26.7-dars: Generativ modellarni baholash

UCH TALAB:
  sifat (fidelity)     - har namuna haqiqiy taqsimotning "ichida"
  qamrov (diversity)   - haqiqiy taqsimotning hamma qismi namunalarda bor
  yangilik (novelty)   - namunalar o'quv to'plamining nusxasi emas

KLASSIK BUZILISHLAR (bu darsdagi sun'iy generatorlar):
  shovqin / xiralik     -> sifat buziladi
  faqat 5 sinf          -> qamrov buziladi (modalar yo'qolgan)
  10 ta prototip        -> sinf ICHIDAGI qamrov yo'q (mode collapse, 26.4)
  o'quvdan nusxa        -> yangilik yo'q (yodlab olish)

NIMA UCHUN BITTA SON YETMAYDI:
  log-likelihood 26.1-bob - VAE/diffusion da hisoblanadi, GAN da yo'q;
    yuqori likelihood yaxshi namunani kafolatlamaydi
  ko'z bilan baholash - kichik tanlov, sub'ektiv, xilma-xillikni ko'rmaydi
  -> bir nechta o'lchov, har biri o'z buzilishiga sezgir

G'OYA (Heusel va boshq., 2017 - FID):
  1. haqiqiy va generatsiya rasmlarini oldindan o'rgatilgan tarmoqdan o'tkazish
     (FID da - Inception-v3 ning 2048 o'lchamli qatlami; bizda - o'z
      klassifikatorimizning 32 o'lchamli qatlami)
  2. har to'plam xususiyatlarini ko'p o'lchovli Gauss deb qarash: (mu, S)
  3. ikki Gauss orasidagi Frechet (Wasserstein-2) masofa:

  FD = ||mu_r - mu_g||^2 + tr(S_r) + tr(S_g) - 2 tr( sqrt(S_r S_g) )

HISOBLASH (barqaror):
  tr(sqrt(S_r S_g)) = sum sqrt( lambda( sqrt(S_r) S_g sqrt(S_r) ) )
  sqrt(S_r) - xos qiymatlar orqali (simmetrik) -> 1-misolda scipy.sqrtm bilan mos

NIMANI KO'RADI:
  o'rtacha siljishi (sifat, sinflar nisbati)
  kovariatsiya farqi (xilma-xillik, qamrov)
  -> sifat VA qamrov buzilishlarini bitta songa aralashtiradi

NIMANI KO'RMAYDI:
  yodlab olishni - nusxa haqiqiy taqsimotning o'zi, FD past
  qaysi komponent buzilgani - bitta son

  generator                 FD
  ideal (yangi haqiqiy)     5.59    <- nol emas: chekli namuna
  shovqin 0.3              15.57
  shovqin 0.6              46.99
  xira (3x3)              188.55
  faqat 5 sinf             44.03
  prototiplar (10 ta)      13.64
  o'quvdan nusxa            4.66    <- idealdan ham PAST

MUAMMO:
  FD - chekli namunadan hisoblangan (mu, S) bilan; kovariatsiya bahosining
  shovqini tr(...) qismiga har doim MUSBAT hissa qo'shadi
  -> ikki AYNAN bir xil taqsimot uchun ham FD > 0, va n kichik bo'lsa katta

1-MISOL (ikki haqiqiy to'plam, 20 tanlov):
  n      FD       n * FD
  25    69.82     1745
  50    31.16     1558
  100   14.55     1455
  200    7.73     1546
  400    3.15     1258
  -> siljish taxminan ~ 1/n

TUZOQ:
  "shovqin 0.3" (n = 200)  FD 18.18
  haqiqiy rasmlar (n = 25) FD 40.25   <- buzilgan generator "yaxshiroq" ko'rinadi

QOIDA:
  FD ni faqat BIR XIL n (va bir xil etalon) bilan solishtiring
  maqolalarda odatda 50 000 namuna; kichik n da ideal chegarani ham o'lchang
  (FID ning siljishsiz variantlari ham taklif qilingan - ekstrapolyatsiya bilan)

IS (Salimans va boshq., 2016):
  IS = exp( E_x [ KL( p(y|x) || p(y) ) ] )
  p(y|x) - klassifikator ehtimollari, p(y) = E_x p(y|x) - marginal

YUQORI BO'LADI, AGAR:
  har rasm uchun p(y|x) o'tkir  (klassifikator ishonchli -> "sifat")
  p(y) tekis                     (hamma sinf bor -> "sinflar qamrovi")
  maksimum = sinflar soni (bizda 10)

2-MISOL:
  etalon haqiqiy rasmlar    IS 9.43
  hamma p(y|x) tekis        IS 1.00
  hamma rasm bitta sinf     IS 1.00
  prototiplar (10 ta)       IS 10.00   <- haqiqiydan "yaxshi"!

KO'R NUQTALARI:
  haqiqiy ma'lumot bilan SOLISHTIRMAYDI - faqat generatsiyaga qaraydi
  sinf ICHIDAGI xilma-xillikni ko'rmaydi (10 ta rasm yetarli)
  yodlab olishni ko'rmaydi
  klassifikatorga bog'liq: boshqa ma'lumotda (ImageNet emas) ma'nosiz

G'OYA (Sajjadi 2018; Kynkaanniemi va boshq., 2019):
  sifat va qamrovni ALOHIDA sonlar bilan o'lchash

HAQIQIY MANIFOLD:
  har haqiqiy nuqta f_i atrofida shar, radiusi r_i = k-chi eng yaqin
  haqiqiy qo'shnigacha masofa (bizda k = 3)
  manifold = sharlar birlashmasi

  precision = generatsiyaning qancha qismi HAQIQIY manifold ichida    (sifat)
  recall    = haqiqiy nuqtalarning qancha qismi GENERATSIYA manifoldi ichida (qamrov)

2-MISOL (N = 200, k = 3):
  generator               precision  recall
  ideal                     0.955     0.928
  shovqin 0.3               0.875     0.865
  faqat 5 sinf              0.947     0.553   <- sifat joyida, qamrov yo'q
  prototiplar               1.000     0.000   <- takror nuqtalar: radius 0
  o'quvdan nusxa            0.938     0.947   <- sezilmaydi

NOZIK JOYLAR:
  k tanlovi: kichik k - qattiq, katta k - yumshoq manifold
  chetdagi (outlier) haqiqiy nuqta katta shar hosil qiladi -> precision oshadi
  ikki to'plam bir xil n bilan (2-misolda etalondan N ta tanlanadi)

SAVOL: namuna o'quv to'plamidagi biror rasmning nusxasimi?

ODDIY TEKSHIRUV (26.1-dars):
  d_oquv = namunadan eng yaqin O'QUV rasmigacha masofa
  kichik d_oquv - shubhali, lekin "kichik" nisbatan nimaga?

TO'G'RI TAQQOSLASH:
  d_boshqa = eng yaqin KO'RILMAGAN haqiqiy rasmgacha masofa
             (o'quv bilan bir xil hajmdagi boshqa to'plam)
  halol generator:  o'quv to'plamini "ko'rmagan" kabi -> d_oquv ~ d_boshqa
  yodlagan:         d_oquv << d_boshqa

  nisbat = median(d_oquv) / median(d_boshqa)     ideal ~ 1
  nusxa ulushi = d_oquv < 0.5 * d_boshqa bo'lgan namunalar

3-MISOL (piksel fazosida):
  generator            FD     nisbat  nusxa ulushi
  ideal               7.80    1.00       0.0%
  o'quvdan nusxa      2.11    0.00     100.0%
  GMM,  20 komponent  6.82    0.89       1.0%
  GMM, 100 komponent  5.79    0.63      28.5%   <- FD idealdan PAST!
  GMM, 400 komponent  8.00    0.27      99.5%

QAYSI FAZODA:
  piksel fazosi - "aynan nusxa"ni yaxshi topadi; siljitilgan nusxani o'tkazib yuboradi
  xususiyat fazosi - semantik o'xshashlik; lekin "bir xil sinf" ni nusxa deb o'ylashi mumkin
  amalda ikkalasi + eng yaqin juftliklarni ko'z bilan ko'rish

TASODIFIYLIK MANBALARI:
  generatsiya urug'i, etalon tanlovi, (va modelning o'zi - o'rgatish urug'i)

1. BIR NECHA URUG' (18-qism):
   har generator 3 urug'da; ideal bilan JUFTLASHGAN farq (bir xil urug' -
   bir xil etalon tanlovi); "sezilarli" = farq > 2 * SE

2. AMALIY CHEGARA:
   juftlashgan SE juda kichik bo'lishi mumkin -> 0.008 farq ham "sezilarli"
   shuning uchun: farq > max(2 * SE, amaliy chegara)
   (2-misolda: IS 5%, precision/recall 0.05)

3. BOOTSTRAP CI (bitta namunadan):
   generatsiya va etalonni qaytarib tanlab, o'lchovni qayta hisoblash
   farq uchun: ikkala generatorga BIR XIL indekslar (juftlashgan)
   4-misol: FD farqi +10.38, 95% CI [+6.84, +12.69]
   ehtiyot: FD ning o'zi bootstrapda yuqoriga siljiydi (takror nuqtalar
   kovariatsiyani kichraytiradi): 191.05 -> o'rtacha 194.04

NEGA KERAK:
  avtomatik o'lchovlar - proksi; "odamga yoqadimi", "so'rovga mosmi",
  "matn to'g'ri yozilganmi" kabi savollarni bevosita o'lchamaydi

USULLAR:
  juftlashgan afzallik (A/B): bir xil so'rov, ikki model, "qaysi biri yaxshi?"
  mutlaq baho (1-5): oddiy, lekin baholovchilar shkalasi farq qiladi
  "haqiqiymi yoki sun'iymi?" testi: odam ajrata olmasa - realistik

STATISTIKA (4-misol, haqiqiy afzallik 0.55):
  juftliklar   quvvat
    100         0.16
    400         0.47
    800         0.82
  -> kichik afzallikni topish uchun yuzlab juftlik kerak

TUZOQLAR:
  joylashuv moyilligi: B doim chapda -> B ulushi 0.598 (haqiqiy 0.55)
    yechim: joylashuvni tasodifiy qilish 0.555-bob
  baholovchilar kelishuvi (Cohen kappa) va ko'rsatmalar
  kichik "yoqqan" misollarni tanlab ko'rsatish (cherry-picking)
  bir baholovchidan ko'p baho - mustaqil emas

import numpy as np
import torch

# Frechet masofa (xususiyatlar fazosida)
s1, s2 = np.cov(a, rowvar=False), np.cov(b, rowvar=False)
w, v = np.linalg.eigh(s1)
ild = (v * np.sqrt(np.clip(w, 0, None))) @ v.T
fd = ((a.mean(0) - b.mean(0)) ** 2).sum() + np.trace(s1) + np.trace(s2) \
    - 2 * np.sqrt(np.clip(np.linalg.eigvalsh(ild @ s2 @ ild), 0, None)).sum()

# Inception Score
py = p.mean(0, keepdims=True)
is_ = np.exp((p * (np.log(p) - np.log(py))).sum(1).mean())

# precision / recall (k-NN radiuslar)
r_real = np.sort(dist(f_real, f_real), 1)[:, k]
precision = (dist(f_gen, f_real) <= r_real[None]).any(1).mean()

# yodlab olish
d_o = torch.cdist(X_gen, X_oquv).min(1).values
d_b = torch.cdist(X_gen, X_boshqa).min(1).values
nisbat = d_o.median() / d_b.median()

sifat, qamrov, yangilik - uch xil talab, uch xil o'lchov
FD: sifat + qamrov bitta sonda; n ga bog'liq; nusxani ko'rmaydi
IS: etalonsiz; sinf ichi xilma-xillik va nusxani ko'rmaydi
precision - sifat, recall - qamrov (k-NN radiuslar)
yodlab olish: o'quvga va ko'rilmagan to'plamga masofa nisbati
bir necha urug', juftlashgan farq, amaliy chegara, inson A/B

"""Frechet masofa noldan: o'z klassifikatorimiz xususiyatlarida, sun'iy buzilgan generatorlar va namuna soni siljishi."""

import math
import warnings

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from scipy import linalg
from sklearn.datasets import load_digits

N = 200                                          # har generatordan namuna soni


class Klassifikator(nn.Module):
    """Baholovchi: 64 -> 128 -> 32 (xususiyatlar) -> 10."""

    def __init__(self):
        super().__init__()
        self.xus = nn.Sequential(nn.Linear(64, 128), nn.ReLU(), nn.Linear(128, 32), nn.ReLU())
        self.bosh = nn.Linear(32, 10)

    def forward(self, x):
        return self.bosh(self.xus(x))


def bolaklar():
    """4 ta ajratilgan bo'lak: klassifikator, generator o'quvi, held-out (ideal), etalon."""
    d = load_digits()
    X = torch.tensor(d.data / 8.0 - 1.0, dtype=torch.float32)
    y = torch.tensor(d.target)
    p = torch.tensor(np.random.default_rng(0).permutation(len(X)))
    X, y = X[p], y[p]
    ch = [0, 450, 850, 1350, len(X)]
    return [(X[a:b], y[a:b]) for a, b in zip(ch[:-1], ch[1:])]


def orgat_klassifikator(X, y, seed=0):
    torch.manual_seed(seed)
    k = Klassifikator()
    opt = torch.optim.Adam(k.parameters(), lr=3e-3, weight_decay=1e-4)
    g = torch.Generator().manual_seed(seed)
    for _ in range(800):
        i = torch.randint(0, len(X), (128,), generator=g)
        loss = F.cross_entropy(k(X[i] + 0.1 * torch.randn(X[i].shape, generator=g)), y[i])
        opt.zero_grad()
        loss.backward()
        opt.step()
    k.eval()
    return k


def xira(X):
    """3x3 o'rtacha filtr (chetlarda nol bilan to'ldirib)."""
    r = F.avg_pool2d(F.pad((X.view(-1, 1, 8, 8) + 1) / 2, (1, 1, 1, 1)), 3, stride=1)
    return (r * 2 - 1).view(-1, 64)


def generatorlar(oquv, held, seed):
    """Sun'iy buzilgan 'generatorlar': har biri N ta rasm qaytaradi."""
    g = torch.Generator().manual_seed(seed)
    (Xg, yg), (Xh, yh) = oquv, held
    idx = torch.randperm(len(Xh), generator=g)[:N]
    ideal = Xh[idx]
    besh = (yh < 5).nonzero()[:, 0]
    besh = besh[torch.randperm(len(besh), generator=g)[:N]]
    proto = torch.stack([Xg[yg == c].mean(0) for c in range(10)])
    return {
        "ideal (yangi haqiqiy)": ideal,
        "shovqin 0.3": (ideal + 0.3 * torch.randn(ideal.shape, generator=g)).clamp(-1, 1),
        "shovqin 0.6": (ideal + 0.6 * torch.randn(ideal.shape, generator=g)).clamp(-1, 1),
        "xira (3x3)": xira(ideal),
        "faqat 5 sinf": Xh[besh],
        "prototiplar (10 ta)": proto.repeat(N // 10, 1),
        "o'quvdan nusxa": Xg[torch.randperm(len(Xg), generator=g)[:N]],
    }


def frechet(a, b):
    """FD = ||mu1 - mu2||^2 + tr(S1) + tr(S2) - 2 tr(sqrt(S1 S2)).

    tr(sqrt(S1 S2)) = sum sqrt(lambda(sqrt(S1) S2 sqrt(S1))) - simmetrik va barqaror.
    """
    a, b = np.asarray(a, dtype=np.float64), np.asarray(b, dtype=np.float64)
    s1, s2 = np.cov(a, rowvar=False), np.cov(b, rowvar=False)
    w, v = np.linalg.eigh(s1)
    ild = (v * np.sqrt(np.clip(w, 0, None))) @ v.T
    oz = np.linalg.eigvalsh(ild @ s2 @ ild)
    return float(((a.mean(0) - b.mean(0)) ** 2).sum() + np.trace(s1) + np.trace(s2)
                 - 2 * np.sqrt(np.clip(oz, 0, None)).sum())


def frechet_scipy(a, b):
    """Tekshiruv uchun: scipy.linalg.sqrtm bilan klassik yozuv."""
    s1, s2 = np.cov(a, rowvar=False), np.cov(b, rowvar=False)
    with warnings.catch_warnings():
        warnings.simplefilter("ignore")                  # "o'lik" ReLU -> singulyar matritsa
        ildiz = linalg.sqrtm(s1 @ s2).real
    return float(((a.mean(0) - b.mean(0)) ** 2).sum() + np.trace(s1 + s2 - 2 * ildiz))


def main() -> None:
    torch.set_num_threads(1)
    klass, oquv, held, etalon = bolaklar()
    k = orgat_klassifikator(*klass)

    def xus(X):
        with torch.no_grad():
            return k.xus(X).double().numpy()

    f_et = xus(etalon[0])
    with torch.no_grad():
        aniq = (k(etalon[0]).argmax(1) == etalon[1]).float().mean().item()
    print("=== 1. Bo'laklar va baholovchi ===")
    for nom, (X, _) in zip(("klassifikator o'quvi", "generator o'quvi", "held-out (ideal)",
                            "etalon (FD uchun)"), (klass, oquv, held, etalon)):
        print(f"  {nom:<22} {len(X):>4} ta")
    print(f"  baholovchi aniqligi (etalonda): {aniq:.3f}; xususiyatlar: 32 o'lcham")
    gen = generatorlar(oquv, held, 0)
    a, b = xus(gen["shovqin 0.3"]), f_et
    print(f"  o'z FD va scipy.sqrtm FD: {frechet(a, b):.4f} va {frechet_scipy(a, b):.4f}")

    print(f"\n=== 2. FD: sun'iy buzilgan generatorlar (N = {N}, etalon {len(f_et)}, 3 urug') ===")
    print("  generator                FD (o'rt +- SE)   ideal dan farq")
    natija = {}
    for u in range(3):
        for nom, X in generatorlar(oquv, held, u).items():
            natija.setdefault(nom, []).append(frechet(xus(X), f_et))
    ideal = np.array(natija["ideal (yangi haqiqiy)"])
    for nom, v in natija.items():
        v = np.array(v)
        f = v - ideal
        se = f.std(ddof=1) / math.sqrt(3) if nom != "ideal (yangi haqiqiy)" else 0.0
        belgi = "-" if nom.startswith("ideal") else ("USHLADI" if f.mean() > 2 * se else "ushlamadi")
        print(f"  {nom:<22} {v.mean():>7.2f} +- {v.std(ddof=1) / math.sqrt(3):<5.2f}   {belgi}")

    print("\n=== 3. Namuna soniga bog'liq siljish: ikki HAQIQIY to'plam orasidagi FD ===")
    print("  n     FD(held_n, etalon_n)   n * FD    (20 tasodifiy tanlov)")
    rng = np.random.default_rng(1)
    f_h = xus(held[0])
    olcham = {}
    for n in (25, 50, 100, 200, 400):
        v = [frechet(f_h[rng.choice(len(f_h), n, replace=False)],
                     f_et[rng.choice(len(f_et), n, replace=False)]) for _ in range(20)]
        olcham[n] = np.mean(v)
        print(f"  {n:<5} {np.mean(v):>10.2f} +- {np.std(v, ddof=1) / math.sqrt(20):<6.2f}   {n * np.mean(v):>8.0f}")
    if olcham[25] > 4 * olcham[400]:
        print("  haqiqiy to'plamlar AYNAN bir taqsimotdan, lekin FD n kichik bo'lsa katta:")
        print("  siljish taxminan ~ 1/n (n * FD kam o'zgaradi) - FD ni faqat bir xil n da solishtiring")
    xavf = frechet(xus(generatorlar(oquv, held, 0)["shovqin 0.3"]), f_et)
    kichik = frechet(f_h[:25], f_et[:25])
    print(f"  tuzoq: 'shovqin 0.3' (n = {N}) FD {xavf:.2f}, haqiqiy rasmlar (n = 25) FD {kichik:.2f}")

    print("\n=== 4. Bootstrap CI: bitta generator, bitta namuna ===")
    fg = xus(generatorlar(oquv, held, 0)["xira (3x3)"])
    rng = np.random.default_rng(2)
    boot = [frechet(fg[rng.integers(0, len(fg), len(fg))], f_et[rng.integers(0, len(f_et), len(f_et))])
            for _ in range(200)]
    print(f"  'xira' FD {frechet(fg, f_et):.2f}; bootstrap 95% CI [{np.percentile(boot, 2.5):.2f}, "
          f"{np.percentile(boot, 97.5):.2f}], o'rtacha {np.mean(boot):.2f}")
    print("  bootstrap o'rtachasi asl qiymatdan yuqori: qayta tanlashda takror nuqtalar")
    print("  kovariatsiyani 'buzadi' - FD ning o'zi siljigan baho, CI ni ehtiyot bilan o'qing")


if __name__ == "__main__":
    main()

=== 1. Bo'laklar va baholovchi ===
  klassifikator o'quvi    450 ta
  generator o'quvi        400 ta
  held-out (ideal)        500 ta
  etalon (FD uchun)       447 ta
  baholovchi aniqligi (etalonda): 0.973; xususiyatlar: 32 o'lcham
  o'z FD va scipy.sqrtm FD: 18.1830 va 18.1830

=== 2. FD: sun'iy buzilgan generatorlar (N = 200, etalon 447, 3 urug') ===
  generator                FD (o'rt +- SE)   ideal dan farq
  ideal (yangi haqiqiy)     5.59 +- 1.30    -
  shovqin 0.3              15.57 +- 1.40    USHLADI
  shovqin 0.6              46.99 +- 1.11    USHLADI
  xira (3x3)              188.55 +- 1.86    USHLADI
  faqat 5 sinf             44.03 +- 0.27    USHLADI
  prototiplar (10 ta)      13.64 +- 0.00    USHLADI
  o'quvdan nusxa            4.66 +- 0.57    ushlamadi

=== 3. Namuna soniga bog'liq siljish: ikki HAQIQIY to'plam orasidagi FD ===
  n     FD(held_n, etalon_n)   n * FD    (20 tasodifiy tanlov)
  25         69.82 +- 6.13         1745
  50         31.16 +- 2.80         1558
  100        14.55 +- 1.15         1455
  200         7.73 +- 0.65         1546
  400         3.15 +- 0.12         1258
  haqiqiy to'plamlar AYNAN bir taqsimotdan, lekin FD n kichik bo'lsa katta:
  siljish taxminan ~ 1/n (n * FD kam o'zgaradi) - FD ni faqat bir xil n da solishtiring
  tuzoq: 'shovqin 0.3' (n = 200) FD 18.18, haqiqiy rasmlar (n = 25) FD 40.25

=== 4. Bootstrap CI: bitta generator, bitta namuna ===
  'xira' FD 191.05; bootstrap 95% CI [177.22, 213.48], o'rtacha 194.04
  bootstrap o'rtachasi asl qiymatdan yuqori: qayta tanlashda takror nuqtalar
  kovariatsiyani 'buzadi' - FD ning o'zi siljigan baho, CI ni ehtiyot bilan o'qing

"""Inception Score va precision/recall (k-NN radiuslar): sifat va qamrovni ajratish."""

import math

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from sklearn.datasets import load_digits

N = 200                                          # har generatordan namuna soni


class Klassifikator(nn.Module):
    """Baholovchi: 64 -> 128 -> 32 (xususiyatlar) -> 10."""

    def __init__(self):
        super().__init__()
        self.xus = nn.Sequential(nn.Linear(64, 128), nn.ReLU(), nn.Linear(128, 32), nn.ReLU())
        self.bosh = nn.Linear(32, 10)

    def forward(self, x):
        return self.bosh(self.xus(x))


def bolaklar():
    """4 ta ajratilgan bo'lak: klassifikator, generator o'quvi, held-out (ideal), etalon."""
    d = load_digits()
    X = torch.tensor(d.data / 8.0 - 1.0, dtype=torch.float32)
    y = torch.tensor(d.target)
    p = torch.tensor(np.random.default_rng(0).permutation(len(X)))
    X, y = X[p], y[p]
    ch = [0, 450, 850, 1350, len(X)]
    return [(X[a:b], y[a:b]) for a, b in zip(ch[:-1], ch[1:])]


def orgat_klassifikator(X, y, seed=0):
    torch.manual_seed(seed)
    k = Klassifikator()
    opt = torch.optim.Adam(k.parameters(), lr=3e-3, weight_decay=1e-4)
    g = torch.Generator().manual_seed(seed)
    for _ in range(800):
        i = torch.randint(0, len(X), (128,), generator=g)
        loss = F.cross_entropy(k(X[i] + 0.1 * torch.randn(X[i].shape, generator=g)), y[i])
        opt.zero_grad()
        loss.backward()
        opt.step()
    k.eval()
    return k


def xira(X):
    """3x3 o'rtacha filtr (chetlarda nol bilan to'ldirib)."""
    r = F.avg_pool2d(F.pad((X.view(-1, 1, 8, 8) + 1) / 2, (1, 1, 1, 1)), 3, stride=1)
    return (r * 2 - 1).view(-1, 64)


def generatorlar(oquv, held, seed):
    """Sun'iy buzilgan 'generatorlar': har biri N ta rasm qaytaradi."""
    g = torch.Generator().manual_seed(seed)
    (Xg, yg), (Xh, yh) = oquv, held
    idx = torch.randperm(len(Xh), generator=g)[:N]
    ideal = Xh[idx]
    besh = (yh < 5).nonzero()[:, 0]
    besh = besh[torch.randperm(len(besh), generator=g)[:N]]
    proto = torch.stack([Xg[yg == c].mean(0) for c in range(10)])
    return {
        "ideal (yangi haqiqiy)": ideal,
        "shovqin 0.3": (ideal + 0.3 * torch.randn(ideal.shape, generator=g)).clamp(-1, 1),
        "shovqin 0.6": (ideal + 0.6 * torch.randn(ideal.shape, generator=g)).clamp(-1, 1),
        "xira (3x3)": xira(ideal),
        "faqat 5 sinf": Xh[besh],
        "prototiplar (10 ta)": proto.repeat(N // 10, 1),
        "o'quvdan nusxa": Xg[torch.randperm(len(Xg), generator=g)[:N]],
    }


def inception_score(p):
    """IS = exp( E_x KL(p(y|x) || p(y)) ) - o'z klassifikatorimiz ehtimollarida."""
    p = np.clip(p, 1e-12, 1.0)
    py = p.mean(0, keepdims=True)
    return float(np.exp((p * (np.log(p) - np.log(py))).sum(1).mean()))


def knn_radius(f, k=3):
    """Har nuqtadan o'z to'plamidagi k-chi eng yaqin qo'shnigacha masofa (o'zi hisobga olinmaydi)."""
    d = np.sqrt(((f[:, None, :] - f[None, :, :]) ** 2).sum(-1))
    return np.sort(d, axis=1)[:, k]


def precision_recall(f_real, f_gen, k=3):
    """Kynkaanniemi va boshq. (2019): k-NN sharlari birlashmasi - 'manifold'.

    precision - generatsiyaning qancha qismi HAQIQIY manifold ichida (sifat)
    recall    - haqiqiy namunalarning qancha qismi GENERATSIYA manifoldi ichida (qamrov)
    """
    r_real, r_gen = knn_radius(f_real, k), knn_radius(f_gen, k)
    d = np.sqrt(((f_gen[:, None, :] - f_real[None, :, :]) ** 2).sum(-1))       # (gen, real)
    precision = (d <= r_real[None, :]).any(1).mean()
    recall = (d.T <= r_gen[None, :]).any(1).mean()
    return float(precision), float(recall)


def main() -> None:
    torch.set_num_threads(1)
    klass, oquv, held, etalon = bolaklar()
    k = orgat_klassifikator(*klass)

    def xus_ehtimol(X):
        with torch.no_grad():
            f = k.xus(X)
            return f.double().numpy(), torch.softmax(k.bosh(f), 1).double().numpy()

    f_et, p_et = xus_ehtimol(etalon[0])
    print("=== 1. Inception Score g'oyasi (o'z klassifikatorimiz bilan) ===")
    print("  IS yuqori, agar: har rasmda p(y|x) o'tkir (ishonchli) VA p(y) tekis (hamma sinf)")
    print(f"  maksimum = sinflar soni = 10; etalon haqiqiy rasmlar: IS {inception_score(p_et):.2f}")
    tekis = np.full((N, 10), 0.1)
    bitta = np.eye(10)[np.zeros(N, dtype=int)]
    print(f"  hamma p(y|x) tekis: IS {inception_score(tekis):.2f};  hamma rasm bitta sinf, "
          f"ishonchli: IS {inception_score(bitta):.2f}")

    print(f"\n=== 2. IS, precision, recall (k = 3, N = {N}, 3 urug') ===")
    print("  generator               IS      precision  recall   sinflar>=5%")
    natija = {}
    for u in range(3):
        rng = np.random.default_rng(10 + u)
        f_ref = f_et[rng.choice(len(f_et), N, replace=False)]          # bir xil N - adolatli
        for nom, X in generatorlar(oquv, held, u).items():
            f, p = xus_ehtimol(X)
            pr, rc = precision_recall(f_ref, f)
            sinf = np.bincount(p.argmax(1), minlength=10) / len(p)
            natija.setdefault(nom, []).append((inception_score(p), pr, rc, (sinf >= 0.05).sum()))
    for nom, v in natija.items():
        v = np.array(v).mean(0)
        print(f"  {nom:<22} {v[0]:>5.2f}   {v[1]:>8.3f}  {v[2]:>7.3f}   {v[3]:>6.1f}")

    print("\n=== 3. Qaysi o'lchov nimani ushladi (ideal bilan juftlashgan farq) ===")
    print("  mezon: farq > 2*SE VA amaliy chegara (IS 5% dan ko'p, precision/recall 0.05 dan ko'p)")
    ideal = np.array(natija["ideal (yangi haqiqiy)"])
    print("  generator               IS pasaydi  precision pasaydi  recall pasaydi")
    for nom, v in natija.items():
        if nom.startswith("ideal"):
            continue
        f = np.array(v) - ideal
        se = f.std(0, ddof=1) / math.sqrt(3) + 1e-9
        chegara = [0.05 * ideal[:, 0].mean(), 0.05, 0.05]
        belgi = ["HA" if f[:, j].mean() < -max(2 * se[j], chegara[j]) else "-" for j in range(3)]
        print(f"  {nom:<22} {belgi[0]:^11} {belgi[1]:^18} {belgi[2]:^14}")

    print("\n=== 4. Tuzoq: IS 'haqiqiydan yaxshi' bo'lishi mumkin ===")
    is_ideal = ideal[:, 0].mean()
    is_proto = np.array(natija["prototiplar (10 ta)"])[:, 0].mean()
    print(f"  prototiplar IS {is_proto:.2f}, haqiqiy rasmlar IS {is_ideal:.2f}")
    if is_proto > is_ideal:
        print("  10 ta rasmni takrorlaydigan generator IS bo'yicha 'yutdi' - IS sinf ICHIDAGI")
        print("  xilma-xillikni ko'rmaydi; recall esa buni ushladi")
    rc_n = np.array(natija["o'quvdan nusxa"])[:, 1:3].mean(0)
    print(f"  o'quvdan nusxa: precision {rc_n[0]:.3f}, recall {rc_n[1]:.3f} - "
          "nusxani precision/recall ham ko'rmaydi (3-misol)")


if __name__ == "__main__":
    main()

=== 1. Inception Score g'oyasi (o'z klassifikatorimiz bilan) ===
  IS yuqori, agar: har rasmda p(y|x) o'tkir (ishonchli) VA p(y) tekis (hamma sinf)
  maksimum = sinflar soni = 10; etalon haqiqiy rasmlar: IS 9.43
  hamma p(y|x) tekis: IS 1.00;  hamma rasm bitta sinf, ishonchli: IS 1.00

=== 2. IS, precision, recall (k = 3, N = 200, 3 urug') ===
  generator               IS      precision  recall   sinflar>=5%
  ideal (yangi haqiqiy)   9.26      0.955    0.928     10.0
  shovqin 0.3             8.85      0.875    0.865     10.0
  shovqin 0.6             7.87      0.572    0.668     10.0
  xira (3x3)              3.50      0.535    0.002      6.0
  faqat 5 sinf            5.12      0.947    0.553      5.0
  prototiplar (10 ta)    10.00      1.000    0.000     10.0
  o'quvdan nusxa          9.30      0.938    0.947     10.0

=== 3. Qaysi o'lchov nimani ushladi (ideal bilan juftlashgan farq) ===
  mezon: farq > 2*SE VA amaliy chegara (IS 5% dan ko'p, precision/recall 0.05 dan ko'p)
  generator               IS pasaydi  precision pasaydi  recall pasaydi
  shovqin 0.3                 -              HA               HA
  shovqin 0.6                HA              HA               HA
  xira (3x3)                 HA              HA               HA
  faqat 5 sinf               HA              -                HA
  prototiplar (10 ta)         -              -                HA
  o'quvdan nusxa              -              -                -

=== 4. Tuzoq: IS 'haqiqiydan yaxshi' bo'lishi mumkin ===
  prototiplar IS 10.00, haqiqiy rasmlar IS 9.26
  10 ta rasmni takrorlaydigan generator IS bo'yicha 'yutdi' - IS sinf ICHIDAGI
  xilma-xillikni ko'rmaydi; recall esa buni ushladi
  o'quvdan nusxa: precision 0.938, recall 0.947 - nusxani precision/recall ham ko'rmaydi (3-misol)

"""Yodlab olish (memorization) tekshiruvi: eng yaqin qo'shni masofasi, o'quv va ko'rilmagan to'plam."""

import math
import warnings

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from sklearn.datasets import load_digits
from sklearn.mixture import GaussianMixture

N = 200                                          # har generatordan namuna soni
BELGI = " .:-=+*#%@"


class Klassifikator(nn.Module):
    """Baholovchi: 64 -> 128 -> 32 (xususiyatlar) -> 10."""

    def __init__(self):
        super().__init__()
        self.xus = nn.Sequential(nn.Linear(64, 128), nn.ReLU(), nn.Linear(128, 32), nn.ReLU())
        self.bosh = nn.Linear(32, 10)

    def forward(self, x):
        return self.bosh(self.xus(x))


def bolaklar():
    """4 ta ajratilgan bo'lak: klassifikator, generator o'quvi, held-out (ideal), etalon."""
    d = load_digits()
    X = torch.tensor(d.data / 8.0 - 1.0, dtype=torch.float32)
    y = torch.tensor(d.target)
    p = torch.tensor(np.random.default_rng(0).permutation(len(X)))
    X, y = X[p], y[p]
    ch = [0, 450, 850, 1350, len(X)]
    return [(X[a:b], y[a:b]) for a, b in zip(ch[:-1], ch[1:])]


def orgat_klassifikator(X, y, seed=0):
    torch.manual_seed(seed)
    k = Klassifikator()
    opt = torch.optim.Adam(k.parameters(), lr=3e-3, weight_decay=1e-4)
    g = torch.Generator().manual_seed(seed)
    for _ in range(800):
        i = torch.randint(0, len(X), (128,), generator=g)
        loss = F.cross_entropy(k(X[i] + 0.1 * torch.randn(X[i].shape, generator=g)), y[i])
        opt.zero_grad()
        loss.backward()
        opt.step()
    k.eval()
    return k


def xira(X):
    """3x3 o'rtacha filtr (chetlarda nol bilan to'ldirib)."""
    r = F.avg_pool2d(F.pad((X.view(-1, 1, 8, 8) + 1) / 2, (1, 1, 1, 1)), 3, stride=1)
    return (r * 2 - 1).view(-1, 64)


def generatorlar(oquv, held, seed):
    """Sun'iy buzilgan 'generatorlar': har biri N ta rasm qaytaradi."""
    g = torch.Generator().manual_seed(seed)
    (Xg, yg), (Xh, yh) = oquv, held
    idx = torch.randperm(len(Xh), generator=g)[:N]
    ideal = Xh[idx]
    besh = (yh < 5).nonzero()[:, 0]
    besh = besh[torch.randperm(len(besh), generator=g)[:N]]
    proto = torch.stack([Xg[yg == c].mean(0) for c in range(10)])
    return {
        "ideal (yangi haqiqiy)": ideal,
        "shovqin 0.3": (ideal + 0.3 * torch.randn(ideal.shape, generator=g)).clamp(-1, 1),
        "shovqin 0.6": (ideal + 0.6 * torch.randn(ideal.shape, generator=g)).clamp(-1, 1),
        "xira (3x3)": xira(ideal),
        "faqat 5 sinf": Xh[besh],
        "prototiplar (10 ta)": proto.repeat(N // 10, 1),
        "o'quvdan nusxa": Xg[torch.randperm(len(Xg), generator=g)[:N]],
    }


def frechet(a, b):
    """FD = ||mu1 - mu2||^2 + tr(S1) + tr(S2) - 2 tr(sqrt(S1 S2)).

    tr(sqrt(S1 S2)) = sum sqrt(lambda(sqrt(S1) S2 sqrt(S1))) - simmetrik va barqaror.
    """
    a, b = np.asarray(a, dtype=np.float64), np.asarray(b, dtype=np.float64)
    s1, s2 = np.cov(a, rowvar=False), np.cov(b, rowvar=False)
    w, v = np.linalg.eigh(s1)
    ild = (v * np.sqrt(np.clip(w, 0, None))) @ v.T
    oz = np.linalg.eigvalsh(ild @ s2 @ ild)
    return float(((a.mean(0) - b.mean(0)) ** 2).sum() + np.trace(s1) + np.trace(s2)
                 - 2 * np.sqrt(np.clip(oz, 0, None)).sum())


def eng_yaqin(A, B):
    """A ning har qatoridan B dagi eng yaqin qatorgacha L2 masofa (piksel fazosida) va indeksi."""
    d = torch.cdist(A, B)
    q, i = d.min(1)
    return q.numpy(), i.numpy()


def gmm_generator(Xg, c, seed=0):
    """Haqiqiy (lekin sodda) generator: o'quv to'plamiga c komponentli GMM (26.1-dars)."""
    with warnings.catch_warnings():
        warnings.simplefilter("ignore")
        gm = GaussianMixture(c, covariance_type="full", reg_covar=0.01, max_iter=100,
                             random_state=seed).fit(Xg.numpy())
        s, _ = gm.sample(N)                          # sonli yaxlitlash ogohlantirishlari ham
    return torch.tensor(s, dtype=torch.float32).clamp(-1, 1)


def ascii_qator(rasmlar):
    satrlar = []
    for r in range(8):
        qism = []
        for x in rasmlar:
            q = ((x.view(8, 8)[r] + 1) / 2).clamp(0, 1)
            qism.append("".join(BELGI[min(9, int(v * 10))] for v in q.tolist()))
        satrlar.append("    " + "   ".join(qism))
    return "\n".join(satrlar)


def main() -> None:
    torch.set_num_threads(1)
    klass, oquv, held, etalon = bolaklar()
    k = orgat_klassifikator(*klass)
    Xg = oquv[0]
    X_boshqa = klass[0][:len(Xg)]              # generator ko'rmagan, o'quv bilan bir xil hajm

    def xus(X):
        with torch.no_grad():
            return k.xus(X).double().numpy()

    f_et = xus(etalon[0])
    print("=== 1. Yodlab olish testi: o'quvga masofa va BOSHQA haqiqiy to'plamga masofa ===")
    print(f"  o'quv to'plami {len(Xg)} ta; taqqoslash to'plami (ko'rilmagan) {len(X_boshqa)} ta")
    print("  har namuna uchun: d_oquv = eng yaqin o'quv rasmi, d_boshqa = eng yaqin boshqa rasm")
    print("  halol generator: d_oquv ~ d_boshqa (nisbat ~ 1); yodlagan: d_oquv << d_boshqa")

    g = torch.Generator().manual_seed(0)
    ideal = held[0][torch.randperm(len(held[0]), generator=g)[:N]]
    nusxa = Xg[torch.randperm(len(Xg), generator=g)[:N]]
    gen = {"ideal (yangi haqiqiy)": ideal,
           "o'quvdan nusxa": nusxa,
           "nusxa + shovqin 0.1": (nusxa + 0.1 * torch.randn(nusxa.shape, generator=g)).clamp(-1, 1),
           "nusxa + shovqin 0.3": (nusxa + 0.3 * torch.randn(nusxa.shape, generator=g)).clamp(-1, 1)}
    for c in (5, 20, 100, 400):
        gen[f"GMM, {c} komponent"] = gmm_generator(Xg, c)

    print(f"\n=== 2. Generatorlar (N = {N}): FD va yodlab olish o'lchovlari ===")
    print("  generator                FD     med d_oquv  med d_boshqa  nisbat  nusxa ulushi")
    natija = {}
    for nom, X in gen.items():
        d_o, _ = eng_yaqin(X, Xg)
        d_b, _ = eng_yaqin(X, X_boshqa)
        ulush = float((d_o < 0.5 * d_b).mean())
        natija[nom] = (frechet(xus(X), f_et), np.median(d_o) / np.median(d_b), ulush)
        print(f"  {nom:<22} {natija[nom][0]:>6.2f}   {np.median(d_o):>8.2f}   {np.median(d_b):>10.2f}"
              f"   {natija[nom][1]:>6.2f}   {ulush:>9.1%}")
    print("  nusxa ulushi: d_oquv < 0.5 * d_boshqa bo'lgan namunalar")

    print("\n=== 3. Tuzoq: FD 'yaxshilanadi', yodlab olish kuchayadi ===")
    for c in (5, 20, 100, 400):
        fd, nisbat, ulush = natija[f"GMM, {c} komponent"]
        print(f"  GMM {c:>3}: FD {fd:>6.2f}, nisbat {nisbat:.2f}, nusxa ulushi {ulush:.1%}")
    eng_fd = min(natija, key=lambda n: natija[n][0])
    print(f"  eng past FD: {eng_fd} ({natija[eng_fd][0]:.2f})")
    if natija[eng_fd][1] < 0.7:
        print("  eng 'yaxshi' FD li generator - yodlagan generator: FD buni jazolamaydi")
    shubhali = sorted(n for n, v in natija.items() if v[1] < 0.7)
    print(f"  nisbat < 0.7 (shubhali): {', '.join(shubhali)}")

    print("\n=== 4. Ko'z bilan tekshirish: GMM 400 namunasi va eng yaqin o'quv rasmi ===")
    X = gen["GMM, 400 komponent"]
    d_o, idx = eng_yaqin(X, Xg)
    tartib = np.argsort(d_o)[:3]
    print(f"  eng yaqin 3 juftlik masofalari: {', '.join(f'{d_o[i]:.2f}' for i in tartib)} "
          f"(ideal generatorda median {np.median(eng_yaqin(ideal, Xg)[0]):.2f})")
    print("  chapdan: namuna | o'quvdagi juft | namuna | juft | namuna | juft")
    print(ascii_qator([r for i in tartib for r in (X[i], Xg[idx[i]])]))


if __name__ == "__main__":
    main()

=== 1. Yodlab olish testi: o'quvga masofa va BOSHQA haqiqiy to'plamga masofa ===
  o'quv to'plami 400 ta; taqqoslash to'plami (ko'rilmagan) 400 ta
  har namuna uchun: d_oquv = eng yaqin o'quv rasmi, d_boshqa = eng yaqin boshqa rasm
  halol generator: d_oquv ~ d_boshqa (nisbat ~ 1); yodlagan: d_oquv << d_boshqa

=== 2. Generatorlar (N = 200): FD va yodlab olish o'lchovlari ===
  generator                FD     med d_oquv  med d_boshqa  nisbat  nusxa ulushi
  ideal (yangi haqiqiy)    7.80       2.42         2.42     1.00        0.0%
  o'quvdan nusxa           2.11       0.00         2.36     0.00      100.0%
  nusxa + shovqin 0.1      2.87       0.66         2.43     0.27      100.0%
  nusxa + shovqin 0.3     11.58       1.92         2.87     0.67        3.5%
  GMM, 5 komponent        14.16       2.90         2.98     0.97        0.0%
  GMM, 20 komponent        6.82       2.26         2.55     0.89        1.0%
  GMM, 100 komponent       5.79       1.57         2.48     0.63       28.5%
  GMM, 400 komponent       8.00       0.65         2.40     0.27       99.5%
  nusxa ulushi: d_oquv < 0.5 * d_boshqa bo'lgan namunalar

=== 3. Tuzoq: FD 'yaxshilanadi', yodlab olish kuchayadi ===
  GMM   5: FD  14.16, nisbat 0.97, nusxa ulushi 0.0%
  GMM  20: FD   6.82, nisbat 0.89, nusxa ulushi 1.0%
  GMM 100: FD   5.79, nisbat 0.63, nusxa ulushi 28.5%
  GMM 400: FD   8.00, nisbat 0.27, nusxa ulushi 99.5%
  eng past FD: o'quvdan nusxa 2.11-bob
  eng 'yaxshi' FD li generator - yodlagan generator: FD buni jazolamaydi
  nisbat < 0.7 (shubhali): GMM, 100 komponent, GMM, 400 komponent, nusxa + shovqin 0.1, nusxa + shovqin 0.3, o'quvdan nusxa

=== 4. Ko'z bilan tekshirish: GMM 400 namunasi va eng yaqin o'quv rasmi ===
  eng yaqin 3 juftlik masofalari: 0.47, 0.48, 0.48 (ideal generatorda median 2.42)
  chapdan: namuna | o'quvdagi juft | namuna | juft | namuna | juft
       =:         +.        %@@:       %@@:       -@*:       -%*.
      .@+        .@+       +@-@+      +@-%+       @+#*       @+%*
      +@         +@        :* **      -* #*       @+=@.      @++@.
      %*         #*          -@:        -@-       *@@@.      *@@@.
      %=         %=         :@+        -@*           @:         @:
      +@@@%      *@@@%     .@#        .@*            @:         @:
      :@.:@-     :@ :@-    -@#+=:     -@%++-      =:=@       =-+@
       :%@*       -%@*      *%@@@      *%@@@      -#%+       -#%+

"""Yig'ma jadval: qaysi o'lchov qaysi buzilishni ushlaydi; bootstrap CI va inson A/B baholashi."""

import math

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from scipy.stats import binomtest
from sklearn.datasets import load_digits

N = 200                                          # har generatordan namuna soni


class Klassifikator(nn.Module):
    """Baholovchi: 64 -> 128 -> 32 (xususiyatlar) -> 10."""

    def __init__(self):
        super().__init__()
        self.xus = nn.Sequential(nn.Linear(64, 128), nn.ReLU(), nn.Linear(128, 32), nn.ReLU())
        self.bosh = nn.Linear(32, 10)

    def forward(self, x):
        return self.bosh(self.xus(x))


def bolaklar():
    """4 ta ajratilgan bo'lak: klassifikator, generator o'quvi, held-out (ideal), etalon."""
    d = load_digits()
    X = torch.tensor(d.data / 8.0 - 1.0, dtype=torch.float32)
    y = torch.tensor(d.target)
    p = torch.tensor(np.random.default_rng(0).permutation(len(X)))
    X, y = X[p], y[p]
    ch = [0, 450, 850, 1350, len(X)]
    return [(X[a:b], y[a:b]) for a, b in zip(ch[:-1], ch[1:])]


def orgat_klassifikator(X, y, seed=0):
    torch.manual_seed(seed)
    k = Klassifikator()
    opt = torch.optim.Adam(k.parameters(), lr=3e-3, weight_decay=1e-4)
    g = torch.Generator().manual_seed(seed)
    for _ in range(800):
        i = torch.randint(0, len(X), (128,), generator=g)
        loss = F.cross_entropy(k(X[i] + 0.1 * torch.randn(X[i].shape, generator=g)), y[i])
        opt.zero_grad()
        loss.backward()
        opt.step()
    k.eval()
    return k


def xira(X):
    """3x3 o'rtacha filtr (chetlarda nol bilan to'ldirib)."""
    r = F.avg_pool2d(F.pad((X.view(-1, 1, 8, 8) + 1) / 2, (1, 1, 1, 1)), 3, stride=1)
    return (r * 2 - 1).view(-1, 64)


def generatorlar(oquv, held, seed):
    """Sun'iy buzilgan 'generatorlar': har biri N ta rasm qaytaradi."""
    g = torch.Generator().manual_seed(seed)
    (Xg, yg), (Xh, yh) = oquv, held
    idx = torch.randperm(len(Xh), generator=g)[:N]
    ideal = Xh[idx]
    besh = (yh < 5).nonzero()[:, 0]
    besh = besh[torch.randperm(len(besh), generator=g)[:N]]
    proto = torch.stack([Xg[yg == c].mean(0) for c in range(10)])
    return {
        "ideal (yangi haqiqiy)": ideal,
        "shovqin 0.3": (ideal + 0.3 * torch.randn(ideal.shape, generator=g)).clamp(-1, 1),
        "shovqin 0.6": (ideal + 0.6 * torch.randn(ideal.shape, generator=g)).clamp(-1, 1),
        "xira (3x3)": xira(ideal),
        "faqat 5 sinf": Xh[besh],
        "prototiplar (10 ta)": proto.repeat(N // 10, 1),
        "o'quvdan nusxa": Xg[torch.randperm(len(Xg), generator=g)[:N]],
    }


def frechet(a, b):
    """FD = ||mu1 - mu2||^2 + tr(S1) + tr(S2) - 2 tr(sqrt(S1 S2)).

    tr(sqrt(S1 S2)) = sum sqrt(lambda(sqrt(S1) S2 sqrt(S1))) - simmetrik va barqaror.
    """
    a, b = np.asarray(a, dtype=np.float64), np.asarray(b, dtype=np.float64)
    s1, s2 = np.cov(a, rowvar=False), np.cov(b, rowvar=False)
    w, v = np.linalg.eigh(s1)
    ild = (v * np.sqrt(np.clip(w, 0, None))) @ v.T
    oz = np.linalg.eigvalsh(ild @ s2 @ ild)
    return float(((a.mean(0) - b.mean(0)) ** 2).sum() + np.trace(s1) + np.trace(s2)
                 - 2 * np.sqrt(np.clip(oz, 0, None)).sum())


def inception_score(p):
    p = np.clip(p, 1e-12, 1.0)
    py = p.mean(0, keepdims=True)
    return float(np.exp((p * (np.log(p) - np.log(py))).sum(1).mean()))


def precision_recall(f_real, f_gen, k=3):
    def radius(f):
        d = np.sqrt(((f[:, None, :] - f[None, :, :]) ** 2).sum(-1))
        return np.sort(d, axis=1)[:, k]
    r_real, r_gen = radius(f_real), radius(f_gen)
    d = np.sqrt(((f_gen[:, None, :] - f_real[None, :, :]) ** 2).sum(-1))
    return float((d <= r_real[None, :]).any(1).mean()), float((d.T <= r_gen[None, :]).any(1).mean())


def main() -> None:
    torch.set_num_threads(1)
    klass, oquv, held, etalon = bolaklar()
    k = orgat_klassifikator(*klass)
    Xg, X_boshqa = oquv[0], klass[0][:len(oquv[0])]

    def xus_ehtimol(X):
        with torch.no_grad():
            f = k.xus(X)
            return f.double().numpy(), torch.softmax(k.bosh(f), 1).double().numpy()

    f_et, _ = xus_ehtimol(etalon[0])
    olchovlar = ("FD", "IS", "precision", "recall", "sinflar", "nisbat")

    def hammasi(X, f_ref):
        f, p = xus_ehtimol(X)
        pr, rc = precision_recall(f_ref, f)
        sinf = (np.bincount(p.argmax(1), minlength=10) / len(p) >= 0.05).sum()
        d_o = torch.cdist(X, Xg).min(1).values.median().item()
        d_b = torch.cdist(X, X_boshqa).min(1).values.median().item()
        return [frechet(f, f_et), inception_score(p), pr, rc, sinf, d_o / d_b]

    print(f"=== 1. Barcha o'lchovlar x barcha buzilishlar (N = {N}, 3 urug' o'rtachasi) ===")
    natija = {}
    for u in range(3):
        rng = np.random.default_rng(10 + u)
        f_ref = f_et[rng.choice(len(f_et), N, replace=False)]
        gen = generatorlar(oquv, held, u)
        gen = {n: gen[n] for n in ("ideal (yangi haqiqiy)", "shovqin 0.3", "xira (3x3)", "faqat 5 sinf",
                                   "prototiplar (10 ta)", "o'quvdan nusxa")}
        g = torch.Generator().manual_seed(100 + u)
        gen["nusxa + shovqin 0.1"] = (gen["o'quvdan nusxa"]
                                      + 0.1 * torch.randn((N, 64), generator=g)).clamp(-1, 1)
        for nom, X in gen.items():
            natija.setdefault(nom, []).append(hammasi(X, f_ref))
    print("  generator                  FD      IS   precision recall sinflar nisbat")
    for nom, v in natija.items():
        v = np.array(v).mean(0)
        print(f"  {nom:<22} {v[0]:>7.2f} {v[1]:>6.2f} {v[2]:>9.3f} {v[3]:>6.3f} {v[4]:>6.1f} {v[5]:>6.2f}")

    print("\n=== 2. Jadval: qaysi o'lchov qaysi buzilishni USHLADI (ideal bilan, 2*SE + amaliy chegara) ===")
    ideal = np.array(natija["ideal (yangi haqiqiy)"])
    im = ideal.mean(0)
    # yo'nalish (+1 - oshsa yomon, -1 - kamaysa yomon) va amaliy chegara
    yonalish = np.array([1, -1, -1, -1, -1, -1])
    chegara = np.array([0.5 * im[0], 0.05 * im[1], 0.05, 0.05, 0.5, 0.3])
    print("  generator               " + "".join(f"{o:>10}" for o in olchovlar))
    ushlash = {}
    for nom, v in natija.items():
        if nom.startswith("ideal"):
            continue
        f = (np.array(v) - ideal) * yonalish
        se = f.std(0, ddof=1) / math.sqrt(3)
        ush = (f.mean(0) > np.maximum(2 * se, chegara))
        ushlash[nom] = ush
        print(f"  {nom:<22}  " + "".join(f"{'HA' if x else '.':>10}" for x in ush))
    hech = [n for n, u in ushlash.items() if not u[:5].any()]
    print("  FD/IS/precision/recall/sinflar hech biri ushlamagan: " + (", ".join(hech) or "yo'q"))
    faqat = {o: sorted(n for n, u in ushlash.items() if u[j] and u.sum() == 1)
             for j, o in enumerate(olchovlar)}
    for o, n in faqat.items():
        if n:
            print(f"  faqat '{o}' ushlagan: {', '.join(n)}")

    print("\n=== 3. Bootstrap CI: FD farqi 'shovqin 0.3' - ideal (1-urug', 300 takror) ===")
    gen = generatorlar(oquv, held, 0)
    fa, _ = xus_ehtimol(gen["ideal (yangi haqiqiy)"])
    fb, _ = xus_ehtimol(gen["shovqin 0.3"])
    rng = np.random.default_rng(5)
    farq = []
    for _ in range(300):
        i, j = rng.integers(0, N, N), rng.integers(0, len(f_et), len(f_et))
        farq.append(frechet(fb[i], f_et[j]) - frechet(fa[i], f_et[j]))    # bir xil indekslar
    past, yuqori = np.percentile(farq, [2.5, 97.5])
    print(f"  farq {frechet(fb, f_et) - frechet(fa, f_et):+.2f}, 95% CI [{past:+.2f}, {yuqori:+.2f}] -> "
          + ("noldan farqli" if past > 0 or yuqori < 0 else "nolni o'z ichiga oladi"))

    print("\n=== 4. Inson baholashi: A/B (juftlashgan afzallik), simulyatsiya ===")
    print("  haqiqiy afzallik B uchun p = 0.55; ikki tomonlama binomial test, alfa = 0.05")
    rng = np.random.default_rng(7)
    print("  juftliklar   quvvat (B ni topish ehtimoli, 400 simulyatsiya)")
    for n in (50, 100, 200, 400, 800):
        sonlar = rng.binomial(n, 0.55, size=400).tolist()
        muhim = {kk: binomtest(kk, n).pvalue < 0.05 for kk in set(sonlar)}   # har k ga bitta test
        quvvat = np.mean([muhim[kk] for kk in sonlar])
        print(f"  {n:>9}    {quvvat:.2f}")
    print("  tuzoq - joylashuv moyilligi: farqni sezmagan baholovchi 60% CHAP rasmni tanlaydi")
    print("  (haqiqiy B ulushi: 0.5 * 0.6 + 0.5 * 0.5 = 0.55)")
    n = 400
    sezdi = rng.random(n) < 0.5                                       # yarmida farqni sezadi
    togri = rng.random(n) < 0.6                                       # sezganlar B ni 60% tanlaydi
    chap_tanlov = rng.random(n) < 0.6
    b_chapda = np.ones(n, dtype=bool)                                 # B har doim chapda
    b_tanladi = np.where(sezdi, togri, chap_tanlov == b_chapda)
    tasodif = rng.random(n) < 0.5                                     # tasodifiy joylashuv
    b_tanladi2 = np.where(sezdi, togri, chap_tanlov == tasodif)
    for nom, v in (("B doim chapda", b_tanladi), ("joylashuv tasodifiy", b_tanladi2)):
        print(f"  {nom:<20} B ulushi {v.mean():.3f}, p = {binomtest(int(v.sum()), n).pvalue:.4f}")


if __name__ == "__main__":
    main()

=== 1. Barcha o'lchovlar x barcha buzilishlar (N = 200, 3 urug' o'rtachasi) ===
  generator                  FD      IS   precision recall sinflar nisbat
  ideal (yangi haqiqiy)     5.59   9.26     0.955  0.928   10.0   1.00
  shovqin 0.3              15.57   8.85     0.875  0.865   10.0   1.00
  xira (3x3)              188.55   3.50     0.535  0.002    6.0   1.01
  faqat 5 sinf             44.03   5.12     0.947  0.553    5.0   1.01
  prototiplar (10 ta)      13.64  10.00     1.000  0.000   10.0   0.93
  o'quvdan nusxa            4.66   9.30     0.938  0.947   10.0   0.00
  nusxa + shovqin 0.1       5.81   9.22     0.938  0.948   10.0   0.27

=== 2. Jadval: qaysi o'lchov qaysi buzilishni USHLADI (ideal bilan, 2*SE + amaliy chegara) ===
  generator                       FD        IS precision    recall   sinflar    nisbat
  shovqin 0.3                     HA         .        HA        HA         .         .
  xira (3x3)                      HA        HA        HA        HA        HA         .
  faqat 5 sinf                    HA        HA         .        HA        HA         .
  prototiplar (10 ta)             HA         .         .        HA         .         .
  o'quvdan nusxa                   .         .         .         .         .        HA
  nusxa + shovqin 0.1              .         .         .         .         .        HA
  FD/IS/precision/recall/sinflar hech biri ushlamagan: o'quvdan nusxa, nusxa + shovqin 0.1
  faqat 'nisbat' ushlagan: nusxa + shovqin 0.1, o'quvdan nusxa

=== 3. Bootstrap CI: FD farqi 'shovqin 0.3' - ideal (1-urug', 300 takror) ===
  farq +10.38, 95% CI [+6.84, +12.69] -> noldan farqli

=== 4. Inson baholashi: A/B (juftlashgan afzallik), simulyatsiya ===
  haqiqiy afzallik B uchun p = 0.55; ikki tomonlama binomial test, alfa = 0.05
  juftliklar   quvvat (B ni topish ehtimoli, 400 simulyatsiya)
         50    0.07
        100    0.16
        200    0.26
        400    0.47
        800    0.82
  tuzoq - joylashuv moyilligi: farqni sezmagan baholovchi 60% CHAP rasmni tanlaydi
  (haqiqiy B ulushi: 0.5 * 0.6 + 0.5 * 0.5 = 0.55)
  B doim chapda        B ulushi 0.598, p = 0.0001
  joylashuv tasodifiy  B ulushi 0.555, p = 0.0314

fd_a = frechet(xus(gen_a[:50]), f_et)                 # 50 ta                # ⚠️
fd_b = frechet(xus(gen_b[:500]), f_et)                # 500 ta
fd_a = frechet(xus(gen_a[:N]), f_et)                                         # ✅
fd_b = frechet(xus(gen_b[:N]), f_et)
fd_ideal = frechet(xus(yangi_haqiqiy[:N]), f_et)      # nol nuqtasi

k = orgat_klassifikator(X_oquv, y_oquv)               # generator ham shunda  # ⚠️
k = orgat_klassifikator(X_klass, y_klass)             # alohida bo'lak        # ✅

tr_ildiz = np.trace(np.sqrt(s1 @ s2))                 # elementlar ildizi!    # ⚠️
w, v = np.linalg.eigh(s1)                                                    # ✅
ild = (v * np.sqrt(np.clip(w, 0, None))) @ v.T
tr_ildiz = np.sqrt(np.clip(np.linalg.eigvalsh(ild @ s2 @ ild), 0, None)).sum()

r = np.sort(d, axis=1)[:, k - 1]                      # 0-ustun - o'zi (0)    # ⚠️
r = np.sort(d, axis=1)[:, k]                          # o'zini tashlab        # ✅

shubhali = d_oquv < 1.0                               # nimaga nisbatan?      # ⚠️
nisbat = np.median(d_oquv) / np.median(d_boshqa)      # ko'rilmagan to'plam   # ✅

chap, ong = rasm_b, rasm_a                            # B doim chapda         # ⚠️
chap, ong = (rasm_b, rasm_a) if rng.random() < 0.5 else (rasm_a, rasm_b)      # ✅

print("B yaxshi" if fd(gen_b(0)) < fd(gen_a(0)) else "A yaxshi")             # ⚠️
farq = [fd(gen_b(u)) - fd(gen_a(u)) for u in range(3)]                        # ✅
se = np.std(farq, ddof=1) / np.sqrt(3)

1.  # FD formulasi?
2.  # ikki bir xil taqsimotdan olingan 25 talik to'plamlar uchun FD nolmi?
3.  # o'quv to'plamini aynan qaytaradigan generatorning FD si qanday bo'ladi?
4.  # IS ning maksimumi (10 sinf)?
5.  # hamma rasm bir sinfda va klassifikator ishonchli bo'lsa IS?
6.  # faqat 5 sinfni qaytaradigan generatorda precision va recall?
7.  # 10 ta rasmni takrorlaydigan generatorda recall (k = 3)?
8.  # yodlab olish nisbati formulasi va ideal qiymati?
9.  # nega d_oquv ning o'zi yetarli emas?
10. # bootstrap da FD ning o'rtachasi nima uchun siljiydi?
11. # afzallik 0.55 bo'lsa, 100 juftlik bilan uni topish ehtimoli taxminan?
12. # A/B da B doim chapda bo'lsa nima bo'ladi?

1.  fd_a = frechet(xus(gen_a[:100]), f_et)
    fd_b = frechet(xus(gen_b[:1000]), f_et)
    print("A yaxshi" if fd_a < fd_b else "B yaxshi")

2.  r = np.sort(d_real, axis=1)[:, 2]          # k = 3
    precision = (d_gen_real <= r[None]).any(1).mean()

3.  k = orgat_klassifikator(X_oquv, y_oquv)
    gen = orgat_generator(X_oquv)
    fd = frechet(xus(gen.sample(200)), xus(X_oquv))

4.  shubhali = (torch.cdist(X_gen, X_oquv).min(1).values < 1.0).float().mean()
    print("yodlamagan" if shubhali < 0.1 else "yodlagan")

5.  ulush = np.mean(b_tanlandi); print("B yaxshi" if ulush > 0.5 else "A yaxshi")