26.3-dars: GAN

IKKI TARMOQ:
  G(z):  z ~ N(0, I) -> soxta x          (generator, "qalbakichi")
  D(x):  x -> ehtimol "x haqiqiy"        (diskriminator, "ekspert")

O'YIN:
  D ning maqsadi: haqiqiyga 1, soxtaga 0 deyish (oddiy binar klassifikatsiya)
  G ning maqsadi: D soxtaga 1 desin

NIMA YO'Q:
  p(x) formulasi yo'q, likelihood yo'q, encoder yo'q
  G - faqat namuna chiqaradigan "implitsit" model (26.1, 2.2: GAN - p(x) YO'Q)
  sifat signali - D ning gradienti: "rasmni qaysi tomonga o'zgartirsang,
  haqiqiyroq ko'rinadi"

26.2 BILAN FARQ:
  VAE: x -> encoder -> z -> decoder -> x, piksel bo'yicha qayta tiklash loss
  GAN: z -> G -> x; "qayta tiklash" yo'q, piksel bo'yicha loss yo'q
  -> GAN piksel o'rtachasini emas, "haqiqiy ko'rinish" ni optimallashtiradi

QIYMAT FUNKSIYASI:
  V(D, G) = E_x~data[log D(x)] + E_z[log(1 - D(G(z)))]
  min_G max_D V(D, G)

OPTIMAL D (G belgilangan bo'lsa):
  D*(x) = p_data(x) / (p_data(x) + p_g(x))
  1-misol, 1D: p_data = N(0, 1), p_g = N(2, 1)
    x = -1: o'rgatilgan 0.984, nazariya 0.982
    x = +1: o'rgatilgan 0.501, nazariya 0.500 (o'rtada - teng)
    x = +3: o'rgatilgan 0.016, nazariya 0.018

D* DA QIYMAT:
  V(D*, G) = -log 4 + 2 * JSD(p_data || p_g)
  JSD - Jensen-Shannon divergensiyasi (simmetrik, 0 .. log 2)
  1-misol: o'rgatilgan D bilan -0.712, nazariya -1.386 + 2 * 0.337 = -0.713

GLOBAL OPTIMUM:
  p_g = p_data  ->  JSD = 0,  V = -log 4 = -1.386,  D* = 0.5 hamma joyda
  ya'ni ideal holda D hech narsani ajrata olmaydi

MINIMAX (asl) G LOSS:     G minimallashtiradi  log(1 - D(G(z)))
NON-SATURATING G LOSS:    G minimallashtiradi  -log D(G(z))
  ikkalasining statsionar nuqtasi bir xil, gradientlari - yo'q

LOGIT BO'YICHA GRADIENT (1-misol, 1-bo'lim):
  D(G(z))    minimax       non-saturating
  0.001      -0.0010       -0.9990
  0.01       -0.0100       -0.9900
  0.5        -0.5000       -0.5000
  0.9        -0.9000       -0.1000

NEGA MUHIM:
  o'rgatish boshida G yomon -> D soxtani oson rad etadi -> D(G(z)) ~ 0
  minimax: aynan shu paytda gradient ~ 0 ("to'yinish") -> G o'rganmaydi
  non-saturating: G eng yomon bo'lganda eng kuchli gradient oladi

1-MISOL, 3-BO'LIM (kuchli D, D(G(z)) = 0.0022, G - oddiy SGD):
  ||grad G||: minimax 0.01362, non-saturating 4.62931 (340 barobar)
  100 qadamdan keyin D(G(z)): minimax 0.0024, non-saturating 0.9972

AMALDA:
  deyarli hamma GAN non-saturating loss bilan o'rgatiladi
  BCE ko'rinishida: G uchun "soxtaga 1 yorlig'i":
    g_loss = BCE_with_logits(D(G(z)), 1)
  Adam gradient masshtabini qisman normallaydi - 2-misolda (muvozanatli D,
  Adam) ikki loss orasida sezilarli farq chiqmadi

HAR QADAM:
  1. D qadami:
     x ~ ma'lumot, soxta = G(z)
     d_loss = BCE(D(x), 1) + BCE(D(soxta.detach()), 0)     <- detach!
  2. G qadami:
     g_loss = BCE(D(soxta), 1)                              <- non-saturating
     faqat G ning optimizatori step qiladi

TIPIK SOZLAMA (DCGAN tavsiyalari asosida):
  Adam, lr = 2e-4, betas = (0.5, 0.999)
  LeakyReLU0.2-bob D da
  rasmlar [-1, 1] ga, G chiqishi tanh
  3-misol: xuddi shu GAN [0, 1] + sigmoid bilan - buzildi
    (Frechet 66.8, sinflar 3 taga qisqardi); tanh bilan - 8.1

DETACH NIMA UCHUN:
  D qadamida G ga gradient kerak emas (va G ning grad lari ifloslanmasin)
  G qadamida esa soxta D orqali G gacha differensiallanadi

MA'LUMOT: radiusi 2 bo'lgan halqada 8 ta Gauss, sigma = 0.05
O'LCHOVLAR:
  sifatli namuna: eng yaqin moda markazidan < 3 sigma = 0.15
  qamrab olingan moda: unga kamida 1% sifatli namuna tushgan
  haqiqiy ma'lumot: 8/8, sifatli 0.988

2-MISOL (1500 qadam, 3 urug'):
  non-saturating: qamrov 8/8 har urug'da; sifatli 0.554, 0.460, 0.592
  minimax:        qamrov 8/8 har urug'da; sifatli 0.513, 0.492, 0.512
  modalardagi sonlar notekis (masalan 32 dan 271 gacha)

NEGA SIFATLI ULUSH 1 GA YETMAYDI:
  G - uzluksiz funksiya, z fazosi (N(0, I)) - yaxlit
  8 ta ajralgan orolni uzluksiz akslantirish bilan "ko'priksiz" hosil qilib bo'lmaydi
  -> namunalarning bir qismi modalar orasidagi "ko'priklarda" (ASCII da ko'rinadi)

MUVOZANAT MA'LUMOTNOMASI:
  D(x) = D(G(z)) = 0.5 bo'lsa:
    D loss = 2 * ln 2 = 1.386,  G loss = ln 2 = 0.693

NIMANI KO'RSATADI:
  D(x) va D(G(z)) - kim ustun: D(x) ~ 1, D(G(z)) ~ 0 va D loss -> 0
    -> D butunlay ustun, G gradienti zaif (3-misol, sigmoid varianti:
       D loss 0.253, D(G(z)) 0.08 - va namunalar buzilgan)
  loss lar keskin tebransa yoki nan - beqarorlik

NIMANI KO'RSATMAYDI:
  namuna sifatini: 3-misolda (tanh) D loss 1000 qadamdan keyin deyarli
  o'zgarmadi (0.90 .. 1.06), Frechet esa 15.8 dan 8.1 gacha yaxshilandi
  xilma-xillikni: G bitta yaxshi rasm chizib ham D ni vaqtincha aldashi mumkin
  G loss ning kamayishi - D zaiflashgani ham bo'lishi mumkin

SHUNING UCHUN:
  har N qadamda namunalarni MUSTAQIL o'lchov bilan baholang:
  hakam ishonchi, sinf entropiyasi, Frechet masofa, yodlash tekshiruvi

G'OYA (FID ning soddalashtirilgan varianti):
  haqiqiy va soxta rasmlarni hakam-klassifikatorning yashirin qatlamiga
  o'tkazamiz (bizda 128 o'lchov), har to'plamga Gauss moslaymiz:
  FD = ||mu_1 - mu_2||^2 + tr(C_1 + C_2 - 2 * sqrt(C_1 * C_2))
  0 - taqsimotlar (shu xususiyatlarda) bir xil

MA'LUMOTNOMA:
  450 ta haqiqiy o'quv rasmi va test: 0.712 (namuna kichikligi tufayli 0 emas)

CHEKLOVLAR:
  xususiyatlar hakamga bog'liq - boshqa hakam boshqa raqam beradi
  namuna soniga bog'liq - bir xil n bilan solishtiring
  sifat va xilma-xillikni bitta songa aralashtiradi
  standart FID (Inception tarmog'i) - Generativ modellarni baholash darsida

4-MISOL (3 urug', bir xil hakam, bir xil 1000 namuna):
  o'lchov            GAN      VAE     haqiqiy
  ishonch>0.9       0.632    0.568    1.000
  sinf entropiya    0.861    0.995    0.998
  Frechet          17.877    6.274    0.712
  eng kam sinf      0.009    0.076    0.087
  sinf ichi xilma   1.715    1.192    2.256
  keskin piksel     0.753    0.452    0.656
  (hamma farqlar 2 * SE dan katta)

TALQIN:
  GAN: o'tkir (keskin piksellar haqiqiydan ham ko'p), sinf ichida xilma-xil,
       lekin ba'zi sinflarni deyarli chizmaydi (eng kam sinf 0.9%)
  VAE: xira, sinf ichida bir xilroq, lekin hamma sinfni tekis qamraydi
  Frechet bu kichik tajribada VAE ni yaxshi deydi - qamrov og'ir tortadi
  GAN Frechet i urug'lar bo'yicha keng tarqalgan (16.6, 14.1, 23.0) - beqarorlik

HALOLLIK:
  GAN 1500 qadam, VAE 40 davr - sozlash byudjetlari teng emas
  3-misolda GAN 3000 qadamda Frechet 8.1 ga yetdi
  "GAN o'tkirroq, VAE qamrovliroq" - adabiyotdagi umumiy manzara bilan mos,
  lekin aniq raqamlar kichik ma'lumot va kichik modellarga xos

HODISA:
  G ma'lumotning faqat bir qismini (bir necha moda/sinf) chizadi
  D ni aldash uchun hamma narsani chizish shart emas - bitta ishonchli rasm yetadi

BU DARSDA KO'RINGAN BELGILAR:
  3-misol, sigmoid varianti: 1000 namunaning 654 tasi "8" (3 ta sinf)
  3-misol, tanh: sinflar 5 va 9 atigi 25 tadan (boshqalari 97 .. 142)
  4-misol: eng kam sinf ulushi GAN da 0.009, VAE da 0.076

O'LCHASH: sinf entropiyasi, eng kam sinf ulushi, 2D da modalar soni
CHUQUR TAHLIL VA YECHIMLAR (minibatch diskriminatsiya, WGAN, spektral
normallash va boshqalar) - keyingi darsda

import torch
import torch.nn as nn
import torch.nn.functional as F

G = nn.Sequential(nn.Linear(16, 128), nn.LeakyReLU(0.2), nn.Linear(128, 128), nn.LeakyReLU(0.2),
                  nn.Linear(128, 64), nn.Tanh())                  # rasmlar [-1, 1] da
D = nn.Sequential(nn.Linear(64, 128), nn.LeakyReLU(0.2), nn.Linear(128, 128), nn.LeakyReLU(0.2),
                  nn.Linear(128, 1))                              # logit
opt_g = torch.optim.Adam(G.parameters(), lr=2e-4, betas=(0.5, 0.999))
opt_d = torch.optim.Adam(D.parameters(), lr=2e-4, betas=(0.5, 0.999))

for x in batchlar:
    soxta = G(torch.randn(len(x), 16, generator=g))
    d_loss = (F.binary_cross_entropy_with_logits(D(x), torch.ones(len(x), 1)) +
              F.binary_cross_entropy_with_logits(D(soxta.detach()), torch.zeros(len(x), 1)))
    opt_d.zero_grad(); d_loss.backward(); opt_d.step()
    g_loss = F.binary_cross_entropy_with_logits(D(soxta), torch.ones(len(x), 1))  # non-saturating
    opt_g.zero_grad(); g_loss.backward(); opt_g.step()

# barqaror minimax ko'rinishi (faqat solishtirish uchun)
g_minimax = F.logsigmoid(-D(soxta)).mean()                       # log(1 - D)

# baholash: loss emas - namunalar
xs = (G(z_bah) + 1) / 2                                           # [0, 1] ga
ishonch, entropiya, frechet = olchovlar(hakam, xs, x_test)

min_G max_D  E[log D(x)] + E[log(1 - D(G(z)))];  D* = p_data / (p_data + p_g)
optimumda JSD = 0, D = 0.5, V = -log 4
G uchun non-saturating: -log D(G(z)) - kuchli D da ham gradient
navbat: D (soxta.detach) -> G; Adam(2e-4, betas = (0.5, 0.999)); tanh + [-1, 1]
loss - o'yin holati; sifat - hakam, entropiya, Frechet, yodlash
GAN: o'tkir, lekin beqaror va mode collapse ga moyil

"""GAN o'yini: optimal diskriminator, minimax qiymati va non-saturating generator loss."""

import math

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F


def tarmoq(kirish, chiqish, h=64):
    return nn.Sequential(nn.Linear(kirish, h), nn.LeakyReLU(0.2), nn.Linear(h, h), nn.LeakyReLU(0.2),
                         nn.Linear(h, chiqish))


def normal_zichlik(x, mu):
    return np.exp(-0.5 * (x - mu) ** 2) / math.sqrt(2 * math.pi)


def main() -> None:
    torch.set_num_threads(1)

    print("=== 1. Generator gradienti: d(loss)/d(logit), logit - D ning soxta namunaga chiqishi ===")
    print("  D(G(z))   minimax: log(1 - D)   non-saturating: -log D")
    for d in (0.001, 0.01, 0.1, 0.5, 0.9):
        logit = torch.tensor(math.log(d / (1 - d)), requires_grad=True)
        torch.log1p(-torch.sigmoid(logit)).backward()               # G buni MINIMALLASHTIRADI
        g_mm = logit.grad.item()
        logit = torch.tensor(math.log(d / (1 - d)), requires_grad=True)
        (-F.logsigmoid(logit)).backward()
        g_ns = logit.grad.item()
        print(f"  {d:<8} {g_mm:>14.4f} {g_ns:>22.4f}")
    print("  D soxtani ishonch bilan rad etganda (D -> 0) minimax gradienti ham -> 0")

    print("\n=== 2. Optimal diskriminator (1D): p_data = N(0, 1), p_g = N(2, 1) ===")
    torch.manual_seed(0)
    D = tarmoq(1, 1, 32)
    opt = torch.optim.Adam(D.parameters(), lr=3e-3)
    g = torch.Generator().manual_seed(0)
    for _ in range(1500):
        xr = torch.randn(256, 1, generator=g)
        xg = 2 + torch.randn(256, 1, generator=g)
        loss = (F.binary_cross_entropy_with_logits(D(xr), torch.ones(256, 1)) +
                F.binary_cross_entropy_with_logits(D(xg), torch.zeros(256, 1)))
        opt.zero_grad()
        loss.backward()
        opt.step()
    print("  x      D(x) o'rgatilgan   D*(x) = p_data / (p_data + p_g)")
    for x in (-1.0, 0.0, 1.0, 2.0, 3.0):
        with torch.no_grad():
            dx = torch.sigmoid(D(torch.tensor([[x]]))).item()
        ds = normal_zichlik(x, 0) / (normal_zichlik(x, 0) + normal_zichlik(x, 2))
        print(f"  {x:+.1f} {dx:>13.3f} {ds:>20.3f}")
    with torch.no_grad():
        xr = torch.randn(100000, 1, generator=g)
        xg = 2 + torch.randn(100000, 1, generator=g)
        v = (F.logsigmoid(D(xr)).mean() + F.logsigmoid(-D(xg)).mean()).item()
    t = np.linspace(-8, 10, 20001)
    pd, pg = normal_zichlik(t, 0), normal_zichlik(t, 2)
    m = 0.5 * (pd + pg)
    jsd = 0.5 * np.trapezoid(pd * np.log(pd / m), t) + 0.5 * np.trapezoid(pg * np.log(pg / m), t)
    print(f"  V(D, G) = E[log D(x)] + E[log(1 - D(G(z)))]: o'rgatilgan D bilan {v:.3f}")
    print(f"  nazariya: -log 4 + 2 * JSD = {-math.log(4):.3f} + 2 * {jsd:.3f} = {-math.log(4) + 2 * jsd:.3f}")
    print(f"  p_g = p_data bo'lsa: JSD = 0, V = -log 4 = {-math.log(4):.3f}, D* = 0.5 hamma joyda")

    print("\n=== 3. Kuchli D qarshisida: generator qaysi loss bilan qochib chiqadi? (2D) ===")
    modalar = torch.tensor([[2 * math.cos(2 * math.pi * k / 8), 2 * math.sin(2 * math.pi * k / 8)]
                            for k in range(8)])
    natija = {}
    print("  avval faqat D 300 qadam o'rgatiladi; keyin D muzlatiladi va G 100 ta SGD qadami qiladi")
    for tur in ("minimax", "non-saturating"):
        torch.manual_seed(1)
        G, D = tarmoq(2, 2), tarmoq(2, 1)
        opt_d = torch.optim.Adam(D.parameters(), lr=1e-3, betas=(0.5, 0.999))
        opt_g = torch.optim.SGD(G.parameters(), lr=0.01)     # oddiy gradient: Adam uni normallab yuboradi
        g = torch.Generator().manual_seed(1)
        for _ in range(300):                                 # faqat D o'rganadi: soxtani yaxshi taniydi
            x = modalar[torch.randint(0, 8, (128,), generator=g)] + 0.05 * torch.randn(128, 2, generator=g)
            with torch.no_grad():
                soxta = G(torch.randn(128, 2, generator=g))
            loss = (F.binary_cross_entropy_with_logits(D(x), torch.ones(128, 1)) +
                    F.binary_cross_entropy_with_logits(D(soxta), torch.zeros(128, 1)))
            opt_d.zero_grad()
            loss.backward()
            opt_d.step()
        z = torch.randn(1000, 2, generator=torch.Generator().manual_seed(7))
        logit = D(G(z))
        gl = F.logsigmoid(-logit).mean() if tur == "minimax" else -F.logsigmoid(logit).mean()
        G.zero_grad()
        gl.backward()
        norma = math.sqrt(sum((p.grad ** 2).sum().item() for p in G.parameters()))
        d0 = torch.sigmoid(logit).mean().item()
        if tur == "minimax":
            print(f"  boshlang'ich holat: D(G(z)) = {d0:.4f} - D soxtani deyarli aniq rad etadi")
        for p in D.parameters():
            p.requires_grad_(False)                          # D muzlatilgan: faqat G qadam qiladi
        yol = []
        for q in range(100):
            logit = D(G(torch.randn(128, 2, generator=g)))
            gl = F.logsigmoid(-logit).mean() if tur == "minimax" else -F.logsigmoid(logit).mean()
            opt_g.zero_grad()
            gl.backward()
            opt_g.step()
            if (q + 1) % 25 == 0:
                with torch.no_grad():
                    yol.append(torch.sigmoid(D(G(z))).mean().item())
        natija[tur] = (d0, norma)
        print(f"  {tur:<15} ||grad G|| = {norma:.5f}; D(G(z)) 25/50/75/100 SGD qadamda: "
              + " ".join(f"{v:.4f}" for v in yol))
    nisbat = natija["non-saturating"][1] / natija["minimax"][1]
    print(f"  gradient normasi nisbati (ns / minimax): {nisbat:.0f} barobar")


if __name__ == "__main__":
    main()

=== 1. Generator gradienti: d(loss)/d(logit), logit - D ning soxta namunaga chiqishi ===
  D(G(z))   minimax: log(1 - D)   non-saturating: -log D
  0.001           -0.0010                -0.9990
  0.01            -0.0100                -0.9900
  0.1             -0.1000                -0.9000
  0.5             -0.5000                -0.5000
  0.9             -0.9000                -0.1000
  D soxtani ishonch bilan rad etganda (D -> 0) minimax gradienti ham -> 0

=== 2. Optimal diskriminator (1D): p_data = N(0, 1), p_g = N(2, 1) ===
  x      D(x) o'rgatilgan   D*(x) = p_data / (p_data + p_g)
  -1.0         0.984                0.982
  +0.0         0.885                0.881
  +1.0         0.501                0.500
  +2.0         0.108                0.119
  +3.0         0.016                0.018
  V(D, G) = E[log D(x)] + E[log(1 - D(G(z)))]: o'rgatilgan D bilan -0.712
  nazariya: -log 4 + 2 * JSD = -1.386 + 2 * 0.337 = -0.713
  p_g = p_data bo'lsa: JSD = 0, V = -log 4 = -1.386, D* = 0.5 hamma joyda

=== 3. Kuchli D qarshisida: generator qaysi loss bilan qochib chiqadi? (2D) ===
  avval faqat D 300 qadam o'rgatiladi; keyin D muzlatiladi va G 100 ta SGD qadami qiladi
  boshlang'ich holat: D(G(z)) = 0.0022 - D soxtani deyarli aniq rad etadi
  minimax         ||grad G|| = 0.01362; D(G(z)) 25/50/75/100 SGD qadamda: 0.0022 0.0023 0.0023 0.0024
  non-saturating  ||grad G|| = 4.62931; D(G(z)) 25/50/75/100 SGD qadamda: 0.9854 0.9940 0.9962 0.9972
  gradient normasi nisbati (ns / minimax): 340 barobar

"""GAN noldan 2D da: 8 ta Gauss halqasi, qamrab olingan modalar va sifatli namunalar ulushi."""

import math

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F

MODALAR = torch.tensor([[2 * math.cos(2 * math.pi * k / 8), 2 * math.sin(2 * math.pi * k / 8)]
                        for k in range(8)])
SIGMA = 0.05


def haqiqiy(n, g):
    k = torch.randint(0, 8, (n,), generator=g)
    return MODALAR[k] + SIGMA * torch.randn(n, 2, generator=g)


def tarmoq(kirish, chiqish, h=64):
    return nn.Sequential(nn.Linear(kirish, h), nn.LeakyReLU(0.2), nn.Linear(h, h), nn.LeakyReLU(0.2),
                         nn.Linear(h, chiqish))


def orgat(seed, g_loss="ns", qadamlar=1500, B=128, z_dim=2):
    """Diskriminator va generator navbat bilan; g_loss: 'ns' (non-saturating) yoki 'minimax'."""
    torch.manual_seed(seed)
    G, D = tarmoq(z_dim, 2), tarmoq(2, 1)
    opt_g = torch.optim.Adam(G.parameters(), lr=1e-3, betas=(0.5, 0.999))
    opt_d = torch.optim.Adam(D.parameters(), lr=1e-3, betas=(0.5, 0.999))
    g = torch.Generator().manual_seed(seed)
    tarix = []
    for q in range(qadamlar):
        x = haqiqiy(B, g)
        soxta = G(torch.randn(B, z_dim, generator=g))
        # D: haqiqiy -> 1, soxta -> 0
        d_loss = (F.binary_cross_entropy_with_logits(D(x), torch.ones(B, 1)) +
                  F.binary_cross_entropy_with_logits(D(soxta.detach()), torch.zeros(B, 1)))
        opt_d.zero_grad()
        d_loss.backward()
        opt_d.step()
        # G
        logit = D(soxta)
        if g_loss == "ns":
            gl = F.binary_cross_entropy_with_logits(logit, torch.ones(B, 1))      # -log D(G(z))
        else:
            gl = -F.binary_cross_entropy_with_logits(logit, torch.zeros(B, 1))    # log(1 - D(G(z)))
        opt_g.zero_grad()
        gl.backward()
        opt_g.step()
        if (q + 1) % 500 == 0:
            tarix.append((q + 1, d_loss.item(), gl.item()))
    return G.eval(), tarix


@torch.no_grad()
def baholash(G, n=2000, z_dim=2, seed=99):
    x = G(torch.randn(n, z_dim, generator=torch.Generator().manual_seed(seed)))
    d = torch.cdist(x, MODALAR)
    eng, k = d.min(1)
    sifatli = eng < 3 * SIGMA
    soni = torch.bincount(k[sifatli], minlength=8)
    qamrov = int((soni >= 0.01 * n).sum())            # kamida 1% sifatli namuna bor modalar
    return qamrov, sifatli.float().mean().item(), soni.tolist(), x


def ascii_zichlik(x, n=21):
    chet = torch.linspace(-2.6, 2.6, n + 1)
    c = torch.histogramdd(x.flip(1), bins=[chet, chet]).hist.flip(0)
    c = (c / c.max()).sqrt()
    return ["".join(" .:-=+*#%@"[min(9, int(v * 10))] for v in q) for q in c.tolist()]


def main() -> None:
    torch.set_num_threads(1)
    print("=== 1. Haqiqiy taqsimot: radiusi 2 bo'lgan halqada 8 ta Gauss (sigma 0.05) ===")
    print("  'sifatli' namuna: eng yaqin moda markazidan < 3 sigma = 0.15")
    print("  moda 'qamrab olingan': unga namunalarning kamida 1% i sifatli tushgan")
    q, s, _, _ = baholash(lambda z: haqiqiy(len(z), torch.Generator().manual_seed(5)))
    print(f"  haqiqiy ma'lumotning o'zi: qamrov {q}/8, sifatli {s:.3f}")

    print("\n=== 2. Non-saturating GAN, 3 urug' (1500 qadam) ===")
    natija = {}
    for tur in ("ns", "minimax"):
        natija[tur] = []
        for s in range(3):
            G, tarix = orgat(s, tur)
            natija[tur].append(baholash(G) + (tarix,))
    for s, (q, sf, soni, _, _) in enumerate(natija["ns"]):
        print(f"  urug' {s}: qamrov {q}/8, sifatli {sf:.3f}, modalar: {soni}")

    print("\n=== 3. Saturating (minimax) generator loss bilan, xuddi shu urug'lar ===")
    for s, (q, sf, soni, _, _) in enumerate(natija["minimax"]):
        print(f"  urug' {s}: qamrov {q}/8, sifatli {sf:.3f}, modalar: {soni}")
    for i, nom in ((0, "qamrov"), (1, "sifatli")):
        f = np.array([r[i] for r in natija["ns"]]) - np.array([r[i] for r in natija["minimax"]])
        se = f.std(ddof=1) / math.sqrt(3)
        print(f"  ns - minimax, {nom:<7}: {f.mean():+.3f}, SE {se:.3f} -> "
              f"{'sezilarli' if abs(f.mean()) > 2 * se else 'sezilarli emas'}")

    print("\n=== 4. Loss lar (urug' 0, ns): D va G loss vaqt bo'yicha ===")
    for qd, dl, gl in natija["ns"][0][4]:
        print(f"  qadam {qd:>4}: D loss {dl:.3f}, G loss {gl:.3f}")
    print(f"  (muvozanatda D(x) = 0.5: D loss = 2 ln 2 = {2 * math.log(2):.3f}, G loss = ln 2 = {math.log(2):.3f})")

    print("\n=== 5. Namunalar zichligi (urug' 0, ns) va haqiqiy ===")
    a = ascii_zichlik(natija["ns"][0][3])
    b = ascii_zichlik(haqiqiy(2000, torch.Generator().manual_seed(5)))
    print(f"  {'GAN':<21}   haqiqiy")
    for p, r in zip(a, b):
        print(f"  {p}   {r}")


if __name__ == "__main__":
    main()

=== 1. Haqiqiy taqsimot: radiusi 2 bo'lgan halqada 8 ta Gauss (sigma 0.05) ===
  'sifatli' namuna: eng yaqin moda markazidan < 3 sigma = 0.15
  moda 'qamrab olingan': unga namunalarning kamida 1% i sifatli tushgan
  haqiqiy ma'lumotning o'zi: qamrov 8/8, sifatli 0.988

=== 2. Non-saturating GAN, 3 urug' (1500 qadam) ===
  urug' 0: qamrov 8/8, sifatli 0.554, modalar: [40, 232, 58, 213, 41, 32, 271, 221]
  urug' 1: qamrov 8/8, sifatli 0.460, modalar: [161, 27, 219, 45, 88, 200, 57, 123]
  urug' 2: qamrov 8/8, sifatli 0.592, modalar: [52, 202, 174, 113, 62, 164, 193, 225]

=== 3. Saturating (minimax) generator loss bilan, xuddi shu urug'lar ===
  urug' 0: qamrov 8/8, sifatli 0.513, modalar: [161, 33, 137, 190, 75, 104, 138, 188]
  urug' 1: qamrov 8/8, sifatli 0.492, modalar: [129, 132, 193, 25, 118, 123, 168, 96]
  urug' 2: qamrov 8/8, sifatli 0.512, modalar: [25, 152, 176, 152, 66, 160, 101, 191]
  ns - minimax, qamrov : +0.000, SE 0.000 -> sezilarli emas
  ns - minimax, sifatli: +0.030, SE 0.033 -> sezilarli emas

=== 4. Loss lar (urug' 0, ns): D va G loss vaqt bo'yicha ===
  qadam  500: D loss 1.261, G loss 0.790
  qadam 1000: D loss 1.128, G loss 1.007
  qadam 1500: D loss 1.144, G loss 1.045
  (muvozanatda D(x) = 0.5: D loss = 2 ln 2 = 1.386, G loss = ln 2 = 0.693)

=== 5. Namunalar zichligi (urug' 0, ns) va haqiqiy ===
  GAN                     haqiqiy

                                    .
          :-=-.                     @
        .::. .:::
      =%.       .-%           #-         -%
      -.          =           -.         .-
     .:           :.
     :             :
    :.             .:
    -               -
    -               -       @               @.
    -               :
    ::              :
    .:.            :
     -.            .
     .-           -           -.         .-
      .-:       .-%           %-         -%
        ....  ::.
         ..-@:                      @
           .                        .

"""GAN digits (8x8) da: o'rgatish dinamikasi, namunalar, tanib olinish va sinf qamrovi."""

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split

BELGI = " .:-=+*#%@"
Z = 16


def ascii_qator(rasmlar, izohlar):
    satrlar = ["  ".join(f"{s:<8}" for s in izohlar)]
    for r in range(8):
        satrlar.append("  ".join("".join(BELGI[min(9, int(v * 10))] for v in im.reshape(8, 8)[r])
                                 for im in np.clip(rasmlar, 0, 1)))
    return satrlar


def malumot():
    d = load_digits()
    x_o, x_t, y_o, y_t = train_test_split(d.data / 16, d.target, test_size=0.25,
                                          stratify=d.target, random_state=0)
    f = lambda a: torch.tensor(a, dtype=torch.float32)
    return f(x_o), f(x_t), torch.tensor(y_o), torch.tensor(y_t)


class Klassifikator(nn.Module):
    def __init__(self):
        super().__init__()
        self.h = nn.Linear(64, 128)
        self.chiq = nn.Linear(128, 10)

    def forward(self, x):
        return self.chiq(F.relu(self.h(x)))


def hakam_orgat(x, y, seed=0):
    torch.manual_seed(seed)
    m = Klassifikator()
    opt = torch.optim.Adam(m.parameters(), lr=1e-2, weight_decay=1e-4)
    g = torch.Generator().manual_seed(seed)
    for _ in range(60):
        for idx in torch.randperm(len(x), generator=g).split(64):
            loss = F.cross_entropy(m(x[idx]), y[idx])
            opt.zero_grad()
            loss.backward()
            opt.step()
    return m.eval()


@torch.no_grad()
def olchovlar(hakam, x, x_ref):
    """Ishonch > 0.9 ulushi, sinf entropiyasi (0..1) va hakam xususiyatlarida Frechet masofa."""
    p = torch.softmax(hakam(x), -1)
    soni = np.bincount(p.argmax(1).numpy(), minlength=10)
    u = soni[soni > 0] / soni.sum()
    entr = float(-(u * np.log(u)).sum() / np.log(10))
    f1, f2 = F.relu(hakam.h(x)).numpy().astype(np.float64), F.relu(hakam.h(x_ref)).numpy().astype(np.float64)
    m1, m2 = f1.mean(0), f2.mean(0)
    c1, c2 = np.cov(f1, rowvar=False), np.cov(f2, rowvar=False)
    w, v = np.linalg.eigh(c1)
    s1 = (v * np.sqrt(np.clip(w, 0, None))) @ v.T                  # sqrt(C1)
    iz = np.sqrt(np.clip(np.linalg.eigvalsh(s1 @ c2 @ s1), 0, None)).sum()   # tr sqrt(C1 C2)
    fd = float(((m1 - m2) ** 2).sum() + np.trace(c1) + np.trace(c2) - 2 * iz)
    return (p.max(1).values > 0.9).float().mean().item(), entr, fd, soni


def gan_orgat(x_o, hakam, x_t, tanh=True, seed=0, qadamlar=3000, B=64, har=500):
    """Non-saturating GAN; tanh=True: rasmlar [-1, 1] ga o'tkaziladi, G chiqishi tanh."""
    torch.manual_seed(seed)
    G = nn.Sequential(nn.Linear(Z, 128), nn.LeakyReLU(0.2), nn.Linear(128, 128), nn.LeakyReLU(0.2),
                      nn.Linear(128, 64), nn.Tanh() if tanh else nn.Sigmoid())
    D = nn.Sequential(nn.Linear(64, 128), nn.LeakyReLU(0.2), nn.Linear(128, 128), nn.LeakyReLU(0.2),
                      nn.Linear(128, 1))
    opt_g = torch.optim.Adam(G.parameters(), lr=2e-4, betas=(0.5, 0.999), foreach=True)
    opt_d = torch.optim.Adam(D.parameters(), lr=2e-4, betas=(0.5, 0.999), foreach=True)
    g = torch.Generator().manual_seed(seed)
    xo = x_o * 2 - 1 if tanh else x_o
    rasm = (lambda t: (t + 1) / 2) if tanh else (lambda t: t)       # [0, 1] ga qaytarish
    z_bah = torch.randn(1000, Z, generator=torch.Generator().manual_seed(9))
    tarix, qator = [], []
    for q in range(1, qadamlar + 1):
        x = xo[torch.randint(0, len(xo), (B,), generator=g)]
        soxta = G(torch.randn(B, Z, generator=g))
        l_h, l_s = D(x), D(soxta.detach())
        d_loss = (F.binary_cross_entropy_with_logits(l_h, torch.ones(B, 1)) +
                  F.binary_cross_entropy_with_logits(l_s, torch.zeros(B, 1)))
        opt_d.zero_grad()
        d_loss.backward()
        opt_d.step()
        g_loss = F.binary_cross_entropy_with_logits(D(soxta), torch.ones(B, 1))   # non-saturating
        opt_g.zero_grad()
        g_loss.backward()
        opt_g.step()
        tarix.append((d_loss.item(), g_loss.item(), torch.sigmoid(l_h).mean().item(),
                      torch.sigmoid(l_s).mean().item()))
        if har and q % har == 0:
            with torch.no_grad():
                xs = rasm(G(z_bah))
            ish, entr, fd, _ = olchovlar(hakam, xs, x_t)
            qator.append((q, *np.mean(tarix[-100:], 0), ish, entr, fd))
    with torch.no_grad():
        xs = rasm(G(z_bah))
    return xs, qator


def main() -> None:
    torch.set_num_threads(1)
    x_o, x_t, y_o, y_t = malumot()
    hakam = hakam_orgat(x_o, y_o)
    ish, entr, fd, _ = olchovlar(hakam, x_o[:450], x_t)
    print("=== 1. Ma'lumotnoma: haqiqiy o'quv rasmlari (450 ta) test ga nisbatan ===")
    print(f"  ishonch>0.9 {ish:.3f}, sinf entropiyasi {entr:.3f}, Frechet masofa {fd:.1f}")

    print("\n=== 2. O'rgatish dinamikasi: G (16 -> 128 -> 128 -> 64, tanh), D (64 -> 128 -> 128 -> 1) ===")
    xs, qator = gan_orgat(x_o, hakam, x_t)
    print("  qadam   D loss   G loss   D(x)   D(G(z))   ishonch>0.9   entropiya   Frechet")
    for q, dl, gl, dx, dg, ish, entr, fd in qator:
        print(f"  {q:>5} {dl:>8.3f} {gl:>8.3f} {dx:>6.2f} {dg:>9.2f} {ish:>12.3f} {entr:>11.3f} {fd:>9.1f}")
    kor = np.corrcoef([r[1] for r in qator], [r[7] for r in qator])[0, 1]
    print(f"  D loss va Frechet korrelyatsiyasi (6 nuqta): {kor:+.2f}")

    print("\n=== 3. Yakuniy namunalar (3000 qadam) ===")
    ish, entr, fd, soni = olchovlar(hakam, xs, x_t)
    print(f"  ishonch>0.9 {ish:.3f}, entropiya {entr:.3f}, Frechet {fd:.1f}; sinflar: {' '.join(map(str, soni))}")
    p = hakam(xs[:8]).argmax(1).tolist()
    for s in ascii_qator(xs[:8].numpy(), [f"GAN:{a}" for a in p]):
        print("  " + s)
    d_test = torch.cdist(x_t, x_o).min(1).values
    chegara = torch.quantile(d_test, 0.05)
    d_gan = torch.cdist(xs, x_o).min(1).values
    print(f"  eng yaqin o'quv rasmigacha: test median {d_test.median():.3f}, GAN median {d_gan.median():.3f}; "
          f"nusxa (< {chegara:.3f}): GAN {(d_gan < chegara).float().mean():.3f}")

    print("\n=== 4. Xuddi shu GAN, lekin rasmlar [0, 1] va G chiqishi sigmoid ===")
    xs2, qator2 = gan_orgat(x_o, hakam, x_t, tanh=False, qadamlar=2000, har=1000)
    for q, dl, gl, dx, dg, ish, entr, fd in qator2:
        print(f"  {q:>5} D loss {dl:.3f}, D(G(z)) {dg:.2f}, ishonch>0.9 {ish:.3f}, "
              f"entropiya {entr:.3f}, Frechet {fd:.1f}")
    _, _, _, soni2 = olchovlar(hakam, xs2, x_t)
    print(f"  sinflar: {' '.join(map(str, soni2))}")


if __name__ == "__main__":
    main()

=== 1. Ma'lumotnoma: haqiqiy o'quv rasmlari (450 ta) test ga nisbatan ===
  ishonch>0.9 1.000, sinf entropiyasi 0.998, Frechet masofa 0.7

=== 2. O'rgatish dinamikasi: G (16 -> 128 -> 128 -> 64, tanh), D (64 -> 128 -> 128 -> 1) ===
  qadam   D loss   G loss   D(x)   D(G(z))   ishonch>0.9   entropiya   Frechet
    500    0.840    1.280   0.72      0.36        0.228       0.518      44.0
   1000    0.983    1.260   0.65      0.35        0.542       0.964      15.8
   1500    0.901    1.432   0.68      0.32        0.594       0.846      16.6
   2000    1.056    1.223   0.63      0.36        0.605       0.936      13.2
   2500    1.002    1.312   0.65      0.35        0.571       0.972       9.3
   3000    0.971    1.237   0.65      0.35        0.601       0.957       8.1
  D loss va Frechet korrelyatsiyasi (6 nuqta): -0.79

=== 3. Yakuniy namunalar (3000 qadam) ===
  ishonch>0.9 0.601, entropiya 0.957, Frechet 8.1; sinflar: 142 115 124 107 97 25 117 117 131 25
  GAN:7     GAN:8     GAN:5     GAN:3     GAN:7     GAN:3     GAN:7     GAN:1
    +@+       -@@##     *@@+       +@%#    .@@=.      :%@*       =++-     %@-.
    +%@:      @.+@:     @++%      @-#@=    ***@       @*-%       %@ %    -=%@
    :%@-      = :+     .@..       : #-      .@@.     .@ .*       %@#%     -@@
   :*=+:.     .@@       @@@.       @@      +.@:*=    .=*@+      : %#%    *%@ -.
   *@@@=.     +@@       .%@-       @@      :#%+%:      -%%     =+@@@.    :@@#*
    @@@       .@@       . *+:      @@       =%@--       .%=    . %@@      -#@*.
    #%@       #:*:      @.#@      --%+     :%+@.      *--@:      %@.     :##@=
    ++        .*.       +@@+       -#:      @@=       .@@=        :       @#@
  eng yaqin o'quv rasmigacha: test median 1.044, GAN median 1.688; nusxa (< 0.714): GAN 0.000

=== 4. Xuddi shu GAN, lekin rasmlar [0, 1] va G chiqishi sigmoid ===
   1000 D loss 0.354, D(G(z)) 0.13, ishonch>0.9 0.220, entropiya 0.281, Frechet 59.8
   2000 D loss 0.253, D(G(z)) 0.08, ishonch>0.9 0.561, entropiya 0.381, Frechet 66.8
  sinflar: 0 0 0 0 0 137 0 0 654 209

"""GAN va VAE ni bir xil o'lchovlarda solishtirish: tanib olinish, qamrov, Frechet, yodlash."""

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split

BELGI = " .:-=+*#%@"
Z = 16


def ascii_qator(rasmlar, izohlar):
    satrlar = ["  ".join(f"{s:<8}" for s in izohlar)]
    for r in range(8):
        satrlar.append("  ".join("".join(BELGI[min(9, int(v * 10))] for v in im.reshape(8, 8)[r])
                                 for im in np.clip(rasmlar, 0, 1)))
    return satrlar


def malumot():
    d = load_digits()
    x_o, x_t, y_o, y_t = train_test_split(d.data / 16, d.target, test_size=0.25,
                                          stratify=d.target, random_state=0)
    f = lambda a: torch.tensor(a, dtype=torch.float32)
    return f(x_o), f(x_t), torch.tensor(y_o), torch.tensor(y_t)


class Klassifikator(nn.Module):
    def __init__(self):
        super().__init__()
        self.h = nn.Linear(64, 128)
        self.chiq = nn.Linear(128, 10)

    def forward(self, x):
        return self.chiq(F.relu(self.h(x)))


def hakam_orgat(x, y, seed=0):
    torch.manual_seed(seed)
    m = Klassifikator()
    opt = torch.optim.Adam(m.parameters(), lr=1e-2, weight_decay=1e-4)
    g = torch.Generator().manual_seed(seed)
    for _ in range(60):
        for idx in torch.randperm(len(x), generator=g).split(64):
            loss = F.cross_entropy(m(x[idx]), y[idx])
            opt.zero_grad()
            loss.backward()
            opt.step()
    return m.eval()


@torch.no_grad()
def olchovlar(hakam, x, x_ref):
    """Ishonch > 0.9 ulushi, sinf entropiyasi (0..1) va hakam xususiyatlarida Frechet masofa."""
    p = torch.softmax(hakam(x), -1)
    soni = np.bincount(p.argmax(1).numpy(), minlength=10)
    u = soni[soni > 0] / soni.sum()
    entr = float(-(u * np.log(u)).sum() / np.log(10))
    f1, f2 = F.relu(hakam.h(x)).numpy().astype(np.float64), F.relu(hakam.h(x_ref)).numpy().astype(np.float64)
    m1, m2 = f1.mean(0), f2.mean(0)
    c1, c2 = np.cov(f1, rowvar=False), np.cov(f2, rowvar=False)
    w, v = np.linalg.eigh(c1)
    s1 = (v * np.sqrt(np.clip(w, 0, None))) @ v.T                  # sqrt(C1)
    iz = np.sqrt(np.clip(np.linalg.eigvalsh(s1 @ c2 @ s1), 0, None)).sum()   # tr sqrt(C1 C2)
    fd = float(((m1 - m2) ** 2).sum() + np.trace(c1) + np.trace(c2) - 2 * iz)
    return (p.max(1).values > 0.9).float().mean().item(), entr, fd, soni


def gan_orgat(x_o, hakam, x_t, tanh=True, seed=0, qadamlar=3000, B=64, har=500):
    """Non-saturating GAN; tanh=True: rasmlar [-1, 1] ga o'tkaziladi, G chiqishi tanh."""
    torch.manual_seed(seed)
    G = nn.Sequential(nn.Linear(Z, 128), nn.LeakyReLU(0.2), nn.Linear(128, 128), nn.LeakyReLU(0.2),
                      nn.Linear(128, 64), nn.Tanh() if tanh else nn.Sigmoid())
    D = nn.Sequential(nn.Linear(64, 128), nn.LeakyReLU(0.2), nn.Linear(128, 128), nn.LeakyReLU(0.2),
                      nn.Linear(128, 1))
    opt_g = torch.optim.Adam(G.parameters(), lr=2e-4, betas=(0.5, 0.999), foreach=True)
    opt_d = torch.optim.Adam(D.parameters(), lr=2e-4, betas=(0.5, 0.999), foreach=True)
    g = torch.Generator().manual_seed(seed)
    xo = x_o * 2 - 1 if tanh else x_o
    rasm = (lambda t: (t + 1) / 2) if tanh else (lambda t: t)       # [0, 1] ga qaytarish
    z_bah = torch.randn(1000, Z, generator=torch.Generator().manual_seed(9))
    tarix, qator = [], []
    for q in range(1, qadamlar + 1):
        x = xo[torch.randint(0, len(xo), (B,), generator=g)]
        soxta = G(torch.randn(B, Z, generator=g))
        l_h, l_s = D(x), D(soxta.detach())
        d_loss = (F.binary_cross_entropy_with_logits(l_h, torch.ones(B, 1)) +
                  F.binary_cross_entropy_with_logits(l_s, torch.zeros(B, 1)))
        opt_d.zero_grad()
        d_loss.backward()
        opt_d.step()
        g_loss = F.binary_cross_entropy_with_logits(D(soxta), torch.ones(B, 1))   # non-saturating
        opt_g.zero_grad()
        g_loss.backward()
        opt_g.step()
        tarix.append((d_loss.item(), g_loss.item(), torch.sigmoid(l_h).mean().item(),
                      torch.sigmoid(l_s).mean().item()))
        if har and q % har == 0:
            with torch.no_grad():
                xs = rasm(G(z_bah))
            ish, entr, fd, _ = olchovlar(hakam, xs, x_t)
            qator.append((q, *np.mean(tarix[-100:], 0), ish, entr, fd))
    with torch.no_grad():
        xs = rasm(G(z_bah))
    return xs, qator


class VAE(nn.Module):
    """26.2 dagi VAE: d = 8, Bernoulli decoder."""

    def __init__(self, d=8):
        super().__init__()
        self.enc = nn.Sequential(nn.Linear(64, 128), nn.ReLU(), nn.Linear(128, 2 * d))
        self.dec = nn.Sequential(nn.Linear(d, 128), nn.ReLU(), nn.Linear(128, 64))


def vae_orgat(x, seed=0, davr=40, B=128):
    torch.manual_seed(seed)
    m = VAE()
    opt = torch.optim.Adam(m.parameters(), lr=3e-3)
    g = torch.Generator().manual_seed(seed)
    for _ in range(davr):
        for idx in torch.randperm(len(x), generator=g).split(B):
            mu, logvar = m.enc(x[idx]).chunk(2, dim=1)
            z = mu + torch.exp(0.5 * logvar) * torch.randn(mu.shape, generator=g)
            qayta = F.binary_cross_entropy_with_logits(m.dec(z), x[idx], reduction="none").sum(1)
            kl = 0.5 * (mu ** 2 + logvar.exp() - 1 - logvar).sum(1)
            loss = (qayta + kl).mean()
            opt.zero_grad()
            loss.backward()
            opt.step()
    with torch.no_grad():
        return torch.sigmoid(m.dec(torch.randn(1000, 8, generator=torch.Generator().manual_seed(9))))


def main() -> None:
    torch.set_num_threads(1)
    x_o, x_t, y_o, y_t = malumot()
    hakam = hakam_orgat(x_o, y_o)
    d_test = torch.cdist(x_t, x_o).min(1).values
    chegara = torch.quantile(d_test, 0.05)

    def hammasi(xs):
        ish, entr, fd, soni = olchovlar(hakam, xs, x_t)
        with torch.no_grad():
            ys = hakam(xs).argmax(1)
        xilma = float(np.mean([torch.pdist(xs[ys == k]).mean().item() for k in range(10) if (ys == k).sum() > 1]))
        nusxa = (torch.cdist(xs, x_o).min(1).values < chegara).float().mean().item()
        keskinlik = (torch.minimum(xs, 1 - xs) < 0.1).float().mean().item()   # 0 yoki 1 ga yaqin piksel
        return [ish, entr, fd, soni.min() / soni.sum(), xilma, nusxa, keskinlik]

    nomlar = ["ishonch>0.9", "sinf entropiya", "Frechet", "eng kam sinf", "sinf ichi xilma",
              "nusxa ulushi", "keskin piksel"]
    print("=== 1. Ma'lumotnoma: haqiqiy o'quv rasmlari (450 ta) ===")
    ref = hammasi(x_o[:450])
    ref[5] = float("nan")                           # o'quv rasmi o'zi bilan - nusxa ma'nosiz
    print("  " + ", ".join(f"{n} {v:.3f}" for n, v in zip(nomlar, ref) if not np.isnan(v)))

    print("\n=== 2. GAN (1500 qadam) va VAE (40 davr), 3 urug' ===")
    natija = {"GAN": [], "VAE": []}
    for s in range(3):
        xs, _ = gan_orgat(x_o, hakam, x_t, seed=s, qadamlar=1500, har=0)
        natija["GAN"].append(hammasi(xs))
        xv = vae_orgat(x_o, seed=s)
        natija["VAE"].append(hammasi(xv))
        if s == 0:
            namuna = {"GAN": xs[:6].numpy(), "VAE": xv[:6].numpy()}
    print(f"  {'olchov':<16} {'GAN':>8} {'VAE':>8} {'haqiqiy':>8}   GAN - VAE (SE)")
    for i, n in enumerate(nomlar):
        a, b = np.array([r[i] for r in natija["GAN"]]), np.array([r[i] for r in natija["VAE"]])
        f = a - b
        se = f.std(ddof=1) / np.sqrt(3)
        belgi = "sezilarli" if abs(f.mean()) > 2 * se else "sezilarli emas"
        haq = "" if np.isnan(ref[i]) else f"{ref[i]:.3f}"
        print(f"  {n:<16} {a.mean():>8.3f} {b.mean():>8.3f} {haq:>8}   {f.mean():+.3f} ({se:.3f}) {belgi}")

    print("\n=== 3. Namunalar (urug' 0) ===")
    for nom in ("GAN", "VAE"):
        with torch.no_grad():
            p = hakam(torch.tensor(namuna[nom])).argmax(1).tolist()
        for s in ascii_qator(namuna[nom], [f"{nom}:{a}" for a in p]):
            print("  " + s)

    print("\n=== 4. Qaror (natijadan) ===")
    g_fd = np.mean([r[2] for r in natija["GAN"]])
    v_fd = np.mean([r[2] for r in natija["VAE"]])
    yaxshi = "GAN" if g_fd < v_fd else "VAE"
    print(f"  Frechet bo'yicha yaxshiroq: {yaxshi} ({min(g_fd, v_fd):.1f} va {max(g_fd, v_fd):.1f})")
    fd = np.array([r[2] for r in natija["GAN"]])
    print(f"  GAN Frechet urug'lar bo'yicha: {' '.join(f'{v:.1f}' for v in fd)} - beqarorlik shu yerda ko'rinadi")


if __name__ == "__main__":
    main()

=== 1. Ma'lumotnoma: haqiqiy o'quv rasmlari (450 ta) ===
  ishonch>0.9 1.000, sinf entropiya 0.998, Frechet 0.712, eng kam sinf 0.087, sinf ichi xilma 2.256, keskin piksel 0.656

=== 2. GAN (1500 qadam) va VAE (40 davr), 3 urug' ===
  olchov                GAN      VAE  haqiqiy   GAN - VAE (SE)
  ishonch>0.9         0.632    0.568    1.000   +0.063 0.022-bob sezilarli
  sinf entropiya      0.861    0.995    0.998   -0.135 0.047-bob sezilarli
  Frechet            17.877    6.274    0.712   +11.603 2.605-bob sezilarli
  eng kam sinf        0.009    0.076    0.087   -0.067 0.005-bob sezilarli
  sinf ichi xilma     1.715    1.192    2.256   +0.524 0.106-bob sezilarli
  nusxa ulushi        0.000    0.006            -0.006 0.001-bob sezilarli
  keskin piksel       0.753    0.452    0.656   +0.301 0.007-bob sezilarli

=== 3. Namunalar (urug' 0) ===
  GAN:7     GAN:7     GAN:3     GAN:5     GAN:3     GAN:3
    ..@=      +*@@*     .@@       *-@@-     .@@       .@@
    @@*@      @@.@=    .@+@.      -@:+.    .@#@%     :@:%.
    @. *      -   -     @         +   .     @#@@     .@ :.
    ..@      .  .#+     .+*+     . *-..      %@       :==#
   .%#%@     .@++@:      @@@     .@@@@       %@@       @%@:
   :@@@      = :+          %.    : @@       .-.:.        %*
    @%@.      :.@       =-%*      . @       @%*@      --#*.
    *@.       *@        @@*-      %@        @#@%      @@%%
  VAE:4     VAE:7     VAE:0     VAE:6     VAE:4     VAE:6
     :#:       -#%#.    .#*        #*        =#-      :%*.
    .%#:      -%=*%.    *%#=      -@=       :%*-      #@-:
    *#:=-     #- ##    .%+.+      %%.       ##--.    .@- .
   :@-=%-    .#-+@=    :%- +:    .@*-.     :%*+-     -@. .
   =@+#@:    .++#@.    :%:.#+    :%%%=     :#*%+.    -@=+=.
   -#*%#       .%+     .@. *+    .#%#+.    .++%*.    .@+=#-
    .-@-       -%       #*-#=     =##=.     :=%+      ##*#:
     -#.       +-       .*%*       #%=       +%=      :%%=

=== 4. Qaror (natijadan) ===
  Frechet bo'yicha yaxshiroq: VAE (6.3 va 17.9)
  GAN Frechet urug'lar bo'yicha: 16.6 14.1 23.0 - beqarorlik shu yerda ko'rinadi

d_loss = bce(D(x), 1) + bce(D(G(z)), 0); d_loss.backward()   # G ga ham grad  # ⚠️
soxta = G(z)
d_loss = bce(D(x), 1) + bce(D(soxta.detach()), 0)                          # ✅

g_loss = torch.log(1 - torch.sigmoid(D(soxta))).mean()         # to'yinadi, nan  # ⚠️
g_loss = F.binary_cross_entropy_with_logits(D(soxta), torch.ones(n, 1))     # ✅

opt = torch.optim.Adam(list(G.parameters()) + list(D.parameters()))        # ⚠️
opt_g = torch.optim.Adam(G.parameters(), lr=2e-4, betas=(0.5, 0.999))
opt_d = torch.optim.Adam(D.parameters(), lr=2e-4, betas=(0.5, 0.999))      # ✅

x = rasm / 16; G = nn.Sequential(..., nn.Tanh())             # [0, 1] va tanh   # ⚠️
x = rasm / 16 * 2 - 1; G = nn.Sequential(..., nn.Tanh())                   # ✅

if g_loss.item() < eng_yaxshi: saqla(G)                                    # ⚠️
if q % 500 == 0: fd = frechet(hakam, G(z_bah), x_test); if fd < eng: saqla(G)   # ✅

xs = G(torch.randn(1000, 16))                                              # ⚠️
xs = G(torch.randn(1000, 16, generator=torch.Generator().manual_seed(9)))  # ✅

print("ishonch:", (hakam(xs).softmax(-1).max(1).values > 0.9).float().mean())   # ⚠️
print("sinflar:", torch.bincount(hakam(xs).argmax(1), minlength=10))            # ✅

1.  # GAN ning qiymat funksiyasi V(D, G)?
2.  # belgilangan G uchun optimal D*(x)?
3.  # p_g = p_data bo'lsa D* va V qanday?
4.  # D(G(z)) = 0.01 da minimax va non-saturating logit gradienti?
5.  # nega D qadamida soxta.detach() kerak?
6.  # muvozanatda (D = 0.5) D loss va G loss qiymatlari?
7.  # 8 ta ajralgan moda: nega GAN namunalarining bir qismi modalar orasida?
8.  # D loss 0.05 ga tushdi, D(G(z)) = 0.02 - bu nimani bildiradi?
9.  # tanh chiqishli G uchun rasmlar qaysi oraliqda bo'lishi kerak?
10. # Frechet masofa nimani solishtiradi?
11. # mode collapse ni qaysi o'lchov bilan ko'rasiz?
12. # nega log(1 - sigmoid(logit)) ni to'g'ridan-to'g'ri hisoblash xavfli?

1.  opt = torch.optim.Adam(list(G.parameters()) + list(D.parameters()), lr=2e-4)
    loss = bce(D(x), 1) + bce(D(G(z)), 0) + bce(D(G(z)), 1)
    opt.zero_grad(); loss.backward(); opt.step()

2.  g_loss = torch.log(1 - torch.sigmoid(D(G(z)))).mean()

3.  x = torch.tensor(digits.data / 16)
    G = nn.Sequential(nn.Linear(16, 128), nn.ReLU(), nn.Linear(128, 64), nn.Tanh())

4.  if g_loss.item() < 0.7:
        print("GAN tayyor!")

5.  sifat = (hakam(G(z)).softmax(-1).max(1).values > 0.9).float().mean()
    print("mode collapse yo'q" if sifat > 0.6 else "")