26.4-dars: GAN muammolari va yechimlari

MA'LUMOT KO'P MODALI:
  p_data - bir nechta "tepalik" (moda) aralashmasi
  masalan: 8 ta raqam uslubi, 12 ta mahsulot turi, 8 ta Gauss halqada

MODE COLLAPSE:
  G bir nechta modani yasaydi, qolganlari YO'Q
  to'liq collapse: barcha z -> deyarli bitta nuqta
  qisman collapse: 8 tadan 5 tasi
  "moda sakrashi" (mode hopping): G bir modadan boshqasiga ko'chadi,
    D uni quvadi, G yana ko'chadi - hech qachon hammasini qamramaydi

NEGA YUZ BERADI:
  G ning maqsadi: E_z[ -log D(G(z)) ] - D ni ALDASH, qamrov emas
  D ayni paytda eng "ishonchsiz" bo'lgan joy - G uchun eng foydali
  G hamma z ni o'sha joyga yuboradi -> D o'rganadi -> G boshqa joyga
  loss da "barcha modalarni qamra" degan had YO'Q

O'LCHASH (2D, modalar ma'lum - bu darsning vazifasi):
  8 ta Gauss, radius 2, STD 0.05; modalar orasi ~1.53
  sifatli namuna:   eng yaqin modagacha masofa < 3 * STD = 0.15
  sifatli ulush:    sifatli namunalar / barcha namunalar (2000)
  qamrab olingan moda: unga >= 25 ta sifatli namuna tushgan (kutilgan 250 ning 10%)
  muvozanat:        sifatli namunalarning modalar bo'yicha entropiyasi / log 8
                    (1 = teng taqsimlangan, 0 = bitta moda)
  median masofa:    namunalar modaga qanchalik yaqin (kichik - yaxshi)

REAL MA'LUMOTDA (modalar noma'lum):
  sinflar bo'yicha taqsimot - o'zimiz o'rgatgan klassifikator bilan (4-misol)
  eng yaqin o'quv namunasigacha masofa, Frechet masofa - 26.7-dars

GAN - MINIMIZATSIYA EMAS, O'YIN:
  min_G max_D V(D, G)
  gradient tushishi egar nuqta atrofida AYLANISHI mumkin (yaqinlashmasdan)
  oddiy misol: min_x max_y x*y - gradient qadamlari aylana bo'ylab
  uzoqlashadi

ADAM beta1 (IMPULS):
  beta1 = 0.9: gradientlarning uzoq "xotirasi"
    raqib allaqachon o'zgargan, lekin eski yo'nalish davom etadi
    -> tebranish kuchayadi
  beta1 = 0.5: DCGAN dan beri GAN uchun odatiy tanlov
  beta1 = 0.0: WGAN-GP maqolasida (impulssiz)

1-MISOL NATIJASI (4 urug', 1000 qadam, 500-1000 qadam o'rtachasi):
  lr     beta1  modalar  sifatli  bitta modaga yig'ilish
  0.002  0.5     6.00    0.304    0 / 4
  0.002  0.9     0.54    0.028    4 / 4
  0.006  0.5     6.29    0.424    0 / 4

MODA SAKRASHI (lr 0.002, beta1 0.5, urug' 1, har 100 qadam):
  0.2.45.. 012.45.7 012.45.7 012345.. .1..456. 01...567
  - qamrab olingan modalar to'plami deyarli har nazoratda o'zgaradi

ASL MINIMAKS (Goodfellow, 2014):
  D:  max  E[log D(x)] + E[log(1 - D(G(z)))]
  G:  min  E[log(1 - D(G(z)))]            <- "to'yinadigan" (saturating)

MUAMMO:
  D = sigmoid(l),  l - logit
  d/dl log(1 - sigmoid(l)) = -sigmoid(l) = -D(G(z))
  D ishonchli bo'lsa (D(G(z)) -> 0): gradient -> 0
  aynan G eng yomon bo'lganda, u eng kam signal oladi!

TO'YINMAYDIGAN (non-saturating) LOSS:
  G:  min  E[-log D(G(z))]
  d/dl (-log sigmoid(l)) = -(1 - D(G(z)))  -> D(G(z)) -> 0 da -1 ga intiladi
  ikkalasi ham bir xil qo'zg'almas nuqtaga ega, lekin gradient dinamikasi boshqa
  amalda: F.softplus(-D(G(z))).mean()   (logit bilan, barqaror)

2-MISOL (G muzlatilgan, D 300 qadam o'rgatiladi):
  D qadami  D(G(z))  ||grad|| log(1-D)  ||grad|| -log D
       0    0.5041       0.0764            0.0758
      30    0.1986       0.1541            0.6338
     300    0.0008       0.0042            2.6635
  to'yinadigan: 18 barobar kamaydi;  to'yinmaydigan: o'sdi

AMALIY XULOSA:
  hech qachon asl log(1 - D) ni ishlatmang - doim -log D
  D ni G dan ko'p marta ko'p o'rgatish (n_D >> 1) oddiy GAN da xavfli

JS-DIVERGENSIYA MUAMMOSI:
  oddiy GAN (optimal D da) Jensen-Shannon divergensiyani kamaytiradi
  p_data va p_G ustma-ust tushmasa (past o'lchamli "tolalar"),
  JS = log 2 - o'zgarmas: G qayerda bo'lishidan qat'i nazar bir xil
  -> ma'noli gradient yo'q

WASSERSTEIN (EARTH MOVER) MASOFASI:
  W(p, q) = "p tuprog'ini q shakliga keltirish uchun minimal ish"
  taqsimotlar ustma-ust tushmasa ham masofa bilan silliq o'zgaradi
  Kantorovich-Rubinshteyn ikkiligi:
    W(p, q) = max_{||f||_L <= 1}  E_p[f(x)] - E_q[f(x)]
  f - 1-Lipschitz funksiya ("kritik", endi ehtimol emas - ixtiyoriy son)

WGAN (2017):
  kritik:  max  E[D(x)] - E[D(G(z))]
  G:       min  -E[D(G(z))]
  sigmoid yo'q -> to'yinish yo'q
  Lipschitz sharti: dastlab vaznlarni qirqish (clipping) - qo'pol

WGAN-GP (2017) - GRADIENT PENALTY:
  x_hat = a * x + (1 - a) * G(z),  a ~ U(0, 1)
  jarima = lam * E[ (||grad_x D(x_hat)|| - 1)^2 ],   lam = 10
  kritik loss = E[D(G(z))] - E[D(x)] + jarima
  torch da: torch.autograd.grad(..., create_graph=True) - ikkinchi tartibli
  odatda n_kritik = 5 (har G qadamiga), Adam(betas=(0.5 yoki 0, 0.9))

NARX:
  bir G qadami = 5 ta kritik qadami, har biri ikki marta backward
  2-misolda: WGAN-GP ning 800 G qadami = 4000 kritik yangilanishi

G'OYA:
  Lipschitz konstantasi <= har qatlam Lipschitz konstantalarining ko'paytmasi
  Linear qatlam uchun: sigma_max(W) - eng katta singular qiymat
  ReLU, LeakyReLU - 1-Lipschitz
  => W ni W / sigma_max(W) ga almashtirsak, butun D <= 1-Lipschitz

SIGMA_MAX NI ARZON BAHOLASH - POWER ITERATION:
  u - saqlanadigan vektor
  v = normalize(W^T u);  u = normalize(W v);  sigma = u^T W v
  har forward da BITTA qadam - u o'rgatish davomida "isib" boradi

TORCH:
  from torch.nn.utils.parametrizations import spectral_norm
  layer = spectral_norm(nn.Linear(64, 64))
  layer.weight - normallangan, layer.parametrizations.weight.original - asl

3-MISOL: asl W sigma_max = 1.087, spectral_norm dan keyin 1.001
  bizning parallel (4 urug'li) qatlamimiz ham 1.000-1.006

GAN DA:
  odatda faqat D ga qo'yiladi (SN-GAN, 2018)
  WGAN-GP dan arzon: qo'shimcha backward yo'q, faqat matritsa-vektor ko'paytmasi
  lekin D ni "sekinlashtiradi" - kichik byudjetda o'rganish sekinroq bo'lishi mumkin

BIR TOMONLAMA LABEL SMOOTHING (Salimans va b., 2016):
  haqiqiy namunalar nishoni 1.0 o'rniga 0.9
  soxta namunalar nishoni 0 QOLADI (ikki tomonlama silliqlash zararli)
  D haddan tashqari ishonchli bo'lmaydi -> G ga gradient yumshoqroq
  torch: F.binary_cross_entropy_with_logits(D(x), torch.full_like(..., 0.9))

MINIBATCH DISKRIMINATSIYA:
  muammo: D har namunani ALOHIDA baholaydi - "hammasi bir xil" ekanini ko'rmaydi
  g'oya: D ga batch darajasidagi xususiyat beriladi
  asl usul (2016): namunalar orasidagi masofalar tenzori - murakkab
  soddalashtirilgani - MINIBATCH STD (ProGAN, 2018):
    h - D ning oxirgi yashirin qatlami (B, H)
    s = h.std(dim=batch).mean()  - bitta son
    h' = [h, s] - har namunaga qo'shiladi
  collapse bo'lsa: soxta batchda s kichik, haqiqiyda katta -> D ajratadi
  -> G xilma-xillikka "majburlanadi"
  TUZOQ: haqiqiy va soxta namunani BITTA batchda D ga bermang - s aralashadi

TAQQOSLASH PROTOKOLI (2- va 3-misol):
  bir xil vazifa (8 ta Gauss), bir xil arxitektura (2 x 64)
  bir xil urug'lar: boshlang'ich vaznlar, z shovqini, haqiqiy batchlar
  4 urug' BITTA tensorda parallel: vazn (S, a, b), torch.baddbmm
    - har urug' mustaqil, lekin Python yuklamasi bir marta
  yakuniy o'lchov - oxirgi 3 nazorat o'rtachasi (bitta nuqta shovqinli)
  farq = usul - oddiy (har urug'da), SE = std(farq) / sqrt(4)
  |farq| > 2 * SE -> sezilarli

3-MISOL NATIJASI (900 qadam, 700-900 o'rtachasi, farq usul - oddiy, (SE)):
  usul              sifatli farq    median masofa   muvozanat
  label smoothing   +0.106 0.079-bob  -0.070 0.038-bob  +0.011 0.024-bob
  spektral          -0.073 0.079-bob  +0.082 0.063-bob  +0.070 0.041-bob
  minibatch std     +0.104 0.052-bob* -0.049 0.034-bob  +0.020 0.044-bob
  * - sezilarli; modalar soni bo'yicha hech biri sezilarli farq qilmadi

2-MISOL: WGAN-GP (800 G qadami) - 2 urug' halqaga yetib bormadi;
  median masofa sezilarli YOMONROQ (+0.30, SE 0.15)

HALOL XULOSA:
  minibatch std - sifatli ulushni sezilarli oshirgan yagona usul
  label smoothing - sifat va masofa yaxshi tomonga, lekin 2*SE ga yetmadi
  spektral - muvozanat tengroq, sifat va masofa yomon tomonga (sezilarsiz)
  WGAN-GP - ma'noli gradient, lekin shu byudjetda eng sekin va eng qimmat
  "davo" ning foydasi byudjet, vazifa va sozlamaga bog'liq

G'OYA (Mirza va Osindero, 2014):
  G(z, y) - y sinf (shart) bo'yicha namuna
  D(x, y) - "x haqiqiymi VA y sinfiga mosmi?"
  eng sodda usul: y ni one-hot qilib kirishga ulash (concat)
    G: [z (16), onehot(y) (10)] -> 64 piksel
    D: [x (64), onehot(y) (10)] -> logit

NEGA ISHLAYDI:
  D soxta "3" ni haqiqiy "3" bilan solishtiradi - sinfga mos kelmasa, jazolaydi
  G har y uchun alohida taqsimotni o'rganishga majbur
  bonus: mode collapse sinflar darajasida YO'Q - har sinfni o'zimiz so'raymiz

BAHOLASH (4-misol):
  o'zimiz o'rgatgan klassifikator (test aniqligi 0.972)
  to'g'ri sinf ulushi: klassifikator(G(z, y)) == y
  sinf bo'yicha ulush - qaysi sinf qiyin
  shartsiz GAN bilan: klassifikator bo'yicha sinflar taqsimoti, entropiya
  yodlab olish: eng yaqin o'quv namunasigacha masofa (test bilan solishtirib)

4-MISOL NATIJASI (3 urug', 1000 qadam):
  to'g'ri sinf ulushi 0.892 - 0.969; eng qiyin sinflar: 3 0.80-bob, 8 0.88-bob, 1 0.90-bob
  shartsiz: 9-10 ta sinf >= 5%, lekin bitta urug'da 5-sinf, boshqasida 9-sinf
  ulushi 0.03 (kutilgan ~0.10), eng katta ulush 0.18

ZAMONAVIY KO'RINISHLAR:
  projection discriminator, class-conditional BatchNorm (BigGAN)
  matn sharti - 26.8-dars (matn-rasm modellari)

import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.nn.utils.parametrizations import spectral_norm

# oddiy GAN (logit bilan, barqaror)
ld = F.softplus(-D(x)).mean() + F.softplus(D(fake.detach())).mean()
lg = F.softplus(-D(fake)).mean()                 # to'yinmaydigan: -log D(G(z))
# lg = -F.softplus(D(fake)).mean()               # to'yinadigan log(1 - D) - ISHLATMANG

opt = torch.optim.Adam(G.parameters(), lr=2e-3, betas=(0.5, 0.999))  # beta1 = 0.5

# bir tomonlama label smoothing
ld = F.binary_cross_entropy_with_logits(D(x), torch.full_like(D(x), 0.9)) \
    + F.softplus(D(fake.detach())).mean()

# WGAN-GP kritik
a = torch.rand(len(x), 1)
xh = (a * x + (1 - a) * fake.detach()).requires_grad_(True)
gr = torch.autograd.grad(D(xh).sum(), xh, create_graph=True)[0]
gp = ((gr.norm(dim=1) - 1) ** 2).mean()
ld = D(fake.detach()).mean() - D(x).mean() + 10 * gp
lg = -D(fake).mean()

# spektral normalizatsiya
layer = spectral_norm(nn.Linear(64, 64))

# minibatch std (D ichida, oxirgi yashirin qatlamdan keyin)
s = h.std(0).mean().expand(len(h), 1)
h = torch.cat([h, s], 1)

# cGAN: shartni ikkala tarmoqqa ulash
fake = G(torch.cat([z, F.one_hot(y, 10).float()], 1))
logit = D(torch.cat([x, F.one_hot(y, 10).float()], 1))

mode collapse - qamrovni o'lchang (modalar / sinflar taqsimoti)
beqarorlik - Adam beta1 = 0.5, lr ni sinab ko'ring, bir necha urug'
gradient so'nishi - doim -log D (to'yinmaydigan), log(1 - D) emas
WGAN-GP - to'yinmaydigan kritik, lekin 5x qimmat va sekin
spektral norm - arzon Lipschitz, D ni sekinlashtirishi mumkin
label smoothing, minibatch std - arzon, foydasi vazifaga bog'liq
yechimlarni bir xil urug'larda, juftlashgan farq + SE bilan taqqoslang
cGAN - shart G ga ham, D ga ham; to'g'ri sinf ulushi bilan baholang

"""Mode collapse: 8 ta Gauss halqasida qamrab olingan modalar soni vaqt bo'yicha, lr va beta1 ta'siri."""

import math

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F

MARKAZ = torch.tensor([[2 * math.cos(k * math.pi / 4), 2 * math.sin(k * math.pi / 4)]
                       for k in range(8)])
STD = 0.05          # har moda juda tor: 3 * STD = 0.15, modalar orasi ~1.53
S = 4               # 4 ta urug' bitta tensorda parallel o'rgatiladi
B, H, Z = 128, 64, 2


class BLinear(nn.Module):
    """S ta mustaqil Linear qatlam: x (S, B, a) -> (S, B, b). Har urug' o'z og'irligida."""

    def __init__(self, a, b):
        super().__init__()
        k = 1 / math.sqrt(a)
        self.w = nn.Parameter(torch.empty(S, a, b).uniform_(-k, k))
        self.b = nn.Parameter(torch.empty(S, 1, b).uniform_(-k, k))

    def forward(self, x):
        return torch.baddbmm(self.b, x, self.w)


def tarmoq(a, b):
    return nn.Sequential(BLinear(a, H), nn.ReLU(), BLinear(H, H), nn.ReLU(), BLinear(H, b))


def haqiqiy(n, g):
    k = torch.randint(0, 8, (S, n), generator=g)
    return MARKAZ[k] + STD * torch.randn(S, n, 2, generator=g)


def olcha(x):
    """x (S, N, 2) -> har urug' uchun: modalar soni, sifatli ulush, qaysi modalar."""
    d = torch.cdist(x, MARKAZ.expand(S, 8, 2))            # (S, N, 8)
    dmin, eng = d.min(2)
    sifatli = dmin < 3 * STD
    natija = []
    for s in range(S):
        soni = torch.bincount(eng[s][sifatli[s]], minlength=8)
        bor = soni >= 25                                  # kutilgan 250 ning 10%
        natija.append((int(bor.sum()), sifatli[s].float().mean().item(),
                       "".join(str(k) if bor[k] else "." for k in range(8))))
    return natija


def orgat(lr, beta1, qadamlar=1000, har=100):
    torch.manual_seed(0)
    G, D = tarmoq(Z, 2), tarmoq(2, 1)
    oG = torch.optim.Adam(G.parameters(), lr=lr, betas=(beta1, 0.999))
    oD = torch.optim.Adam(D.parameters(), lr=lr, betas=(beta1, 0.999))
    g = torch.Generator().manual_seed(1)
    z_baho = torch.randn(S, 2000, Z, generator=torch.Generator().manual_seed(99))
    tarix = []
    for q in range(1, qadamlar + 1):
        x = haqiqiy(B, g)
        f = G(torch.randn(S, B, Z, generator=g))
        # D: haqiqiy -> 1, soxta -> 0 (logit bilan BCE = softplus)
        ld = F.softplus(-D(x)).mean() + F.softplus(D(f.detach())).mean()
        oD.zero_grad()
        ld.backward()
        oD.step()
        # G: to'yinmaydigan (non-saturating) loss: -log D(G(z))
        lg = F.softplus(-D(f)).mean()
        oG.zero_grad()
        lg.backward()
        oG.step()
        if q % har == 0:
            with torch.no_grad():
                tarix.append(olcha(G(z_baho)))
    return tarix


def main() -> None:
    torch.set_num_threads(1)
    print("=== 1. Vazifa: 8 ta Gauss halqasi ===")
    x = haqiqiy(2000, torch.Generator().manual_seed(5))
    m, s, q = olcha(x)[0]
    print(f"  modalar radiusi 2, STD {STD}; sifatli namuna: eng yaqin modagacha < {3 * STD:.2f}")
    print(f"  moda 'qamrab olingan': unga >= 25 ta sifatli namuna tushsa (2000 tadan)")
    print(f"  haqiqiy ma'lumot: modalar {m}/8, sifatli {s:.3f}, {q}")
    print(f"  G va D: 2 yashirin qatlam x {H}; {S} urug' parallel (bitta tensorda)")

    print("\n=== 2. Oddiy GAN (lr 0.002, beta1 0.5): modalar soni vaqt bo'yicha ===")
    sozlamalar = [(0.002, 0.5), (0.002, 0.9), (0.006, 0.5)]
    natija = {k: orgat(*k) for k in sozlamalar}
    t = natija[(0.002, 0.5)]
    print("  qadam  " + "  ".join(f"urug' {s}" for s in range(S)))
    for i in range(1, len(t), 2):
        print(f"  {(i + 1) * 100:>5}  " + "  ".join(f"{t[i][s][0]:>7}" for s in range(S)))
    print("  qaysi modalar (urug' 1, har 100 qadam):")
    print("   " + " ".join(t[i][1][2] for i in range(3, 10)))

    print("\n=== 3. lr va beta1 ta'siri (oxirgi 6 nazorat nuqtasi, 500-1000 qadam) ===")
    print("  lr     beta1  modalar  sifatli  modalar sakrashi  bitta modaga yig'ilish")
    xulosa = {}
    for k in sozlamalar:
        t = natija[k][-6:]
        mod = np.mean([[r[s][0] for s in range(S)] for r in t])
        sif = np.mean([[r[s][1] for s in range(S)] for r in t])
        # sakrash: ketma-ket nazoratlarda qamrab olingan modalar to'plami o'zgargan holatlar
        sakrash = np.mean([sum(t[i][s][2] != t[i - 1][s][2] for i in range(1, 6)) for s in range(S)])
        yakka = sum(any(r[s][0] <= 1 for r in t) for s in range(S))
        xulosa[k] = (mod, sif, yakka)
        print(f"  {k[0]:<6} {k[1]:<5} {mod:>7.2f} {sif:>8.3f} {sakrash:>12.2f} / 5 {yakka:>13} / {S}")
    print("  lr 0.002, beta1 0.9 (har 100 qadam, 500-1000), urug' 0 va 1:")
    for s in (0, 1):
        print("   " + " ".join(r[s][2] for r in natija[(0.002, 0.9)][-6:]))

    print("\n=== 4. Xulosa (natijadan) ===")
    eng = max(sozlamalar, key=lambda k: (xulosa[k][0], xulosa[k][1]))
    print(f"  eng yaxshi sozlama: lr {eng[0]}, beta1 {eng[1]} - "
          f"modalar {xulosa[eng][0]:.2f}, sifatli {xulosa[eng][1]:.3f}")
    for k in sozlamalar:
        if xulosa[k][2] > 0:
            print(f"  lr {k[0]}, beta1 {k[1]}: {xulosa[k][2]} urug'da G bir yoki nol modaga yig'ildi")
    print("  mode collapse - bir martalik hodisa emas: modalar paydo bo'ladi va yo'qoladi")


if __name__ == "__main__":
    main()

=== 1. Vazifa: 8 ta Gauss halqasi ===
  modalar radiusi 2, STD 0.05; sifatli namuna: eng yaqin modagacha < 0.15
  moda 'qamrab olingan': unga >= 25 ta sifatli namuna tushsa (2000 tadan)
  haqiqiy ma'lumot: modalar 8/8, sifatli 0.989, 01234567
  G va D: 2 yashirin qatlam x 64; 4 urug' parallel (bitta tensorda)

=== 2. Oddiy GAN (lr 0.002, beta1 0.5): modalar soni vaqt bo'yicha ===
  qadam  urug' 0  urug' 1  urug' 2  urug' 3
    200        0        0        0        0
    400        3        0        5        5
    600        4        6        6        7
    800        6        6        7        8
   1000        7        5        7        5
  qaysi modalar (urug' 1, har 100 qadam):
   ........ 0.2.45.. 012.45.7 012.45.7 012345.. .1..456. 01...567

=== 3. lr va beta1 ta'siri (oxirgi 6 nazorat nuqtasi, 500-1000 qadam) ===
  lr     beta1  modalar  sifatli  modalar sakrashi  bitta modaga yig'ilish
  0.002  0.5      6.00    0.304         4.50 / 5             0 / 4
  0.002  0.9      0.54    0.028         3.25 / 5             4 / 4
  0.006  0.5      6.29    0.424         4.50 / 5             0 / 4
  lr 0.002, beta1 0.9 (har 100 qadam, 500-1000), urug' 0 va 1:
   ........ ..2....7 .....5.. ........ ........ 0.......
   .......7 ........ ..2..... ........ 0....... 0.......

=== 4. Xulosa (natijadan) ===
  eng yaxshi sozlama: lr 0.006, beta1 0.5 - modalar 6.29, sifatli 0.424
  lr 0.002, beta1 0.9: 4 urug'da G bir yoki nol modaga yig'ildi
  mode collapse - bir martalik hodisa emas: modalar paydo bo'ladi va yo'qoladi

"""Kuchli diskriminator va gradient so'nishi; WGAN-GP ni noldan yozib oddiy GAN bilan juftlashgan taqqoslash."""

import math

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F

MARKAZ = torch.tensor([[2 * math.cos(k * math.pi / 4), 2 * math.sin(k * math.pi / 4)]
                       for k in range(8)])
STD = 0.05
S = 4
B, H, Z = 128, 64, 2


class BLinear(nn.Module):
    """S ta mustaqil Linear qatlam: x (S, B, a) -> (S, B, b)."""

    def __init__(self, a, b):
        super().__init__()
        k = 1 / math.sqrt(a)
        self.w = nn.Parameter(torch.empty(S, a, b).uniform_(-k, k))
        self.b = nn.Parameter(torch.empty(S, 1, b).uniform_(-k, k))

    def forward(self, x):
        return torch.baddbmm(self.b, x, self.w)


def tarmoq(a, b):
    return nn.Sequential(BLinear(a, H), nn.ReLU(), BLinear(H, H), nn.ReLU(), BLinear(H, b))


def haqiqiy(n, g):
    k = torch.randint(0, 8, (S, n), generator=g)
    return MARKAZ[k] + STD * torch.randn(S, n, 2, generator=g)


def olcha(x):
    """x (S, N, 2) -> (S, 4): modalar soni, sifatli ulush, median masofa, muvozanat."""
    d = torch.cdist(x, MARKAZ.expand(S, 8, 2))
    dmin, eng = d.min(2)
    sifatli = dmin < 3 * STD
    natija = []
    for s in range(S):
        soni = torch.bincount(eng[s][sifatli[s]], minlength=8).float()
        p = soni / soni.sum().clamp_min(1)
        # muvozanat: sifatli namunalar modalar bo'yicha entropiyasi / log 8 (1 = teng)
        ent = -(p[p > 0] * p[p > 0].log()).sum().item() / math.log(8)
        natija.append([(soni >= 25).sum().item(), sifatli[s].float().mean().item(),
                       dmin[s].median().item(), ent])
    return np.array(natija)


def kritik_loss(D, x, f, g, lam=10.0):
    """WGAN-GP: E[D(soxta)] - E[D(haqiqiy)] + lam * E[(||grad D(x_hat)|| - 1)^2].

    x_hat - haqiqiy va soxta nuqtalar orasidagi tasodifiy nuqta."""
    a = torch.rand(S, x.shape[1], 1, generator=g)
    xh = (a * x + (1 - a) * f).requires_grad_(True)
    gr = torch.autograd.grad(D(xh).sum(), xh, create_graph=True)[0]
    gp = ((gr.norm(dim=2) - 1) ** 2).mean()
    return D(f).mean() - D(x).mean() + lam * gp


def g_gradient_normasi(G, loss):
    """Har urug' uchun G parametrlari gradientining normasi (S,)."""
    gr = torch.autograd.grad(loss, list(G.parameters()), retain_graph=True)
    return torch.sqrt(sum((t.reshape(S, -1) ** 2).sum(1) for t in gr))


def gradient_tajriba():
    torch.manual_seed(0)
    G = tarmoq(Z, 2)                          # G muzlatilgan (boshlang'ich holat)
    g = torch.Generator().manual_seed(1)
    natija = {}
    for tur in ("gan", "wgan"):
        torch.manual_seed(1)
        D = tarmoq(2, 1)
        opt = torch.optim.Adam(D.parameters(), lr=0.002, betas=(0.5, 0.999))
        qator = []
        for q in range(301):
            if q in (0, 10, 30, 100, 300):
                z = torch.randn(S, 1000, Z, generator=torch.Generator().manual_seed(7))
                o = D(G(z))
                if tur == "gan":
                    d_soxta = torch.sigmoid(o).mean().item()
                    sat = g_gradient_normasi(G, -F.softplus(o).mean(1).sum())  # log(1 - D) ni kamaytirish
                    ns = g_gradient_normasi(G, F.softplus(-o).mean(1).sum())   # -log D ni kamaytirish
                    qator.append((q, d_soxta, sat.mean().item(), ns.mean().item()))
                else:
                    w = g_gradient_normasi(G, -o.mean(1).sum())
                    qator.append((q, w.mean().item()))
            x = haqiqiy(B, g)
            with torch.no_grad():
                f = G(torch.randn(S, B, Z, generator=g))
            if tur == "gan":
                loss = F.softplus(-D(x)).mean() + F.softplus(D(f)).mean()
            else:
                loss = kritik_loss(D, x, f, g)
            opt.zero_grad()
            loss.backward()
            opt.step()
        natija[tur] = qator
    return natija


def orgat(tur, qadamlar=800, har=100):
    torch.manual_seed(0)
    G, D = tarmoq(Z, 2), tarmoq(2, 1)
    if tur == "gan":
        lr, betas, n_kritik = 0.002, (0.5, 0.999), 1
    else:
        lr, betas, n_kritik = 0.001, (0.5, 0.9), 5
    oG = torch.optim.Adam(G.parameters(), lr=lr, betas=betas)
    oD = torch.optim.Adam(D.parameters(), lr=lr, betas=betas)
    g = torch.Generator().manual_seed(1)
    z_baho = torch.randn(S, 2000, Z, generator=torch.Generator().manual_seed(99))
    tarix = []
    for q in range(1, qadamlar + 1):
        for _ in range(n_kritik):
            x = haqiqiy(B, g)
            with torch.no_grad():
                f = G(torch.randn(S, B, Z, generator=g))
            if tur == "gan":
                ld = F.softplus(-D(x)).mean() + F.softplus(D(f)).mean()
            else:
                ld = kritik_loss(D, x, f, g)
            oD.zero_grad()
            ld.backward()
            oD.step()
        o = D(G(torch.randn(S, B, Z, generator=g)))
        lg = -o.mean() if tur == "wgan" else F.softplus(-o).mean()
        oG.zero_grad()
        lg.backward()
        oG.step()
        if q % har == 0:
            with torch.no_grad():
                tarix.append(olcha(G(z_baho)))
    return np.array(tarix)                       # (nazorat, S, 4)


def main() -> None:
    torch.set_num_threads(1)
    print("=== 1. G muzlatilgan, D ko'p o'rgatiladi: G ga qancha gradient yetadi ===")
    t = gradient_tajriba()
    print("  D qadami  D(G(z))  ||grad|| log(1-D)  ||grad|| -log D  ||grad|| WGAN")
    for (q, d, sat, ns), (_, w) in zip(t["gan"], t["wgan"]):
        print(f"  {q:>8} {d:>8.4f} {sat:>17.5f} {ns:>15.4f} {w:>13.4f}")
    sat0, sat1 = t["gan"][0][2], t["gan"][-1][2]
    print(f"  log(1-D) gradienti {sat0 / sat1:.0f} barobar kamaydi" if sat1 < sat0 / 10 else
          "  log(1-D) gradienti sezilarli kamaymadi")

    print("\n=== 2. Oddiy GAN va WGAN-GP: 800 ta G qadami, 4 urug' ===")
    natija = {tur: orgat(tur) for tur in ("gan", "wgan")}
    print("  qadam   GAN modalar       WGAN-GP modalar")
    for i in (1, 3, 5, 7):
        print(f"  {(i + 1) * 100:>5}   " + " ".join(f"{int(v):>2}" for v in natija["gan"][i, :, 0])
              + "       " + " ".join(f"{int(v):>2}" for v in natija["wgan"][i, :, 0]))
    print("  D yangilanishlari: GAN 800, WGAN-GP 4000 (har G qadamiga 5 kritik qadam)")

    print("\n=== 3. Yakuniy o'lchovlar (600-800 qadam o'rtachasi) ===")
    nomlar = ["modalar", "sifatli", "median masofa", "muvozanat"]
    yak = {tur: natija[tur][-3:].mean(0) for tur in natija}   # (S, 4)
    print("  o'lchov         GAN (urug'lar)            WGAN-GP (urug'lar)        farq    SE")
    xulosa = []
    for j, nom in enumerate(nomlar):
        a, b = yak["gan"][:, j], yak["wgan"][:, j]
        farq = b - a
        se = farq.std(ddof=1) / math.sqrt(S)
        print(f"  {nom:<14} " + " ".join(f"{v:5.2f}" for v in a) + "   "
              + " ".join(f"{v:5.2f}" for v in b) + f"  {farq.mean():+6.2f} {se:5.2f}")
        xulosa.append((nom, farq.mean(), se))

    print("\n=== 4. Xulosa (natijadan) ===")
    for nom, f, se in xulosa:
        belgi = "sezilarli" if abs(f) > 2 * se else "sezilarli emas"
        print(f"  WGAN-GP - GAN, {nom}: {f:+.2f} (SE {se:.2f}) -> {belgi}")
    tiqilgan = int((yak["wgan"][:, 0] < 4).sum())
    if tiqilgan:
        print(f"  WGAN-GP {tiqilgan} ta urug'da 800 qadamda halqaga yetib bormadi")
    print("  WGAN kritigi to'yinmaydi, lekin bir G qadami ~5 barobar qimmat")


if __name__ == "__main__":
    main()

=== 1. G muzlatilgan, D ko'p o'rgatiladi: G ga qancha gradient yetadi ===
  D qadami  D(G(z))  ||grad|| log(1-D)  ||grad|| -log D  ||grad|| WGAN
         0   0.5041           0.07637          0.0758        0.1522
        10   0.4599           0.13876          0.1634        1.3103
        30   0.1986           0.15411          0.6338        1.9316
       100   0.0096           0.02699          2.0428        0.8494
       300   0.0008           0.00422          2.6635        0.7903
  log(1-D) gradienti 18 barobar kamaydi

=== 2. Oddiy GAN va WGAN-GP: 800 ta G qadami, 4 urug' ===
  qadam   GAN modalar       WGAN-GP modalar
    200    0  0  0  0        0  0  0  0
    400    4  4  8  0        0  0  0  7
    600    7  7  5  6        0  0  6  8
    800    6  6  7  2        0  1  8  7
  D yangilanishlari: GAN 800, WGAN-GP 4000 (har G qadamiga 5 kritik qadam)

=== 3. Yakuniy o'lchovlar (600-800 qadam o'rtachasi) ===
  o'lchov         GAN (urug'lar)            WGAN-GP (urug'lar)        farq    SE
  modalar         6.00  7.00  6.67  3.67    0.33  0.33  7.00  7.33   -2.08  2.46
  sifatli         0.30  0.26  0.38  0.18    0.02  0.02  0.23  0.39   -0.11  0.11
  median masofa   0.24  0.27  0.20  0.27    0.81  0.81  0.35  0.22   +0.30  0.15
  muvozanat       0.84  0.93  0.85  0.72    0.54  0.76  0.94  0.93   -0.05  0.12

=== 4. Xulosa (natijadan) ===
  WGAN-GP - GAN, modalar: -2.08 (SE 2.46) -> sezilarli emas
  WGAN-GP - GAN, sifatli: -0.11 (SE 0.11) -> sezilarli emas
  WGAN-GP - GAN, median masofa: +0.30 (SE 0.15) -> sezilarli
  WGAN-GP - GAN, muvozanat: -0.05 (SE 0.12) -> sezilarli emas
  WGAN-GP 2 ta urug'da 800 qadamda halqaga yetib bormadi
  WGAN kritigi to'yinmaydi, lekin bir G qadami ~5 barobar qimmat

"""Yechimlar: label smoothing, spektral normalizatsiya, minibatch std - oddiy GAN bilan juftlashgan taqqoslash."""

import math

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.nn.utils.parametrizations import spectral_norm

MARKAZ = torch.tensor([[2 * math.cos(k * math.pi / 4), 2 * math.sin(k * math.pi / 4)]
                       for k in range(8)])
STD = 0.05
S = 4
B, H, Z = 128, 64, 2


class BLinear(nn.Module):
    """S ta mustaqil Linear qatlam; sn=True bo'lsa W / sigma_max(W) (power iteration)."""

    def __init__(self, a, b, sn=False):
        super().__init__()
        k = 1 / math.sqrt(a)
        self.w = nn.Parameter(torch.empty(S, a, b).uniform_(-k, k))
        self.b = nn.Parameter(torch.empty(S, 1, b).uniform_(-k, k))
        self.sn = sn
        self.register_buffer("u", F.normalize(torch.randn(S, b, 1), dim=1))

    def vazn(self):
        if not self.sn:
            return self.w
        with torch.no_grad():                          # bitta power iteration qadami
            v = F.normalize(self.w @ self.u, dim=1)    # (S, a, 1)
            self.u = F.normalize(self.w.transpose(1, 2) @ v, dim=1)
        sigma = (v.transpose(1, 2) @ self.w @ self.u).reshape(S, 1, 1)
        return self.w / sigma

    def forward(self, x):
        return torch.baddbmm(self.b, x, self.vazn())


class Disk(nn.Module):
    def __init__(self, sn=False, mb=False):
        super().__init__()
        self.mb = mb
        self.l1, self.l2 = BLinear(2, H, sn), BLinear(H, H, sn)
        self.l3 = BLinear(H + int(mb), 1, sn)

    def forward(self, x):
        h = F.relu(self.l2(F.relu(self.l1(x))))
        if self.mb:
            # minibatch std: batch ichidagi xilma-xillik D ga qo'shimcha belgi sifatida
            s = h.std(1, keepdim=True).mean(2, keepdim=True).expand(-1, h.shape[1], 1)
            h = torch.cat([h, s], 2)
        return self.l3(h)


def generator():
    return nn.Sequential(BLinear(Z, H), nn.ReLU(), BLinear(H, H), nn.ReLU(), BLinear(H, 2))


def haqiqiy(n, g):
    k = torch.randint(0, 8, (S, n), generator=g)
    return MARKAZ[k] + STD * torch.randn(S, n, 2, generator=g)


def olcha(x):
    d = torch.cdist(x, MARKAZ.expand(S, 8, 2))
    dmin, eng = d.min(2)
    sifatli = dmin < 3 * STD
    natija = []
    for s in range(S):
        soni = torch.bincount(eng[s][sifatli[s]], minlength=8).float()
        p = soni / soni.sum().clamp_min(1)
        ent = -(p[p > 0] * p[p > 0].log()).sum().item() / math.log(8)
        natija.append([(soni >= 25).sum().item(), sifatli[s].float().mean().item(),
                       dmin[s].median().item(), ent])
    return np.array(natija)


def orgat(usul, qadamlar=900, har=100):
    torch.manual_seed(0)
    G = generator()
    D = Disk(sn=usul == "spektral", mb=usul == "minibatch std")
    oG = torch.optim.Adam(G.parameters(), lr=0.002, betas=(0.5, 0.999))
    oD = torch.optim.Adam(D.parameters(), lr=0.002, betas=(0.5, 0.999))
    g = torch.Generator().manual_seed(1)
    z_baho = torch.randn(S, 2000, Z, generator=torch.Generator().manual_seed(99))
    nishon = 0.9 if usul == "label smoothing" else 1.0     # bir tomonlama: faqat haqiqiy
    tarix = []
    for q in range(1, qadamlar + 1):
        x = haqiqiy(B, g)
        f = G(torch.randn(S, B, Z, generator=g))
        ld = (F.binary_cross_entropy_with_logits(D(x), torch.full((S, B, 1), nishon))
              + F.softplus(D(f.detach())).mean())
        oD.zero_grad()
        ld.backward()
        oD.step()
        lg = F.softplus(-D(f)).mean()
        oG.zero_grad()
        lg.backward()
        oG.step()
        if q % har == 0:
            with torch.no_grad():
                tarix.append(olcha(G(z_baho)))
    return np.array(tarix)


def main() -> None:
    torch.set_num_threads(1)
    print("=== 1. Spektral normalizatsiya: torch API va o'zimizniki ===")
    torch.manual_seed(3)
    lin = spectral_norm(nn.Linear(64, 64))
    for _ in range(30):                     # har forward da bitta power iteration
        lin(torch.zeros(1, 64))
    w_asl = lin.parametrizations.weight.original
    print(f"  asl W: sigma_max = {torch.linalg.matrix_norm(w_asl, 2).item():.3f}")
    print(f"  spectral_norm(W): sigma_max = {torch.linalg.matrix_norm(lin.weight, 2).item():.3f}")
    bl = BLinear(64, 64, sn=True)
    for _ in range(30):
        w = bl.vazn()
    print(f"  bizning BLinear (4 urug'): sigma_max = "
          + ", ".join(f"{torch.linalg.matrix_norm(w[s].detach(), 2).item():.3f}" for s in range(S)))

    print("\n=== 2. To'rt usul, bir xil urug'lar va batchlar (900 qadam) ===")
    usullar = ["oddiy", "label smoothing", "spektral", "minibatch std"]
    natija = {u: orgat(u) for u in usullar}
    print("  usul              modalar (300, 500, 700, 900 qadam; 4 urug')")
    for u in usullar:
        print(f"  {u:<17} " + " | ".join(" ".join(str(int(v)) for v in natija[u][i, :, 0])
                                        for i in (2, 4, 6, 8)))

    print("\n=== 3. Yakuniy o'lchovlar (700-900 qadam o'rtachasi, 4 urug' o'rtachasi) ===")
    nomlar = ["modalar", "sifatli", "median masofa", "muvozanat"]
    yak = {u: natija[u][-3:].mean(0) for u in usullar}
    print("  usul              " + "".join(f"{n:>15}" for n in nomlar))
    for u in usullar:
        print(f"  {u:<17} " + "".join(f"{v:>15.3f}" for v in yak[u].mean(0)))

    print("\n=== 4. Juftlashgan farq (usul - oddiy), SE, 2*SE qoidasi ===")
    yaxshi = []
    for u in usullar[1:]:
        qator = []
        for j, nom in enumerate(nomlar):
            farq = yak[u][:, j] - yak["oddiy"][:, j]
            se = farq.std(ddof=1) / math.sqrt(S)
            belgi = "*" if abs(farq.mean()) > 2 * se else " "
            qator.append(f"{nom} {farq.mean():+.3f} ({se:.3f}){belgi}")
            if nom == "sifatli" and farq.mean() > 2 * se:
                yaxshi.append(u)
        print(f"  {u:<17} " + "  ".join(qator))
    print("  (* - sezilarli: |farq| > 2 * SE)")
    if yaxshi:
        print(f"  sifatli ulushni sezilarli oshirdi: {', '.join(yaxshi)}")
    else:
        print("  hech bir usul sifatli ulushni sezilarli oshirmadi")
    print("  4 urug' - kichik tanlama: sezilarli emas = 'yordam bermaydi' degani emas")


if __name__ == "__main__":
    main()

=== 1. Spektral normalizatsiya: torch API va o'zimizniki ===
  asl W: sigma_max = 1.087
  spectral_norm(W): sigma_max = 1.001
  bizning BLinear (4 urug'): sigma_max = 1.006, 1.001, 1.000, 1.000

=== 2. To'rt usul, bir xil urug'lar va batchlar (900 qadam) ===
  usul              modalar (300, 500, 700, 900 qadam; 4 urug')
  oddiy             0 2 4 1 | 4 8 8 3 | 5 8 7 3 | 4 8 6 7
  label smoothing   5 6 6 5 | 6 8 6 7 | 8 6 7 6 | 4 7 8 7
  spektral          0 0 6 4 | 0 5 7 4 | 1 5 6 8 | 5 5 8 8
  minibatch std     0 4 8 1 | 4 7 8 4 | 7 5 8 1 | 8 7 8 4

=== 3. Yakuniy o'lchovlar (700-900 qadam o'rtachasi, 4 urug' o'rtachasi) ===
  usul                      modalar        sifatli  median masofa      muvozanat
  oddiy                       6.250          0.318          0.244          0.851
  label smoothing             6.417          0.425          0.174          0.862
  spektral                    5.750          0.246          0.326          0.920
  minibatch std               6.333          0.423          0.195          0.870

=== 4. Juftlashgan farq (usul - oddiy), SE, 2*SE qoidasi ===
  label smoothing   modalar +0.167 0.631-bob   sifatli +0.106 0.079-bob   median masofa -0.070 0.038-bob   muvozanat +0.011 0.024-bob
  spektral          modalar -0.500 1.266-bob   sifatli -0.073 0.079-bob   median masofa +0.082 0.063-bob   muvozanat +0.070 0.041-bob
  minibatch std     modalar +0.083 0.927-bob   sifatli +0.104 0.052-bob*  median masofa -0.049 0.034-bob   muvozanat +0.020 0.044-bob
  (* - sezilarli: |farq| > 2 * SE)
  sifatli ulushni sezilarli oshirdi: minibatch std
  4 urug' - kichik tanlama: sezilarli emas = 'yordam bermaydi' degani emas

"""Shartli GAN (cGAN) digits 8x8 da: sinf bo'yicha generatsiya, to'g'ri sinf ulushi va sinf qamrovi."""

import math

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split

S = 3                   # 3 urug' parallel
B, H, Z = 128, 128, 16
BELGI = " .:-=+*#%@"


class BLinear(nn.Module):
    def __init__(self, a, b):
        super().__init__()
        k = 1 / math.sqrt(a)
        self.w = nn.Parameter(torch.empty(S, a, b).uniform_(-k, k))
        self.b = nn.Parameter(torch.empty(S, 1, b).uniform_(-k, k))

    def forward(self, x):
        return torch.baddbmm(self.b, x, self.w)


def tarmoq(a, b, oxiri=None):
    qatlam = [BLinear(a, H), nn.LeakyReLU(0.2), BLinear(H, H), nn.LeakyReLU(0.2), BLinear(H, b)]
    return nn.Sequential(*qatlam, *([oxiri] if oxiri else []))


def klassifikator(X, y, Xt, yt):
    """Baholash uchun o'zimiz o'rgatgan kichik MLP (GAN dan mustaqil)."""
    torch.manual_seed(0)
    m = nn.Sequential(nn.Linear(64, 128), nn.ReLU(), nn.Linear(128, 10))
    opt = torch.optim.Adam(m.parameters(), lr=0.003, weight_decay=1e-4)
    for _ in range(300):
        loss = F.cross_entropy(m(X), y)
        opt.zero_grad()
        loss.backward()
        opt.step()
    m.eval()
    with torch.no_grad():
        acc = (m(Xt).argmax(1) == yt).float().mean().item()
    return m, acc


def orgat(X, y, shartli, qadamlar=1000):
    torch.manual_seed(0)
    c = 10 if shartli else 0
    G = tarmoq(Z + c, 64, nn.Tanh())
    D = tarmoq(64 + c, 1)
    oG = torch.optim.Adam(G.parameters(), lr=0.001, betas=(0.5, 0.999))
    oD = torch.optim.Adam(D.parameters(), lr=0.001, betas=(0.5, 0.999))
    g = torch.Generator().manual_seed(1)
    for _ in range(qadamlar):
        i = torch.randint(0, len(X), (S, B), generator=g)
        x, yx = X[i], F.one_hot(y[i], 10).float()
        z = torch.randn(S, B, Z, generator=g)
        yz = F.one_hot(torch.randint(0, 10, (S, B), generator=g), 10).float()
        if shartli:                      # shart (one-hot) G ga ham, D ga ham beriladi
            f = G(torch.cat([z, yz], 2))
            dx, df = torch.cat([x, yx], 2), torch.cat([f, yz], 2)
        else:
            f = G(z)
            dx, df = x, f
        ld = F.softplus(-D(dx)).mean() + F.softplus(D(df.detach())).mean()
        oD.zero_grad()
        ld.backward()
        oD.step()
        lg = F.softplus(-D(df)).mean()
        oG.zero_grad()
        lg.backward()
        oG.step()
    return G


def ascii_qator(rasmlar):
    """rasmlar: (n, 64) [-1, 1] -> 8 qator matn, raqamlar yonma-yon."""
    q = np.clip(((rasmlar + 1) / 2 * 9.99).astype(int), 0, 9).reshape(-1, 8, 8)
    return ["  " + "  ".join("".join(BELGI[v] for v in r[i]) for r in q) for i in range(8)]


def main() -> None:
    torch.set_num_threads(1)
    d = load_digits()
    X_np = d.data / 8.0 - 1.0                          # [0, 16] -> [-1, 1]
    Xo, Xt, yo, yt = train_test_split(X_np, d.target, test_size=0.3, random_state=0,
                                      stratify=d.target)
    Xo, Xt = torch.tensor(Xo, dtype=torch.float32), torch.tensor(Xt, dtype=torch.float32)
    yo, yt = torch.tensor(yo), torch.tensor(yt)

    print("=== 1. Baholovchi klassifikator ===")
    clf, acc = klassifikator(Xo, yo, Xt, yt)
    print(f"  o'quv {len(Xo)}, test {len(Xt)}; test aniqligi {acc:.3f}")

    print("\n=== 2. Shartli GAN: har sinfdan 100 tadan so'raymiz (3 urug') ===")
    Gc = orgat(Xo, yo, shartli=True)
    yz = torch.arange(10).repeat_interleave(100)
    z = torch.randn(S, 1000, Z, generator=torch.Generator().manual_seed(5))
    with torch.no_grad():
        xc = Gc(torch.cat([z, F.one_hot(yz, 10).float().expand(S, -1, -1)], 2))
        pc = torch.softmax(clf(xc), 2)
    togri = (pc.argmax(2) == yz).float()
    print("  urug'   to'g'ri sinf ulushi   o'rtacha ishonch")
    for s in range(S):
        print(f"  {s:>5} {togri[s].mean().item():>21.3f} {pc[s].max(1).values.mean().item():>18.3f}")
    print("  sinf bo'yicha (3 urug' o'rtachasi):")
    sinf = togri.reshape(S, 10, 100).mean((0, 2))
    print("   " + " ".join(f"{k}:{v:.2f}" for k, v in enumerate(sinf.tolist())))
    print("  urug' 0, har sinfdan bitta namuna (shart 0..9):")
    for qator in ascii_qator(xc[0, ::100].numpy()):
        print(qator)

    print("\n=== 3. Shartsiz GAN: qaysi sinflar chiqadi? ===")
    Gu = orgat(Xo, yo, shartli=False)
    with torch.no_grad():
        xu = Gu(z)
        pr = torch.softmax(clf(xu), 2)
        pu = pr.argmax(2)
    print("  urug'   sinflar ulushi (0..9)                                   >=5%  entropiya  ishonch")
    for s in range(S):
        p = torch.bincount(pu[s], minlength=10).float() / 1000
        ent = -(p[p > 0] * p[p > 0].log()).sum().item() / math.log(10)
        print(f"  {s:>5}   " + " ".join(f"{v:.2f}" for v in p.tolist())
              + f" {int((p >= 0.05).sum()):>4} {ent:>9.3f} {pr[s].max(1).values.mean().item():>8.3f}")
    p_c = torch.bincount(pc.argmax(2).reshape(-1), minlength=10).float() / (S * 1000)
    print(f"  shartli (taqqoslash uchun): max ulush {p_c.max().item():.2f}, "
          f"min ulush {p_c.min().item():.2f}")

    print("\n=== 4. Yodlab olish tekshiruvi: eng yaqin o'quv namunasigacha masofa ===")
    oquv_ichida = torch.cdist(Xt, Xo).min(1).values.median().item()
    gen = torch.cdist(xc[0], Xo).min(1).values
    print(f"  test -> o'quv (haqiqiy yangi rasmlar): median {oquv_ichida:.2f}")
    print(f"  cGAN -> o'quv:                        median {gen.median().item():.2f}, "
          f"< 1.0 bo'lganlar {(gen < 1.0).float().mean().item():.3f}")
    if gen.median().item() > 0.5 * oquv_ichida:
        print("  generatsiyalar o'quv namunalarining nusxasi emas")
    else:
        print("  generatsiyalar o'quv namunalariga shubhali darajada yaqin")


if __name__ == "__main__":
    main()

=== 1. Baholovchi klassifikator ===
  o'quv 1257, test 540; test aniqligi 0.972

=== 2. Shartli GAN: har sinfdan 100 tadan so'raymiz (3 urug') ===
  urug'   to'g'ri sinf ulushi   o'rtacha ishonch
      0                 0.892              0.963
      1                 0.969              0.971
      2                 0.967              0.969
  sinf bo'yicha (3 urug' o'rtachasi):
   0:1.00 1:0.90 2:1.00 3:0.80 4:0.97 5:0.99 6:1.00 7:1.00 8:0.88 9:0.91
  urug' 0, har sinfdan bitta namuna (shart 0..9):
     %:=      :@%       -@=       #-@:.      *%.      %%@@#      *%.       @@:      -=%=      #%:
    %@#.      .@@      @%%@-     *#.=+       @@      .@:::      .# .      *@@@     .#:*@.    .@%@+
    @# ==     =@@      :+=@      #% *+.     =@+      -%.        #           -@.    :#.+@:    =@@@@.
   +@: +:     @@@        :@       @@@#      @@:=:    -%++       %=-:        =@*     #@@*      @@@%
   #%  @*     @@@.       =%       .#@%     -@%@@:     -...      @@@+      =#@@      %@#.      .  @%
    # .@*     *@@:      *@= .       .==     -*@@        :+.     -*:.#     .@@.      #+:.        .*+
    %-@@      @@@:      @@+:+     .- @#     .-@*      ..*#      +# ##      =.       #+ :      ***@=
    :%@-      *@@      ++@@%%     +@@@.     .=@       @@-        *@@.      @        +#+=      %@@=

=== 3. Shartsiz GAN: qaysi sinflar chiqadi? ===
  urug'   sinflar ulushi (0..9)                                   >=5%  entropiya  ishonch
      0   0.10 0.08 0.09 0.08 0.07 0.12 0.10 0.10 0.15 0.10   10     0.989    0.833
      1   0.18 0.15 0.09 0.07 0.07 0.03 0.09 0.05 0.14 0.13    9     0.956    0.853
      2   0.06 0.13 0.10 0.17 0.14 0.14 0.05 0.08 0.10 0.03    9     0.960    0.878
  shartli (taqqoslash uchun): max ulush 0.12, min ulush 0.08

=== 4. Yodlab olish tekshiruvi: eng yaqin o'quv namunasigacha masofa ===
  test -> o'quv (haqiqiy yangi rasmlar): median 2.13
  cGAN -> o'quv:                        median 2.70, < 1.0 bo'lganlar 0.000
  generatsiyalar o'quv namunalarining nusxasi emas

lg = torch.log(1 - torch.sigmoid(D(fake))).mean()                  # ⚠️
lg = F.softplus(-D(fake)).mean()                                    # ✅ -log D(G(z))

opt = torch.optim.Adam(G.parameters(), lr=2e-3)                     # ⚠️ beta1 = 0.9
opt = torch.optim.Adam(G.parameters(), lr=2e-3, betas=(0.5, 0.999)) # ✅

gr = torch.autograd.grad(D(xh).sum(), xh)[0]                        # ⚠️ jarima D ga ta'sir qilmaydi
gr = torch.autograd.grad(D(xh).sum(), xh, create_graph=True)[0]     # ✅

ld = torch.sigmoid(D(fake)).mean() - torch.sigmoid(D(x)).mean()     # ⚠️ yana to'yinadi
ld = D(fake).mean() - D(x).mean() + 10 * gp                         # ✅

o = D(torch.cat([x, fake.detach()]))                                # ⚠️ std haqiqiy+soxtadan
o_x, o_f = D(x), D(fake.detach())                                   # ✅ alohida batchlar

ld = bce(D(x), 0.9) + bce(D(fake.detach()), 0.1)                    # ⚠️
ld = bce(D(x), 0.9) + bce(D(fake.detach()), 0.0)                    # ✅ faqat haqiqiy

fake = G(torch.cat([z, y1h], 1)); logit = D(fake)                   # ⚠️ D shartni bilmaydi
logit = D(torch.cat([fake, y1h], 1))                                # ✅

print("SN yaxshi" if sifat(orgat("sn", seed=0)) > sifat(orgat("oddiy", seed=1)) else "")   # ⚠️
farq = [sifat(orgat("sn", s)) - sifat(orgat("oddiy", s)) for s in range(4)]               # ✅

1.  # mode collapse va past sifat - farqi nima?
2.  # 8 ta modali vazifada G 3 ta modani yasaydi, har biri juda aniq. Sifatli ulush? Modalar?
3.  # d/dl log(1 - sigmoid(l)) = ? D(G(z)) -> 0 da nima bo'ladi?
4.  # -log D(G(z)) ning gradienti D(G(z)) -> 0 da?
5.  # Adam beta1 = 0.9 GAN da nega xavfli?
6.  # WGAN kritigi nega sigmoidsiz?
7.  # gradient penalty qaysi nuqtalarda hisoblanadi?
8.  # spektral norm dan keyin Linear qatlamning sigma_max?
9.  # label smoothing ni soxta nishonlarga ham qo'llasa?
10. # minibatch std collapse ni qanday "ko'radi"?
11. # cGAN da D nimani baholaydi?
12. # WGAN-GP va oddiy GAN ni 1000 G qadamida taqqoslash adolatlimi?

1.  lg = torch.log(1 - torch.sigmoid(D(G(z)))).mean()

2.  opt_D = torch.optim.Adam(D.parameters(), lr=2e-3)

3.  xh = a * x + (1 - a) * fake
    gr = torch.autograd.grad(D(xh).sum(), xh)[0]
    gp = (gr.norm(dim=1) - 1).mean()

4.  o = D(torch.cat([x, fake.detach()]))        # D ichida minibatch std bor
    ld = F.softplus(-o[:B]).mean() + F.softplus(o[B:]).mean()

5.  fake = G(torch.cat([z, F.one_hot(y, 10).float()], 1))
    ld = F.softplus(-D(x)).mean() + F.softplus(D(fake.detach())).mean()