26.2-dars: Avtoenkoder va VAE

TUZILISH:
  x (64) -> encoder -> z (d) -> decoder -> x_hat (64)
  "tor bo'g'iz": d < 64 -> tarmoq faqat muhim ma'lumotni saqlashga majbur
  loss = qayta tiklash xatosi: MSE yoki BCE (piksellar [0, 1] da)

PCA BILAN BOG'LIQLIK:
  chiziqli encoder/decoder + MSE -> PCA bilan bir xil fazoni topadi
  nochiziqli (ReLU) -> egri sirtlarni ham siqadi
  1-misol, test MSE (piksel [0, 1]):
    d = 2:   PCA 0.0530,  AE 0.0365
    d = 8:   PCA 0.0246,  AE 0.0122  (ikki barobar kam)
    d = 16:  PCA 0.0115,  AE 0.0065

QAYERDA ISHLATILADI:
  siqish, shovqinni tozalash (denoising AE), anomaliya aniqlash
  (qayta tiklash xatosi katta - g'ayrioddiy), oldindan o'rgatish,
  diffusion modellarda "latent fazo" (Matn-rasm modellari darsida)

MUAMMO:
  AE loss faqat "koddan rasmni tikla" deydi
  kodlar QAYERDA va QANDAY zichlikda joylashishi - nazoratsiz
  kodlar orasidagi bo'shliqlarni decoder hech qachon ko'rmagan

1-MISOL (d = 8):
  o'quv kodlari: o'rtacha norma 7.07, koordinata std 1.64 .. 2.51
  -> N(0, I) umuman boshqa joyda (normasi ~ sqrt(8) = 2.8)
  "teshik" = eng yaqin o'quv kodigacha masofa > test kodlarining 95-foizili
    z manbasi       ishonch>0.9   teshikda
    N(0, I)            0.348        0.964
    N(mu, Sigma)       0.594        0.502   (kodlarga Gauss moslab)
    test kodlari       0.920        0.051   (ma'lumotnoma)

XULOSA:
  hatto kodlarning o'rtacha va kovariatsiyasiga moslangan Gauss ham
  namunalarning yarmini teshiklarga tashlaydi
  -> kodlar taqsimoti Gauss emas; decoder u joylarda nima chizishni bilmaydi

GENERATIV MODEL:
  z ~ p(z) = N(0, I)                prior
  x ~ p(x | z) = Bernoulli(decoder(z))  (har piksel)
  p(x) = integral p(x | z) p(z) dz  <- hisoblab bo'lmaydi

ENCODER = TAXMINIY POSTERIOR:
  q(z | x) = N(mu(x), diag(sigma(x)^2))
  encoder bitta nuqta emas, TAQSIMOT beradi

ELBO (evidence lower bound):
  log p(x) >= E_q[log p(x | z)] - KL(q(z | x) || p(z)) = ELBO
               ^ qayta tiklash      ^ kod priorga yaqin bo'lsin
  loss = -ELBO = qayta_tiklash + KL

KL NING YOPIQ FORMULASI (ikkita Gauss):
  KL = 0.5 * sum_j (mu_j^2 + sigma_j^2 - 1 - log sigma_j^2)
  mu = 0, sigma = 1 bo'lsa - nol

NEGA TESHIKLAR YO'QOLADI:
  KL har kodni N(0, I) ga tortadi -> kodlar prior atrofida zich
  sigma > 0 -> decoder har kod ATROFIDAGI butun sohani ko'radi
  2-misol: VAE, z ~ N(0, I) -> teshikda 0.094 (AE N(mu, Sigma): 0.581)

2-MISOL TEST QIYMATLARI (nat/rasm, d = 8):
  qayta tiklash 21.07 + KL 3.40 = -ELBO 24.47
  AE qayta tiklashi 16.54 - KL jarimasi yo'q, shuning uchun yaxshiroq

MUAMMO:
  loss = E_{z ~ q(z|x)}[f(z)]; z - TASODIFIY namuna
  z = torch.normal(mu, sigma)  -> z.requires_grad = False
  gradient mu va sigma ga (demak encoderga) yetib bormaydi

YECHIM:
  eps ~ N(0, I)                     (parametrga bog'liq emas)
  z = mu + sigma * eps              (mu va sigma ning differensiallanuvchi funksiyasi)
  tasodif "chetga" chiqarildi; gradient mu va sigma orqali oqadi

2-MISOL TEKSHIRUVI: f(z) = (z - 3)^2, mu = 1, sigma = 1
  aniq: E[f] = (mu - 3)^2 + sigma^2 -> d/dmu = -4, d/dlogvar = sigma^2 = 1
  reparam (10000 eps): -4.021 va 1.025

MUQOBIL - REINFORCE (score function):
  grad = E[f(z) * d log q(z) / d mu] - har qanday f uchun ishlaydi
  lekin dispersiyasi katta: bitta namunali baho std
    reparam 2.00,  REINFORCE 9.44  (4.7 barobar shovqinliroq)
  diskret latentlar (reparam imkonsiz) uchun ishlatiladi

2D LATENT XARITASI (3-misol):
  z to'ri [-2, 2]^2 -> decoder -> hakam sinfi
  qo'shni nuqtalar - o'xshash raqamlar; sinflar yaxlit "orollar"
  orollar orasidagi chegaralarda - noaniq rasmlar ('.' ulushi 0.140)
  test kodlarining 0.940 qismi [-2, 2]^2 ichida - prior sohasi to'la ishlatilgan

INTERPOLATSIYA:
  z(t) = (1 - t) * z_0 + t * z_1,  t = 0 .. 1
  latentda: shakl asta-sekin o'zgaradi (0 -> ... -> 1)
  pikselda: ikki rasm ustma-ust tushadi (shaffof "arvoh")
  3-misol: 200 juftning o'rta nuqtasi, ishonch > 0.9:
    latentda 0.555, pikselda 0.515 - o'rta nuqta ko'pincha noaniq raqam

LATENT ARIFMETIKA (umumiy ma'lum g'oya):
  "tabassumli yuz" - "neytral yuz" + "boshqa odam" -> boshqa odam tabassumi
  bu faqat latent fazo silliq va tuzilgan bo'lsa ishlaydi

loss = qayta_tiklash + beta * KL

beta < 1: kodlar ko'proq ma'lumot saqlaydi -> qayta tiklash yaxshi,
          lekin prior bilan moslik yomonlashadi
beta > 1: kod priorga kuchliroq tortiladi -> "ajralgan" (disentangled)
          omillar, lekin qayta tiklash yomon; juda katta beta - collapse

3-MISOL (3 urug', d = 8, test):
  beta   qayta tiklash   KL     ishonch>0.9   sinf entropiyasi   faol o'lchov
  0.5       19.85        5.60     0.607          0.994              8.0
  1.0       21.90        2.95     0.586          0.994              7.7
  4.0       27.29        0.00     0.995          0.000              0.0

  beta 0.5: qayta tiklash sezilarli yaxshi (-2.049, SE 0.115),
            namuna ishonchi farqi sezilarli emas (+0.021, SE 0.023)
  beta 4:   KL = 0 - latent butunlay o'chdi; 1000 namunaning hammasi bir xil
            "o'rtacha raqam", hakam uni 8 deb 0.995 ishonch bilan taniydi

p(x | z, y), q(z | x, y); y - sinf (one-hot), encoder va decoderga qo'shiladi
generatsiya: y ni o'zimiz beramiz, z ~ N(0, I)
  "menga 7 chiz" -> decoder(z, y = 7)

MA'NO BO'LINISHI:
  y - QAYSI raqam; z - QANDAY yozilgan (qalinlik, qiyalik, uslub)
  bitta z + turli y -> bir xil uslubda turli raqamlar (ideal holda)

4-MISOL:
  so'ralgan sinf to'g'ri: 1.000 (hamma 10 sinfda)
  lekin sinf ichidagi xilma-xillik: CVAE 0.803, haqiqiy test 2.284
  z juda kam ma'lumot tashiydi (KL 0.91 nat) - "uslub" deyarli yo'q

HODISA:
  KL(q(z | x) || p(z)) -> 0: encoder x ga qaramay N(0, I) ni qaytaradi
  decoder z ni e'tiborsiz qoldirib, "o'rtacha" rasmni chizadi
  "faol o'lchov": test ustida Var(mu_j(x)) > 0.01

NEGA YUZ BERADI:
  KL jarimasi har nat axborot uchun "narx" oladi
  agar decoder z siz ham yetarli yaxshi bo'lsa (kuchli decoder, katta beta,
  yoki shart y allaqachon asosiy ma'lumotni bersa) - z ishlatilmaydi

4-MISOL (o'lchovlar bo'yicha KL, nat):
  VAE  beta=1:  jami 3.17, faol 7
  CVAE beta=1:  jami 0.91, faol 3  <- sinf y da, z ga kam ish qoldi
  VAE  beta=4:  jami 0.00, faol 0  -> bitta sinf, hamma namuna bir xil
  CVAE beta=4:  jami 0.00, faol 0  -> 10 sinf to'g'ri, lekin xilma-xillik 0.205

YECHIMLAR:
  KL annealing: beta ni 0 dan 1 ga asta-sekin oshirish
  free bits: har o'lchov KL i ma'lum chegaradan past bo'lsa jarima yo'q
  kuchsizroq decoder yoki kichikroq beta
  diagnostika: har doim KL ni o'lchov bo'yicha va faol o'lchovlar sonini chop eting

import torch
import torch.nn as nn
import torch.nn.functional as F

class VAE(nn.Module):
    def __init__(self, d):
        super().__init__()
        self.enc = nn.Sequential(nn.Linear(64, 128), nn.ReLU(), nn.Linear(128, 2 * d))
        self.dec = nn.Sequential(nn.Linear(d, 128), nn.ReLU(), nn.Linear(128, 64))

    def forward(self, x, g=None):
        mu, logvar = self.enc(x).chunk(2, dim=1)
        z = mu + torch.exp(0.5 * logvar) * torch.randn(mu.shape, generator=g)  # reparam
        return self.dec(z), mu, logvar

logit, mu, logvar = model(x, g)
qayta = F.binary_cross_entropy_with_logits(logit, x, reduction="none").sum(1)
kl = 0.5 * (mu ** 2 + logvar.exp() - 1 - logvar).sum(1)
loss = (qayta + beta * kl).mean()                     # -ELBO (beta = 1)

# namuna olish
z = torch.randn(n, d, generator=torch.Generator().manual_seed(0))
x_new = torch.sigmoid(model.dec(z))

# diagnostika
faol = (mu_test.var(0) > 0.01).sum()                  # faol latent o'lchovlar
kl_olchov = kl_har_olchov.mean(0)                     # posterior collapse ni ko'rish

# shartli: decoder(cat([z, one_hot(y)]))

AE: encoder -> tor latent -> decoder; nochiziqli PCA; namuna olishga yaroqsiz
VAE: q(z|x) = N(mu, sigma^2), p(z) = N(0, I); loss = qayta tiklash + KL
reparametrizatsiya: z = mu + sigma * eps
beta: qayta tiklash <-> prior; katta beta -> posterior collapse
CVAE: y - nima, z - qanday; xilma-xillikni alohida o'lchang

"""Avtoenkoder: siqish, qayta tiklash, PCA bilan taqqoslash va latent fazodagi "teshiklar"."""

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
from sklearn.model_selection import train_test_split

BELGI = " .:-=+*#%@"


def ascii_qator(rasmlar, izohlar):
    """[0, 1] oralig'idagi 8x8 rasmlarni yonma-yon chizadi."""
    satrlar = ["  ".join(f"{s:<8}" for s in izohlar)]
    for r in range(8):
        satrlar.append("  ".join("".join(BELGI[min(9, int(v * 10))] for v in im.reshape(8, 8)[r])
                                 for im in np.clip(rasmlar, 0, 1)))
    return satrlar


def malumot():
    d = load_digits()
    x_o, x_t, y_o, y_t = train_test_split(d.data / 16, d.target, test_size=0.25,
                                          stratify=d.target, random_state=0)
    f = lambda a: torch.tensor(a, dtype=torch.float32)
    return f(x_o), f(x_t), torch.tensor(y_o), torch.tensor(y_t)


class Klassifikator(nn.Module):
    """Hakam: [0, 1] oralig'idagi 64 piksel -> 10 sinf; h - xususiyatlar (128)."""

    def __init__(self):
        super().__init__()
        self.h = nn.Linear(64, 128)
        self.chiq = nn.Linear(128, 10)

    def forward(self, x):
        return self.chiq(F.relu(self.h(x)))


def hakam_orgat(x, y, seed=0):
    torch.manual_seed(seed)
    m = Klassifikator()
    opt = torch.optim.Adam(m.parameters(), lr=1e-2, weight_decay=1e-4)
    g = torch.Generator().manual_seed(seed)
    for _ in range(60):
        for idx in torch.randperm(len(x), generator=g).split(64):
            loss = F.cross_entropy(m(x[idx]), y[idx])
            opt.zero_grad()
            loss.backward()
            opt.step()
    return m.eval()


@torch.no_grad()
def tanish(hakam, x):
    """Hakam ishonchi > 0.9 bo'lgan namunalar ulushi va sinflar soni."""
    p = torch.softmax(hakam(x), -1)
    return (p.max(1).values > 0.9).float().mean().item(), np.bincount(p.argmax(1).numpy(), minlength=10)


class AE(nn.Module):
    def __init__(self, d):
        super().__init__()
        self.enc = nn.Sequential(nn.Linear(64, 128), nn.ReLU(), nn.Linear(128, d))
        self.dec = nn.Sequential(nn.Linear(d, 128), nn.ReLU(), nn.Linear(128, 64))

    def forward(self, x):
        return torch.sigmoid(self.dec(self.enc(x)))


def ae_orgat(x, d, seed=0, davr=100):
    torch.manual_seed(seed)
    m = AE(d)
    opt = torch.optim.Adam(m.parameters(), lr=3e-3)
    g = torch.Generator().manual_seed(seed)
    for _ in range(davr):
        for idx in torch.randperm(len(x), generator=g).split(64):
            loss = F.mse_loss(m(x[idx]), x[idx])
            opt.zero_grad()
            loss.backward()
            opt.step()
    return m.eval()


def main() -> None:
    torch.set_num_threads(1)
    x_o, x_t, y_o, y_t = malumot()
    hakam = hakam_orgat(x_o, y_o)

    print("=== 1. Siqish: 64 piksel -> d o'lchovli latent -> 64 piksel (test MSE) ===")
    print("  d    PCA (chiziqli)   AE (nochiziqli)   AE / PCA")
    modellar = {}
    for d in (2, 8, 16):
        pca = PCA(d, random_state=0).fit(x_o.numpy())
        mse_pca = np.mean((pca.inverse_transform(pca.transform(x_t.numpy())) - x_t.numpy()) ** 2)
        ae = ae_orgat(x_o, d)
        modellar[d] = ae
        with torch.no_grad():
            mse_ae = F.mse_loss(ae(x_t), x_t).item()
        print(f"  {d:<4} {mse_pca:>14.4f} {mse_ae:>17.4f} {mse_ae / mse_pca:>10.2f}")

    print("\n=== 2. Qayta tiklash (d = 8): asl | tiklangan ===")
    ae = modellar[8]
    with torch.no_grad():
        qayta = ae(x_t[:5])
    juft = torch.stack([x_t[:5], qayta], 1).reshape(10, 64).numpy()
    for s in ascii_qator(juft, ["asl", "AE"] * 5):
        print("  " + s)
    ish_q, _ = tanish(hakam, ae(x_t).detach())
    ish_t, _ = tanish(hakam, x_t)
    print(f"  hakam: haqiqiy test ishonch > 0.9 ulushi {ish_t:.3f}, tiklanganlar {ish_q:.3f}")

    print("\n=== 3. AE latentidan namuna olish (d = 8) ===")
    with torch.no_grad():
        z_o, z_t = ae.enc(x_o), ae.enc(x_t)
    mu, cov = z_o.mean(0), torch.cov(z_o.T)
    print(f"  o'quv kodlari: o'rtacha norma {z_o.norm(dim=1).mean():.2f}, "
          f"koordinata std {z_o.std(0).min():.2f} .. {z_o.std(0).max():.2f}")
    g = torch.Generator().manual_seed(1)
    L = torch.linalg.cholesky(cov)
    usullar = {
        "N(0, I)": torch.randn(1000, 8, generator=g),
        "N(mu, Sigma)": mu + torch.randn(1000, 8, generator=g) @ L.T,
        "test kodlari": z_t,
    }
    d_test = torch.cdist(z_t, z_o).min(1).values
    chegara = torch.quantile(d_test, 0.95)
    print(f"  'teshik' = eng yaqin o'quv kodigacha masofa > {chegara:.2f} (test kodlarining 95-foizili)")
    print("  z manbasi        ishonch>0.9   teshikda   sinflar (0..9)")
    for nom, z in usullar.items():
        with torch.no_grad():
            x = torch.sigmoid(ae.dec(z))
        ish, soni = tanish(hakam, x)
        teshik = (torch.cdist(z, z_o).min(1).values > chegara).float().mean().item()
        print(f"  {nom:<16} {ish:>10.3f} {teshik:>10.3f}   {' '.join(map(str, soni))}")
        if nom == "N(mu, Sigma)":
            namuna = x[:6].numpy()
    for s in ascii_qator(namuna, ["N(mu,S)"] * 6):
        print("  " + s)


if __name__ == "__main__":
    main()

=== 1. Siqish: 64 piksel -> d o'lchovli latent -> 64 piksel (test MSE) ===
  d    PCA (chiziqli)   AE (nochiziqli)   AE / PCA
  2            0.0530            0.0365       0.69
  8            0.0246            0.0122       0.50
  16           0.0115            0.0065       0.56

=== 2. Qayta tiklash (d = 8): asl | tiklangan ===
  asl       AE        asl       AE        asl       AE        asl       AE        asl       AE
    =@%.      :@#=      .%%.      .%*.       *+        +%       +@*.      .%+        *@.       +%:
    +%*@      =##%      %%+*      *%**      -@  +-    :@: -:    #%%*      %@@+      =@=.-     -@.#-
     - @-        @.     @= #     .%: #:     %* =@:    %= .%.    *%+@.     %%+@     .@=.@*    .@: @*
       @-       :@:    .@- *-    :%. --    -@=-@-    =@::@%     :@@@+     -*@@:    +% *%.    +@.*@-
      .@:       +@     -@  +-    -#  :-    *@@@%     =@%@%.      .+*%        @=    *@@@*     -@%@+
      #%        =#.    .@  *-    .%  +-    .:*@+      --@:         -@:       .%    .:+@.      -#%
    -#@@%     :*%*+     @+-%.     %*=@.      =@.       :@        ..=@-    .:.:@      +%        ##
    +@%*%.    -#*%#.    :@@+      .%@=       ##        %%       .%@@*     .#@@%      #+        #*
  hakam: haqiqiy test ishonch > 0.9 ulushi 0.947, tiklanganlar 0.920

=== 3. AE latentidan namuna olish (d = 8) ===
  o'quv kodlari: o'rtacha norma 7.07, koordinata std 1.64 .. 2.51
  'teshik' = eng yaqin o'quv kodigacha masofa > 2.85 (test kodlarining 95-foizili)
  z manbasi        ishonch>0.9   teshikda   sinflar (0..9)
  N(0, I)               0.348      0.964   39 30 32 37 154 87 43 58 260 260
  N(mu, Sigma)          0.594      0.502   63 112 116 102 102 64 74 86 194 87
  test kodlari          0.920      0.051   43 51 46 44 42 46 44 47 42 45
  N(mu,S)   N(mu,S)   N(mu,S)   N(mu,S)   N(mu,S)   N(mu,S)
     :==        %*      .#%%=     #@@@#      :*:      .#=.
    :###.      %@+.     *%-%#:    -%@@-      *%+      =%-+.
    +- @-     #% --     =  #%     .%@@      :#@=      %-=#
    +.:@:    =@-.%.     .  @=     .@@.     :%%@-      %=@-
    *#*@.    -@@@@.    .=+=@.     :@@+     .#%@=     =%+#=.
     ..@        .*-    .-:+%      .#%:       =@:     +@+%+
     .*%        +#:     .-#       +%-        :%      :*-+
     :+#        %%:     .#-       @%         :#:      :#:

"""VAE noldan: reparametrizatsiya, ELBO = qayta tiklash - KL va prior dan namuna olish."""

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split

BELGI = " .:-=+*#%@"


def ascii_qator(rasmlar, izohlar):
    """[0, 1] oralig'idagi 8x8 rasmlarni yonma-yon chizadi."""
    satrlar = ["  ".join(f"{s:<8}" for s in izohlar)]
    for r in range(8):
        satrlar.append("  ".join("".join(BELGI[min(9, int(v * 10))] for v in im.reshape(8, 8)[r])
                                 for im in np.clip(rasmlar, 0, 1)))
    return satrlar


def malumot():
    d = load_digits()
    x_o, x_t, y_o, y_t = train_test_split(d.data / 16, d.target, test_size=0.25,
                                          stratify=d.target, random_state=0)
    f = lambda a: torch.tensor(a, dtype=torch.float32)
    return f(x_o), f(x_t), torch.tensor(y_o), torch.tensor(y_t)


class Klassifikator(nn.Module):
    """Hakam: [0, 1] oralig'idagi 64 piksel -> 10 sinf; h - xususiyatlar (128)."""

    def __init__(self):
        super().__init__()
        self.h = nn.Linear(64, 128)
        self.chiq = nn.Linear(128, 10)

    def forward(self, x):
        return self.chiq(F.relu(self.h(x)))


def hakam_orgat(x, y, seed=0):
    torch.manual_seed(seed)
    m = Klassifikator()
    opt = torch.optim.Adam(m.parameters(), lr=1e-2, weight_decay=1e-4)
    g = torch.Generator().manual_seed(seed)
    for _ in range(60):
        for idx in torch.randperm(len(x), generator=g).split(64):
            loss = F.cross_entropy(m(x[idx]), y[idx])
            opt.zero_grad()
            loss.backward()
            opt.step()
    return m.eval()


@torch.no_grad()
def tanish(hakam, x):
    """Hakam ishonchi > 0.9 bo'lgan namunalar ulushi va sinflar soni."""
    p = torch.softmax(hakam(x), -1)
    return (p.max(1).values > 0.9).float().mean().item(), np.bincount(p.argmax(1).numpy(), minlength=10)


class AE(nn.Module):
    def __init__(self, d):
        super().__init__()
        self.enc = nn.Sequential(nn.Linear(64, 128), nn.ReLU(), nn.Linear(128, d))
        self.dec = nn.Sequential(nn.Linear(d, 128), nn.ReLU(), nn.Linear(128, 64))


class VAE(nn.Module):
    """Encoder q(z|x) = N(mu, sigma^2), decoder p(x|z) - piksel bo'yicha Bernoulli (logit)."""

    def __init__(self, d):
        super().__init__()
        self.enc = nn.Sequential(nn.Linear(64, 128), nn.ReLU(), nn.Linear(128, 2 * d))
        self.dec = nn.Sequential(nn.Linear(d, 128), nn.ReLU(), nn.Linear(128, 64))

    def kodla(self, x):
        mu, logvar = self.enc(x).chunk(2, dim=1)
        return mu, logvar

    def forward(self, x, g=None):
        mu, logvar = self.kodla(x)
        eps = torch.randn(mu.shape, generator=g)
        z = mu + torch.exp(0.5 * logvar) * eps                # reparametrizatsiya
        return self.dec(z), mu, logvar


def elbo_qismlari(model, x, g=None):
    """Bir rasm uchun o'rtacha: qayta tiklash (-log p(x|z), nat) va KL(q(z|x) || N(0, I))."""
    logit, mu, logvar = model(x, g)
    qayta = F.binary_cross_entropy_with_logits(logit, x, reduction="none").sum(1)
    kl = 0.5 * (mu ** 2 + logvar.exp() - 1 - logvar).sum(1)
    return qayta, kl


def orgat(tur, x, d=8, beta=1.0, seed=0, davr=100):
    torch.manual_seed(seed)
    m = AE(d) if tur == "AE" else VAE(d)
    opt = torch.optim.Adam(m.parameters(), lr=3e-3)
    g = torch.Generator().manual_seed(seed)
    for _ in range(davr):
        for idx in torch.randperm(len(x), generator=g).split(64):
            if tur == "AE":
                loss = F.binary_cross_entropy_with_logits(m.dec(m.enc(x[idx])), x[idx],
                                                          reduction="none").sum(1).mean()
            else:
                qayta, kl = elbo_qismlari(m, x[idx], g)
                loss = (qayta + beta * kl).mean()
            opt.zero_grad()
            loss.backward()
            opt.step()
    return m.eval()


def main() -> None:
    torch.set_num_threads(1)
    x_o, x_t, y_o, y_t = malumot()
    hakam = hakam_orgat(x_o, y_o)

    print("=== 1. Reparametrizatsiya: gradient z orqali o'tadimi? ===")
    mu = torch.tensor([1.0], requires_grad=True)
    logvar = torch.tensor([0.0], requires_grad=True)
    z_yomon = torch.normal(mu.detach().expand(10000), torch.exp(0.5 * logvar.detach()).expand(10000),
                           generator=torch.Generator().manual_seed(0))
    print(f"  z = torch.normal(mu, sigma):        z.requires_grad = {z_yomon.requires_grad}"
          f" -> mu va sigma gradient olmaydi")
    eps = torch.randn(10000, generator=torch.Generator().manual_seed(0))
    z = mu + torch.exp(0.5 * logvar) * eps
    ((z - 3) ** 2).mean().backward()
    print(f"  z = mu + sigma * eps:               z.requires_grad = {z.requires_grad}")
    print(f"  maqsad E[(z - 3)^2] = (mu - 3)^2 + sigma^2; aniq gradient: d/dmu = -4.000, d/dlogvar = 1.000")
    print(f"  reparametrizatsiya bahosi (10000 eps): d/dmu = {mu.grad.item():.3f}, "
          f"d/dlogvar = {logvar.grad.item():.3f}")
    with torch.no_grad():
        f = (z - 3) ** 2
        rep = 2 * (z - 3)                                     # har eps uchun d/dmu
        skor = f * (z - mu)                                   # REINFORCE: f(z) * d log q / d mu
    print(f"  d/dmu ning bitta-namunali bahosi: reparam std {rep.std():.2f}, "
          f"REINFORCE std {skor.std():.2f} (o'rtacha {skor.mean():.3f})")

    print("\n=== 2. VAE (d = 8) va AE: test ELBO qismlari (nat/rasm) ===")
    vae = orgat("VAE", x_o)
    ae = orgat("AE", x_o)
    with torch.no_grad():
        qayta, kl = elbo_qismlari(vae, x_t, torch.Generator().manual_seed(1))
        mu_t, _ = vae.kodla(x_t)
        qayta_mu = F.binary_cross_entropy_with_logits(vae.dec(mu_t), x_t, reduction="none").sum(1)
        qayta_ae = F.binary_cross_entropy_with_logits(ae.dec(ae.enc(x_t)), x_t, reduction="none").sum(1)
    print(f"  VAE: qayta tiklash {qayta.mean():.2f}, KL {kl.mean():.2f}, "
          f"-ELBO = {qayta.mean() + kl.mean():.2f}")
    print(f"  VAE qayta tiklash z = mu bilan (shovqinsiz): {qayta_mu.mean():.2f}")
    print(f"  AE  qayta tiklash:                          {qayta_ae.mean():.2f}  (KL jarimasi yo'q)")
    print(f"  bit ga: -ELBO = {(qayta.mean() + kl.mean()) / np.log(2):.1f} bit/rasm "
          f"(uzluksiz piksellar - 26.1 dagi binar bitlar bilan solishtirib bo'lmaydi)")

    print("\n=== 3. Prior dan namuna: z ~ N(0, I) -> decoder ===")
    g = torch.Generator().manual_seed(2)
    z0 = torch.randn(1000, 8, generator=g)
    with torch.no_grad():
        z_o_ae, z_t_ae = ae.enc(x_o), ae.enc(x_t)
        mu_o, lv_o = vae.kodla(x_o)
        _, lv_t = vae.kodla(x_t)
        g2 = torch.Generator().manual_seed(3)
        q_o = mu_o + torch.exp(0.5 * lv_o) * torch.randn(mu_o.shape, generator=g2)   # z ~ q(z|x)
        q_t = mu_t + torch.exp(0.5 * lv_t) * torch.randn(mu_t.shape, generator=g2)
        L = torch.linalg.cholesky(torch.cov(z_o_ae.T))
        manbalar = {
            "AE, N(0, I)": (torch.sigmoid(ae.dec(z0)), z0, z_o_ae, z_t_ae),
            "AE, N(mu, Sigma)": (torch.sigmoid(ae.dec(z_o_ae.mean(0) + z0 @ L.T)),
                                 z_o_ae.mean(0) + z0 @ L.T, z_o_ae, z_t_ae),
            "VAE, N(0, I)": (torch.sigmoid(vae.dec(z0)), z0, q_o, q_t),
        }
    print("  manba              ishonch>0.9   teshikda   sinflar (0..9)")
    for nom, (x, z, baza, test) in manbalar.items():
        ish, soni = tanish(hakam, x)
        chegara = torch.quantile(torch.cdist(test, baza).min(1).values, 0.95)
        teshik = (torch.cdist(z, baza).min(1).values > chegara).float().mean().item()
        print(f"  {nom:<18} {ish:>10.3f} {teshik:>10.3f}   {' '.join(map(str, soni))}")
    ish_t, _ = tanish(hakam, x_t)
    print(f"  {'haqiqiy test':<18} {ish_t:>10.3f}")
    print(f"  VAE o'quv kodlari z ~ q(z|x): o'rtacha {q_o.mean():+.2f}, "
          f"koordinata std {q_o.std(0).min():.2f} .. {q_o.std(0).max():.2f} (prior: 0 va 1)")
    for s in ascii_qator(manbalar["VAE, N(0, I)"][0][:6].numpy(), ["VAE"] * 6):
        print("  " + s)


if __name__ == "__main__":
    main()

=== 1. Reparametrizatsiya: gradient z orqali o'tadimi? ===
  z = torch.normal(mu, sigma):        z.requires_grad = False -> mu va sigma gradient olmaydi
  z = mu + sigma * eps:               z.requires_grad = True
  maqsad E[(z - 3)^2] = (mu - 3)^2 + sigma^2; aniq gradient: d/dmu = -4.000, d/dlogvar = 1.000
  reparametrizatsiya bahosi (10000 eps): d/dmu = -4.021, d/dlogvar = 1.025
  d/dmu ning bitta-namunali bahosi: reparam std 2.00, REINFORCE std 9.44 (o'rtacha -4.046)

=== 2. VAE (d = 8) va AE: test ELBO qismlari (nat/rasm) ===
  VAE: qayta tiklash 21.07, KL 3.40, -ELBO = 24.47
  VAE qayta tiklash z = mu bilan (shovqinsiz): 19.29
  AE  qayta tiklash:                          16.54  (KL jarimasi yo'q)
  bit ga: -ELBO = 35.3 bit/rasm (uzluksiz piksellar - 26.1 dagi binar bitlar bilan solishtirib bo'lmaydi)

=== 3. Prior dan namuna: z ~ N(0, I) -> decoder ===
  manba              ishonch>0.9   teshikda   sinflar (0..9)
  AE, N(0, I)             0.329      0.982   214 67 98 18 204 132 82 104 65 16
  AE, N(mu, Sigma)        0.601      0.581   55 113 96 114 98 77 89 104 172 82
  VAE, N(0, I)            0.661      0.094   88 82 76 119 113 85 112 108 129 88
  haqiqiy test            0.947
  VAE o'quv kodlari z ~ q(z|x): o'rtacha -0.03, koordinata std 0.96 .. 1.03 (prior: 0 va 1)
  VAE       VAE       VAE       VAE       VAE       VAE
    -@%=      .#*.      +%#*.     *@@*      +%#-      .**.
    *##*      *%+=     :%+++     :%=*%     :%##*      :%%:
    ..#*     .%:.+.    :#:::      :.%#     .+-#+      -@%:
    :+@%-    -%..=.    :#**:       +@=      -#%:      -%%.
   :%@@*:    -%:-+:    .+**=        *%.     +@*.      -%%
    =@+      .@:.*=     :.-#.       .%=     *%=       *%#-
    :%.       %*=#-     +-+#      ::+%-     ##-.      *@%+.
    +*        .#@+      +@#:      #@%=      *#=.      .#%#-

"""VAE latent fazosi: 2D xarita, beta-VAE murosasi va interpolatsiya."""

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split

BELGI = " .:-=+*#%@"


def ascii_qator(rasmlar, izohlar):
    """[0, 1] oralig'idagi 8x8 rasmlarni yonma-yon chizadi."""
    satrlar = ["  ".join(f"{s:<8}" for s in izohlar)]
    for r in range(8):
        satrlar.append("  ".join("".join(BELGI[min(9, int(v * 10))] for v in im.reshape(8, 8)[r])
                                 for im in np.clip(rasmlar, 0, 1)))
    return satrlar


def malumot():
    d = load_digits()
    x_o, x_t, y_o, y_t = train_test_split(d.data / 16, d.target, test_size=0.25,
                                          stratify=d.target, random_state=0)
    f = lambda a: torch.tensor(a, dtype=torch.float32)
    return f(x_o), f(x_t), torch.tensor(y_o), torch.tensor(y_t)


class Klassifikator(nn.Module):
    """Hakam: [0, 1] oralig'idagi 64 piksel -> 10 sinf; h - xususiyatlar (128)."""

    def __init__(self):
        super().__init__()
        self.h = nn.Linear(64, 128)
        self.chiq = nn.Linear(128, 10)

    def forward(self, x):
        return self.chiq(F.relu(self.h(x)))


def hakam_orgat(x, y, seed=0):
    torch.manual_seed(seed)
    m = Klassifikator()
    opt = torch.optim.Adam(m.parameters(), lr=1e-2, weight_decay=1e-4)
    g = torch.Generator().manual_seed(seed)
    for _ in range(60):
        for idx in torch.randperm(len(x), generator=g).split(64):
            loss = F.cross_entropy(m(x[idx]), y[idx])
            opt.zero_grad()
            loss.backward()
            opt.step()
    return m.eval()


@torch.no_grad()
def tanish(hakam, x):
    """Hakam ishonchi > 0.9 bo'lgan namunalar ulushi va sinflar soni."""
    p = torch.softmax(hakam(x), -1)
    return (p.max(1).values > 0.9).float().mean().item(), np.bincount(p.argmax(1).numpy(), minlength=10)


class VAE(nn.Module):
    def __init__(self, d):
        super().__init__()
        self.enc = nn.Sequential(nn.Linear(64, 128), nn.ReLU(), nn.Linear(128, 2 * d))
        self.dec = nn.Sequential(nn.Linear(d, 128), nn.ReLU(), nn.Linear(128, 64))

    def kodla(self, x):
        return self.enc(x).chunk(2, dim=1)

    def forward(self, x, g=None):
        mu, logvar = self.kodla(x)
        z = mu + torch.exp(0.5 * logvar) * torch.randn(mu.shape, generator=g)
        return self.dec(z), mu, logvar


def elbo_qismlari(model, x, g=None):
    logit, mu, logvar = model(x, g)
    qayta = F.binary_cross_entropy_with_logits(logit, x, reduction="none").sum(1)
    kl = 0.5 * (mu ** 2 + logvar.exp() - 1 - logvar).sum(1)
    return qayta, kl


def orgat(x, d=8, beta=1.0, seed=0, davr=40, B=128):
    torch.manual_seed(seed)
    m = VAE(d)
    opt = torch.optim.Adam(m.parameters(), lr=3e-3)
    g = torch.Generator().manual_seed(seed)
    for _ in range(davr):
        for idx in torch.randperm(len(x), generator=g).split(B):
            qayta, kl = elbo_qismlari(m, x[idx], g)
            loss = (qayta + beta * kl).mean()
            opt.zero_grad()
            loss.backward()
            opt.step()
    return m.eval()


@torch.no_grad()
def olchovlar(m, hakam, x_t, d=8):
    qayta, kl = elbo_qismlari(m, x_t, torch.Generator().manual_seed(1))
    mu, _ = m.kodla(x_t)
    faol = int((mu.var(0) > 0.01).sum())                     # faol latent o'lchovlar
    x = torch.sigmoid(m.dec(torch.randn(1000, d, generator=torch.Generator().manual_seed(2))))
    ish, soni = tanish(hakam, x)
    u = soni[soni > 0] / soni.sum()
    entropiya = float(-(u * np.log(u)).sum() / np.log(10))   # 1 - hamma sinf teng, 0 - bitta sinf
    return qayta.mean().item(), kl.mean().item(), ish, faol, entropiya, soni


def main() -> None:
    torch.set_num_threads(1)
    x_o, x_t, y_o, y_t = malumot()
    hakam = hakam_orgat(x_o, y_o)

    print("=== 1. 2D latent xaritasi: z to'rini decoder dan o'tkazib, hakam bilan o'qiymiz ===")
    m2 = orgat(x_o, d=2)
    to_r = torch.linspace(-2, 2, 11)
    zz = torch.stack(torch.meshgrid(to_r, to_r.flip(0), indexing="xy"), -1).reshape(-1, 2)
    with torch.no_grad():
        p = torch.softmax(hakam(torch.sigmoid(m2.dec(zz))), -1)
    belgi = [str(s) if q > 0.7 else "." for s, q in zip(p.argmax(1).tolist(), p.max(1).values.tolist())]
    print("  z2 ^   (raqam - hakam sinfi, '.' - ishonch < 0.7)")
    for r in range(11):
        print(f"  {to_r.flip(0)[r].item():+4.1f} " + " ".join(belgi[r * 11:(r + 1) * 11]))
    print("        -2" + " " * 16 + "+2  -> z1")
    with torch.no_grad():
        mu2, _ = m2.kodla(x_t)
    print(f"  test kodlarining {((mu2.abs() < 2).all(1)).float().mean():.3f} qismi [-2, 2]^2 kvadrat ichida; "
          f"xaritada '.' ulushi {belgi.count('.') / len(belgi):.3f}")

    print("\n=== 2. beta-VAE (d = 8): 3 urug', test ustida (nat/rasm) ===")
    betalar = (0.5, 1.0, 4.0)
    nat = {b: [olchovlar(orgat(x_o, beta=b, seed=s), hakam, x_t) for s in range(3)] for b in betalar}
    print("  beta   qayta tiklash     KL   ishonch>0.9   sinf entropiyasi   faol o'lchov")
    for b in betalar:
        v = np.array([r[:5] for r in nat[b]])
        print(f"  {b:<5} {v[:, 0].mean():>11.2f} {v[:, 1].mean():>9.2f} {v[:, 2].mean():>10.3f}"
              f" {v[:, 4].mean():>15.3f} {v[:, 3].mean():>14.1f}")
    for b in (0.5, 4.0):
        for i, nom in ((0, "qayta tiklash"), (2, "ishonch>0.9")):
            f = np.array([r[i] for r in nat[b]]) - np.array([r[i] for r in nat[1.0]])
            se = f.std(ddof=1) / np.sqrt(3)
            print(f"  beta {b} - beta 1, {nom:<13}: {f.mean():+7.3f}, SE {se:.3f} -> "
                  f"{'sezilarli' if abs(f.mean()) > 2 * se else 'sezilarli emas'}")
    soni = nat[4.0][0][5]
    print(f"  beta 4 (urug' 0) prior namunalari sinflari: {' '.join(map(str, soni))}")

    print("\n=== 3. Interpolatsiya (beta = 1, d = 8): 0 -> 1 raqami, latent chiziq bo'yicha ===")
    m = orgat(x_o, beta=1.0, seed=0)
    i0, i1 = int((y_t == 0).nonzero()[0]), int((y_t == 1).nonzero()[0])
    with torch.no_grad():
        mu, _ = m.kodla(x_t[[i0, i1]])
        t = torch.linspace(0, 1, 7)[:, None]
        x = torch.sigmoid(m.dec((1 - t) * mu[0] + t * mu[1]))
        p = torch.softmax(hakam(x), -1)
    izoh = [f"{a}:{q:.2f}" for a, q in zip(p.argmax(1).tolist(), p.max(1).values.tolist())]
    for s in ascii_qator(x.numpy(), izoh):
        print("  " + s)
    g = torch.Generator().manual_seed(4)
    juft = torch.randint(0, len(x_t), (200, 2), generator=g)
    with torch.no_grad():
        mu, _ = m.kodla(x_t)
        orta = torch.sigmoid(m.dec(0.5 * (mu[juft[:, 0]] + mu[juft[:, 1]])))
        piksel = 0.5 * (x_t[juft[:, 0]] + x_t[juft[:, 1]])
    print(f"  200 tasodifiy juftning o'rta nuqtasi, ishonch>0.9: latentda {tanish(hakam, orta)[0]:.3f}, "
          f"pikselda (ikki rasm ustma-ust) {tanish(hakam, piksel)[0]:.3f}")


if __name__ == "__main__":
    main()

=== 1. 2D latent xaritasi: z to'rini decoder dan o'tkazib, hakam bilan o'qiymiz ===
  z2 ^   (raqam - hakam sinfi, '.' - ishonch < 0.7)
  +2.0 9 9 9 9 9 . 7 7 7 7 7
  +1.6 9 9 9 9 9 . 7 7 7 7 7
  +1.2 9 9 9 9 9 . 7 7 7 7 7
  +0.8 9 9 9 3 . 8 . 7 7 7 7
  +0.4 9 9 . 3 3 8 8 . . 4 4
  +0.0 9 9 . 3 . . 8 1 . 4 4
  -0.4 9 9 9 . 2 2 8 1 4 4 4
  -0.8 0 0 0 0 . . 6 6 4 4 4
  -1.2 0 0 0 0 6 6 6 6 . 4 4
  -1.6 0 0 0 0 6 6 6 6 6 4 4
  -2.0 0 0 0 0 6 6 6 6 6 . 4
        -2                +2  -> z1
  test kodlarining 0.940 qismi [-2, 2]^2 kvadrat ichida; xaritada '.' ulushi 0.140

=== 2. beta-VAE (d = 8): 3 urug', test ustida (nat/rasm) ===
  beta   qayta tiklash     KL   ishonch>0.9   sinf entropiyasi   faol o'lchov
  0.5         19.85      5.60      0.607           0.994            8.0
  1.0         21.90      2.95      0.586           0.994            7.7
  4.0         27.29      0.00      0.995           0.000            0.0
  beta 0.5 - beta 1, qayta tiklash:  -2.049, SE 0.115 -> sezilarli
  beta 0.5 - beta 1, ishonch>0.9  :  +0.021, SE 0.023 -> sezilarli emas
  beta 4.0 - beta 1, qayta tiklash:  +5.388, SE 0.137 -> sezilarli
  beta 4.0 - beta 1, ishonch>0.9  :  +0.409, SE 0.011 -> sezilarli
  beta 4 (urug' 0) prior namunalari sinflari: 0 0 0 0 0 0 0 0 1000 0

=== 3. Interpolatsiya (beta = 1, d = 8): 0 -> 1 raqami, latent chiziq bo'yicha ===
  0:1.00    0:1.00    0:0.99    0:0.60    4:0.93    1:0.65    1:0.98
    :%*.      .#*.      .**:       =#-       -#=       :#+       .#*.
    %%*+      #%*+      +#*+      =##+      :*#+.     .*%*.     .+%*.
   :@: *.    :%-.*.    .%+:*:    .#*-*:     +#=*:     =%**:     -%%#.
   -%. +=    -%:.+=    :%-.*-    :%=-*-    :#*+*:    .####.    .*%%#.
   =%. *+    =%:.*=    -#::*=    -#--#-    -#+*#:    :**##.    .*#%*.
   :@. *+    :%. *=    :%::#-    :#--#:    :*=*#.    .++##.    .=+%*
    %*=%:     #*+#:     *++#:     +=*#.     --##.     :-#*      .-%*
    :%%=      :#%=      .*%=       =%+       -#+       :#+       .#+
  200 tasodifiy juftning o'rta nuqtasi, ishonch>0.9: latentda 0.555, pikselda (ikki rasm ustma-ust) 0.515

"""Shartli VAE va posterior collapse: sinf berib generatsiya, uslub va latent o'lchovlar KL i."""

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split

BELGI = " .:-=+*#%@"


def ascii_qator(rasmlar, izohlar):
    """[0, 1] oralig'idagi 8x8 rasmlarni yonma-yon chizadi."""
    satrlar = ["  ".join(f"{s:<8}" for s in izohlar)]
    for r in range(8):
        satrlar.append("  ".join("".join(BELGI[min(9, int(v * 10))] for v in im.reshape(8, 8)[r])
                                 for im in np.clip(rasmlar, 0, 1)))
    return satrlar


def malumot():
    d = load_digits()
    x_o, x_t, y_o, y_t = train_test_split(d.data / 16, d.target, test_size=0.25,
                                          stratify=d.target, random_state=0)
    f = lambda a: torch.tensor(a, dtype=torch.float32)
    return f(x_o), f(x_t), torch.tensor(y_o), torch.tensor(y_t)


class Klassifikator(nn.Module):
    """Hakam: [0, 1] oralig'idagi 64 piksel -> 10 sinf; h - xususiyatlar (128)."""

    def __init__(self):
        super().__init__()
        self.h = nn.Linear(64, 128)
        self.chiq = nn.Linear(128, 10)

    def forward(self, x):
        return self.chiq(F.relu(self.h(x)))


def hakam_orgat(x, y, seed=0):
    torch.manual_seed(seed)
    m = Klassifikator()
    opt = torch.optim.Adam(m.parameters(), lr=1e-2, weight_decay=1e-4)
    g = torch.Generator().manual_seed(seed)
    for _ in range(60):
        for idx in torch.randperm(len(x), generator=g).split(64):
            loss = F.cross_entropy(m(x[idx]), y[idx])
            opt.zero_grad()
            loss.backward()
            opt.step()
    return m.eval()


@torch.no_grad()
def tanish(hakam, x):
    """Hakam ishonchi > 0.9 bo'lgan namunalar ulushi va sinflar soni."""
    p = torch.softmax(hakam(x), -1)
    return (p.max(1).values > 0.9).float().mean().item(), np.bincount(p.argmax(1).numpy(), minlength=10)


class CVAE(nn.Module):
    """Shartli VAE: encoder q(z|x, y), decoder p(x|z, y); y - raqam sinfi (one-hot)."""

    def __init__(self, d, k=10):
        super().__init__()
        self.k = k
        self.enc = nn.Sequential(nn.Linear(64 + k, 128), nn.ReLU(), nn.Linear(128, 2 * d))
        self.dec = nn.Sequential(nn.Linear(d + k, 128), nn.ReLU(), nn.Linear(128, 64))

    def kodla(self, x, y):
        return self.enc(torch.cat([x, F.one_hot(y, self.k).float()], 1)).chunk(2, dim=1)

    def dekodla(self, z, y):
        return self.dec(torch.cat([z, F.one_hot(y, self.k).float()], 1))

    def forward(self, x, y, g=None):
        mu, logvar = self.kodla(x, y)
        z = mu + torch.exp(0.5 * logvar) * torch.randn(mu.shape, generator=g)
        return self.dekodla(z, y), mu, logvar


def qismlar(m, x, y, g=None):
    logit, mu, logvar = m(x, y, g)
    qayta = F.binary_cross_entropy_with_logits(logit, x, reduction="none").sum(1)
    kl_olchov = 0.5 * (mu ** 2 + logvar.exp() - 1 - logvar)       # har latent o'lchov uchun
    return qayta, kl_olchov


def orgat(x, y, beta=1.0, seed=0, d=8, davr=40, B=128):
    torch.manual_seed(seed)
    m = CVAE(d)
    opt = torch.optim.Adam(m.parameters(), lr=3e-3)
    g = torch.Generator().manual_seed(seed)
    for _ in range(davr):
        for idx in torch.randperm(len(x), generator=g).split(B):
            qayta, kl = qismlar(m, x[idx], y[idx], g)
            loss = (qayta + beta * kl.sum(1)).mean()
            opt.zero_grad()
            loss.backward()
            opt.step()
    return m.eval()


class VAE(nn.Module):
    def __init__(self, d=8):
        super().__init__()
        self.enc = nn.Sequential(nn.Linear(64, 128), nn.ReLU(), nn.Linear(128, 2 * d))
        self.dec = nn.Sequential(nn.Linear(d, 128), nn.ReLU(), nn.Linear(128, 64))


def qismlar_vae(m, x, g=None):
    mu, logvar = m.enc(x).chunk(2, dim=1)
    z = mu + torch.exp(0.5 * logvar) * torch.randn(mu.shape, generator=g)
    qayta = F.binary_cross_entropy_with_logits(m.dec(z), x, reduction="none").sum(1)
    return qayta, 0.5 * (mu ** 2 + logvar.exp() - 1 - logvar)


def orgat_vae(x, beta, seed=0, davr=40, B=128):
    torch.manual_seed(seed)
    m = VAE()
    opt = torch.optim.Adam(m.parameters(), lr=3e-3)
    g = torch.Generator().manual_seed(seed)
    for _ in range(davr):
        for idx in torch.randperm(len(x), generator=g).split(B):
            qayta, kl = qismlar_vae(m, x[idx], g)
            loss = (qayta + beta * kl.sum(1)).mean()
            opt.zero_grad()
            loss.backward()
            opt.step()
    return m.eval()


def sinf_ichidagi_xilma(x, y):
    """Har sinf ichida rasmlar orasidagi o'rtacha Evklid masofa (sinflar bo'yicha o'rtacha)."""
    return float(np.mean([torch.pdist(x[y == k]).mean().item() for k in range(10)]))


def main() -> None:
    torch.set_num_threads(1)
    x_o, x_t, y_o, y_t = malumot()
    hakam = hakam_orgat(x_o, y_o)
    y_kerak = torch.arange(10).repeat_interleave(100)               # har sinfdan 100 ta

    print("=== 1. Shartli generatsiya: 'menga 7 chiz' (har sinfdan 100 namuna) ===")
    cvae = orgat(x_o, y_o)
    with torch.no_grad():
        z = torch.randn(1000, 8, generator=torch.Generator().manual_seed(2))
        x = torch.sigmoid(cvae.dekodla(z, y_kerak))
        p = torch.softmax(hakam(x), -1)
    togri = (p.argmax(1) == y_kerak).float()
    print(f"  hakam so'ralgan sinfni tanidi: {togri.mean():.3f} (tasodifiy taxmin 0.100)")
    print("  sinf bo'yicha: " + " ".join(f"{k}:{togri[y_kerak == k].mean():.2f}" for k in range(10)))
    print(f"  ishonch > 0.9 ulushi {(p.max(1).values > 0.9).float().mean():.3f}; "
          f"haqiqiy test {tanish(hakam, x_t)[0]:.3f}")

    print("\n=== 2. Uslubni saqlab sinfni almashtirish: bitta z, o'nta y ===")
    for i in (int((y_t == 3).nonzero()[0]), int((y_t == 7).nonzero()[1])):
        with torch.no_grad():
            mu, _ = cvae.kodla(x_t[i:i + 1], y_t[i:i + 1])
            x10 = torch.sigmoid(cvae.dekodla(mu.repeat(10, 1), torch.arange(10)))
            p10 = hakam(x10).argmax(1)
        izoh = [f"asl {int(y_t[i])}"] + [f"y={k}->{int(a)}" for k, a in enumerate(p10)]
        for s in ascii_qator(torch.cat([x_t[i:i + 1], x10]).numpy(), izoh):
            print("  " + s)

    print("\n=== 3. Posterior collapse: latent o'lchovlar bo'yicha KL (test, nat) ===")
    modellar = {"VAE beta=1": orgat_vae(x_o, 1.0), "CVAE beta=1": cvae,
                "VAE beta=4": orgat_vae(x_o, 4.0), "CVAE beta=4": orgat(x_o, y_o, beta=4.0)}
    print("  model          KL (8 o'lchov, kamayish tartibida)                 jami   faol")
    xilma = {}
    for nom, m in modellar.items():
        with torch.no_grad():
            if nom.startswith("CVAE"):
                _, kl = qismlar(m, x_t, y_t, torch.Generator().manual_seed(1))
                xs = torch.sigmoid(m.dekodla(z, y_kerak))
            else:
                _, kl = qismlar_vae(m, x_t, torch.Generator().manual_seed(1))
                xs = torch.sigmoid(m.dec(z))
        k = kl.mean(0).sort(descending=True).values
        print(f"  {nom:<14} {' '.join(f'{v:5.2f}' for v in k.tolist())}   {k.sum():5.2f}   {int((k > 0.05).sum())}")
        xilma[nom] = xs
    print("\n  sinf ichidagi xilma-xillik (hakam sinfi bo'yicha guruhlab, o'rtacha juft masofa):")
    print(f"    haqiqiy test: {sinf_ichidagi_xilma(x_t, y_t):.3f}")
    for nom, xs in xilma.items():
        with torch.no_grad():
            ys = hakam(xs).argmax(1)
        bor = [k for k in range(10) if (ys == k).sum() >= 2]
        xm = float(np.mean([torch.pdist(xs[ys == k]).mean().item() for k in bor]))
        print(f"    {nom:<14} {xm:.3f}  (sinflar soni: {len(bor)})")


if __name__ == "__main__":
    main()

=== 1. Shartli generatsiya: 'menga 7 chiz' (har sinfdan 100 namuna) ===
  hakam so'ralgan sinfni tanidi: 1.000 (tasodifiy taxmin 0.100)
  sinf bo'yicha: 0:1.00 1:1.00 2:1.00 3:1.00 4:1.00 5:1.00 6:1.00 7:1.00 8:1.00 9:1.00
  ishonch > 0.9 ulushi 0.998; haqiqiy test 0.947

=== 2. Uslubni saqlab sinfni almashtirish: bitta z, o'nta y ===
  asl 3     y=0->0    y=1->1    y=2->2    y=3->3    y=4->4    y=5->5    y=6->6    y=7->7    y=8->8    y=9->9
    +@*       :%*.      .*+.      #@=       *@%-       =*       *%%#.      #+       =%%*.     -##:      =%*:
    +.@:      ####      .%%-     =@##.     -%+#*      .%+      -@#+:      -@=      .#*##.    .#***     :%+*#
     -%.     :%-.%:     :@@:     -=-%:     .::%=      +%-:.    -%-.       ##        ..**     .*+*#     :#-*%.
    .@@=     -#. *=     =@%:       -%.       *@-     :@==+-    -%%*:     .%+:.      :+%#-     =%%-     .+%%@:
      .@.    =#  ++     -%%:       **        :*#.    *@+%%=    .===+.    :@%##:    .*@@*-     -@#:       ::#=
   :-  %=    :%. *=     -*#-      -%-         .%+    +%%@*.      .-+:    .@*:**     =%+.      #+=+.        *+
   -% -@.     %**%.     =#%+.    .%@##+     +-=%=     :+%:      ==*+      *%-##     -%:      .%==#.     +::#=
    *@@-      :%%-      .*%#-.    #@%#*.    *@@*.      -#.      #@*.       *@%-     +*        -#%+      =%%#.
  asl 7     y=0->0    y=1->1    y=2->2    y=3->3    y=4->4    y=5->5    y=6->6    y=7->7    y=8->8    y=9->9
    .%@+      :%*.      .+*:      #@+       *%%-       -#       *%%#:      **       -%@#-     -##-      =%#-
    :#@*      #%##      .%@+     =@#%:     -#+##      .%+      -@#+-      -@=      .*##%:    .#**#.    :%+*#.
      @#     :%-.%-     :@@=     -=:%-     ..:%+      +#--:    -%-.       #*        ..+#.    .*+*%.    :#-*@:
     .@+     -#. *=     =@@-       :%.       +@-     :@-=*=    -%#*:     .%+:       :=%%-     =%%=     .+#%@:
   .-*@#.    =#  ++     =%%:       **        :##.    *@+%%=    .=++=.    :@%#*:    .*@@*-     -@%:      .::#-
   @@@@%.    :%. #-     -#%-      -%=         .%=    +%%@*.      .=+.    .@#-**     -%*.     .#+++         *=
   ::%#       %**%.     -#%=     .%@#*=    .+-+%=     .+%:      =+*=      #%-##     :%:      .%==*.     =:-#-
    .@=       :%%-      .+%*.     #%##+.    *@@*       =#       #@+        *@%-     +*        -#%=      =%%*.

=== 3. Posterior collapse: latent o'lchovlar bo'yicha KL (test, nat) ===
  model          KL (8 o'lchov, kamayish tartibida)                 jami   faol
  VAE beta=1      0.88  0.81  0.74  0.43  0.15  0.07  0.06  0.02    3.17   7
  CVAE beta=1     0.40  0.40  0.08  0.01  0.01  0.01  0.01  0.00    0.91   3
  VAE beta=4      0.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00    0.00   0
  CVAE beta=4     0.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00    0.00   0

  sinf ichidagi xilma-xillik (hakam sinfi bo'yicha guruhlab, o'rtacha juft masofa):
    haqiqiy test: 2.284
    VAE beta=1     1.232  (sinflar soni: 10)
    CVAE beta=1    0.803  (sinflar soni: 10)
    VAE beta=4     0.220  (sinflar soni: 1)
    CVAE beta=4    0.205  (sinflar soni: 10)

z = torch.normal(mu, torch.exp(0.5 * logvar))                       # ⚠️
z = mu + torch.exp(0.5 * logvar) * torch.randn(mu.shape, generator=g)   # ✅

z = mu + logvar * eps                                               # ⚠️
z = mu + torch.exp(0.5 * logvar) * eps                              # ✅

loss = F.binary_cross_entropy_with_logits(logit, x) + kl.sum(1).mean()   # o'rtacha vs yig'indi  # ⚠️
qayta = F.binary_cross_entropy_with_logits(logit, x, reduction="none").sum(1)
loss = (qayta + beta * kl.sum(1)).mean()                            # ✅

x_new = ae.dec(torch.randn(n, d))                                   # ⚠️
# VAE o'rgating yoki kamida kodlar taqsimotiga mos model (GMM) qo'ying   # ✅

print("KL:", kl.mean())                   # 0.00 - "zo'r!"          # ⚠️
print("faol:", (mu.var(0) > 0.01).sum(), "KL o'lchov:", kl.mean(0)) # ✅

eng_beta = max(betalar, key=lambda b: ishonch[b])                   # ⚠️
# qayta tiklash, KL, ishonch, sinf entropiyasi, faol o'lchov - birga   # ✅

print((hakam(x).argmax(1) == y_kerak).float().mean())               # ⚠️
print(togri_sinf, sinf_ichidagi_xilma(x, y_kerak), faol_olchov)     # ✅

1.  # chiziqli AE + MSE qaysi klassik usulga teng?
2.  # nega AE latentidan N(0, I) bilan namuna olish yomon?
3.  # ELBO ning ikki qismi?
4.  # mu = 0, sigma = 1 da KL(q || N(0, 1)) = ?
5.  # z = torch.normal(mu, sigma) dan keyin z.requires_grad?
6.  # E[(z - 3)^2] ning mu bo'yicha gradienti (mu = 1)?
7.  # beta oshsa qayta tiklash va KL qanday o'zgaradi?
8.  # "faol o'lchov" qanday aniqlanadi?
9.  # KL = 0 va hamma namuna bir xil - bu nima?
10. # CVAE da y va z nimani kodlaydi?
11. # posterior collapse ga qarshi ikki usul?
12. # nega beta ni faqat "ishonch > 0.9" bo'yicha tanlab bo'lmaydi?

1.  mu, logvar = model.enc(x).chunk(2, dim=1)
    z = torch.normal(mu, logvar)

2.  qayta = F.mse_loss(torch.sigmoid(logit), x)
    kl = 0.5 * (mu ** 2 + logvar.exp() - 1 - logvar).sum()
    loss = qayta + kl

3.  x_new = torch.sigmoid(ae.dec(torch.randn(100, 8)))

4.  kl = 0.5 * (mu ** 2 + logvar.exp() - 1 - logvar).sum(1).mean()
    print("KL juda kichik - model zo'r!" if kl < 0.01 else "")

5.  togri = (hakam(cvae_namuna).argmax(1) == y_kerak).float().mean()
    print("CVAE tayyor" if togri > 0.99 else "yana o'rgat")