26.5-dars: Diffusion asoslari

FORWARD (O'RGANILMAYDI):
  x_0 (rasm) -> x_1 -> x_2 -> ... -> x_T (toza shovqin)
  har qadamda: x_t = sqrt(1 - beta_t) * x_{t-1} + sqrt(beta_t) * eps,  eps ~ N(0, I)
  beta_t - kichik son (masalan, 0.0001 ... 0.02)
  sqrt(1 - beta_t) ko'paytmasi signalni biroz "so'ndiradi" -
    shuning uchun dispersiya portlamaydi va x_T ~ N(0, I)

REVERSE (O'RGANILADI):
  x_T ~ N(0, I) -> x_{T-1} -> ... -> x_0 (yangi namuna)
  p_theta(x_{t-1} | x_t) - neyron tarmoq bilan
  beta_t kichik bo'lsa, teskari qadam ham taxminan Gauss -
    faqat uning O'RTACHASINI o'rganish yetarli

NEGA ISHLAYDI - INTUITSIYA:
  bitta katta sakrash (shovqin -> rasm) - juda qiyin
  1000 ta kichik "biroz tozalash" qadami - har biri oson
  har qadamda tarmoq bitta savolga javob beradi: "bu yerda qaysi shovqin bor?"

VAE VA GAN BILAN (26.2, 26.3):
  VAE:       encoder o'rganiladi, 1 ta yashirin qadam, loss - ELBO
  GAN:       o'yin, 1 ta qadam (G(z)), loss - raqib
  Diffusion: encoder QAT'IY (shovqin qo'shish), T ta qadam, loss - MSE
             yashirin o'lcham = rasm o'lchami (siqish yo'q)

BELGILAR:
  alfa_t     = 1 - beta_t
  alfa_bar_t = alfa_1 * alfa_2 * ... * alfa_t     (kumulyativ ko'paytma)

YOPIQ FORMULA (t ta qadamni bitta qadamga yig'ish):
  q(x_t | x_0) = N( sqrt(alfa_bar_t) * x_0,  (1 - alfa_bar_t) * I )
  x_t = sqrt(alfa_bar_t) * x_0 + sqrt(1 - alfa_bar_t) * eps,   eps ~ N(0, I)

ISBOT G'OYASI (ikki qadam):
  x_2 = sqrt(a_2) * (sqrt(a_1) x_0 + sqrt(1 - a_1) e_1) + sqrt(1 - a_2) e_2
  mustaqil Gausslar yig'indisi - Gauss, dispersiyalar qo'shiladi:
    a_2 (1 - a_1) + (1 - a_2) = 1 - a_1 a_2
  => x_2 = sqrt(a_1 a_2) x_0 + sqrt(1 - a_1 a_2) eps

NEGA MUHIM:
  o'rgatishda t ta qadamni simulyatsiya qilish SHART EMAS
  tasodifiy t tanlanadi va x_t bir zumda hisoblanadi
  1-misol: qadamma-qadam va yopiq formula bir xil statistika beradi
    corr(x_50, x_0): 0.567 (qadamma-qadam), 0.571 (yopiq), 0.573 (nazariya)
  oxirida: x_100 o'rtachasi -0.002, std 1.002 - N(0, 1)

SIGNAL/SHOVQIN NISBATI:
  SNR(t) = alfa_bar_t / (1 - alfa_bar_t)
  signal: sqrt(alfa_bar_t) * x_0,  shovqin: sqrt(1 - alfa_bar_t) * eps
  log10 SNR > 0 - signal ustun;  log10 SNR < -2 - deyarli toza shovqin

CHIZIQLI JADVAL (DDPM, Ho va b., 2020):
  beta_t: 0.0001 dan 0.02 gacha chiziqli, T = 1000
  kichik T uchun masshtab: beta_t in [0.1 / T, 20 / T]

KOSINUS JADVAL (Nichol va Dhariwal, 2021):
  alfa_bar_t = f(t) / f(0),  f(t) = cos^2( ((t/T + s) / (1 + s)) * pi/2 ),  s = 0.008
  beta_t = 1 - alfa_bar_t / alfa_bar_{t-1},  0.999 bilan chegaralanadi

1-MISOL (T = 1000):
  t/T    chiziqli alfa_bar  log10 SNR    kosinus alfa_bar  log10 SNR
  0.25        0.5214          0.04           0.8459          0.74
  0.5         0.0778         -1.07           0.4923         -0.01
  0.75        0.0033         -2.48           0.1432         -0.78
  SNR < 0.01: chiziqli t/T = 0.67 dan, kosinus - 0.94 dan

NIMA UCHUN MUHIM:
  chiziqli jadvalda qadamlarning ~1/3 qismi deyarli toza shovqin ustida -
    tarmoq bu yerda o'rganadigan narsa kam, sampling da ham bu qadamlar "bo'sh"
  kichik rasmlarda (8x8, 32x32) bu ayniqsa sezilarli:
    digits da klassifikator x_t ni t/T = 0.6 dan (chiziqli) va
    0.9 dan (kosinus) boshlab deyarli tanimaydi (aniqlik < 0.2)
  kosinus: SNR log-shkalada tekisroq kamayadi - qadamlar teng "foydali"

MAQSAD:
  eps_theta(x_t, t) - tarmoq, x_t ga QAYSI shovqin qo'shilganini bashorat qiladi

O'RGATISH ALGORITMI (bitta qadam):
  x_0 ~ ma'lumot,  t ~ U{0, ..., T-1},  eps ~ N(0, I)
  x_t = sqrt(alfa_bar_t) * x_0 + sqrt(1 - alfa_bar_t) * eps
  loss = || eps - eps_theta(x_t, t) ||^2          (MSE)

NEGA AYNAN SHOVQIN:
  teskari qadam o'rtachasi:
    mu(x_t, x_0) = (1 / sqrt(alfa_t)) * ( x_t - beta_t / sqrt(1 - alfa_bar_t) * eps )
  eps ni bilsak - mu ni bilamiz; x_0 = (x_t - sqrt(1 - alfa_bar_t) eps) / sqrt(alfa_bar_t)
  Ho va b. (2020): ELBO ning soddalashtirilgan ko'rinishi - vaznlarsiz MSE -
    amalda eng yaxshi namunalar beradi
  eps ning masshtabi har t da bir xil (N(0, I)) - regressiya nishoni barqaror

BOSHQA VARIANTLAR (qisqa):
  x_0-prediction - x_0 ni to'g'ridan-to'g'ri bashorat
  v-prediction   - v = sqrt(alfa_bar) eps - sqrt(1 - alfa_bar) x_0
  hammasi o'zaro almashtiriladi; farq - loss vaznlari va barqarorlik

LOSS QIYMATI HAQIDA:
  eps loss turli T lar orasida TAQQOSLANMAYDI (t taqsimoti boshqa)
  4-misol: T=10 da loss 0.2896, T=100 da 0.3247 - lekin T=10 namunalari yomonroq

ALGORITM (Ho va b., 2020):
  x_T ~ N(0, I)
  t = T-1, ..., 0 uchun:
    eps_hat = eps_theta(x_t, t)
    x_{t-1} = (1 / sqrt(alfa_t)) * ( x_t - beta_t / sqrt(1 - alfa_bar_t) * eps_hat ) + sigma_t * z
    z ~ N(0, I) (t > 0 da), sigma_t^2 = beta_t
  natija: x_0

NARX:
  T ta tarmoq chaqiruvi BITTA namuna uchun (GAN da - 1 ta)
  2-misol: T = 100 -> 100 marta forward; tezlashtirish (DDIM, kam qadamli
  sampling) - 26.6-dars

X_0 NI KESISH (clip_denoised):
  har qadamda x_0 bahosi:  x0_hat = (x_t - sqrt(1 - alfa_bar_t) eps_hat) / sqrt(alfa_bar_t)
  x0_hat ni ma'lumot oralig'iga kesish: [-1, 1]
  so'ng posterior q(x_{t-1} | x_t, x0_hat) o'rtachasi:
    mu = ( beta_t sqrt(alfa_bar_{t-1}) x0_hat + (1 - alfa_bar_{t-1}) sqrt(alfa_t) x_t ) / (1 - alfa_bar_t)
    sigma_t^2 = beta_t (1 - alfa_bar_{t-1}) / (1 - alfa_bar_t)
  NEGA KERAK: kosinus jadvalda oxirgi beta ~ 0.999 -> 1 / sqrt(alfa_t) ~ 32
    eps_hat dagi kichik xato 32 barobar kattalashadi
  3-misol (digits): kesishsiz piksellarning 0.93 qismi -1 yoki +1 ga "yopishdi",
    o'quv rasmigacha median masofa 7.31 (test rasmlarda 2.13);
    kesish bilan - 2.94

NEGA TARMOQ t NI BILISHI KERAK:
  bir xil x_t turli t larda turli shovqinni anglatadi:
    t kichik - x_t deyarli x_0, shovqin ulushi kichik: sqrt(1 - alfa_bar) = 0.105 (t = 5)
    t katta  - x_t deyarli shovqin: sqrt(1 - alfa_bar) = 0.998 (t = 95)
  t siz tarmoq barcha t lar uchun BITTA o'rtacha javob beradi
  4-misol: t siz model 0 ta moda (4 urug'da), median masofa +0.242 (sezilarli)

SINUSOIDAL EMBEDDING (24-qismdagi pozitsion kodlash kabi):
  emb(t) = [sin(t * w_1), ..., sin(t * w_k), cos(t * w_1), ..., cos(t * w_k)]
  w_i - geometrik ketma-ketlik (past va yuqori chastotalar)
  kirishga ulanadi (2D misol) yoki har yashirin qatlamga qo'shiladi (digits)

QADAMLAR SONI T:
  katta T - har qadam kichik, teskari qadam "Gaussga yaqin" taxmini aniqroq
  kichik T - arzon sampling, lekin har qadam katta, xato yig'iladi
  4-misol (MLP 3 x 64): T=10 va T=100 - sifatli ulush bir xil (-0.003, SE 0.009),
    lekin median masofa T=10 da sezilarli katta (+0.039, SE 0.013)
  kichik modelda asosiy cheklov - tarmoq sig'imi; T ni oshirish uni bartaraf qilmaydi

NEGA DIFFUSION MODE COLLAPSE GA MOYIL EMAS:
  loss har bir o'quv namunasi uchun: "shu x_0 dan yasalgan x_t dagi shovqinni top"
  hech bir o'quv namunasi "e'tibordan chetda" qolmaydi - MSE hammasini jazolaydi
  GAN da G ning maqsadi - D ni aldash; qamrov haqida hech qanday had yo'q
  (bu maksimal o'xshashlik oilasiga xos xususiyat - 26.1-dars)

2-MISOL (8 ta Gauss, 4 urug', bir xil o'lchovlar 26.4 dagi kabi):
  o'lchov         GAN (4 urug')         DDPM (4 urug')        farq, SE
  modalar         8, 5, 7, 6            8, 8, 8, 8            +1.500 0.645-bob *
  sifatli ulush   0.47 0.15 0.50 0.36   0.32 0.49 0.46 0.43   +0.058 0.104-bob
  median masofa   0.16 0.26 0.15 0.20   0.20 0.15 0.16 0.17   -0.022 0.031-bob
  muvozanat       0.79 - 0.90           0.98 - 1.00           +0.156 0.023-bob *

NARX:
  GAN: 1000 qadam o'rgatish, sampling 1 forward
  DDPM: 2000 qadam x 256, sampling 100 forward
  diffusion ning qamrovi bepul emas - u sampling vaqti bilan to'lanadi

DIGITS DA (3-misol va 26.4, 4-misol):
  diffusion: klassifikator ishonchi 0.828, 10 sinf >= 5%, entropiya 0.945
  shartsiz GAN 26.4-bob: ishonchi 0.833 - 0.878, 9-10 sinf, entropiya 0.956 - 0.989
  bu kichik byudjetda diffusion digits da GAN dan YAXSHI EMAS

BU DARSDA (KICHIK):
  2D: MLP [x (2), emb(t) (16)] -> 3 x 128 (SiLU) -> 2
  digits: MLP 64 -> 256, 3 qatlam, qoldiq ulanish, emb(t) har qatlamga qo'shiladi

AMALDA - U-NET (22.13-dars g'oyasi):
  kirish va chiqish bir xil o'lchamli (shovqin rasm bilan bir xil shaklda)
  encoder-decoder + skip ulanishlar: ham global shakl, ham mayda detallar
  vaqt embeddingi har rezidual blokka (masshtab va siljish sifatida)
  past o'lchamlarda self-attention (24-qism)
  Stable Diffusion: U-Net rasmning o'zida emas, VAE ning yashirin fazosida
    ishlaydi (26.2 dagi VAE g'oyasi) - "latent diffusion"
  yangi modellarda U-Net o'rniga Transformer (DiT) ham ishlatiladi

import math

import torch
import torch.nn.functional as F


def kosinus_jadval(T, s=0.008):
    f = torch.cos((torch.arange(T + 1) / T + s) / (1 + s) * math.pi / 2) ** 2
    ab = f[1:] / f[0]
    return (1 - ab / torch.cat([torch.ones(1), ab[:-1]])).clamp(max=0.999)


beta = kosinus_jadval(100)
alfa = 1 - beta
ab = torch.cumprod(alfa, 0)                        # alfa_bar
snr = ab / (1 - ab)                                # SNR(t)

# o'rgatish qadami (epsilon-prediction)
t = torch.randint(0, 100, (B,), generator=g)
eps = torch.randn(x0.shape, generator=g)
a = ab[t][:, None]
xt = a.sqrt() * x0 + (1 - a).sqrt() * eps          # q(x_t | x_0)
loss = F.mse_loss(model(xt, t), eps)

# DDPM sampling
x = torch.randn(n, d, generator=g)
for t in range(99, -1, -1):
    eps_hat = model(x, torch.full((n,), t))
    x = (x - beta[t] / (1 - ab[t]).sqrt() * eps_hat) / alfa[t].sqrt()
    if t > 0:
        x = x + beta[t].sqrt() * torch.randn(x.shape, generator=g)

# x0 bahosi va kesish (rasm uchun)
x0_hat = ((x - (1 - ab[t]).sqrt() * eps_hat) / ab[t].sqrt()).clamp(-1, 1)

# sinusoidal vaqt embeddingi
w = torch.exp(-math.log(1000.0) * torch.arange(8) / 8)
emb = torch.cat([torch.sin(t[:, None] * w), torch.cos(t[:, None] * w)], 1)

from diffusers import DDPMScheduler

sched = DDPMScheduler(num_train_timesteps=1000, beta_schedule="squaredcos_cap_v2")
noisy = sched.add_noise(x0, eps, t)                # q(x_t | x_0)
sched.set_timesteps(100)
for t in sched.timesteps:
    eps_hat = unet(x, t).sample
    x = sched.step(eps_hat, t, x).prev_sample

forward: x_t = sqrt(alfa_bar_t) x_0 + sqrt(1 - alfa_bar_t) eps - o'rganilmaydi
SNR(t) = alfa_bar / (1 - alfa_bar); kosinus jadval - tekisroq, chiziqli - erta shovqin
loss: MSE(eps, eps_theta(x_t, t)) - oddiy regressiya
sampling: T ta qadam, har birida eps_hat -> o'rtacha + shovqin
rasm uchun x0_hat ni [-1, 1] ga kesing
t embeddingi majburiy; T - sifat va narx murosasi
qamrov: GAN dan yaxshi (8/8 moda), narx: T barobar ko'p forward

"""Forward jarayon: beta jadvallari (chiziqli va kosinus), SNR(t), yopiq formula va digits da shovqin."""

import math

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split

BELGI = " .:-=+*#%@"


def chiziqli_jadval(T):
    """DDPM (Ho va b., 2020): T = 1000 da beta 1e-4 dan 0.02 gacha; boshqa T ga masshtablanadi."""
    return torch.linspace(0.1 / T, 20.0 / T, T, dtype=torch.float64)


def kosinus_jadval(T, s=0.008):
    """alfa_bar(t) = cos^2(((t/T + s) / (1 + s)) * pi/2) - Nichol va Dhariwal (2021)."""
    f = torch.cos((torch.arange(T + 1, dtype=torch.float64) / T + s) / (1 + s) * math.pi / 2) ** 2
    ab = f[1:] / f[0]
    return (1 - ab / torch.cat([torch.ones(1, dtype=torch.float64), ab[:-1]])).clamp(max=0.999)


def alfa_bar(beta):
    return torch.cumprod(1 - beta, 0)


def klassifikator(X, y):
    torch.manual_seed(0)
    m = nn.Sequential(nn.Linear(64, 128), nn.ReLU(), nn.Linear(128, 10))
    opt = torch.optim.Adam(m.parameters(), lr=0.003, weight_decay=1e-4)
    for _ in range(300):
        loss = F.cross_entropy(m(X), y)
        opt.zero_grad()
        loss.backward()
        opt.step()
    return m.eval()


def ascii_qator(rasmlar):
    q = np.clip(((rasmlar + 1) / 2 * 9.99).astype(int), 0, 9).reshape(-1, 8, 8)
    return ["  " + "  ".join("".join(BELGI[v] for v in r[i]) for r in q) for i in range(8)]


def main() -> None:
    torch.set_num_threads(1)
    print("=== 1. Beta jadvallari: alfa_bar(t) va SNR(t) = alfa_bar / (1 - alfa_bar) ===")
    for T in (1000, 100):
        jad = {"chiziqli": alfa_bar(chiziqli_jadval(T)), "kosinus": alfa_bar(kosinus_jadval(T))}
        print(f"  T = {T}")
        print("    t/T     chiziqli: alfa_bar   log10 SNR     kosinus: alfa_bar   log10 SNR")
        for u in (0.0, 0.1, 0.25, 0.5, 0.75, 0.9, 1.0):
            t = min(T - 1, int(round(u * T)))
            qator = f"    {u:<5}"
            for nom in ("chiziqli", "kosinus"):
                ab = jad[nom][t].item()
                qator += f" {ab:>19.4f} {math.log10(ab / (1 - ab)):>11.2f}   "
            print(qator)
        for nom, ab in jad.items():
            snr = ab / (1 - ab)
            t1 = int((snr < 0.01).nonzero()[0]) if (snr < 0.01).any() else T
            print(f"    {nom}: SNR < 0.01 (deyarli toza shovqin) t/T = {t1 / T:.2f} dan boshlab")

    print("\n=== 2. Yopiq formula q(x_t | x_0) va qadamma-qadam shovqin (2D, T = 100) ===")
    T = 100
    beta = kosinus_jadval(T).float()
    ab = alfa_bar(beta)
    g = torch.Generator().manual_seed(0)
    burchak = torch.randint(0, 8, (20000,), generator=g) * math.pi / 4
    x0 = torch.stack([burchak.cos(), burchak.sin()], 1) + 0.025 * torch.randn(20000, 2, generator=g)
    x = x0.clone()
    print("    t   alfa_bar  corr(x_t, x_0): qadamma-qadam  yopiq formula  nazariya")
    for t in range(T):
        x = (1 - beta[t]).sqrt() * x + beta[t].sqrt() * torch.randn(20000, 2, generator=g)
        if t in (9, 29, 49, 69, 89, 99):
            xy = ab[t].sqrt() * x0 + (1 - ab[t]).sqrt() * torch.randn(20000, 2, generator=g)
            c1 = np.corrcoef(x[:, 0], x0[:, 0])[0, 1]
            c2 = np.corrcoef(xy[:, 0], x0[:, 0])[0, 1]
            s2 = x0[:, 0].var().item()
            naz = math.sqrt(ab[t] * s2 / (ab[t] * s2 + 1 - ab[t]))
            print(f"  {t + 1:>3} {ab[t].item():>10.4f} {c1:>24.3f} {c2:>14.3f} {naz:>9.3f}")
    print(f"  oxirida: x_T o'rtachasi {x.mean().item():+.3f}, std {x.std().item():.3f} (N(0, 1) ga yaqin)")

    print("\n=== 3. Digits: toza rasmlarda o'rgatilgan klassifikator x_t ni taniydimi? ===")
    d = load_digits()
    X = torch.tensor(d.data / 8.0 - 1.0, dtype=torch.float32)
    Xo, Xt, yo, yt = train_test_split(X, torch.tensor(d.target), test_size=0.3, random_state=0,
                                      stratify=d.target)
    clf = klassifikator(Xo, yo)
    print("    t/T    chiziqli: alfa_bar  aniqlik     kosinus: alfa_bar  aniqlik")
    chegara = {}
    for u in (0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9):
        t = int(u * T)
        qator = f"    {u:<5}"
        for nom, b in (("chiziqli", chiziqli_jadval(T).float()), ("kosinus", beta)):
            a = alfa_bar(b)[t]
            eps = torch.randn(Xt.shape, generator=torch.Generator().manual_seed(t))
            with torch.no_grad():
                acc = (clf(a.sqrt() * Xt + (1 - a).sqrt() * eps).argmax(1) == yt).float().mean().item()
            if acc < 0.2 and nom not in chegara:
                chegara[nom] = u
            qator += f" {a.item():>18.3f} {acc:>8.3f}   "
        print(qator)
    for nom in ("chiziqli", "kosinus"):
        print(f"    {nom}: aniqlik < 0.2 (raqam deyarli tanib bo'lmaydi) t/T = {chegara.get(nom, 1.0)} dan")

    print("\n=== 4. Bitta raqam (test[0]) shovqin ostida: t = 0, 20, 40, 60, 80 ===")
    for nom, b in (("chiziqli", chiziqli_jadval(T).float()), ("kosinus", beta)):
        a = alfa_bar(b)
        eps = torch.randn(64, generator=torch.Generator().manual_seed(1))
        rasmlar = [Xt[0]] + [a[t].sqrt() * Xt[0] + (1 - a[t]).sqrt() * eps for t in (19, 39, 59, 79)]
        print(f"  {nom} (haqiqiy sinf {yt[0].item()}):")
        for qator in ascii_qator(torch.stack(rasmlar).numpy()):
            print(qator)


if __name__ == "__main__":
    main()

=== 1. Beta jadvallari: alfa_bar(t) va SNR(t) = alfa_bar / (1 - alfa_bar) ===
  T = 1000
    t/T     chiziqli: alfa_bar   log10 SNR     kosinus: alfa_bar   log10 SNR
    0.0                0.9999        4.00                 1.0000        4.38
    0.1                0.8951        0.93                 0.9716        1.53
    0.25               0.5214        0.04                 0.8459        0.74
    0.5                0.0778       -1.07                 0.4923       -0.01
    0.75               0.0033       -2.48                 0.1432       -0.78
    0.9                0.0003       -3.57                 0.0236       -1.62
    1.0                0.0000       -4.39                 0.0000       -8.61
  ...

=== 2. Yopiq formula q(x_t | x_0) va qadamma-qadam shovqin (2D, T = 100) ===
    t   alfa_bar  corr(x_t, x_0): qadamma-qadam  yopiq formula  nazariya
   10     0.9721                    0.973          0.972     0.973
   30     0.7869                    0.804          0.804     0.806
   50     0.4938                    0.567          0.571     0.573
   70     0.2031                    0.334          0.344     0.337
   90     0.0241                    0.109          0.119     0.111
  100     0.0000                   -0.010         -0.009     0.000
  oxirida: x_T o'rtachasi -0.002, std 1.002 (N(0, 1) ga yaqin)

=== 3. Digits: toza rasmlarda o'rgatilgan klassifikator x_t ni taniydimi? ===
    t/T    chiziqli: alfa_bar  aniqlik     kosinus: alfa_bar  aniqlik
    0.0                0.999    0.974                 0.999    0.974
    0.1                0.885    0.943                 0.967    0.970
    0.2                0.638    0.759                 0.889    0.924
    0.3                0.373    0.533                 0.774    0.867
    0.4                0.176    0.330                 0.633    0.756
    0.5                0.067    0.250                 0.478    0.607
    0.6                0.020    0.143                 0.326    0.470
    0.7                0.005    0.109                 0.191    0.311
  ...

=== 4. Bitta raqam (test[0]) shovqin ostida: t = 0, 20, 40, 60, 80 ===
  chiziqli (haqiqiy sinf 1):
      *%.       **        +-      . =:     .. =:
     .@@.    . .%@::   = :*@==   + -=@+*  .* -=@*#
     *@%      :#@% -    +*@# *    #*@* %  . %*@*.%
    .@@@    =  =@*    #:  #= :  @-  *: -  @-  +: =
    %@@%    *.#*@#.   @:#-%*-   @-*.*+=.  @=+.*++:
   -@+@%     =%+@=+   :+*+@.@   -*==@ @.  =*==@ @.
      #@     :. @@=*   +- @@%@   *=:@#@@   #+:%#@@
      +%    .-  @+    -#. @:    =%-.@.    +@-.@  .
  ...

"""DDPM noldan: 8 ta Gauss halqasida kichik MLP, GAN bilan bir xil o'lchovlarda juftlashgan taqqoslash."""

import math

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F

MARKAZ = torch.tensor([[2 * math.cos(k * math.pi / 4), 2 * math.sin(k * math.pi / 4)]
                       for k in range(8)])
STD = 0.05
S = 4
B, H = 256, 128
T = 100


class BLinear(nn.Module):
    """S ta mustaqil Linear qatlam: x (S, B, a) -> (S, B, b)."""

    def __init__(self, a, b):
        super().__init__()
        k = 1 / math.sqrt(a)
        self.w = nn.Parameter(torch.empty(S, a, b).uniform_(-k, k))
        self.b = nn.Parameter(torch.empty(S, 1, b).uniform_(-k, k))

    def forward(self, x):
        return torch.baddbmm(self.b, x, self.w)


def haqiqiy(n, g):
    k = torch.randint(0, 8, (S, n), generator=g)
    return MARKAZ[k] + STD * torch.randn(S, n, 2, generator=g)


def olcha(x):
    """x (S, N, 2) -> (S, 4): modalar soni, sifatli ulush, median masofa, muvozanat."""
    d = torch.cdist(x, MARKAZ.expand(S, 8, 2))
    dmin, eng = d.min(2)
    sifatli = dmin < 3 * STD
    natija = []
    for s in range(S):
        soni = torch.bincount(eng[s][sifatli[s]], minlength=8).float()
        p = soni / soni.sum().clamp_min(1)
        ent = -(p[p > 0] * p[p > 0].log()).sum().item() / math.log(8)
        natija.append([(soni >= 25).sum().item(), sifatli[s].float().mean().item(),
                       dmin[s].median().item(), ent])
    return np.array(natija)


def kosinus_jadval(T, s=0.008):
    """alfa_bar(t) = cos^2(((t/T + s) / (1 + s)) * pi/2) - Nichol va Dhariwal (2021)."""
    f = torch.cos((torch.arange(T + 1) / T + s) / (1 + s) * math.pi / 2) ** 2
    ab = f[1:] / f[0]
    beta = (1 - ab / torch.cat([torch.ones(1), ab[:-1]])).clamp(max=0.999)
    return beta


def vaqt_embedding(t, T, d=16):
    """Sinusoidal embedding: t (S, N) butun son -> (S, N, d)."""
    chastota = torch.exp(-math.log(1000.0) * torch.arange(d // 2) / (d // 2))
    burchak = (t.float() / T * 1000.0)[..., None] * chastota * 0.1
    return torch.cat([burchak.sin(), burchak.cos()], -1)


class Denoiser(nn.Module):
    """eps_theta(x_t, t): [x_t, emb(t)] -> bashorat qilingan shovqin."""

    def __init__(self):
        super().__init__()
        self.tarmoq = nn.Sequential(BLinear(2 + 16, H), nn.SiLU(), BLinear(H, H), nn.SiLU(),
                                    BLinear(H, H), nn.SiLU(), BLinear(H, 2))

    def forward(self, x, t):
        return self.tarmoq(torch.cat([x, vaqt_embedding(t, T)], -1))


def ddpm_orgat(qadamlar=2000):
    torch.manual_seed(0)
    model = Denoiser()
    opt = torch.optim.Adam(model.parameters(), lr=2e-3)
    beta = kosinus_jadval(T)
    ab = torch.cumprod(1 - beta, 0)
    g = torch.Generator().manual_seed(1)
    for _ in range(qadamlar):
        x0 = haqiqiy(B, g) / 2                                # masshtab: radius 1
        t = torch.randint(0, T, (S, B), generator=g)
        eps = torch.randn(S, B, 2, generator=g)
        a = ab[t][..., None]
        xt = a.sqrt() * x0 + (1 - a).sqrt() * eps            # q(x_t | x_0) yopiq ko'rinishda
        loss = F.mse_loss(model(xt, t), eps)
        opt.zero_grad()
        loss.backward()
        opt.step()
    return model, beta


@torch.no_grad()
def ddpm_namuna(model, beta, n, g):
    alfa = 1 - beta
    ab = torch.cumprod(alfa, 0)
    x = torch.randn(S, n, 2, generator=g)                      # x_T ~ N(0, I)
    for t in range(T - 1, -1, -1):
        eps = model(x, torch.full((S, n), t))
        x = (x - beta[t] / (1 - ab[t]).sqrt() * eps) / alfa[t].sqrt()
        if t > 0:
            x = x + beta[t].sqrt() * torch.randn(S, n, 2, generator=g)
    return x * 2


def gan_orgat(qadamlar=1000):
    """26.4-darsdagi oddiy GAN: lr 0.002, beta1 0.5, to'yinmaydigan loss."""
    torch.manual_seed(0)
    G = nn.Sequential(BLinear(2, 64), nn.ReLU(), BLinear(64, 64), nn.ReLU(), BLinear(64, 2))
    D = nn.Sequential(BLinear(2, 64), nn.ReLU(), BLinear(64, 64), nn.ReLU(), BLinear(64, 1))
    oG = torch.optim.Adam(G.parameters(), lr=0.002, betas=(0.5, 0.999))
    oD = torch.optim.Adam(D.parameters(), lr=0.002, betas=(0.5, 0.999))
    g = torch.Generator().manual_seed(1)
    for _ in range(qadamlar):
        x = haqiqiy(128, g)
        f = G(torch.randn(S, 128, 2, generator=g))
        ld = F.softplus(-D(x)).mean() + F.softplus(D(f.detach())).mean()
        oD.zero_grad()
        ld.backward()
        oD.step()
        lg = F.softplus(-D(f)).mean()
        oG.zero_grad()
        lg.backward()
        oG.step()
    return G


def main() -> None:
    torch.set_num_threads(1)
    print("=== 1. Model va byudjet ===")
    model, beta = ddpm_orgat()
    n_par = sum(p.numel() for p in model.parameters()) // S
    print(f"  DDPM: T = {T}, kosinus jadval, MLP 3 x {H} + vaqt embeddingi (16), "
          f"{n_par} parametr (bitta urug')")
    print(f"  o'rgatish: 2000 qadam x {B}; sampling: {T} qadam (har namuna {T} marta forward)")
    G = gan_orgat()
    print(f"  GAN: 26.4 dagi oddiy GAN, 1000 qadam; sampling - 1 forward")

    print("\n=== 2. Namunalar sifati va qamrovi (2000 namuna, 4 urug') ===")
    x = ddpm_namuna(model, beta, 2000, torch.Generator().manual_seed(5))
    dd = olcha(x)
    with torch.no_grad():
        ga = olcha(G(torch.randn(S, 2000, 2, generator=torch.Generator().manual_seed(5))))
    hq = olcha(haqiqiy(2000, torch.Generator().manual_seed(6)))
    nomlar = ["modalar", "sifatli", "median masofa", "muvozanat"]
    print("  o'lchov        haqiqiy    GAN (urug'lar)            DDPM (urug'lar)")
    for j, nom in enumerate(nomlar):
        print(f"  {nom:<14} {hq[:, j].mean():>6.3f}   " + " ".join(f"{v:5.2f}" for v in ga[:, j])
              + "    " + " ".join(f"{v:5.2f}" for v in dd[:, j]))

    print("\n=== 3. Juftlashgan farq: DDPM - GAN ===")
    for j, nom in enumerate(nomlar):
        farq = dd[:, j] - ga[:, j]
        se = farq.std(ddof=1) / math.sqrt(S)
        belgi = "sezilarli" if abs(farq.mean()) > 2 * se else "sezilarli emas"
        print(f"  {nom:<14} {farq.mean():+.3f} (SE {se:.3f}) -> {belgi}")

    print("\n=== 4. Qayerda xato: radius yoki modalar orasi? ===")
    r = x.norm(dim=2)
    burchak = torch.atan2(x[..., 1], x[..., 0]) / (math.pi / 4)
    oraliq = (burchak - burchak.round()).abs()             # 0 - moda ustida, 0.5 - o'rtada
    print(f"  radius: o'rtacha {r.mean().item():.3f} (haqiqiy 2), "
          f"|r - 2| > 0.15 ulushi {((r - 2).abs() > 0.15).float().mean().item():.3f}")
    radial = ((r - 2).abs() > 0.15).float().mean().item()
    oraliq_ulush = (oraliq > 0.25).float().mean().item()
    print(f"  burchak bo'yicha modalar orasida (oraliq > 0.25) ulushi {oraliq_ulush:.3f}")
    if radial > oraliq_ulush:
        print("  xatolar ko'proq radius bo'yicha: namunalar halqa atrofida 'yoyilgan'")
    else:
        print("  xatolar ko'proq modalar orasida: 'ko'priklar'")
    if dd[:, 0].min() == 8:
        print("  DDPM 4 urug'ning hammasida 8/8 modani qamradi - mode collapse yo'q")


if __name__ == "__main__":
    main()

=== 1. Model va byudjet ===
  DDPM: T = 100, kosinus jadval, MLP 3 x 128 + vaqt embeddingi (16), 35714 parametr (bitta urug')
  o'rgatish: 2000 qadam x 256; sampling: 100 qadam (har namuna 100 marta forward)
  GAN: 26.4 dagi oddiy GAN, 1000 qadam; sampling - 1 forward

=== 2. Namunalar sifati va qamrovi (2000 namuna, 4 urug') ===
  o'lchov        haqiqiy    GAN (urug'lar)            DDPM (urug'lar)
  modalar         8.000    8.00  5.00  7.00  6.00     8.00  8.00  8.00  8.00
  sifatli         0.989    0.47  0.15  0.50  0.36     0.32  0.49  0.46  0.43
  median masofa   0.059    0.16  0.26  0.15  0.20     0.20  0.15  0.16  0.17
  muvozanat       0.999    0.90  0.80  0.85  0.79     0.99  0.99  1.00  0.98

=== 3. Juftlashgan farq: DDPM - GAN ===
  modalar        +1.500 (SE 0.645) -> sezilarli
  sifatli        +0.058 (SE 0.104) -> sezilarli emas
  median masofa  -0.022 (SE 0.031) -> sezilarli emas
  muvozanat      +0.156 (SE 0.023) -> sezilarli

=== 4. Qayerda xato: radius yoki modalar orasi? ===
  radius: o'rtacha 1.990 (haqiqiy 2), |r - 2| > 0.15 ulushi 0.228
  burchak bo'yicha modalar orasida (oraliq > 0.25) ulushi 0.105
  xatolar ko'proq radius bo'yicha: namunalar halqa atrofida 'yoyilgan'
  DDPM 4 urug'ning hammasida 8/8 modani qamradi - mode collapse yo'q

"""Digits 8x8 da kichik diffusion model: tanib olinish, sinf qamrovi va yodlab olish tekshiruvi."""

import math

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split

T = 100
BELGI = " .:-=+*#%@"


def kosinus_jadval(T, s=0.008):
    f = torch.cos((torch.arange(T + 1) / T + s) / (1 + s) * math.pi / 2) ** 2
    ab = f[1:] / f[0]
    return (1 - ab / torch.cat([torch.ones(1), ab[:-1]])).clamp(max=0.999)


def vaqt_embedding(t, d=32):
    chastota = torch.exp(-math.log(1000.0) * torch.arange(d // 2) / (d // 2))
    burchak = (t.float() * 10.0)[:, None] * chastota
    return torch.cat([burchak.sin(), burchak.cos()], 1)


class Denoiser(nn.Module):
    """eps_theta(x_t, t): MLP, vaqt embeddingi har yashirin qatlamga qo'shiladi."""

    def __init__(self, h=256):
        super().__init__()
        self.kirish = nn.Linear(64, h)
        self.vaqt = nn.Sequential(nn.Linear(32, h), nn.SiLU(), nn.Linear(h, h))
        self.bloklar = nn.ModuleList([nn.Linear(h, h) for _ in range(2)])
        self.chiqish = nn.Linear(h, 64)

    def forward(self, x, t):
        e = self.vaqt(vaqt_embedding(t))
        h = F.silu(self.kirish(x) + e)
        for blok in self.bloklar:
            h = h + F.silu(blok(h) + e)          # qoldiq ulanish + vaqt
        return self.chiqish(h)


def klassifikator(X, y):
    torch.manual_seed(0)
    m = nn.Sequential(nn.Linear(64, 128), nn.ReLU(), nn.Linear(128, 10))
    opt = torch.optim.Adam(m.parameters(), lr=0.003, weight_decay=1e-4)
    for _ in range(300):
        loss = F.cross_entropy(m(X), y)
        opt.zero_grad()
        loss.backward()
        opt.step()
    return m.eval()


def orgat(X, qadamlar=2000, B=128):
    torch.manual_seed(0)
    model = Denoiser()
    opt = torch.optim.Adam(model.parameters(), lr=1e-3)
    ab = torch.cumprod(1 - kosinus_jadval(T), 0)
    g = torch.Generator().manual_seed(1)
    tarix = []
    for q in range(qadamlar):
        x0 = X[torch.randint(0, len(X), (B,), generator=g)]
        t = torch.randint(0, T, (B,), generator=g)
        eps = torch.randn(x0.shape, generator=g)
        a = ab[t][:, None]
        loss = F.mse_loss(model(a.sqrt() * x0 + (1 - a).sqrt() * eps, t), eps)
        opt.zero_grad()
        loss.backward()
        opt.step()
        tarix.append(loss.item())
    return model, tarix


@torch.no_grad()
def namuna(model, n, g, kesish=True):
    """DDPM sampling. kesish=True: har qadamda x0 bahosini [-1, 1] ga kesib,
    q(x_{t-1} | x_t, x0) posterior o'rtachasidan foydalanamiz."""
    beta = kosinus_jadval(T)
    alfa, ab = 1 - beta, torch.cumprod(1 - beta, 0)
    ab_old = torch.cat([torch.ones(1), ab[:-1]])
    x = torch.randn(n, 64, generator=g)
    for t in range(T - 1, -1, -1):
        eps = model(x, torch.full((n,), t))
        if kesish:
            x0 = ((x - (1 - ab[t]).sqrt() * eps) / ab[t].sqrt()).clamp(-1, 1)
            x = (beta[t] * ab_old[t].sqrt() * x0 + (1 - ab_old[t]) * alfa[t].sqrt() * x) / (1 - ab[t])
            sigma = (beta[t] * (1 - ab_old[t]) / (1 - ab[t])).sqrt()
        else:
            x = (x - beta[t] / (1 - ab[t]).sqrt() * eps) / alfa[t].sqrt()
            sigma = beta[t].sqrt()
        if t > 0:
            x = x + sigma * torch.randn(x.shape, generator=g)
    return x.clamp(-1, 1)


def baho(clf, x):
    with torch.no_grad():
        p = torch.softmax(clf(x), 1)
    ishonch, sinf = p.max(1)
    ulush = torch.bincount(sinf, minlength=10).float() / len(x)
    ent = -(ulush[ulush > 0] * ulush[ulush > 0].log()).sum().item() / math.log(10)
    return ishonch, sinf, ulush, ent


def ascii_qator(rasmlar):
    q = np.clip(((rasmlar + 1) / 2 * 9.99).astype(int), 0, 9).reshape(-1, 8, 8)
    return ["  " + "  ".join("".join(BELGI[v] for v in r[i]) for r in q) for i in range(8)]


def main() -> None:
    torch.set_num_threads(1)
    d = load_digits()
    X = torch.tensor(d.data / 8.0 - 1.0, dtype=torch.float32)
    Xo, Xt, yo, yt = train_test_split(X, torch.tensor(d.target), test_size=0.3, random_state=0,
                                      stratify=d.target)
    clf = klassifikator(Xo, yo)

    print("=== 1. Model va o'rgatish ===")
    model, tarix = orgat(Xo)
    print(f"  denoiser: MLP 3 x 256, vaqt embeddingi har qatlamga; "
          f"{sum(p.numel() for p in model.parameters())} parametr")
    print(f"  T = {T} (kosinus), 2000 qadam x 128, o'quv rasmlari {len(Xo)}")
    for a, b in ((0, 100), (900, 1000), (1900, 2000)):
        print(f"  loss (eps MSE) {a:>4}-{b:<4} qadam: {np.mean(tarix[a:b]):.4f}")

    print("\n=== 2. 1000 ta namuna: tanib olinish va sinf qamrovi ===")
    x = namuna(model, 1000, torch.Generator().manual_seed(5), kesish=True)
    x_k = namuna(model, 1000, torch.Generator().manual_seed(5), kesish=False)
    print("  manba              ishonch  ishonch>0.9  >=5% sinf  entropiya  o'quvgacha masofa")
    d_test = torch.cdist(Xt, Xo).min(1).values
    for nom, v in (("test rasm", Xt), ("x0 kesish bilan", x), ("kesishsiz", x_k)):
        ish, sinf, ulush, ent = baho(clf, v)
        dm = d_test if nom == "test rasm" else torch.cdist(v, Xo).min(1).values
        print(f"  {nom:<17} {ish.mean().item():>8.3f} {(ish > 0.9).float().mean().item():>12.3f} "
              f"{int((ulush >= 0.05).sum()):>10} {ent:>10.3f} {dm.median().item():>18.2f}")
    ish, sinf, ulush, ent = baho(clf, x)
    print("  x0 kesish bilan, sinflar ulushi (0..9): " + " ".join(f"{v:.2f}" for v in ulush.tolist()))
    tuyingan = (x_k.abs() > 0.99).float().mean().item()
    print(f"  kesishsiz: piksellarning {tuyingan:.2f} qismi -1 yoki +1 (test rasmlarda "
          f"{(Xt.abs() > 0.99).float().mean().item():.2f})")

    print("\n=== 3. Yodlab olish: eng yaqin o'quv namunasigacha masofa ===")
    d_gen = torch.cdist(x, Xo).min(1).values
    print(f"  test -> o'quv:      median {d_test.median().item():.2f}, "
          f"5-persentil {d_test.quantile(0.05).item():.2f}")
    print(f"  diffusion -> o'quv: median {d_gen.median().item():.2f}, "
          f"5-persentil {d_gen.quantile(0.05).item():.2f}")
    if d_gen.quantile(0.05) < 0.5 * d_test.quantile(0.05):
        print("  diqqat: generatsiyalarning bir qismi o'quv rasmlariga juda yaqin")
    else:
        print("  generatsiyalar o'quv rasmlariga testdagidan yaqin emas - nusxa ko'chirish belgisi yo'q")

    print("\n=== 4. Namunalar (x0 kesish bilan) ===")
    print("  birinchi 10 ta, klassifikator bo'yicha sinf va ishonch:")
    ish, sinf, _, _ = baho(clf, x[:10])
    print("  " + "".join(f"{s}:{i:.2f}".ljust(10) for s, i in zip(sinf.tolist(), ish.tolist())))
    for qator in ascii_qator(x[:10].numpy()):
        print(qator)
    print("  kesishsiz, xuddi shu shovqindan (birinchi 5 ta):")
    for qator in ascii_qator(x_k[:5].numpy()):
        print(qator)


if __name__ == "__main__":
    main()

=== 1. Model va o'rgatish ===
  denoiser: MLP 3 x 256, vaqt embeddingi har qatlamga; 238912 parametr
  T = 100 (kosinus), 2000 qadam x 128, o'quv rasmlari 1257
  loss (eps MSE)    0-100  qadam: 0.5216
  loss (eps MSE)  900-1000 qadam: 0.2500
  loss (eps MSE) 1900-2000 qadam: 0.2263

=== 2. 1000 ta namuna: tanib olinish va sinf qamrovi ===
  manba              ishonch  ishonch>0.9  >=5% sinf  entropiya  o'quvgacha masofa
  test rasm            0.979        0.939         10      0.999               2.13
  x0 kesish bilan      0.828        0.513         10      0.945               2.94
  kesishsiz            0.909        0.729          6      0.848               7.31
  x0 kesish bilan, sinflar ulushi (0..9): 0.05 0.06 0.15 0.18 0.08 0.08 0.05 0.06 0.20 0.08
  kesishsiz: piksellarning 0.93 qismi -1 yoki +1 (test rasmlarda 0.58)

=== 3. Yodlab olish: eng yaqin o'quv namunasigacha masofa ===
  test -> o'quv:      median 2.13, 5-persentil 1.45
  diffusion -> o'quv: median 2.94, 5-persentil 2.37
  generatsiyalar o'quv rasmlariga testdagidan yaqin emas - nusxa ko'chirish belgisi yo'q

=== 4. Namunalar (x0 kesish bilan) ===
  birinchi 10 ta, klassifikator bo'yicha sinf va ishonch:
  3:0.98    3:0.92    8:1.00    0:0.95    4:0.80    4:1.00    2:0.90    9:0.83    6:0.96    8:0.93
    =@@-.     *%#+.     .%#.      :%@:      .+-        =#      .=@#       =%=-.     .-@=      =@%.
    #-#@:    .@*=#:     **#+     .#*@@      **.       :@@      :@@*=      ++#+-     -%*.     .@==@
    :.-@.     * #=     -:.+#     :=-+@     :@- -.     =%==:    -% .@.     =-=+:    .=#=      =@  %=
     +@+      . +.      +=#+     :..:--    :% .:     =@=#%=    .+:=*     .=#@%:     **+:     :+::*=
     =#+     . *% .     +@+*.    =%..+:    +=:%*.    *@%@@-    -:+.:=    ::#@%-    .=%%*     .+#%#-
      -*.     .-%.      #+ %=    .+  @-    =@%-*=    .-#@#     .*+.--      .:@.    :@#*#:     %+-@-
   .*+ *:     --*:      *==@-    .%+=@    . %+*.:      *@.     .%%+@-     ..:#      %+==..    %#-@-
    =@@#-:    ##:       .@@*.     -=%*      .*#-:      ==.      +@%=      +#*.      ::%=:     -%%*
  kesishsiz, xuddi shu shovqindan (birinchi 5 ta):
  @  @@.@   @@@@ @@    @@@  %   @     @    *@@  :
   @@@#     @@    @    % @       @ @   @   @@@
  :@@  @     @  @ @@    @@        @   @@  =@@@+
  @@@@   @  @ @@  @   @  @   @  @ @    @  @  @  @@
    @@      @@  @ @    +@@=     @@@ @  @  -     #*
  @  @ @@   @@@ @ @   =  @@ +   @@@@@@@@   . @@
    @@  @   @@@  @ @    @@@@@@    @   @@    @@@@@=
   #  @@@@   @@@ @@=    @@@@@@   @  @@@:   *@@@@@@

"""Qadamlar soni T va vaqt embeddingining ta'siri: 8 ta Gauss halqasida juftlashgan taqqoslash."""

import math

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F

MARKAZ = torch.tensor([[2 * math.cos(k * math.pi / 4), 2 * math.sin(k * math.pi / 4)]
                       for k in range(8)])
STD = 0.05
S = 4
B, H = 256, 64


class BLinear(nn.Module):
    def __init__(self, a, b):
        super().__init__()
        k = 1 / math.sqrt(a)
        self.w = nn.Parameter(torch.empty(S, a, b).uniform_(-k, k))
        self.b = nn.Parameter(torch.empty(S, 1, b).uniform_(-k, k))

    def forward(self, x):
        return torch.baddbmm(self.b, x, self.w)


def haqiqiy(n, g):
    k = torch.randint(0, 8, (S, n), generator=g)
    return MARKAZ[k] + STD * torch.randn(S, n, 2, generator=g)


def olcha(x):
    d = torch.cdist(x, MARKAZ.expand(S, 8, 2))
    dmin, eng = d.min(2)
    sifatli = dmin < 3 * STD
    natija = []
    for s in range(S):
        soni = torch.bincount(eng[s][sifatli[s]], minlength=8).float()
        p = soni / soni.sum().clamp_min(1)
        ent = -(p[p > 0] * p[p > 0].log()).sum().item() / math.log(8)
        natija.append([(soni >= 25).sum().item(), sifatli[s].float().mean().item(),
                       dmin[s].median().item(), ent])
    return np.array(natija)


def kosinus_jadval(T, s=0.008):
    f = torch.cos((torch.arange(T + 1) / T + s) / (1 + s) * math.pi / 2) ** 2
    ab = f[1:] / f[0]
    return (1 - ab / torch.cat([torch.ones(1), ab[:-1]])).clamp(max=0.999)


def vaqt_embedding(t, T, d=16):
    """t/T in [0, 1) -> sinusoidal xususiyatlar (T dan qat'i nazar bir xil shkala)."""
    chastota = torch.exp(-math.log(1000.0) * torch.arange(d // 2) / (d // 2))
    burchak = (t.float() / T * 100.0)[..., None] * chastota
    return torch.cat([burchak.sin(), burchak.cos()], -1)


class Denoiser(nn.Module):
    def __init__(self, T, embedding=True):
        super().__init__()
        self.T, self.embedding = T, embedding
        self.tarmoq = nn.Sequential(BLinear(2 + 16, H), nn.SiLU(), BLinear(H, H), nn.SiLU(),
                                    BLinear(H, H), nn.SiLU(), BLinear(H, 2))

    def forward(self, x, t):
        e = vaqt_embedding(t, self.T)
        if not self.embedding:
            e = torch.zeros_like(e)                  # model t ni "ko'rmaydi"
        return self.tarmoq(torch.cat([x, e], -1))


def orgat(T, embedding, qadamlar=1500):
    torch.manual_seed(0)
    model = Denoiser(T, embedding)
    opt = torch.optim.Adam(model.parameters(), lr=2e-3)
    ab = torch.cumprod(1 - kosinus_jadval(T), 0)
    g = torch.Generator().manual_seed(1)
    oxirgi = []
    for q in range(qadamlar):
        x0 = haqiqiy(B, g) / 2
        t = torch.randint(0, T, (S, B), generator=g)
        eps = torch.randn(S, B, 2, generator=g)
        a = ab[t][..., None]
        loss = F.mse_loss(model(a.sqrt() * x0 + (1 - a).sqrt() * eps, t), eps)
        opt.zero_grad()
        loss.backward()
        opt.step()
        if q >= qadamlar - 100:
            oxirgi.append(loss.item())
    return model, float(np.mean(oxirgi))


@torch.no_grad()
def namuna(model, n, g):
    T = model.T
    beta = kosinus_jadval(T)
    alfa, ab = 1 - beta, torch.cumprod(1 - beta, 0)
    x = torch.randn(S, n, 2, generator=g)
    for t in range(T - 1, -1, -1):
        eps = model(x, torch.full((S, n), t))
        x = (x - beta[t] / (1 - ab[t]).sqrt() * eps) / alfa[t].sqrt()
        if t > 0:
            x = x + beta[t].sqrt() * torch.randn(S, n, 2, generator=g)
    return x * 2


def main() -> None:
    torch.set_num_threads(1)
    sozlamalar = [("T=10", 10, True), ("T=100", 100, True), ("T=100, t siz", 100, False)]
    print("=== 1. Uch sozlama, bir xil urug'lar, 1500 qadam, MLP 3 x 64 ===")
    natija, loss, modellar = {}, {}, {}
    for nom, T, emb in sozlamalar:
        modellar[nom], loss[nom] = orgat(T, emb)
        natija[nom] = olcha(namuna(modellar[nom], 2000, torch.Generator().manual_seed(5)))
    nomlar = ["modalar", "sifatli", "median masofa", "muvozanat"]
    print("  sozlama        eps loss  sampling forward  " + "".join(f"{n:>14}" for n in nomlar))
    for nom, T, _ in sozlamalar:
        print(f"  {nom:<14} {loss[nom]:>8.4f} {T:>17}  "
              + "".join(f"{v:>14.3f}" for v in natija[nom].mean(0)))

    print("\n=== 2. Juftlashgan farq: sozlama - (T=100, embedding bilan) ===")
    asos = natija["T=100"]
    xulosa = {}
    for nom, _, _ in sozlamalar:
        if nom == "T=100":
            continue
        qator = []
        for j, n in enumerate(nomlar[1:3], start=1):
            farq = natija[nom][:, j] - asos[:, j]
            se = farq.std(ddof=1) / math.sqrt(S)
            belgi = "*" if abs(farq.mean()) > 2 * se else " "
            qator.append(f"{n} {farq.mean():+.3f} ({se:.3f}){belgi}")
            if j == 1:
                xulosa[nom] = (farq.mean(), se)
        print(f"  {nom:<14} " + "   ".join(qator))
    print("  (* - sezilarli: |farq| > 2 * SE)")

    print("\n=== 3. Vaqt embeddingisiz model nima o'rganadi? ===")
    x = torch.tensor([[[0.6, 0.0]]]).expand(S, 1, 2)     # masshtablangan fazoda nuqta
    for nom in ("T=100", "T=100, t siz"):
        with torch.no_grad():
            e = [modellar[nom](x, torch.full((S, 1), t))[0, 0].tolist() for t in (5, 50, 95)]
        print(f"  {nom:<13} x_t = (0.6, 0) uchun eps bashorati, t = 5, 50, 95: "
              + ", ".join(f"({a:+.2f}, {b:+.2f})" for a, b in e))
    print("  t ni bilmasa, model barcha t lar uchun BITTA o'rtacha javob beradi")
    ab = torch.cumprod(1 - kosinus_jadval(100), 0)
    for t in (5, 50, 95):
        print(f"    t = {t:>2}: sqrt(1 - alfa_bar) = {(1 - ab[t]).sqrt().item():.3f} "
              f"- kerakli eps masshtabi t ga kuchli bog'liq")

    print("\n=== 4. Xulosa (natijadan) ===")
    for nom, (f, se) in xulosa.items():
        if f < -2 * se:
            print(f"  {nom}: sifatli ulush sezilarli past ({f:+.3f})")
        elif f > 2 * se:
            print(f"  {nom}: sifatli ulush sezilarli yuqori ({f:+.3f})")
        else:
            print(f"  {nom}: T=100 dan sezilarli farq yo'q ({f:+.3f}, SE {se:.3f})")
    print("  sampling narxi T ga proporsional: T=100 - 100 ta forward, GAN - 1 ta")


if __name__ == "__main__":
    main()

=== 1. Uch sozlama, bir xil urug'lar, 1500 qadam, MLP 3 x 64 ===
  sozlama        eps loss  sampling forward         modalar       sifatli median masofa     muvozanat
  T=10             0.2896                10           5.000         0.111         0.456         0.973
  T=100            0.3247               100           4.250         0.114         0.416         0.984
  T=100, t siz     0.4039               100           0.000         0.031         0.658         0.968

=== 2. Juftlashgan farq: sozlama - (T=100, embedding bilan) ===
  T=10           sifatli -0.003 0.009-bob    median masofa +0.039 0.013-bob*
  T=100, t siz   sifatli -0.083 0.020-bob*   median masofa +0.242 0.028-bob*
  (* - sezilarli: |farq| > 2 * SE)

=== 3. Vaqt embeddingisiz model nima o'rganadi? ===
  T=100         x_t = (0.6, 0) uchun eps bashorati, t = 5, 50, 95: (-1.20, -0.11), (+0.46, -0.05), (+0.53, -0.04)
  T=100, t siz  x_t = (0.6, 0) uchun eps bashorati, t = 5, 50, 95: (+0.07, +0.02), (+0.07, +0.02), (+0.07, +0.02)
  t ni bilmasa, model barcha t lar uchun BITTA o'rtacha javob beradi
    t =  5: sqrt(1 - alfa_bar) = 0.105 - kerakli eps masshtabi t ga kuchli bog'liq
    t = 50: sqrt(1 - alfa_bar) = 0.722 - kerakli eps masshtabi t ga kuchli bog'liq
    t = 95: sqrt(1 - alfa_bar) = 0.998 - kerakli eps masshtabi t ga kuchli bog'liq

=== 4. Xulosa (natijadan) ===
  T=10: T=100 dan sezilarli farq yo'q (-0.003, SE 0.009)
  T=100, t siz: sifatli ulush sezilarli past (-0.083)
  sampling narxi T ga proporsional: T=100 - 100 ta forward, GAN - 1 ta

xt = (1 - beta[t]).sqrt() * x0 + beta[t].sqrt() * eps               # ⚠️ bitta qadam
ab = torch.cumprod(1 - beta, 0)
xt = ab[t].sqrt() * x0 + (1 - ab[t]).sqrt() * eps                   # ✅

xt = ab[t] * x0 + (1 - ab[t]) * eps                                 # ⚠️ dispersiya noto'g'ri
xt = ab[t].sqrt() * x0 + (1 - ab[t]).sqrt() * eps                   # ✅

loss = F.mse_loss(model(xt), eps)                                   # ⚠️
loss = F.mse_loss(model(xt, t), eps)                                # ✅

x = mu + beta[t].sqrt() * torch.randn(x.shape, generator=g)         # ⚠️ t = 0 da ham
if t > 0:
    x = mu + beta[t].sqrt() * torch.randn(x.shape, generator=g)     # ✅

x0 = torch.tensor(digits.data)                                      # ⚠️ 0..16, shovqin N(0, 1)
x0 = torch.tensor(digits.data / 8.0 - 1.0)                          # ✅ [-1, 1]

beta = 1 - ab / ab_oldingi                                          # ⚠️ oxirida 1 ga yaqin
beta = (1 - ab / ab_oldingi).clamp(max=0.999)                       # ✅

x = (x - beta[t] / (1 - ab[t]).sqrt() * eps_hat) / alfa[t].sqrt()    # ⚠️ to'yinib ketadi
x0_hat = ((x - (1 - ab[t]).sqrt() * eps_hat) / ab[t].sqrt()).clamp(-1, 1)
x = (beta[t] * ab_old[t].sqrt() * x0_hat
     + (1 - ab_old[t]) * alfa[t].sqrt() * x) / (1 - ab[t])          # ✅ posterior o'rtachasi

eng_yaxshi = min(sozlamalar, key=lambda s: eps_loss[s])             # ⚠️
eng_yaxshi = max(sozlamalar, key=lambda s: sifatli_ulush[s])        # ✅ namunalar bo'yicha

1.  # alfa_bar_t qanday hisoblanadi?
2.  # q(x_t | x_0) ning o'rtachasi va dispersiyasi?
3.  # SNR(t) formulasi? log10 SNR = 0 nimani anglatadi?
4.  # chiziqli jadvalda (T = 1000) t/T = 0.5 da alfa_bar taxminan?
5.  # nega o'rgatishda t tasodifiy tanlanadi?
6.  # eps-prediction loss formulasi?
7.  # sampling da x_T qayerdan olinadi?
8.  # t = 0 qadamida shovqin qo'shiladimi?
9.  # nega diffusion mode collapse ga moyil emas?
10. # T = 100 bo'lsa bitta namuna uchun nechta forward?
11. # x0_hat qanday hisoblanadi?
12. # tarmoq t ni bilmasa nima bo'ladi?

1.  ab = 1 - beta
    xt = ab[t].sqrt() * x0 + (1 - ab[t]).sqrt() * eps

2.  t = torch.randint(1, T + 1, (B,))
    a = ab[t][:, None]

3.  for t in range(T):
        x = (x - beta[t] / (1 - ab[t]).sqrt() * model(x, t)) / alfa[t].sqrt()

4.  loss = F.mse_loss(model(xt, t), x0)        # "eps-prediction"

5.  x = torch.randn(n, 64)
    for t in reversed(range(T)):
        x = step(model, x, t) + beta[t].sqrt() * torch.randn(n, 64)