23.11-dars: Seq2seq va attention

ENCODER (kirishni o'qiydi):
  x_1 ... x_T  ->  Embedding -> GRU (bu darsda ikki tomonlama)
  chiqishlar: e_1 ... e_T   (har pozitsiya uchun vektor)
  yakuniy holat: h_T        (butun kirishning "xulosasi")

DECODER (shartli til modeli):
  boshlang'ich holat s_0 = f(h_T)
  kirish: <bos> y_1 y_2 ... y_{m-1}
  nishon: y_1 y_2 ... y_m <eos>
  har qadamda: s_t = GRU(y_{t-1}, s_{t-1});  P(y_t) = softmax(W s_t)

MAXSUS TOKENLAR:
  <pad> - to'ldirish (loss da ignore_index=PAD)
  <bos> - decoder boshlanishi
  <eos> - "tugadi" signali: chiqish uzunligini model O'ZI hal qiladi

SHAKLLAR (1-misol, sana vazifasi):
  X: (B, 10)          "2035-08-16"
  Y: (B, 17)          <bos> 1 6 - a v g u s t   2 0 3 5 <eos> <pad>...
  decoder kirishi Y[:, :-1], nishoni Y[:, 1:] - til modelidagi kabi surilgan

O'RGATISH - TEACHER FORCING:
  decoder kirishi = HAQIQIY oldingi belgilar (Y[:, :-1])
  butun chiqish bitta forward da, loss hamma pozitsiyada
  tez va barqaror

BASHORAT - GREEDY INFERENCE:
  y_0 = <bos>
  takrorla: logit, s = decoder(y_{t-1}, s); y_t = argmax(logit)
  <eos> chiqsa yoki maks uzunlikka yetsa - to'xta
  decoder kirishi = MODELNING O'Z bashoratlari

FARQ - EXPOSURE BIAS:
  o'rgatishda model hech qachon o'z xatosini kirishda ko'rmagan
  bashoratda bitta xato -> keyingi qadamlar "begona" kontekstda
  4-misol: birinchi xatodan keyingi token aniqligi greedy da 0.61,
           xuddi shu pozitsiyalarda teacher forcing da 0.9985

O'LCHOVLARNI ADASHTIRMANG:
  TF token aniqligi - optimistik (shpargalka bilan imtihon)
  haqiqiy sifat - greedy (yoki beam search) dan keyingi ANIQ MOSLIK
  1-misol: TF token aniqligi 0.84, greedy aniq moslik 0.011

ATTENTION SIZ SEQ2SEQ:
  decoder faqat s_0 = f(h_T) ni oladi
  T ta belgi haqidagi HAMMA ma'lumot bitta h o'lchamli vektorda
  encoder chiqishlari e_1 ... e_T ishlatilmaydi

MUAMMOLAR:
  1. sig'im: vektor o'lchami qat'iy, kirish uzunligi esa o'sadi
  2. masofa: kirish boshidagi ma'lumot ko'p qadam "yuradi"
     sanada: yil kirish BOSHIDA, chiqish OXIRIDA -> eng uzoq yo'l
  3. decoder har qadamda kontekstni o'z holatida olib yurishi kerak

O'LCHOV (bu dars):
  1-misol: 400 qadamdan keyin kun va oy 100%, yil - 1.1%
           yilning 1-2 raqami 0.50 (19 yoki 20 - tanga tashlash),
           3- va 4-raqami 0.09 va 0.10 (tasodif ~0.1)
  2-misol: teskari aylantirish, uzunlik 4-6 da 0.24,
           11-14 da 0.00 - bitta ham to'liq to'g'ri javob yo'q

G'OYA: decoder har qadamda encoderning BARCHA chiqishlariga qaraydi

1. SKOR - s_t (decoder holati) va har e_i qanchalik mos:
   dot-product (Luong):  skor_i = s_t . e_i
   additiv (Bahdanau):   skor_i = v^T tanh(W_s s_t + W_h e_i)
2. OG'IRLIKLAR:  a_t = softmax(skor)          yig'indisi 1
   (padding pozitsiyalari: skor = -cheksiz -> og'irlik 0)
3. KONTEKST:     c_t = yig'indi_i a_ti * e_i  (vaznli o'rtacha)
4. CHIQISH:      z_t = tanh(W [s_t ; c_t]);   P(y_t) = softmax(U z_t)

TENSOR SHAKLIDA (parallel, teacher forcing):
  s: (B, t, h), enc: (B, T, h)
  skor = s @ enc.transpose(1, 2)        (B, t, T)
  w = softmax(skor, dim=2)
  kontekst = w @ enc                     (B, t, h)

VARIANTLAR:
  Luong: skor s_t (joriy holat) bilan, kontekst RNN dan KEYIN qo'shiladi
         -> teacher forcing da butun chiqish parallel (bu darsdagi kod)
  Bahdanau (asl): s_{t-1} bilan, kontekst decoder GRU KIRISHIGA beriladi
         -> faqat qadamma-qadam sikl
  bu darsdagi "additiv" - Bahdanau skori, Luong joylashuvida

NARXI:
  har chiqish qadami x har kirish pozitsiyasi: O(t * T) skor
  parametr: dot - qo'shimcha skor parametri yo'q; additiv - W_s, W_h, v

a_t VEKTORI - "t-chiqish qaysi kirishga qaraydi":
  sana vazifasida kutilgan moslik ma'lum:
    kun raqamlari  -> kirishning 8-9 pozitsiyasi (DD)
    oy harflari    -> 5-6 pozitsiya (MM)
    yilning j-raqami -> j-pozitsiya (YYYY)

TEKSHIRISH (3-misol, 1000 test):
  argmax a_t kutilgan pozitsiyadami?  kun 0.980, yil 0.874, oy 0.493
  kutilgan pozitsiya yoki qo'shnisi:   kun 0.995, yil 1.000, oy 1.000
  tasodifiy qarash: ~0.1-0.2 (qo'shnisi bilan 0.2-0.4)

NIMA UCHUN OY "QO'SHNIGA" QARAYDI:
  encoder ikki tomonlama: '-' belgisi (4- va 7-pozitsiya) holatida
  orqaga yo'nalish oy raqamlarini allaqachon "ko'rgan"
  model ma'lumotni ENG QULAY joydan oladi - aynan belgidan emas

TALQIN CHEGARASI:
  attention - modelning "qayerdan ma'lumot olgani", tushuntirish emas
  encoder holatlari allaqachon qo'shnilar bilan aralashgan
  og'irlikni "sabab" deb o'qishdan ehtiyot bo'ling

BU DARSDA:
  attention RNN ga QO'SHIMCHA: encoder va decoder hali ham GRU
  ketma-ket hisob: t-qadam (t-1) ni kutadi

KEYINGI QADAM - "RNN ni olib tashlash":
  agar decoder encoderga attention bilan qaray olsa,
  nega har pozitsiya O'Z ketma-ketligining boshqa pozitsiyalariga ham
  attention bilan qaramasin?  -> SELF-ATTENTION
  so'rov, kalit, qiymat (query, key, value) - bir ketma-ketlikdan
  barcha pozitsiyalar PARALLEL hisoblanadi - GPU uchun ideal

TRANSFORMER:
  self-attention + to'liq bog'langan qatlamlar + pozitsiya kodlash
  encoder-decoder tuzilishi saqlanadi, rekurrentlik yo'qoladi
  decoder dagi "cross-attention" - aynan bu darsdagi attention

BU DARSDAN O'TADIGANLAR:
  skor -> softmax -> vaznli o'rtacha
  padding maskasi (-cheksiz)
  teacher forcing, <bos>/<eos>, greedy inference

import torch
import torch.nn as nn

# ma'lumot: <bos> ... <eos>, <pad> bilan to'ldirilgan
Y_kir, Y_nishon = Y[:, :-1], Y[:, 1:]

# encoder
enc, h = encoder(emb_kir(X))                  # enc: (B, T, h)

# decoder + dot-product attention (teacher forcing, parallel)
s, _ = decoder(emb_chiq(Y_kir), h0)           # (B, t, h)
skor = s @ enc.transpose(1, 2)                # (B, t, T)
skor = skor.masked_fill(~maska[:, None, :], float("-inf"))
w = torch.softmax(skor, dim=2)                # kirish o'qi bo'yicha
kontekst = w @ enc                            # (B, t, h)
z = torch.tanh(birlash(torch.cat([s, kontekst], 2)))
logit = chiqish(z)
loss = nn.functional.cross_entropy(logit.reshape(-1, V),
                                   Y_nishon.reshape(-1), ignore_index=PAD)

# additiv (Bahdanau) skor
skor = v(torch.tanh(W_s(s)[:, :, None] + W_h(enc)[:, None])).squeeze(3)

# greedy inference
y = torch.full((B, 1), BOS)
for _ in range(maks):
    logit, h, w = dekod(y, h, enc)            # bitta qadam, holat uzatiladi
    y = logit[:, -1].argmax(1, keepdim=True)

encoder -> kontekst -> decoder (shartli til modeli)
o'rgatish: teacher forcing; baholash: greedy + aniq moslik
bitta kontekst vektor - uzun kirishda "tor bo'g'iz"
attention: skor -> softmax -> vaznli o'rtacha; padding maskasi
og'irliklar - yumshoq hizalash, tekshirsa bo'ladi
Transformer = attention, RNN siz

"""Sanani o'girish: attention siz encoder-decoder, teacher forcing va greedy."""

import calendar

import numpy as np
import torch
import torch.nn as nn

OYLAR = ["yanvar", "fevral", "mart", "aprel", "may", "iyun", "iyul", "avgust",
         "sentabr", "oktabr", "noyabr", "dekabr"]
KIR = ["<pad>"] + list("0123456789-")
CHIQ = ["<pad>", "<bos>", "<eos>"] + sorted(set("0123456789- " + "".join(OYLAR)))
PAD, BOS, EOS = 0, 1, 2
k2i = {c: i for i, c in enumerate(KIR)}
c2i = {c: i for i, c in enumerate(CHIQ)}


def sanalar(n, seed):
    rng = np.random.default_rng(seed)
    juftlar = []
    for _ in range(n):
        yil = int(rng.integers(1950, 2050))
        oy = int(rng.integers(1, 13))
        kun = int(rng.integers(1, calendar.monthrange(yil, oy)[1] + 1))
        juftlar.append((f"{yil}-{oy:02d}-{kun:02d}", f"{kun}-{OYLAR[oy - 1]} {yil}"))
    return juftlar


def tensorla(juftlar):
    X = torch.tensor([[k2i[c] for c in a] for a, _ in juftlar])
    T = max(len(b) for _, b in juftlar) + 2
    Y = torch.full((len(juftlar), T), PAD)
    for i, (_, b) in enumerate(juftlar):
        s = [BOS] + [c2i[c] for c in b] + [EOS]
        Y[i, :len(s)] = torch.tensor(s)
    return X, Y


class Seq2seq(nn.Module):
    """attention: None, 'dot' yoki 'additiv' (Bahdanau skori)."""

    def __init__(self, attention=None, h=64, d=32):
        super().__init__()
        self.attention = attention
        self.emb_kir = nn.Embedding(len(KIR), d)
        self.emb_chiq = nn.Embedding(len(CHIQ), d, padding_idx=PAD)
        self.encoder = nn.GRU(d, h, batch_first=True, bidirectional=True)
        self.enc_proj = nn.Linear(2 * h, h)
        self.decoder = nn.GRU(d, h, batch_first=True)
        if attention == "additiv":
            self.W_s = nn.Linear(h, h, bias=False)
            self.W_h = nn.Linear(h, h, bias=False)
            self.v = nn.Linear(h, 1, bias=False)
        if attention is not None:
            self.birlash = nn.Linear(2 * h, h)
        self.chiqish = nn.Linear(h, len(CHIQ))

    def kodla(self, X):
        o, h = self.encoder(self.emb_kir(X))
        enc = self.enc_proj(o)                              # (B, T_kir, h)
        h0 = torch.tanh(self.enc_proj(torch.cat([h[0], h[1]], 1)))
        return enc, h0.unsqueeze(0)

    def dekod(self, y_kir, h, enc):
        s, h = self.decoder(self.emb_chiq(y_kir), h)       # (B, t, h)
        if self.attention is None:
            return self.chiqish(s), h, None
        if self.attention == "dot":
            skor = s @ enc.transpose(1, 2)
        else:
            skor = self.v(torch.tanh(self.W_s(s)[:, :, None]
                                     + self.W_h(enc)[:, None])).squeeze(3)
        w = torch.softmax(skor, dim=2)                      # (B, t, T_kir)
        z = torch.tanh(self.birlash(torch.cat([s, w @ enc], 2)))
        return self.chiqish(z), h, w

    def forward(self, X, Y_kir):
        enc, h = self.kodla(X)
        return self.dekod(Y_kir, h, enc)

    @torch.no_grad()
    def greedy(self, X, maks):
        enc, h = self.kodla(X)
        y = torch.full((len(X), 1), BOS)
        chiq, ogirlik = [], []
        for _ in range(maks):
            logit, h, w = self.dekod(y, h, enc)
            y = logit[:, -1].argmax(1, keepdim=True)
            chiq.append(y)
            if w is not None:
                ogirlik.append(w[:, -1])
        return torch.cat(chiq, 1), (torch.stack(ogirlik, 1) if ogirlik else None)


def orgat(attention, seed, qadamlar, X, Y, kuzatuv=None):
    torch.manual_seed(seed)
    model = Seq2seq(attention)
    opt = torch.optim.Adam(model.parameters(), lr=0.003)
    g = torch.Generator().manual_seed(seed)
    for q in range(qadamlar):
        idx = torch.randint(0, len(X), (64,), generator=g)
        logit, _, _ = model(X[idx], Y[idx, :-1])
        loss = nn.functional.cross_entropy(logit.reshape(-1, len(CHIQ)),
                                           Y[idx, 1:].reshape(-1),
                                           ignore_index=PAD)
        opt.zero_grad()
        loss.backward()
        nn.utils.clip_grad_norm_(model.parameters(), 1.0)
        opt.step()
        if kuzatuv is not None:
            kuzatuv(q + 1, model)
    model.eval()
    return model


def matnga(qator):
    s = []
    for i in qator:
        if i == EOS:
            break
        s.append(CHIQ[i])
    return "".join(s)


def main() -> None:
    torch.set_num_threads(1)
    oquv = sanalar(5000, 0)
    test = sanalar(1000, 1)
    Xo, Yo = tensorla(oquv)
    Xt, Yt = tensorla(test)

    print("=== 1. Ma'lumot ===")
    for a, b in oquv[:3]:
        print(f"  {a}  ->  {b}")
    print(f"  kirish lug'ati {len(KIR)}, chiqish lug'ati {len(CHIQ)}")
    print(f"  X {tuple(Xo.shape)}, Y {tuple(Yo.shape)}  (Y: <bos> ... <eos> <pad>)")
    print(f"  decoder kirishi Y[:, :-1]: {[CHIQ[i] for i in Yo[0, :5].tolist()]}...")
    print(f"  decoder nishoni Y[:, 1:]:  {[CHIQ[i] for i in Yo[0, 1:6].tolist()]}...")

    print("\n=== 2. Encoder-decoder (attention siz) ===")
    model = Seq2seq(None)
    with torch.no_grad():
        enc, h = model.kodla(Xo[:4])
    print(f"  encoder chiqishlari {tuple(enc.shape)} - attention siz ISHLATILMAYDI")
    print(f"  kontekst vektor h   {tuple(h.shape)} - decoder faqat shuni oladi")
    print(f"  parametrlar: {sum(p.numel() for p in model.parameters())}")

    print("\n=== 3. O'rgatish (teacher forcing) va greedy dekodlash ===")
    print(f"  {'qadam':>6} {'TF token aniq':>14} {'greedy aniq moslik':>19}")
    maska = Yt[:, 1:] != PAD
    natija = {}

    def kuzat(qadam, m):
        if qadam in (100, 200, 400):
            m.eval()
            with torch.no_grad():
                logit, _, _ = m(Xt, Yt[:, :-1])
            tf = (logit.argmax(2) == Yt[:, 1:])[maska].float().mean().item()
            b, _ = m.greedy(Xt, Yt.shape[1] - 1)
            chiq = [matnga(q) for q in b.tolist()]
            em = np.mean([c == s for c, (_, s) in zip(chiq, test)])
            natija[qadam] = chiq
            print(f"  {qadam:>6} {tf:>14.4f} {em:>19.4f}")
            m.train()

    orgat(None, 0, 400, Xo, Yo, kuzatuv=kuzat)
    chiq = natija[400]

    print("\n=== 4. Qaysi qism xato: kun, oy, yil (400 qadam) ===")
    kun = np.mean([c.split("-")[0] == s.split("-")[0]
                   for c, (_, s) in zip(chiq, test)])
    oy = np.mean([c.split("-")[1][:-5] == s.split("-")[1][:-5]
                  if c.count("-") == 1 and len(c) > 5 else False
                  for c, (_, s) in zip(chiq, test)])
    yil = np.mean([c[-4:] == s[-4:] for c, (_, s) in zip(chiq, test)])
    print(f"  kun {kun:.3f}   oy {oy:.3f}   yil {yil:.3f}")
    for j in range(4):
        r = np.mean([len(c) >= 4 and c[-4 + j] == s[-4 + j]
                     for c, (_, s) in zip(chiq, test)])
        print(f"  yilning {j + 1}-raqami: {r:.3f}")
    for (a, s), c in list(zip(test, chiq))[:4]:
        print(f"  {a} -> {c:<18} (to'g'ri: {s})")
    if yil < kun:
        print("  kirishning BOSHIDAGI yil - chiqishning OXIRIDA: eng uzoq masofa")
    print("  ⭐ Bitta kontekst vektor - butun kirishni siqishga majbur 'tor bo'g'iz'")


if __name__ == "__main__":
    main()

=== 1. Ma'lumot ===
  2035-08-16  ->  16-avgust 2035
  1976-04-02  ->  2-aprel 1976
  1957-01-06  ->  6-yanvar 1957
  kirish lug'ati 12, chiqish lug'ati 34
  X (5000, 10), Y (5000, 17)  (Y: <bos> ... <eos> <pad>)
  decoder kirishi Y[:, :-1]: ['<bos>', '1', '6', '-', 'a']...
  decoder nishoni Y[:, 1:]:  ['1', '6', '-', 'a', 'v']...

=== 2. Encoder-decoder (attention siz) ===
  encoder chiqishlari (4, 10, 64) - attention siz ISHLATILMAYDI
  kontekst vektor h   (1, 4, 64) - decoder faqat shuni oladi
  parametrlar: 68386

=== 3. O'rgatish (teacher forcing) va greedy dekodlash ===
   qadam  TF token aniq  greedy aniq moslik
     100         0.7640              0.0000
     200         0.7986              0.0040
     400         0.8426              0.0110

=== 4. Qaysi qism xato: kun, oy, yil (400 qadam) ===
  kun 1.000   oy 1.000   yil 0.011
  yilning 1-raqami: 0.496
  yilning 2-raqami: 0.496
  yilning 3-raqami: 0.093
  yilning 4-raqami: 0.102
  1997-07-24 -> 24-iyul 1958       (to'g'ri: 24-iyul 1997)
  2045-01-05 -> 5-yanvar 1958      (to'g'ri: 5-yanvar 2045)
  2032-12-08 -> 8-dekabr 1958      (to'g'ri: 8-dekabr 2032)
  1981-11-13 -> 13-noyabr 1958     (to'g'ri: 13-noyabr 1981)
  kirishning BOSHIDAGI yil - chiqishning OXIRIDA: eng uzoq masofa
  ⭐ Bitta kontekst vektor - butun kirishni siqishga majbur 'tor bo'g'iz'

"""Teskari aylantirish: attention siz va dot-product attention bilan, uzunlik bo'yicha."""

import numpy as np
import torch
import torch.nn as nn

PAD, BOS, EOS = 0, 1, 2
HARFLAR = "abcdefghijklmnopqrstuvwxyz"[:20]
V = 3 + len(HARFLAR)


def malumot(n, g, uz_min=4, uz_max=14):
    uz = torch.randint(uz_min, uz_max + 1, (n,), generator=g)
    T = int(uz.max())
    X = torch.full((n, T), PAD)
    Y = torch.full((n, T + 2), PAD)
    for b in range(n):
        k = int(uz[b])
        s = torch.randint(3, V, (k,), generator=g)
        X[b, :k] = s
        Y[b, 0] = BOS
        Y[b, 1:k + 1] = s.flip(0)
        Y[b, k + 1] = EOS
    return X, uz, Y


class Seq2seq(nn.Module):
    def __init__(self, attention, h=64, d=32):
        super().__init__()
        self.attention = attention
        self.emb_kir = nn.Embedding(V, d, padding_idx=PAD)
        self.emb_chiq = nn.Embedding(V, d, padding_idx=PAD)
        self.encoder = nn.GRU(d, h, batch_first=True)
        self.decoder = nn.GRU(d, h, batch_first=True)
        self.birlash = nn.Linear(2 * h, h)
        self.chiqish = nn.Linear(h, V)

    def kodla(self, X, uz):
        o, _ = self.encoder(self.emb_kir(X))                # (B, T_kir, h)
        h = o[torch.arange(len(uz)), uz - 1].unsqueeze(0)   # haqiqiy oxirgi holat
        return o, h

    def dekod_qadam(self, y_kir, h, enc, maska):
        s, h = self.decoder(self.emb_chiq(y_kir), h)       # (B, t, h)
        if not self.attention:
            return self.chiqish(s), h, None
        skor = s @ enc.transpose(1, 2)                      # (B, t, T_kir)
        skor = skor.masked_fill(~maska[:, None, :], float("-inf"))
        w = torch.softmax(skor, dim=2)
        kontekst = w @ enc                                  # (B, t, h)
        z = torch.tanh(self.birlash(torch.cat([s, kontekst], 2)))
        return self.chiqish(z), h, w

    def forward(self, X, uz, Y_kir):
        enc, h = self.kodla(X, uz)
        maska = torch.arange(X.shape[1])[None, :] < uz[:, None]
        logit, _, _ = self.dekod_qadam(Y_kir, h, enc, maska)
        return logit

    @torch.no_grad()
    def greedy(self, X, uz, maks):
        enc, h = self.kodla(X, uz)
        maska = torch.arange(X.shape[1])[None, :] < uz[:, None]
        y = torch.full((len(X), 1), BOS)
        chiq = []
        for _ in range(maks):
            logit, h, _ = self.dekod_qadam(y, h, enc, maska)
            y = logit[:, -1].argmax(1, keepdim=True)
            chiq.append(y)
        return torch.cat(chiq, 1)


def orgat(attention, seed, qadamlar):
    torch.manual_seed(seed)
    model = Seq2seq(attention)
    opt = torch.optim.Adam(model.parameters(), lr=0.003)
    g = torch.Generator().manual_seed(seed)
    for _ in range(qadamlar):
        X, uz, Y = malumot(64, g)
        logit = model(X, uz, Y[:, :-1])
        loss = nn.functional.cross_entropy(logit.reshape(-1, V),
                                           Y[:, 1:].reshape(-1),
                                           ignore_index=PAD)
        opt.zero_grad()
        loss.backward()
        nn.utils.clip_grad_norm_(model.parameters(), 1.0)
        opt.step()
    model.eval()
    return model


def aniq_moslik(model, X, uz, Y):
    b = model.greedy(X, uz, Y.shape[1] - 1)
    nishon = Y[:, 1:]
    maska = nishon != PAD
    return ((b == nishon) | ~maska).all(1).float()


def main() -> None:
    torch.set_num_threads(1)
    print("=== 1. Vazifa: ketma-ketlikni teskari aylantirish ===")
    X, uz, Y = malumot(2, torch.Generator().manual_seed(4), 4, 8)
    for b in range(2):
        kir = "".join(HARFLAR[i - 3] for i in X[b, :uz[b]].tolist())
        chiq = "".join(HARFLAR[i - 3] for i in Y[b, 1:uz[b] + 1].tolist())
        print(f"  {kir:<10} -> {chiq}")
    print("  o'rgatish: uzunlik 4..14, har qadamda yangi tasodifiy batch")

    Xt, uzt, Yt = malumot(1000, torch.Generator().manual_seed(99))
    guruhlar = [(4, 6), (7, 10), (11, 14)]
    natija = {False: [], True: []}
    for s in range(3):
        for att in (False, True):
            m = orgat(att, s, 200)
            a = aniq_moslik(m, Xt, uzt, Yt)
            natija[att].append([a[(uzt >= lo) & (uzt <= hi)].mean().item()
                                for lo, hi in guruhlar])
    print("\n=== 2. Aniq moslik uzunlik bo'yicha (200 qadam, 3 seed o'rtachasi) ===")
    print(f"  {'uzunlik':<9} {'attention siz':>14} {'attention':>10} "
          f"{'farq':>8} {'SE':>7} {'sezilarli':>10}")
    for k, (lo, hi) in enumerate(guruhlar):
        a0 = np.array([r[k] for r in natija[False]])
        a1 = np.array([r[k] for r in natija[True]])
        farq = a1 - a0
        se = farq.std(ddof=1) / np.sqrt(len(farq))
        print(f"  {f'{lo}-{hi}':<9} {a0.mean():>14.4f} {a1.mean():>10.4f} "
              f"{farq.mean():>+8.4f} {se:>7.4f} "
              f"{str(abs(farq.mean()) > 2 * se):>10}")

    print("\n=== 3. Uzunlik oshganda yo'qotish ===")
    for att, nom in [(False, "attention siz"), (True, "attention")]:
        q = np.mean([r[0] for r in natija[att]])
        u = np.mean([r[-1] for r in natija[att]])
        print(f"  {nom:<14} 4-6: {q:.4f} -> 11-14: {u:.4f}  "
              f"(saqlangan ulush {u / q:.0%})")
    siz = np.mean([r[-1] for r in natija[False]])
    if siz < 0.05:
        print("  attention siz model uzun ketma-ketlikni deyarli hech qachon "
              "to'liq to'g'ri qaytarmadi")
    print("  ⭐ Bitta kontekst vektor uzunlik bilan to'yinadi; attention ham pasayadi,"
          " lekin ancha sekin")


if __name__ == "__main__":
    main()

=== 1. Vazifa: ketma-ketlikni teskari aylantirish ===
  lnjq       -> qjnl
  tqcbsqpn   -> npqsbcqt
  o'rgatish: uzunlik 4..14, har qadamda yangi tasodifiy batch

=== 2. Aniq moslik uzunlik bo'yicha (200 qadam, 3 seed o'rtachasi) ===
  uzunlik    attention siz  attention     farq      SE  sezilarli
  4-6               0.2395     0.9457  +0.7062  0.0118       True
  7-10              0.0029     0.8429  +0.8400  0.0125       True
  11-14             0.0000     0.6527  +0.6527  0.0154       True

=== 3. Uzunlik oshganda yo'qotish ===
  attention siz  4-6: 0.2395 -> 11-14: 0.0000  (saqlangan ulush 0%)
  attention      4-6: 0.9457 -> 11-14: 0.6527  (saqlangan ulush 69%)
  attention siz model uzun ketma-ketlikni deyarli hech qachon to'liq to'g'ri qaytarmadi
  ⭐ Bitta kontekst vektor uzunlik bilan to'yinadi; attention ham pasayadi, lekin ancha sekin

"""Sanani o'girishda attention: dot va additiv (Bahdanau) skor, og'irliklar matritsasi."""

import calendar

import numpy as np
import torch
import torch.nn as nn

OYLAR = ["yanvar", "fevral", "mart", "aprel", "may", "iyun", "iyul", "avgust",
         "sentabr", "oktabr", "noyabr", "dekabr"]
KIR = ["<pad>"] + list("0123456789-")
CHIQ = ["<pad>", "<bos>", "<eos>"] + sorted(set("0123456789- " + "".join(OYLAR)))
PAD, BOS, EOS = 0, 1, 2
k2i = {c: i for i, c in enumerate(KIR)}
c2i = {c: i for i, c in enumerate(CHIQ)}


def sanalar(n, seed):
    rng = np.random.default_rng(seed)
    juftlar = []
    for _ in range(n):
        yil = int(rng.integers(1950, 2050))
        oy = int(rng.integers(1, 13))
        kun = int(rng.integers(1, calendar.monthrange(yil, oy)[1] + 1))
        juftlar.append((f"{yil}-{oy:02d}-{kun:02d}", f"{kun}-{OYLAR[oy - 1]} {yil}"))
    return juftlar


def tensorla(juftlar):
    X = torch.tensor([[k2i[c] for c in a] for a, _ in juftlar])
    T = max(len(b) for _, b in juftlar) + 2
    Y = torch.full((len(juftlar), T), PAD)
    for i, (_, b) in enumerate(juftlar):
        s = [BOS] + [c2i[c] for c in b] + [EOS]
        Y[i, :len(s)] = torch.tensor(s)
    return X, Y


class Seq2seq(nn.Module):
    """attention: None, 'dot' yoki 'additiv' (Bahdanau skori)."""

    def __init__(self, attention=None, h=64, d=32):
        super().__init__()
        self.attention = attention
        self.emb_kir = nn.Embedding(len(KIR), d)
        self.emb_chiq = nn.Embedding(len(CHIQ), d, padding_idx=PAD)
        self.encoder = nn.GRU(d, h, batch_first=True, bidirectional=True)
        self.enc_proj = nn.Linear(2 * h, h)
        self.decoder = nn.GRU(d, h, batch_first=True)
        if attention == "additiv":
            self.W_s = nn.Linear(h, h, bias=False)
            self.W_h = nn.Linear(h, h, bias=False)
            self.v = nn.Linear(h, 1, bias=False)
        if attention is not None:
            self.birlash = nn.Linear(2 * h, h)
        self.chiqish = nn.Linear(h, len(CHIQ))

    def kodla(self, X):
        o, h = self.encoder(self.emb_kir(X))
        enc = self.enc_proj(o)                              # (B, T_kir, h)
        h0 = torch.tanh(self.enc_proj(torch.cat([h[0], h[1]], 1)))
        return enc, h0.unsqueeze(0)

    def dekod(self, y_kir, h, enc):
        s, h = self.decoder(self.emb_chiq(y_kir), h)       # (B, t, h)
        if self.attention is None:
            return self.chiqish(s), h, None
        if self.attention == "dot":
            skor = s @ enc.transpose(1, 2)
        else:
            skor = self.v(torch.tanh(self.W_s(s)[:, :, None]
                                     + self.W_h(enc)[:, None])).squeeze(3)
        w = torch.softmax(skor, dim=2)                      # (B, t, T_kir)
        z = torch.tanh(self.birlash(torch.cat([s, w @ enc], 2)))
        return self.chiqish(z), h, w

    def forward(self, X, Y_kir):
        enc, h = self.kodla(X)
        return self.dekod(Y_kir, h, enc)

    @torch.no_grad()
    def greedy(self, X, maks):
        enc, h = self.kodla(X)
        y = torch.full((len(X), 1), BOS)
        chiq, ogirlik = [], []
        for _ in range(maks):
            logit, h, w = self.dekod(y, h, enc)
            y = logit[:, -1].argmax(1, keepdim=True)
            chiq.append(y)
            if w is not None:
                ogirlik.append(w[:, -1])
        return torch.cat(chiq, 1), (torch.stack(ogirlik, 1) if ogirlik else None)


def orgat(attention, seed, qadamlar, X, Y, kuzatuv=None):
    torch.manual_seed(seed)
    model = Seq2seq(attention)
    opt = torch.optim.Adam(model.parameters(), lr=0.003)
    g = torch.Generator().manual_seed(seed)
    for q in range(qadamlar):
        idx = torch.randint(0, len(X), (64,), generator=g)
        logit, _, _ = model(X[idx], Y[idx, :-1])
        loss = nn.functional.cross_entropy(logit.reshape(-1, len(CHIQ)),
                                           Y[idx, 1:].reshape(-1),
                                           ignore_index=PAD)
        opt.zero_grad()
        loss.backward()
        nn.utils.clip_grad_norm_(model.parameters(), 1.0)
        opt.step()
        if kuzatuv is not None:
            kuzatuv(q + 1, model)
    model.eval()
    return model


def matnga(qator):
    s = []
    for i in qator:
        if i == EOS:
            break
        s.append(CHIQ[i])
    return "".join(s)


def kutilgan_joy(chiqish):
    """Har chiqish belgisi uchun kirishdagi (YYYY-MM-DD) kutilgan pozitsiyalar."""
    kun, qolgan = chiqish.split("-", 1)
    oy, yil = qolgan.split(" ")
    joylar = []
    for _ in kun:
        joylar.append(("kun", {8, 9}))
    joylar.append(("ajratgich", None))
    for _ in oy:
        joylar.append(("oy", {5, 6}))
    joylar.append(("ajratgich", None))
    for j in range(4):
        joylar.append(("yil", {j}))
    return joylar


def main() -> None:
    torch.set_num_threads(1)
    Xo, Yo = tensorla(sanalar(5000, 0))
    test = sanalar(1000, 1)
    Xt, Yt = tensorla(test)

    print("=== 1. Ikki attention turi, bir xil byudjet (300 qadam) ===")
    modellar = {}
    for att in ["dot", "additiv"]:
        m = orgat(att, 0, 300, Xo, Yo)
        b, w = m.greedy(Xt, Yt.shape[1] - 1)
        em = np.mean([matnga(q) == s for q, (_, s) in zip(b.tolist(), test)])
        n = sum(p.numel() for p in m.parameters())
        modellar[att] = (m, b, w)
        print(f"  {str(att):<8} parametr {n:>6}   aniq moslik {em:.4f}")

    print("\n=== 2. Attention og'irliklari (additiv, x100) ===")
    m, b, w = modellar["additiv"]
    i = 3
    kir, togri = test[i]
    chiq = matnga(b[i].tolist())
    print(f"  {kir} -> {chiq}")
    print("       " + "".join(f"{c:>4}" for c in kir))
    for t, c in enumerate(chiq):
        qator = (w[i, t] * 100).round().int().tolist()
        belgi = [f"{v:>4}" for v in qator]
        eng = int(w[i, t].argmax())
        belgi[eng] = f"{'[' + str(qator[eng]) + ']':>4}"
        print(f"  {c!r:<4} " + "".join(belgi))
    print("  [..] - shu chiqish belgisi eng ko'p qaragan kirish belgisi")

    print("\n=== 3. Hizalanishni raqam bilan tekshirish (1000 test) ===")
    hisob = {}
    for k, (_, s) in enumerate(test):
        if matnga(b[k].tolist()) != s:
            continue
        for t, (tur, joy) in enumerate(kutilgan_joy(s)):
            if joy is None:
                continue
            eng = int(w[k, t].argmax())
            massa = w[k, t, sorted(joy)].sum().item()
            qoshni = {p + d for p in joy for d in (-1, 0, 1)}
            hisob.setdefault(tur, []).append((eng in joy, massa, eng in qoshni))
    print(f"  {'qism':<6} {'belgilar':>9} {'argmax aniq':>12} {'argmax +-1':>11} "
          f"{'massa':>7}")
    for tur in ["kun", "oy", "yil"]:
        a = np.array(hisob[tur])
        print(f"  {tur:<6} {len(a):>9} {a[:, 0].mean():>12.3f} "
              f"{a[:, 2].mean():>11.3f} {a[:, 1].mean():>7.3f}")
    print("  argmax +-1 - kutilgan pozitsiya yoki uning qo'shnisi")
    print("  massa - kutilgan pozitsiyalardagi og'irliklar yig'indisi")
    print("  yil: j-raqam kirishning AYNAN j-pozitsiyasiga qarashi tekshirildi")
    tasodif = np.mean([len(j) / 10 for j in [{8, 9}, {5, 6}, {0}]])
    print(f"  tasodifiy qarashda kutilgan ulush ~{tasodif:.2f}")
    print("  ⭐ Attention - o'rgatilgan 'yumshoq hizalash': kim qayerga qaraydi")


if __name__ == "__main__":
    main()

=== 1. Ikki attention turi, bir xil byudjet (300 qadam) ===
  dot      parametr  76642   aniq moslik 1.0000
  additiv  parametr  84898   aniq moslik 1.0000

=== 2. Attention og'irliklari (additiv, x100) ===
  1981-11-13 -> 13-noyabr 1981
          1   9   8   1   -   1   1   -   1   3
  '1'    13   2   1   6   1  13   5   3[43]  13
  '3'     1   0   0   1   1   1   0   1   4[90]
  '-'     3   2   2   2[41]   8   6  27   6   4
  'n'     3   0   0   0   1[67]  18   4   5   1
  'o'     2   0   0   0  12  11  28[43]   2   1
  'y'    11   1   2   2  19[23]  16  20   4   2
  'a'     5   1   1   1[47]  11   8  24   2   1
  'b'     3   1   1   0[41]   9   4  36   3   2
  'r'     3   1   3   0[43]   7   3  37   3   1
  ' '     6   1   4   1[38]  13   2  27   5   2
  '1'     0[79]   1  18   0   0   0   0   0   0
  '9'     0[77]  11   8   2   0   1   1   0   0
  '8'     0   4[88]   6   2   0   0   0   0   0
  '1'     2   6   6[67]   6   2   3   2   2   3
  [..] - shu chiqish belgisi eng ko'p qaragan kirish belgisi

=== 3. Hizalanishni raqam bilan tekshirish (1000 test) ===
  qism    belgilar  argmax aniq  argmax +-1   massa
  kun         1710        0.980       0.995   0.847
  oy          5271        0.493       1.000   0.494
  yil         4000        0.874       1.000   0.699
  argmax +-1 - kutilgan pozitsiya yoki uning qo'shnisi
  massa - kutilgan pozitsiyalardagi og'irliklar yig'indisi
  yil: j-raqam kirishning AYNAN j-pozitsiyasiga qarashi tekshirildi
  tasodifiy qarashda kutilgan ulush ~0.17
  ⭐ Attention - o'rgatilgan 'yumshoq hizalash': kim qayerga qaraydi

"""Teacher forcing va greedy inference: xato zanjiri va buzuq prefiks tajribasi."""

import calendar

import numpy as np
import torch
import torch.nn as nn

OYLAR = ["yanvar", "fevral", "mart", "aprel", "may", "iyun", "iyul", "avgust",
         "sentabr", "oktabr", "noyabr", "dekabr"]
KIR = ["<pad>"] + list("0123456789-")
CHIQ = ["<pad>", "<bos>", "<eos>"] + sorted(set("0123456789- " + "".join(OYLAR)))
PAD, BOS, EOS = 0, 1, 2
k2i = {c: i for i, c in enumerate(KIR)}
c2i = {c: i for i, c in enumerate(CHIQ)}


def sanalar(n, seed):
    rng = np.random.default_rng(seed)
    juftlar = []
    for _ in range(n):
        yil = int(rng.integers(1950, 2050))
        oy = int(rng.integers(1, 13))
        kun = int(rng.integers(1, calendar.monthrange(yil, oy)[1] + 1))
        juftlar.append((f"{yil}-{oy:02d}-{kun:02d}", f"{kun}-{OYLAR[oy - 1]} {yil}"))
    return juftlar


def tensorla(juftlar):
    X = torch.tensor([[k2i[c] for c in a] for a, _ in juftlar])
    T = max(len(b) for _, b in juftlar) + 2
    Y = torch.full((len(juftlar), T), PAD)
    for i, (_, b) in enumerate(juftlar):
        s = [BOS] + [c2i[c] for c in b] + [EOS]
        Y[i, :len(s)] = torch.tensor(s)
    return X, Y


class Seq2seq(nn.Module):
    """attention: None, 'dot' yoki 'additiv' (Bahdanau skori)."""

    def __init__(self, attention=None, h=64, d=32):
        super().__init__()
        self.attention = attention
        self.emb_kir = nn.Embedding(len(KIR), d)
        self.emb_chiq = nn.Embedding(len(CHIQ), d, padding_idx=PAD)
        self.encoder = nn.GRU(d, h, batch_first=True, bidirectional=True)
        self.enc_proj = nn.Linear(2 * h, h)
        self.decoder = nn.GRU(d, h, batch_first=True)
        if attention == "additiv":
            self.W_s = nn.Linear(h, h, bias=False)
            self.W_h = nn.Linear(h, h, bias=False)
            self.v = nn.Linear(h, 1, bias=False)
        if attention is not None:
            self.birlash = nn.Linear(2 * h, h)
        self.chiqish = nn.Linear(h, len(CHIQ))

    def kodla(self, X):
        o, h = self.encoder(self.emb_kir(X))
        enc = self.enc_proj(o)                              # (B, T_kir, h)
        h0 = torch.tanh(self.enc_proj(torch.cat([h[0], h[1]], 1)))
        return enc, h0.unsqueeze(0)

    def dekod(self, y_kir, h, enc):
        s, h = self.decoder(self.emb_chiq(y_kir), h)       # (B, t, h)
        if self.attention is None:
            return self.chiqish(s), h, None
        if self.attention == "dot":
            skor = s @ enc.transpose(1, 2)
        else:
            skor = self.v(torch.tanh(self.W_s(s)[:, :, None]
                                     + self.W_h(enc)[:, None])).squeeze(3)
        w = torch.softmax(skor, dim=2)                      # (B, t, T_kir)
        z = torch.tanh(self.birlash(torch.cat([s, w @ enc], 2)))
        return self.chiqish(z), h, w

    def forward(self, X, Y_kir):
        enc, h = self.kodla(X)
        return self.dekod(Y_kir, h, enc)

    @torch.no_grad()
    def greedy(self, X, maks):
        enc, h = self.kodla(X)
        y = torch.full((len(X), 1), BOS)
        chiq, ogirlik = [], []
        for _ in range(maks):
            logit, h, w = self.dekod(y, h, enc)
            y = logit[:, -1].argmax(1, keepdim=True)
            chiq.append(y)
            if w is not None:
                ogirlik.append(w[:, -1])
        return torch.cat(chiq, 1), (torch.stack(ogirlik, 1) if ogirlik else None)


def orgat(attention, seed, qadamlar, X, Y, kuzatuv=None):
    torch.manual_seed(seed)
    model = Seq2seq(attention)
    opt = torch.optim.Adam(model.parameters(), lr=0.003)
    g = torch.Generator().manual_seed(seed)
    for q in range(qadamlar):
        idx = torch.randint(0, len(X), (64,), generator=g)
        logit, _, _ = model(X[idx], Y[idx, :-1])
        loss = nn.functional.cross_entropy(logit.reshape(-1, len(CHIQ)),
                                           Y[idx, 1:].reshape(-1),
                                           ignore_index=PAD)
        opt.zero_grad()
        loss.backward()
        nn.utils.clip_grad_norm_(model.parameters(), 1.0)
        opt.step()
        if kuzatuv is not None:
            kuzatuv(q + 1, model)
    model.eval()
    return model


def matnga(qator):
    s = []
    for i in qator:
        if i == EOS:
            break
        s.append(CHIQ[i])
    return "".join(s)


@torch.no_grad()
def davom_ettir(model, X, prefiks, maks):
    """BOS + berilgan prefiks (teacher forcing) dan keyin greedy davom."""
    enc, h = model.kodla(X)
    kir = torch.cat([torch.full((len(X), 1), BOS), prefiks], 1)
    logit, h, _ = model.dekod(kir, h, enc)
    y = logit[:, -1].argmax(1, keepdim=True)
    chiq = [y]
    for _ in range(maks - prefiks.shape[1] - 1):
        logit, h, _ = model.dekod(y, h, enc)
        y = logit[:, -1].argmax(1, keepdim=True)
        chiq.append(y)
    return torch.cat([prefiks] + chiq, 1)


def main() -> None:
    torch.set_num_threads(1)
    Xo, Yo = tensorla(sanalar(5000, 0))
    test = sanalar(1000, 1)
    Xt, Yt = tensorla(test)
    model = orgat("dot", 0, 150, Xo, Yo)       # ataylab qisqa o'rgatilgan
    nishon = Yt[:, 1:]
    maska = nishon != PAD

    print("=== 1. Teacher forcing va greedy: bir model, ikki rejim ===")
    with torch.no_grad():
        logit, _, _ = model(Xt, Yt[:, :-1])
    tf = logit.argmax(2)
    gr, _ = model.greedy(Xt, nishon.shape[1])
    tf_tok = (tf == nishon)[maska].float().mean().item()
    gr_tok = (gr == nishon)[maska].float().mean().item()
    tf_em = ((tf == nishon) | ~maska).all(1).float().mean().item()
    gr_em = np.mean([matnga(q) == s for q, (_, s) in zip(gr.tolist(), test)])
    print(f"  {'':<16} {'token aniqligi':>15} {'aniq moslik':>12}")
    print(f"  {'teacher forcing':<16} {tf_tok:>15.4f} {tf_em:>12.4f}")
    print(f"  {'greedy':<16} {gr_tok:>15.4f} {gr_em:>12.4f}")
    print("  TF da har qadamda to'g'ri prefiks beriladi - bu 'imtihonda shpargalka'")
    if abs(tf_em - gr_em) < 1e-6:
        print("  aniq moslik teng: TF da hammasi to'g'ri bo'lsa, greedy ham aynan"
              " shu yo'ldan yuradi")

    print("\n=== 2. Xato zanjiri: birinchi xatodan keyin ===")
    togri_gr = (gr == nishon) | ~maska
    xatoli = ~togri_gr.all(1)
    birinchi = (~togri_gr).float().argmax(1)
    keyin_tf, keyin_gr = [], []
    for k in torch.where(xatoli)[0].tolist():
        j = int(birinchi[k])
        m = maska[k].clone()
        m[:j + 1] = False
        if m.any():
            keyin_gr.append((gr[k] == nishon[k])[m].float().mean().item())
            keyin_tf.append((tf[k] == nishon[k])[m].float().mean().item())
    print(f"  greedy da xatoli jumlalar: {int(xatoli.sum())} / {len(test)}")
    print(f"  birinchi xato pozitsiyasi (o'rtacha): "
          f"{birinchi[xatoli].float().mean().item():.1f}")
    print(f"  birinchi xatodan KEYINGI tokenlar aniqligi:")
    print(f"    greedy (o'z xatosi ustiga): {np.mean(keyin_gr):.4f}")
    print(f"    teacher forcing (shu pozitsiyalar): {np.mean(keyin_tf):.4f}")

    print("\n=== 3. Tajriba: birinchi belgini ataylab buzish ===")
    toliq = torch.where(togri_gr.all(1))[0]
    prefiks = nishon[toliq, :1]
    buzuq = torch.where(prefiks == c2i["1"], c2i["2"], c2i["1"])
    davom = davom_ettir(model, Xt[toliq], buzuq, nishon.shape[1])
    kun_ok, qolgan_ok = [], []
    for q, k in zip(davom.tolist(), toliq.tolist()):
        chiq, togri = matnga(q), test[k][1]
        kun_ok.append(chiq.split("-")[0] == togri.split("-")[0])
        qolgan_ok.append(chiq.split("-", 1)[-1] == togri.split("-", 1)[1])
    print(f"  {len(toliq)} ta to'g'ri jumlada birinchi belgi almashtirildi")
    for q, k in list(zip(davom.tolist(), toliq.tolist()))[:4]:
        print(f"    {test[k][0]}:  to'g'ri {test[k][1]:<16} buzuq boshdan -> "
              f"{matnga(q)}")
    print(f"  kun baribir to'g'ri chiqdi (tuzatdi): {np.mean(kun_ok):.1%}")
    print(f"  oy va yil to'g'ri qoldi:              {np.mean(qolgan_ok):.1%}")
    print("  bitta noto'g'ri token - model uni 'haqiqat' deb davom ettiradi")

    print("\n=== 4. Inference sikli: <eos> da to'xtash ===")
    uzunlik = [len(matnga(q)) for q in gr.tolist()]
    eos_bor = (gr == EOS).any(1).float().mean().item()
    print(f"  maks qadam {nishon.shape[1]}, <eos> chiqargan jumlalar {eos_bor:.1%}")
    print(f"  chiqish uzunligi {min(uzunlik)}..{max(uzunlik)} belgi")
    print("  ⭐ O'rgatish - teacher forcing, bashorat - o'z chiqishi ustiga: farqni o'lchang")


if __name__ == "__main__":
    main()

=== 1. Teacher forcing va greedy: bir model, ikki rejim ===
                    token aniqligi  aniq moslik
  teacher forcing           0.9866       0.8160
  greedy                    0.9376       0.8160
  TF da har qadamda to'g'ri prefiks beriladi - bu 'imtihonda shpargalka'
  aniq moslik teng: TF da hammasi to'g'ri bo'lsa, greedy ham aynan shu yo'ldan yuradi

=== 2. Xato zanjiri: birinchi xatodan keyin ===
  greedy da xatoli jumlalar: 184 / 1000
  birinchi xato pozitsiyasi (o'rtacha): 2.6
  birinchi xatodan KEYINGI tokenlar aniqligi:
    greedy (o'z xatosi ustiga): 0.6053
    teacher forcing (shu pozitsiyalar): 0.9985

=== 3. Tajriba: birinchi belgini ataylab buzish ===
  816 ta to'g'ri jumlada birinchi belgi almashtirildi
    1997-07-24:  to'g'ri 24-iyul 1997     buzuq boshdan -> 14-iyul 1997
    2045-01-05:  to'g'ri 5-yanvar 2045    buzuq boshdan -> 15-yanvar 2045
    2032-12-08:  to'g'ri 8-dekabr 2032    buzuq boshdan -> 18-dekabr 2032
    1981-11-13:  to'g'ri 13-noyabr 1981   buzuq boshdan -> 21-noyabr 1981
  kun baribir to'g'ri chiqdi (tuzatdi): 0.0%
  oy va yil to'g'ri qoldi:              98.0%
  bitta noto'g'ri token - model uni 'haqiqat' deb davom ettiradi

=== 4. Inference sikli: <eos> da to'xtash ===
  maks qadam 16, <eos> chiqargan jumlalar 100.0%
  chiqish uzunligi 9..15 belgi
  ⭐ O'rgatish - teacher forcing, bashorat - o'z chiqishi ustiga: farqni o'lchang

logit = model(X, Y); loss = ce(logit, Y)                           # ⚠️
logit = model(X, Y[:, :-1]); loss = ce(logit, Y[:, 1:])            # ✅

nn.functional.cross_entropy(logit, y)                              # ⚠️
nn.functional.cross_entropy(logit, y, ignore_index=PAD)            # ✅

w = torch.softmax(s @ enc.transpose(1, 2), dim=2)                  # ⚠️
skor = skor.masked_fill(~maska[:, None, :], float("-inf"))         # ✅

w = torch.softmax(skor, dim=1)       # chiqish o'qi bo'yicha       # ⚠️
w = torch.softmax(skor, dim=2)       # kirish pozitsiyalari bo'yicha # ✅

aniq = (model(X, Y[:, :-1]).argmax(2) == Y[:, 1:]).float().mean() # ⚠️
chiq = model.greedy(X, maks); aniq = (chiq == nishon).all(1).mean()  # ✅

matn = "".join(CHIQ[i] for i in qator)                             # ⚠️
matn = matnga(qator)                 # <eos> da to'xtaydi           # ✅

logit, _, _ = dekod(y, h0, enc)      # har safar boshlang'ich holat # ⚠️
logit, h, _ = dekod(y, h, enc)                                     # ✅

1.  # Y = <bos> a b c <eos> bo'lsa decoder kirishi va nishoni?
2.  # attention siz decoder encoderdan nimani oladi?
3.  # dot-product skor formulasi?
4.  # additiv (Bahdanau) skor formulasi?
5.  # skor shakli: s (B, t, h), enc (B, T, h)?
6.  # softmax qaysi o'q bo'yicha?
7.  # padding pozitsiyalariga qanday og'irlik berilishi kerak?
8.  # kontekst vektor formulasi?
9.  # teacher forcing da decoder kirishi nima?
10. # greedy da decoder kirishi nima?
11. # nima uchun TF va greedy aniq moslik doim teng?
12. # sanada attention siz model qaysi qismni eng ko'p xato qiladi va nima uchun?

1.  logit, _, _ = model(X, Y)
    loss = nn.functional.cross_entropy(logit.reshape(-1, V), Y.reshape(-1))

2.  w = torch.softmax(s @ enc.transpose(1, 2), dim=1)

3.  aniq = (model(X, Y[:, :-1])[0].argmax(2) == Y[:, 1:]).float().mean()
    print("model tayyor" if aniq > 0.85 else "yana o'rgatamiz")

4.  for _ in range(maks):
        logit, _, _ = model.dekod(y, h0, enc)
        y = logit[:, -1].argmax(1, keepdim=True)

5.  kontekst = enc.mean(1)          # attention "soddalashtirilgan"