25.1-dars: LLM nima

24.9-DARSDAGI GPT:
  token + pozitsiya embedding -> N ta kauzal blok -> LayerNorm -> Linear(d, V)
  maqsad: loss = cross_entropy(logit[:, t], x[:, t+1])
  bizniki: 2 qatlam, d = 48, belgi darajasi, 43 133 parametr

ZAMONAVIY LLM - O'SHA SKELET, BOSHQA MASSHTAB:
  qatlamlar       2        ->  30-100+
  kenglik d       48       ->  4 000-16 000
  lug'at V        29 belgi ->  30 000-200 000 subword token (BPE, 23.3)
  kontekst L      48       ->  o'n minglab - yuz minglab token
  parametrlar     43 ming  ->  milliardlar - yuzlab milliard
  o'quv tokenlari ~0.2 mln ->  trillionlab

MAYDA ARXITEKTURA FARQLARI (g'oya o'zgarmaydi):
  pozitsiya: o'rgatiladigan embedding o'rniga aylanma (RoPE) kodlash
  normallash: LayerNorm o'rniga RMSNorm, "pre-norm"
  FFN: GELU o'rniga "gated" variantlar (SwiGLU)
  attention: bir nechta bosh bitta K/V ni bo'lishadi (GQA) - KV-cache kichrayadi
  ba'zilarida "mixture of experts": har token FFN larning faqat bir qismidan o'tadi

1. PRETRAINING (oldindan o'rgatish)            -> BASE MODEL
   ma'lumot: trillionlab token (veb, kitoblar, kod), yorliqsiz
   maqsad:   keyingi token (24.9 bilan aynan bir xil)
   natija:   tilni, faktlarni, uslublarni biladi; lekin "hujjatni davom ettiradi"
   narx:     hisobning 90%+ qismi shu yerda

2. INSTRUCTION TUNING (SFT, supervised fine-tuning)   -> INSTRUCT MODEL
   ma'lumot: o'n minglab (ko'rsatma, javob) juftlari, odamlar yozgan yoki tanlagan
   maqsad:   keyingi token, lekin loss FAQAT javob tokenlarida (savol niqoblanadi)
   natija:   savolga JAVOB beradi, formatga amal qiladi, suhbat rollarini biladi

3. AFZALLIK BO'YICHA MOSLASH (RLHF / DPO)         -> "CHAT" MODEL
   ma'lumot: bir savolga ikki javob + qaysi biri yaxshi (odam yoki model belgilaydi)
   maqsad:   yaxshi javob ehtimolini oshirish, yomonini pasaytirish,
             boshlang'ich modeldan juda uzoqlashmaslik (KL cheklov)
   natija:   foydaliroq, xavfsizroq, uslubi mos javoblar

1-MISOL NATIJASI (15 ta SFT ko'rmagan ism x 2 seed):
  usul              format (javob:)   to'g'ri kasb
  base zero-shot         0.0%            0.0%     <- savolni savol bilan davom ettiradi
  base 2 misolli       100.0%           76.7%     <- in-context learning
  SFT zero-shot        100.0%          100.0%

BASE MODEL:
  "savol : gulnora kim ?"  ->  "savol : temur kim"      (1-misol, seed 0)
  hujjat qanday davom etishi ehtimolli bo'lsa, shunday davom ettiradi
  foydali: matn davom ettirish, few-shot, o'z fine-tuning ingiz uchun asos

INSTRUCT / CHAT MODEL:
  "savol : gulnora kim ?"  ->  "javob : oshpaz ."
  maxsus format: rollar (system, user, assistant), maxsus tokenlar
  API orqali ishlatiladigan modellarning deyarli hammasi shu turda

AMALIY OQIBAT:
  base modelga "ko'rsatma" berish ishonchsiz - uni hujjat ko'rinishida bering
  ("Savol: ... Javob:" shabloni yoki bir nechta misol)
  instruct modelga esa aniq ko'rsatma bering (25.4-dars)

KETMA-KETLIK:  <d> savol : laylo kim ?  javob : dehqon .
               |------ savol ------|   |--- javob ---|
YORLIQLAR:     -100 -100 -100 ... -100  javob : dehqon .
                (loss hisoblanmaydi)    (loss faqat shu yerda)

NEGA NIQOBLAYMIZ:
  model savol matnini "yodlashi" kerak emas, faqat javob yozishni o'rganadi
  F.cross_entropy(..., ignore_index=-100) - PyTorch standart usuli

AMALDA:
  suhbat shabloni: har rolning boshi/oxiri maxsus tokenlar bilan
  loss faqat "assistant" qismlarida
  kichik lr va kam qadam - pretraining bilimini buzmaslik uchun
  (katastrofik unutish, 24.10-dars)

PARAMETRLAR 24.11-bob:  N ~ 12 * L * d^2  +  V * d
  2-misol: L=6, d=256 da bloklar 4 738 560, formula 4 718 592 (farq 0.4%)

HISOB:  C ~ 6 * N * D  FLOP   (D - o'quv tokenlari soni)
  forward ~2N FLOP/token, backward ~4N FLOP/token
  2-misol: torch FlopCounterMode o'lchagani == 6*N*D (nisbat 1.00)
  attention o'z hissasini qo'shadi (12 * L * T * d / token): kichik T da 5-10%

MA'LUMOT VA MODEL NISBATI ("hisob-optimal" qoidasi, taxminiy):
  berilgan hisob byudjetida D ~ 20 * N token atrofida eng past loss
  amalda ko'p modellar undan ANCHA ko'p tokenda o'rgatiladi -
  chunki kichik modelni ishlatish (inference) arzonroq

2-MISOL (FARAZIY apparat: 1e15 FLOP/s, 40% foydalanish):
  7B sinf:  N = 6.7B, D = 134B token, C = 5.4e21 FLOP
            1 tezlatgichda 156 kun, 1024 tada 0.2 kun
  70B sinf: C = 5.1e23 FLOP - 1024 tezlatgichda 14.5 kun

XOTIRA (faqat vaznlar):   N x bayt
                 fp32  bf16  int8  int4
  7B sinf (GB)   26.8  13.4   6.7   3.4
  o'rgatish (AdamW, fp32 master + m + v + grad) ~16 bayt/param -> 107 GB
  KV-cache: 2 * L * d * bayt har token -> 7B sinfda 0.52 MB/token,
            8000 token 4.2 GB, 32 parallel so'rov 134 GB

SIMMETRIK KVANTLASH:
  w ~ shkala * q,  q - butun son [-(2^(b-1)-1), 2^(b-1)-1]
  shkala = max|w| / (2^(b-1)-1)       int8: 127 daraja, int4: 7 daraja

SHKALA QAYERDAN OLINADI:
  butun matritsa bo'yicha bitta shkala - sodda, lekin bitta katta qiymat
  hammasini buzadi
  qator (kanal) bo'yicha - har chiqish kanali o'z shkalasi bilan

2-MISOL (512x512, x @ w nisbiy xatosi):
                          int8 butun  int8 qator  int4 butun  int4 qator
  oddiy (normal)            0.0105      0.0074      0.1899      0.1343
  20 ta chetli qiymat       0.1025      0.0217      0.9128      0.2170

IN-CONTEXT LEARNING (ICL, kontekstdan o'rganish):
  vaznlar O'ZGARMAYDI; model promptdagi misollardan vazifani "tushunadi"
  1-misol: base model, prompt boshida 2 ta savol-javob:
    format 0.0% -> 100.0%, to'g'ri kasb 0.0% -> 76.7%

NEGA ISHLAYDI:
  pretraining korpusida FAQ hujjatlari bor edi (10%):
    "savol : X kim ? javob : Y . savol : Z kim ? javob : ..."
  model "bu hujjat FAQ turida - demak savoldan keyin javob keladi" degan
  qonuniyatni o'rgangan; misollar hujjat turini aniqlashga yordam beradi

"EMERGENT" (paydo bo'luvchi) QOBILIYATLAR:
  kuzatuv: ba'zi vazifalar kichik modellarda deyarli nol, ma'lum masshtabdan
  keyin keskin o'sadi (ko'p bosqichli arifmetika, ko'rsatmaga amal qilish)
  ehtiyot: "keskinlik" ko'pincha METRIKADAN - "aniq moslik" kabi hammasi-yoki-hech
  metrika silliq yaxshilanishni ham sakrash qilib ko'rsatadi;
  token darajasidagi loss bilan qaralsa, o'sish silliqroq ko'rinadi

SABAB - MAQSAD FUNKSIYASINING O'ZIDA:
  model P(keyingi token | kontekst) ni o'rganadi
  "X kasbi ..." dan keyin korpusda DOIM kasb keladi
  -> X haqida fakt bo'lmasa ham model kasb aytadi: "bilmayman" davomi
     korpusda yo'q edi

3-MISOL (40 ism, fakt 0/1/3/10 marta, 2 seed):
  fakt soni  kasbga massa  ishonch  aniqlik  ishonch - aniqlik
      0          0.987      0.727    0.150       +0.577   <- to'qish
      1          0.979      0.679    0.800       -0.121
      3          0.988      0.857    0.900       -0.043
     10          0.990      0.983    1.000       -0.017
  ko'rilmagan faktda aniqlik tasodif darajasida (1/6), ishonch esa 0.727

KALIBRLASH:
  yaxshi kalibrlangan model: "0.8 ishonch" bilan aytganlarining ~80% i to'g'ri
  ECE = sum (ulush * |o'rtacha ishonch - aniqlik|) ishonch oraliqlari bo'yicha
  3-misol: ECE 0.099; eng katta xato - hech ko'rilmagan faktlarda

KAMAYTIRISH YO'LLARI (keyingi darslarda):
  RAG - javobni topilgan hujjatga bog'lash (25.8-25.9)
  "bilmayman" javoblarini SFT/afzallik ma'lumotiga qo'shish
  manbani keltirishni talab qilish va uni avtomatik tekshirish
  bir necha namunaning o'zaro mosligini tekshirish (25.10)

RLHF (klassik, uch qadam):
  1. afzallik ma'lumoti: (savol, javob_A, javob_B, qaysi yaxshi)
  2. mukofot modeli r(x, y): "yaxshi" javobga yuqori ball berishni o'rganadi
  3. RL (PPO): siyosat pi mukofotni oshiradi, KL(pi || pi_ref) jarimasi bilan
  murakkab: to'rtta model (siyosat, reference, mukofot, qiymat), beqaror

DPO (Direct Preference Optimization):
  mukofot modelisiz, to'g'ridan-to'g'ri juftlar ustida:
  loss = -log sigmoid( beta * [ (log pi(y_w|x) - log ref(y_w|x))
                              - (log pi(y_l|x) - log ref(y_l|x)) ] )
  y_w - tanlangan, y_l - rad etilgan javob; ref - muzlatilgan SFT model
  beta - KL "tasmasi": katta beta -> ref dan kam uzoqlashadi

4-MISOL (15 juft: qisqa javob > "suvli" javob; B guruhda o'lchov, 40 qadam):
  usul              P(qisqa)  KL(pi||ref)  kasb (15 tadan)
  boshlang'ich        0.444       -             15
  DPO beta=0.1        0.938     0.793           13
  DPO beta=1.0        0.798     0.357           15
  SFT (tanlanganda)   0.948     0.790           14

SAVOL                         NIMA QARAYDI
sifat                         o'z vazifangizdagi baho to'plami 25.10-bob, umumiy
                              reyting emas
narx                          kirish va chiqish tokenlari narxi 25.2-bob, hajm
kechikish                     birinchi token vaqti, token/s; interaktivmi, fondami
kontekst oynasi               hujjatlaringiz sig'adimi 25.2-bob
tillar                        o'zbekcha sifati va token "solig'i" 25.2-bob
maxfiylik va joylashuv        ma'lumot qayerga ketadi; o'z serverida kerakmi
ochiq vazn / API              fine-tuning, kvantlash imkoniyati va xizmat yuki
litsenziya                    tijorat maqsadida ishlatish shartlari

AMALIY QOIDA:
  eng kichik/arzon modeldan boshlang, o'z baho to'plamingizda o'lchang,
  yetmasa kattasiga o'ting - "eng yaxshisidan sezilarli yomon bo'lmagan
  eng arzon" (18-qismdagi qaror qoidasining o'zi)

# masshtab
N = 12 * qatlamlar * d * d + V * d            # parametrlar (taxminan)
C = 6 * N * D                                 # o'rgatish FLOP
kun = C / (choqqi_flops * foydalanish) / 86400
xotira_gb = N * bayt / 1e9                    # fp32 4, bf16 2, int8 1, int4 0.5
orgatish_gb = N * 16 / 1e9                    # AdamW: vazn + grad + m + v
kv_bayt_token = 2 * qatlamlar * d * 2         # K va V, bf16

# SFT: savol niqoblangan loss
y = [-100] * (len(savol) - 1) + javob
loss = F.cross_entropy(logit.reshape(-1, V), y.reshape(-1), ignore_index=-100)

# DPO
loss = -F.logsigmoid(beta * ((pi_w - ref_w) - (pi_l - ref_l))).mean()

# kalibrlash (ECE)
ece = sum(ulush * abs(ishonch.mean() - aniqlik.mean()) for ulush, ishonch, aniqlik in oraliqlar)

# instruct modelni API orqali chaqirish (faqat ko'rinish; kalit muhit o'zgaruvchisidan)
import anthropic

client = anthropic.Anthropic()                # ANTHROPIC_API_KEY muhitdan o'qiladi
javob = client.messages.create(
    model="claude-haiku-4-5-20251001",
    max_tokens=200,
    system="Sen kadrlar bo'limi yordamchisisan. Bilmasang, 'bilmayman' de.",
    messages=[{"role": "user", "content": "Ta'til uchun ariza qanday yoziladi?"}],
)
print(javob.content[0].text)
print(javob.usage.input_tokens, javob.usage.output_tokens)

LLM = katta masshtabdagi GPT (kauzal decoder, keyingi token)
pretraining -> base (davom ettiradi); SFT -> instruct (javob beradi); DPO/RLHF -> did
bilim pretrainingdan, format SFT dan
N ~ 12Ld^2, C ~ 6ND, xotira = N x bayt (+ KV-cache)
ICL: misollar promptda, vaznlar o'zgarmaydi
gallyutsinatsiya: ehtimolli davom; ishonch != to'g'rilik

"""Base model va instruction tuning (SFT): bir xil savolga davom yozish va javob berish."""

import copy
import math

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F

ISMLAR = ("anvar dilnoza bobur malika sardor nodira jasur zarina otabek madina "
          "sherzod gulnora farhod laylo rustam sevara aziz kamola bahrom shahlo "
          "temur nigora dilshod feruza jamshid mohira sanjar yulduz akmal lola").split()
KASBLAR = "shifokor muhandis oqituvchi dehqon haydovchi oshpaz".split()
JOYLAR = "bozorga maktabga shaharga bekatga dalaga bogga".split()
L = 24


def dunyo(seed=0):
    """Har ismga bitta kasb - bu "fakt". A guruh SFT da, B guruh faqat baholashda."""
    rng = np.random.default_rng(seed)
    kasb = {ism: str(rng.choice(KASBLAR)) for ism in ISMLAR}
    aralash = rng.permutation(ISMLAR)
    return kasb, [str(x) for x in aralash[:15]], [str(x) for x in aralash[15:]]


def fakt_gap(ism, k, rng):
    if rng.random() < 0.5:
        return f"{ism} {k} bo'lib ishlaydi ."
    return f"{ism} kasbi {k} ."


def pretrain_korpus(kasb, n_hujjat, rng):
    """Internet o'xshatmasi: fakt matnlari, savollar ro'yxati va kam uchraydigan FAQ."""
    hujjatlar = []
    for _ in range(n_hujjat):
        tur = rng.random()
        ismlar = rng.choice(ISMLAR, 4, replace=False)
        if tur < 0.60:                                     # oddiy matn
            gaplar = [fakt_gap(i, kasb[i], rng) if rng.random() < 0.7
                      else f"{i} {rng.choice(JOYLAR)} bordi ." for i in ismlar]
        elif tur < 0.90:                                   # savollar ro'yxati (test varag'i)
            gaplar = [f"savol : {i} kim ?" for i in ismlar]
        else:                                              # FAQ: savol-javob juftlari
            gaplar = [f"savol : {i} kim ? javob : {kasb[i]} ." for i in ismlar[:3]]
        hujjatlar.append("<d> " + " ".join(gaplar))
    return " ".join(hujjatlar)


class Blok(nn.Module):
    def __init__(self, d, boshlar):
        super().__init__()
        self.boshlar = boshlar
        self.ln1, self.ln2 = nn.LayerNorm(d), nn.LayerNorm(d)
        self.qkv = nn.Linear(d, 3 * d)
        self.proj = nn.Linear(d, d)
        self.ffn = nn.Sequential(nn.Linear(d, 4 * d), nn.GELU(), nn.Linear(4 * d, d))

    def forward(self, x):
        B, T, d = x.shape
        q, k, v = self.qkv(self.ln1(x)).split(d, dim=2)
        q, k, v = [t.view(B, T, self.boshlar, -1).transpose(1, 2) for t in (q, k, v)]
        o = F.scaled_dot_product_attention(q, k, v, is_causal=True)
        x = x + self.proj(o.transpose(1, 2).reshape(B, T, d))
        return x + self.ffn(self.ln2(x))


class GPT(nn.Module):
    def __init__(self, V, d=32, boshlar=4, qatlamlar=2):
        super().__init__()
        self.emb = nn.Embedding(V, d)
        self.poz = nn.Embedding(L, d)
        self.bloklar = nn.Sequential(*[Blok(d, boshlar) for _ in range(qatlamlar)])
        self.ln = nn.LayerNorm(d)
        self.chiqish = nn.Linear(d, V)

    def forward(self, x):
        x = x[:, -L:]
        h = self.emb(x) + self.poz(torch.arange(x.shape[1]))
        return self.chiqish(self.ln(self.bloklar(h)))


def pretrain(T, V, seed, qadamlar=200, B=32):
    """Keyingi token maqsadi, butun matn bo'yicha - hech qanday "vazifa" yo'q."""
    torch.manual_seed(seed)
    model = GPT(V)
    opt = torch.optim.AdamW(model.parameters(), lr=1e-2, weight_decay=0.01)
    g = torch.Generator().manual_seed(seed)
    for _ in range(qadamlar):
        bosh = torch.randint(0, len(T) - L - 1, (B,), generator=g)
        w = torch.stack([T[i:i + L + 1] for i in bosh.tolist()])
        loss = F.cross_entropy(model(w[:, :-1]).reshape(-1, V), w[:, 1:].reshape(-1))
        opt.zero_grad()
        loss.backward()
        opt.step()
    return model


def sft(model, juftlar, s2i, seed, qadamlar=60, B=16):
    """Instruction tuning: (savol, javob) juftlari, loss FAQAT javob tokenlarida."""
    V = model.chiqish.out_features
    opt = torch.optim.AdamW(model.parameters(), lr=1e-3, weight_decay=0.01)
    g = torch.Generator().manual_seed(1000 + seed)
    for _ in range(qadamlar):
        idx = torch.randint(0, len(juftlar), (B,), generator=g).tolist()
        x, y = [], []
        for i in idx:
            savol, javob = juftlar[i]
            s = [s2i[w] for w in savol.split()]
            j = [s2i[w] for w in javob.split()]
            t = s + j
            x.append(t[:-1])
            y.append([-100] * (len(s) - 1) + j)            # savol qismi niqoblangan
        loss = F.cross_entropy(model(torch.tensor(x)).reshape(-1, V),
                               torch.tensor(y).reshape(-1), ignore_index=-100)
        opt.zero_grad()
        loss.backward()
        opt.step()
    return model


@torch.no_grad()
def greedy(model, prompt, s2i, i2s, n=4):
    x = torch.tensor([[s2i[w] for w in prompt.split()]])
    for _ in range(n):
        x = torch.cat([x, model(x)[:, -1].argmax(-1, keepdim=True)], 1)
    return " ".join(i2s[i] for i in x[0, -n:].tolist())


def baholash(model, ismlar, kasb, s2i, i2s, oldin=""):
    """Har savol uchun: format (javob bilan boshladimi) va to'g'rilik (kasb to'g'rimi)."""
    fmt, togri = [], []
    for ism in ismlar:
        davom = greedy(model, f"<d> {oldin}savol : {ism} kim ?", s2i, i2s).split()
        fmt.append(float(davom[0] == "javob"))
        togri.append(float(davom[:3] == ["javob", ":", kasb[ism]]))
    return np.array(fmt), np.array(togri)


def main() -> None:
    torch.set_num_threads(1)
    kasb, A, B_guruh = dunyo()
    rng = np.random.default_rng(0)
    matn = pretrain_korpus(kasb, 3000, rng)
    sozlar = sorted(set(matn.split()) | {"javob"})
    s2i = {w: i for i, w in enumerate(sozlar)}
    i2s = {i: w for w, i in s2i.items()}
    T = torch.tensor([s2i[w] for w in matn.split()])
    sft_juftlar = [(f"<d> savol : {i} kim ?", f"javob : {kasb[i]} .") for i in A]

    print("=== 1. Ma'lumot ===")
    print(f"  pretraining: {len(T)} token, lug'at {len(sozlar)} so'z, 3000 hujjat")
    print("  hujjat turlari: 60% matn, 30% savollar ro'yxati, 10% savol-javob (FAQ)")
    print(f"  SFT: {len(sft_juftlar)} ta juft (A guruh), baholash - B guruhning 15 ismi")
    print(f"  SFT misol: {sft_juftlar[0][0][4:]} -> {sft_juftlar[0][1]}")

    natija = {k: ([], []) for k in ("base zero-shot", "base 2 misolli", "SFT zero-shot")}
    for seed in range(2):
        base = pretrain(T, len(sozlar), seed)
        oldin = "".join(f"savol : {a} kim ? javob : {kasb[a]} . " for a in A[:2])
        for nom, m, o in (("base zero-shot", base, ""), ("base 2 misolli", base, oldin)):
            f_, t_ = baholash(m, B_guruh, kasb, s2i, i2s, o)
            natija[nom][0].append(f_)
            natija[nom][1].append(t_)
        tuned = sft(copy.deepcopy(base), sft_juftlar, s2i, seed)
        if seed == 0:
            base0, tuned0 = base, tuned
        f_, t_ = baholash(tuned, B_guruh, kasb, s2i, i2s)
        natija["SFT zero-shot"][0].append(f_)
        natija["SFT zero-shot"][1].append(t_)

    print("\n=== 2. Bir savolga uch xil javob (seed 0) ===")
    ism = B_guruh[0]
    fs = f"savol : {A[0]} kim ? javob : {kasb[A[0]]} . "
    print(f"  savol: 'savol : {ism} kim ?'   (to'g'ri kasb: {kasb[ism]})")
    print(f"  base, zero-shot : {greedy(base0, f'<d> savol : {ism} kim ?', s2i, i2s)}")
    print(f"  base, 1 misolli : {greedy(base0, f'<d> {fs}savol : {ism} kim ?', s2i, i2s)}")
    print(f"  SFT dan keyin   : {greedy(tuned0, f'<d> savol : {ism} kim ?', s2i, i2s)}")

    print("\n=== 3. B guruh (SFT ko'rmagan 15 ism) x 2 seed ===")
    print("  usul             format (javob:)   to'g'ri kasb")
    for nom, (f_, t_) in natija.items():
        print(f"  {nom:<16} {np.mean(f_):>13.1%} {np.mean(t_):>14.1%}")

    print("\n=== 4. Juftlashgan farq (30 = 15 savol x 2 seed) ===")
    sft_t = np.concatenate(natija["SFT zero-shot"][1])
    for nom in ("base zero-shot", "base 2 misolli"):
        d = sft_t - np.concatenate(natija[nom][1])
        se = d.std(ddof=1) / math.sqrt(len(d))
        hukm = ("SFT sezilarli yaxshi" if d.mean() > 2 * se else
                "SFT sezilarli yomon" if d.mean() < -2 * se else "sezilarli farq yo'q")
        print(f"  SFT - {nom:<15}: {d.mean():+.3f}, SE {se:.3f} -> {hukm}")
    if np.mean(natija["base zero-shot"][0]) < 0.5:
        print("  base model savolni ko'pincha 'savol' bilan davom ettiradi - u javob bermaydi")
    print("  ⭐ B guruh faktlari SFT da YO'Q edi: bilim pretrainingdan, format SFT dan")


if __name__ == "__main__":
    main()

=== 1. Ma'lumot ===
  pretraining: 60024 token, lug'at 53 so'z, 3000 hujjat
  hujjat turlari: 60% matn, 30% savollar ro'yxati, 10% savol-javob (FAQ)
  SFT: 15 ta juft (A guruh), baholash - B guruhning 15 ismi
  SFT misol: savol : laylo kim ? -> javob : dehqon .

=== 2. Bir savolga uch xil javob (seed 0) ===
  savol: 'savol : gulnora kim ?'   (to'g'ri kasb: oshpaz)
  base, zero-shot : savol : temur kim
  base, 1 misolli : javob : oshpaz .
  SFT dan keyin   : javob : oshpaz .

=== 3. B guruh (SFT ko'rmagan 15 ism) x 2 seed ===
  usul             format (javob:)   to'g'ri kasb
  base zero-shot            0.0%           0.0%
  base 2 misolli          100.0%          76.7%
  SFT zero-shot           100.0%         100.0%

=== 4. Juftlashgan farq (30 = 15 savol x 2 seed) ===
  SFT - base zero-shot : +1.000, SE 0.000 -> SFT sezilarli yaxshi
  SFT - base 2 misolli : +0.233, SE 0.079 -> SFT sezilarli yaxshi
  base model savolni ko'pincha 'savol' bilan davom ettiradi - u javob bermaydi
  ⭐ B guruh faktlari SFT da YO'Q edi: bilim pretrainingdan, format SFT dan

"""Masshtab hisob-kitobi: parametrlar, 6ND FLOP, xotira va kvantlash xatosi."""

import warnings

import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.flop_counter import FlopCounterMode

L_KONTEKST = 64

# FARAZIY konfiguratsiyalar: qatlamlar, d, lug'at (haqiqiy modellar boshqacha bo'lishi mumkin)
KONFIG = {
    "kichik (125M sinf)": (12, 768, 50000),
    "o'rta (1B sinf)": (16, 2048, 64000),
    "katta (7B sinf)": (32, 4096, 64000),
    "juda katta (70B sinf)": (80, 8192, 128000),
}
# FARAZIY apparat: bitta tezlatgichning cho'qqi tezligi va real foydalanish ulushi
CHOQQI_FLOPS = 1e15
FOYDALANISH = 0.40


class Blok(nn.Module):
    def __init__(self, d, boshlar):
        super().__init__()
        self.boshlar = boshlar
        self.ln1, self.ln2 = nn.LayerNorm(d), nn.LayerNorm(d)
        self.qkv = nn.Linear(d, 3 * d)
        self.proj = nn.Linear(d, d)
        self.ffn = nn.Sequential(nn.Linear(d, 4 * d), nn.GELU(), nn.Linear(4 * d, d))

    def forward(self, x):
        B, T, d = x.shape
        q, k, v = self.qkv(self.ln1(x)).split(d, dim=2)
        q, k, v = [t.view(B, T, self.boshlar, -1).transpose(1, 2) for t in (q, k, v)]
        o = F.scaled_dot_product_attention(q, k, v, is_causal=True)
        x = x + self.proj(o.transpose(1, 2).reshape(B, T, d))
        return x + self.ffn(self.ln2(x))


class GPT(nn.Module):
    def __init__(self, V, d, qatlamlar, boshlar=4):
        super().__init__()
        self.emb = nn.Embedding(V, d)
        self.poz = nn.Embedding(L_KONTEKST, d)
        self.bloklar = nn.Sequential(*[Blok(d, boshlar) for _ in range(qatlamlar)])
        self.ln = nn.LayerNorm(d)
        self.chiqish = nn.Linear(d, V, bias=False)

    def forward(self, x):
        h = self.emb(x) + self.poz(torch.arange(x.shape[1]))
        return self.chiqish(self.ln(self.bloklar(h)))


def parametrlar(qatlam, d, V):
    """12*L*d^2 bloklar + V*d embedding (chiqish qatlami embedding bilan bog'langan deb)."""
    return 12 * qatlam * d * d, V * d


def odam(n, birlik=""):
    for chegara, qoshimcha in ((1e12, "T"), (1e9, "B"), (1e6, "M"), (1e3, "K")):
        if n >= chegara:
            return f"{n / chegara:.1f}{qoshimcha}{birlik}"
    return f"{n:.0f}{birlik}"


def kvantla(w, bit, qator_boyicha):
    """Simmetrik kvantlash: w ~ shkala * butun_son, butun son [-(2^(b-1)-1), 2^(b-1)-1]."""
    qmax = 2 ** (bit - 1) - 1
    maks = w.abs().amax(dim=1, keepdim=True) if qator_boyicha else w.abs().max()
    shkala = maks / qmax
    return torch.clamp(torch.round(w / shkala), -qmax, qmax) * shkala


def main() -> None:
    torch.set_num_threads(1)
    torch.manual_seed(0)

    print("=== 1. Parametr soni: formula va torch sanog'i ===")
    for q, d, V in ((2, 64, 1000), (4, 128, 5000), (6, 256, 8000)):
        m = GPT(V, d, q)
        sanoq = sum(p.numel() for n, p in m.named_parameters() if n.startswith("bloklar"))
        blok, _ = parametrlar(q, d, V)
        print(f"  L={q} d={d:>3}: bloklar {sanoq:>9,}, 12*L*d^2 = {blok:>9,}"
              f"  (farq {abs(sanoq - blok) / sanoq:.1%})")

    print("\n=== 2. 6*N*D: torch FLOP hisoblagichi bilan tekshiruv ===")
    for q, d, V in ((2, 64, 1000), (4, 128, 5000)):
        m = GPT(V, d, q)
        x = torch.randint(0, V, (8, L_KONTEKST))
        with warnings.catch_warnings():
            warnings.simplefilter("ignore")
            with FlopCounterMode(display=False) as fc:
                loss = F.cross_entropy(m(x).reshape(-1, V), x.reshape(-1))
                loss.backward()
        olchangan = fc.get_total_flops()
        n_mat = sum(p.numel() for n, p in m.named_parameters()
                    if p.dim() == 2 and not n.startswith(("emb", "poz")))
        taxmin = 6 * n_mat * x.numel()
        attn = 6 * 2 * q * L_KONTEKST * d * x.numel()      # QK^T va w@V, forward + backward
        print(f"  L={q} d={d:>3}: o'lchangan {odam(olchangan)}, 6*N*D = {odam(taxmin)}, "
              f"nisbat {olchangan / taxmin:.2f}; attention qo'shimchasi +{attn / taxmin:.0%}")
    print("  (hisoblagich faqat mm/addmm ni sanadi; attention ni formula bilan qo'shdik)")

    print("\n=== 3. O'rgatish hisobi (D = 20 * N token, FARAZIY apparat) ===")
    print("  model                     N      D       FLOP     1 tezlatgichda   1024 tada")
    sekund_kun = 86400
    for nom, (q, d, V) in KONFIG.items():
        blok, emb = parametrlar(q, d, V)
        N = blok + emb
        D = 20 * N
        flop = 6 * N * D
        kun = flop / (CHOQQI_FLOPS * FOYDALANISH) / sekund_kun
        print(f"  {nom:<22} {odam(N):>7} {odam(D):>6} {flop:>10.1e} {kun:>11,.1f} kun "
              f"{kun / 1024:>8.1f} kun")

    print("\n=== 4. Og'irliklar xotirasi (faqat vaznlar, GB) ===")
    print("  model                   fp32    bf16    int8    int4   o'rgatish (16 B/param)")
    for nom, (q, d, V) in KONFIG.items():
        N = sum(parametrlar(q, d, V))
        qator = " ".join(f"{N * b / 1e9:>7.1f}" for b in (4, 2, 1, 0.5))
        print(f"  {nom:<22} {qator} {N * 16 / 1e9:>10.0f}")
    q, d, _ = KONFIG["katta (7B sinf)"]
    kv = 2 * q * d * 2                                  # K va V, har qatlam, bf16 (2 bayt)
    print(f"  KV-cache (7B sinf, bf16): token boshiga {kv / 1e6:.2f} MB; "
          f"8000 token -> {kv * 8000 / 1e9:.1f} GB; 32 so'rov x 8000 -> {kv * 8000 * 32 / 1e9:.0f} GB")

    print("\n=== 5. Kvantlash xatosi: haqiqiy matritsada o'lchaymiz ===")
    g = torch.Generator().manual_seed(0)
    w = torch.randn(512, 512, generator=g) * 0.02
    w_chetli = w.clone()
    w_chetli[torch.randint(0, 512, (20,), generator=g), torch.randint(0, 512, (20,), generator=g)] = 1.0
    x = torch.randn(64, 512, generator=g)
    print("  matritsa         bit  shkala       nisbiy xato (w)  nisbiy xato (x @ w)")
    xatolar = {}
    for nom, m in (("oddiy", w), ("20 ta chetli", w_chetli)):
        for bit in (8, 4):
            for qb in (False, True):
                wq = kvantla(m, bit, qb)
                e_w = ((wq - m).norm() / m.norm()).item()
                e_y = ((x @ wq - x @ m).norm() / (x @ m).norm()).item()
                xatolar[(nom, bit, qb)] = e_y
                print(f"  {nom:<16} {bit:>3}  {'qator' if qb else 'butun':<10} {e_w:>14.4f} {e_y:>18.4f}")
    if xatolar[("20 ta chetli", 4, False)] > 3 * xatolar[("oddiy", 4, False)]:
        print("  chetli qiymatlar butun matritsa shkalasini buzadi -> qator bo'yicha shkala yordam beradi")
    print("  ⭐ Xotira = N x bayt; hisob = 6ND; ikkalasi ham bir qatorda baholanadi")


if __name__ == "__main__":
    main()

=== 1. Parametr soni: formula va torch sanog'i ===
  L=2 d= 64: bloklar    99,968, 12*L*d^2 =    98,304  (farq 1.7%)
  L=4 d=128: bloklar   793,088, 12*L*d^2 =   786,432  (farq 0.8%)
  L=6 d=256: bloklar 4,738,560, 12*L*d^2 = 4,718,592  (farq 0.4%)

=== 2. 6*N*D: torch FLOP hisoblagichi bilan tekshiruv ===
  L=2 d= 64: o'lchangan 498.6M, 6*N*D = 498.6M, nisbat 1.00; attention qo'shimchasi +10%
  L=4 d=128: o'lchangan 4.4B, 6*N*D = 4.4B, nisbat 1.00; attention qo'shimchasi +5%
  (hisoblagich faqat mm/addmm ni sanadi; attention ni formula bilan qo'shdik)

=== 3. O'rgatish hisobi (D = 20 * N token, FARAZIY apparat) ===
  model                     N      D       FLOP     1 tezlatgichda   1024 tada
  kichik (125M sinf)      123.3M   2.5B    1.8e+18         0.1 kun      0.0 kun
  o'rta (1B sinf)         936.4M  18.7B    1.1e+20         3.0 kun      0.0 kun
  katta (7B sinf)           6.7B 134.1B    5.4e+21       156.1 kun      0.2 kun
  juda katta (70B sinf)    65.5B   1.3T    5.1e+23    14,884.5 kun     14.5 kun

=== 4. Og'irliklar xotirasi (faqat vaznlar, GB) ===
  model                   fp32    bf16    int8    int4   o'rgatish (16 B/param)
  kichik (125M sinf)         0.5     0.2     0.1     0.1          2
  o'rta (1B sinf)            3.7     1.9     0.9     0.5         15
  katta (7B sinf)           26.8    13.4     6.7     3.4        107
  juda katta (70B sinf)    261.9   130.9    65.5    32.7       1048
  KV-cache (7B sinf, bf16): token boshiga 0.52 MB; 8000 token -> 4.2 GB; 32 so'rov x 8000 -> 134 GB

=== 5. Kvantlash xatosi: haqiqiy matritsada o'lchaymiz ===
  matritsa         bit  shkala       nisbiy xato (w)  nisbiy xato (x @ w)
  oddiy              8  butun              0.0106             0.0105
  oddiy              8  qator              0.0075             0.0074
  oddiy              4  butun              0.1920             0.1899
  oddiy              4  qator              0.1354             0.1343
  20 ta chetli       8  butun              0.1041             0.1025
  20 ta chetli       8  qator              0.0215             0.0217
  20 ta chetli       4  butun              0.9159             0.9128
  20 ta chetli       4  qator              0.2168             0.2170
  chetli qiymatlar butun matritsa shkalasini buzadi -> qator bo'yicha shkala yordam beradi
  ⭐ Xotira = N x bayt; hisob = 6ND; ikkalasi ham bir qatorda baholanadi

"""Gallyutsinatsiya: model ko'rmagan faktni ham ishonch bilan "to'qiydi"."""

import math

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F

ISMLAR = ("anvar dilnoza bobur malika sardor nodira jasur zarina otabek madina "
          "sherzod gulnora farhod laylo rustam sevara aziz kamola bahrom shahlo "
          "temur nigora dilshod feruza jamshid mohira sanjar yulduz akmal lola "
          "botir oydin hamid zuhra ilhom saida karim nilufar komil diyora").split()
KASBLAR = "shifokor muhandis oqituvchi dehqon haydovchi oshpaz".split()
JOYLAR = "bozorga maktabga shaharga bekatga dalaga bogga".split()
CHASTOTA = (0, 1, 3, 10)              # har guruhda fakt korpusda necha marta uchraydi
L = 24


def korpus(seed=0):
    """40 ism, 4 guruh: fakt 0, 1, 3 yoki 10 marta. Hamma ism boshqa gaplarda ham bor."""
    rng = np.random.default_rng(seed)
    ismlar = [str(x) for x in rng.permutation(ISMLAR)]
    kasb = {i: str(rng.choice(KASBLAR)) for i in ismlar}
    guruh = {i: CHASTOTA[k // 10] for k, i in enumerate(ismlar)}
    gaplar = []
    for i in ismlar:
        gaplar += [f"{i} kasbi {kasb[i]} ." for _ in range(guruh[i])]
        gaplar += [f"{i} {rng.choice(JOYLAR)} bordi ." for _ in range(8)]
    gaplar += ["mening kasbim yaxshi ." for _ in range(40)]
    rng.shuffle(gaplar)
    return " ".join(gaplar), kasb, guruh


class Blok(nn.Module):
    def __init__(self, d, boshlar):
        super().__init__()
        self.boshlar = boshlar
        self.ln1, self.ln2 = nn.LayerNorm(d), nn.LayerNorm(d)
        self.qkv = nn.Linear(d, 3 * d)
        self.proj = nn.Linear(d, d)
        self.ffn = nn.Sequential(nn.Linear(d, 4 * d), nn.GELU(), nn.Linear(4 * d, d))

    def forward(self, x):
        B, T, d = x.shape
        q, k, v = self.qkv(self.ln1(x)).split(d, dim=2)
        q, k, v = [t.view(B, T, self.boshlar, -1).transpose(1, 2) for t in (q, k, v)]
        o = F.scaled_dot_product_attention(q, k, v, is_causal=True)
        x = x + self.proj(o.transpose(1, 2).reshape(B, T, d))
        return x + self.ffn(self.ln2(x))


class GPT(nn.Module):
    def __init__(self, V, d=32, boshlar=4, qatlamlar=2):
        super().__init__()
        self.emb = nn.Embedding(V, d)
        self.poz = nn.Embedding(L, d)
        self.bloklar = nn.Sequential(*[Blok(d, boshlar) for _ in range(qatlamlar)])
        self.ln = nn.LayerNorm(d)
        self.chiqish = nn.Linear(d, V)

    def forward(self, x):
        x = x[:, -L:]
        h = self.emb(x) + self.poz(torch.arange(x.shape[1]))
        return self.chiqish(self.ln(self.bloklar(h)))


def orgat(T, V, seed, qadamlar=200, B=32):
    torch.manual_seed(seed)
    model = GPT(V)
    opt = torch.optim.AdamW(model.parameters(), lr=1e-2, weight_decay=0.01)
    g = torch.Generator().manual_seed(seed)
    for _ in range(qadamlar):
        bosh = torch.randint(0, len(T) - L - 1, (B,), generator=g)
        w = torch.stack([T[i:i + L + 1] for i in bosh.tolist()])
        loss = F.cross_entropy(model(w[:, :-1]).reshape(-1, V), w[:, 1:].reshape(-1))
        opt.zero_grad()
        loss.backward()
        opt.step()
    model.eval()
    return model


@torch.no_grad()
def sora(model, ism, s2i):
    """'<ism> kasbi' dan keyingi taqsimot: kasblar ulushi va kasblar ichidagi eng yuqori p."""
    x = torch.tensor([[s2i["."], s2i[ism], s2i["kasbi"]]])
    p = torch.softmax(model(x)[0, -1], -1)
    pk = torch.tensor([p[s2i[k]].item() for k in KASBLAR])
    return pk.sum().item(), pk / pk.sum()


def main() -> None:
    torch.set_num_threads(1)
    matn, kasb, guruh = korpus()
    sozlar = sorted(set(matn.split()))
    s2i = {w: i for i, w in enumerate(sozlar)}
    T = torch.tensor([s2i[w] for w in matn.split()])
    print("=== 1. Korpus ===")
    print(f"  {len(T)} token, 40 ism; har ism 8 marta oddiy gapda ('... bozorga bordi .')")
    print("  kasb fakti: 10 ismda 0 marta, 10 tasida 1, 10 tasida 3, 10 tasida 10 marta")
    print(f"  tasodifiy taxmin aniqligi: 1/{len(KASBLAR)} = {1 / len(KASBLAR):.3f}")

    print("\n=== 2. '<ism> kasbi ...' - javob, ishonch va to'g'rilik (2 seed) ===")
    qatorlar = []
    for seed in range(2):
        model = orgat(T, len(sozlar), seed)
        for ism in kasb:
            ulush, pk = sora(model, ism, s2i)
            k = int(pk.argmax())
            qatorlar.append((seed, guruh[ism], ulush, pk.max().item(), float(KASBLAR[k] == kasb[ism])))
    q = np.array(qatorlar)
    print("  fakt soni   kasbga massa   ishonch (maks p)   aniqlik   ishonch - aniqlik")
    ishonch, aniqlik = {}, {}
    for c in CHASTOTA:
        m = q[:, 1] == c
        ishonch[c], aniqlik[c] = q[m, 3].mean(), q[m, 4].mean()
        print(f"  {c:>9} {q[m, 2].mean():>14.3f} {ishonch[c]:>18.3f} {aniqlik[c]:>9.3f} "
              f"{ishonch[c] - aniqlik[c]:>+18.3f}")

    print("\n=== 3. Ko'rilmagan fakt: javob bermay qo'yadimi? ===")
    m0 = q[:, 1] == 0
    print(f"  0 marta ko'rilgan ismlarda kasbga massa {q[m0, 2].mean():.3f} - "
          f"model deyarli doim biror kasb aytadi")
    d = q[m0, 3] - 1 / len(KASBLAR)
    se = d.std(ddof=1) / math.sqrt(len(d))
    print(f"  ishonch - tasodif (1/6): {d.mean():+.3f}, SE {se:.3f} -> "
          f"{'tasodifdan sezilarli yuqori' if d.mean() > 2 * se else 'tasodifga yaqin'}")
    d2 = q[m0, 4] - 1 / len(KASBLAR)
    se2 = d2.std(ddof=1) / math.sqrt(len(d2))
    print(f"  aniqlik - tasodif:         {d2.mean():+.3f}, SE {se2:.3f} -> "
          f"{'tasodifdan sezilarli yuqori' if d2.mean() > 2 * se2 else 'tasodifdan farqsiz'}")

    print("\n=== 4. Kalibrlash: ishonch oraliqlari bo'yicha (hamma guruhlar) ===")
    chegaralar = [0.0, 0.4, 0.6, 0.8, 1.01]
    ece = 0.0
    for a, b in zip(chegaralar, chegaralar[1:]):
        m = (q[:, 3] >= a) & (q[:, 3] < b)
        if m.sum() == 0:
            continue
        ece += m.mean() * abs(q[m, 3].mean() - q[m, 4].mean())
        print(f"  ishonch {a:.1f}-{min(b, 1.0):.1f}: {int(m.sum()):>3} savol, "
              f"o'rtacha ishonch {q[m, 3].mean():.3f}, aniqlik {q[m, 4].mean():.3f}")
    print(f"  ECE (kutilgan kalibrlash xatosi): {ece:.3f}")
    if ishonch[0] - aniqlik[0] > ishonch[10] - aniqlik[10]:
        print("  eng katta ortiqcha ishonch - kam ko'rilgan faktlarda: bu gallyutsinatsiya")
    print("  ⭐ Model 'bilmayman' demaydi: u ehtimolli davomni beradi, bilim chegarasini emas")


if __name__ == "__main__":
    main()

=== 1. Korpus ===
  2000 token, 40 ism; har ism 8 marta oddiy gapda ('... bozorga bordi .')
  kasb fakti: 10 ismda 0 marta, 10 tasida 1, 10 tasida 3, 10 tasida 10 marta
  tasodifiy taxmin aniqligi: 1/6 = 0.167

=== 2. '<ism> kasbi ...' - javob, ishonch va to'g'rilik (2 seed) ===
  fakt soni   kasbga massa   ishonch (maks p)   aniqlik   ishonch - aniqlik
          0          0.987              0.727     0.150             +0.577
          1          0.979              0.679     0.800             -0.121
          3          0.988              0.857     0.900             -0.043
         10          0.990              0.983     1.000             -0.017

=== 3. Ko'rilmagan fakt: javob bermay qo'yadimi? ===
  0 marta ko'rilgan ismlarda kasbga massa 0.987 - model deyarli doim biror kasb aytadi
  ishonch - tasodif (1/6): +0.561, SE 0.050 -> tasodifdan sezilarli yuqori
  aniqlik - tasodif:         -0.017, SE 0.082 -> tasodifdan farqsiz

=== 4. Kalibrlash: ishonch oraliqlari bo'yicha (hamma guruhlar) ===
  ishonch 0.0-0.4:   3 savol, o'rtacha ishonch 0.327, aniqlik 0.000
  ishonch 0.4-0.6:  14 savol, o'rtacha ishonch 0.511, aniqlik 0.500
  ishonch 0.6-0.8:  14 savol, o'rtacha ishonch 0.714, aniqlik 0.643
  ishonch 0.8-1.0:  49 savol, o'rtacha ishonch 0.955, aniqlik 0.837
  ECE (kutilgan kalibrlash xatosi): 0.099
  eng katta ortiqcha ishonch - kam ko'rilgan faktlarda: bu gallyutsinatsiya
  ⭐ Model 'bilmayman' demaydi: u ehtimolli davomni beradi, bilim chegarasini emas

"""DPO g'oyasi noldan: afzallik juftlari bilan javob uslubini siljitish."""

import copy
import math

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F

ISMLAR = ("anvar dilnoza bobur malika sardor nodira jasur zarina otabek madina "
          "sherzod gulnora farhod laylo rustam sevara aziz kamola bahrom shahlo "
          "temur nigora dilshod feruza jamshid mohira sanjar yulduz akmal lola").split()
KASBLAR = "shifokor muhandis oqituvchi dehqon haydovchi oshpaz".split()
L = 24


def qisqa(k):
    return f"javob : {k} ."


def uzun(k):
    return f"javob : hmm , yaxshi savol , menimcha {k} ."


def dunyo(seed=0):
    rng = np.random.default_rng(seed)
    kasb = {i: str(rng.choice(KASBLAR)) for i in ISMLAR}
    aralash = [str(x) for x in rng.permutation(ISMLAR)]
    return kasb, aralash[:15], aralash[15:]


def sft_korpus(kasb, n, rng):
    """SFT dan chiqqan model o'xshatmasi: javoblarning yarmi qisqa, yarmi "suvli"."""
    qator = []
    for _ in range(n):
        i = str(rng.choice(ISMLAR))
        javob = qisqa(kasb[i]) if rng.random() < 0.5 else uzun(kasb[i])
        qator.append(f"<d> savol : {i} kim ? {javob}")
    return " ".join(qator)


class Blok(nn.Module):
    def __init__(self, d, boshlar):
        super().__init__()
        self.boshlar = boshlar
        self.ln1, self.ln2 = nn.LayerNorm(d), nn.LayerNorm(d)
        self.qkv = nn.Linear(d, 3 * d)
        self.proj = nn.Linear(d, d)
        self.ffn = nn.Sequential(nn.Linear(d, 4 * d), nn.GELU(), nn.Linear(4 * d, d))

    def forward(self, x):
        B, T, d = x.shape
        q, k, v = self.qkv(self.ln1(x)).split(d, dim=2)
        q, k, v = [t.view(B, T, self.boshlar, -1).transpose(1, 2) for t in (q, k, v)]
        o = F.scaled_dot_product_attention(q, k, v, is_causal=True)
        x = x + self.proj(o.transpose(1, 2).reshape(B, T, d))
        return x + self.ffn(self.ln2(x))


class GPT(nn.Module):
    def __init__(self, V, d=32, boshlar=4, qatlamlar=2):
        super().__init__()
        self.emb = nn.Embedding(V, d)
        self.poz = nn.Embedding(L, d)
        self.bloklar = nn.Sequential(*[Blok(d, boshlar) for _ in range(qatlamlar)])
        self.ln = nn.LayerNorm(d)
        self.chiqish = nn.Linear(d, V)

    def forward(self, x):
        x = x[:, -L:]
        h = self.emb(x) + self.poz(torch.arange(x.shape[1]))
        return self.chiqish(self.ln(self.bloklar(h)))


def orgat(T, V, seed=0, qadamlar=250, B=32):
    torch.manual_seed(seed)
    model = GPT(V)
    opt = torch.optim.AdamW(model.parameters(), lr=1e-2, weight_decay=0.01)
    g = torch.Generator().manual_seed(seed)
    for _ in range(qadamlar):
        bosh = torch.randint(0, len(T) - L - 1, (B,), generator=g)
        w = torch.stack([T[i:i + L + 1] for i in bosh.tolist()])
        loss = F.cross_entropy(model(w[:, :-1]).reshape(-1, V), w[:, 1:].reshape(-1))
        opt.zero_grad()
        loss.backward()
        opt.step()
    return model


def javob_logp(model, savollar, javoblar, s2i):
    """log pi(javob | savol) = javob tokenlari log-ehtimollari yig'indisi (batch)."""
    s = torch.tensor([[s2i[w] for w in x.split()] for x in savollar])
    j = torch.tensor([[s2i[w] for w in x.split()] for x in javoblar])
    t = torch.cat([s, j], 1)
    logp = torch.log_softmax(model(t[:, :-1]), -1)
    tok = logp.gather(2, t[:, 1:].unsqueeze(2)).squeeze(2)
    return tok[:, s.shape[1] - 1:].sum(1)


def dpo(ref, juftlar, s2i, beta, qadamlar=40, faqat_sft=False):
    """DPO: -log sigmoid(beta * [(log pi_w - log ref_w) - (log pi_l - log ref_l)])."""
    model = copy.deepcopy(ref)
    opt = torch.optim.AdamW(model.parameters(), lr=3e-4, weight_decay=0.0)
    sav = [s for s, _, _ in juftlar]
    yw = [w for _, w, _ in juftlar]
    yl = [l_ for _, _, l_ in juftlar]
    with torch.no_grad():
        ref_w, ref_l = javob_logp(ref, sav, yw, s2i), javob_logp(ref, sav, yl, s2i)
    for _ in range(qadamlar):
        pw = javob_logp(model, sav, yw, s2i)
        if faqat_sft:                                      # taqqoslash: faqat tanlanganda SFT
            loss = -pw.mean()
        else:
            pl = javob_logp(model, sav, yl, s2i)
            loss = -F.logsigmoid(beta * ((pw - ref_w) - (pl - ref_l))).mean()
        opt.zero_grad()
        loss.backward()
        opt.step()
    return model


@torch.no_grad()
def olcha(model, ref, ismlar, kasb, s2i):
    """Har ism: P(qisqa uslub), greedy kasb to'g'rimi, KL(pi || ref) javob boshida."""
    p_qisqa, togri, kl = [], [], []
    for i in ismlar:
        x = torch.tensor([[s2i[w] for w in f"<d> savol : {i} kim ? javob :".split()]])
        lp, lr = torch.log_softmax(model(x)[0, -1], -1), torch.log_softmax(ref(x)[0, -1], -1)
        p = lp.exp()
        p_qisqa.append(1 - p[s2i["hmm"]].item())
        kl.append((p * (lp - lr)).sum().item())
        k = max(KASBLAR, key=lambda k: p[s2i[k]].item())
        togri.append(float(k == kasb[i]))
    return np.array(p_qisqa), np.array(togri), np.array(kl)


def main() -> None:
    torch.set_num_threads(1)
    kasb, A, B_guruh = dunyo()
    matn = sft_korpus(kasb, 2500, np.random.default_rng(0))
    sozlar = sorted(set(matn.split()))
    s2i = {w: i for i, w in enumerate(sozlar)}
    T = torch.tensor([s2i[w] for w in matn.split()])
    ref = orgat(T, len(sozlar))
    juftlar = [(f"<d> savol : {i} kim ?", qisqa(kasb[i]), uzun(kasb[i])) for i in A]

    print("=== 1. Boshlang'ich (reference) model ===")
    print(f"  korpus: 2500 savol-javob, javoblarning ~50% i qisqa: '{qisqa('oshpaz')}'")
    print(f"                              ~50% i suvli: '{uzun('oshpaz')}'")
    print(f"  afzallik juftlari: {len(juftlar)} ta (A guruh), tanlangan = qisqa, rad = suvli")
    p0, t0, _ = olcha(ref, ref, B_guruh, kasb, s2i)
    print(f"  B guruhda: P(qisqa) {p0.mean():.3f}, kasb aniqligi {t0.mean():.3f}")

    print("\n=== 2. DPO va 'faqat tanlanganda SFT' (40 qadam, B guruh - juftlarda yo'q) ===")
    print("  usul                 P(qisqa)   kasb aniqligi   KL(pi||ref)")
    natija = {}
    for nom, beta, fs in (("DPO beta=0.1", 0.1, False), ("DPO beta=1.0", 1.0, False),
                          ("SFT tanlanganda", None, True)):
        torch.manual_seed(0)
        m = dpo(ref, juftlar, s2i, beta, faqat_sft=fs)
        natija[nom] = olcha(m, ref, B_guruh, kasb, s2i)
        pq, tg, kl = natija[nom]
        print(f"  {nom:<18} {pq.mean():>10.3f} {tg.mean():>15.3f} {kl.mean():>13.3f}")

    print("\n=== 3. Juftlashgan farqlar (15 ism) ===")
    for nom in natija:
        d = natija[nom][0] - p0
        se = d.std(ddof=1) / math.sqrt(len(d))
        print(f"  {nom:<18} P(qisqa) o'zgarishi {d.mean():+.3f}, SE {se:.3f} -> "
              f"{'sezilarli' if abs(d.mean()) > 2 * se else 'sezilarli emas'}")
    d = natija["DPO beta=1.0"][2] - natija["DPO beta=0.1"][2]
    se = d.std(ddof=1) / math.sqrt(len(d))
    print(f"  KL: beta=1.0 - beta=0.1 = {d.mean():+.3f}, SE {se:.3f}")

    print("\n=== 4. Nima o'zgardi, nima o'zgarmadi ===")
    for nom, (pq, tg, kl) in natija.items():
        d = tg - t0
        se = d.std(ddof=1) / math.sqrt(len(d)) if d.any() else 0.0
        hukm = "sezilarli tushdi" if d.mean() < -2 * se else "sezilarli o'zgarmadi"
        print(f"  {nom:<18} kasb: {int(t0.sum())}/15 -> {int(tg.sum())}/15, "
              f"farq {d.mean():+.3f}, SE {se:.3f} -> {hukm}")
    print("  ⭐ DPO mukofot modelisiz: faqat (tanlangan, rad etilgan) juftlar va reference model")


if __name__ == "__main__":
    main()

=== 1. Boshlang'ich (reference) model ===
  korpus: 2500 savol-javob, javoblarning ~50% i qisqa: 'javob : oshpaz .'
                              ~50% i suvli: 'javob : hmm , yaxshi savol , menimcha oshpaz .'
  afzallik juftlari: 15 ta (A guruh), tanlangan = qisqa, rad = suvli
  B guruhda: P(qisqa) 0.444, kasb aniqligi 1.000

=== 2. DPO va 'faqat tanlanganda SFT' (40 qadam, B guruh - juftlarda yo'q) ===
  usul                 P(qisqa)   kasb aniqligi   KL(pi||ref)
  DPO beta=0.1            0.938           0.867         0.793
  DPO beta=1.0            0.798           1.000         0.357
  SFT tanlanganda         0.948           0.933         0.790

=== 3. Juftlashgan farqlar (15 ism) ===
  DPO beta=0.1       P(qisqa) o'zgarishi +0.493, SE 0.030 -> sezilarli
  DPO beta=1.0       P(qisqa) o'zgarishi +0.354, SE 0.021 -> sezilarli
  SFT tanlanganda    P(qisqa) o'zgarishi +0.504, SE 0.031 -> sezilarli
  KL: beta=1.0 - beta=0.1 = -0.436, SE 0.048

=== 4. Nima o'zgardi, nima o'zgarmadi ===
  DPO beta=0.1       kasb: 15/15 -> 13/15, farq -0.133, SE 0.091 -> sezilarli o'zgarmadi
  DPO beta=1.0       kasb: 15/15 -> 15/15, farq +0.000, SE 0.000 -> sezilarli o'zgarmadi
  SFT tanlanganda    kasb: 15/15 -> 14/15, farq -0.067, SE 0.067 -> sezilarli o'zgarmadi
  ⭐ DPO mukofot modelisiz: faqat (tanlangan, rad etilgan) juftlar va reference model

y = savol_va_javob[1:]                                              # ⚠️
y = [-100] * (len(savol) - 1) + javob                               # ✅
loss = F.cross_entropy(logit, y, ignore_index=-100)

prompt = "Menga Toshkent haqida qisqa ma'lumot ber."                # ⚠️ base model
prompt = "Savol: Toshkent qanday shahar?\nJavob:"                   # ✅ hujjat shaklida

kerak_gb = N * 4 / 1e9                                              # ⚠️
kerak_gb = N * 16 / 1e9 + aktivatsiyalar_gb                         # ✅ AdamW

kerak = vaznlar_gb                                                  # ⚠️
kerak = vaznlar_gb + 2 * L * d * bayt * tokenlar * parallel / 1e9   # ✅

shkala = w.abs().max() / 7                                          # ⚠️ chetli qiymat
shkala = w.abs().amax(dim=1, keepdim=True) / 7                      # ✅ qator bo'yicha

ref_w = javob_logp(model, savol, y_w)                               # ⚠️ o'sha model
with torch.no_grad():
    ref_w = javob_logp(ref_muzlatilgan, savol, y_w)                 # ✅

if p_max > 0.7: javobni_qabul_qil()                                 # ⚠️
if manba_topildi and p_max > chegara: javobni_qabul_qil()           # ✅ + kalibrlash

1.  # LLM va 24.9 dagi GPT ning asosiy arxitektura farqi?
2.  # base model "Savol: ...?" ga ko'pincha nima bilan javob beradi va nega?
3.  # SFT da qaysi tokenlar loss ga kiradi?
4.  # L = 32, d = 4096 bo'lsa, bloklardagi parametrlar taxminan?
5.  # N = 7e9, D = 1.4e11 bo'lsa, o'rgatish FLOP?
6.  # 7B model int4 da vaznlar uchun qancha xotira oladi?
7.  # AdamW bilan fp32 o'rgatishda parametr boshiga taxminan necha bayt?
8.  # in-context learning da vaznlar o'zgaradimi?
9.  # nega model ko'rmagan faktni ham ishonch bilan aytadi?
10. # DPO da beta oshirilsa, KL(pi || ref) qanday o'zgaradi?
11. # butun matritsa shkalasida bitta juda katta vazn nimaga olib keladi?
12. # "hisob-optimal" qoidada D va N nisbati taxminan?

1.  x = savol + javob
    loss = F.cross_entropy(model(x[:-1]), x[1:])        # SFT

2.  xotira_gb = 7e9 * 2 / 1e9                            # "7B modelni 32 ta 8000 tokenli
                                                          #  suhbat bilan ishlatamiz"

3.  ref = model                                          # DPO
    loss = -F.logsigmoid(beta * ((pi_w - ref_w) - (pi_l - ref_l))).mean()

4.  shkala = w.abs().max() / 7
    wq = torch.round(w / shkala) * shkala                # int4

5.  if javob_ishonchi > 0.9:
        print("fakt to'g'ri")