23.6-dars: So'z embeddinglari

G'OYA (Harris 1954, Firth 1957):
  "so'zni u bilan birga keladigan so'zlar orqali bilasiz"
  bir xil kontekstlarda keladigan so'zlar - ma'nosi yaqin

MISOL:
  shifokor bemorni tekshirdi       vrach bemorni tekshirdi
  shifokor dori yozib berdi        vrach dori yozib berdi
  -> "shifokor" va "vrach" kontekstlari deyarli bir xil

ONE-HOT 23.4-bob BILAN FARQ:
  one-hot: har so'z alohida o'q, istalgan ikki so'z orasidagi
           kosinus = 0  ("shifokor" va "vrach" ham, "shifokor" va "gol" ham)
  embedding: d o'lchovli zich vektor (d = 20..300),
             o'xshash so'zlar - kichik burchak

"O'XSHASH" NIMANI BILDIRADI:
  sinonimlar (shifokor - vrach)                     yaqin
  bir mavzudagi so'zlar (shifokor - bemor)          o'rtacha yaqin
  ANTONIMLAR ham ko'pincha yaqin (oshdi - tushdi) - kontekstlari bir xil!
  -> embedding "almashtirsa bo'ladigan" so'zlarni topadi,
     "bir xil ma'noli" so'zlarni emas

BIRGA UCHRASH MATRITSASI:
  C[w, c] = w so'zidan +-oyna masofada c so'zi necha marta kelgan
  oyna = 2: "bugun shifokor bemor ko'rdi" da
    shifokor uchun kontekstlar: bugun, bemor, ko'rdi
  shakl: (V, V), V - lug'at hajmi; asosan nollardan iborat

XOM SANOQ MUAMMOSI:
  ko'p uchraydigan so'zlar ("bugun", "juda", "bu") hamma joyda
  -> har so'zning eng katta sanoqlari aynan shular
  -> ikki so'z "bugun" bilan birga kelgani uchun o'xshash bo'lib qoladi

PMI (pointwise mutual information):
  PMI(w, c) = log( P(w, c) / (P(w) * P(c)) )
            = log( C[w,c] * N / (C[w,:].sum() * C[:,c].sum()) )
  "tasodifdan qanchalik KO'P birga keladi"
  PMI > 0 - tasodifdan ko'p, PMI < 0 - kam, C = 0 da -inf

PPMI (positive PMI):
  PPMI = max(PMI, 0), C = 0 bo'lsa 0
  manfiy qiymatlar ishonchsiz (kam ma'lumot) -> tashlanadi
  natija: "bugun" kabi hamma joydagi so'zlarning og'irligi kichrayadi

PPMI MATRITSASI: (V, V), siyrak, katta (V = 50 000 -> 2.5 mlrd katak)

QISQARTIRILGAN SVD (10-qism):
  PPMI ~ U_k * S_k * V_k^T,   k << V
  so'z vektori: E = U_k * S_k   (yoki U_k * sqrt(S_k))
  shakl: (V, k), k = 20..300

NIMA BERADI:
  1. zich va qisqa vektorlar
  2. shovqin kamayadi: kichik singular qiymatlar tashlanadi
  3. umumlashtirish: "shifokor" va "vrach" hech qachon bir gapda
     kelmasa ham, ularning satrlari bir xil "yo'nalish"ga tushadi

k NI TANLASH:
  juda kichik -> mavzular aralashadi (sport va salomatlik bir joyda)
  juda katta  -> shovqin qaytadi, siqish foydasi yo'qoladi
  -> validatsiya vazifasida tanlanadi (bizda: sinonim topish)

SKIP-GRAM VAZIFASI:
  markaziy so'zdan uning qo'shnilarini bashorat qil
  "bugun shifokor bemor ko'rdi", oyna=2:
    (shifokor, bugun), (shifokor, bemor), (shifokor, ko'rdi), ...
  har so'zning IKKI vektori: markaz v_w va kontekst u_c

TO'LIQ SOFTMAX (qimmat):
  P(c | w) = exp(u_c . v_w) / sum_{c'} exp(u_c' . v_w)
  maxraj - butun lug'at bo'yicha (V = 100 000 bo'lsa, har juftda 100 000)

NEGATIVE SAMPLING (SGNS):
  ko'p klassli vazifa -> ikkilik vazifa:
    haqiqiy juft (w, c) -> 1,   tasodifiy k ta (w, n) -> 0
  loss = -log sigmoid(u_c . v_w) - sum_{i=1..k} log sigmoid(-u_ni . v_w)
  k = 5..20; manfiy so'zlar P(n) ~ count(n)^0.75 dan olinadi
  (0.75 - kam uchraydigan so'zlarga biroz ko'proq imkon)

TORCH DA:
  markaz = nn.Embedding(V, d);  kontekst = nn.Embedding(V, d)
  ijobiy = (markaz(w) * kontekst(c)).sum(1)
  salbiy = bmm(kontekst(neg), markaz(w).unsqueeze(2))
  o'rgatishdan keyin: markaz.weight - so'z vektorlari

SGNS VA PPMI BOG'LIQLIGI (Levy va Goldberg, 2014):
  SGNS yashirin tarzda PMI - log(k) matritsasini faktorlaydi
  -> ikki yondashuv bir xil statistikaning ikki ko'rinishi

KOSINUS O'XSHASHLIK:
  cos(a, b) = a . b / (|a| * |b|),   -1 .. 1
  vektor UZUNLIGI ko'pincha chastotani aks ettiradi -> uzunlik emas,
  yo'nalish taqqoslanadi

ENG YAQIN SO'ZLAR:
  En = E / |E|  (satr bo'yicha normallash)
  S = En @ En.T;  S[w, w] = -inf;  S[w].argsort()[::-1][:5]

SINONIMLARNI O'LCHASH (bizning korpusda):
  top-1: sinonimi eng yaqin qo'shni chiqqan juftlar ulushi
  cos(sinonimlar) ni cos(bir mavzudagi boshqa so'zlar) va
  cos(boshqa mavzudagi so'zlar) bilan solishtiramiz
  tasodifiy vektorlar bilan ham (o'rgatishdan OLDIN) - bazaviy

ANALOGIYA (nazariya):
  "erkak : qirol = ayol : ?"
  vec(qirol) - vec(erkak) + vec(ayol) ~ vec(malika)
  ishlashi uchun: farq vektori ("erkak -> ayol") ko'p juftlarda
  bir xil bo'lishi kerak - bu faqat JUDA KATTA korpusda (milliardlab
  so'z) va muntazam munosabatlarda (poytaxt, rod, zamon) kuzatiladi
  kichik korpusda analogiya ishonchsiz; va u embedding sifatining
  yagona o'lchovi emas (natijalar test to'plamiga sezgir)

NEGA:
  so'z vektori uning kontekstlari statistikasidan quriladi
  so'z 5 marta uchragan bo'lsa - 5 ta kontekst, statistikasi shovqinli
  kam uchraydigan sinonim ("vrach" "shifokor"dan 3 barobar kam) -
  eng birinchi zarar ko'radi

AMALDA:
  word2vec ning klassik vektorlari milliardlab so'zda o'rgatilgan
  o'n minglab gapda - faqat ko'p uchraydigan so'zlar ishonchli
  min_count (odatda 5) - juda kam uchraganlar lug'atdan chiqariladi

HALOL TAJRIBA (4-misol):
  bir xil generatordan 50, 150, 500, 2000 gap
  bir xil model va bir xil qadamlar soni
  -> farq faqat ma'lumot hajmidan

OLDINDAN O'RGATILGAN EMBEDDING:
  katta umumiy korpusda bir marta o'rgatiladi, keyin kichik vazifada
  ishlatiladi (keyingi dars) - kam ma'lumotli vazifaning asosiy yordamchisi

import numpy as np
import torch
import torch.nn as nn

# 1. Birga uchrash (oyna = 2)
C = np.zeros((V, V))
for gap in gaplar:
    ids = [idx[s] for s in gap]
    for i, w in enumerate(ids):
        for j in range(max(0, i - 2), min(len(ids), i + 3)):
            if j != i:
                C[w, ids[j]] += 1

# 2. PPMI
N = C.sum()
pmi = np.log(C * N / (C.sum(1, keepdims=True) * C.sum(0, keepdims=True))
             + 1e-12)
ppmi = np.where(C > 0, np.maximum(pmi, 0), 0)

# 3. SVD
U, S, Vt = np.linalg.svd(ppmi)
E = U[:, :20] * S[:20]

# 4. Skip-gram + negative sampling
markaz, kontekst = nn.Embedding(V, d), nn.Embedding(V, d)
vm = markaz(w)                                        # (B, d)
ijobiy = (vm * kontekst(c)).sum(1)                    # (B,)
salbiy = torch.bmm(kontekst(neg), vm.unsqueeze(2)).squeeze(2)  # (B, k)
loss = -(nn.functional.logsigmoid(ijobiy).mean()
         + nn.functional.logsigmoid(-salbiy).sum(1).mean())

# 5. Eng yaqin so'zlar
En = E / np.linalg.norm(E, axis=1, keepdims=True)
S = En @ En.T
np.fill_diagonal(S, -np.inf)
yaqin = S[idx["shifokor"]].argsort()[::-1][:5]

taqsimot gipotezasi: kontekst o'xshash -> ma'no yaqin
sanash: C -> PPMI -> SVD (k ~ 20..300)
bashorat: skip-gram + negative sampling, P(neg) ~ count^0.75
kosinus, o'zini chiqarib tashlab eng yaqinlar
sifat: sinonimlar, mavzu, tasodifiy bazaviy bilan o'lchang
korpus kichik -> kam uchraydigan so'zlar ishonchsiz

"""Mavzuli sintetik korpus: xom sanoq va PPMI qo'shnilari."""

import numpy as np

MAVZULAR = {
    "sport": (["o'yin", "mavsum", "chempionat", "sport", "final"], [
        [("futbolchi", "o'yinchi"), ("darvozaga",), ("gol",),
         ("urdi", "kiritdi")],
        [("jamoa", "komanda"), ("turnirda", "musobaqada"), ("g'alaba",),
         ("qozondi",)],
        [("murabbiy", "trener"), ("futbolchi", "o'yinchi"), ("maqtadi",)],
        [("jamoa", "komanda"), ("stadionda", "maydonda"), ("mashq",),
         ("qildi",)]]),
    "iqtisod": (["bozor", "iqtisodiyot", "foiz", "hukumat", "davlat"], [
        [("narx", "baho"), ("bozorda",), ("oshdi", "ko'tarildi")],
        [("bank",), ("kredit", "qarz"), ("berdi",)],
        [("hukumat",), ("soliqni",), ("kamaytirdi", "pasaytirdi")],
        [("kompaniya", "firma"), ("pul", "mablag'"), ("sarfladi",)],
        [("investor", "sarmoyador"), ("kompaniya", "firma"),
         ("qo'llab-quvvatladi",)]]),
    "texnologiya": (["raqamli", "texnologiya", "tizim", "onlayn",
                     "yangilanish"], [
        [("dasturchi", "muhandis"), ("dastur", "ilova"),
         ("yozdi", "yaratdi")],
        [("foydalanuvchi",), ("telefon", "smartfon"), ("sotib",), ("oldi",)],
        [("internet", "tarmoq"), ("tezligi",), ("oshdi", "ko'tarildi")],
        [("kompaniya", "firma"), ("dastur", "ilova"), ("chiqardi",)]]),
    "salomatlik": (["sog'liq", "tibbiyot", "kasallik", "davolash",
                    "klinika"], [
        [("shifokor", "vrach"), ("bemor", "kasal"), ("ko'rdi", "tekshirdi")],
        [("shifokor", "vrach"), ("dori", "preparat"), ("yozib",),
         ("berdi",)],
        [("bemor", "kasal"), ("kasalxonada", "shifoxonada"),
         ("davolandi",)],
        [("bemor", "kasal"), ("dori", "preparat"), ("ichdi",)]]),
}
BOSH = ["bugun", "kecha", "shu", "bu", "o'tgan"]
OXIR = ["yana", "juda", "ham", "tez", "ko'p"]
YORDAMCHI = ["va", "bu", "ham", "endi", "esa"]


def gap_yarat(rng):
    """Mavzu -> freym (Zipf bo'yicha) -> sinonim (75% / 25%) -> fon va
    yordamchi so'zlar (hamma mavzuda bir xil, tez-tez uchraydi)."""
    fon, freymlar = MAVZULAR[list(MAVZULAR)[int(rng.integers(0, 4))]]
    w = 1.0 / np.arange(1, len(freymlar) + 1)
    f = freymlar[rng.choice(len(freymlar), p=w / w.sum())]
    sozlar = [t[0] if len(t) == 1 or rng.random() < 0.75 else t[1]
              for t in f]
    for _ in range(int(rng.integers(1, 3))):
        sozlar.insert(int(rng.integers(0, len(sozlar) + 1)),
                      str(rng.choice(fon)))
    if rng.random() < 0.6:
        sozlar.insert(0, str(rng.choice(BOSH)))
    if rng.random() < 0.3:
        sozlar.insert(len(sozlar) - 1, str(rng.choice(OXIR)))
    for _ in range(int(rng.integers(0, 6))):
        sozlar.insert(int(rng.integers(0, len(sozlar) + 1)),
                      str(rng.choice(YORDAMCHI)))
    return sozlar


def birga_uchrash(gaplar, idx, oyna=2):
    C = np.zeros((len(idx), len(idx)))
    for g in gaplar:
        ids = [idx[s] for s in g]
        for i, w in enumerate(ids):
            for j in range(max(0, i - oyna), min(len(ids), i + oyna + 1)):
                if j != i:
                    C[w, ids[j]] += 1
    return C


def ppmi(C):
    N = C.sum()
    kutilgan = C.sum(1, keepdims=True) * C.sum(0, keepdims=True) / N
    with np.errstate(divide="ignore"):
        pmi = np.log(C / kutilgan)
    return np.where(C > 0, np.maximum(pmi, 0), 0.0)


def kos(a, b):
    return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))


def main() -> None:
    rng = np.random.default_rng(0)
    gaplar = [gap_yarat(rng) for _ in range(1000)]
    lugat = sorted({s for g in gaplar for s in g})
    idx = {s: i for i, s in enumerate(lugat)}
    C = birga_uchrash(gaplar, idx)

    print("=== 1. Korpus ===")
    print(f"  gaplar: {len(gaplar)}, so'zlar: {sum(map(len, gaplar))}, "
          f"lug'at: {len(lugat)}")
    for g in gaplar[:3]:
        print(f"  {' '.join(g)}")
    soni = {s: sum(g.count(s) for g in gaplar)
            for s in ["shifokor", "vrach", "bugun", "gol"]}
    print(f"  chastotalar: {soni}")
    print(f"  C shakli: {C.shape}, nol bo'lmagan kataklar: "
          f"{(C > 0).mean():.1%}")

    print("\n=== 2. Matritsa bo'lagi (xom sanoq) ===")
    qator = ["shifokor", "vrach", "bemor", "futbolchi"]
    ustun = ["bemor", "dori", "tekshirdi", "gol", "bugun", "tibbiyot"]
    print("  " + " " * 10 + "".join(f"{u:>10}" for u in ustun))
    for q in qator:
        print(f"  {q:<10}" + "".join(f"{C[idx[q], idx[u]]:>10.0f}"
                                     for u in ustun))
    print(f"  shifokor va vrach bir gapda: "
          f"{C[idx['shifokor'], idx['vrach']]:.0f} marta")

    P = ppmi(C)
    print("\n=== 3. 'shifokor' ning eng kuchli kontekstlari ===")
    for nom, M in [("xom sanoq", C), ("PPMI", P)]:
        top = np.argsort(-M[idx["shifokor"]], kind="stable")[:6]
        print(f"  {nom:<10} " + ", ".join(
            f"{lugat[i]} ({M[idx['shifokor'], i]:.1f})" for i in top))

    print("\n=== 4. Kosinus: xom sanoq va PPMI satrlari ===")
    juftlar = [("shifokor", "vrach"), ("shifokor", "bemor"),
               ("shifokor", "futbolchi"), ("narx", "baho"),
               ("narx", "gol")]
    print(f"  {'juft':<22} {'xom':>7} {'PPMI':>7}")
    natija = {}
    for a, b in juftlar:
        x = kos(C[idx[a]], C[idx[b]])
        p = kos(P[idx[a]], P[idx[b]])
        natija[(a, b)] = (x, p)
        print(f"  {a + ' - ' + b:<22} {x:>7.3f} {p:>7.3f}")
    x_s, p_s = natija[("shifokor", "vrach")]
    x_b, p_b = natija[("shifokor", "futbolchi")]
    print(f"\n  sinonim / boshqa mavzu nisbati: xom {x_s / x_b:.1f}, "
          f"PPMI {p_s / p_b:.1f}")
    if p_s / p_b > x_s / x_b:
        print("  PPMI sinonimni boshqa mavzudan yaxshiroq ajratdi")
    else:
        print("  PPMI ajratishni yaxshilamadi")
    print("  ⭐ Sinonimlar bir gapda deyarli kelmaydi - lekin kontekstlari bir xil")


if __name__ == "__main__":
    main()

=== 1. Korpus ===
  gaplar: 1000, so'zlar: 8324, lug'at: 100
  bu shifokor bemor davolash klinika tekshirdi endi bu
  kecha shifokor bemor sog'liq yana tekshirdi
  jamoa ham turnirda final esa bu g'alaba endi esa final qozondi
  chastotalar: {'shifokor': 138, 'vrach': 45, 'bugun': 117, 'gol': 127}
  C shakli: (100, 100), nol bo'lmagan kataklar: 29.0%

=== 2. Matritsa bo'lagi (xom sanoq) ===
                 bemor      dori tekshirdi       gol     bugun  tibbiyot
  shifokor          58        21         3         0        10        25
  vrach             21         7         2         0         5         7
  bemor              0        15        15         0         8        32
  futbolchi          0         0         0        22         9         0
  shifokor va vrach bir gapda: 0 marta

=== 3. 'shifokor' ning eng kuchli kontekstlari ===
  xom sanoq  bemor 58.0-bob, esa 39.0-bob, bu 34.0-bob, endi 34.0-bob, ham 31.0-bob, va 30.0-bob
  PPMI       kasal 2.1-bob, preparat 1.9-bob, sog'liq 1.8-bob, bemor 1.8-bob, davolash 1.7-bob, tibbiyot 1.6-bob

=== 4. Kosinus: xom sanoq va PPMI satrlari ===
  juft                       xom    PPMI
  shifokor - vrach         0.941   0.960
  shifokor - bemor         0.711   0.585
  shifokor - futbolchi     0.397   0.053
  narx - baho              0.910   0.822
  narx - gol               0.192   0.000

  sinonim / boshqa mavzu nisbati: xom 2.4, PPMI 18.0
  PPMI sinonimni boshqa mavzudan yaxshiroq ajratdi
  ⭐ Sinonimlar bir gapda deyarli kelmaydi - lekin kontekstlari bir xil

"""PPMI -> SVD: o'lchov k, sinonim topish va eng yaqin so'zlar."""

import numpy as np

MAVZULAR = {
    "sport": (["o'yin", "mavsum", "chempionat", "sport", "final"], [
        [("futbolchi", "o'yinchi"), ("darvozaga",), ("gol",),
         ("urdi", "kiritdi")],
        [("jamoa", "komanda"), ("turnirda", "musobaqada"), ("g'alaba",),
         ("qozondi",)],
        [("murabbiy", "trener"), ("futbolchi", "o'yinchi"), ("maqtadi",)],
        [("jamoa", "komanda"), ("stadionda", "maydonda"), ("mashq",),
         ("qildi",)]]),
    "iqtisod": (["bozor", "iqtisodiyot", "foiz", "hukumat", "davlat"], [
        [("narx", "baho"), ("bozorda",), ("oshdi", "ko'tarildi")],
        [("bank",), ("kredit", "qarz"), ("berdi",)],
        [("hukumat",), ("soliqni",), ("kamaytirdi", "pasaytirdi")],
        [("kompaniya", "firma"), ("pul", "mablag'"), ("sarfladi",)],
        [("investor", "sarmoyador"), ("kompaniya", "firma"),
         ("qo'llab-quvvatladi",)]]),
    "texnologiya": (["raqamli", "texnologiya", "tizim", "onlayn",
                     "yangilanish"], [
        [("dasturchi", "muhandis"), ("dastur", "ilova"),
         ("yozdi", "yaratdi")],
        [("foydalanuvchi",), ("telefon", "smartfon"), ("sotib",), ("oldi",)],
        [("internet", "tarmoq"), ("tezligi",), ("oshdi", "ko'tarildi")],
        [("kompaniya", "firma"), ("dastur", "ilova"), ("chiqardi",)]]),
    "salomatlik": (["sog'liq", "tibbiyot", "kasallik", "davolash",
                    "klinika"], [
        [("shifokor", "vrach"), ("bemor", "kasal"), ("ko'rdi", "tekshirdi")],
        [("shifokor", "vrach"), ("dori", "preparat"), ("yozib",),
         ("berdi",)],
        [("bemor", "kasal"), ("kasalxonada", "shifoxonada"),
         ("davolandi",)],
        [("bemor", "kasal"), ("dori", "preparat"), ("ichdi",)]]),
}
BOSH = ["bugun", "kecha", "shu", "bu", "o'tgan"]
OXIR = ["yana", "juda", "ham", "tez", "ko'p"]
YORDAMCHI = ["va", "bu", "ham", "endi", "esa"]


def gap_yarat(rng):
    fon, freymlar = MAVZULAR[list(MAVZULAR)[int(rng.integers(0, 4))]]
    w = 1.0 / np.arange(1, len(freymlar) + 1)
    f = freymlar[rng.choice(len(freymlar), p=w / w.sum())]
    sozlar = [t[0] if len(t) == 1 or rng.random() < 0.75 else t[1]
              for t in f]
    for _ in range(int(rng.integers(1, 3))):
        sozlar.insert(int(rng.integers(0, len(sozlar) + 1)),
                      str(rng.choice(fon)))
    if rng.random() < 0.6:
        sozlar.insert(0, str(rng.choice(BOSH)))
    if rng.random() < 0.3:
        sozlar.insert(len(sozlar) - 1, str(rng.choice(OXIR)))
    for _ in range(int(rng.integers(0, 6))):
        sozlar.insert(int(rng.integers(0, len(sozlar) + 1)),
                      str(rng.choice(YORDAMCHI)))
    return sozlar


def sinonimlar():
    return sorted({t for _, fr in MAVZULAR.values() for f in fr for t in f
                   if len(t) == 2})


def birga_uchrash(gaplar, idx, oyna=2):
    C = np.zeros((len(idx), len(idx)))
    for g in gaplar:
        ids = [idx[s] for s in g]
        for i, w in enumerate(ids):
            for j in range(max(0, i - oyna), min(len(ids), i + oyna + 1)):
                if j != i:
                    C[w, ids[j]] += 1
    return C


def ppmi(C):
    N = C.sum()
    kutilgan = C.sum(1, keepdims=True) * C.sum(0, keepdims=True) / N
    with np.errstate(divide="ignore"):
        pmi = np.log(C / kutilgan)
    return np.where(C > 0, np.maximum(pmi, 0), 0.0)


def oxshashlik(E):
    En = E / np.linalg.norm(E, axis=1, keepdims=True)
    S = En @ En.T
    np.fill_diagonal(S, -np.inf)
    return S


def sinonim_top1(S, idx, juftlar):
    return np.mean([S[idx[a]].argmax() == idx[b] for a, b in juftlar])


def main() -> None:
    rng = np.random.default_rng(0)
    gaplar = [gap_yarat(rng) for _ in range(1000)]
    lugat = sorted({s for g in gaplar for s in g})
    idx = {s: i for i, s in enumerate(lugat)}
    juftlar = sinonimlar()
    C = birga_uchrash(gaplar, idx)
    P = ppmi(C)
    U, S_, _ = np.linalg.svd(P)

    print("=== 1. Singular qiymatlar ===")
    print("  birinchi 8: " + " ".join(f"{s:.1f}" for s in S_[:8]))
    energiya = np.cumsum(S_ ** 2) / np.sum(S_ ** 2)
    for k in (5, 20, 50):
        print(f"  k = {k:>2}: energiyaning {energiya[k - 1]:.1%} i")

    print(f"\n=== 2. Sinonim top-1 ({len(juftlar)} juft) ===")
    print(f"  {'vektorlar':<22} {'o_lcham':>8} {'top-1':>7}")
    Uc, Sc, _ = np.linalg.svd(C)
    satrlar = [("xom sanoq satri", C, len(lugat)),
               ("PPMI satri", P, len(lugat))]
    for k in (2, 5, 10, 20, 50):
        satrlar.append((f"PPMI + SVD k={k}", U[:, :k] * S_[:k], k))
    satrlar.append(("xom sanoq + SVD k=20", Uc[:, :20] * Sc[:20], 20))
    top1 = {}
    for nom, E, d in satrlar:
        top1[nom] = sinonim_top1(oxshashlik(E), idx, juftlar)
        print(f"  {nom:<22} {d:>8} {top1[nom]:>7.3f}")
    eng_k = max((k for k in (2, 5, 10, 20, 50)),
                key=lambda k: (top1[f"PPMI + SVD k={k}"], -k))
    print(f"  eng yaxshi k (teng bo'lsa kichigi): {eng_k}")

    print("\n=== 3. Eng yaqin so'zlar (PPMI + SVD k=20) ===")
    S = oxshashlik(U[:, :20] * S_[:20])
    for s in ["shifokor", "narx", "dastur", "jamoa", "bugun"]:
        yaqin = np.argsort(-S[idx[s]], kind="stable")[:4]
        print(f"  {s:<9} -> " + ", ".join(
            f"{lugat[i]} ({S[idx[s], i]:.2f})" for i in yaqin))

    print("\n=== 4. Xato qilingan sinonimlar (k=20) ===")
    xatolar = [(a, b) for a, b in juftlar if S[idx[a]].argmax() != idx[b]]
    for a, b in xatolar[:4]:
        eng = lugat[int(S[idx[a]].argmax())]
        print(f"  {a:<10} sinonimi {b:<11} eng yaqin: {eng}")
    if not xatolar:
        print("  hamma sinonimlar topildi")
    print("  ⭐ SVD siqadi va umumlashtiradi - lekin k ni o'lchab tanlang")


if __name__ == "__main__":
    main()

=== 1. Singular qiymatlar ===
  birinchi 8: 23.9 21.5 19.0 18.8 11.7 11.4 11.3 10.2
  k =  5: energiyaning 49.0% i
  k = 20: energiyaning 79.9% i
  k = 50: energiyaning 97.3% i

=== 2. Sinonim top-1 (23 juft) ===
  vektorlar               o_lcham   top-1
  xom sanoq satri             100   0.870
  PPMI satri                  100   1.000
  PPMI + SVD k=2                2   0.043
  PPMI + SVD k=5                5   0.739
  PPMI + SVD k=10              10   0.957
  PPMI + SVD k=20              20   1.000
  PPMI + SVD k=50              50   1.000
  xom sanoq + SVD k=20         20   0.783
  eng yaxshi k (teng bo'lsa kichigi): 20

=== 3. Eng yaqin so'zlar (PPMI + SVD k=20) ===
  shifokor  -> vrach 1.00-bob, ichdi 0.83-bob, ko'rdi 0.81-bob, tibbiyot 0.78-bob
  narx      -> baho 0.97-bob, bozorda 0.83-bob, ko'tarildi 0.74-bob, oshdi 0.70-bob
  dastur    -> ilova 0.98-bob, texnologiya 0.84-bob, yangilanish 0.82-bob, raqamli 0.78-bob
  jamoa     -> komanda 0.97-bob, qildi 0.73-bob, g'alaba 0.71-bob, qozondi 0.66-bob
  bugun     -> shu 0.96-bob, kecha 0.88-bob, o'tgan 0.84-bob, bu 0.68-bob

=== 4. Xato qilingan sinonimlar (k=20) ===
  hamma sinonimlar topildi
  ⭐ SVD siqadi va umumlashtiradi - lekin k ni o'lchab tanlang

"""SGNS ni torch da noldan o'rgatish va sinonimlarni o'lchash."""

import numpy as np
import torch
import torch.nn as nn

MAVZULAR = {
    "sport": (["o'yin", "mavsum", "chempionat", "sport", "final"], [
        [("futbolchi", "o'yinchi"), ("darvozaga",), ("gol",),
         ("urdi", "kiritdi")],
        [("jamoa", "komanda"), ("turnirda", "musobaqada"), ("g'alaba",),
         ("qozondi",)],
        [("murabbiy", "trener"), ("futbolchi", "o'yinchi"), ("maqtadi",)],
        [("jamoa", "komanda"), ("stadionda", "maydonda"), ("mashq",),
         ("qildi",)]]),
    "iqtisod": (["bozor", "iqtisodiyot", "foiz", "hukumat", "davlat"], [
        [("narx", "baho"), ("bozorda",), ("oshdi", "ko'tarildi")],
        [("bank",), ("kredit", "qarz"), ("berdi",)],
        [("hukumat",), ("soliqni",), ("kamaytirdi", "pasaytirdi")],
        [("kompaniya", "firma"), ("pul", "mablag'"), ("sarfladi",)],
        [("investor", "sarmoyador"), ("kompaniya", "firma"),
         ("qo'llab-quvvatladi",)]]),
    "texnologiya": (["raqamli", "texnologiya", "tizim", "onlayn",
                     "yangilanish"], [
        [("dasturchi", "muhandis"), ("dastur", "ilova"),
         ("yozdi", "yaratdi")],
        [("foydalanuvchi",), ("telefon", "smartfon"), ("sotib",), ("oldi",)],
        [("internet", "tarmoq"), ("tezligi",), ("oshdi", "ko'tarildi")],
        [("kompaniya", "firma"), ("dastur", "ilova"), ("chiqardi",)]]),
    "salomatlik": (["sog'liq", "tibbiyot", "kasallik", "davolash",
                    "klinika"], [
        [("shifokor", "vrach"), ("bemor", "kasal"), ("ko'rdi", "tekshirdi")],
        [("shifokor", "vrach"), ("dori", "preparat"), ("yozib",),
         ("berdi",)],
        [("bemor", "kasal"), ("kasalxonada", "shifoxonada"),
         ("davolandi",)],
        [("bemor", "kasal"), ("dori", "preparat"), ("ichdi",)]]),
}
BOSH = ["bugun", "kecha", "shu", "bu", "o'tgan"]
OXIR = ["yana", "juda", "ham", "tez", "ko'p"]
YORDAMCHI = ["va", "bu", "ham", "endi", "esa"]


def gap_yarat(rng):
    fon, freymlar = MAVZULAR[list(MAVZULAR)[int(rng.integers(0, 4))]]
    w = 1.0 / np.arange(1, len(freymlar) + 1)
    f = freymlar[rng.choice(len(freymlar), p=w / w.sum())]
    sozlar = [t[0] if len(t) == 1 or rng.random() < 0.75 else t[1]
              for t in f]
    for _ in range(int(rng.integers(1, 3))):
        sozlar.insert(int(rng.integers(0, len(sozlar) + 1)),
                      str(rng.choice(fon)))
    if rng.random() < 0.6:
        sozlar.insert(0, str(rng.choice(BOSH)))
    if rng.random() < 0.3:
        sozlar.insert(len(sozlar) - 1, str(rng.choice(OXIR)))
    for _ in range(int(rng.integers(0, 6))):
        sozlar.insert(int(rng.integers(0, len(sozlar) + 1)),
                      str(rng.choice(YORDAMCHI)))
    return sozlar


def sinonimlar():
    return sorted({t for _, fr in MAVZULAR.values() for f in fr for t in f
                   if len(t) == 2})


def mavzu_sozlari():
    """So'z -> u tegishli mavzular to'plami (umumiy so'zlarsiz)."""
    m = {}
    for nom, (fon, fr) in MAVZULAR.items():
        for s in fon + [s for f in fr for t in f for s in t]:
            m.setdefault(s, set()).add(nom)
    return m


def juftlar_top(gaplar, idx, oyna=2):
    m, c = [], []
    for g in gaplar:
        ids = [idx[s] for s in g]
        for i, w in enumerate(ids):
            for j in range(max(0, i - oyna), min(len(ids), i + oyna + 1)):
                if j != i:
                    m.append(w)
                    c.append(ids[j])
    return torch.tensor(m), torch.tensor(c)


class SkipGram(nn.Module):
    def __init__(self, v, d):
        super().__init__()
        self.markaz = nn.Embedding(v, d)
        self.kontekst = nn.Embedding(v, d)
        nn.init.uniform_(self.markaz.weight, -0.5 / d, 0.5 / d)
        nn.init.zeros_(self.kontekst.weight)

    def forward(self, m, c, neg):
        vm = self.markaz(m)                                    # (B, d)
        ijobiy = (vm * self.kontekst(c)).sum(1)                # (B,)
        salbiy = torch.bmm(self.kontekst(neg),
                           vm.unsqueeze(2)).squeeze(2)         # (B, k)
        return -(nn.functional.logsigmoid(ijobiy).mean()
                 + nn.functional.logsigmoid(-salbiy).sum(1).mean())


def baho(E, idx, juftlar, mavzu):
    En = E / np.linalg.norm(E, axis=1, keepdims=True)
    S = En @ En.T
    np.fill_diagonal(S, -np.inf)
    sin = {frozenset(j) for j in juftlar}
    soz = sorted(s for s in mavzu if s in idx)
    bir, boshqa = [], []
    for i, a in enumerate(soz):
        for b in soz[i + 1:]:
            if frozenset((a, b)) in sin:
                continue
            (bir if mavzu[a] & mavzu[b] else boshqa).append(
                S[idx[a], idx[b]])
    return (np.mean([S[idx[a]].argmax() == idx[b] for a, b in juftlar]),
            np.mean([S[idx[a], idx[b]] for a, b in juftlar]),
            np.mean(bir), np.mean(boshqa), S)


def main() -> None:
    rng = np.random.default_rng(0)
    gaplar = [gap_yarat(rng) for _ in range(1000)]
    lugat = sorted({s for g in gaplar for s in g})
    idx = {s: i for i, s in enumerate(lugat)}
    M, Cx = juftlar_top(gaplar, idx)
    soni = np.bincount(M.numpy(), minlength=len(lugat)).astype(float)
    P = torch.tensor(soni ** 0.75 / (soni ** 0.75).sum(), dtype=torch.float)
    V, d, k = len(lugat), 20, 5

    print("=== 1. O'quv juftlari va manfiy namunalar ===")
    print(f"  lug'at {V}, (markaz, kontekst) juftlari: {len(M)}")
    print(f"  birinchi gap: {' '.join(gaplar[0])}")
    print(f"  undan juftlar: " + ", ".join(
        f"({lugat[M[i]]}, {lugat[Cx[i]]})" for i in range(4)))
    ko, kam = int(soni.argmax()), int(soni.argmin())
    for nom, i in [("eng ko'p", ko), ("eng kam", kam)]:
        print(f"  {nom} uchragan '{lugat[i]}': chastota ulushi "
              f"{soni[i] / soni.sum():.4f}, count^0.75 ulushi {P[i]:.4f}")

    torch.manual_seed(0)
    model = SkipGram(V, d)
    mavzu = mavzu_sozlari()
    juftlar = sinonimlar()
    oldin = baho(model.markaz.weight.detach().numpy().copy(), idx,
                 juftlar, mavzu)

    print("\n=== 2. O'rgatish (Adam lr=0.01, batch 1024, k=5) ===")
    opt = torch.optim.Adam(model.parameters(), lr=0.01)
    g = torch.Generator().manual_seed(0)
    qadam, jami, n = 0, 0.0, 0
    while qadam < 600:
        tartib = torch.randperm(len(M), generator=g)
        for i in range(0, len(M), 1024):
            b = tartib[i:i + 1024]
            neg = torch.multinomial(P, len(b) * k, replacement=True,
                                    generator=g).view(len(b), k)
            opt.zero_grad()
            loss = model(M[b], Cx[b], neg)
            loss.backward()
            opt.step()
            jami, n, qadam = jami + loss.item(), n + 1, qadam + 1
            if qadam in (1, 50, 150, 300, 600):
                print(f"  qadam {qadam:>4}: o'rtacha loss {jami / n:.4f}")
                jami, n = 0.0, 0
            if qadam >= 600:
                break
    print(f"  boshlang'ich loss nazariyasi: (1 + k) * log 2 = "
          f"{(1 + k) * np.log(2):.4f}")

    E = model.markaz.weight.detach().numpy()
    top1, sin, bir, boshqa, S = baho(E, idx, juftlar, mavzu)

    print("\n=== 3. Eng yaqin so'zlar ===")
    for s in ["shifokor", "narx", "dastur", "jamoa", "bugun"]:
        yaqin = np.argsort(-S[idx[s]], kind="stable")[:4]
        print(f"  {s:<9} -> " + ", ".join(
            f"{lugat[i]} ({S[idx[s], i]:.2f})" for i in yaqin))

    print(f"\n=== 4. Sinonimlar ({len(juftlar)} juft) ===")
    print(f"  {'':<24} {'oldin':>8} {'keyin':>8}")
    print(f"  {'top-1 sinonim':<24} {oldin[0]:>8.3f} {top1:>8.3f}")
    print(f"  {'cos sinonimlar':<24} {oldin[1]:>8.3f} {sin:>8.3f}")
    print(f"  {'cos bir mavzu':<24} {oldin[2]:>8.3f} {bir:>8.3f}")
    print(f"  {'cos boshqa mavzu':<24} {oldin[3]:>8.3f} {boshqa:>8.3f}")
    if sin > bir > boshqa:
        print("  tartib: sinonim > bir mavzu > boshqa mavzu - "
              "taqsimot gipotezasi ishladi")
    else:
        print("  kutilgan tartib buzildi")
    print("  ⭐ Vektorlar faqat 'qo'shnini tanish' o'yinidan hosil bo'ldi")


if __name__ == "__main__":
    main()

=== 1. O'quv juftlari va manfiy namunalar ===
  lug'at 100, (markaz, kontekst) juftlari: 27296
  birinchi gap: bu shifokor bemor davolash klinika tekshirdi endi bu
  undan juftlar: (bu, shifokor), (bu, bemor), (shifokor, bu), (shifokor, bemor)
  eng ko'p uchragan 'bu': chastota ulushi 0.0696, count^0.75 ulushi 0.0470
  eng kam uchragan 'mablag'': chastota ulushi 0.0010, count^0.75 ulushi 0.0019

=== 2. O'rgatish (Adam lr=0.01, batch 1024, k=5) ===
  qadam    1: o'rtacha loss 4.1589
  qadam   50: o'rtacha loss 3.1764
  qadam  150: o'rtacha loss 2.6449
  qadam  300: o'rtacha loss 2.2980
  qadam  600: o'rtacha loss 2.1538
  boshlang'ich loss nazariyasi: (1 + k) * log 2 = 4.1589

=== 3. Eng yaqin so'zlar ===
  shifokor  -> vrach 0.99-bob, ichdi 0.84-bob, tibbiyot 0.80-bob, tekshirdi 0.80-bob
  narx      -> baho 0.93-bob, hukumat 0.77-bob, bozorda 0.75-bob, davlat 0.73-bob
  dastur    -> ilova 0.96-bob, dasturchi 0.73-bob, texnologiya 0.71-bob, muhandis 0.71-bob
  jamoa     -> komanda 0.97-bob, qozondi 0.78-bob, g'alaba 0.75-bob, musobaqada 0.75-bob
  bugun     -> kecha 0.92-bob, o'tgan 0.90-bob, shu 0.90-bob, bu 0.72-bob

=== 4. Sinonimlar (23 juft) ===
                              oldin    keyin
  top-1 sinonim               0.000    1.000
  cos sinonimlar              0.043    0.969
  cos bir mavzu               0.016    0.637
  cos boshqa mavzu           -0.001    0.261
  tartib: sinonim > bir mavzu > boshqa mavzu - taqsimot gipotezasi ishladi
  ⭐ Vektorlar faqat 'qo'shnini tanish' o'yinidan hosil bo'ldi

"""Bir xil model, bir xil qadamlar: 50 dan 2000 gapgacha korpus."""

import numpy as np
import torch
import torch.nn as nn

MAVZULAR = {
    "sport": (["o'yin", "mavsum", "chempionat", "sport", "final"], [
        [("futbolchi", "o'yinchi"), ("darvozaga",), ("gol",),
         ("urdi", "kiritdi")],
        [("jamoa", "komanda"), ("turnirda", "musobaqada"), ("g'alaba",),
         ("qozondi",)],
        [("murabbiy", "trener"), ("futbolchi", "o'yinchi"), ("maqtadi",)],
        [("jamoa", "komanda"), ("stadionda", "maydonda"), ("mashq",),
         ("qildi",)]]),
    "iqtisod": (["bozor", "iqtisodiyot", "foiz", "hukumat", "davlat"], [
        [("narx", "baho"), ("bozorda",), ("oshdi", "ko'tarildi")],
        [("bank",), ("kredit", "qarz"), ("berdi",)],
        [("hukumat",), ("soliqni",), ("kamaytirdi", "pasaytirdi")],
        [("kompaniya", "firma"), ("pul", "mablag'"), ("sarfladi",)],
        [("investor", "sarmoyador"), ("kompaniya", "firma"),
         ("qo'llab-quvvatladi",)]]),
    "texnologiya": (["raqamli", "texnologiya", "tizim", "onlayn",
                     "yangilanish"], [
        [("dasturchi", "muhandis"), ("dastur", "ilova"),
         ("yozdi", "yaratdi")],
        [("foydalanuvchi",), ("telefon", "smartfon"), ("sotib",), ("oldi",)],
        [("internet", "tarmoq"), ("tezligi",), ("oshdi", "ko'tarildi")],
        [("kompaniya", "firma"), ("dastur", "ilova"), ("chiqardi",)]]),
    "salomatlik": (["sog'liq", "tibbiyot", "kasallik", "davolash",
                    "klinika"], [
        [("shifokor", "vrach"), ("bemor", "kasal"), ("ko'rdi", "tekshirdi")],
        [("shifokor", "vrach"), ("dori", "preparat"), ("yozib",),
         ("berdi",)],
        [("bemor", "kasal"), ("kasalxonada", "shifoxonada"),
         ("davolandi",)],
        [("bemor", "kasal"), ("dori", "preparat"), ("ichdi",)]]),
}
BOSH = ["bugun", "kecha", "shu", "bu", "o'tgan"]
OXIR = ["yana", "juda", "ham", "tez", "ko'p"]
YORDAMCHI = ["va", "bu", "ham", "endi", "esa"]


def gap_yarat(rng):
    fon, freymlar = MAVZULAR[list(MAVZULAR)[int(rng.integers(0, 4))]]
    w = 1.0 / np.arange(1, len(freymlar) + 1)
    f = freymlar[rng.choice(len(freymlar), p=w / w.sum())]
    sozlar = [t[0] if len(t) == 1 or rng.random() < 0.75 else t[1]
              for t in f]
    for _ in range(int(rng.integers(1, 3))):
        sozlar.insert(int(rng.integers(0, len(sozlar) + 1)),
                      str(rng.choice(fon)))
    if rng.random() < 0.6:
        sozlar.insert(0, str(rng.choice(BOSH)))
    if rng.random() < 0.3:
        sozlar.insert(len(sozlar) - 1, str(rng.choice(OXIR)))
    for _ in range(int(rng.integers(0, 6))):
        sozlar.insert(int(rng.integers(0, len(sozlar) + 1)),
                      str(rng.choice(YORDAMCHI)))
    return sozlar


def sinonimlar():
    return sorted({t for _, fr in MAVZULAR.values() for f in fr for t in f
                   if len(t) == 2})


def juftlar_top(gaplar, idx, oyna=2):
    m, c = [], []
    for g in gaplar:
        ids = [idx[s] for s in g]
        for i, w in enumerate(ids):
            for j in range(max(0, i - oyna), min(len(ids), i + oyna + 1)):
                if j != i:
                    m.append(w)
                    c.append(ids[j])
    return torch.tensor(m), torch.tensor(c)


class SkipGram(nn.Module):
    def __init__(self, v, d):
        super().__init__()
        self.markaz = nn.Embedding(v, d)
        self.kontekst = nn.Embedding(v, d)
        nn.init.uniform_(self.markaz.weight, -0.5 / d, 0.5 / d)
        nn.init.zeros_(self.kontekst.weight)

    def forward(self, m, c, neg):
        vm = self.markaz(m)
        ijobiy = (vm * self.kontekst(c)).sum(1)
        salbiy = torch.bmm(self.kontekst(neg), vm.unsqueeze(2)).squeeze(2)
        return -(nn.functional.logsigmoid(ijobiy).mean()
                 + nn.functional.logsigmoid(-salbiy).sum(1).mean())


def sgns(gaplar, idx, seed, d=20, k=5, qadamlar=300):
    M, C = juftlar_top(gaplar, idx)
    soni = np.bincount(M.numpy(), minlength=len(idx)).astype(float)
    P = torch.tensor(soni ** 0.75 / (soni ** 0.75).sum(), dtype=torch.float)
    torch.manual_seed(seed)
    model = SkipGram(len(idx), d)
    opt = torch.optim.Adam(model.parameters(), lr=0.02)
    g = torch.Generator().manual_seed(seed)
    qadam = 0
    while qadam < qadamlar:
        tartib = torch.randperm(len(M), generator=g)
        for i in range(0, len(M), 1024):
            b = tartib[i:i + 1024]
            neg = torch.multinomial(P, len(b) * k, replacement=True,
                                    generator=g).view(len(b), k)
            opt.zero_grad()
            model(M[b], C[b], neg).backward()
            opt.step()
            qadam += 1
            if qadam >= qadamlar:
                break
    return model.markaz.weight.detach().numpy()


def top1(E, idx, juftlar):
    En = E / np.linalg.norm(E, axis=1, keepdims=True)
    S = En @ En.T
    np.fill_diagonal(S, -np.inf)
    return np.array([S[idx[a]].argmax() == idx[b] for a, b in juftlar])


def main() -> None:
    hajmlar = [50, 150, 500, 2000]
    seedlar = [0, 1, 2]
    barcha = sinonimlar()

    print("=== 1. Korpus hajmi va sinonim top-1 (3 seed) ===")
    print(f"  {'gaplar':>6} {'lug_at':>7} {'juftlar':>8} "
          f"{'noyob sinonim':>14} {'top-1':>7} {'SE':>6}")
    natija = {}
    for n in hajmlar:
        qiymat, noyob, v, nj = [], [], [], []
        for s in seedlar:
            rng = np.random.default_rng(100 + s)
            gaplar = [gap_yarat(rng) for _ in range(n)]
            lugat = sorted({w for g in gaplar for w in g})
            idx = {w: i for i, w in enumerate(lugat)}
            juftlar = [j for j in barcha if j[0] in idx and j[1] in idx]
            v.append(len(lugat))
            nj.append(len(juftlar))
            soni = {w: sum(g.count(w) for g in gaplar) for w in lugat}
            noyob.append(np.median([soni[b] for _, b in juftlar]))
            E = sgns(gaplar, idx, seed=s)
            t = top1(E, idx, juftlar)
            qiymat.append(t.mean())
        q = np.array(qiymat)
        natija[n] = q
        se = q.std(ddof=1) / np.sqrt(len(q))
        print(f"  {n:>6} {np.mean(v):>7.0f} {np.mean(nj):>5.1f}/{len(barcha)} "
              f"{np.median(noyob):>14.0f} {q.mean():>7.3f} {se:>6.3f}")
    print("  (lug'at, juftlar - 3 seed o'rtachasi; noyob sinonim - ikkinchi,")
    print("   kam uchraydigan variantning mediana chastotasi)")

    print("\n=== 2. Qo'shni hajmlar orasidagi farq ===")
    for a, b in zip(hajmlar, hajmlar[1:]):
        f = natija[b] - natija[a]
        se = f.std(ddof=1) / np.sqrt(len(f))
        belgi = "sezilarli" if f.mean() > 2 * se else "sezilarli emas"
        print(f"  {a:>5} -> {b:>5}: {f.mean():+.3f}  SE {se:.3f}  {belgi}")
    print("  ⭐ Bir xil model - farq faqat ma'lumot hajmidan")


if __name__ == "__main__":
    main()

=== 1. Korpus hajmi va sinonim top-1 (3 seed) ===
  gaplar  lug_at  juftlar  noyob sinonim   top-1     SE
      50      85  12.3/23              2   0.139  0.032
     150      97  21.0/23              3   0.413  0.008
     500     100  23.0/23             10   0.899  0.038
    2000     100  23.0/23             46   1.000  0.000
  (lug'at, juftlar - 3 seed o'rtachasi; noyob sinonim - ikkinchi,
   kam uchraydigan variantning mediana chastotasi)

=== 2. Qo'shni hajmlar orasidagi farq ===
     50 ->   150: +0.274  SE 0.024  sezilarli
    150 ->   500: +0.486  SE 0.035  sezilarli
    500 ->  2000: +0.101  SE 0.038  sezilarli
  ⭐ Bir xil model - farq faqat ma'lumot hajmidan

pmi = np.log(C * N / (qator * ustun))          # -inf va ogohlantirish  # ⚠️
ppmi = np.where(C > 0, np.maximum(pmi, 0), 0)   # errstate bilan         # ✅

S[w].argsort()[::-1][:5]                   # birinchisi - o'zi     # ⚠️
np.fill_diagonal(S, -np.inf); S[w].argsort()[::-1][:5]              # ✅

np.linalg.norm(E - E[w], axis=1).argsort()  # uzunlik = chastota    # ⚠️
En = E / np.linalg.norm(E, axis=1, keepdims=True); En @ En[w]       # ✅

neg = torch.randint(0, V, (B, k))                                    # ⚠️
neg = torch.multinomial(soni ** 0.75 / Z, B * k, replacement=True)   # ✅

E = model.kontekst.weight                  # nol bilan boshlangan   # ⚠️
E = model.markaz.weight                    # (yoki ikkalasi yig'indisi)  # ✅

print(cos(E[a], E[b]))                     # 0.7 - yaxshimi?        # ⚠️
# o'rgatishdan OLDIN va boshqa mavzu juftlari bilan solishtiring    # ✅

yaqin(E, "noyob_soz")                      # 3 marta uchragan       # ⚠️
if soni[w] >= 5: yaqin(E, w)               # min_count             # ✅

1.  # one-hot da ikki xil so'z orasidagi kosinus?
2.  # C[w, c] nimani sanaydi?
3.  # PMI formulasi?
4.  # nima uchun PPMI da manfiy qiymatlar tashlanadi?
5.  # SVD dan keyingi so'z vektori?
6.  # skip-gram nimani bashorat qiladi?
7.  # negative sampling loss formulasi?
8.  # manfiy namunalar taqsimoti?
9.  # SGNS boshlang'ich loss i (k = 5)?
10. # nega kosinus, Evklid emas?
11. # analogiya formulasi?
12. # nega kam uchraydigan sinonim birinchi zarar ko'radi?

1.  pmi = np.log(C * N / np.outer(C.sum(1), C.sum(0)))
    ppmi = np.maximum(pmi, 0)

2.  yaqin = (En @ En[w]).argsort()[::-1][:5]

3.  neg = torch.randint(0, V, (B, 5))

4.  E = model.kontekst.weight.detach().numpy()

5.  masofa = np.linalg.norm(E - E[w], axis=1); yaqin = masofa.argsort()[:5]