14.14-dars: Amaliyot — to'liq klassifikatsiya loyihasi

1. VAZIFA      — nima bashorat qilinadi, qaror nima, metrika qaysi (12.2, 12.7)
2. AJRATISH    — vaqt bo'yicha 12.3-bob, stratifikatsiya bilan
3. BAZA        — joriy qoida, DummyClassifier, chiziqli model (12.6, 14.1)
4. BELGILAR    — matn (TF-IDF - 14.11) + jadval belgilari
5. NOMZODLAR   — 2-3 ta, bir xil CV da (14.7, 14.12)
6. SOZLASH     — GridSearch/Randomized, faqat o'quvda
7. QAROR       — chegara 12.7-bob yoki byudjet; kalibrlash 14.10-bob
8. SELEKTIV    — past ishonchli holatlarni insonga 14.13-bob
9. XATOLAR     — segment, ishonch, turlar 14.13-bob
10. HISOBOT    — natija, cheklovlar, monitoring

YO'NALTIRISH (ko'p sinfli, muvozanatli):
  metrika: F1 macro (barcha bo'limlar teng muhim) + chalkashlik matritsasi
  qaror: eng yuqori ehtimolli sinf yoki "operatorga"

ESKALATSIYA (binar, 2-3% musbat):
  metrika: PR AUC + recall@byudjet 12.7-bob
  qaror: kuniga N ta eng yuqori ballni belgilash (byudjet)

MUHIM: bitta metrika ikkala vazifaga mos kelmaydi

Yo'naltirish bazalari:
  · eng ko'p uchraydigan bo'lim (DummyClassifier)
  · kalit so'z qoidalari (joriy tizim)
Nomzodlar: TF-IDF + LinearSVC / LogReg / ComplementNB 14.11-bob

Eskalatsiya bazalari:
  · tasodifiy tanlov (byudjet ichida)
  · oddiy qoida ("uchinchi marta murojaat")
Nomzodlar: LogReg (kalibrlangan), boosting, chiziqli SVM + kalibrlash

Yo'naltirishda: chegara emas, ISHONCH chegarasi
  max(proba) < t bo'lsa -> operatorga (selektiv bashorat - 14.13)
  t ni validatsiyada tanlang: "qancha avtomatlashtiramiz va qanday aniqlikda"

Eskalatsiyada: byudjet (top-N) yoki narx chegarasi 12.7-bob
  ehtimol qarorga kirsa -> kalibrlash (14.10)

Qamrov (coverage) — avtomatlashtirilgan ulush
Aniqlik@qamrov — avtomatlashtirilgan qismdagi aniqlik

Egri chiziq: qamrovni kamaytirsangiz, aniqlik ortadi
  100% qamrov -> 0.83 aniqlik
   88% qamrov -> 0.91 aniqlik
   70% qamrov -> 0.96 aniqlik

Tanlov: operator quvvati va xato narxidan

1. Savol va qaror (ikki vazifa)
2. Ma'lumot, ajratish, baza
3. Model natijalari (CV +- SD va test)
4. Qaror siyosati: ishonch chegarasi va byudjet
5. Xatolar tahlili: segmentlar, ishonch, turlar
6. Cheklovlar va xavflar
7. Monitoring va qayta o'qitish rejasi
8. Kutilgan ta'sir (vaqt tejash, eskalatsiya kamayishi)

import numpy as np
from sklearn.calibration import CalibratedClassifierCV
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import average_precision_score, f1_score
from sklearn.pipeline import Pipeline

# yo'naltirish (ko'p sinfli)
yol = Pipeline([("vec", TfidfVectorizer(min_df=2, ngram_range=(1, 2))),
                ("m", LogisticRegression(max_iter=3000))])

# selektiv bashorat
proba = yol.predict_proba(X_val)
ishonch = proba.max(axis=1)
avtomat = ishonch >= t

# eskalatsiya (binar, byudjet)
eskal = CalibratedClassifierCV(asos, method="sigmoid", cv=5)
top = np.argsort(eskal.predict_proba(X_te)[:, 1])[::-1][:BYUDJET]
QOIDA: har vazifaga o'z metrikasi · joriy tizim — baza · qaror siyosati alohida

1. Vazifa/metrika  2. Ajratish  3. Baza  4. Belgilar  5. Nomzodlar
6. Sozlash  7. Chegara/byudjet/kalibrlash  8. Selektiv  9. Xatolar  10. Hisobot

"""1-bosqich: ikki vazifa, baza va nomzodlar (real pandas/sklearn)."""

import numpy as np
import pandas as pd
from sklearn.dummy import DummyClassifier
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import average_precision_score, f1_score
from sklearn.naive_bayes import ComplementNB
from sklearn.pipeline import Pipeline
from sklearn.svm import LinearSVC

BOLIMLAR = ["internet", "mobil", "tolov", "uskuna", "shartnoma"]
LUGAT = {
    "internet": ["internet", "tezlik", "uzilish", "router", "signal"],
    "mobil": ["mobil", "aloqa", "raqam", "sim", "tarif"],
    "tolov": ["tolov", "hisob", "chegirma", "qarz", "kvitansiya"],
    "uskuna": ["uskuna", "modem", "kabel", "qurilma", "almashtirish"],
    "shartnoma": ["shartnoma", "bekor", "muddat", "band", "shart"],
}
UMUMIY = ["iltimos", "yordam", "muammo", "javob", "kutyapman", "hurmat",
          "murojaat", "masala"]


def yarat(n: int, seed: int) -> pd.DataFrame:
    """Murojaat matni + jadval belgilari; eskalatsiya kam uchraydi."""
    rng = np.random.default_rng(seed)
    qatorlar = []
    for i in range(n):
        bolim = str(rng.choice(BOLIMLAR))
        sozlar = list(rng.choice(LUGAT[bolim], 4))
        sozlar += list(rng.choice(UMUMIY, 7))
        boshqa = str(rng.choice(BOLIMLAR))
        sozlar += list(rng.choice(LUGAT[boshqa], 3))       # chalkashtiruvchi
        rng.shuffle(sozlar)
        takror = int(rng.poisson(0.6))                     # oldingi murojaatlar
        kutish = float(rng.gamma(2.0, 8.0))                # kutish daqiqalari
        vip = bool(rng.random() < 0.12)
        ball = (-4.2 + 0.75 * takror + 0.035 * kutish + 0.9 * vip
                + 0.5 * (bolim == "tolov"))
        eskalatsiya = int(rng.random() < 1 / (1 + np.exp(-ball)))
        qatorlar.append({"matn": " ".join(sozlar), "bolim": bolim,
                         "takror": takror, "kutish": kutish, "vip": vip,
                         "eskalatsiya": eskalatsiya})
    return pd.DataFrame(qatorlar)


def main() -> None:
    tr = yarat(6000, seed=1)
    val = yarat(2500, seed=2)

    print("=== 1. Ma'lumot ===")
    print(f"  o'quv {len(tr)}, validatsiya {len(val)}")
    print(f"  bo'limlar ulushi: "
          f"{ {k: round(v, 3) for k, v in tr['bolim'].value_counts(normalize=True).items()} }")
    print(f"  eskalatsiya ulushi: {tr['eskalatsiya'].mean():.2%} "
          f"({tr['eskalatsiya'].sum()} ta)")

    print("\n=== 2. Yo'naltirish: baza va nomzodlar ===")
    baza = DummyClassifier(strategy="most_frequent").fit(tr["matn"], tr["bolim"])
    print(f"  baza (eng ko'p bo'lim): F1 macro "
          f"{f1_score(val['bolim'], baza.predict(val['matn']), average='macro', zero_division=0):.4f}")
    nomzodlar = {
        "ComplementNB": ComplementNB(alpha=0.3),
        "LogReg": LogisticRegression(max_iter=3000, C=5.0),
        "LinearSVC": LinearSVC(C=1.0, max_iter=10_000),
    }
    for nom, m in nomzodlar.items():
        q = Pipeline([("vec", TfidfVectorizer(min_df=2, ngram_range=(1, 2))),
                      ("m", m)]).fit(tr["matn"], tr["bolim"])
        print(f"  {nom:<13}: F1 macro "
              f"{f1_score(val['bolim'], q.predict(val['matn']), average='macro'):.4f}")

    print("\n=== 3. Eskalatsiya: baza va nomzodlar ===")
    SON = ["takror", "kutish", "vip"]
    Xtr = tr[SON].astype(float)
    Xval = val[SON].astype(float)
    ytr, yval = tr["eskalatsiya"], val["eskalatsiya"]
    print(f"  baza (tasodifiy tanlov) PR AUC = {yval.mean():.4f}")
    qoida = tr["takror"].to_numpy()
    print(f"  qoida (takror soni): PR AUC "
          f"{average_precision_score(yval, val['takror']):.4f}")
    from sklearn.preprocessing import StandardScaler
    log = Pipeline([("sc", StandardScaler()),
                    ("m", LogisticRegression(max_iter=3000))]).fit(Xtr, ytr)
    p = log.predict_proba(Xval)[:, 1]
    print(f"  LogReg: PR AUC {average_precision_score(yval, p):.4f}")

    print("\n=== 4. Metrikalar har xil ===")
    print("  yo'naltirish: F1 macro (bo'limlar teng muhim)")
    print(f"  eskalatsiya: PR AUC (baza {yval.mean():.4f}) — accuracy ma'nosiz")
    print(f"  'hech kim eskalatsiya qilmaydi' aniqligi: {1 - yval.mean():.4f}")
    print("  ⭐ Har vazifa o'z metrikasi bilan baholanadi")


if __name__ == "__main__":
    main()

=== 1. Ma'lumot ===
  o'quv 6000, validatsiya 2500
  bo'limlar ulushi: {'shartnoma': 0.205, 'tolov': 0.203, 'mobil': 0.203, 'uskuna': 0.196, 'internet': 0.194}
  eskalatsiya ulushi: 6.15% (369 ta)

=== 2. Yo'naltirish: baza va nomzodlar ===
  baza (eng ko'p bo'lim): F1 macro 0.0694
  ComplementNB : F1 macro 0.9358
  LogReg       : F1 macro 0.9996
  LinearSVC    : F1 macro 1.0000

=== 3. Eskalatsiya: baza va nomzodlar ===
  baza (tasodifiy tanlov) PR AUC = 0.0660
  qoida (takror soni): PR AUC 0.1164
  LogReg: PR AUC 0.1902

=== 4. Metrikalar har xil ===
  yo'naltirish: F1 macro (bo'limlar teng muhim)
  eskalatsiya: PR AUC (baza 0.0660) — accuracy ma'nosiz
  'hech kim eskalatsiya qilmaydi' aniqligi: 0.9340
  ⭐ Har vazifa o'z metrikasi bilan baholanadi

"""2-bosqich: yo'naltirish modelini sozlash va ishonch chegarasi (real sklearn)."""

import numpy as np
import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import f1_score
from sklearn.model_selection import GridSearchCV, StratifiedKFold
from sklearn.pipeline import Pipeline

BOLIMLAR = ["internet", "mobil", "tolov", "uskuna", "shartnoma"]
LUGAT = {
    "internet": ["internet", "tezlik", "uzilish", "router", "signal"],
    "mobil": ["mobil", "aloqa", "raqam", "sim", "tarif"],
    "tolov": ["tolov", "hisob", "chegirma", "qarz", "kvitansiya"],
    "uskuna": ["uskuna", "modem", "kabel", "qurilma", "almashtirish"],
    "shartnoma": ["shartnoma", "bekor", "muddat", "band", "shart"],
}
UMUMIY = ["iltimos", "yordam", "muammo", "javob", "kutyapman", "hurmat",
          "murojaat", "masala"]


def yarat(n: int, seed: int) -> pd.DataFrame:
    rng = np.random.default_rng(seed)
    qatorlar = []
    for _ in range(n):
        bolim = str(rng.choice(BOLIMLAR))
        sozlar = list(rng.choice(LUGAT[bolim], 4))
        sozlar += list(rng.choice(UMUMIY, 7))
        boshqa = str(rng.choice(BOLIMLAR))
        sozlar += list(rng.choice(LUGAT[boshqa], 3))
        rng.shuffle(sozlar)
        qatorlar.append({"matn": " ".join(sozlar), "bolim": bolim})
    return pd.DataFrame(qatorlar)


def main() -> None:
    tr = yarat(6000, seed=1)
    val = yarat(2500, seed=2)
    cv = StratifiedKFold(4, shuffle=True, random_state=0)

    print("=== 1. Sozlash ===")
    quvur = Pipeline([("vec", TfidfVectorizer()),
                      ("m", LogisticRegression(max_iter=3000))])
    setka = {"vec__ngram_range": [(1, 1), (1, 2)],
             "vec__min_df": [1, 2],
             "m__C": [1.0, 5.0, 20.0]}
    qidiruv = GridSearchCV(quvur, setka, cv=cv, scoring="f1_macro").fit(
        tr["matn"], tr["bolim"])
    print(f"  eng yaxshi: {qidiruv.best_params_}")
    print(f"  CV F1 macro = {qidiruv.best_score_:.4f}  (siljigan — 14.12)")

    model = qidiruv.best_estimator_
    proba = model.predict_proba(val["matn"])
    pred = model.classes_[proba.argmax(axis=1)]
    ishonch = proba.max(axis=1)
    togri = pred == val["bolim"].to_numpy()

    print("\n=== 2. Validatsiya natijasi ===")
    print(f"  F1 macro = {f1_score(val['bolim'], pred, average='macro'):.4f}")
    print(f"  aniqlik  = {togri.mean():.4f}")

    print("\n=== 3. Selektiv bashorat: qamrov va aniqlik ===")
    print(f"  {'chegara':>8} {'qamrov':>8} {'avtomat aniqligi':>18}")
    for t in [0.0, 0.35, 0.45, 0.55, 0.70]:
        avtomat = ishonch >= t
        if avtomat.sum() == 0:
            continue
        print(f"  {t:>8.2f} {avtomat.mean():>8.1%} "
              f"{togri[avtomat].mean():>18.4f}")

    print("\n=== 4. Operator quvvatiga moslash ===")
    for quvvat in [0.05, 0.12, 0.25]:
        chegara = np.quantile(ishonch, quvvat)
        avtomat = ishonch > chegara
        print(f"  operatorga {quvvat:.0%}: chegara {chegara:.3f}, "
              f"avtomat aniqligi {togri[avtomat].mean():.4f}, "
              f"operatordagi aniqlik {togri[~avtomat].mean():.4f}")
    print("  ⭐ Model hamma narsani hal qilishi shart emas")


if __name__ == "__main__":
    main()

=== 1. Sozlash ===
  eng yaxshi: {'m__C': 1.0, 'vec__min_df': 1, 'vec__ngram_range': (1, 1)}
  CV F1 macro = 1.0000  (siljigan — 14.12)

=== 2. Validatsiya natijasi ===
  F1 macro = 1.0000
  aniqlik  = 1.0000

=== 3. Selektiv bashorat: qamrov va aniqlik ===
   chegara   qamrov   avtomat aniqligi
      0.00   100.0%             1.0000
      0.35   100.0%             1.0000
      0.45   100.0%             1.0000
      0.55   100.0%             1.0000
      0.70   100.0%             1.0000

=== 4. Operator quvvatiga moslash ===
  operatorga 5%: chegara 0.835, avtomat aniqligi 1.0000, operatordagi aniqlik 1.0000
  operatorga 12%: chegara 0.851, avtomat aniqligi 1.0000, operatordagi aniqlik 1.0000
  operatorga 25%: chegara 0.869, avtomat aniqligi 1.0000, operatordagi aniqlik 1.0000
  ⭐ Model hamma narsani hal qilishi shart emas

"""3-bosqich: nomutanosib vazifa, kalibrlash va top-N qaror (real sklearn)."""

import numpy as np
import pandas as pd
from sklearn.calibration import CalibratedClassifierCV, calibration_curve
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import average_precision_score, brier_score_loss
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler


def yarat(n: int, seed: int) -> pd.DataFrame:
    rng = np.random.default_rng(seed)
    takror = rng.poisson(0.6, n).astype(float)
    kutish = rng.gamma(2.0, 8.0, n)
    vip = (rng.random(n) < 0.12).astype(float)
    tolov = (rng.random(n) < 0.2).astype(float)
    ball = -4.2 + 0.75 * takror + 0.035 * kutish + 0.9 * vip + 0.5 * tolov
    y = (rng.random(n) < 1 / (1 + np.exp(-ball))).astype(int)
    return pd.DataFrame({"takror": takror, "kutish": kutish, "vip": vip,
                         "tolov": tolov, "eskalatsiya": y})


def main() -> None:
    tr = yarat(12_000, seed=1)
    te = yarat(6000, seed=2)
    SON = ["takror", "kutish", "vip", "tolov"]
    Xtr, ytr = tr[SON], tr["eskalatsiya"]
    Xte, yte = te[SON], te["eskalatsiya"]

    print("=== 1. Vazifa ===")
    print(f"  o'quv {len(tr)} (musbat {ytr.sum()}), "
          f"test {len(te)} (musbat {yte.sum()}, {yte.mean():.2%})")

    print("\n=== 2. Nomzodlar (PR AUC) ===")
    modellar = {
        "LogReg": Pipeline([("sc", StandardScaler()),
                            ("m", LogisticRegression(max_iter=3000))]),
        "O'rmon": RandomForestClassifier(n_estimators=300, min_samples_leaf=20,
                                         random_state=0),
        "O'rmon+kalib": CalibratedClassifierCV(
            RandomForestClassifier(n_estimators=300, min_samples_leaf=20,
                                   random_state=0),
            method="isotonic", cv=5),
    }
    bashoratlar = {}
    for nom, m in modellar.items():
        m.fit(Xtr, ytr)
        p = m.predict_proba(Xte)[:, 1]
        bashoratlar[nom] = p
        print(f"  {nom:<13}: PR AUC {average_precision_score(yte, p):.4f} "
              f"(baza {yte.mean():.4f}), Brier {brier_score_loss(yte, p):.5f}")

    print("\n=== 3. Kalibrlash sifati ===")
    for nom in ["LogReg", "O'rmon", "O'rmon+kalib"]:
        h, b = calibration_curve(yte, bashoratlar[nom], n_bins=4,
                                 strategy="quantile")
        juftlar = ", ".join(f"{bb:.3f}->{hh:.3f}" for bb, hh in zip(b, h))
        print(f"  {nom:<13}: {juftlar}")

    print("\n=== 4. Byudjet qarori (kuniga 40 ta) ===")
    kunlik = 40
    ulush = kunlik / 6000 * len(te)
    N = int(ulush)
    print(f"  test davrida byudjet: {N} ta murojaat")
    print(f"  {'model':<13} {'precision@N':>12} {'recall@N':>10} {'lift':>7}")
    for nom, p in bashoratlar.items():
        top = np.argsort(p)[::-1][:N]
        prec = yte.to_numpy()[top].mean()
        rec = yte.to_numpy()[top].sum() / yte.sum()
        print(f"  {nom:<13} {prec:>12.4f} {rec:>10.4f} "
              f"{prec / yte.mean():>7.1f}x")
    tasodif = np.random.default_rng(0).permutation(len(te))[:N]
    print(f"  tasodifiy tanlov: precision {yte.to_numpy()[tasodif].mean():.4f}")
    print("  ⭐ Byudjet cheklovi chegarani o'zi belgilaydi")


if __name__ == "__main__":
    main()

=== 1. Vazifa ===
  o'quv 12000 (musbat 744), test 6000 (musbat 371, 6.18%)

=== 2. Nomzodlar (PR AUC) ===
  LogReg       : PR AUC 0.2314 (baza 0.0618), Brier 0.05308
  O'rmon       : PR AUC 0.1769 (baza 0.0618), Brier 0.05544
  O'rmon+kalib : PR AUC 0.1809 (baza 0.0618), Brier 0.05495

=== 3. Kalibrlash sifati ===
  LogReg       : 0.017->0.019, 0.030->0.026, 0.052->0.057, 0.150->0.145
  O'rmon       : 0.002->0.030, 0.019->0.033, 0.059->0.057, 0.166->0.127
  O'rmon+kalib : 0.019->0.024, 0.036->0.039, 0.052->0.048, 0.141->0.137

=== 4. Byudjet qarori (kuniga 40 ta) ===
  test davrida byudjet: 40 ta murojaat
  model          precision@N   recall@N    lift
  LogReg              0.6250     0.0674    10.1x
  O'rmon              0.4000     0.0431     6.5x
  O'rmon+kalib        0.4250     0.0458     6.9x
  tasodifiy tanlov: precision 0.0250
  ⭐ Byudjet cheklovi chegarani o'zi belgilaydi

"""4-bosqich: segmentlar, ishonch va hisobot (real pandas/sklearn)."""

import numpy as np
import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix, f1_score
from sklearn.pipeline import Pipeline

BOLIMLAR = ["internet", "mobil", "tolov", "uskuna", "shartnoma"]
LUGAT = {
    "internet": ["internet", "tezlik", "uzilish", "router", "signal"],
    "mobil": ["mobil", "aloqa", "raqam", "sim", "tarif"],
    "tolov": ["tolov", "hisob", "chegirma", "qarz", "kvitansiya"],
    "uskuna": ["uskuna", "modem", "kabel", "qurilma", "almashtirish"],
    "shartnoma": ["shartnoma", "bekor", "muddat", "band", "shart"],
}
UMUMIY = ["iltimos", "yordam", "muammo", "javob", "kutyapman", "hurmat",
          "murojaat", "masala"]


def yarat(n: int, seed: int) -> pd.DataFrame:
    """Qisqa murojaatlarda belgilar kam — bu segment qiyinroq."""
    rng = np.random.default_rng(seed)
    qatorlar = []
    for _ in range(n):
        bolim = str(rng.choice(BOLIMLAR))
        qisqa = rng.random() < 0.25
        mavzu_soni = 2 if qisqa else 4
        umumiy_soni = 2 if qisqa else 7
        sozlar = list(rng.choice(LUGAT[bolim], mavzu_soni))
        sozlar += list(rng.choice(UMUMIY, umumiy_soni))
        boshqa = str(rng.choice(BOLIMLAR))
        sozlar += list(rng.choice(LUGAT[boshqa], 3))
        rng.shuffle(sozlar)
        qatorlar.append({"matn": " ".join(sozlar), "bolim": bolim,
                         "qisqa": qisqa})
    return pd.DataFrame(qatorlar)


def main() -> None:
    tr = yarat(7000, seed=1)
    val = yarat(3000, seed=2)
    model = Pipeline([("vec", TfidfVectorizer(min_df=2, ngram_range=(1, 2))),
                      ("m", LogisticRegression(max_iter=3000, C=5.0))]).fit(
        tr["matn"], tr["bolim"])

    proba = model.predict_proba(val["matn"])
    pred = model.classes_[proba.argmax(axis=1)]
    nat = val.copy()
    nat["pred"] = pred
    nat["ishonch"] = proba.max(axis=1)
    nat["xato"] = nat["pred"] != nat["bolim"]

    print("=== 1. Umumiy natija ===")
    print(f"  F1 macro {f1_score(val['bolim'], pred, average='macro'):.4f}, "
          f"aniqlik {1 - nat['xato'].mean():.4f}")

    print("\n=== 2. Segment: qisqa va uzun murojaatlar ===")
    for qiymat, qism in nat.groupby("qisqa"):
        nom = "qisqa" if qiymat else "uzun"
        print(f"  {nom:<6}: n={len(qism):>4}, xato ulushi "
              f"{qism['xato'].mean():.4f}, "
              f"barcha xatolarning {qism['xato'].sum() / nat['xato'].sum():.1%} i, "
              f"o'rtacha ishonch {qism['ishonch'].mean():.3f}")

    print("\n=== 3. Chalkashlik: eng ko'p adashtiradigan juftliklar ===")
    cm = confusion_matrix(val["bolim"], pred, labels=model.classes_.tolist())
    cm_toza = cm.copy()
    np.fill_diagonal(cm_toza, 0)
    tartib = np.argsort(cm_toza, axis=None)[::-1][:4]
    for idx in tartib:
        a, b = np.unravel_index(idx, cm.shape)
        if cm_toza[a, b]:
            print(f"  {model.classes_[a]} -> {model.classes_[b]}: "
                  f"{cm_toza[a, b]} marta "
                  f"({cm_toza[a, b] / cm[a].sum():.1%} shu bo'limdan)")

    print("\n=== 4. Yakuniy hisobot ===")
    chegara = np.quantile(nat["ishonch"], 0.12)
    avtomat = nat["ishonch"] > chegara
    print("  1) Vazifa: murojaatni 5 bo'limga yo'naltirish + eskalatsiya xavfi")
    print(f"  2) Ma'lumot: o'quv {len(tr)}, validatsiya {len(val)}")
    print(f"  3) Yo'naltirish: F1 macro "
          f"{f1_score(val['bolim'], pred, average='macro'):.4f}")
    print(f"  4) Siyosat: eng past ishonchli 12% operatorga "
          f"(chegara {chegara:.3f})")
    print(f"     avtomat qismda aniqlik {(~nat['xato'])[avtomat].mean():.4f}, "
          f"operatorda {(~nat['xato'])[~avtomat].mean():.4f}")
    qisqa_ulush = nat.loc[nat['xato'], 'qisqa'].mean()
    print(f"  5) Xatolar tahlili: xatolarning {qisqa_ulush:.0%} i qisqa "
          f"murojaatlarda")
    print("  6) Cheklovlar: sun'iy ma'lumot, yorliq sifati tekshirilmagan, drift")
    print("  7) Monitoring: haftalik F1 macro, segment xatolari, ishonch taqsimoti")
    print("  8) Kutilgan ta'sir: operator vaqti 88% murojaatda tejaladi")
    print("  ⭐ 14-qism yakuni: vazifa - metrika - qaror siyosati - tahlil")


if __name__ == "__main__":
    main()

=== 1. Umumiy natija ===
  F1 macro 0.7191, aniqlik 0.7190

=== 2. Segment: qisqa va uzun murojaatlar ===
  uzun  : n=2222, xato ulushi 0.1458, barcha xatolarning 38.4% i, o'rtacha ishonch 0.807
  qisqa : n= 778, xato ulushi 0.6671, barcha xatolarning 61.6% i, o'rtacha ishonch 0.829

=== 3. Chalkashlik: eng ko'p adashtiradigan juftliklar ===
  shartnoma -> uskuna: 54 marta (8.8% shu bo'limdan)
  shartnoma -> internet: 51 marta (8.3% shu bo'limdan)
  uskuna -> internet: 50 marta (7.8% shu bo'limdan)
  tolov -> shartnoma: 47 marta (8.0% shu bo'limdan)

=== 4. Yakuniy hisobot ===
  1) Vazifa: murojaatni 5 bo'limga yo'naltirish + eskalatsiya xavfi
  2) Ma'lumot: o'quv 7000, validatsiya 3000
  3) Yo'naltirish: F1 macro 0.7191
  4) Siyosat: eng past ishonchli 12% operatorga (chegara 0.583)
     avtomat qismda aniqlik 0.7458, operatorda 0.5222
  5) Xatolar tahlili: xatolarning 62% i qisqa murojaatlarda
  6) Cheklovlar: sun'iy ma'lumot, yorliq sifati tekshirilmagan, drift
  7) Monitoring: haftalik F1 macro, segment xatolari, ishonch taqsimoti
  8) Kutilgan ta'sir: operator vaqti 88% murojaatda tejaladi
  ⭐ 14-qism yakuni: vazifa - metrika - qaror siyosati - tahlil

print(f"loyiha aniqligi: {acc:.3f}")                              # ⚠️
# yo'naltirish: F1 macro; eskalatsiya: PR AUC                     # ✅

print("F1 0.83 — yaxshi")                                         # ⚠️
print("F1 0.83 (joriy qoida 0.61, dummy 0.07)")                   # ✅

X = TfidfVectorizer().fit_transform(hamma_matn)                   # ⚠️
Pipeline([("vec", TfidfVectorizer()), ("m", model)])              # ✅

avtomat = model.predict_proba(X).max(axis=1) > 0.9                # ⚠️
# kalibrlangan modelda yoki chegarani validatsiyada tanlang       # ✅

print(model.score(X_te, y_te))         # 97.6%                    # ⚠️
print(average_precision_score(y_te, p))                           # ✅

# test xatolarini tahlil qilib, o'sha testda baholash             # ⚠️
# tahlil validatsiyada 14.13-bob                                    # ✅

# modelni ishga tushirib, unutish                                 # ⚠️
# haftalik metrika, segment xatolari, drift kuzatuvi              # ✅

1.  # ikki vazifa qanday metrikalar?
2.  # yo'naltirishda qaror qanday?
3.  # eskalatsiyada?
4.  # selektiv bashorat nima?
5.  # qamrov nima?
6.  # ishonch qanday hisoblanadi?
7.  # kalibrlash nega kerak?
8.  # baza nima bo'lishi kerak?
9.  # xatolar tahlili qaysi to'plamda?
10. # matn belgilari qayerda yaratiladi?
11. # byudjet qarori nima?
12. # hisobotda nechta bo'lim?

1.  print(f"loyiha aniqligi {acc:.3f}")   # ikki vazifa

2.  print("F1 0.83 — yaxshi")   # bazasiz

3.  X = TfidfVectorizer().fit_transform(hammasi)

4.  print(model.score(X_te, y_te))   # 2.4% musbat sinf

5.  # test xatolarini tahlil qilib, o'sha testda baholash