14.13-dars: Xatolar tahlili

Umumiy metrika bitta son beradi; segmentlar qayerda muammo borligini ko'rsatadi

Segmentlar: kategoriyalar (hudud, tarif), diapazonlar (yosh, summa),
            vaqt (oy, hafta kuni), ma'lumot manbai, til, qurilma

Har segment uchun: n, sinf ulushi, precision/recall/F1, xato ulushi
Diqqat: kichik segmentlarda metrika juda shovqinli (n >= 50-100 bo'lsin)

Qidirish kerak:
  · metrikasi umumiy o'rtachadan sezilarli past segment
  · xatolarning katta ulushini beradigan segment (n katta bo'lgani uchun)

Bashoratlarni ishonch bo'yicha guruhlang: max(predict_proba)

Kutilgan naqsh:
  yuqori ishonch -> kam xato;  past ishonch -> ko'p xato

Anomaliyalar:
  · YUQORI ishonchli XATOLAR — eng qimmatli topilma
      (yorliq xatosi, leakage, model tuzilmasi noto'g'ri)
  · past ishonchda ham aniqlik yuqori — model ehtiyotkor (kalibrlash - 14.10)

Amaliy: past ishonchli holatlarni INSON ko'rigiga yo'naltirish
        ("abstain" yoki "selective prediction")

Belgilar: bir xil (yoki juda o'xshash) namunalarda TURLI yorliq
          model ishonch bilan yanglishgan holatlar
          yorliqlovchilar orasidagi kelishmovchilik

Tekshirish usullari:
  1. Dublikat/yaqin namunalarni topib, yorliqlarini solishtirish
  2. Yuqori ishonchli xatolarni QO'LDA ko'rib chiqish (20-50 ta)
  3. Cross-validation bashorati bilan yorliq mos kelmagan holatlar (confident learning)

Yorliq shovqini 5-10% bo'lsa — u Bayes xatosiga o'xshab ishlaydi (14.1)

Xatolarni SABABI bo'yicha guruhlang:

1. YORLIQ xatosi        — haqiqiy yorliq noto'g'ri
2. BELGI yetishmasligi  — kerakli ma'lumot belgilar ichida yo'q
3. AJRALMAS holat       — sinflar haqiqatan ustma-ust (Bayes xatosi - 14.1)
4. KAM NAMUNA           — shu turdagi holatlar o'quvda kam
5. TAQSIMOT siljishi    — test holati o'quvdan farq qiladi (drift)
6. MODEL cheklovi       — chegara shakli mos emas 14.7-bob

Har tur BOSHQA yechim talab qiladi:
  yorliq -> tozalash;  belgi -> yangi belgi;  kam namuna -> ma'lumot yig'ish
  ajralmas -> qabul qilish;  drift -> qayta o'qitish;  model -> boshqa algoritm

from sklearn.inspection import permutation_importance

nat = permutation_importance(model, X_te, y_te, n_repeats=10,
                             random_state=0, scoring="f1")
nat.importances_mean, nat.importances_std

Chiziqli modelda: hissa_j = koeffitsiyent_j * belgi_qiymati_j   (13.3, 14.11)
  -> to'g'ridan-to'g'ri o'qiladi

Daraxt/ansamblda: qaror yo'li yoki SHAP qiymatlari

Amaliy foyda:
  · xatoni tekshirish ("nega bu holatda yanglishdi?")
  · foydalanuvchiga sabab ko'rsatish (kredit rad etish sababi)
  · leakage topish (kutilmagan belgi hukmron - 12.9)

import numpy as np
import pandas as pd
from sklearn.inspection import permutation_importance
from sklearn.metrics import classification_report, confusion_matrix

df = pd.DataFrame({"y": y_val, "p": proba, "pred": pred, **segmentlar})
df["xato"] = df["y"] != df["pred"]
df["ishonch"] = np.maximum(df["p"], 1 - df["p"])

# segmentlar bo'yicha
df.groupby("hudud").agg(n=("y", "size"), xato_ulushi=("xato", "mean"),
                        xatolar=("xato", "sum"))

# ishonch bo'yicha
df["guruh"] = pd.qcut(df["ishonch"], 5)
df.groupby("guruh", observed=True)["xato"].mean()

# yuqori ishonchli xatolar — qo'lda ko'rib chiqish
df[df.xato].nlargest(20, "ishonch")

permutation_importance(model, X_val, y_val, n_repeats=10, random_state=0)
QOIDA: validatsiyada tahlil qil · segment + ishonch · 20-50 xatoni qo'lda ko'r

Segmentlar: qaysi yomon (sifat) va qaysi ko'p xato beradi (hajm)
Ishonch: yuqori ishonchli xatolar -> yorliq/leakage/vazifa muammosi
Turlari: yorliq, belgi, ajralmas, kam namuna, drift, model
Har turga boshqa yechim; natija — aniq reja

"""Model qayerda yomon va qayerda ko'p xato qiladi (real pandas/sklearn)."""

import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import f1_score
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler


def yarat(seed: int = 7, n: int = 9000) -> pd.DataFrame:
    """Kredit: 'yangi mijoz' segmentida belgilar kam ma'lumot beradi."""
    rng = np.random.default_rng(seed)
    yangi = rng.random(n) < 0.22
    hudud = rng.choice(["markaz", "shimol", "janub", "chekka"], n,
                       p=[0.4, 0.25, 0.2, 0.15])
    daromad = rng.lognormal(14.6, 0.5, n)
    kechikish = rng.poisson(np.where(yangi, 0.15, 0.6), n).astype(float)
    muddat = rng.integers(3, 48, n).astype(float)
    ball = (-1.0 - 0.55 * (np.log(daromad) - 14.6) + 0.85 * kechikish
            - 0.012 * muddat)
    # yangi mijozlarda tarix yo'q -> ko'proq tasodifiylik
    ball = ball + np.where(yangi, rng.normal(0, 1.6, n), rng.normal(0, 0.5, n))
    y = (rng.random(n) < 1 / (1 + np.exp(-ball))).astype(int)
    return pd.DataFrame({"yangi": yangi, "hudud": hudud, "daromad": daromad,
                         "kechikish": kechikish, "muddat": muddat,
                         "qaytarmadi": y})


def main() -> None:
    df = yarat()
    SON = ["daromad", "kechikish", "muddat"]
    KAT = ["hudud"]
    X = df[SON + KAT]
    y = df["qaytarmadi"]
    Xtr, Xval, ytr, yval, dtr, dval = train_test_split(
        X, y, df, test_size=0.35, random_state=0, stratify=y)

    model = Pipeline([
        ("t", ColumnTransformer([("son", StandardScaler(), SON),
                                 ("kat", OneHotEncoder(handle_unknown="ignore",
                                                       drop="first"), KAT)])),
        ("m", LogisticRegression(max_iter=3000)),
    ]).fit(Xtr, ytr)

    nat = dval.copy()
    nat["p"] = model.predict_proba(Xval)[:, 1]
    nat["pred"] = (nat["p"] >= 0.5).astype(int)
    nat["xato"] = nat["pred"] != nat["qaytarmadi"]

    print("=== 1. Umumiy natija ===")
    print(f"  validatsiya {len(nat)} qator, musbat sinf {yval.mean():.1%}")
    print(f"  aniqlik {1 - nat['xato'].mean():.4f}, "
          f"F1 {f1_score(nat['qaytarmadi'], nat['pred']):.4f}")

    print("\n=== 2. Segment: yangi va eski mijozlar ===")
    for qiymat, qism in nat.groupby("yangi"):
        nom = "yangi" if qiymat else "eski"
        print(f"  {nom:<6}: n={len(qism):>4}, xato ulushi "
              f"{qism['xato'].mean():.4f}, "
              f"F1 {f1_score(qism['qaytarmadi'], qism['pred']):.4f}, "
              f"barcha xatolarning {qism['xato'].sum() / nat['xato'].sum():.1%} i")

    print("\n=== 3. Segment: hudud ===")
    for qiymat, qism in nat.groupby("hudud"):
        print(f"  {qiymat:<8}: n={len(qism):>4}, xato ulushi "
              f"{qism['xato'].mean():.4f}, "
              f"barcha xatolarning {qism['xato'].sum() / nat['xato'].sum():.1%} i")

    print("\n=== 4. Segment: daromad diapazoni ===")
    nat["daromad_guruh"] = pd.qcut(nat["daromad"], 4,
                                   labels=["past", "o_rta", "yuqori", "eng yuqori"])
    for qiymat, qism in nat.groupby("daromad_guruh", observed=True):
        print(f"  {str(qiymat):<10}: n={len(qism):>4}, xato ulushi "
              f"{qism['xato'].mean():.4f}, "
              f"musbat sinf {qism['qaytarmadi'].mean():.3f}")
    print("  ⭐ 'Eng yomon segment' va 'eng ko'p xato beruvchi segment' har xil")


if __name__ == "__main__":
    main()

=== 1. Umumiy natija ===
  validatsiya 3150 qator, musbat sinf 32.8%
  aniqlik 0.6860, F1 0.2636

=== 2. Segment: yangi va eski mijozlar ===
  eski  : n=2477, xato ulushi 0.3121, F1 0.3141, barcha xatolarning 78.2% i
  yangi : n= 673, xato ulushi 0.3210, F1 0.0000, barcha xatolarning 21.8% i

=== 3. Segment: hudud ===
  chekka  : n= 465, xato ulushi 0.2796, barcha xatolarning 13.1% i
  janub   : n= 655, xato ulushi 0.3130, barcha xatolarning 20.7% i
  markaz  : n=1266, xato ulushi 0.3065, barcha xatolarning 39.2% i
  shimol  : n= 764, xato ulushi 0.3482, barcha xatolarning 26.9% i

=== 4. Segment: daromad diapazoni ===
  past      : n= 788, xato ulushi 0.3579, musbat sinf 0.385
  o_rta     : n= 787, xato ulushi 0.3380, musbat sinf 0.341
  yuqori    : n= 787, xato ulushi 0.2757, musbat sinf 0.307
  eng yuqori: n= 788, xato ulushi 0.2843, musbat sinf 0.280
  ⭐ 'Eng yomon segment' va 'eng ko'p xato beruvchi segment' har xil

"""Ishonch darajasi bo'yicha xatolar (real pandas/sklearn)."""

import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler


def yarat(seed: int = 11, n: int = 9000, yorliq_shovqini: float = 0.05):
    """Ba'zi yorliqlar ataylab NOTO'G'RI (yorliq shovqini)."""
    rng = np.random.default_rng(seed)
    X = rng.normal(0, 1, (n, 6))
    ball = 1.2 * X[:, 0] - 1.0 * X[:, 1] + 0.7 * X[:, 2]
    y = (rng.random(n) < 1 / (1 + np.exp(-ball))).astype(int)
    buzuq = rng.random(n) < yorliq_shovqini
    y[buzuq] = 1 - y[buzuq]
    return X, y, buzuq


def main() -> None:
    X, y, buzuq = yarat()
    idx = np.arange(len(X))
    Xtr, Xval, ytr, yval, itr, ival = train_test_split(
        X, y, idx, test_size=0.35, random_state=0, stratify=y)
    model = Pipeline([("sc", StandardScaler()),
                      ("m", LogisticRegression(max_iter=3000))]).fit(Xtr, ytr)
    p = model.predict_proba(Xval)[:, 1]

    nat = pd.DataFrame({"y": yval, "p": p, "buzuq": buzuq[ival]})
    nat["pred"] = (nat["p"] >= 0.5).astype(int)
    nat["xato"] = nat["pred"] != nat["y"]
    nat["ishonch"] = np.maximum(nat["p"], 1 - nat["p"])

    print("=== 1. Umumiy ===")
    print(f"  {len(nat)} qator, xato ulushi {nat['xato'].mean():.4f}")
    print(f"  ataylab buzilgan yorliqlar: {nat['buzuq'].mean():.1%}")

    print("\n=== 2. Ishonch guruhlari bo'yicha xato ===")
    nat["guruh"] = pd.qcut(nat["ishonch"], 5,
                           labels=["eng past", "past", "o_rta", "yuqori",
                                   "eng yuqori"])
    for qiymat, qism in nat.groupby("guruh", observed=True):
        print(f"  {str(qiymat):<11}: ishonch {qism['ishonch'].mean():.3f}, "
              f"xato ulushi {qism['xato'].mean():.4f}, "
              f"n={len(qism):>4}")

    print("\n=== 3. Yuqori ishonchli xatolar ===")
    yuqori_xato = nat[nat["xato"] & (nat["ishonch"] > 0.9)]
    print(f"  ishonch > 0.9 bo'lgan xatolar: {len(yuqori_xato)} ta")
    print(f"  ulardan buzilgan yorliqli: {yuqori_xato['buzuq'].mean():.1%}")
    oddiy_xato = nat[nat["xato"] & (nat["ishonch"] <= 0.9)]
    print(f"  qolgan xatolarda buzilgan yorliq: {oddiy_xato['buzuq'].mean():.1%}")
    print("  (yuqori ishonchli xatolar — yorliq muammosining indikatori)")

    print("\n=== 4. Past ishonchda insonga yo'naltirish ===")
    for ulush in [0.05, 0.10, 0.20]:
        chegara = nat["ishonch"].quantile(ulush)
        qolgan = nat[nat["ishonch"] > chegara]
        print(f"  eng past {ulush:.0%} ni insonga bersak: "
              f"qolganda aniqlik {1 - qolgan['xato'].mean():.4f} "
              f"(umumiy {1 - nat['xato'].mean():.4f})")
    print("  ⭐ Selektiv bashorat: ishonchsiz holatlarni insonga bering")


if __name__ == "__main__":
    main()

=== 1. Umumiy ===
  3150 qator, xato ulushi 0.2822
  ataylab buzilgan yorliqlar: 4.4%

=== 2. Ishonch guruhlari bo'yicha xato ===
  eng past   : ishonch 0.542, xato ulushi 0.4444, n= 630
  past       : ishonch 0.623, xato ulushi 0.4111, n= 630
  o_rta      : ishonch 0.709, xato ulushi 0.2508, n= 630
  yuqori     : ishonch 0.794, xato ulushi 0.2016, n= 630
  eng yuqori : ishonch 0.901, xato ulushi 0.1032, n= 630

=== 3. Yuqori ishonchli xatolar ===
  ishonch > 0.9 bo'lgan xatolar: 19 ta
  ulardan buzilgan yorliqli: 47.4%
  qolgan xatolarda buzilgan yorliq: 11.3%
  (yuqori ishonchli xatolar — yorliq muammosining indikatori)

=== 4. Past ishonchda insonga yo'naltirish ===
  eng past 5% ni insonga bersak: qolganda aniqlik 0.7269 (umumiy 0.7178)
  eng past 10% ni insonga bersak: qolganda aniqlik 0.7383 (umumiy 0.7178)
  eng past 20% ni insonga bersak: qolganda aniqlik 0.7583 (umumiy 0.7178)
  ⭐ Selektiv bashorat: ishonchsiz holatlarni insonga bering

"""Xatolarni sababi bo'yicha guruhlash (real numpy/sklearn)."""

import numpy as np
import pandas as pd
from sklearn.inspection import permutation_importance
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler


def yarat(seed: int = 15, n: int = 8000):
    """Uch xil xato manbai: yorliq shovqini, yashirin belgi, ajralmas zona."""
    rng = np.random.default_rng(seed)
    X = rng.normal(0, 1, (n, 5))
    yashirin = rng.normal(0, 1, n)                 # modelga BERILMAYDI
    ball = 1.1 * X[:, 0] - 0.9 * X[:, 1] + 1.3 * yashirin
    y = (rng.random(n) < 1 / (1 + np.exp(-ball))).astype(int)
    buzuq = rng.random(n) < 0.04
    y[buzuq] = 1 - y[buzuq]
    return X, y, yashirin, buzuq


def main() -> None:
    X, y, yashirin, buzuq = yarat()
    idx = np.arange(len(X))
    Xtr, Xval, ytr, yval, itr, ival = train_test_split(
        X, y, idx, test_size=0.35, random_state=0, stratify=y)
    model = Pipeline([("sc", StandardScaler()),
                      ("m", LogisticRegression(max_iter=3000))]).fit(Xtr, ytr)
    p = model.predict_proba(Xval)[:, 1]
    pred = (p >= 0.5).astype(int)
    xato = pred != yval

    print("=== 1. Umumiy xato ===")
    print(f"  {len(yval)} qator, xato ulushi {xato.mean():.4f}")

    print("\n=== 2. Xato turlari ===")
    buzuq_val = buzuq[ival]
    yashirin_val = np.abs(yashirin[ival])
    ishonch = np.maximum(p, 1 - p)
    turlar = {
        "yorliq xatosi": xato & buzuq_val,
        "yashirin belgi kuchli": xato & ~buzuq_val & (yashirin_val > 1.2),
        "ajralmas zona (past ishonch)": xato & ~buzuq_val & (yashirin_val <= 1.2)
                                        & (ishonch < 0.65),
        "boshqa": xato & ~buzuq_val & (yashirin_val <= 1.2) & (ishonch >= 0.65),
    }
    for nom, mos in turlar.items():
        print(f"  {nom:<30}: {mos.sum():>4} ta "
              f"({mos.sum() / xato.sum():.1%} barcha xatolardan)")

    print("\n=== 3. Yashirin belgi qo'shilsa ===")
    X2tr = np.column_stack([Xtr, yashirin[itr]])
    X2val = np.column_stack([Xval, yashirin[ival]])
    model2 = Pipeline([("sc", StandardScaler()),
                       ("m", LogisticRegression(max_iter=3000))]).fit(X2tr, ytr)
    xato2 = (model2.predict(X2val) != yval)
    print(f"  belgi qo'shilmagan: xato {xato.mean():.4f}")
    print(f"  belgi qo'shilgan:   xato {xato2.mean():.4f}")
    print(f"  yaxshilanish: {(xato.mean() - xato2.mean()) / xato.mean():.1%}")

    print("\n=== 4. Permutation importance ===")
    nat = permutation_importance(model, Xval, yval, n_repeats=10,
                                 random_state=0, scoring="accuracy")
    tartib = np.argsort(nat.importances_mean)[::-1]
    for i in tartib:
        print(f"  belgi {i}: {nat.importances_mean[i]:+.4f} "
              f"+- {nat.importances_std[i]:.4f}")
    print("  ⭐ Har xato turiga boshqa yechim mos keladi")


if __name__ == "__main__":
    main()

=== 1. Umumiy xato ===
  2800 qator, xato ulushi 0.3193

=== 2. Xato turlari ===
  yorliq xatosi                 :   78 ta (8.7% barcha xatolardan)
  yashirin belgi kuchli         :  234 ta (26.2% barcha xatolardan)
  ajralmas zona (past ishonch)  :  384 ta (43.0% barcha xatolardan)
  boshqa                        :  198 ta (22.1% barcha xatolardan)

=== 3. Yashirin belgi qo'shilsa ===
  belgi qo'shilmagan: xato 0.3193
  belgi qo'shilgan:   xato 0.2482
  yaxshilanish: 22.3%

=== 4. Permutation importance ===
  belgi 0: +0.1248 +- 0.0061
  belgi 1: +0.0676 +- 0.0060
  belgi 3: +0.0005 +- 0.0004
  belgi 2: +0.0002 +- 0.0015
  belgi 4: -0.0006 +- 0.0003
  ⭐ Har xato turiga boshqa yechim mos keladi

"""Xatolar tahlilidan aniq rejaga (real pandas/sklearn)."""

import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import f1_score
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler


def yarat(seed: int = 7, n: int = 9000) -> pd.DataFrame:
    rng = np.random.default_rng(seed)
    yangi = rng.random(n) < 0.22
    hudud = rng.choice(["markaz", "shimol", "janub", "chekka"], n,
                       p=[0.4, 0.25, 0.2, 0.15])
    daromad = rng.lognormal(14.6, 0.5, n)
    kechikish = rng.poisson(np.where(yangi, 0.15, 0.6), n).astype(float)
    muddat = rng.integers(3, 48, n).astype(float)
    ball = (-1.0 - 0.55 * (np.log(daromad) - 14.6) + 0.85 * kechikish
            - 0.012 * muddat)
    ball = ball + np.where(yangi, rng.normal(0, 1.6, n), rng.normal(0, 0.5, n))
    y = (rng.random(n) < 1 / (1 + np.exp(-ball))).astype(int)
    return pd.DataFrame({"yangi": yangi, "hudud": hudud, "daromad": daromad,
                         "kechikish": kechikish, "muddat": muddat,
                         "qaytarmadi": y})


def main() -> None:
    df = yarat()
    SON = ["daromad", "kechikish", "muddat"]
    KAT = ["hudud"]
    X, y = df[SON + KAT], df["qaytarmadi"]
    Xtr, Xval, ytr, yval, dtr, dval = train_test_split(
        X, y, df, test_size=0.35, random_state=0, stratify=y)
    model = Pipeline([
        ("t", ColumnTransformer([("son", StandardScaler(), SON),
                                 ("kat", OneHotEncoder(handle_unknown="ignore",
                                                       drop="first"), KAT)])),
        ("m", LogisticRegression(max_iter=3000)),
    ]).fit(Xtr, ytr)

    nomlar = model.named_steps["t"].get_feature_names_out()
    w = model.named_steps["m"].coef_[0]
    kesma = model.named_steps["m"].intercept_[0]
    Z = model.named_steps["t"].transform(Xval)
    p = model.predict_proba(Xval)[:, 1]
    pred = (p >= 0.5).astype(int)
    xato = pred != yval.to_numpy()
    ishonch = np.maximum(p, 1 - p)

    print("=== 1. Eng ishonchli xatolar ===")
    tartib = np.argsort(np.where(xato, ishonch, -1))[::-1][:3]
    for i in tartib:
        qator = dval.iloc[i]
        print(f"  #{i}: haqiqiy {int(yval.iloc[i])}, bashorat {pred[i]} "
              f"(p={p[i]:.3f}), yangi={bool(qator['yangi'])}, "
              f"kechikish={qator['kechikish']:.0f}")

    print("\n=== 2. Bitta bashoratni tushuntirish ===")
    i = int(tartib[0])
    hissa = Z[i] * w
    tartib_h = np.argsort(np.abs(hissa))[::-1][:4]
    print(f"  kesma: {kesma:+.3f}")
    for j in tartib_h:
        print(f"  {nomlar[j]:<16}: {hissa[j]:+.3f}")
    z = float(hissa.sum() + kesma)
    print(f"  jami log-odds {z:+.3f} -> ehtimol {1 / (1 + np.exp(-z)):.3f}")

    print("\n=== 3. Topilmalar jadvali ===")
    yangi_val = dval["yangi"].to_numpy()
    topilmalar = [
        ("yangi mijozlar segmenti",
         f"xato ulushi {xato[yangi_val].mean():.3f} vs "
         f"{xato[~yangi_val].mean():.3f}",
         "tarix belgilari yo'q",
         "alohida model yoki qo'shimcha belgilar"),
        ("barcha xatolarning ulushi",
         f"{xato[yangi_val].sum() / xato.sum():.1%} yangi mijozlardan",
         f"segment hajmi {yangi_val.mean():.0%}",
         "shu segmentga ustuvorlik berish"),
        ("yuqori ishonchli xatolar",
         f"{(xato & (ishonch > 0.85)).sum()} ta",
         "yorliq yoki ziddiyatli yozuv",
         "20-50 tasini qo'lda ko'rib chiqish"),
    ]
    for nom, olchov, sabab, harakat in topilmalar:
        print(f"  [{nom}]")
        print(f"    o'lchov: {olchov}")
        print(f"    taxminiy sabab: {sabab}")
        print(f"    harakat: {harakat}")

    print("\n=== 4. Rejani sinash: segment belgisi qo'shish ===")
    X2tr = Xtr.assign(yangi=dtr["yangi"].astype(float))
    X2val = Xval.assign(yangi=dval["yangi"].astype(float))
    model2 = Pipeline([
        ("t", ColumnTransformer([("son", StandardScaler(), SON + ["yangi"]),
                                 ("kat", OneHotEncoder(handle_unknown="ignore",
                                                       drop="first"), KAT)])),
        ("m", LogisticRegression(max_iter=3000)),
    ]).fit(X2tr, ytr)
    pred2 = model2.predict(X2val)
    print(f"  F1: {f1_score(yval, pred):.4f} -> {f1_score(yval, pred2):.4f}")
    print(f"  yangi mijozlarda xato: {xato[yangi_val].mean():.4f} -> "
          f"{(pred2 != yval.to_numpy())[yangi_val].mean():.4f}")
    print("  ⭐ Tahlil natijasi — aniq reja, keyin uni sinash")


if __name__ == "__main__":
    main()

=== 1. Eng ishonchli xatolar ===
  #788: haqiqiy 0, bashorat 1 (p=0.891), yangi=False, kechikish=4
  #1795: haqiqiy 0, bashorat 1 (p=0.878), yangi=False, kechikish=4
  #2623: haqiqiy 1, bashorat 0 (p=0.131), yangi=False, kechikish=0

=== 2. Bitta bashoratni tushuntirish ===
  kesma: -0.819
  son__kechikish  : +2.671
  son__daromad    : +0.190
  son__muddat     : +0.058
  kat__hudud_markaz: +0.007
  jami log-odds +2.106 -> ehtimol 0.891

=== 3. Topilmalar jadvali ===
  [yangi mijozlar segmenti]
    o'lchov: xato ulushi 0.321 vs 0.312
    taxminiy sabab: tarix belgilari yo'q
    harakat: alohida model yoki qo'shimcha belgilar
  [barcha xatolarning ulushi]
    o'lchov: 21.8% yangi mijozlardan
    taxminiy sabab: segment hajmi 21%
    harakat: shu segmentga ustuvorlik berish
  [yuqori ishonchli xatolar]
    o'lchov: 7 ta
    taxminiy sabab: yorliq yoki ziddiyatli yozuv
    harakat: 20-50 tasini qo'lda ko'rib chiqish

=== 4. Rejani sinash: segment belgisi qo'shish ===
  F1: 0.2636 -> 0.2674
  yangi mijozlarda xato: 0.3210 -> 0.3254
  ⭐ Tahlil natijasi — aniq reja, keyin uni sinash

print(f"F1 = {f1:.3f}")                                           # ⚠️
df.groupby("segment")["xato"].agg(["mean", "sum", "size"])        # ✅

# test xatolarini tahlil qilib, shu testda qayta baholash          # ⚠️
# tahlil validatsiyada, test faqat yakuniy baho uchun 12.3-bob      # ✅

# n = 12 bo'lgan segmentda F1 = 0.42 -> "muammo"                  # ⚠️
# n >= 50-100 bo'lgan segmentlarni ko'ring                        # ✅

# faqat sonlarga qarash                                           # ⚠️
df[df.xato].nlargest(30, "ishonch")   # va o'qib chiqing          # ✅

# "model 8% xato qiladi" (yorliqlarning 5% i noto'g'ri)           # ⚠️
# yorliq sifatini alohida o'lchang                                # ✅

# "ikkala belgi ham muhim emas" (ular bir-birini almashtiradi)    # ⚠️
# guruh bo'lib aralashtiring yoki korrelyatsiyani tekshiring      # ✅

# "yangi mijozlarda yomon" — va shu yerda to'xtash                # ⚠️
# har topilma uchun: sabab + harakat + kutilgan ta'sir            # ✅

1.  # segment tahlilining ikki savoli?
2.  # kichik segment chegarasi?
3.  # ishonch qanday hisoblanadi?
4.  # yuqori ishonchli xato nimani anglatadi?
5.  # selektiv bashorat nima?
6.  # yorliq shovqinini qanday topish?
7.  # xato turlari?
8.  # yorliq xatosiga yechim?
9.  # belgi yetishmasligiga?
10. # permutation importance nima o'lchaydi?
11. # korrelyatsiyada u qanday buziladi?
12. # tahlil qaysi to'plamda?

1.  print(f"F1 = {f1:.3f}")   # boshqa tahlil yo'q

2.  # test xatolarini tahlil qilib, o'sha testda baholash

3.  # n = 15 bo'lgan segmentda xulosa chiqarish

4.  # "model 8% xato" (yorliqlarning 5% i noto'g'ri)

5.  # "yangi mijozlarda yomon" — xulosa shu yerda tugadi