17.8-dars: Leakage

1. MAQSAD leakage (target leakage)
   Belgi maqsaddan hosil bo'lgan yoki u bilan sababiy bog'langan
   "qaytarish summasi", "yopilish sanasi", "tashxis kodi"

2. TAYYORLASH leakage (preprocessing leakage)
   Transformatsiya butun ma'lumotda fit qilingan
   scaler, imputer, encoder, PCA, belgi tanlash

3. VAQT leakage (temporal leakage)
   Belgi bashorat vaqtida hali ma'lum emas
   rolling da shift yo'q, tasodifiy CV, kelajak agregatsiyasi

4. GURUH leakage (group leakage)
   Bir obyektning qatorlari o'quv va testga bo'linib ketgan
   bir bemor, bir mijoz, bir qurilma

5. DUPLIKATLAR
   Bir xil (yoki deyarli bir xil) qatorlar o'quv va testda

6. TASHQI leakage
   Test to'plami ommaviy ma'lumotdan olingan va model uni ko'rgan

TA'RIFI: belgi maqsad HAQIDAGI ma'lumotni o'z ichiga oladi

KLASSIK MISOLLAR:
  defolt bashorati    -> "qaytarilgan summa" (defoltda 0)
  churn bashorati     -> "yopilish sababi" (faqat ketganlarda bor)
  kasallik tashxisi   -> "buyurilgan dori"
  buyurtma bekor      -> "qaytarish sanasi"
  firibgarlik         -> "tekshiruvchi izohi"

ASOSIY SAVOL: "bu qiymat qachon yoziladi?"
  maqsad ma'lum bo'lgandan KEYIN -> leakage

NOZIK HOLAT: sababiy zanjir
  "tekshiruv turi" tashxisga sabab emas, lekin shifokor
  gumoniga qarab tanlanadi -> bilvosita leakage

NOTO'G'RI                                 TO'G'RI
scaler.fit_transform(X); CV               Pipeline([("sc", ...), ("m", ...)])
imputer.fit(X)                            Pipeline ichida
SelectKBest.fit(X, y); CV                 Pipeline ichida
PCA.fit(X); CV                            Pipeline ichida
TargetEncoder qo'lda                      Pipeline ichida (cv bilan)
SMOTE butun ma'lumotda                    imblearn.Pipeline ichida

TA'SIR DARAJASI:
  scaler          kichik (0.001-0.01)
  imputer         kichik
  PCA             o'rtacha
  belgi tanlash   KATTA (p >> n da halokatli)
  target encoding KATTA
  oversampling    JUDA KATTA

from sklearn.model_selection import (GroupKFold, StratifiedGroupKFold,
                                     TimeSeriesSplit)

# vaqt
cv = TimeSeriesSplit(n_splits=5, gap=7)

# guruh (bir mijozning barcha qatorlari bir foldda)
cv = GroupKFold(n_splits=5)
cv.split(X, y, groups=df["mijoz_id"])

# guruh + sinf muvozanati
cv = StratifiedGroupKFold(n_splits=5, shuffle=True, random_state=0)

GURUH LEAKAGE I BELGISI:
  guruhsiz CV >> guruhli CV
  (masalan 0.94 va 0.71)

QACHON GURUH KERAK:
  bir mijozning ko'p tranzaksiyasi
  bir bemorning ko'p tashrifi
  bir qurilmaning ko'p o'lchovi
  bir maqolaning ko'p jumlasi
  bir rasmning ko'p kesimi (crop)

df.duplicated().sum()                      # aniq duplikatlar
df.duplicated(subset=belgilar).sum()       # belgilar bo'yicha
df.drop_duplicates(subset=belgilar)

# deyarli duplikatlar (matnda)
from sklearn.metrics.pairwise import cosine_similarity

NEGA XAVFLI:
  bir xil qator o'quvda va testda -> model uni YODLAB qo'ygan
  -> test bahosi optimistik

QAYERDA UCHRAYDI:
  bir necha marta eksport qilingan ma'lumot
  qayta yuborilgan forma
  matn korpusida takroriy hujjatlar
  tasvir to'plamida bir xil rasmlar

TARTIB: duplikatlarni BO'LISHDAN OLDIN olib tashlang

TOPISH USULLARI:
  1. "JUDA YAXSHI natija" -> shubha (AUC > 0.95 kamdan-kam real)
  2. Har belgining YAKKA CV natijasi (0.9+ -> shubhali)
  3. Permutation importance: bitta belgi hukmron bo'lsa
  4. Guruhsiz va guruhli CV ni solishtirish
  5. Vaqt bo'yicha validatsiya bilan solishtirish
  6. CV va ISHLAB CHIQARISH natijasini solishtirish
  7. Domen mutaxassisidan so'rash: "bu qachon yoziladi?"

OLDINI OLISH TUZILMASI:
  [ ] Barcha tayyorlash Pipeline ichida
  [ ] Validatsiya strategiyasi vazifaga mos (vaqt/guruh)
  [ ] Duplikatlar bo'lishdan oldin olib tashlangan
  [ ] Har belgi uchun "qachon ma'lum?" hujjatlashtirilgan
  [ ] Test to'plami BIR MARTA ishlatilgan
  [ ] Natija domen mutaxassisi bilan muhokama qilingan

import pandas as pd
from sklearn.model_selection import (GroupKFold, StratifiedGroupKFold,
                                     TimeSeriesSplit, cross_val_score)
from sklearn.pipeline import Pipeline

df.duplicated().sum()                       # duplikatlar (bo'lishdan OLDIN)

# har belgining yakka kuchi
for ustun in belgilar:
    print(ustun, cross_val_score(model, X[[ustun]], y, cv=5,
                                 scoring="roc_auc").mean())

cross_val_score(quvur, X, y, cv=GroupKFold(5), groups=df["mijoz_id"])
cross_val_score(quvur, X, y, cv=TimeSeriesSplit(5, gap=7))
Pipeline([("sc", ...), ("s", ...), ("m", model)])   # hamma tayyorlash ichida
QOIDA: "juda yaxshi" - shubhali · Pipeline ichida · guruh/vaqtni hisobga ol ·
       duplikatlarni avval tozala

Turlari: maqsad, tayyorlash, vaqt, guruh, duplikat, tashqi
Asosiy savol: "bu qiymat qachon yoziladi?"
Topish: yuqori natijaga shubha, yakka CV, guruhsiz/guruhli farq
Oldini olish: Pipeline + to'g'ri validatsiya + duplikat tozalash

"""Har belgining yakka kuchi bilan audit (real pandas/sklearn)."""

import numpy as np
import pandas as pd
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.inspection import permutation_importance
from sklearn.model_selection import StratifiedKFold, cross_val_score, train_test_split


def yarat(seed: int = 5, n: int = 6000) -> pd.DataFrame:
    """Kredit defolti; uchta leakage belgisi bor."""
    rng = np.random.default_rng(seed)
    daromad = rng.lognormal(14.2, 0.5, n)
    qarz = daromad * rng.lognormal(-0.5, 0.6, n)
    yosh = rng.integers(21, 65, n)
    muddat = rng.integers(6, 60, n)
    kuch = -1.3 + 2.4 * (qarz / daromad) - 0.015 * (yosh - 40) \
        + 0.012 * muddat
    defolt = (rng.random(n) < 1 / (1 + np.exp(-kuch))).astype(int)

    # LEAKAGE 1: qaytarilgan summa - defoltda deyarli 0
    qaytarilgan = np.where(defolt == 1, qarz * rng.uniform(0, 0.25, n),
                           qarz * rng.uniform(0.85, 1.0, n))
    # LEAKAGE 2: yopilish sababi (kod) - faqat defoltda 3
    yopilish_kodi = np.where(defolt == 1, 3,
                             rng.choice([1, 2], n))
    # LEAKAGE 3: bilvosita - undiruv bo'limiga o'tkazilganmi
    undiruv = np.where(defolt == 1, rng.random(n) < 0.85,
                       rng.random(n) < 0.05).astype(int)
    return pd.DataFrame({"daromad": daromad, "qarz": qarz, "yosh": yosh,
                         "muddat": muddat, "qaytarilgan": qaytarilgan,
                         "yopilish_kodi": yopilish_kodi, "undiruv": undiruv,
                         "defolt": defolt})


def main() -> None:
    df = yarat()
    y = df["defolt"].to_numpy()
    belgilar = [c for c in df.columns if c != "defolt"]
    cv = StratifiedKFold(5, shuffle=True, random_state=0)

    def model():
        return HistGradientBoostingClassifier(learning_rate=0.1, max_iter=200,
                                              random_state=0)

    print("=== 1. Barcha belgilar bilan ===")
    b = cross_val_score(model(), df[belgilar], y, cv=cv, scoring="roc_auc")
    print(f"  CV ROC AUC: {b.mean():.4f} (+-{b.std():.4f})")
    print(f"  SHUBHA: kredit skoringida 0.95+ deyarli bo'lmaydi")

    print("\n=== 2. Har belgining yakka kuchi ===")
    print(f"  {'belgi':<16} {'yakka CV AUC':>14} {'baho':<14}")
    shubhalilar = []
    for ustun in belgilar:
        bb = cross_val_score(model(), df[[ustun]], y, cv=cv,
                             scoring="roc_auc").mean()
        baho = "SHUBHALI" if bb > 0.85 else ("kuchli" if bb > 0.65 else "oddiy")
        if bb > 0.85:
            shubhalilar.append(ustun)
        print(f"  {ustun:<16} {bb:>14.4f} {baho:<14}")

    print("\n=== 3. 'Bu qiymat qachon yoziladi?' ===")
    savollar = {
        "daromad": "ariza berishda - OK",
        "qarz": "ariza berishda - OK",
        "yosh": "ariza berishda - OK",
        "muddat": "shartnomada - OK",
        "qaytarilgan": "kredit YOPILGANDAN keyin - LEAKAGE",
        "yopilish_kodi": "kredit YOPILGANDAN keyin - LEAKAGE",
        "undiruv": "defolt aniqlangandan keyin - LEAKAGE",
    }
    for ustun, javob in savollar.items():
        belgi = "  <-" if "LEAKAGE" in javob else ""
        print(f"  {ustun:<16}: {javob}{belgi}")

    print("\n=== 4. Tozalashdan keyin ===")
    toza = ["daromad", "qarz", "yosh", "muddat"]
    bt = cross_val_score(model(), df[toza], y, cv=cv, scoring="roc_auc")
    print(f"  leakage bilan: {b.mean():.4f}")
    print(f"  leakage siz:   {bt.mean():.4f} (+-{bt.std():.4f})")
    print(f"  farq: {b.mean() - bt.mean():+.4f}")
    Xtr, Xte, ytr, yte = train_test_split(df[belgilar], y, test_size=0.3,
                                          random_state=0, stratify=y)
    m = model().fit(Xtr, ytr)
    r = permutation_importance(m, Xte, yte, n_repeats=10, scoring="roc_auc",
                               random_state=0, n_jobs=1)
    print(f"  {'belgi':<16} {'permutation muhimligi':>23}")
    for i in np.argsort(-r.importances_mean)[:4]:
        print(f"  {belgilar[i]:<16} {r.importances_mean[i]:>+23.4f}")
    print("  ⭐ 'Bu qiymat qachon yoziladi?' - eng samarali savol")


if __name__ == "__main__":
    main()

=== 1. Barcha belgilar bilan ===
  CV ROC AUC: 1.0000 (+-0.0000)
  SHUBHA: kredit skoringida 0.95+ deyarli bo'lmaydi

=== 2. Har belgining yakka kuchi ===
  belgi              yakka CV AUC baho
  daromad                  0.5099 oddiy
  qarz                     0.6213 oddiy
  yosh                     0.5542 oddiy
  muddat                   0.5228 oddiy
  qaytarilgan              0.9109 SHUBHALI
  yopilish_kodi            1.0000 SHUBHALI
  undiruv                  0.9015 SHUBHALI

=== 3. 'Bu qiymat qachon yoziladi?' ===
  daromad         : ariza berishda - OK
  qarz            : ariza berishda - OK
  yosh            : ariza berishda - OK
  muddat          : shartnomada - OK
  qaytarilgan     : kredit YOPILGANDAN keyin - LEAKAGE  <-
  yopilish_kodi   : kredit YOPILGANDAN keyin - LEAKAGE  <-
  undiruv         : defolt aniqlangandan keyin - LEAKAGE  <-

=== 4. Tozalashdan keyin ===
  leakage bilan: 1.0000
  leakage siz:   0.6720 (+-0.0190)
  farq: +0.3280
  belgi              permutation muhimligi
  yopilish_kodi                    +0.5043
  daromad                          +0.0000
  yosh                             +0.0000
  qarz                             +0.0000
  ⭐ 'Bu qiymat qachon yoziladi?' - eng samarali savol

"""Model obyektni eslab qoladi (real pandas/sklearn)."""

import numpy as np
import pandas as pd
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.model_selection import (GroupKFold, StratifiedGroupKFold,
                                     StratifiedKFold, cross_val_score)


def yarat(seed: int = 9, bemorlar: int = 400) -> pd.DataFrame:
    """Har bemorning bir necha tashrifi; kasallik BEMOR darajasida."""
    rng = np.random.default_rng(seed)
    qatorlar = []
    for b in range(bemorlar):
        # bemorning o'ziga xos "imzosi" - modelga uni tanib olish imkonini beradi
        imzo = rng.normal(0, 1, 4)
        kasal = int(rng.random() < 0.35)
        tashriflar = int(rng.integers(3, 15))
        for _ in range(tashriflar):
            olchovlar = imzo + rng.normal(0, 0.25, 4)
            # kasallik belgisi ZAIF
            olchovlar[0] += 0.55 * kasal
            qatorlar.append([b, *olchovlar, kasal])
    return pd.DataFrame(qatorlar, columns=["bemor", "x0", "x1", "x2", "x3",
                                           "kasal"])


def main() -> None:
    df = yarat()
    X = df[["x0", "x1", "x2", "x3"]]
    y = df["kasal"].to_numpy()
    guruh = df["bemor"].to_numpy()

    print("=== 1. Ma'lumot ===")
    print(f"  {len(df)} tashrif, {df['bemor'].nunique()} bemor")
    print(f"  har bemorda: mediana "
          f"{int(df.groupby('bemor').size().median())} tashrif")
    print(f"  kasal ulushi (tashriflar bo'yicha): {y.mean():.2%}")
    bemor_kasal = df.groupby("bemor")["kasal"].first()
    print(f"  kasal ulushi (bemorlar bo'yicha): {bemor_kasal.mean():.2%}")

    def model():
        return HistGradientBoostingClassifier(learning_rate=0.1, max_iter=200,
                                              random_state=0)

    print("\n=== 2. Guruhsiz va guruhli CV ===")
    cvlar = {
        "StratifiedKFold (NOTO'G'RI)": (StratifiedKFold(5, shuffle=True,
                                                        random_state=0), None),
        "GroupKFold (to'g'ri)": (GroupKFold(5), guruh),
        "StratifiedGroupKFold": (StratifiedGroupKFold(5, shuffle=True,
                                                      random_state=0), guruh),
    }
    print(f"  {'validatsiya':<30} {'CV ROC AUC':>12} {'std':>8}")
    for nom, (cv, g) in cvlar.items():
        b = cross_val_score(model(), X, y, cv=cv, groups=g, scoring="roc_auc")
        print(f"  {nom:<30} {b.mean():>12.4f} {b.std():>8.4f}")
    print("  (farq katta - model bemorni eslab qolgan)")

    print("\n=== 3. Qanday 'eslab qoladi' ===")
    # bemorning imzosi x1..x3 da - kasallik bilan bog'liq EMAS
    faqat_imzo = df[["x1", "x2", "x3"]]
    b1 = cross_val_score(model(), faqat_imzo, y,
                         cv=StratifiedKFold(5, shuffle=True, random_state=0),
                         scoring="roc_auc").mean()
    b2 = cross_val_score(model(), faqat_imzo, y, cv=GroupKFold(5),
                         groups=guruh, scoring="roc_auc").mean()
    print(f"  faqat 'imzo' belgilari (x1, x2, x3):")
    print(f"    guruhsiz CV: {b1:.4f}")
    print(f"    guruhli CV:  {b2:.4f}")
    print("  (bu belgilar kasallik haqida hech narsa bilmaydi)")

    print("\n=== 4. To'g'ri yondashuv: bemor darajasida agregatsiya ===")
    agregat = df.groupby("bemor").agg(
        x0_ort=("x0", "mean"), x0_std=("x0", "std"),
        x1_ort=("x1", "mean"), x2_ort=("x2", "mean"),
        x3_ort=("x3", "mean"), tashriflar=("x0", "count"),
        kasal=("kasal", "first")).fillna(0).reset_index()
    ya = agregat["kasal"].to_numpy()
    Xa = agregat[["x0_ort", "x0_std", "x1_ort", "x2_ort", "x3_ort",
                  "tashriflar"]]
    ba = cross_val_score(model(), Xa, ya,
                         cv=StratifiedKFold(5, shuffle=True, random_state=0),
                         scoring="roc_auc")
    print(f"  bemor darajasida: {len(agregat)} qator")
    print(f"  CV ROC AUC: {ba.mean():.4f} (+-{ba.std():.4f})")
    print(f"  tashrif darajasida (GroupKFold): "
          f"{cross_val_score(model(), X, y, cv=GroupKFold(5), groups=guruh, scoring='roc_auc').mean():.4f}")
    print("  ⭐ Guruh tuzilmasi bo'lsa GroupKFold yoki agregatsiya")


if __name__ == "__main__":
    main()

=== 1. Ma'lumot ===
  3532 tashrif, 400 bemor
  har bemorda: mediana 9 tashrif
  kasal ulushi (tashriflar bo'yicha): 36.18%
  kasal ulushi (bemorlar bo'yicha): 36.75%

=== 2. Guruhsiz va guruhli CV ===
  validatsiya                      CV ROC AUC      std
  StratifiedKFold (NOTO'G'RI)          0.8164   0.0157
  GroupKFold (to'g'ri)                 0.6047   0.0194
  StratifiedGroupKFold                 0.5575   0.0602
  (farq katta - model bemorni eslab qolgan)

=== 3. Qanday 'eslab qoladi' ===
  faqat 'imzo' belgilari (x1, x2, x3):
    guruhsiz CV: 0.6740
    guruhli CV:  0.5125
  (bu belgilar kasallik haqida hech narsa bilmaydi)

=== 4. To'g'ri yondashuv: bemor darajasida agregatsiya ===
  bemor darajasida: 400 qator
  CV ROC AUC: 0.6164 (+-0.0685)
  tashrif darajasida (GroupKFold): 0.6047
  ⭐ Guruh tuzilmasi bo'lsa GroupKFold yoki agregatsiya

"""Har transformatsiyaning ta'sir darajasi (real numpy/sklearn)."""

import numpy as np
from sklearn.decomposition import PCA
from sklearn.feature_selection import SelectKBest, f_classif
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold, cross_val_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import QuantileTransformer, StandardScaler


def yarat(seed: int = 3, n: int = 400, p: int = 200, signal: int = 5):
    rng = np.random.default_rng(seed)
    X = rng.normal(0, 1, (n, p))
    if signal:
        kuch = X[:, :signal] @ rng.normal(0, 1.2, signal)
        y = (rng.random(n) < 1 / (1 + np.exp(-kuch))).astype(int)
    else:
        y = rng.integers(0, 2, n)
    # ba'zi yo'qolgan qiymatlar
    yoq = rng.random((n, p)) < 0.05
    X[yoq] = np.nan
    return X, y


def main() -> None:
    X, y = yarat()
    cv = StratifiedKFold(5, shuffle=True, random_state=0)
    model = LogisticRegression(max_iter=3000)

    print("=== 1. Ma'lumot ===")
    print(f"  {X.shape[0]} qator, {X.shape[1]} belgi (5 tasi foydali)")
    print(f"  yo'qolgan qiymatlar: {np.isnan(X).mean():.1%}")

    print("\n=== 2. Har transformatsiyaning leakage ta'siri ===")
    bosqichlar = {
        "imputer": SimpleImputer(strategy="median"),
        "scaler": StandardScaler(),
        "quantile": QuantileTransformer(n_quantiles=100, random_state=0),
        "PCA(20)": PCA(n_components=20, random_state=0),
        "SelectKBest(20)": SelectKBest(f_classif, k=20),
    }
    print(f"  {'bosqich':<18} {'leakage bilan':>15} {'Pipeline bilan':>16} "
          f"{'farq':>9}")
    for nom, bosqich in bosqichlar.items():
        # NOTO'G'RI: butun ma'lumotda fit
        Xi = SimpleImputer(strategy="median").fit_transform(X)
        if nom == "imputer":
            Xa = Xi
        elif nom == "SelectKBest(20)":
            Xa = bosqich.fit(Xi, y).transform(Xi)
        else:
            Xa = bosqich.fit_transform(Xi)
        leak = cross_val_score(model, Xa, y, cv=cv, scoring="roc_auc").mean()
        # TO'G'RI: Pipeline
        qadamlar = [("imp", SimpleImputer(strategy="median"))]
        if nom != "imputer":
            qadamlar.append(("t", bosqich))
        qadamlar.append(("m", model))
        togri = cross_val_score(Pipeline(qadamlar), X, y, cv=cv,
                                scoring="roc_auc").mean()
        print(f"  {nom:<18} {leak:>15.4f} {togri:>16.4f} "
              f"{leak - togri:>+9.4f}")

    print("\n=== 3. Signalsiz ma'lumotda (eng aniq ko'rinadi) ===")
    X0, y0 = yarat(seed=7, n=200, p=3000, signal=0)
    X0i = SimpleImputer(strategy="median").fit_transform(X0)
    print(f"  {X0.shape[0]} qator, {X0.shape[1]} belgi, maqsad TASODIFIY")
    print(f"  {'k':>6} {'leakage bilan':>15} {'Pipeline bilan':>16}")
    for k in [10, 50, 200]:
        Xs = SelectKBest(f_classif, k=k).fit(X0i, y0).transform(X0i)
        leak = cross_val_score(model, Xs, y0, cv=cv, scoring="roc_auc").mean()
        quvur = Pipeline([("imp", SimpleImputer(strategy="median")),
                          ("s", SelectKBest(f_classif, k=k)), ("m", model)])
        togri = cross_val_score(quvur, X0, y0, cv=cv,
                                scoring="roc_auc").mean()
        print(f"  {k:>6} {leak:>15.4f} {togri:>16.4f}")

    print("\n=== 4. Bosqichlar birgalikda ===")
    Xi = SimpleImputer(strategy="median").fit_transform(X)
    Xs = StandardScaler().fit_transform(Xi)
    Xk = SelectKBest(f_classif, k=20).fit(Xs, y).transform(Xs)
    leak = cross_val_score(model, Xk, y, cv=cv, scoring="roc_auc").mean()
    quvur = Pipeline([("imp", SimpleImputer(strategy="median")),
                      ("sc", StandardScaler()),
                      ("s", SelectKBest(f_classif, k=20)), ("m", model)])
    togri = cross_val_score(quvur, X, y, cv=cv, scoring="roc_auc").mean()
    print(f"  imputer + scaler + tanlash (butun ma'lumotda): {leak:.4f}")
    print(f"  bir xil bosqichlar Pipeline ichida:            {togri:.4f}")
    print(f"  jamlangan optimizm: {leak - togri:+.4f}")
    print("  ⭐ Belgi tanlash va oversampling eng katta leakage beradi")


if __name__ == "__main__":
    main()

=== 1. Ma'lumot ===
  400 qator, 200 belgi (5 tasi foydali)
  yo'qolgan qiymatlar: 4.9%

=== 2. Har transformatsiyaning leakage ta'siri ===
  bosqich              leakage bilan   Pipeline bilan      farq
  imputer                     0.7440           0.7443   -0.0004
  scaler                      0.7432           0.7427   +0.0005
  quantile                    0.7825           0.7817   +0.0008
  PCA(20)                     0.6599           0.6109   +0.0490
  SelectKBest(20)             0.9102           0.8739   +0.0364

=== 3. Signalsiz ma'lumotda (eng aniq ko'rinadi) ===
  200 qator, 3000 belgi, maqsad TASODIFIY
       k   leakage bilan   Pipeline bilan
      10          0.8140           0.5622
      50          0.9140           0.5707
     200          0.9745           0.4447

=== 4. Bosqichlar birgalikda ===
  imputer + scaler + tanlash (butun ma'lumotda): 0.9102
  bir xil bosqichlar Pipeline ichida:            0.8734
  jamlangan optimizm: +0.0369
  ⭐ Belgi tanlash va oversampling eng katta leakage beradi

"""Takroriy qatorlar va nazorat ro'yxati (real pandas/sklearn)."""

import numpy as np
import pandas as pd
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.metrics import roc_auc_score
from sklearn.model_selection import StratifiedKFold, cross_val_score, train_test_split


def yarat(seed: int = 4, n: int = 3000, duplikat_ulush: float = 0.25):
    rng = np.random.default_rng(seed)
    X = rng.normal(0, 1, (n, 8))
    kuch = X[:, :3] @ np.array([1.1, -0.9, 0.8])
    y = (rng.random(n) < 1 / (1 + np.exp(-kuch))).astype(int)
    # duplikatlar qo'shish
    nechta = int(n * duplikat_ulush)
    idx = rng.choice(n, nechta, replace=True)
    Xd = np.vstack([X, X[idx]])
    yd = np.concatenate([y, y[idx]])
    return Xd, yd, n


def main() -> None:
    X, y, asl_n = yarat()
    cv = StratifiedKFold(5, shuffle=True, random_state=0)

    def model():
        return HistGradientBoostingClassifier(learning_rate=0.1, max_iter=200,
                                              random_state=0)

    print("=== 1. Duplikatlarni aniqlash ===")
    df = pd.DataFrame(X)
    dup = df.duplicated()
    print(f"  jami qatorlar: {len(df)}")
    print(f"  duplikatlar: {int(dup.sum())} ({dup.mean():.1%})")
    print(f"  noyob qatorlar: {len(df) - int(dup.sum())}")

    print("\n=== 2. Duplikatlarning CV ga ta'siri ===")
    b_dup = cross_val_score(model(), X, y, cv=cv, scoring="roc_auc")
    toza_idx = ~dup.to_numpy()
    b_toza = cross_val_score(model(), X[toza_idx], y[toza_idx], cv=cv,
                             scoring="roc_auc")
    print(f"  duplikatlar bilan: {b_dup.mean():.4f} (+-{b_dup.std():.4f})")
    print(f"  duplikatlarsiz:    {b_toza.mean():.4f} (+-{b_toza.std():.4f})")
    print(f"  optimizm: {b_dup.mean() - b_toza.mean():+.4f}")

    print("\n=== 3. Duplikat ulushiga qarab ===")
    print(f"  {'ulush':>7} {'duplikat bilan':>16} {'duplikatsiz':>13} "
          f"{'optimizm':>10}")
    for ulush in [0.0, 0.1, 0.3, 0.6]:
        Xu, yu, _ = yarat(duplikat_ulush=ulush)
        dupu = pd.DataFrame(Xu).duplicated().to_numpy()
        a = cross_val_score(model(), Xu, yu, cv=cv, scoring="roc_auc").mean()
        b = cross_val_score(model(), Xu[~dupu], yu[~dupu], cv=cv,
                            scoring="roc_auc").mean()
        print(f"  {ulush:>7.0%} {a:>16.4f} {b:>13.4f} {a - b:>+10.4f}")

    print("\n=== 4. Audit nazorat ro'yxati ===")
    tekshiruvlar = []
    # 1. duplikatlar
    tekshiruvlar.append(("Duplikatlar",
                         f"{int(dup.sum())} ta ({dup.mean():.1%})",
                         "TOZALANSIN" if dup.sum() else "OK"))
    # 2. har belgining yakka kuchi
    eng_kuchli = 0.0
    for i in range(X.shape[1]):
        bb = cross_val_score(model(), X[toza_idx][:, [i]], y[toza_idx], cv=cv,
                             scoring="roc_auc").mean()
        eng_kuchli = max(eng_kuchli, bb)
    tekshiruvlar.append(("Eng kuchli yakka belgi", f"AUC {eng_kuchli:.4f}",
                         "SHUBHALI" if eng_kuchli > 0.85 else "OK"))
    # 3. umumiy natija
    umumiy = b_toza.mean()
    tekshiruvlar.append(("Umumiy CV AUC", f"{umumiy:.4f}",
                         "SHUBHALI" if umumiy > 0.95 else "OK"))
    # 4. o'quv va test farqi
    Xtr, Xte, ytr, yte = train_test_split(X[toza_idx], y[toza_idx],
                                          test_size=0.3, random_state=0,
                                          stratify=y[toza_idx])
    m = model().fit(Xtr, ytr)
    oquv = roc_auc_score(ytr, m.predict_proba(Xtr)[:, 1])
    test = roc_auc_score(yte, m.predict_proba(Xte)[:, 1])
    tekshiruvlar.append(("O'quv va test farqi", f"{oquv - test:+.4f}",
                         "OVERFITTING" if oquv - test > 0.15 else "OK"))
    print(f"  {'tekshiruv':<26} {'qiymat':>18} {'holat':<14}")
    for nom, qiymat, holat in tekshiruvlar:
        print(f"  {nom:<26} {qiymat:>18} {holat:<14}")
    print("  ⭐ Duplikatlarni bo'lishdan OLDIN tozalang")


if __name__ == "__main__":
    main()

=== 1. Duplikatlarni aniqlash ===
  jami qatorlar: 3750
  duplikatlar: 750 (20.0%)
  noyob qatorlar: 3000

=== 2. Duplikatlarning CV ga ta'siri ===
  duplikatlar bilan: 0.8786 (+-0.0158)
  duplikatlarsiz:    0.7894 (+-0.0117)
  optimizm: +0.0892

=== 3. Duplikat ulushiga qarab ===
    ulush   duplikat bilan   duplikatsiz   optimizm
       0%           0.7894        0.7894    +0.0000
      10%           0.8282        0.7894    +0.0388
      30%           0.8833        0.7894    +0.0940
      60%           0.9288        0.7894    +0.1395

=== 4. Audit nazorat ro'yxati ===
  tekshiruv                              qiymat holat
  Duplikatlar                    750 ta (20.0%) TOZALANSIN
  Eng kuchli yakka belgi             AUC 0.7018 OK
  Umumiy CV AUC                          0.7894 OK
  O'quv va test farqi                   +0.1993 OVERFITTING
  ⭐ Duplikatlarni bo'lishdan OLDIN tozalang

# "AUC 0.97 - ajoyib!"                                            # ⚠️
# har belgining yakka CV natijasini tekshiring                    # ✅

Xs = scaler.fit_transform(X); cross_val_score(m, Xs, y)           # ⚠️
cross_val_score(Pipeline([("sc", scaler), ("m", m)]), X, y)       # ✅

cross_val_score(m, X, y, cv=StratifiedKFold(5))    # bir bemor ko'p qator # ⚠️
cross_val_score(m, X, y, cv=GroupKFold(5), groups=df["bemor"])           # ✅

Xtr, Xte = train_test_split(X); Xtr = drop_duplicates(Xtr)        # ⚠️
X = X.drop_duplicates(); Xtr, Xte = train_test_split(X)           # ✅

Xr, yr = SMOTE().fit_resample(X, y); cross_val_score(m, Xr, yr)   # ⚠️
imblearn.pipeline.Pipeline([("s", SMOTE()), ("m", m)])            # ✅

cross_val_score(m, X, y, cv=KFold(5, shuffle=True))               # ⚠️
cross_val_score(m, X, y, cv=TimeSeriesSplit(5, gap=7))            # ✅

for variant in variantlar: print(variant.score(Xte, yte))         # ⚠️
# CV da tanlang, testda BIR MARTA baholang                        # ✅

1.  # leakage nima?
2.  # nega uni topish qiyin?
3.  # besh asosiy tur?
4.  # eng samarali savol?
5.  # tayyorlashda eng katta leakage?
6.  # guruh leakage i belgisi?
7.  # qaysi CV guruh uchun?
8.  # duplikatlar qachon tozalanadi?
9.  # vaqt uchun qaysi CV?
10. # gap nima uchun?
11. # topishning asosiy signali?
12. # yakuniy tekshiruv?

1.  Xs = scaler.fit_transform(X); cross_val_score(m, Xs, y)

2.  cross_val_score(m, X, y, cv=StratifiedKFold(5))   # bir bemor ko'p qator

3.  Xtr, Xte = train_test_split(X); Xtr = Xtr.drop_duplicates()

4.  Xr, yr = SMOTE().fit_resample(X, y); cross_val_score(m, Xr, yr)

5.  for variant in variantlar: print(variant.score(Xte, yte))