IlmHamroh
Data Science va sun'iy intellekt/ML asoslari9/10-dars18 daqiqa
Mundarija (22)

12.9-dars: Pipeline va ma'lumot sizib chiqishi

12-QISM — MACHINE LEARNING ASOSLARI · 9-dars

1. Kirish va motivatsiya

ML loyihalarining eng keng tarqalgan va eng qimmat xatosi — ma'lumot sizib chiqishi (data leakage): model o'qitish paytida ishlab chiqarishda mavjud bo'lmaydigan ma'lumotni ko'radi. Natija: validatsiyada ajoyib ball, realda muvaffaqiyatsizlik. Leakage shovqinli emas — u jim ishlaydi va natijani yaxshilab ko'rsatadi, shuning uchun uni hech kim shubha ostiga olmaydi.

Bu darsda: leakage turlari (tayyorlashda, belgilarda, vaqtda, guruhlarda, dublikatlarda), pipeline nega asosiy himoya vositasi ekani, ColumnTransformer bilan turli ustunlarga turli ishlov, custom transformer yozish, fit/transform/fit_transform semantikasi va leakage'ni aniqlash usullari.

Real vaziyat. Jamoa kredit modelini quradi: AUC 0.94 — sohada rekord. Ishga tushirilgandan keyin AUC 0.71. Sabablar uchta bo'ldi: (1) StandardScaler butun ma'lumotda fit qilingan; (2) "oxirgi to'lov sanasi" belgisi — u faqat kredit yakunlangandan keyin ma'lum; (3) bitta mijozning bir necha arizasi o'quv va test'ga bo'linib tushgan. Pipeline, vaqt bo'yicha ajratish va GroupKFold bu uchalasini ham hal qildi.

Bu darsda pipeline va leakage'ni o'rganamiz.

Bu darsda:

  • Leakage turlari
  • Pipeline — asosiy himoya
  • ColumnTransformer
  • fit / transform semantikasi
  • Custom transformer
  • Leakage'ni aniqlash
  • Tuzoqlar
  • Amaliy: leakage'siz oqim

ℹ Misollar real numpy/pandas/sklearn bilan (Python 3.14).

2. Nazariya — chuqur tushuntirish

2.1. Leakage turlari

text
1. TAYYORLASH leakage — scaler/imputer/encoder butun ma'lumotda fit qilingan
     → pipeline bilan hal qilinadi

2. BELGI leakage — belgi maqsaddan keyin paydo bo'ladi yoki uni o'z ichiga oladi
     "bekor qilish sanasi", "oxirgi to'lov", "shikoyat bormi" 12.2-bob

3. VAQT leakage — kelajakdagi ma'lumot o'tmishni bashorat qilishda
     → TimeSeriesSplit, vaqt bo'yicha ajratish 12.3-bob

4. GURUH leakage — bir mijoz/bemor o'quv va test'da
     → GroupKFold, GroupShuffleSplit 12.3-bob

5. DUBLIKAT leakage — bir xil qatorlar ikkala to'plamda
     → dublikatlarni ajratishdan OLDIN olib tashlash

6. TANLOV leakage — belgi tanlash yoki sozlash butun ma'lumotda
     → hamma narsa CV ICHIDA

Leakage — "modelga kelajakni ko'rsatish". Olti turning hammasi bitta savolga qaytadi: bashorat paytida bu ma'lumot bormi? 12.2-bob. Eng ko'p uchraydigani — tayyorlash leakage'i, eng qimmati — belgi leakage'i (uni faqat domen bilimi aniqlaydi).

2.2. Pipeline — asosiy himoya

python
from sklearn.pipeline import Pipeline

pipe = Pipeline([("imp", SimpleImputer()), ("sc", StandardScaler()), ("m", Ridge())])

# CV da: har bo'lakda imp va sc FAQAT o'quv qismida fit bo'ladi
cross_val_score(pipe, X, y, cv=5)          # ✅ to'g'ri

X_sc = StandardScaler().fit_transform(X)   # ⚠️ butun ma'lumotda
cross_val_score(Ridge(), X_sc, y, cv=5)    # ⚠️ leakage

Pipeline — qadamlar ketma-ketligi: oxirgisidan boshqa hammasi transform ga ega bo'lishi kerak. fit chaqirilganda har qadam faqat berilgan ma'lumotda o'rganadi — CV bo'lagida bu o'quv qismi. Shu tufayli pipeline tayyorlash leakage'ini butunlay yo'q qiladi, shuningdek GridSearchCV bilan butun oqim bo'ylab giperparametrni sozlashga imkon beradi (m__alpha, imp__strategy).

2.3. ColumnTransformer

python
from sklearn.compose import ColumnTransformer, make_column_selector

tayyor = ColumnTransformer([
    ("son", Pipeline([("imp", SimpleImputer(strategy="median")),
                      ("sc", StandardScaler())]),
     make_column_selector(dtype_include=np.number)),
    ("kat", Pipeline([("imp", SimpleImputer(strategy="most_frequent")),
                      ("oh", OneHotEncoder(handle_unknown="ignore", min_frequency=10))]),
     make_column_selector(dtype_include=object)),
], remainder="drop", verbose_feature_names_out=False)

tayyor.get_feature_names_out()        # yakuniy belgi nomlari

ColumnTransformer — turli ustun guruhlariga turli ishlov: sonlarga imputer+scaler, kategoriyalarga imputer+one-hot. handle_unknown="ignore" — test'da yangi kategoriya uchrasa xato bermaydi (majburiy); min_frequency — kam uchraydigan kategoriyalarni birlashtiradi. remainder — qolgan ustunlar bilan nima qilish ("drop" yoki "passthrough").

2.4. fit / transform semantikasi

text
fit(X_train)            — parametrlarni O'RGANADI (o'rtacha, SD, kategoriyalar ro'yxati)
transform(X)            — o'rganilgan parametrlar bilan o'zgartiradi
fit_transform(X_train)  — ikkalasi (faqat O'QUVda!)

TO'G'RI:                       NOTO'G'RI:
sc.fit_transform(X_tr)         sc.fit_transform(X)      ⚠️ butun ma'lumot
sc.transform(X_te)             sc.fit_transform(X_te)   ⚠️ test'da fit

Bu eng muhim qoida: fit faqat o'quv ma'lumotida. Test va ishlab chiqarishda faqat transform. Xatoning tipik ko'rinishi — fit_transform(X_test): u test'ning o'z o'rtachasidan foydalanadi, ya'ni model test taqsimotini "ko'radi" va natija optimistik bo'ladi. Pipeline bu xatoni texnik jihatdan imkonsiz qiladi.

2.5. Custom transformer

python
from sklearn.base import BaseEstimator, TransformerMixin

class LogBelgi(BaseEstimator, TransformerMixin):
    def __init__(self, ustunlar=None):
        self.ustunlar = ustunlar
    def fit(self, X, y=None):
        self.n_ = X.shape[1]              # o'rganilgan narsa _ bilan tugaydi
        return self
    def transform(self, X):
        X = X.copy()
        X[self.ustunlar] = np.log1p(X[self.ustunlar])
        return X

# yoki holatsiz o'zgartirish uchun:
from sklearn.preprocessing import FunctionTransformer
log = FunctionTransformer(np.log1p, feature_names_out="one-to-one")

Custom transformer — BaseEstimator va TransformerMixin dan meros olib, fit va transform yozish (get_params/set_params va fit_transform avtomatik keladi). Qoida: fit da o'rganilgan atributlar pastki chiziq bilan tugaydi (self.median_), __init__ da hech qanday hisob-kitob bo'lmaydi (faqat parametrlarni saqlash). Holatsiz funksiyalar uchun FunctionTransformer yetarli.

2.6. Leakage'ni aniqlash

text
Shubha belgilari:
  · ajoyib natija (AUC > 0.98, R^2 > 0.99) — birinchi shubha
  · bitta belgi hukmron (importance 0.9+)
  · natija ishlab chiqarishda keskin tushadi
  · o'quv va test natijasi juda yaqin (haddan tashqari)

Tekshirish usullari:
  1. Har belgi uchun: "bashorat paytida bu bormi?" 12.2-bob
  2. Belgi va maqsad korrelyatsiyasini ko'rish (juda yuqori — shubha)
  3. Bitta belgi bilan model o'qitish (AUC 0.95 — leakage)
  4. Vaqt bo'yicha ajratib qayta tekshirish
  5. Dublikat va guruhlarni sanash

Birinchi qoida: juda yaxshi natijaga shubha qiling. Tajribali Data Scientist uchun AUC 0.99 — bayram emas, ogohlantirish. Eng tez tekshiruv: har bir belgini alohida model qilib sinash — bittasi deyarli mukammal natija bersa, u leakage.

2.7. Tuzoqlar

Asosiy tuzoqlar: fit_transform ni butun ma'lumotda; pipeline'siz belgi tanlash yoki SMOTE (CV dan tashqarida); maqsaddan hosil qilingan belgi (target encoding — CV ichida qilish kerak); dublikatlarni ajratishdan keyin olib tashlash; vaqt qatorida tasodifiy ajratish; guruhlarni hisobga olmaslik; test'ni bir necha marta ishlatish 12.3-bob; pipeline'ni saqlamaslik (ishlab chiqarishda qadamlar farq qiladi).

2.8. Pipeline — intizom vositasi

Leakage — modelga bashorat paytida mavjud bo'lmaydigan ma'lumotni ko'rsatish; u natijani yaxshilab ko'rsatadi va shuning uchun xavfli. Olti turi: tayyorlash, belgi, vaqt, guruh, dublikat, tanlov. Pipeline tayyorlash va tanlov leakage'ini texnik jihatdan yo'q qiladi (fit faqat o'quv bo'lagida), ColumnTransformer turli ustunlarga turli ishlov beradi, custom transformer o'z mantiqingizni shu himoya ichiga kiritadi. Belgi, vaqt va guruh leakage'i esa domen bilimi va to'g'ri ajratish bilan hal qilinadi. Keyingi dars — amaliyot: to'liq ML loyihasi.

3. Tez ma'lumotnoma

python
import numpy as np
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.compose import ColumnTransformer, make_column_selector
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import FunctionTransformer, OneHotEncoder, StandardScaler

tayyor = ColumnTransformer([
    ("son", Pipeline([("imp", SimpleImputer(strategy="median")),
                      ("sc", StandardScaler())]), son_ustunlar),
    ("kat", OneHotEncoder(handle_unknown="ignore", min_frequency=10), kat_ustunlar),
])
pipe = Pipeline([("t", tayyor), ("m", model)])

pipe.fit(X_tr, y_tr)                  # barcha qadamlar faqat o'quvda
cross_val_score(pipe, X_tr, y_tr, cv=5)
pipe.named_steps["t"].get_feature_names_out()

import joblib
joblib.dump(pipe, "model.joblib")     # butun oqim saqlanadi
QOIDA: fit faqat o'quvda · hamma tayyorlash pipeline ichida · juda yaxshi natijaga shubha

Leakage xulosasi

Tayyorlash — pipeline · Belgi — domen bilimi · Vaqt — TimeSeriesSplit
Guruh — GroupKFold · Dublikat — ajratishdan oldin tozalash · Tanlov — CV ichida
Shubha: AUC > 0.98, bitta belgi hukmron, ishlab chiqarishda tushish

4. Batafsil misollar

Misollar real numpy/pandas/sklearn bilan (Python 3.14).

Misol 1 — Tayyorlash leakage'i: raqamlar bilan

python
"""Butun ma'lumotda fit qilish qanchalik aldaydi (real numpy/sklearn)."""

import numpy as np
from sklearn.feature_selection import SelectKBest, f_classif
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold, cross_val_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler


def main() -> None:
    rng = np.random.default_rng(0)
    n, p = 200, 2000                      # kam namuna, ko'p belgi
    X = rng.normal(size=(n, p))
    y = rng.integers(0, 2, n)             # maqsad belgilarga BOG'LIQ EMAS
    cv = StratifiedKFold(5, shuffle=True, random_state=0)

    print("=== 1. Ma'lumot: maqsad butunlay tasodifiy ===")
    print(f"  {n} namuna, {p} belgi; haqiqiy bog'liqlik YO'Q")
    print("  to'g'ri natija ~0.50 bo'lishi kerak")

    print("\n=== 2. NOTO'G'RI: belgi tanlash CV dan TASHQARIDA ===")
    tanlangan = SelectKBest(f_classif, k=20).fit_transform(X, y)   # butun ma'lumotda
    ball = cross_val_score(LogisticRegression(max_iter=1000), tanlangan, y, cv=cv)
    print(f"  CV aniqlik = {ball.mean():.3f} ± {ball.std():.3f}  ← soxta natija!")

    print("\n=== 3. TO'G'RI: tanlash pipeline ICHIDA ===")
    pipe = Pipeline([("tanla", SelectKBest(f_classif, k=20)),
                     ("sc", StandardScaler()),
                     ("m", LogisticRegression(max_iter=1000))])
    ball2 = cross_val_score(pipe, X, y, cv=cv)
    print(f"  CV aniqlik = {ball2.mean():.3f} ± {ball2.std():.3f}  ← haqiqat")

    print("\n=== 4. Scaler leakage'i — bu yerda deyarli sezilmaydi ===")
    Xs = StandardScaler().fit_transform(X[:, :20])                 # butun ma'lumotda
    a = cross_val_score(LogisticRegression(max_iter=1000), Xs, y, cv=cv).mean()
    b = cross_val_score(Pipeline([("sc", StandardScaler()),
                                  ("m", LogisticRegression(max_iter=1000))]),
                        X[:, :20], y, cv=cv).mean()
    print(f"  tashqarida fit: {a:.3f}, pipeline ichida: {b:.3f}")
    print("  (o'rtacha/SD ko'p namunadan hisoblanadi — farq kichik)")
    print("  ⭐ Belgi tanlash leakage'i eng xavflisi, scaler'niki eng kichigi")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Ma'lumot: maqsad butunlay tasodifiy ===
  200 namuna, 2000 belgi; haqiqiy bog'liqlik YO'Q
  to'g'ri natija ~0.50 bo'lishi kerak

=== 2. NOTO'G'RI: belgi tanlash CV dan TASHQARIDA ===
  CV aniqlik = 0.765 ± 0.068  ← soxta natija!

=== 3. TO'G'RI: tanlash pipeline ICHIDA ===
  CV aniqlik = 0.520 ± 0.100  ← haqiqat

=== 4. Scaler leakage'i — bu yerda deyarli sezilmaydi ===
  tashqarida fit: 0.530, pipeline ichida: 0.530
  (o'rtacha/SD ko'p namunadan hisoblanadi — farq kichik)
  ⭐ Belgi tanlash leakage'i eng xavflisi, scaler'niki eng kichigi

Nima ko'rsatdi: 2.1, 2.2-bo'limlar.

Misol 2 — ColumnTransformer bilan to'liq tayyorlash

python
"""Aralash ustunlarga to'g'ri ishlov (real pandas/sklearn)."""

import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler


def yarat(seed: int = 12, n: int = 4000) -> pd.DataFrame:
    rng = np.random.default_rng(seed)
    shahar = rng.choice(["Toshkent", "Samarqand", "Namangan", "Nukus", "Xiva"],
                        n, p=[0.45, 0.22, 0.18, 0.1, 0.05])
    tarif = rng.choice(["start", "standart", "premium"], n, p=[0.5, 0.35, 0.15])
    yosh = rng.integers(18, 65, n).astype(float)
    daromad = rng.lognormal(14.6, 0.5, n)
    muddat = rng.integers(1, 48, n).astype(float)
    ball = (-1.2 - 0.03 * (yosh - 35) + 0.5 * (tarif == "start")
            - 0.4 * (shahar == "Toshkent") - 0.02 * muddat)
    y = (rng.random(n) < 1 / (1 + np.exp(-ball))).astype(int)
    df = pd.DataFrame({"shahar": shahar, "tarif": tarif, "yosh": yosh,
                       "daromad": daromad.round(0), "muddat": muddat, "ketdi": y})
    df.loc[rng.choice(n, 250, replace=False), "daromad"] = np.nan
    df.loc[rng.choice(n, 120, replace=False), "shahar"] = None
    return df


def main() -> None:
    df = yarat()
    X, y = df.drop(columns=["ketdi"]), df["ketdi"]
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)

    print("=== 1. Ma'lumot ===")
    print(f"  {len(df)} qator; ketish ulushi {y.mean():.1%}")
    print(f"  yetishmovchilik: {df.isna().sum()[df.isna().sum() > 0].to_dict()}")

    son = ["yosh", "daromad", "muddat"]
    kat = ["shahar", "tarif"]
    tayyor = ColumnTransformer([
        ("son", Pipeline([("imp", SimpleImputer(strategy="median")),
                          ("sc", StandardScaler())]), son),
        ("kat", Pipeline([("imp", SimpleImputer(strategy="most_frequent")),
                          ("oh", OneHotEncoder(handle_unknown="ignore",
                                               min_frequency=200))]), kat),
    ], verbose_feature_names_out=False)

    pipe = Pipeline([("t", tayyor), ("m", LogisticRegression(max_iter=1000))])
    pipe.fit(Xtr, ytr)

    print("\n=== 2. Yaratilgan belgilar ===")
    nomlar = pipe.named_steps["t"].get_feature_names_out()
    print(f"  {len(nomlar)} ta: {list(nomlar)}")
    print("  (min_frequency=200 kam uchraydigan shaharlarni bitta ustunga birlashtirdi)")

    print("\n=== 3. Natija ===")
    p = pipe.predict_proba(Xte)[:, 1]
    print(f"  test ROC AUC = {roc_auc_score(yte, p):.3f}")

    print("\n=== 4. Yangi kategoriya test'da ===")
    yangi = Xte.iloc[:3].copy()
    yangi.loc[:, "shahar"] = "Qarshi"                # o'quvda bo'lmagan qiymat
    print(f"  bashorat ishladi: {pipe.predict_proba(yangi)[:, 1].round(3)}")
    print("  ⭐ handle_unknown='ignore' — majburiy")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Ma'lumot ===
  4000 qator; ketish ulushi 14.9%
  yetishmovchilik: {'shahar': 120, 'daromad': 250}

=== 2. Yaratilgan belgilar ===
  11 ta: ['yosh', 'daromad', 'muddat', 'shahar_Namangan', 'shahar_Nukus', 'shahar_Samarqand', 'shahar_Toshkent', 'shahar_infrequent_sklearn', 'tarif_premium', 'tarif_standart', 'tarif_start']
  (min_frequency=200 kam uchraydigan shaharlarni bitta ustunga birlashtirdi)

=== 3. Natija ===
  test ROC AUC = 0.635

=== 4. Yangi kategoriya test'da ===
  bashorat ishladi: [0.186 0.241 0.177]
  ⭐ handle_unknown='ignore' — majburiy

Nima ko'rsatdi: 2.3, 2.4-bo'limlar.

Misol 3 — Custom transformer va target encoding

python
"""O'z transformeringiz va CV ichida target encoding (real pandas/sklearn)."""

import numpy as np
import pandas as pd
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.model_selection import StratifiedKFold, cross_val_score
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline


class TargetEncoder(BaseEstimator, TransformerMixin):
    """Kategoriyani maqsad o'rtachasi bilan almashtirish (silliqlash bilan)."""

    def __init__(self, ustun: str = "id", silliq: float = 20.0):
        self.ustun = ustun
        self.silliq = silliq

    def fit(self, X, y):
        y = np.asarray(y)
        umumiy = y.mean()
        g = pd.DataFrame({"k": X[self.ustun].to_numpy(), "y": y}).groupby("k")["y"]
        n, o = g.count(), g.mean()
        self.xarita_ = ((n * o + self.silliq * umumiy) / (n + self.silliq)).to_dict()
        self.umumiy_ = float(umumiy)
        return self

    def transform(self, X):
        kod = X[self.ustun].map(self.xarita_).fillna(self.umumiy_)
        return np.column_stack([kod.to_numpy(), X.drop(columns=[self.ustun]).to_numpy()])


def main() -> None:
    rng = np.random.default_rng(3)
    n = 3000
    idn = rng.integers(0, 900, n)                    # ko'p darajali kategoriya
    x1 = rng.normal(size=n)
    y = (rng.random(n) < 1 / (1 + np.exp(-(0.8 * x1 - 0.3)))).astype(int)
    df = pd.DataFrame({"id": idn, "x1": x1})
    cv = StratifiedKFold(5, shuffle=True, random_state=0)

    print("=== 1. Vazifa ===")
    print(f"  {n} qator, 'id' — {df['id'].nunique()} daraja, maqsadga BOG'LIQ EMAS")
    print(f"  faqat x1 haqiqiy belgi; ketish ulushi {y.mean():.1%}")

    print("\n=== 2. NOTO'G'RI: encoding butun ma'lumotda ===")
    te = TargetEncoder(ustun="id").fit(df, y)
    Xleak = te.transform(df)
    ball = cross_val_score(LogisticRegression(max_iter=1000), Xleak, y, cv=cv,
                           scoring="roc_auc")
    print(f"  CV ROC AUC = {ball.mean():.3f} ← maqsad ma'lumoti sizib chiqdi")

    print("\n=== 3. TO'G'RI: encoding pipeline ICHIDA ===")
    pipe = Pipeline([("te", TargetEncoder(ustun="id")),
                     ("m", LogisticRegression(max_iter=1000))])
    ball2 = cross_val_score(pipe, df, y, cv=cv, scoring="roc_auc")
    print(f"  CV ROC AUC = {ball2.mean():.3f} ± {ball2.std():.3f}")

    print("\n=== 4. Faqat x1 bilan (haqiqiy chegara) ===")
    ball3 = cross_val_score(LogisticRegression(max_iter=1000), df[["x1"]], y, cv=cv,
                            scoring="roc_auc")
    print(f"  CV ROC AUC = {ball3.mean():.3f}")
    print("  ⭐ To'g'ri variant haqiqiy chegaraga yaqin, leakage undan yuqori")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Vazifa ===
  3000 qator, 'id' — 871 daraja, maqsadga BOG'LIQ EMAS
  faqat x1 haqiqiy belgi; ketish ulushi 43.4%

=== 2. NOTO'G'RI: encoding butun ma'lumotda ===
  CV ROC AUC = 0.812 ← maqsad ma'lumoti sizib chiqdi

=== 3. TO'G'RI: encoding pipeline ICHIDA ===
  CV ROC AUC = 0.677 ± 0.018

=== 4. Faqat x1 bilan (haqiqiy chegara) ===
  CV ROC AUC = 0.697
  ⭐ To'g'ri variant haqiqiy chegaraga yaqin, leakage undan yuqori

Nima ko'rsatdi: 2.5, 2.7-bo'limlar.

Misol 4 — Leakage'ni aniqlash: diagnostika

python
"""Shubhali belgini topish usullari (real pandas/sklearn)."""

import numpy as np
import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
from sklearn.model_selection import train_test_split


def yarat(seed: int = 21, n: int = 4000) -> pd.DataFrame:
    rng = np.random.default_rng(seed)
    yosh = rng.integers(18, 65, n).astype(float)
    muddat = rng.integers(1, 48, n).astype(float)
    shikoyat = rng.poisson(0.7, n).astype(float)
    ball = -1.0 - 0.02 * (yosh - 35) - 0.03 * muddat + 0.45 * shikoyat
    y = (rng.random(n) < 1 / (1 + np.exp(-ball))).astype(int)
    # leakage belgisi: bekor qilish arizasi faqat ketganlarda to'ldiriladi
    bekor_ariza = np.where(y == 1, rng.random(n) < 0.93, rng.random(n) < 0.02)
    return pd.DataFrame({"yosh": yosh, "muddat": muddat, "shikoyat": shikoyat,
                         "bekor_ariza": bekor_ariza.astype(float), "ketdi": y})


def main() -> None:
    df = yarat()
    X, y = df.drop(columns=["ketdi"]), df["ketdi"]
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)

    print("=== 1. Model 'juda yaxshi' natija beradi ===")
    m = RandomForestClassifier(n_estimators=200, random_state=0).fit(Xtr, ytr)
    print(f"  test ROC AUC = {roc_auc_score(yte, m.predict_proba(Xte)[:, 1]):.3f}"
          f"  ← shubha")

    print("\n=== 2. Tekshiruv A: belgi muhimligi ===")
    for nom, v in sorted(zip(X.columns, m.feature_importances_),
                         key=lambda t: -t[1]):
        print(f"  {nom:<12}: {v:.3f}")

    print("\n=== 3. Tekshiruv B: har belgi alohida ===")
    for ustun in X.columns:
        mm = LogisticRegression(max_iter=1000).fit(Xtr[[ustun]], ytr)
        auc = roc_auc_score(yte, mm.predict_proba(Xte[[ustun]])[:, 1])
        belgi = "  ← LEAKAGE" if auc > 0.9 else ""
        print(f"  {ustun:<12}: AUC {auc:.3f}{belgi}")

    print("\n=== 4. Shubhali belgini olib tashlash ===")
    toza = [c for c in X.columns if c != "bekor_ariza"]
    m2 = RandomForestClassifier(n_estimators=200, random_state=0).fit(Xtr[toza], ytr)
    print(f"  test ROC AUC = {roc_auc_score(yte, m2.predict_proba(Xte[toza])[:, 1]):.3f}"
          f"  ← realistik")
    print("  ⭐ 'Bashorat paytida bu belgi bormi?' — asosiy savol")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Model 'juda yaxshi' natija beradi ===
  test ROC AUC = 0.957  ← shubha

=== 2. Tekshiruv A: belgi muhimligi ===
  bekor_ariza : 0.802
  yosh        : 0.091
  muddat      : 0.086
  shikoyat    : 0.022

=== 3. Tekshiruv B: har belgi alohida ===
  yosh        : AUC 0.589
  muddat      : AUC 0.614
  shikoyat    : AUC 0.602
  bekor_ariza : AUC 0.954  ← LEAKAGE

=== 4. Shubhali belgini olib tashlash ===
  test ROC AUC = 0.568  ← realistik
  ⭐ 'Bashorat paytida bu belgi bormi?' — asosiy savol

Nima ko'rsatdi: 2.1, 2.6-bo'limlar.

5. To'g'ri va noto'g'ri tushunishlar

Noto'g'ri fikr To'g'risi
"Scaler leakage'i kichik" Belgi tanlashda halokatli
"Pipeline — qulaylik" Himoya vositasi
"AUC 0.99 — zo'r model" Birinchi shubha
"Leakage'ni kod ko'rsatadi" Ko'pincha domen bilimi kerak
"Target encoding oddiy" CV ichida bo'lishi shart
"Dublikatlar zararsiz" Test'ga sizib o'tadi
"fit_transform har joyda" Faqat o'quvda
"Test bir marta emas, ko'p marta" Bir marta

6. Keng tarqalgan xatolar va yechimlari

1. Butun ma'lumotda fit

python
X = StandardScaler().fit_transform(X)                             # ⚠️
Pipeline([("sc", StandardScaler()), ("m", model)])                # ✅

2. Belgi tanlash CV dan tashqarida

python
X = SelectKBest(k=20).fit_transform(X, y)                         # ⚠️
Pipeline([("tanla", SelectKBest(k=20)), ("m", model)])            # ✅

3. Target encoding tashqarida

python
df["kod"] = df.groupby("id")["y"].transform("mean")               # ⚠️
Pipeline([("te", TargetEncoder()), ("m", model)])                 # ✅

4. Yangi kategoriya

python
OneHotEncoder()                        # test'da xato beradi       # ⚠️
OneHotEncoder(handle_unknown="ignore")                            # ✅

5. Dublikatlar

python
train_test_split(df, ...)              # dublikatlar bor          # ⚠️
df = df.drop_duplicates(); train_test_split(df, ...)              # ✅

6. Faqat modelni saqlash

python
joblib.dump(model, "model.joblib")     # tayyorlash yo'q          # ⚠️
joblib.dump(pipe, "model.joblib")      # butun oqim               # ✅

7. Shubhasiz qabul qilish

python
print("AUC 0.99 — ishga tushiramiz")                              # ⚠️
# har belgini tekshiring: bashorat paytida mavjudmi?              # ✅

7. Integratsiya — bu bilim qayerda kerak bo'ladi

  • 12.2-dars (o'tilgan): Belgi leakage'i va vaqt kesimi
  • 12.3-dars (o'tilgan): Guruh va vaqt bo'yicha ajratish
  • 12.6-dars (o'tilgan): Pipeline oqimda
  • 12.10-dars: To'liq loyiha
  • Feature engineering qismi: Kengaytirilgan transformerlar

8. Eng yaxshi amaliyotlar

  1. Hamma tayyorlashni pipeline ichiga qo'ying.

  2. fit faqat o'quvda.

  3. handle_unknown='ignore' qo'ying.

  4. Har belgi uchun "bashorat paytida bormi?" savolini bering.

  5. Dublikatlarni ajratishdan oldin tozalang.

  6. Juda yaxshi natijaga shubha qiling.

  7. Butun pipeline'ni saqlang.

  8. Har belgini alohida sinab ko'ring.

9. Amaliy topshiriq

Vazifa 1: Bashorat qiling

python
1.  # leakage nima?
2.  # tayyorlash leakage'i qanday hal qilinadi?
3.  # belgi leakage'i?
4.  # fit qayerda chaqiriladi?
5.  # fit_transform(X_test) — nima xato?
6.  # ColumnTransformer nima uchun?
7.  # handle_unknown='ignore' nima qiladi?
8.  # custom transformer'da o'rganilgan atribut qanday nomlanadi?
9.  # target encoding qayerda bajariladi?
10. # AUC 0.99 — birinchi fikr?
11. # dublikatlar qachon olib tashlanadi?
12. # nimani saqlash kerak?
Javoblar
  1. Bashorat paytida yo'q ma'lumotni ko'rish
  2. Pipeline
  3. Maqsaddan keyin paydo bo'ladigan belgi
  4. Faqat o'quvda
  5. Test taqsimotini o'rganadi
  6. Turli ustunlarga turli ishlov
  7. Yangi kategoriyada xato bermaydi
  8. Pastki chiziq bilan (median_)
  9. Pipeline/CV ichida
  10. Shubha — leakage tekshirish
  11. Ajratishdan oldin
  12. Butun pipeline

Vazifa 2: Xatolarni tuzating

python
1.  X = StandardScaler().fit_transform(X)

2.  X = SelectKBest(k=30).fit_transform(X, y)

3.  df["hudud_kod"] = df.groupby("hudud")["target"].transform("mean")

4.  enc = OneHotEncoder().fit(X_train)   # test'da yangi qiymat bor

5.  joblib.dump(model, "m.joblib")   # scaler alohida
Javoblar
python
1.  Pipeline([("sc", StandardScaler()), ("m", model)])

2.  Pipeline([("tanla", SelectKBest(k=30)), ("m", model)])

3.  Pipeline([("te", TargetEncoder(ustun="hudud")), ("m", model)])

4.  OneHotEncoder(handle_unknown="ignore")

5.  joblib.dump(pipe, "m.joblib")

Vazifa 3: Leakage tajribasi

Modellang:

  1. Tasodifiy ma'lumot
  2. Tanlash tashqarida/ichida
  3. Farqni o'lchash
  4. Xulosa

Vazifa 4: To'liq pipeline

Modellang:

  1. Aralash ustunlar
  2. ColumnTransformer
  3. GridSearchCV
  4. Saqlash va yuklash

Vazifa 5: Custom transformer

Modellang:

  1. O'z transformeringiz
  2. fit/transform
  3. Pipeline'ga qo'shish
  4. CV bilan tekshirish

Vazifa 6: Diagnostika

Modellang:

  1. Leakage belgisi qo'shing
  2. Uchta tekshiruv
  3. Olib tashlash
  4. Hisobot

Vazifa 7: O'ylash

Leakage'ning ko'pchiligi texnik xato emas — u ma'lumot qanday yig'ilganini bilmaslikdan kelib chiqadi. Data Scientist ma'lumot bazasini o'zi yaratmaydi va har ustunning ma'nosini bilmaydi. Bu muammoni jamoa darajasida qanday hal qilish kerak?

Javob

Qisqa javob: leakage — tashkiliy muammo: uning yechimi ham tashkiliy — ma'lumot hujjati (data dictionary), domen mutaxassisi bilan belgi ko'rigi va ishlab chiqarish sharoitida takrorlanadigan tekshiruv.

1. Nega texnik yechim yetarli emas

  • Pipeline faqat tayyorlash leakage'ini to'xtatadi
  • "bekor_ariza" ustuni sintaktik jihatdan normal ko'rinadi
  • Faqat domen bilimi "bu qachon to'ldiriladi?" degan savolga javob beradi

2. Jamoa amaliyotlari

Amaliyot Nima beradi
Data dictionary (har ustun: manba, to'ldirilish vaqti) Belgi leakage'ini oldini oladi
Domen mutaxassisi bilan belgi ko'rigi Shubhali ustunlarni topadi
Feature store (vaqt kesimi bilan) Texnik kafolat
Offline/online moslik testi Ishlab chiqarishdagi farqni topadi
Shadow rejim (real trafikda kuzatish) Yakuniy tasdiq

3. Hujjatda nima bo'lishi kerak

  • Ustun manbai va yangilanish vaqti
  • Qachon to'ldiriladi (hodisagacha yoki keyin)
  • Yetishmovchilik sababi (MCAR/MAR/MNAR — 6-qism)

4. Data Scientist roli

  • Har belgi uchun vaqt savolini berish (12.2)
  • Ajoyib natijani shubha bilan tekshirish
  • Topilgan leakage'ni hujjatlashtirish (keyingi loyihaga foyda)

5. Xulosa

  1. Leakage — jarayon muammosi
  2. Pipeline — faqat bir qismi
  3. Hujjat va domen ko'rigi majburiy
  4. Shadow rejim — yakuniy himoya

Nimani mustahkamlaydi: 2.1, 2.6-bo'limlar.

Xulosa

Bu darsda pipeline va ma'lumot sizib chiqishini o'rgandik.

Eng muhim uch fikr:

  1. Leakage natijani yaxshilab ko'rsatadi. Shuning uchun u jim va xavfli: validatsiyada AUC 0.99, ishlab chiqarishda 0.71. Olti turi bor: tayyorlash (butun ma'lumotda fit), belgi (maqsaddan keyin paydo bo'ladigan ustun), vaqt, guruh, dublikat, tanlov (belgi tanlash/sozlash CV dan tashqarida).

  2. Pipeline — texnik himoya. Pipeline + ColumnTransformer barcha tayyorlash qadamlarini modelga bog'laydi: CV da har bo'lakda fit faqat o'quv qismida bajariladi, ya'ni tayyorlash va tanlov leakage'i imkonsiz bo'ladi. Qoida o'zgarmas: fit faqat o'quvda, test'da faqat transform; handle_unknown="ignore" majburiy; butun pipeline saqlanadi.

  3. Qolgan turlari — domen bilimi. Belgi leakage'ini kod topmaydi: har ustun uchun "bashorat paytida bu ma'lumot bormi?" savolini berish kerak 12.2-bob. Diagnostika: juda yaxshi natijaga shubha, belgi muhimligi, har belgini alohida model qilib sinash (AUC > 0.9 — leakage), vaqt va guruh bo'yicha qayta tekshirish 12.3-bob.

Keyingi darsda amaliyot — 12-qismning to'liq loyihasi: ma'lumotdan boshlab, baholangan va hujjatlashtirilgan modelgacha.

Ulashish:Telegram'da

Izohlar (0)

Izoh yozish uchun kiring.

  • Hozircha izoh yo'q. Birinchi bo'ling!