14.12-dars: Modellarni tanlash va taqqoslash

Muammo: 40 ta konfiguratsiyadan ENG YAXSHISINI tanlasangiz,
        uning CV bahosi OPTIMISTIK bo'ladi

Sabab: har baho shovqinli; ko'p urinishdan eng yuqorisini tanlash —
       shovqinning eng yaxshi tomonini tanlash 11.9-bob

Siljish kattaroq bo'ladi, agar:
  · nomzodlar soni ko'p bo'lsa
  · CV baholari shovqinli bo'lsa (kichik ma'lumot)
  · nomzodlar bir-biriga yaqin bo'lsa

Natija: best_score_ — YAKUNIY BAHO EMAS (12.6)

TASHQI sikl  — modelni BAHOLASH uchun (K bo'lak)
  har tashqi bo'lakda:
    ICHKI sikl — giperparametrni TANLASH uchun (faqat o'quv qismida)
    tanlangan model tashqi test bo'lagida baholanadi

Natija: tanlov jarayonining O'ZI baholanadi -> siljimagan baho

ichki = GridSearchCV(model, setka, cv=StratifiedKFold(5, shuffle=True, random_state=1))
baho = cross_val_score(ichki, X, y, cv=StratifiedKFold(5, shuffle=True, random_state=0))

Narxi: K_tashqi x K_ichki x |setka| ta o'qitish (qimmat)

from sklearn.model_selection import RepeatedStratifiedKFold

cv = RepeatedStratifiedKFold(n_splits=5, n_repeats=5, random_state=0)
ballar = cross_val_score(model, X, y, cv=cv, scoring="f1_macro")
print(f"{ballar.mean():.4f} +- {ballar.std():.4f}  (n = {len(ballar)})")

Ikki modelni solishtirish:
  1. BIR XIL bo'linishlarda baholang (juftlashtirilgan - 11.4)
  2. Har bo'lakdagi farqni hisoblang: d_i = ball_A - ball_B
  3. O'rtacha farqqa VA uning tarqoqligiga qarang

Muammo: CV ballari bog'liq -> oddiy t-test p-qiymatini PASAYTIRADI
Yechimlar:
  · takrorlangan CV + farqlar oralig'i (ehtiyotkor talqin)
  · 5x2cv F-test yoki tuzatilgan t-test (Nadeau-Bengio)
  · eng amaliysi: farq o'z SD sidan kattami?

Amaliy qoida: farq < 1 SD -> "farq yo'q" deb qarang

"1-SE rule": eng yaxshi ballning 1 standart xatosi ichidagi
             ENG SODDA modelni tanlang

Misol: C=100 -> 0.842 +- 0.021 (SE)
       C=1   -> 0.828            (0.842 - 0.021 = 0.821 dan yuqori)
  -> C=1 ni tanlang (soddaroq, kuchliroq regularizatsiya)

Nega: eng yuqori ball shovqin tufayli bo'lishi mumkin;
      soddaroq model yangi ma'lumotda barqarorroq (12.5)

Aniqlikdan tashqari:
  · bashorat vaqti va o'tkazuvchanlik
  · o'qitish vaqti va qayta o'qitish chastotasi
  · xotira va model hajmi
  · ehtimol kerakmi (kalibrlash - 14.10)
  · talqin talabi (regulyator, foydalanuvchi)
  · barqarorlik (drift'ga chidamlilik)
  · qo'llab-quvvatlash murakkabligi

Qaror matritsasi: har mezon uchun ball + vazn -> jami

import numpy as np
from sklearn.model_selection import (GridSearchCV, RepeatedStratifiedKFold,
                                     StratifiedKFold, cross_val_score)

# ichma-ich CV — siljimagan baho
ichki = GridSearchCV(model, setka, cv=StratifiedKFold(5, shuffle=True, random_state=1))
baho = cross_val_score(ichki, X, y, cv=StratifiedKFold(5, shuffle=True, random_state=0))

# takrorlangan CV — noaniqlik
cv = RepeatedStratifiedKFold(n_splits=5, n_repeats=5, random_state=0)
b = cross_val_score(model, X, y, cv=cv, scoring="f1_macro")
print(f"{b.mean():.4f} +- {b.std():.4f}")

# juftlashtirilgan taqqoslash (bir xil bo'linishlarda)
farq = ballar_A - ballar_B
print(farq.mean(), farq.std(), (farq > 0).mean())
QOIDA: best_score_ yakuniy emas · juftlashtir · 1-SE qoidasi · narxni hisobla

Optimizm siljishi: ko'p nomzod -> eng yaxshi ball siljigan
Ichma-ich CV — tanlov jarayonini baholaydi (qimmat, kichik ma'lumot uchun)
Takrorlangan CV — noaniqlik; juftlashtirilgan farq — adolatli taqqoslash
1-SE qoidasi: eng yaxshining 1 SE ichidagi eng soddasi
Yakuniy mezon: aniqlik + tezlik + talqin + qo'llab-quvvatlash

"""Ko'p nomzoddan eng yaxshisini tanlash bahoni qanday siljitadi (real numpy/sklearn)."""

import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold, cross_val_score


def main() -> None:
    rng = np.random.default_rng(3)
    n = 300
    y = rng.integers(0, 2, n)
    cv = StratifiedKFold(5, shuffle=True, random_state=0)

    print("=== 1. Maqsad belgilarga BOG'LIQ EMAS ===")
    print(f"  {n} namuna, haqiqiy aniqlik ~0.50 bo'lishi kerak")

    print("\n=== 2. Nomzodlar soni ortganda 'eng yaxshi' ball ===")
    print(f"  {'nomzodlar':>10} {'eng yaxshi CV':>14} {'o_rtacha CV':>13}")
    for nomzodlar in [1, 5, 20, 100]:
        ballar = []
        for _ in range(nomzodlar):
            X = rng.normal(0, 1, (n, 5))          # butunlay tasodifiy belgilar
            ballar.append(cross_val_score(LogisticRegression(max_iter=1000),
                                          X, y, cv=cv).mean())
        ballar = np.array(ballar)
        print(f"  {nomzodlar:>10} {ballar.max():>14.4f} {ballar.mean():>13.4f}")
    print("  (eng yaxshi ball nomzodlar soni bilan o'sadi — bu SHOVQIN)")

    print("\n=== 3. Siljish kattaligi ===")
    for nomzodlar in [10, 50]:
        siljishlar = []
        for _ in range(15):
            ballar = []
            for _ in range(nomzodlar):
                X = rng.normal(0, 1, (n, 5))
                ballar.append(cross_val_score(LogisticRegression(max_iter=1000),
                                              X, y, cv=cv).mean())
            siljishlar.append(max(ballar) - 0.5)
        print(f"  {nomzodlar:>3} nomzod: o'rtacha siljish "
              f"{np.mean(siljishlar):+.4f}")

    print("\n=== 4. Xulosa ===")
    print("  best_score_ — tanlov jarayonining natijasi, siljigan")
    print("  ⭐ Yakuniy baho alohida test yoki ichma-ich CV bilan")


if __name__ == "__main__":
    main()

=== 1. Maqsad belgilarga BOG'LIQ EMAS ===
  300 namuna, haqiqiy aniqlik ~0.50 bo'lishi kerak

=== 2. Nomzodlar soni ortganda 'eng yaxshi' ball ===
   nomzodlar  eng yaxshi CV   o_rtacha CV
           1         0.5067        0.5067
           5         0.5267        0.4780
          20         0.5967        0.5140
         100         0.5767        0.4915
  (eng yaxshi ball nomzodlar soni bilan o'sadi — bu SHOVQIN)

=== 3. Siljish kattaligi ===
   10 nomzod: o'rtacha siljish +0.0547
   50 nomzod: o'rtacha siljish +0.0776

=== 4. Xulosa ===
  best_score_ — tanlov jarayonining natijasi, siljigan
  ⭐ Yakuniy baho alohida test yoki ichma-ich CV bilan

"""Oddiy CV va ichma-ich CV baholarini solishtirish (real numpy/sklearn)."""

import numpy as np
from sklearn.metrics import f1_score
from sklearn.model_selection import (GridSearchCV, StratifiedKFold,
                                     cross_val_score, train_test_split)
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC


def yarat(n: int = 500, p: int = 12, seed: int = 7):
    rng = np.random.default_rng(seed)
    X = rng.normal(0, 1, (n, p))
    z = 0.9 * X[:, 0] - 0.8 * X[:, 1] + 0.7 * X[:, 2] * X[:, 3]
    y = (rng.random(n) < 1 / (1 + np.exp(-z))).astype(int)
    return X, y


def main() -> None:
    X, y = yarat()
    quvur = Pipeline([("sc", StandardScaler()), ("m", SVC())])
    setka = {"m__C": [0.1, 1.0, 10.0, 100.0],
             "m__gamma": [0.001, 0.01, 0.1, 1.0]}

    print("=== 1. Ma'lumot va setka ===")
    print(f"  {len(X)} namuna, {X.shape[1]} belgi")
    print(f"  setka: {len(setka['m__C']) * len(setka['m__gamma'])} ta konfiguratsiya")

    print("\n=== 2. Oddiy GridSearchCV (siljigan baho) ===")
    ichki_cv = StratifiedKFold(5, shuffle=True, random_state=1)
    qidiruv = GridSearchCV(quvur, setka, cv=ichki_cv, scoring="f1").fit(X, y)
    print(f"  eng yaxshi parametrlar: "
          f"{ {k.split('__')[1]: v for k, v in qidiruv.best_params_.items()} }")
    print(f"  best_score_ = {qidiruv.best_score_:.4f}   <- optimistik")

    print("\n=== 3. Ichma-ich CV (siljimagan baho) ===")
    tashqi_cv = StratifiedKFold(5, shuffle=True, random_state=0)
    ichma_ich = cross_val_score(
        GridSearchCV(quvur, setka, cv=ichki_cv, scoring="f1"),
        X, y, cv=tashqi_cv, scoring="f1")
    print(f"  ichma-ich CV = {ichma_ich.mean():.4f} +- {ichma_ich.std():.4f}")
    print(f"  bo'laklar: {np.round(ichma_ich, 4)}")
    print(f"  optimizm siljishi = {qidiruv.best_score_ - ichma_ich.mean():+.4f}")

    print("\n=== 4. Mustaqil test bilan tekshirish ===")
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0,
                                          stratify=y)
    q2 = GridSearchCV(quvur, setka, cv=ichki_cv, scoring="f1").fit(Xtr, ytr)
    test_f1 = f1_score(yte, q2.predict(Xte))
    print(f"  o'quvda best_score_ = {q2.best_score_:.4f}")
    print(f"  test F1             = {test_f1:.4f}")
    print(f"  ichma-ich CV bahosi = {ichma_ich.mean():.4f}")
    print("  ⭐ Ichma-ich CV test natijasiga yaqinroq")


if __name__ == "__main__":
    main()

=== 1. Ma'lumot va setka ===
  500 namuna, 12 belgi
  setka: 16 ta konfiguratsiya

=== 2. Oddiy GridSearchCV (siljigan baho) ===
  eng yaxshi parametrlar: {'C': 10.0, 'gamma': 0.01}
  best_score_ = 0.6771   <- optimistik

=== 3. Ichma-ich CV (siljimagan baho) ===
  ichma-ich CV = 0.6760 +- 0.0527
  bo'laklar: [0.7723 0.6737 0.6452 0.6154 0.6737]
  optimizm siljishi = +0.0011

=== 4. Mustaqil test bilan tekshirish ===
  o'quvda best_score_ = 0.6354
  test F1             = 0.6423
  ichma-ich CV bahosi = 0.6760
  ⭐ Ichma-ich CV test natijasiga yaqinroq

"""Farq haqiqiymi va qaysi modelni tanlash (real numpy/sklearn)."""

import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import RepeatedStratifiedKFold, cross_val_score
from sklearn.neighbors import KNeighborsClassifier
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC


def yarat(n: int = 800, p: int = 10, seed: int = 11):
    rng = np.random.default_rng(seed)
    X = rng.normal(0, 1, (n, p))
    z = 1.0 * X[:, 0] - 0.9 * X[:, 1] + 0.5 * X[:, 2] * X[:, 3]
    y = (rng.random(n) < 1 / (1 + np.exp(-z))).astype(int)
    return X, y


def main() -> None:
    X, y = yarat()
    cv = RepeatedStratifiedKFold(n_splits=5, n_repeats=6, random_state=0)
    sc = lambda m: Pipeline([("sc", StandardScaler()), ("m", m)])

    print("=== 1. Takrorlangan CV (5x6 = 30 ta baho) ===")
    modellar = {
        "LogReg": sc(LogisticRegression(max_iter=3000)),
        "SVM-rbf": sc(SVC(C=10.0, gamma="scale")),
        "KNN(25)": sc(KNeighborsClassifier(25)),
    }
    ballar = {}
    for nom, m in modellar.items():
        b = cross_val_score(m, X, y, cv=cv, scoring="f1")
        ballar[nom] = b
        print(f"  {nom:<9}: {b.mean():.4f} +- {b.std():.4f}  "
              f"(min {b.min():.4f}, maks {b.max():.4f})")

    print("\n=== 2. Juftlashtirilgan farqlar ===")
    for a, b in [("SVM-rbf", "LogReg"), ("SVM-rbf", "KNN(25)"),
                 ("LogReg", "KNN(25)")]:
        d = ballar[a] - ballar[b]
        print(f"  {a} - {b}: o'rtacha {d.mean():+.4f} +- {d.std():.4f}, "
              f"{a} yutgan bo'laklar {(d > 0).mean():.0%}")
    print("  (farq o'z SD sidan kichik bo'lsa — ishonchsiz)")

    print("\n=== 3. C bo'yicha 1-SE qoidasi (LogReg) ===")
    natijalar = {}
    for C in [0.001, 0.01, 0.1, 1.0, 10.0, 100.0]:
        b = cross_val_score(sc(LogisticRegression(C=C, max_iter=3000)), X, y,
                            cv=cv, scoring="f1")
        natijalar[C] = (b.mean(), b.std() / np.sqrt(len(b)))
        print(f"  C = {C:>7}: {b.mean():.4f} +- {natijalar[C][1]:.4f} (SE)")

    eng_C = max(natijalar, key=lambda c: natijalar[c][0])
    eng_ball, eng_se = natijalar[eng_C]
    chegara = eng_ball - eng_se
    nomzodlar = [c for c in natijalar if natijalar[c][0] >= chegara]
    print(f"\n  eng yaxshi C = {eng_C} ({eng_ball:.4f})")
    print(f"  1-SE chegarasi: {chegara:.4f}")
    print(f"  chegaradan yuqori C lar: {sorted(nomzodlar)}")
    print(f"  1-SE tanlovi (eng kuchli regularizatsiya): C = {min(nomzodlar)}")

    print("\n=== 4. Tanlov oqibati ===")
    for C in sorted({min(nomzodlar), eng_C}):
        m = sc(LogisticRegression(C=C, max_iter=3000)).fit(X, y)
        w = m.named_steps["m"].coef_[0]
        print(f"  C = {C:>6}: ||w|| = {np.linalg.norm(w):.3f}, "
              f"CV F1 {natijalar[C][0]:.4f}")
    print("  ⭐ Soddaroq model deyarli teng natija beradi")


if __name__ == "__main__":
    main()

=== 1. Takrorlangan CV (5x6 = 30 ta baho) ===
  LogReg   : 0.7155 +- 0.0367  (min 0.6497, maks 0.7843)
  SVM-rbf  : 0.6687 +- 0.0384  (min 0.5616, maks 0.7485)
  KNN(25)  : 0.7020 +- 0.0418  (min 0.6154, maks 0.7861)

=== 2. Juftlashtirilgan farqlar ===
  SVM-rbf - LogReg: o'rtacha -0.0468 +- 0.0409, SVM-rbf yutgan bo'laklar 17%
  SVM-rbf - KNN(25): o'rtacha -0.0332 +- 0.0425, SVM-rbf yutgan bo'laklar 17%
  LogReg - KNN(25): o'rtacha +0.0135 +- 0.0408, LogReg yutgan bo'laklar 63%
  (farq o'z SD sidan kichik bo'lsa — ishonchsiz)

=== 3. C bo'yicha 1-SE qoidasi (LogReg) ===
  C =   0.001: 0.6993 +- 0.0064 (SE)
  C =    0.01: 0.7154 +- 0.0069 (SE)
  C =     0.1: 0.7153 +- 0.0068 (SE)
  C =     1.0: 0.7155 +- 0.0067 (SE)
  C =    10.0: 0.7155 +- 0.0067 (SE)
  C =   100.0: 0.7155 +- 0.0067 (SE)

  eng yaxshi C = 1.0 0.7155-bob
  1-SE chegarasi: 0.7088
  chegaradan yuqori C lar: [0.01, 0.1, 1.0, 10.0, 100.0]
  1-SE tanlovi (eng kuchli regularizatsiya): C = 0.01

=== 4. Tanlov oqibati ===
  C =   0.01: ||w|| = 0.750, CV F1 0.7154
  C =    1.0: ||w|| = 1.384, CV F1 0.7155
  ⭐ Soddaroq model deyarli teng natija beradi

"""Aniqlik, narx va talablarni birga baholash (real numpy/sklearn)."""

import numpy as np
from sklearn.calibration import CalibratedClassifierCV
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import brier_score_loss, f1_score, roc_auc_score
from sklearn.model_selection import (RepeatedStratifiedKFold, cross_val_score,
                                     train_test_split)
from sklearn.naive_bayes import GaussianNB
from sklearn.neighbors import KNeighborsClassifier
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC


def yarat(n: int = 4000, p: int = 12, seed: int = 21):
    rng = np.random.default_rng(seed)
    X = rng.normal(0, 1, (n, p))
    z = -0.6 + 1.0 * X[:, 0] - 0.9 * X[:, 1] + 0.6 * X[:, 2] * X[:, 3]
    y = (rng.random(n) < 1 / (1 + np.exp(-z))).astype(int)
    return X, y


def main() -> None:
    X, y = yarat()
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0,
                                          stratify=y)
    cv = RepeatedStratifiedKFold(n_splits=5, n_repeats=3, random_state=0)
    sc = lambda m: Pipeline([("sc", StandardScaler()), ("m", m)])

    nomzodlar = {
        "GaussNB": GaussianNB(),
        "LogReg": sc(LogisticRegression(max_iter=3000)),
        "KNN(25)": sc(KNeighborsClassifier(25)),
        "SVM-rbf": sc(CalibratedClassifierCV(SVC(C=10.0, random_state=0),
                                             ensemble=False, cv=5)),
        "O'rmon": RandomForestClassifier(n_estimators=200, min_samples_leaf=5,
                                         random_state=0),
    }

    print("=== 1. Sifat (takrorlangan CV, F1) ===")
    cv_natija = {}
    for nom, m in nomzodlar.items():
        b = cross_val_score(m, Xtr, ytr, cv=cv, scoring="f1")
        cv_natija[nom] = (b.mean(), b.std())
        print(f"  {nom:<9}: {b.mean():.4f} +- {b.std():.4f}")

    eng = max(cv_natija, key=lambda k: cv_natija[k][0])
    chegara = cv_natija[eng][0] - cv_natija[eng][1]
    yaqinlar = [k for k, v in cv_natija.items() if v[0] >= chegara]
    print(f"  eng yaxshi: {eng}; 1 SD ichidagilar: {yaqinlar}")

    print("\n=== 2. Test natijasi va ehtimol sifati ===")
    print(f"  {'model':<9} {'test F1':>9} {'AUC':>8} {'Brier':>9} {'model hajmi':>15}")
    for nom, m in nomzodlar.items():
        m.fit(Xtr, ytr)
        p = m.predict_proba(Xte)[:, 1]
        pred = (p >= 0.5).astype(int)
        if nom == "KNN(25)":
            hajm = f"{Xtr.nbytes // 1024} KB ma'lumot"
        elif nom == "O'rmon":
            hajm = f"{m.n_estimators} daraxt"
        elif nom == "SVM-rbf":
            hajm = "tayanch vektorlar"
        else:
            hajm = "koeffitsiyentlar"
        print(f"  {nom:<9} {f1_score(yte, pred):>9.4f} "
              f"{roc_auc_score(yte, p):>8.4f} {brier_score_loss(yte, p):>9.5f} "
              f"{hajm:>15}")

    print("\n=== 3. Qaror matritsasi (1-5 ball) ===")
    mezonlar = {
        "sifat": {"GaussNB": 2, "LogReg": 4, "KNN(25)": 3, "SVM-rbf": 5,
                  "O'rmon": 5},
        "tezlik": {"GaussNB": 5, "LogReg": 5, "KNN(25)": 2, "SVM-rbf": 2,
                   "O'rmon": 3},
        "ehtimol": {"GaussNB": 1, "LogReg": 5, "KNN(25)": 2, "SVM-rbf": 4,
                    "O'rmon": 3},
        "talqin": {"GaussNB": 3, "LogReg": 5, "KNN(25)": 2, "SVM-rbf": 1,
                   "O'rmon": 3},
        "qollab": {"GaussNB": 5, "LogReg": 5, "KNN(25)": 3, "SVM-rbf": 3,
                   "O'rmon": 4},
    }
    vaznlar = {"sifat": 0.35, "tezlik": 0.2, "ehtimol": 0.2, "talqin": 0.15,
               "qollab": 0.1}
    print(f"  {'model':<9} " + "".join(f"{k:>9}" for k in mezonlar) + f"{'jami':>8}")
    jamilar = {}
    for nom in nomzodlar:
        ballar = [mezonlar[k][nom] for k in mezonlar]
        jami = sum(mezonlar[k][nom] * vaznlar[k] for k in mezonlar)
        jamilar[nom] = jami
        print(f"  {nom:<9} " + "".join(f"{b:>9}" for b in ballar) + f"{jami:>8.2f}")

    print("\n=== 4. Yakuniy tavsiya ===")
    tanlov = max(jamilar, key=jamilar.get)
    print(f"  qaror matritsasi bo'yicha: {tanlov} ({jamilar[tanlov]:.2f})")
    print(f"  faqat sifat bo'yicha: {eng} ({cv_natija[eng][0]:.4f})")
    print("  vaznlar biznes talablaridan kelib chiqadi va hujjatlashtiriladi")
    print("  ⭐ Yakuniy tanlov — aniqlik emas, mezonlar yig'indisi")


if __name__ == "__main__":
    main()

=== 1. Sifat (takrorlangan CV, F1) ===
  GaussNB  : 0.6000 +- 0.0227
  LogReg   : 0.6226 +- 0.0209
  KNN(25)  : 0.5351 +- 0.0288
  SVM-rbf  : 0.5432 +- 0.0239
  O'rmon   : 0.6105 +- 0.0234
  eng yaxshi: LogReg; 1 SD ichidagilar: ['LogReg', "O'rmon"]

=== 2. Test natijasi va ehtimol sifati ===
  model       test F1      AUC     Brier     model hajmi
  GaussNB      0.6075   0.7717   0.18915 koeffitsiyentlar
  LogReg       0.6131   0.7743   0.18766 koeffitsiyentlar
  KNN(25)      0.5366   0.7589   0.19701 262 KB ma'lumot
  SVM-rbf      0.5708   0.7425   0.19937 tayanch vektorlar
  O'rmon       0.6126   0.7766   0.18871      200 daraxt

=== 3. Qaror matritsasi (1-5 ball) ===
  model         sifat   tezlik  ehtimol   talqin   qollab    jami
  GaussNB           2        5        1        3        5    2.85
  LogReg            4        5        5        5        5    4.65
  KNN(25)           3        2        2        2        3    2.45
  SVM-rbf           5        2        4        1        3    3.40
  O'rmon            5        3        3        3        4    3.80

=== 4. Yakuniy tavsiya ===
  qaror matritsasi bo'yicha: LogReg 4.65-bob
  faqat sifat bo'yicha: LogReg 0.6226-bob
  vaznlar biznes talablaridan kelib chiqadi va hujjatlashtiriladi
  ⭐ Yakuniy tanlov — aniqlik emas, mezonlar yig'indisi

print(f"model F1 = {qidiruv.best_score_:.3f}")                    # ⚠️
print(f"test F1 = {f1_score(y_te, qidiruv.predict(X_te)):.3f}")   # ✅

# test F1 bo'yicha eng yaxshi modelni tanlash                     # ⚠️
# validatsiya/CV da tanlab, testda faqat baholash                 # ✅

cross_val_score(m1, X, y, cv=5); cross_val_score(m2, X, y, cv=3)  # ⚠️
cv = StratifiedKFold(5, shuffle=True, random_state=0)             # ✅

print(f"SVM {a:.4f} vs LogReg {b:.4f}")                           # ⚠️
print(f"farq {(a - b):+.4f} +- {farq.std():.4f}")                 # ✅

cross_val_score(m, X, y, cv=5)                                    # ⚠️
RepeatedStratifiedKFold(n_splits=5, n_repeats=5, random_state=0)  # ✅

# eng yuqori balli C = 1000 ni tanlash                            # ⚠️
# 1 SE ichidagi eng kuchli regularizatsiyani tanlash              # ✅

# eng aniq modelni tanlash (50x sekinroq)                         # ⚠️
# qaror matritsasi: sifat + tezlik + talqin + qo'llab-quvvatlash  # ✅

1.  # tanlov optimizmi nima?
2.  # siljish qachon katta?
3.  # best_score_ nima?
4.  # ichma-ich CV tuzilishi?
5.  # ichma-ich CV nimani beradi?
6.  # narxi qancha?
7.  # takrorlangan CV nega kerak?
8.  # CV ballari mustaqilmi?
9.  # juftlashtirish nima beradi?
10. # 1-SE qoidasi?
11. # farq qachon ishonchsiz?
12. # yakuniy mezonlar?

1.  print(f"model sifati: {gs.best_score_:.3f}")

2.  # test bo'yicha eng yaxshi nomzodni tanlash

3.  cross_val_score(m1, X, y, cv=5); cross_val_score(m2, X, y, cv=10)

4.  print(f"A {a:.4f} B {b:.4f}")   # SD yo'q

5.  # eng yuqori balli C=1000 ni tanlash (C=1 dan 0.002 yaxshi)