18.8-dars: O'rganish egri chiziqlari

learning_curve(model, X, y, train_sizes=[...], cv=cv, scoring=...)

Har o'quv hajmi uchun:
  o'quv to'plamidan shu qadar qator olinadi
  model o'rgatiladi
  O'QUV ballari va VALIDATSIYA ballari qaytariladi

NATIJA: ikki egri chiziq
  o'quv balli      - odatda pastga tushadi (ko'p ma'lumot = qiyinroq)
  validatsiya balli - odatda yuqoriga chiqadi va tekislanadi

MUHIM: validatsiya to'plami HAR DOIM to'liq (CV dan)
       faqat o'quv hajmi o'zgaradi

1. YUQORI BIAS (underfitting)
   o'quv balli PAST, validatsiya balli unga YAQIN
   ikkalasi ham tez tekislanadi
   -> ko'proq ma'lumot YORDAM BERMAYDI
   -> murakkabroq model, yangi belgilar kerak

2. YUQORI DISPERSIYA (overfitting)
   o'quv balli YUQORI, validatsiya balli PAST
   orasida katta bo'shliq, bo'shliq sekin kamayadi
   -> ko'proq ma'lumot YORDAM BERADI
   -> yoki regularizatsiya, soddaroq model

3. YAXSHI MUVOZANAT
   ikki egri chiziq yaqin va yuqori darajada tekislangan
   -> model ma'lumotdan olishi mumkin bo'lgan narsani olgan
   -> yangi BELGILAR yoki boshqa ma'lumot manbai kerak

validation_curve(model, X, y, param_name=..., param_range=[...],
                 cv=cv, scoring=...)

Bitta giperparametr bo'yicha o'quv va validatsiya ballari.

O'QISH:
  chap tomon (sodda model):  ikkalasi past -> underfitting
  o'rta:                     validatsiya eng yuqori -> optimum
  o'ng tomon (murakkab):     o'quv yuqori, validatsiya tushadi
                             -> overfitting

DIQQAT: bu 1D kesim - boshqa parametrlar qat'iy
        bog'liq parametrlar bo'lsa yanglishtirishi mumkin

QARORNI SHUNDAY QABUL QILING:

1. Oxirgi uch nuqtaga qarang (eng katta o'quv hajmlari)
2. Validatsiya balli hali o'sayaptimi?
   ha  -> ma'lumot qo'shish FOYDALI
   yo'q -> tekislangan, ma'lumot yordam bermaydi

3. Qancha foyda? oxirgi ikki nuqta farqini ekstrapolyatsiya qiling
   (odatda log-chiziqli: hajm 2x -> o'sish deyarli bir xil)

4. Narxni solishtiring:
   "10 000 qator qo'shish +0.004 beradi, narxi 5000 dollar"
   "yangi belgi guruhi +0.02 berishi mumkin, narxi 1 hafta"

                       o'quv ball   valid. ball   bo'shliq
  yuqori bias            past          past         kichik
  yuqori dispersiya      yuqori        past         KATTA
  ikkalasi               o'rta         past         o'rta
  yaxshi                 yuqori        yuqori       kichik

DIQQAT: "past" va "yuqori" MUTLAQ emas, VAZIFAGA nisbatan
  shovqinli vazifada 0.75 - yaxshi natija bo'lishi mumkin

CHEGARA (Bayes xatosi):
  hech bir model o'ta olmaydigan daraja
  shovqin (flip_y), belgilar yetishmasligi

EGRI CHIZIQ                  QAROR
  bo'shliq katta, valid o'sib   -> ko'proq ma'lumot
  bo'shliq katta, valid tekis   -> regularizatsiya/soddalashtirish
  bo'shliq kichik, daraja past  -> murakkabroq model, YANGI BELGILAR
  ikkalasi yuqori, tekis        -> tayyor; belgilar/ma'lumot manbai

QO'SHIMCHA TEKSHIRUV:
  - eng kichik hajmda ham natija yaxshi bo'lsa -> leakage?
  - egri chiziq tebransa -> CV shovqinli, takrorlang
  - o'quv balli 1.0 bo'lsa -> to'liq yodlab olgan

import numpy as np
from sklearn.model_selection import (StratifiedKFold, learning_curve,
                                     validation_curve)

hajmlar, oquv, valid = learning_curve(
    quvur, X, y, train_sizes=np.logspace(-1.3, 0, 8),
    cv=StratifiedKFold(5, shuffle=True, random_state=0),
    scoring="roc_auc", n_jobs=1)
print(oquv.mean(axis=1).round(3), valid.mean(axis=1).round(3))

oquv, valid = validation_curve(
    quvur, X, y, param_name="m__max_leaf_nodes",
    param_range=[4, 8, 16, 32, 64], cv=cv, scoring="roc_auc")

bo_shliq = oquv.mean(axis=1) - valid.mean(axis=1)
QOIDA: bo'shliq -> ma'lumot · daraja -> model/belgilar ·
       log train_sizes · Pipeline ichida

Yuqori bias:      ikkalasi past, bo'shliq kichik -> model/belgilar
Yuqori dispersiya: bo'shliq katta -> ma'lumot/regularizatsiya
Tekislangan:      ma'lumot qo'shish foydasiz
validation_curve: bitta parametr bo'yicha 1D kesim

"""Bias, dispersiya va muvozanat egri chiziqlari (real numpy/sklearn)."""

import numpy as np
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold, learning_curve
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler


def egri(model, X, y, cv):
    hajmlar, oquv, valid = learning_curve(
        model, X, y, train_sizes=np.logspace(-1.5, 0, 7),
        cv=cv, scoring="roc_auc", n_jobs=1, shuffle=True, random_state=0)
    return hajmlar, oquv.mean(axis=1), valid.mean(axis=1)


def main() -> None:
    # nochiziqli vazifa: chiziqli model uchun yuqori bias
    rng = np.random.default_rng(0)
    n = 4000
    X = rng.normal(0, 1, (n, 8))
    kuch = (1.6 * (X[:, 0] * X[:, 1] > 0) + 1.4 * (X[:, 2] ** 2 - 1)
            + 0.8 * X[:, 3] - 1.0)
    y = (rng.random(n) < 1 / (1 + np.exp(-kuch))).astype(int)
    cv = StratifiedKFold(4, shuffle=True, random_state=0)

    modellar = {
        "chiziqli (yuqori bias)": make_pipeline(
            StandardScaler(), LogisticRegression(max_iter=2000)),
        "RF cheklanmagan (dispersiya)": RandomForestClassifier(
            n_estimators=150, min_samples_leaf=1, random_state=0, n_jobs=1),
        "RF sozlangan (muvozanat)": RandomForestClassifier(
            n_estimators=150, min_samples_leaf=15, max_features=0.5,
            random_state=0, n_jobs=1),
    }

    for nom, m in modellar.items():
        hajmlar, oquv, valid = egri(m, X, y, cv)
        print(f"=== {nom} ===")
        print(f"  {'hajm':>7} {'o_quv':>8} {'valid':>8} {'bo_shliq':>10}")
        for h, o, v in zip(hajmlar, oquv, valid):
            print(f"  {int(h):>7} {o:>8.4f} {v:>8.4f} {o - v:>10.4f}")
        print(f"  yakuniy bo'shliq: {oquv[-1] - valid[-1]:.4f}")
        print(f"  oxirgi ikki nuqtada valid o'sishi: "
              f"{valid[-1] - valid[-2]:+.4f}")
        print()

    print("=== Talqin ===")
    print(f"  {'model':<30} {'daraja':>8} {'bo_shliq':>10} {'xulosa':<28}")
    for nom, m in modellar.items():
        _, oquv, valid = egri(m, X, y, cv)
        bosh = oquv[-1] - valid[-1]
        if bosh > 0.05:
            xulosa = "ko'proq ma'lumot / reg."
        elif valid[-1] < 0.72:
            xulosa = "murakkabroq model/belgilar"
        else:
            xulosa = "muvozanat"
        print(f"  {nom:<30} {valid[-1]:>8.4f} {bosh:>10.4f} {xulosa:<28}")
    print("  ⭐ Bo'shliq va daraja - ikki alohida savol")


if __name__ == "__main__":
    main()

=== chiziqli (yuqori bias) ===
     hajm    o_quv    valid   bo_shliq
       94   0.6929   0.5816     0.1114
      168   0.6684   0.6024     0.0661
      300   0.6754   0.6161     0.0594
      533   0.6619   0.6217     0.0402
      948   0.6532   0.6253     0.0278
     1687   0.6449   0.6331     0.0118
     3000   0.6391   0.6355     0.0036
  yakuniy bo'shliq: 0.0036
  oxirgi ikki nuqtada valid o'sishi: +0.0024

=== RF cheklanmagan (dispersiya) ===
     hajm    o_quv    valid   bo_shliq
       94   1.0000   0.7475     0.2525
      168   1.0000   0.7749     0.2251
      300   1.0000   0.7886     0.2114
      533   1.0000   0.7971     0.2029
      948   1.0000   0.8032     0.1968
     1687   1.0000   0.8147     0.1853
     3000   1.0000   0.8240     0.1760
  yakuniy bo'shliq: 0.1760
  oxirgi ikki nuqtada valid o'sishi: +0.0093

=== RF sozlangan (muvozanat) ===
     hajm    o_quv    valid   bo_shliq
       94   0.8545   0.6743     0.1801
      168   0.9050   0.7716     0.1335
      300   0.9221   0.7978     0.1243
      533   0.9163   0.8042     0.1121
      948   0.9199   0.8139     0.1060
     1687   0.9230   0.8197     0.1033
     3000   0.9251   0.8280     0.0970
  yakuniy bo'shliq: 0.0970
  oxirgi ikki nuqtada valid o'sishi: +0.0083

=== Talqin ===
  model                            daraja   bo_shliq xulosa
  chiziqli (yuqori bias)           0.6355     0.0036 murakkabroq model/belgilar
  RF cheklanmagan (dispersiya)     0.8240     0.1760 ko'proq ma'lumot / reg.
  RF sozlangan (muvozanat)         0.8280     0.0970 ko'proq ma'lumot / reg.
  ⭐ Bo'shliq va daraja - ikki alohida savol

"""Egri chiziqdan ekstrapolyatsiya (real numpy/sklearn)."""

import numpy as np
from sklearn.datasets import make_classification
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.model_selection import StratifiedKFold, learning_curve


def main() -> None:
    # ikki vazifa: biri shovqinli (chegara past), biri toza
    vazifalar = {
        "shovqinli (flip_y=0.30)": make_classification(
            n_samples=24000, n_features=20, n_informative=6, n_redundant=4,
            flip_y=0.30, class_sep=0.9, random_state=0),
        "toza (flip_y=0.02)": make_classification(
            n_samples=24000, n_features=20, n_informative=6, n_redundant=4,
            flip_y=0.02, class_sep=0.9, random_state=0),
    }
    cv = StratifiedKFold(3, shuffle=True, random_state=0)

    for nom, (X, y) in vazifalar.items():
        model = HistGradientBoostingClassifier(learning_rate=0.1,
                                               max_iter=200,
                                               early_stopping=False,
                                               random_state=0)
        hajmlar, oquv, valid = learning_curve(
            model, X, y, train_sizes=np.logspace(-2.4, 0, 7), cv=cv,
            scoring="roc_auc", n_jobs=1, shuffle=True, random_state=0)
        o, v = oquv.mean(axis=1), valid.mean(axis=1)
        print(f"=== {nom} ===")
        print(f"  {'hajm':>7} {'o_quv':>8} {'valid':>8} "
              f"{'oldingidan o_sish':>19}")
        oldingi = None
        for h, oo, vv in zip(hajmlar, o, v):
            osish = "-" if oldingi is None else f"{vv - oldingi:+.4f}"
            print(f"  {int(h):>7} {oo:>8.4f} {vv:>8.4f} {osish:>19}")
            oldingi = vv
        # log-chiziqli ekstrapolyatsiya: hajm 2x -> so'nggi o'sish
        songgi = v[-1] - v[-2]
        nisbat = hajmlar[-1] / hajmlar[-2]
        bir_ikki = songgi * np.log(2) / np.log(nisbat)
        print(f"  hajmni 2x oshirsak taxminan: {bir_ikki:+.4f}")
        print(f"  hajmni 10x oshirsak taxminan: "
              f"{bir_ikki * np.log(10) / np.log(2):+.4f}")
        print()

    print("=== Qaror ===")
    print(f"  {'vazifa':<26} {'yakuniy':>9} {'2x foyda':>10} {'qaror':<26}")
    for nom, (X, y) in vazifalar.items():
        model = HistGradientBoostingClassifier(learning_rate=0.1,
                                               max_iter=200,
                                               early_stopping=False,
                                               random_state=0)
        hajmlar, oquv, valid = learning_curve(
            model, X, y, train_sizes=np.logspace(-2.4, 0, 7), cv=cv,
            scoring="roc_auc", n_jobs=1, shuffle=True, random_state=0)
        v = valid.mean(axis=1)
        songgi = v[-1] - v[-2]
        bir_ikki = songgi * np.log(2) / np.log(hajmlar[-1] / hajmlar[-2])
        qaror = ("ma'lumot foydali" if bir_ikki > 0.005
                 else "ma'lumot foydasiz")
        print(f"  {nom:<26} {v[-1]:>9.4f} {bir_ikki:>+10.4f} {qaror:<26}")
    print("  ⭐ Ma'lumot yig'ishdan oldin egri chiziqni chizing")


if __name__ == "__main__":
    main()

=== shovqinli (flip_y=0.30) ===
     hajm    o_quv    valid   oldingidan o_sish
       63   1.0000   0.6705                   -
      160   1.0000   0.7083             +0.0378
      401   1.0000   0.7537             +0.0455
     1009   1.0000   0.7746             +0.0209
     2535   1.0000   0.7964             +0.0218
     6369   0.9983   0.8097             +0.0133
    16000   0.9588   0.8226             +0.0128
  hajmni 2x oshirsak taxminan: +0.0096
  hajmni 10x oshirsak taxminan: +0.0320

=== toza (flip_y=0.02) ===
     hajm    o_quv    valid   oldingidan o_sish
       63   1.0000   0.7686                   -
      160   1.0000   0.8640             +0.0954
      401   1.0000   0.9062             +0.0422
     1009   1.0000   0.9373             +0.0311
     2535   1.0000   0.9581             +0.0208
     6369   1.0000   0.9668             +0.0087
    16000   0.9984   0.9724             +0.0055
  hajmni 2x oshirsak taxminan: +0.0042
  hajmni 10x oshirsak taxminan: +0.0138

=== Qaror ===
  vazifa                       yakuniy   2x foyda qaror
  shovqinli (flip_y=0.30)       0.8226    +0.0096 ma'lumot foydali
  toza (flip_y=0.02)            0.9724    +0.0042 ma'lumot foydasiz
  ⭐ Ma'lumot yig'ishdan oldin egri chiziqni chizing

"""Bitta parametr bo'yicha underfitting va overfitting (real sklearn)."""

import numpy as np
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold, validation_curve
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler


def main() -> None:
    X, y = make_classification(n_samples=1400, n_features=40,
                               n_informative=8, n_redundant=10, flip_y=0.2,
                               class_sep=0.8, random_state=0)
    cv = StratifiedKFold(4, shuffle=True, random_state=0)

    print("=== 1. LogisticRegression: C ===")
    quvur = Pipeline([("s", StandardScaler()),
                      ("m", LogisticRegression(max_iter=3000))])
    C_lar = np.logspace(-4, 3, 8)
    oquv, valid = validation_curve(quvur, X, y, param_name="m__C",
                                   param_range=C_lar, cv=cv,
                                   scoring="roc_auc", n_jobs=1)
    o, v = oquv.mean(axis=1), valid.mean(axis=1)
    print(f"  {'C':>10} {'o_quv':>8} {'valid':>8} {'bo_shliq':>10} "
          f"{'hudud':<14}")
    for C, oo, vv in zip(C_lar, o, v):
        hudud = ("underfitting" if oo - vv < 0.01 and vv < v.max() - 0.005
                 else "overfitting" if oo - vv > 0.05 else "optimum")
        print(f"  {C:>10.4g} {oo:>8.4f} {vv:>8.4f} {oo - vv:>10.4f} "
              f"{hudud:<14}")
    print(f"  eng yaxshi C = {C_lar[int(np.argmax(v))]:.4g}")

    print("\n=== 2. RandomForest: min_samples_leaf ===")
    barglar = [1, 2, 5, 10, 25, 60, 150]
    oquv, valid = validation_curve(
        RandomForestClassifier(n_estimators=80, random_state=0, n_jobs=1),
        X, y, param_name="min_samples_leaf", param_range=barglar, cv=cv,
        scoring="roc_auc", n_jobs=1)
    o, v = oquv.mean(axis=1), valid.mean(axis=1)
    print(f"  {'min_leaf':>10} {'o_quv':>8} {'valid':>8} {'bo_shliq':>10}")
    for b, oo, vv in zip(barglar, o, v):
        print(f"  {b:>10} {oo:>8.4f} {vv:>8.4f} {oo - vv:>10.4f}")
    print(f"  eng yaxshi min_samples_leaf = {barglar[int(np.argmax(v))]}")
    print(f"  min_leaf=1 da bo'shliq: {o[0] - v[0]:.4f} (yodlab olish)")

    print("\n=== 3. 1D kesim cheklovi ===")
    # max_features ni o'zgartirsak, optimal min_samples_leaf o'zgaradimi
    print(f"  {'max_features':>13} {'eng yaxshi min_leaf':>21} "
          f"{'eng yaxshi ball':>17}")
    for mf in [0.2, 0.5, 1.0]:
        _, valid2 = validation_curve(
            RandomForestClassifier(n_estimators=80, max_features=mf,
                                   random_state=0, n_jobs=1),
            X, y, param_name="min_samples_leaf", param_range=barglar,
            cv=cv, scoring="roc_auc", n_jobs=1)
        v2 = valid2.mean(axis=1)
        print(f"  {mf:>13} {barglar[int(np.argmax(v2))]:>21} "
              f"{v2.max():>17.4f}")
    print("  optimal qiymat boshqa parametrga bog'liq")

    print("\n=== 4. Shaklni o'qish ===")
    print("  chap tomon (sodda): ikkalasi past -> underfitting")
    print("  o'rta: valid eng yuqori -> optimum")
    print("  o'ng tomon (murakkab): o'quv o'sadi, valid tushadi")
    print("  ⭐ validation_curve optimal qiymat emas, SHAKL beradi")


if __name__ == "__main__":
    main()

=== 1. LogisticRegression: C ===
           C    o_quv    valid   bo_shliq hudud
      0.0001   0.8648   0.8577     0.0070 underfitting
       0.001   0.8751   0.8660     0.0091 underfitting
        0.01   0.8849   0.8717     0.0132 optimum
         0.1   0.8864   0.8682     0.0181 optimum
           1   0.8863   0.8674     0.0190 optimum
          10   0.8864   0.8674     0.0190 optimum
         100   0.8864   0.8674     0.0190 optimum
        1000   0.8864   0.8674     0.0190 optimum
  eng yaxshi C = 0.01

=== 2. RandomForest: min_samples_leaf ===
    min_leaf    o_quv    valid   bo_shliq
           1   1.0000   0.8851     0.1149
           2   0.9998   0.8871     0.1127
           5   0.9898   0.8902     0.0996
          10   0.9695   0.8867     0.0828
          25   0.9340   0.8814     0.0526
          60   0.9035   0.8709     0.0326
         150   0.8730   0.8541     0.0189
  eng yaxshi min_samples_leaf = 5
  min_leaf=1 da bo'shliq: 0.1149 (yodlab olish)

=== 3. 1D kesim cheklovi ===
   max_features   eng yaxshi min_leaf   eng yaxshi ball
            0.2                     1            0.8876
            0.5                     5            0.8856
            1.0                    10            0.8821
  optimal qiymat boshqa parametrga bog'liq

=== 4. Shaklni o'qish ===
  chap tomon (sodda): ikkalasi past -> underfitting
  o'rta: valid eng yuqori -> optimum
  o'ng tomon (murakkab): o'quv o'sadi, valid tushadi
  ⭐ validation_curve optimal qiymat emas, SHAKL beradi

"""To'rt holatni diagnostika qilish (real numpy/sklearn)."""

import numpy as np
from sklearn.datasets import make_classification
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold, learning_curve
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler


def tahlil(nom: str, model, X, y, cv) -> dict:
    hajmlar, oquv, valid = learning_curve(
        model, X, y, train_sizes=np.logspace(-1.7, 0, 6), cv=cv,
        scoring="roc_auc", n_jobs=1, shuffle=True, random_state=0)
    o, v = oquv.mean(axis=1), valid.mean(axis=1)
    bosh = float(o[-1] - v[-1])
    osish = float(v[-1] - v[-2])
    if bosh > 0.06 and osish > 0.004:
        qaror = "ko'proq ma'lumot"
    elif bosh > 0.06:
        qaror = "regularizatsiya"
    elif v[-1] < 0.75:
        qaror = "murakkabroq model"
    else:
        qaror = "belgilar / manba"
    return {"nom": nom, "oquv": float(o[-1]), "valid": float(v[-1]),
            "boshliq": bosh, "osish": osish, "qaror": qaror}


def main() -> None:
    rng = np.random.default_rng(0)
    cv = StratifiedKFold(3, shuffle=True, random_state=0)

    # A: nochiziqli vazifa, chiziqli model
    n = 3000
    Xa = rng.normal(0, 1, (n, 8))
    kuch = 1.8 * (Xa[:, 0] * Xa[:, 1] > 0) + 1.5 * (Xa[:, 2] ** 2 - 1) - 0.5
    ya = (rng.random(n) < 1 / (1 + np.exp(-kuch))).astype(int)

    # B: kichik ma'lumot, murakkab model
    Xb, yb = make_classification(n_samples=600, n_features=40,
                                 n_informative=10, n_redundant=10,
                                 flip_y=0.12, class_sep=0.9, random_state=1)

    # C: katta toza ma'lumot, mos model
    Xc, yc = make_classification(n_samples=8000, n_features=20,
                                 n_informative=8, n_redundant=4,
                                 flip_y=0.05, class_sep=1.0, random_state=2)

    # D: shovqin chegarasi
    Xd, yd = make_classification(n_samples=8000, n_features=20,
                                 n_informative=6, n_redundant=4,
                                 flip_y=0.35, class_sep=0.9, random_state=3)

    holatlar = [
        tahlil("A: nochiziqli + chiziqli model",
               make_pipeline(StandardScaler(),
                             LogisticRegression(max_iter=2000)), Xa, ya, cv),
        tahlil("B: kichik ma'lumot + murakkab",
               HistGradientBoostingClassifier(max_leaf_nodes=63,
                                              min_samples_leaf=2,
                                              max_iter=300,
                                              early_stopping=False,
                                              random_state=0), Xb, yb, cv),
        tahlil("C: katta toza ma'lumot",
               HistGradientBoostingClassifier(max_iter=200,
                                              early_stopping=False,
                                              random_state=0), Xc, yc, cv),
        tahlil("D: shovqin chegarasi",
               HistGradientBoostingClassifier(max_iter=200,
                                              early_stopping=False,
                                              random_state=0), Xd, yd, cv),
    ]

    print("=== 1. Diagnostika jadvali ===")
    print(f"  {'holat':<34} {'o_quv':>8} {'valid':>8} {'bo_shliq':>10} "
          f"{'o_sish':>9}")
    for h in holatlar:
        print(f"  {h['nom']:<34} {h['oquv']:>8.4f} {h['valid']:>8.4f} "
              f"{h['boshliq']:>10.4f} {h['osish']:>+9.4f}")

    print("\n=== 2. Qarorlar ===")
    print(f"  {'holat':<34} {'qaror':<22}")
    for h in holatlar:
        print(f"  {h['nom']:<34} {h['qaror']:<22}")

    print("\n=== 3. A holatini tuzatish: murakkabroq model ===")
    oldin = tahlil("chiziqli",
                   make_pipeline(StandardScaler(),
                                 LogisticRegression(max_iter=2000)),
                   Xa, ya, cv)
    keyin = tahlil("boosting",
                   HistGradientBoostingClassifier(max_iter=200,
                                                  early_stopping=False,
                                                  random_state=0),
                   Xa, ya, cv)
    print(f"  chiziqli:  valid {oldin['valid']:.4f}, "
          f"bo'shliq {oldin['boshliq']:.4f}")
    print(f"  boosting:  valid {keyin['valid']:.4f}, "
          f"bo'shliq {keyin['boshliq']:.4f}")
    print(f"  o'sish: {keyin['valid'] - oldin['valid']:+.4f}")

    print("\n=== 4. B holatini tuzatish: regularizatsiya ===")
    oldin = tahlil("cheklanmagan",
                   HistGradientBoostingClassifier(max_leaf_nodes=63,
                                                  min_samples_leaf=2,
                                                  max_iter=300,
                                                  early_stopping=False,
                                                  random_state=0),
                   Xb, yb, cv)
    keyin = tahlil("cheklangan",
                   HistGradientBoostingClassifier(max_leaf_nodes=8,
                                                  min_samples_leaf=20,
                                                  l2_regularization=2.0,
                                                  max_iter=300,
                                                  early_stopping=False,
                                                  random_state=0),
                   Xb, yb, cv)
    print(f"  cheklanmagan: valid {oldin['valid']:.4f}, "
          f"bo'shliq {oldin['boshliq']:.4f}")
    print(f"  cheklangan:   valid {keyin['valid']:.4f}, "
          f"bo'shliq {keyin['boshliq']:.4f}")
    print(f"  bo'shliq kamayishi: "
          f"{oldin['boshliq'] - keyin['boshliq']:+.4f}")
    print("  ⭐ Egri chiziq keyingi qadamni aniq ko'rsatadi")


if __name__ == "__main__":
    main()

=== 1. Diagnostika jadvali ===
  holat                                 o_quv    valid   bo_shliq    o_sish
  A: nochiziqli + chiziqli model       0.5318   0.5008     0.0310   -0.0073
  B: kichik ma'lumot + murakkab        1.0000   0.8634     0.1366   +0.1231
  C: katta toza ma'lumot               1.0000   0.9694     0.0306   +0.0013
  D: shovqin chegarasi                 0.9995   0.7792     0.2204   +0.0134

=== 2. Qarorlar ===
  holat                              qaror
  A: nochiziqli + chiziqli model     murakkabroq model
  B: kichik ma'lumot + murakkab      ko'proq ma'lumot
  C: katta toza ma'lumot             belgilar / manba
  D: shovqin chegarasi               ko'proq ma'lumot

=== 3. A holatini tuzatish: murakkabroq model ===
  chiziqli:  valid 0.5008, bo'shliq 0.0310
  boosting:  valid 0.7816, bo'shliq 0.2184
  o'sish: +0.2807

=== 4. B holatini tuzatish: regularizatsiya ===
  cheklanmagan: valid 0.8634, bo'shliq 0.1366
  cheklangan:   valid 0.8748, bo'shliq 0.1252
  bo'shliq kamayishi: +0.0114
  ⭐ Egri chiziq keyingi qadamni aniq ko'rsatadi

train_sizes=np.linspace(0.1, 1.0, 5)                             # ⚠️
train_sizes=np.logspace(-1.5, 0, 7)                              # ✅

learning_curve(m, X, y, cv=5)          # tartib saqlanadi        # ⚠️
learning_curve(m, X, y, cv=cv, shuffle=True, random_state=0)     # ✅

print(valid.mean(axis=1))                                        # ⚠️
print(oquv.mean(axis=1), valid.mean(axis=1))    # ikkalasi       # ✅

learning_curve(SVC(), X, y, ...)      # masshtablanmagan         # ⚠️
learning_curve(make_pipeline(StandardScaler(), SVC()), X, y, ...) # ✅

learning_curve(m, X, y, cv=ShuffleSplit(1))                      # ⚠️
learning_curve(m, X, y, cv=StratifiedKFold(5, shuffle=True,
                                           random_state=0))      # ✅

# validation_curve dan optimal min_samples_leaf olib,
# max_features ni alohida sozlash                                # ⚠️
RandomizedSearchCV(m, {"min_samples_leaf": ..., "max_features": ...})  # ✅

# "valid tekislandi -> model yomon, boshqasini olaman"           # ⚠️
# "valid tekislandi -> ma'lumot yetarli, endi BELGILAR"          # ✅

1.  # o'rganish egri chizig'i nimani ko'rsatadi?
2.  # bo'shliq nimani o'lchaydi?
3.  # daraja nimani o'lchaydi?
4.  # yuqori bias naqshi?
5.  # yuqori dispersiya naqshi?
6.  # tekislangan egri chiziq nima deydi?
7.  # validation_curve nima beradi?
8.  # 1D kesim cheklovi?
9.  # train_sizes qanday?
10. # o'quv balli 1.0 nima?
11. # vazifa chegarasi nima?
12. # ekstrapolyatsiya qanday?

1.  train_sizes=np.linspace(0.1, 1.0, 5)

2.  learning_curve(m, X, y, cv=5)

3.  print(valid.mean(axis=1))

4.  learning_curve(SVC(), X, y, ...)

5.  learning_curve(m, X, y, cv=ShuffleSplit(1))