15.3-dars: Daraxt beqarorligi

BEQAROR model — o'quv ma'lumotining kichik o'zgarishi natijani KATTA o'zgartiradi

Daraxt: juda beqaror
  ildizdagi bo'linish o'zgarsa -> BUTUN daraxt boshqacha
Chiziqli model: barqaror
  koeffitsiyentlar biroz siljiydi, tuzilma o'zgarmaydi
KNN (katta k): barqaror

Beqarorlik = YUQORI DISPERSIYA (variance, 12.5)

Ikki xil beqarorlik:

1. TUZILMA beqarorligi — ildiz belgisi, bo'linishlar tartibi o'zgaradi
   Talqin uchun halokatli
   Deyarli HAR DOIM mavjud

2. BASHORAT beqarorligi — bir xil namuna uchun har xil javob
   Aniqlik uchun muhim
   Tuzilma o'zgarsa ham bashorat o'xshash bo'lishi MUMKIN

1. IERARXIK tuzilma — ildizdagi xato pastga tarqaladi
2. OCHKO'Z tanlov — teng sifatli bo'linishdan biri tanlanadi (tasodifiy)
3. DISKRET chegara — bitta namuna chegarani siljitadi
4. KORRELYATSIYALI belgilar — qaysi biri tanlanishi deyarli tasodif

Eng kuchli sabab: korrelyatsiyali belgilar + teng sifatli bo'linishlar

1. Bootstrap: B ta namuna, har birida daraxt qurib, taqqoslash
   - ildiz belgisining taqsimoti
   - barglar sonining tarqalishi
   - bashoratlar bo'yicha kelishmovchilik ulushi

2. Bashorat dispersiyasi:
   var(x) = B ta daraxtning x uchun bashoratlari dispersiyasi

3. Kelishuv (agreement): ikki daraxt bir xil javob beradigan namunalar ulushi

4. random_state ni o'zgartirish — faqat teng bo'linishlar ta'siri

Agar B ta mustaqil model dispersiyasi sigma^2 bo'lsa:
  o'rtacha dispersiyasi = sigma^2 / B

Lekin daraxtlar bir xil ma'lumotdan o'qitilsa — MUSTAQIL EMAS:
  var(o'rtacha) = rho * sigma^2 + (1 - rho) * sigma^2 / B
  rho — daraxtlar orasidagi korrelyatsiya

B -> cheksiz:  var -> rho * sigma^2

XULOSA: dispersiyani kamaytirish uchun KORRELYATSIYANI kamaytirish kerak
  bootstrap 15.4-bob + tasodifiy belgi tanlash (15.5)

Chuqur daraxt:  past bias, YUQORI dispersiya  -> bagging yordam beradi
Sayoz daraxt:   yuqori bias, past dispersiya  -> boosting yordam beradi

Bagging 15.4-bob:  dispersiyani kamaytiradi, bias ni deyarli o'zgartirmaydi
Boosting 15.7-bob: bias ni kamaytiradi, dispersiyani oshirishi mumkin

import numpy as np
from sklearn.tree import DecisionTreeClassifier

# bootstrap bilan beqarorlikni o'lchash
rng = np.random.default_rng(0)
ildizlar, bashoratlar = [], []
for _ in range(50):
    idx = rng.integers(0, len(X), len(X))
    d = DecisionTreeClassifier(max_depth=5, random_state=0).fit(X[idx], y[idx])
    ildizlar.append(d.tree_.feature[0])
    bashoratlar.append(d.predict(Xte))
np.bincount(ildizlar)                       # ildiz belgisining taqsimoti
np.mean(np.std(bashoratlar, axis=0))        # bashorat dispersiyasi
QOIDA: ma'lumotni o'zgartir (random_state emas) · korrelyatsiyali belgilarga
       ehtiyot bo'l · beqarorlik -> ansambl

Beqarorlik = yuqori dispersiya; daraxtda juda kuchli
Sabablar: ierarxiya, ochko'z tanlov, korrelyatsiyali belgilar
Tuzilma beqarorligi talqinni, bashorat beqarorligi aniqlikni buzadi
var(o'rtacha) = rho*sigma^2 + (1-rho)*sigma^2/B  -> korrelyatsiyani kamaytir

"""Bootstrap namunalarda daraxt tuzilmasi qanday o'zgaradi (real numpy/sklearn)."""

import numpy as np
from sklearn.tree import DecisionTreeClassifier, export_text


def yarat(seed: int = 12, n: int = 1500):
    """daromad va xarajat - bir yashirin omilning ikki o'lchovi."""
    rng = np.random.default_rng(seed)
    holat = rng.normal(0, 1, n)                     # yashirin moliyaviy holat
    daromad = holat + rng.normal(0, 0.35, n)
    xarajat = holat + rng.normal(0, 0.35, n)
    kechikish = rng.poisson(0.6, n).astype(float)
    yosh = rng.integers(21, 65, n).astype(float)
    kuch = -1.6 * holat + 1.0 * (kechikish >= 1)
    y = (rng.random(n) < 1 / (1 + np.exp(-kuch))).astype(int)
    X = np.column_stack([daromad, xarajat, kechikish, yosh])
    return X, y, ["daromad", "xarajat", "kechikish", "yosh"]


def main() -> None:
    X, y, nomlar = yarat()
    rng = np.random.default_rng(0)

    print("=== 1. Belgilar korrelyatsiyasi ===")
    K = np.corrcoef(X.T)
    print(f"  daromad - xarajat: {K[0, 1]:.4f}")
    print(f"  daromad - kechikish: {K[0, 2]:.4f}")
    print("  (ikkalasi ham bir yashirin omilning o'lchovi)")

    print("\n=== 2. 100 bootstrap namunada ildiz belgisi ===")
    ildizlar, chuqurliklar, barglar = [], [], []
    for _ in range(100):
        idx = rng.integers(0, len(X), len(X))
        d = DecisionTreeClassifier(max_depth=4, min_samples_leaf=20,
                                   random_state=0).fit(X[idx], y[idx])
        ildizlar.append(int(d.tree_.feature[0]))
        chuqurliklar.append(d.get_depth())
        barglar.append(d.get_n_leaves())
    sanoq = np.bincount(ildizlar, minlength=len(nomlar))
    for i, nom in enumerate(nomlar):
        print(f"  {nom:<10}: {sanoq[i]:>3} marta ildizda")
    print("  (tanlov amalda tanga tashlashga yaqin)")

    print("\n=== 3. Daraxt o'lchami tarqalishi ===")
    print(f"  barglar: min {min(barglar)}, mediana "
          f"{int(np.median(barglar))}, max {max(barglar)}")
    print(f"  chuqurlik: min {min(chuqurliklar)}, max {max(chuqurliklar)}")

    print("\n=== 4. Ikki bootstrap daraxtining tuzilmasi ===")
    for nechanchi in [0, 1]:
        r2 = np.random.default_rng(100 + nechanchi)
        idx = r2.integers(0, len(X), len(X))
        d = DecisionTreeClassifier(max_depth=2, min_samples_leaf=50,
                                   random_state=0).fit(X[idx], y[idx])
        print(f"  --- namuna {nechanchi + 1}: ildiz = "
              f"{nomlar[int(d.tree_.feature[0])]} ---")
        for qator in export_text(d, feature_names=nomlar,
                                 decimals=2).splitlines():
            print("    " + qator)
    print("  ⭐ Bir xil taqsimot, boshqa tuzilma")


if __name__ == "__main__":
    main()

=== 1. Belgilar korrelyatsiyasi ===
  daromad - xarajat: 0.8891
  daromad - kechikish: -0.0070
  (ikkalasi ham bir yashirin omilning o'lchovi)

=== 2. 100 bootstrap namunada ildiz belgisi ===
  daromad   :  33 marta ildizda
  xarajat   :  67 marta ildizda
  kechikish :   0 marta ildizda
  yosh      :   0 marta ildizda
  (tanlov amalda tanga tashlashga yaqin)

=== 3. Daraxt o'lchami tarqalishi ===
  barglar: min 11, mediana 15, max 16
  chuqurlik: min 4, max 4

=== 4. Ikki bootstrap daraxtining tuzilmasi ===
  --- namuna 1: ildiz = xarajat ---
    |--- xarajat <= -0.02
    |   |--- xarajat <= -0.72
    |   |   |--- class: 1
    |   |--- xarajat >  -0.72
    |   |   |--- class: 1
    |--- xarajat >  -0.02
    |   |--- daromad <= 1.07
    |   |   |--- class: 0
    |   |--- daromad >  1.07
    |   |   |--- class: 0
  --- namuna 2: ildiz = daromad ---
    |--- daromad <= 0.38
    |   |--- xarajat <= -0.54
    |   |   |--- class: 1
    |   |--- xarajat >  -0.54
    |   |   |--- class: 1
    |--- daromad >  0.38
    |   |--- daromad <= 1.07
    |   |   |--- class: 0
    |   |--- daromad >  1.07
    |   |   |--- class: 0
  ⭐ Bir xil taqsimot, boshqa tuzilma

"""Bir xil namuna uchun har xil javoblar (real numpy/sklearn)."""

import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.tree import DecisionTreeClassifier


def yarat(seed: int = 8, n: int = 2500, shovqin: float = 0.12):
    """Qoidaga asoslangan ma'lumot: daraxt uchun tabiiy vazifa."""
    rng = np.random.default_rng(seed)
    X = rng.normal(0, 1, (n, 8))
    qoida = (((X[:, 0] > 0.4) & (X[:, 1] < 0.0))
             | ((X[:, 2] > 0.0) & (X[:, 3] > 0.5))
             | (X[:, 4] < -1.2))
    y = qoida.astype(int)
    alm = rng.random(n) < shovqin
    y[alm] = 1 - y[alm]
    return X, y


def modellar():
    return {
        "daraxt(to'liq)": DecisionTreeClassifier(random_state=0),
        "daraxt(4)": DecisionTreeClassifier(max_depth=4, random_state=0),
        "LogReg": Pipeline([("sc", StandardScaler()),
                            ("m", LogisticRegression(max_iter=2000))]),
        "KNN(25)": Pipeline([("sc", StandardScaler()),
                             ("m", KNeighborsClassifier(25))]),
    }


def main() -> None:
    X, y = yarat()
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0,
                                          stratify=y)
    rng = np.random.default_rng(1)

    print("=== 1. 40 bootstrapda bashoratlar tarqalishi ===")
    natijalar = {nom: [] for nom in modellar()}
    for _ in range(40):
        idx = rng.integers(0, len(Xtr), len(Xtr))
        for nom, m in modellar().items():
            m.fit(Xtr[idx], ytr[idx])
            natijalar[nom].append(m.predict_proba(Xte)[:, 1])

    print(f"  {'model':<15} {'o_rtacha std':>14} {'aniqlik o_rt':>14} "
          f"{'aniqlik std':>13}")
    for nom, qatorlar in natijalar.items():
        P = np.array(qatorlar)
        aniqliklar = [((p > 0.5).astype(int) == yte).mean() for p in P]
        print(f"  {nom:<15} {P.std(axis=0).mean():>14.4f} "
              f"{np.mean(aniqliklar):>14.4f} {np.std(aniqliklar):>13.4f}")

    print("\n=== 2. Bitta namuna uchun 40 ta bashorat ===")
    for nom in ["daraxt(to'liq)", "LogReg"]:
        P = np.array(natijalar[nom])[:, 0]
        print(f"  {nom:<15}: min {P.min():.3f}, mediana {np.median(P):.3f}, "
              f"max {P.max():.3f}")

    print("\n=== 3. Ikki daraxt qancha namunada kelishmaydi ===")
    P = np.array(natijalar["daraxt(to'liq)"]) > 0.5
    kelishmovchilik = [(P[i] != P[j]).mean()
                       for i in range(10) for j in range(i + 1, 10)]
    Pl = np.array(natijalar["LogReg"]) > 0.5
    kl = [(Pl[i] != Pl[j]).mean() for i in range(10) for j in range(i + 1, 10)]
    print(f"  daraxt: o'rtacha {np.mean(kelishmovchilik):.4f} namunada farq")
    print(f"  LogReg: o'rtacha {np.mean(kl):.4f} namunada farq")

    print("\n=== 4. 40 daraxtni o'rtachalashtirish ===")
    Pd = np.array(natijalar["daraxt(to'liq)"])
    for nechta in [1, 5, 10, 20, 40]:
        ortacha = Pd[:nechta].mean(axis=0)
        print(f"  {nechta:>2} daraxt: aniqlik "
              f"{((ortacha > 0.5).astype(int) == yte).mean():.4f}")
    print("  ⭐ O'rtachalashtirish dispersiyani kamaytiradi 15.4-bob")


if __name__ == "__main__":
    main()

=== 1. 40 bootstrapda bashoratlar tarqalishi ===
  model             o_rtacha std   aniqlik o_rt   aniqlik std
  daraxt(to'liq)          0.2803         0.7730        0.0151
  daraxt(4)               0.0848         0.8396        0.0139
  LogReg                  0.0284         0.6921        0.0074
  KNN(25)                 0.0886         0.7293        0.0094

=== 2. Bitta namuna uchun 40 ta bashorat ===
  daraxt(to'liq) : min 0.000, mediana 1.000, max 1.000
  LogReg         : min 0.798, mediana 0.849, max 0.885

=== 3. Ikki daraxt qancha namunada kelishmaydi ===
  daraxt: o'rtacha 0.2138 namunada farq
  LogReg: o'rtacha 0.0402 namunada farq

=== 4. 40 daraxtni o'rtachalashtirish ===
   1 daraxt: aniqlik 0.7613
   5 daraxt: aniqlik 0.8227
  10 daraxt: aniqlik 0.8387
  20 daraxt: aniqlik 0.8560
  40 daraxt: aniqlik 0.8613
  ⭐ O'rtachalashtirish dispersiyani kamaytiradi (15.4)

"""Korrelyatsiyali belgilar va teng sifatli bo'linishlar (real numpy/sklearn)."""

import numpy as np
from sklearn.tree import DecisionTreeClassifier


def yarat(korrelyatsiya: float, seed: int = 3, n: int = 1500):
    """y faqat `a` ga bog'liq; `b` - uning korrelyatsiyali nusxasi."""
    rng = np.random.default_rng(seed)
    a = rng.normal(0, 1, n)
    b = korrelyatsiya * a + np.sqrt(1 - korrelyatsiya ** 2) * rng.normal(0, 1, n)
    shovqin = rng.normal(0, 1, (n, 3))
    X = np.column_stack([a, b, shovqin])
    y = (rng.random(n) < 1 / (1 + np.exp(-1.8 * a))).astype(int)
    return X, y


def ildiz_taqsimoti(korrelyatsiya: float, n: int = 1500, chuqurlik: int = 3,
                    toplamlar: int = 10, bootstraplar: int = 10):
    """Bir necha o'quv to'plami va ularning bootstrap namunalari bo'yicha."""
    sanoq = np.zeros(5, dtype=int)
    for s in range(toplamlar):
        X, y = yarat(korrelyatsiya, seed=s, n=n)
        rng = np.random.default_rng(s)
        for _ in range(bootstraplar):
            idx = rng.integers(0, len(X), len(X))
            d = DecisionTreeClassifier(max_depth=chuqurlik, min_samples_leaf=30,
                                       random_state=0).fit(X[idx], y[idx])
            sanoq[int(d.tree_.feature[0])] += 1
    return sanoq


def main() -> None:
    print("=== 1. Korrelyatsiya va ildiz tanlovi ===")
    print(f"  {'korr':>6} {'belgi_a':>9} {'belgi_b':>9} {'shovqin':>9} "
          f"{'eng ko_p':>10}")
    for k in [0.0, 0.5, 0.9, 0.99, 0.999]:
        s = ildiz_taqsimoti(k)
        print(f"  {k:>6.3f} {s[0]:>9} {s[1]:>9} {s[2:].sum():>9} "
              f"{s.max() / s.sum():>9.1%}")
    print("  (y faqat `a` ga bog'liq, lekin `b` uni almashtira oladi)")

    print("\n=== 2. random_state ta'siri (ma'lumot o'zgarmaydi) ===")
    X, y = yarat(0.99, seed=0)
    ildizlar = [int(DecisionTreeClassifier(max_depth=3, min_samples_leaf=30,
                                           random_state=s).fit(X, y)
                    .tree_.feature[0]) for s in range(40)]
    s = np.bincount(ildizlar, minlength=5)
    print(f"  40 ta random_state: belgi_a {s[0]}, belgi_b {s[1]}")
    print("  (ma'lumot o'zgarmasa daraxt ham o'zgarmaydi -")
    print("   demak random_state beqarorlikni O'LCHAMAYDI)")

    print("\n=== 3. Ma'lumot hajmi ta'siri (korr = 0.99) ===")
    print(f"  {'n':>6} {'eng ko_p ildiz':>16} {'bashorat std':>14}")
    for n in [200, 800, 3000, 10000]:
        s = ildiz_taqsimoti(0.99, n=n, chuqurlik=4, toplamlar=8, bootstraplar=6)
        X, y = yarat(0.99, seed=0, n=n)
        rng = np.random.default_rng(0)
        P = []
        for _ in range(30):
            idx = rng.integers(0, len(X), len(X))
            d = DecisionTreeClassifier(max_depth=4, min_samples_leaf=20,
                                       random_state=0).fit(X[idx], y[idx])
            P.append(d.predict_proba(X[:200])[:, 1])
        print(f"  {n:>6} {s.max() / s.sum():>15.1%} "
              f"{np.array(P).std(axis=0).mean():>14.4f}")
    print("  (ko'proq ma'lumot kamaytiradi, lekin yo'qotmaydi)")

    print("\n=== 4. Chuqurlik va beqarorlik ===")
    X, y = yarat(0.5, n=1500)
    rng = np.random.default_rng(0)
    print(f"  {'max_depth':>10} {'bashorat std':>14}")
    for chuqurlik in [2, 3, 5, 8, None]:
        P = []
        for _ in range(30):
            idx = rng.integers(0, len(X), len(X))
            d = DecisionTreeClassifier(max_depth=chuqurlik,
                                       random_state=0).fit(X[idx], y[idx])
            P.append(d.predict_proba(X)[:, 1])
        nom = "None" if chuqurlik is None else str(chuqurlik)
        print(f"  {nom:>10} {np.array(P).std(axis=0).mean():>14.4f}")
    print("  ⭐ Chuqurroq daraxt - beqarorroq")


if __name__ == "__main__":
    main()

=== 1. Korrelyatsiya va ildiz tanlovi ===
    korr   belgi_a   belgi_b   shovqin   eng ko_p
   0.000       100         0         0    100.0%
   0.500       100         0         0    100.0%
   0.900       100         0         0    100.0%
   0.990        69        31         0     69.0%
   0.999        44        56         0     56.0%
  (y faqat `a` ga bog'liq, lekin `b` uni almashtira oladi)

=== 2. random_state ta'siri (ma'lumot o'zgarmaydi) ===
  40 ta random_state: belgi_a 40, belgi_b 0
  (ma'lumot o'zgarmasa daraxt ham o'zgarmaydi -
   demak random_state beqarorlikni O'LCHAMAYDI)

=== 3. Ma'lumot hajmi ta'siri (korr = 0.99) ===
       n   eng ko_p ildiz   bashorat std
     200           68.8%         0.1478
     800           72.9%         0.1324
    3000           89.6%         0.0779
   10000           95.8%         0.0488
  (ko'proq ma'lumot kamaytiradi, lekin yo'qotmaydi)

=== 4. Chuqurlik va beqarorlik ===
   max_depth   bashorat std
           2         0.0643
           3         0.0821
           5         0.1313
           8         0.2108
        None         0.2705
  ⭐ Chuqurroq daraxt - beqarorroq

"""var(o'rtacha) = rho*s^2 + (1-rho)*s^2/B ni tekshirish (real numpy/sklearn)."""

import numpy as np
from sklearn.tree import DecisionTreeRegressor


def yarat(seed: int, n: int = 600):
    rng = np.random.default_rng(seed)
    X = rng.uniform(-3, 3, (n, 4))
    f = np.sin(X[:, 0]) + 0.5 * X[:, 1] - 0.3 * X[:, 2] ** 2
    return X, f + rng.normal(0, 1.0, n), f


def toplam(Xte, M: int = 20, B: int = 10, max_features=None, n: int = 600):
    """M ta MUSTAQIL o'quv to'plami, har birida B ta bootstrap daraxti."""
    P = []
    for m in range(M):
        Xtr, ytr, _ = yarat(m + 1, n)
        rng = np.random.default_rng(1000 + m)
        qator = []
        for _ in range(B):
            idx = rng.integers(0, len(Xtr), len(Xtr))
            d = DecisionTreeRegressor(max_features=max_features,
                                      random_state=int(rng.integers(1_000_000)))
            d.fit(Xtr[idx], ytr[idx])
            qator.append(d.predict(Xte))
        P.append(qator)
    return np.array(P)                      # (M, B, test)


def main() -> None:
    Xte, yte, fte = yarat(999, 400)

    print("=== 1. Bitta daraxt va o'rtachalashtirilgan daraxtlar ===")
    P = toplam(Xte, M=20, B=16)
    M, B, _ = P.shape
    bitta = float(((P.reshape(M * B, -1) - fte) ** 2).mean())
    print(f"  bitta daraxt MSE: {bitta:.4f}")
    for b in [1, 2, 4, 8, 16]:
        ansambl = P[:, :b].mean(axis=1)
        print(f"  {b:>2} daraxt o'rtachasi MSE: "
              f"{((ansambl - fte) ** 2).mean():.4f}")

    print("\n=== 2. Dispersiya va korrelyatsiya ===")
    sigma2 = float(P.reshape(M * B, -1).var(axis=0, ddof=1).mean())
    vB = float(P.mean(axis=1).var(axis=0, ddof=1).mean())
    rho = (vB / sigma2 - 1 / B) / (1 - 1 / B)       # formuladan yechilgan
    print(f"  bitta daraxt dispersiyasi (s^2): {sigma2:.4f}")
    print(f"  {B} ta daraxt o'rtachasining dispersiyasi: {vB:.4f}")
    print(f"  daraxtlar korrelyatsiyasi (rho): {rho:.4f}")

    print("\n=== 3. Formula bashorati va haqiqat ===")
    print(f"  {'B':>4} {'formula':>10} {'haqiqiy':>10}")
    for b in [1, 2, 4, 8, 16]:
        formula = rho * sigma2 + (1 - rho) * sigma2 / b
        haqiqiy = float(P[:, :b].mean(axis=1).var(axis=0, ddof=1).mean())
        print(f"  {b:>4} {formula:>10.4f} {haqiqiy:>10.4f}")
    print(f"  chegara qiymati (B -> cheksiz): {rho * sigma2:.4f}")

    print("\n=== 4. Korrelyatsiyani kamaytirish ta'siri ===")
    print(f"  {'max_features':>13} {'rho':>8} {'bias^2':>9} {'dispersiya':>12} "
          f"{'MSE':>9}")
    for mf in [4, 3, 2, 1]:
        P2 = toplam(Xte, M=20, B=10, max_features=mf)
        M2, B2, _ = P2.shape
        ansambl = P2.mean(axis=1)
        s2 = float(P2.reshape(M2 * B2, -1).var(axis=0, ddof=1).mean())
        v2 = float(ansambl.var(axis=0, ddof=1).mean())
        rho2 = (v2 / s2 - 1 / B2) / (1 - 1 / B2)
        bias2 = float(((ansambl.mean(axis=0) - fte) ** 2).mean())
        mse = float(((ansambl - fte) ** 2).mean())
        print(f"  {mf:>13} {rho2:>8.4f} {bias2:>9.4f} {v2:>12.4f} {mse:>9.4f}")
    print("  ⭐ rho kamayadi, lekin bias oshadi - o'rtada optimum bor 15.5-bob")


if __name__ == "__main__":
    main()

=== 1. Bitta daraxt va o'rtachalashtirilgan daraxtlar ===
  bitta daraxt MSE: 1.3409
   1 daraxt o'rtachasi MSE: 1.3242
   2 daraxt o'rtachasi MSE: 0.7830
   4 daraxt o'rtachasi MSE: 0.4822
   8 daraxt o'rtachasi MSE: 0.3507
  16 daraxt o'rtachasi MSE: 0.2773

=== 2. Dispersiya va korrelyatsiya ===
  bitta daraxt dispersiyasi (s^2): 1.3100
  16 ta daraxt o'rtachasining dispersiyasi: 0.2550
  daraxtlar korrelyatsiyasi (rho): 0.1409

=== 3. Formula bashorati va haqiqat ===
     B    formula    haqiqiy
     1     1.3100     1.3033
     2     0.7473     0.7571
     4     0.4660     0.4612
     8     0.3253     0.3281
    16     0.2550     0.2550
  chegara qiymati (B -> cheksiz): 0.1846

=== 4. Korrelyatsiyani kamaytirish ta'siri ===
   max_features      rho    bias^2   dispersiya       MSE
              4   0.1478    0.0373       0.3067    0.3287
              3   0.1312    0.0434       0.2901    0.3190
              2   0.0967    0.0600       0.2670    0.3136
              1   0.0604    0.1358       0.2627    0.3854
  ⭐ rho kamayadi, lekin bias oshadi - o'rtada optimum bor (15.5)

# "ildiz belgi — kechikish" deb qoidaga yozish                     # ⚠️
# bootstrapda ildiz taqsimotini tekshiring                         # ✅

for s in range(10): DecisionTreeClassifier(random_state=s)         # ⚠️
for _ in range(100): idx = rng.integers(0, n, n)   # bootstrap     # ✅

d.feature_importances_                   # korrelyatsiya buzadi    # ⚠️
# avval korrelyatsiya matritsasini ko'ring 15.11-bob                 # ✅

model = DecisionTreeClassifier(max_depth=8).fit(X, y)              # ⚠️
RandomForestClassifier(n_estimators=300)                           # ✅

DecisionTreeClassifier()                 # eng beqaror variant     # ⚠️
DecisionTreeClassifier(max_depth=3)      # talqin uchun barqarorroq # ✅

RandomForestClassifier(n_estimators=10000)   # rho*sigma^2 qoladi  # ⚠️
RandomForestClassifier(n_estimators=500, max_features="sqrt")      # ✅

# "daraxt har safar boshqacha — algoritm buzuq"                    # ⚠️
# bu tuzilmaviy xususiyat; ansambl aynan shundan foydalanadi       # ✅

1.  # beqarorlik = qaysi tushuncha?
2.  # daraxt beqarormi?
3.  # ikki xil beqarorlik?
4.  # asosiy sabab?
5.  # random_state yetarlimi?
6.  # o'rtachalashtirish formulasi?
7.  # B -> cheksiz da nima qoladi?
8.  # nimani kamaytirish kerak?
9.  # bagging nimani kamaytiradi?
10. # boosting nimani kamaytiradi?
11. # chuqurroq daraxt qanday?
12. # ko'p ma'lumot yordam beradimi?

1.  # "ildiz belgi = kechikish" deb qoidaga yozish

2.  for s in range(10): DecisionTreeClassifier(random_state=s)

3.  d.feature_importances_   # korrelyatsiya tekshirilmagan

4.  model = DecisionTreeClassifier(max_depth=12).fit(X, y)

5.  RandomForestClassifier(n_estimators=10000)