18.5-dars: Giperparametrlar

PARAMETR - model MA'LUMOTDAN o'rganadi
  chiziqli regressiya koeffitsiyentlari
  daraxtning bo'linish nuqtalari
  neyron tarmoq og'irliklari

GIPERPARAMETR - siz OLDINDAN belgilaysiz
  alpha, C, max_depth, n_estimators, learning_rate

sklearn da:
  giperparametr -> __init__ argumenti  (model.get_params())
  parametr      -> fit dan keyin, oxirida _ bilan
                   (model.coef_, model.feature_importances_)

CHIZIQLI (Ridge/Lasso/LogisticRegression)
  1. alpha yoki C      <- deyarli yagona muhim parametr
  2. penalty / l1_ratio
  (masshtablash SHART)

KNN
  1. n_neighbors
  2. weights (uniform/distance)
  3. metric

SVM (RBF)
  1. C
  2. gamma
  (ikkalasi BIRGA sozlanadi, masshtablash SHART)

RANDOM FOREST
  1. max_features        <- eng muhimi
  2. min_samples_leaf
  3. n_estimators (ko'proq = yaxshiroq, to'yinadi)

GRADIENT BOOSTING
  1. learning_rate + n_estimators (BOG'LIQ)
  2. max_depth / max_leaf_nodes
  3. min_samples_leaf, subsample, regularizatsiya

LOGARIFMIK (kattalik tartibi muhim bo'lganda):
  alpha, C, gamma, learning_rate, l2_regularization
  np.logspace(-4, 2, 13)   ->  1e-4 ... 100

CHIZIQLI (tabiiy chegarasi bor):
  max_depth (2..12), n_neighbors (1..50),
  min_samples_leaf (1..100), max_features (0.1..1.0)

QOIDA: eng yaxshi qiymat maydonning CHETIDA chiqsa,
       maydonni kengaytiring va qayta qidiring

MASSHTAB:
  C = 0.001, 0.01, 0.1, 1, 10   (to'g'ri)
  C = 1, 2, 3, 4, 5             (odatda foydasiz)

learning_rate va n_estimators (boosting):
  lr ikki barobar kichik -> n_estimators ikki barobar katta
  amaliyot: lr ni qat'iy kichik qilib (0.03-0.05),
            n_estimators ni erta to'xtatish bilan topish

C va gamma (SVM RBF):
  gamma katta + C katta -> kuchli overfitting
  ikkalasini BIRGA setkada qidiring (1D qidiruv ishlamaydi)

max_depth va min_samples_leaf (daraxtlar):
  ikkalasi ham murakkablikni cheklaydi - biri yetarli

alpha va belgilar soni (Ridge):
  ko'proq belgi -> kattaroq alpha kerak

SOZLAMANG - qat'iy qiymat qo'ying:
  random_state      -> takrorlanuvchanlik uchun
  n_jobs            -> tezlik, natijaga ta'sir qilmaydi
  verbose           -> chiqish
  max_iter          -> yetarlicha katta (yaqinlashish uchun)
  tol               -> sukut bo'yicha

SOZLAMANG - domendan keladi:
  class_weight      -> biznes narxidan
  scoring           -> maqsaddan
  threshold         -> qaror narxidan (18.9-dars)

"KO'PROQ YAXSHIROQ" (to'yinadi):
  n_estimators (RF)  -> 300-500 dan keyin foyda yo'q

1. BAZAVIY: sukut parametrlar bilan natija oling
2. MUHIM 2-3 tasini keng logarifmik maydonda qidiring
   (RandomizedSearchCV, 20-40 nomzod)
3. Eng yaxshi atrofida TOR maydonda aniqlashtiring
4. Qolgan parametrlarni bittalab sinab ko'ring
   (yaxshilanish SE dan katta bo'lsa qoldiring)
5. Yakuniy modelni nested CV yoki alohida testda tasdiqlang

BYUDJET: umumiy vaqtning 20% idan ko'pini
         sozlashga sarflamang - belgilar ko'proq beradi

import numpy as np
from scipy.stats import loguniform, randint, uniform

# chiziqli
{"C": loguniform(1e-4, 1e2)}
# KNN
{"n_neighbors": randint(1, 50), "weights": ["uniform", "distance"]}
# SVM RBF (birga!)
{"C": loguniform(1e-2, 1e3), "gamma": loguniform(1e-4, 1e0)}
# Random Forest
{"max_features": uniform(0.1, 0.8), "min_samples_leaf": randint(1, 40)}
# Gradient boosting
{"learning_rate": loguniform(0.01, 0.3), "max_leaf_nodes": randint(5, 60),
 "min_samples_leaf": randint(5, 100), "l2_regularization": loguniform(1e-3, 10)}

model.get_params()                      # barcha giperparametrlar
QOIDA: 2-3 muhim parametr · logarifmik oraliq ·
       bog'liqlarni birga · chetda chiqsa kengaytir

Chiziqli: alpha/C
KNN: n_neighbors, weights
SVM: C va gamma (birga)
RF: max_features, min_samples_leaf
Boosting: learning_rate + n_estimators, chuqurlik
Sozlanmaydi: random_state, n_jobs, max_iter

"""Bitta parametrni o'zgartirib, ta'sirni o'lchash (real numpy/sklearn)."""

import numpy as np
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import StratifiedKFold, cross_val_score


def main() -> None:
    X, y = make_classification(n_samples=1500, n_features=30,
                               n_informative=8, n_redundant=8, flip_y=0.15,
                               class_sep=0.85, random_state=0)
    cv = StratifiedKFold(5, shuffle=True, random_state=0)

    def baho(**kw):
        m = RandomForestClassifier(n_estimators=120, random_state=0,
                                   n_jobs=1, **kw)
        b = cross_val_score(m, X, y, cv=cv, scoring="roc_auc")
        return b.mean(), b.std()

    asos, asos_std = baho()
    print("=== 1. Bazaviy (sukut parametrlar) ===")
    print(f"  CV AUC {asos:.4f} (+-{asos_std:.4f})")

    print("\n=== 2. Har parametrni alohida o'zgartirish ===")
    tekshiruvlar = {
        "max_features": [("sqrt", {}), ("0.3", {"max_features": 0.3}),
                         ("0.6", {"max_features": 0.6}),
                         ("1.0", {"max_features": 1.0})],
        "min_samples_leaf": [("1", {}), ("5", {"min_samples_leaf": 5}),
                             ("20", {"min_samples_leaf": 20}),
                             ("60", {"min_samples_leaf": 60})],
        "max_depth": [("None", {}), ("4", {"max_depth": 4}),
                      ("8", {"max_depth": 8}), ("16", {"max_depth": 16})],
        "criterion": [("gini", {}), ("entropy", {"criterion": "entropy"}),
                      ("log_loss", {"criterion": "log_loss"})],
        "bootstrap": [("True", {}), ("False", {"bootstrap": False})],
    }
    print(f"  {'parametr':<20} {'eng past':>10} {'eng yuqori':>11} "
          f"{'oraliq':>9}")
    tasir = {}
    for nom, variantlar in tekshiruvlar.items():
        ballar = [baho(**kw)[0] for _, kw in variantlar]
        oraliq = max(ballar) - min(ballar)
        tasir[nom] = oraliq
        print(f"  {nom:<20} {min(ballar):>10.4f} {max(ballar):>11.4f} "
              f"{oraliq:>9.4f}")

    print("\n=== 3. Muhimlik tartibi ===")
    print(f"  {'parametr':<20} {'oraliq':>9} {'SE ga nisbatan':>16}")
    for nom, oraliq in sorted(tasir.items(), key=lambda kv: -kv[1]):
        print(f"  {nom:<20} {oraliq:>9.4f} {oraliq / asos_std:>15.1f}x")
    muhim = [n for n, o in tasir.items() if o > 2 * asos_std]
    print(f"  SE ning 2 barobaridan katta ta'sir: {muhim}")

    print("\n=== 4. n_estimators - giperparametr emas, resurs ===")
    print(f"  {'n_estimators':>13} {'CV AUC':>9} {'oldingidan':>11}")
    oldingi = None
    for n in [10, 50, 100, 250]:
        m = RandomForestClassifier(n_estimators=n, random_state=0, n_jobs=1)
        b = cross_val_score(m, X, y, cv=cv, scoring="roc_auc").mean()
        osish = "-" if oldingi is None else f"{b - oldingi:+.4f}"
        print(f"  {n:>13} {b:>9.4f} {osish:>11}")
        oldingi = b
    print("  ko'paytirish yomonlashtirmaydi, faqat to'yinadi")
    print("  ⭐ Har oilada 2-3 parametr natijaning ko'pini belgilaydi")


if __name__ == "__main__":
    main()

=== 1. Bazaviy (sukut parametrlar) ===
  CV AUC 0.8987 (+-0.0158)

=== 2. Har parametrni alohida o'zgartirish ===
  parametr               eng past  eng yuqori    oraliq
  max_features             0.8878      0.8987    0.0109
  min_samples_leaf         0.8719      0.8987    0.0268
  max_depth                0.8828      0.8987    0.0159
  criterion                0.8984      0.8987    0.0003
  bootstrap                0.8987      0.9021    0.0034

=== 3. Muhimlik tartibi ===
  parametr                oraliq   SE ga nisbatan
  min_samples_leaf        0.0268             1.7x
  max_depth               0.0159             1.0x
  max_features            0.0109             0.7x
  bootstrap               0.0034             0.2x
  criterion               0.0003             0.0x
  SE ning 2 barobaridan katta ta'sir: []

=== 4. n_estimators - giperparametr emas, resurs ===
   n_estimators    CV AUC  oldingidan
             10    0.8661           -
             50    0.8936     +0.0275
            100    0.8978     +0.0042
            250    0.8988     +0.0010
  ko'paytirish yomonlashtirmaydi, faqat to'yinadi
  ⭐ Har oilada 2-3 parametr natijaning ko'pini belgilaydi

"""Oraliqni to'g'ri tanlash (real numpy/sklearn)."""

import numpy as np
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold, cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler


def main() -> None:
    X, y = make_classification(n_samples=1500, n_features=60,
                               n_informative=10, n_redundant=20,
                               flip_y=0.2, class_sep=0.8, random_state=0)
    cv = StratifiedKFold(5, shuffle=True, random_state=0)

    def baho(C: float) -> float:
        m = make_pipeline(StandardScaler(),
                          LogisticRegression(C=C, max_iter=3000))
        return cross_val_score(m, X, y, cv=cv, scoring="roc_auc").mean()

    print("=== 1. Chiziqli oraliq: C = 1..5 ===")
    chiziqli = {C: baho(C) for C in [1, 2, 3, 4, 5]}
    print(f"  {'C':>8} {'CV AUC':>9}")
    for C, b in chiziqli.items():
        print(f"  {C:>8} {b:>9.4f}")
    print(f"  oraliq: {max(chiziqli.values()) - min(chiziqli.values()):.5f}")
    print("  besh nuqta ham deyarli bir xil - qidiruv behuda")

    print("\n=== 2. Logarifmik oraliq: C = 1e-4..1e2 ===")
    log_C = np.logspace(-4, 2, 13)
    log_ballar = {float(C): baho(float(C)) for C in log_C}
    print(f"  {'C':>10} {'CV AUC':>9}")
    for C, b in list(log_ballar.items())[::2]:
        print(f"  {C:>10.4g} {b:>9.4f}")
    print(f"  oraliq: {max(log_ballar.values()) - min(log_ballar.values()):.4f}")
    eng_C = max(log_ballar, key=log_ballar.get)
    print(f"  eng yaxshi C = {eng_C:.4g} (AUC {log_ballar[eng_C]:.4f})")

    print("\n=== 3. Maydon cheti muammosi ===")
    tor = np.logspace(0, 2, 5)
    tor_ballar = {float(C): baho(float(C)) for C in tor}
    eng_tor = max(tor_ballar, key=tor_ballar.get)
    print(f"  tor maydon [1, 100]: eng yaxshi C = {eng_tor:.4g}")
    print(f"  chetdami: {eng_tor in (float(tor[0]), float(tor[-1]))}")
    print(f"  keng maydonda esa C = {eng_C:.4g}")
    print(f"  tor maydon natijasi: {tor_ballar[eng_tor]:.4f}")
    print(f"  keng maydon natijasi: {log_ballar[eng_C]:.4f}")
    print(f"  yo'qotish: {log_ballar[eng_C] - tor_ballar[eng_tor]:+.4f}")

    print("\n=== 4. Masshtablashning ahamiyati ===")
    print(f"  {'variant':<28} {'eng yaxshi C':>13} {'CV AUC':>9}")
    for nom, quvur_yasovchi in [
            ("masshtablangan",
             lambda C: make_pipeline(StandardScaler(),
                                     LogisticRegression(C=C,
                                                        max_iter=3000))),
            ("masshtablanmagan",
             lambda C: LogisticRegression(C=C, max_iter=3000))]:
        ballar = {}
        for C in np.logspace(-4, 2, 7):
            m = quvur_yasovchi(float(C))
            ballar[float(C)] = cross_val_score(m, X, y, cv=cv,
                                               scoring="roc_auc").mean()
        eng = max(ballar, key=ballar.get)
        print(f"  {nom:<28} {eng:>13.4g} {ballar[eng]:>9.4f}")
    print("  ⭐ C va alpha uchun logarifmik oraliq - yagona to'g'ri yo'l")


if __name__ == "__main__":
    main()

=== 1. Chiziqli oraliq: C = 1..5 ===
         C    CV AUC
         1    0.6533
         2    0.6532
         3    0.6532
         4    0.6532
         5    0.6532
  oraliq: 0.00019
  besh nuqta ham deyarli bir xil - qidiruv behuda

=== 2. Logarifmik oraliq: C = 1e-4..1e2 ===
           C    CV AUC
      0.0001    0.6626
       0.001    0.6652
        0.01    0.6584
         0.1    0.6539
           1    0.6533
          10    0.6532
         100    0.6532
  oraliq: 0.0120
  eng yaxshi C = 0.001 (AUC 0.6652)

=== 3. Maydon cheti muammosi ===
  tor maydon [1, 100]: eng yaxshi C = 1
  chetdami: True
  keng maydonda esa C = 0.001
  tor maydon natijasi: 0.6533
  keng maydon natijasi: 0.6652
  yo'qotish: +0.0118

=== 4. Masshtablashning ahamiyati ===
  variant                       eng yaxshi C    CV AUC
  masshtablangan                       0.001    0.6652
  masshtablanmagan                     0.001    0.6716
  ⭐ C va alpha uchun logarifmik oraliq - yagona to'g'ri yo'l

"""learning_rate + n_estimators va C + gamma (real numpy/sklearn)."""

import numpy as np
from sklearn.datasets import make_classification
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.model_selection import StratifiedKFold, cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC


def main() -> None:
    X, y = make_classification(n_samples=2000, n_features=20,
                               n_informative=7, n_redundant=4, flip_y=0.15,
                               class_sep=0.85, random_state=0)
    cv = StratifiedKFold(4, shuffle=True, random_state=0)

    print("=== 1. learning_rate va max_iter bog'liqligi ===")
    print(f"  {'lr':>7} {'max_iter':>10} {'CV AUC':>9}")
    eng_yaxshi = {}
    for lr in [0.3, 0.1, 0.03]:
        for it in [50, 150, 500]:
            m = HistGradientBoostingClassifier(learning_rate=lr, max_iter=it,
                                               early_stopping=False,
                                               random_state=0)
            b = cross_val_score(m, X, y, cv=cv, scoring="roc_auc").mean()
            eng_yaxshi[(lr, it)] = b
            print(f"  {lr:>7} {it:>10} {b:>9.4f}")
    par = max(eng_yaxshi, key=eng_yaxshi.get)
    print(f"  eng yaxshi: lr={par[0]}, max_iter={par[1]} "
          f"({eng_yaxshi[par]:.4f})")
    print("  kichik lr uchun ko'proq iteratsiya kerak")

    print("\n=== 2. Faqat lr ni sozlash (max_iter qat'iy 100) ===")
    print(f"  {'lr':>7} {'CV AUC':>9}")
    bir_olchovli = {}
    for lr in [0.3, 0.1, 0.03, 0.01]:
        m = HistGradientBoostingClassifier(learning_rate=lr, max_iter=100,
                                           early_stopping=False,
                                           random_state=0)
        bir_olchovli[lr] = cross_val_score(m, X, y, cv=cv,
                                           scoring="roc_auc").mean()
        print(f"  {lr:>7} {bir_olchovli[lr]:>9.4f}")
    eng_1d = max(bir_olchovli, key=bir_olchovli.get)
    print(f"  1D qidiruv: lr={eng_1d} -> {bir_olchovli[eng_1d]:.4f}")
    print(f"  2D qidiruv: {eng_yaxshi[par]:.4f}")
    print(f"  yo'qotish: {eng_yaxshi[par] - bir_olchovli[eng_1d]:+.4f}")

    print("\n=== 3. SVM: C va gamma birga ===")
    print(f"  {'gamma':>8}", end="")
    C_lar = [0.1, 1, 10, 100]
    for C in C_lar:
        print(f" {'C=' + str(C):>9}", end="")
    print()
    setka = {}
    for gamma in [0.001, 0.01, 0.1, 1.0]:
        print(f"  {gamma:>8}", end="")
        for C in C_lar:
            m = make_pipeline(StandardScaler(),
                              SVC(C=C, gamma=gamma, random_state=0))
            b = cross_val_score(m, X, y, cv=cv, scoring="roc_auc").mean()
            setka[(C, gamma)] = b
            print(f" {b:>9.4f}", end="")
        print()
    eng = max(setka, key=setka.get)
    print(f"  eng yaxshi: C={eng[0]}, gamma={eng[1]} ({setka[eng]:.4f})")

    print("\n=== 4. SVM da bittalab sozlash ishlaydimi ===")
    # avval gamma ni sukut C=1 bilan
    gamma_ballar = {g: setka[(1, g)] for g in [0.001, 0.01, 0.1, 1.0]}
    eng_gamma = max(gamma_ballar, key=gamma_ballar.get)
    # keyin C ni shu gamma bilan
    C_ballar = {C: setka[(C, eng_gamma)] for C in C_lar}
    eng_C = max(C_ballar, key=C_ballar.get)
    print(f"  1-qadam (C=1): eng yaxshi gamma = {eng_gamma}")
    print(f"  2-qadam: eng yaxshi C = {eng_C}")
    print(f"  ketma-ket natija: {setka[(eng_C, eng_gamma)]:.4f}")
    print(f"  to'liq setka:     {setka[eng]:.4f}")
    print(f"  farq: {setka[eng] - setka[(eng_C, eng_gamma)]:+.4f}")
    print("  ⭐ Bog'liq parametrlarni birga qidiring")


if __name__ == "__main__":
    main()

=== 1. learning_rate va max_iter bog'liqligi ===
       lr   max_iter    CV AUC
      0.3         50    0.8652
      0.3        150    0.8702
      0.3        500    0.8707
      0.1         50    0.8667
      0.1        150    0.8713
      0.1        500    0.8732
     0.03         50    0.8546
     0.03        150    0.8681
     0.03        500    0.8712
  eng yaxshi: lr=0.1, max_iter=500 0.8732-bob
  kichik lr uchun ko'proq iteratsiya kerak

=== 2. Faqat lr ni sozlash (max_iter qat'iy 100) ===
       lr    CV AUC
      0.3    0.8685
      0.1    0.8701
     0.03    0.8647
     0.01    0.8457
  1D qidiruv: lr=0.1 -> 0.8701
  2D qidiruv: 0.8732
  yo'qotish: +0.0031

=== 3. SVM: C va gamma birga ===
     gamma     C=0.1       C=1      C=10     C=100
     0.001    0.7068    0.7148    0.7402    0.8175
      0.01    0.7421    0.8164    0.8526    0.8483
       0.1    0.8261    0.8620    0.8326    0.8313
       1.0    0.7272    0.7273    0.7273    0.7273
  eng yaxshi: C=1, gamma=0.1 0.8620-bob

=== 4. SVM da bittalab sozlash ishlaydimi ===
  1-qadam (C=1): eng yaxshi gamma = 0.1
  2-qadam: eng yaxshi C = 1
  ketma-ket natija: 0.8620
  to'liq setka:     0.8620
  farq: +0.0000
  ⭐ Bog'liq parametrlarni birga qidiring

"""Bazaviy -> keng qidiruv -> aniqlashtirish (real numpy/sklearn)."""

import numpy as np
from scipy.stats import loguniform, randint
from sklearn.datasets import make_classification
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.model_selection import (RandomizedSearchCV, StratifiedKFold,
                                     cross_val_score)


def main() -> None:
    X, y = make_classification(n_samples=2000, n_features=25,
                               n_informative=8, n_redundant=6, flip_y=0.18,
                               class_sep=0.8, random_state=0)
    cv = StratifiedKFold(4, shuffle=True, random_state=0)

    def baho(model) -> tuple:
        b = cross_val_score(model, X, y, cv=cv, scoring="roc_auc")
        return b.mean(), b.std()

    print("=== 1. Bazaviy ===")
    asos, asos_std = baho(HistGradientBoostingClassifier(random_state=0))
    print(f"  sukut parametrlar: {asos:.4f} (+-{asos_std:.4f})")

    print("\n=== 2. Keng qidiruv (muhim 3 parametr, 18 nomzod) ===")
    keng = {"learning_rate": loguniform(0.01, 0.4),
            "max_leaf_nodes": randint(5, 60),
            "min_samples_leaf": randint(5, 120)}
    q1 = RandomizedSearchCV(
        HistGradientBoostingClassifier(max_iter=200, early_stopping=False,
                                       random_state=0),
        keng, n_iter=18, cv=cv, scoring="roc_auc", random_state=0,
        n_jobs=1).fit(X, y)
    p1 = q1.best_params_
    print(f"  eng yaxshi: lr={p1['learning_rate']:.4f}, "
          f"barglar={p1['max_leaf_nodes']}, "
          f"min_leaf={p1['min_samples_leaf']}")
    print(f"  ball: {q1.best_score_:.4f} "
          f"(bazaviydan {q1.best_score_ - asos:+.4f})")

    print("\n=== 3. Tor maydonda aniqlashtirish ===")
    lr = p1["learning_rate"]
    barg = p1["max_leaf_nodes"]
    tor = {"learning_rate": loguniform(lr / 2, lr * 2),
           "max_leaf_nodes": randint(max(4, barg - 8), barg + 9),
           "min_samples_leaf": randint(
               max(2, p1["min_samples_leaf"] - 20),
               p1["min_samples_leaf"] + 21)}
    q2 = RandomizedSearchCV(
        HistGradientBoostingClassifier(max_iter=200, early_stopping=False,
                                       random_state=0),
        tor, n_iter=12, cv=cv, scoring="roc_auc", random_state=1,
        n_jobs=1).fit(X, y)
    print(f"  ball: {q2.best_score_:.4f} "
          f"(keng qidiruvdan {q2.best_score_ - q1.best_score_:+.4f})")

    print("\n=== 4. Qo'shimcha parametrlarni bittalab sinash ===")
    eng = dict(q2.best_params_)
    joriy = q2.best_score_
    print(f"  {'qo_shimcha':<34} {'ball':>9} {'o_sish':>9} {'qaror':<12}")
    for nom, qiymat in [("l2_regularization=1.0",
                         {"l2_regularization": 1.0}),
                        ("l2_regularization=5.0",
                         {"l2_regularization": 5.0}),
                        ("max_features=0.6", {"max_features": 0.6}),
                        ("interaction_cst='no_interactions'",
                         {"interaction_cst": "no_interactions"})]:
        m = HistGradientBoostingClassifier(max_iter=200,
                                           early_stopping=False,
                                           random_state=0, **eng, **qiymat)
        b, s = baho(m)
        qaror = "QOLDIRISH" if b - joriy > s else "tashlash"
        print(f"  {nom:<34} {b:>9.4f} {b - joriy:>+9.4f} {qaror:<12}")

    print("\n=== 5. Yakuniy hisobot ===")
    print(f"  {'bosqich':<28} {'ball':>9} {'o_sish':>9}")
    print(f"  {'bazaviy':<28} {asos:>9.4f} {'-':>9}")
    print(f"  {'keng qidiruv (18)':<28} {q1.best_score_:>9.4f} "
          f"{q1.best_score_ - asos:>+9.4f}")
    print(f"  {'aniqlashtirish (12)':<28} {q2.best_score_:>9.4f} "
          f"{q2.best_score_ - q1.best_score_:>+9.4f}")
    print(f"  jami o'sish: {q2.best_score_ - asos:+.4f} "
          f"(bazaviy SE: {asos_std:.4f})")
    print("  ⭐ Bazaviy -> keng -> tor -> bittalab")


if __name__ == "__main__":
    main()

=== 1. Bazaviy ===
  sukut parametrlar: 0.8485 (+-0.0189)

=== 2. Keng qidiruv (muhim 3 parametr, 18 nomzod) ===
  eng yaxshi: lr=0.0331, barglar=10, min_leaf=43
  ball: 0.8507 (bazaviydan +0.0022)

=== 3. Tor maydonda aniqlashtirish ===
  ball: 0.8523 (keng qidiruvdan +0.0016)

=== 4. Qo'shimcha parametrlarni bittalab sinash ===
  qo_shimcha                              ball    o_sish qaror
  l2_regularization=1.0                 0.8518   -0.0006 tashlash
  l2_regularization=5.0                 0.8506   -0.0017 tashlash
  max_features=0.6                      0.8518   -0.0005 tashlash
  interaction_cst='no_interactions'     0.8127   -0.0397 tashlash

=== 5. Yakuniy hisobot ===
  bosqich                           ball    o_sish
  bazaviy                         0.8485         -
  keng qidiruv (18)               0.8507   +0.0022
  aniqlashtirish (12)             0.8523   +0.0016
  jami o'sish: +0.0038 (bazaviy SE: 0.0189)
  ⭐ Bazaviy -> keng -> tor -> bittalab

{"C": [1, 2, 3, 4, 5]}                                           # ⚠️
{"C": np.logspace(-4, 2, 13)}                                    # ✅

{"max_depth": ..., "criterion": ..., "bootstrap": ...,
 "min_impurity_decrease": ..., "ccp_alpha": ...}                 # ⚠️
{"max_features": ..., "min_samples_leaf": ...}                   # ✅

{"learning_rate": [0.3, 0.1, 0.03]}      # max_iter qat'iy       # ⚠️
{"learning_rate": [...], "max_iter": [...]}                      # ✅

# eng yaxshi C = 100 (maydon chegarasi) -> tayyor             # ⚠️
# maydonni 1e3 gacha kengaytirib qayta qidiring               # ✅

{"random_state": [0, 1, 2, 3, 4]}                                # ⚠️
RandomForestClassifier(random_state=0)                           # ✅

GridSearchCV(SVC(), {"C": ..., "gamma": ...})                    # ⚠️
GridSearchCV(make_pipeline(StandardScaler(), SVC()), {...})      # ✅

{"max_iter": [100, 500, 1000]}   # LogisticRegression uchun      # ⚠️
LogisticRegression(max_iter=3000)   # yetarlicha katta           # ✅

1.  # parametr va giperparametr farqi?
2.  # sklearn da qaysi atribut o'rganilgan?
3.  # chiziqli modelda muhim parametr?
4.  # SVM RBF da?
5.  # RF da eng muhimi?
6.  # boostingda bog'liq juftlik?
7.  # qaysi oraliq C uchun?
8.  # max_depth uchun?
9.  # chetda chiqsa nima qilish?
10. # sozlanmaydigan parametrlar?
11. # n_estimators (RF) nima?
12. # amaliy tartib?

1.  {"C": [1, 2, 3, 4, 5]}

2.  {"learning_rate": [0.3, 0.1, 0.03]}   # max_iter qat'iy

3.  {"random_state": [0, 1, 2, 3, 4]}

4.  GridSearchCV(SVC(), {"C": ..., "gamma": ...})

5.  {"max_iter": [100, 500, 1000]}