18.6-dars: Grid va random search

GridSearchCV(estimator, param_grid, cv, scoring, refit=True)

param_grid = {"C": [0.1, 1, 10], "gamma": [0.01, 0.1]}
  -> 3 * 2 = 6 kombinatsiya, har biri cv marta

RO'YXAT ko'rinishida ham bo'ladi (turli maydonlar):
  [{"kernel": ["linear"], "C": [...]},
   {"kernel": ["rbf"], "C": [...], "gamma": [...]}]

QACHON YAXSHI:
  parametrlar 1-2 ta
  qiymatlar diskret va oz (kernel, weights)
  maydon kichik va to'liq aylanish arzon

RandomizedSearchCV(estimator, param_distributions, n_iter, cv)

Har iteratsiyada har parametr uchun TASODIFIY qiymat olinadi.

NIMA UCHUN YAXSHIROQ (Bergstra & Bengio 2012):

  Setka 5x5 = 25 nuqta:
    muhim parametr uchun atigi 5 ta TURLI qiymat
    befoyda parametr 5 marta takrorlanadi

  Tasodifiy 25 nuqta:
    muhim parametr uchun 25 ta TURLI qiymat
    befoyda parametr ham 25 ta, lekin zarari yo'q

  Parametrlarning KO'PCHILIGI befoyda bo'lgani uchun
  tasodifiy qidiruv muhimini yaxshiroq qamrab oladi

BONUS: n_iter ni xohlagancha o'zgartirasiz (byudjet nazorati)

scipy.stats dan:
  loguniform(1e-4, 1e2)   -> C, alpha, gamma, learning_rate
  uniform(0.1, 0.8)       -> 0.1 dan 0.9 gacha (loc, SCALE!)
  randint(5, 60)          -> 5..59 butun son
  ro'yxat ["a", "b"]      -> teng ehtimol bilan tanlov

DIQQAT: uniform(a, b) -> [a, a+b] oralig'i (b - KENGLIK)
  uniform(0.1, 0.8) -> 0.1 .. 0.9

Aralash: {"C": loguniform(...), "kernel": ["rbf", "linear"]}

qidiruv.cv_results_ -> dict (DataFrame ga aylantiring)

MUHIM USTUNLAR:
  params                  har nomzodning parametrlari
  mean_test_score         CV o'rtachasi
  std_test_score          foldlar std i
  rank_test_score         o'rin
  mean_fit_time           o'rgatish vaqti
  split0_test_score, ...  har fold ballari

TAHLIL:
  - eng yaxshi 10 tasini ko'ring: parametrlar o'xshashmi?
  - ball va parametr grafigi: trend bormi, chetdami?
  - std katta bo'lsa: tanlov shovqinli
  - 1-o'rin va 5-o'rin farqi std dan kichik bo'lsa:
    ularning orasida ma'noli farq YO'Q

scoring = {"auc": "roc_auc", "ap": "average_precision",
           "f1": "f1"}
refit = "auc"        # qaysi metrika bo'yicha yakuniy model

cv_results_ da: mean_test_auc, mean_test_ap, mean_test_f1

FOYDASI:
  - bitta qidiruvda bir necha metrikani ko'rasiz
  - metrikalar zid bo'lsa bilib olasiz

refit=False -> best_estimator_ yaratilmaydi (faqat tahlil)
return_train_score=True -> overfitting ni ko'rish

UMUMIY QOIDA:
  n_iter = 20-30   -> tez, 2-3 parametr uchun yetarli
  n_iter = 60-100  -> jiddiy qidiruv
  n_iter > 200     -> kamdan-kam oqlanadi

IKKI BOSQICHLI (samaraliroq):
  1. keng maydon, n_iter=30
  2. eng yaxshi atrofida tor maydon, n_iter=20

NARX HISOBI:
  modellar = n_iter * k_fold
  nested bo'lsa: * k_tashqi

VAQT: bitta fit vaqtini o'lchang va ko'paytiring

import pandas as pd
from scipy.stats import loguniform, randint, uniform
from sklearn.model_selection import GridSearchCV, RandomizedSearchCV

taqsimot = {"m__C": loguniform(1e-3, 1e3),
            "m__gamma": loguniform(1e-4, 1e0),
            "m__kernel": ["rbf", "poly"]}

q = RandomizedSearchCV(quvur, taqsimot, n_iter=40, cv=cv,
                       scoring={"auc": "roc_auc", "ap": "average_precision"},
                       refit="auc", return_train_score=True,
                       random_state=0, n_jobs=1).fit(X, y)

natija = pd.DataFrame(q.cv_results_).sort_values("rank_test_auc")
print(natija[["params", "mean_test_auc", "std_test_auc",
              "mean_train_auc"]].head())
QOIDA: random search sukut bo'lsin · uniform(a, b) = [a, a+b] ·
       cv_results_ ni ko'r · best_score_ ni e'lon qilma

GridSearchCV      1-2 diskret parametr
RandomizedSearch  sukut tanlov, byudjet nazorati
Taqsimotlar: loguniform, randint, uniform, ro'yxat
cv_results_: rank, std, train vs test
Ikki bosqich: keng -> tor

"""Bergstra-Bengio tajribasi (real numpy/sklearn)."""

import numpy as np
from scipy.stats import loguniform, randint
from sklearn.datasets import make_classification
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.model_selection import (GridSearchCV, RandomizedSearchCV,
                                     StratifiedKFold)


def main() -> None:
    X, y = make_classification(n_samples=900, n_features=25,
                               n_informative=8, n_redundant=6, flip_y=0.18,
                               class_sep=0.8, random_state=0)
    cv = StratifiedKFold(3, shuffle=True, random_state=0)

    def model():
        return HistGradientBoostingClassifier(max_iter=50,
                                              early_stopping=False,
                                              random_state=0)

    print("=== 1. Setka: 3 parametr x 3 qiymat = 27 nomzod ===")
    setka = {"learning_rate": [0.03, 0.1, 0.3],
             "max_leaf_nodes": [8, 20, 50],
             "min_samples_leaf": [5, 25, 80]}
    g = GridSearchCV(model(), setka, cv=cv, scoring="roc_auc",
                     n_jobs=1).fit(X, y)
    print(f"  nomzodlar: {len(g.cv_results_['params'])}")
    print(f"  eng yaxshi: {g.best_params_}")
    print(f"  ball: {g.best_score_:.4f}")
    print(f"  har parametr uchun TURLI qiymat: 3 ta")

    print("\n=== 2. Tasodifiy: bir xil byudjet (27 nomzod) ===")
    taqsimot = {"learning_rate": loguniform(0.02, 0.4),
                "max_leaf_nodes": randint(6, 60),
                "min_samples_leaf": randint(4, 100)}
    r = RandomizedSearchCV(model(), taqsimot, n_iter=27, cv=cv,
                           scoring="roc_auc", random_state=0,
                           n_jobs=1).fit(X, y)
    p = r.best_params_
    print(f"  nomzodlar: {len(r.cv_results_['params'])}")
    print(f"  eng yaxshi: lr={p['learning_rate']:.4f}, "
          f"barglar={p['max_leaf_nodes']}, "
          f"min_leaf={p['min_samples_leaf']}")
    print(f"  ball: {r.best_score_:.4f}")
    print(f"  har parametr uchun TURLI qiymat: 27 ta")

    print("\n=== 3. Turli byudjetlarda taqqoslash ===")
    print(f"  {'byudjet':>8} {'setka':>9} {'tasodifiy':>11} {'farq':>9}")
    setkalar = {
        8: {"learning_rate": [0.05, 0.2], "max_leaf_nodes": [10, 40],
            "min_samples_leaf": [10, 60]},
        27: setka,
        64: {"learning_rate": [0.03, 0.07, 0.15, 0.3],
             "max_leaf_nodes": [8, 16, 32, 56],
             "min_samples_leaf": [5, 20, 50, 90]},
    }
    for byudjet, s in setkalar.items():
        gg = GridSearchCV(model(), s, cv=cv, scoring="roc_auc",
                          n_jobs=1).fit(X, y)
        rr = RandomizedSearchCV(model(), taqsimot, n_iter=byudjet, cv=cv,
                                scoring="roc_auc", random_state=0,
                                n_jobs=1).fit(X, y)
        print(f"  {byudjet:>8} {gg.best_score_:>9.4f} "
              f"{rr.best_score_:>11.4f} "
              f"{rr.best_score_ - gg.best_score_:>+9.4f}")

    print("\n=== 4. Nima uchun shunday ===")
    print("  setka 4x4x4 da: lr uchun atigi 4 ta turli qiymat")
    print("  tasodifiy 64 da: lr uchun 64 ta turli qiymat")
    lr_lar = [p["learning_rate"] for p in r.cv_results_["params"]]
    print(f"  tasodifiy qidiruvdagi lr lar (27 ta, birinchi 6 tasi):")
    print(f"    {[round(v, 4) for v in sorted(lr_lar)[:6]]}")
    print(f"  eng past {min(lr_lar):.4f}, eng yuqori {max(lr_lar):.4f}")
    print("  ⭐ Tasodifiy qidiruv muhim parametrni zichroq qamrab oladi")


if __name__ == "__main__":
    main()

=== 1. Setka: 3 parametr x 3 qiymat = 27 nomzod ===
  nomzodlar: 27
  eng yaxshi: {'learning_rate': 0.3, 'max_leaf_nodes': 50, 'min_samples_leaf': 5}
  ball: 0.8447
  har parametr uchun TURLI qiymat: 3 ta

=== 2. Tasodifiy: bir xil byudjet (27 nomzod) ===
  nomzodlar: 27
  eng yaxshi: lr=0.1255, barglar=7, min_leaf=69
  ball: 0.8417
  har parametr uchun TURLI qiymat: 27 ta

=== 3. Turli byudjetlarda taqqoslash ===
   byudjet     setka   tasodifiy      farq
         8    0.8379      0.8365   -0.0014
        27    0.8447      0.8417   -0.0030
        64    0.8469      0.8417   -0.0052

=== 4. Nima uchun shunday ===
  setka 4x4x4 da: lr uchun atigi 4 ta turli qiymat
  tasodifiy 64 da: lr uchun 64 ta turli qiymat
  tasodifiy qidiruvdagi lr lar (27 ta, birinchi 6 tasi):
    [np.float64(0.026), np.float64(0.0269), np.float64(0.0275), np.float64(0.035), np.float64(0.0374), np.float64(0.0376)]
  eng past 0.0260, eng yuqori 0.3863
  ⭐ Tasodifiy qidiruv muhim parametrni zichroq qamrab oladi

"""loguniform, uniform, randint va ularning tuzoqlari (real numpy/scipy)."""

import numpy as np
from scipy.stats import loguniform, randint, uniform
from sklearn.datasets import make_classification
from sklearn.model_selection import RandomizedSearchCV, StratifiedKFold
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC


def main() -> None:
    rng = np.random.default_rng(0)

    print("=== 1. uniform(a, b) tuzog'i ===")
    n = uniform(0.1, 0.8).rvs(10000, random_state=0)
    print(f"  uniform(0.1, 0.8): eng past {n.min():.3f}, "
          f"eng yuqori {n.max():.3f}")
    print("  ya'ni [0.1, 0.9] - ikkinchi argument KENGLIK")
    n2 = uniform(0.1, 0.7).rvs(10000, random_state=0)
    print(f"  [0.1, 0.8] kerak bo'lsa: uniform(0.1, 0.7) -> "
          f"[{n2.min():.3f}, {n2.max():.3f}]")

    print("\n=== 2. loguniform va uniform farqi ===")
    lu = loguniform(1e-4, 1e2).rvs(10000, random_state=0)
    un = uniform(1e-4, 1e2).rvs(10000, random_state=0)
    print(f"  {'oraliq':<18} {'loguniform':>12} {'uniform':>10}")
    chegaralar = [(1e-4, 1e-2), (1e-2, 1e0), (1e0, 1e2)]
    for past, yuqori in chegaralar:
        a = float(((lu >= past) & (lu < yuqori)).mean())
        b = float(((un >= past) & (un < yuqori)).mean())
        print(f"  [{past:g}, {yuqori:g})".ljust(20)
              + f"{a:>10.1%} {b:>10.1%}")
    print("  loguniform har kattalik tartibiga teng vaqt ajratadi")

    print("\n=== 3. randint va ro'yxat ===")
    ri = randint(5, 60).rvs(10, random_state=0)
    print(f"  randint(5, 60) 10 ta namuna: {ri.tolist()}")
    print(f"  chegaralar: [5, 59] (yuqorisi KIRMAYDI)")
    tanlov = rng.choice(["rbf", "poly", "sigmoid"], 10).tolist()
    print(f"  ro'yxatdan tanlov: {tanlov}")

    print("\n=== 4. Amaliy qidiruvda ===")
    X, y = make_classification(n_samples=1200, n_features=18,
                               n_informative=6, flip_y=0.15,
                               class_sep=0.85, random_state=0)
    cv = StratifiedKFold(4, shuffle=True, random_state=0)
    quvur = make_pipeline(StandardScaler(), SVC(random_state=0))

    variantlar = {
        "loguniform": {"svc__C": loguniform(1e-2, 1e3),
                       "svc__gamma": loguniform(1e-4, 1e0)},
        "uniform (noto'g'ri)": {"svc__C": uniform(0.01, 1000),
                                "svc__gamma": uniform(0.0001, 1.0)},
    }
    print(f"  {'taqsimot':<22} {'ball':>8} {'eng yaxshi C':>14} "
          f"{'gamma':>10}")
    for nom, t in variantlar.items():
        q = RandomizedSearchCV(quvur, t, n_iter=30, cv=cv,
                               scoring="roc_auc", random_state=0,
                               n_jobs=1).fit(X, y)
        print(f"  {nom:<22} {q.best_score_:>8.4f} "
              f"{q.best_params_['svc__C']:>14.4g} "
              f"{q.best_params_['svc__gamma']:>10.4g}")
    print("  uniform da kichik qiymatlar deyarli tanlanmaydi")
    print("  ⭐ uniform(a, b) = [a, a+b], loguniform(a, b) = [a, b]")


if __name__ == "__main__":
    main()

=== 1. uniform(a, b) tuzog'i ===
  uniform(0.1, 0.8): eng past 0.100, eng yuqori 0.900
  ya'ni [0.1, 0.9] - ikkinchi argument KENGLIK
  [0.1, 0.8] kerak bo'lsa: uniform(0.1, 0.7) -> [0.100, 0.800]

=== 2. loguniform va uniform farqi ===
  oraliq               loguniform    uniform
  [0.0001, 0.01)         33.8%       0.0%
  [0.01, 1)              33.2%       1.1%
  [1, 100)               33.0%      98.9%
  loguniform har kattalik tartibiga teng vaqt ajratadi

=== 3. randint va ro'yxat ===
  randint(5, 60) 10 ta namuna: [49, 52, 58, 5, 8, 8, 44, 14, 24, 26]
  chegaralar: [5, 59] (yuqorisi KIRMAYDI)
  ro'yxatdan tanlov: ['sigmoid', 'poly', 'poly', 'rbf', 'rbf', 'rbf', 'rbf', 'rbf', 'rbf', 'sigmoid']

=== 4. Amaliy qidiruvda ===
  taqsimot                   ball   eng yaxshi C      gamma
  loguniform               0.8666          1.313    0.03833
  uniform (noto'g'ri)      0.8365          208.9     0.1614
  uniform da kichik qiymatlar deyarli tanlanmaydi
  ⭐ uniform(a, b) = [a, a+b], loguniform(a, b) = [a, b]

"""Qidiruv natijalarini o'qish (real numpy/pandas/sklearn)."""

import numpy as np
import pandas as pd
from scipy.stats import loguniform, randint
from sklearn.datasets import make_classification
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.model_selection import RandomizedSearchCV, StratifiedKFold


def main() -> None:
    X, y = make_classification(n_samples=2000, n_features=22,
                               n_informative=7, n_redundant=5, flip_y=0.2,
                               class_sep=0.8, random_state=0)
    cv = StratifiedKFold(4, shuffle=True, random_state=0)
    taqsimot = {"learning_rate": loguniform(0.01, 0.5),
                "max_leaf_nodes": randint(4, 80),
                "min_samples_leaf": randint(3, 120)}
    q = RandomizedSearchCV(
        HistGradientBoostingClassifier(max_iter=200, early_stopping=False,
                                       random_state=0),
        taqsimot, n_iter=40, cv=cv,
        scoring={"auc": "roc_auc", "ap": "average_precision"},
        refit="auc", return_train_score=True, random_state=0,
        n_jobs=1).fit(X, y)

    natija = pd.DataFrame(q.cv_results_)
    print("=== 1. Eng yaxshi 5 nomzod ===")
    top = natija.nsmallest(5, "rank_test_auc")
    print(f"  {'o_rin':>6} {'lr':>8} {'barg':>6} {'leaf':>6} "
          f"{'test AUC':>10} {'std':>8} {'train AUC':>11}")
    for _, q_ in top.iterrows():
        p = q_["params"]
        print(f"  {int(q_['rank_test_auc']):>6} "
              f"{p['learning_rate']:>8.4f} {p['max_leaf_nodes']:>6} "
              f"{p['min_samples_leaf']:>6} {q_['mean_test_auc']:>10.4f} "
              f"{q_['std_test_auc']:>8.4f} {q_['mean_train_auc']:>11.4f}")

    print("\n=== 2. 1-o'rin va boshqalar orasidagi farq ===")
    tartib = natija.sort_values("rank_test_auc")
    birinchi = float(tartib["mean_test_auc"].iloc[0])
    std1 = float(tartib["std_test_auc"].iloc[0])
    print(f"  1-o'rin balli: {birinchi:.4f} (std {std1:.4f})")
    print(f"  {'o_rin':>6} {'ball':>9} {'1-o_rindan farq':>17} "
          f"{'std dan kichikmi':>18}")
    for i in [2, 5, 10, 20]:
        ball = float(tartib["mean_test_auc"].iloc[i - 1])
        farq = birinchi - ball
        print(f"  {i:>6} {ball:>9.4f} {farq:>+17.4f} "
              f"{str(farq < std1):>18}")
    yaqin = int((tartib["mean_test_auc"] > birinchi - std1).sum())
    print(f"  1 std ichidagi nomzodlar: {yaqin} ta")

    print("\n=== 3. Overfitting diagnostikasi ===")
    natija["farq"] = natija["mean_train_auc"] - natija["mean_test_auc"]
    print(f"  {'guruh':<24} {'train':>9} {'test':>9} {'farq':>9}")
    for nom, kesim in [
            ("eng yaxshi 10", tartib.head(10)),
            ("eng yomon 10", tartib.tail(10)),
            ("barglar > 40", natija[natija["param_max_leaf_nodes"] > 40]),
            ("barglar <= 15", natija[natija["param_max_leaf_nodes"] <= 15])]:
        print(f"  {nom:<24} {kesim['mean_train_auc'].mean():>9.4f} "
              f"{kesim['mean_test_auc'].mean():>9.4f} "
              f"{(kesim['mean_train_auc'] - kesim['mean_test_auc']).mean():>9.4f}")

    print("\n=== 4. Ikki metrika zid keladimi ===")
    auc_eng = tartib.index[0]
    ap_eng = natija.sort_values("rank_test_ap").index[0]
    print(f"  AUC bo'yicha eng yaxshi nomzod indeksi: {auc_eng}")
    print(f"  AP bo'yicha eng yaxshi nomzod indeksi:  {ap_eng}")
    print(f"  bir xilmi: {auc_eng == ap_eng}")
    korr = float(natija["mean_test_auc"].corr(natija["mean_test_ap"]))
    print(f"  ikki metrika korrelyatsiyasi: {korr:.4f}")
    print(f"  AUC eng yaxshi nomzodning AP o'rni: "
          f"{int(natija.loc[auc_eng, 'rank_test_ap'])}")
    print("  ⭐ cv_results_ ni ko'rmasdan qidiruvni tugatmang")


if __name__ == "__main__":
    main()

=== 1. Eng yaxshi 5 nomzod ===
   o_rin       lr   barg   leaf   test AUC      std   train AUC
       1   0.0296     15      5     0.8512   0.0210      0.9850
       2   0.0154     52     52     0.8507   0.0208      0.9600
       3   0.1676     69     12     0.8501   0.0197      1.0000
       4   0.0207     59     31     0.8499   0.0199      0.9975
       5   0.0151     27     38     0.8495   0.0181      0.9737

=== 2. 1-o'rin va boshqalar orasidagi farq ===
  1-o'rin balli: 0.8512 (std 0.0210)
   o_rin      ball   1-o_rindan farq   std dan kichikmi
       2    0.8507           +0.0005               True
       5    0.8495           +0.0016               True
      10    0.8482           +0.0030               True
      20    0.8419           +0.0093               True
  1 std ichidagi nomzodlar: 35 ta

=== 3. Overfitting diagnostikasi ===
  guruh                        train      test      farq
  eng yaxshi 10               0.9801    0.8495    0.1306
  eng yomon 10                0.9775    0.8302    0.1473
  barglar > 40                0.9883    0.8409    0.1474
  barglar <= 15               0.9442    0.8393    0.1049

=== 4. Ikki metrika zid keladimi ===
  AUC bo'yicha eng yaxshi nomzod indeksi: 39
  AP bo'yicha eng yaxshi nomzod indeksi:  17
  bir xilmi: False
  ikki metrika korrelyatsiyasi: 0.9191
  AUC eng yaxshi nomzodning AP o'rni: 2
  ⭐ cv_results_ ni ko'rmasdan qidiruvni tugatmang

"""Bitta katta qidiruv va ikki bosqichli qidiruv (real numpy/sklearn)."""

import numpy as np
from scipy.stats import loguniform, randint
from sklearn.datasets import make_classification
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.metrics import roc_auc_score
from sklearn.model_selection import RandomizedSearchCV, StratifiedKFold


def main() -> None:
    X, y = make_classification(n_samples=21000, n_features=25,
                               n_informative=8, n_redundant=6, flip_y=0.18,
                               class_sep=0.8, random_state=0)
    X_ish, y_ish = X[:1000], y[:1000]
    X_haq, y_haq = X[1000:], y[1000:]
    cv = StratifiedKFold(3, shuffle=True, random_state=0)

    def model():
        return HistGradientBoostingClassifier(max_iter=50,
                                              early_stopping=False,
                                              random_state=0)

    keng = {"learning_rate": loguniform(0.01, 0.5),
            "max_leaf_nodes": randint(4, 80),
            "min_samples_leaf": randint(3, 120)}

    print("=== 1. Byudjet va natija ===")
    print(f"  {'n_iter':>8} {'CV ball':>9} {'haqiqiy':>9} "
          f"{'modellar':>10}")
    oldingi = None
    for n_iter in [10, 20, 40]:
        q = RandomizedSearchCV(model(), keng, n_iter=n_iter, cv=cv,
                               scoring="roc_auc", random_state=0,
                               n_jobs=1).fit(X_ish, y_ish)
        h = roc_auc_score(y_haq,
                          q.best_estimator_.predict_proba(X_haq)[:, 1])
        print(f"  {n_iter:>8} {q.best_score_:>9.4f} {h:>9.4f} "
              f"{n_iter * 3:>10}")
        if n_iter == 40:
            oldingi = q

    print("\n=== 2. Ikki bosqichli qidiruv (30 + 20 = 50) ===")
    q1 = RandomizedSearchCV(model(), keng, n_iter=30, cv=cv,
                            scoring="roc_auc", random_state=0,
                            n_jobs=1).fit(X_ish, y_ish)
    p = q1.best_params_
    print(f"  1-bosqich (30): {q1.best_score_:.4f}")
    print(f"    lr={p['learning_rate']:.4f}, "
          f"barglar={p['max_leaf_nodes']}, leaf={p['min_samples_leaf']}")
    tor = {"learning_rate": loguniform(p["learning_rate"] / 2.5,
                                       p["learning_rate"] * 2.5),
           "max_leaf_nodes": randint(max(4, p["max_leaf_nodes"] - 12),
                                     p["max_leaf_nodes"] + 13),
           "min_samples_leaf": randint(max(2, p["min_samples_leaf"] - 25),
                                       p["min_samples_leaf"] + 26)}
    q2 = RandomizedSearchCV(model(), tor, n_iter=20, cv=cv,
                            scoring="roc_auc", random_state=1,
                            n_jobs=1).fit(X_ish, y_ish)
    print(f"  2-bosqich (20): {q2.best_score_:.4f} "
          f"({q2.best_score_ - q1.best_score_:+.4f})")

    print("\n=== 3. Bir bosqichli 50 bilan taqqoslash ===")
    q50 = RandomizedSearchCV(model(), keng, n_iter=50, cv=cv,
                             scoring="roc_auc", random_state=0,
                             n_jobs=1).fit(X_ish, y_ish)
    h_ikki = roc_auc_score(y_haq,
                           q2.best_estimator_.predict_proba(X_haq)[:, 1])
    h_bir = roc_auc_score(y_haq,
                          q50.best_estimator_.predict_proba(X_haq)[:, 1])
    print(f"  {'usul':<26} {'CV ball':>9} {'haqiqiy':>9}")
    print(f"  {'bir bosqich (50)':<26} {q50.best_score_:>9.4f} "
          f"{h_bir:>9.4f}")
    print(f"  {'ikki bosqich (30+20)':<26} {q2.best_score_:>9.4f} "
          f"{h_ikki:>9.4f}")

    print("\n=== 4. Qaytish nuqtasi ===")
    print(f"  {'n_iter':>8} {'CV ball':>9} {'oldingidan o_sish':>19}")
    oldingi_ball = None
    for n_iter in [5, 10, 20, 40]:
        q = RandomizedSearchCV(model(), keng, n_iter=n_iter, cv=cv,
                               scoring="roc_auc", random_state=0,
                               n_jobs=1).fit(X_ish, y_ish)
        osish = ("-" if oldingi_ball is None
                 else f"{q.best_score_ - oldingi_ball:+.4f}")
        print(f"  {n_iter:>8} {q.best_score_:>9.4f} {osish:>19}")
        oldingi_ball = q.best_score_
    print("  o'sish kamaya boshlagan joyda to'xtang")
    print("  ⭐ 20-40 nomzod ko'p hollarda yetarli")


if __name__ == "__main__":
    main()

=== 1. Byudjet va natija ===
    n_iter   CV ball   haqiqiy   modellar
        10    0.8718    0.8755         30
        20    0.8760    0.8754         60
        40    0.8771    0.8727        120

=== 2. Ikki bosqichli qidiruv (30 + 20 = 50) ===
  1-bosqich (30): 0.8771
    lr=0.1078, barglar=16, leaf=45
  2-bosqich (20): 0.8760 (-0.0012)

=== 3. Bir bosqichli 50 bilan taqqoslash ===
  usul                         CV ball   haqiqiy
  bir bosqich (50)              0.8771    0.8727
  ikki bosqich (30+20)          0.8760    0.8754

=== 4. Qaytish nuqtasi ===
    n_iter   CV ball   oldingidan o_sish
         5    0.8703                   -
        10    0.8718             +0.0015
        20    0.8760             +0.0042
        40    0.8771             +0.0011
  o'sish kamaya boshlagan joyda to'xtang
  ⭐ 20-40 nomzod ko'p hollarda yetarli

GridSearchCV(m, {5 ta parametr x 5 qiymat})   # 3125            # ⚠️
RandomizedSearchCV(m, taqsimot, n_iter=40)                       # ✅

{"max_features": uniform(0.1, 0.8)}    # [0.1, 0.9]             # ⚠️
{"max_features": uniform(0.1, 0.7)}    # [0.1, 0.8]             # ✅

print(q.best_params_); exit()                                    # ⚠️
pd.DataFrame(q.cv_results_).sort_values("rank_test_score")       # ✅

X = StandardScaler().fit_transform(X); GridSearchCV(SVC(), ...)  # ⚠️
GridSearchCV(make_pipeline(StandardScaler(), SVC()), ...)        # ✅

print(f"Natija: {q.best_score_}")                                # ⚠️
print(f"Natija: {roc_auc_score(y_test, ...)}")                   # ✅

RandomizedSearchCV(m, t, n_iter=40)      # takrorlanmaydi        # ⚠️
RandomizedSearchCV(m, t, n_iter=40, random_state=0)              # ✅

RandomizedSearchCV(RandomForestClassifier(n_jobs=-1), ...,
                   n_jobs=-1)                                    # ⚠️
RandomizedSearchCV(RandomForestClassifier(n_jobs=1), ...,
                   n_jobs=-1)                                    # ✅

1.  # GridSearchCV qachon yaxshi?
2.  # nima uchun tasodifiy qidiruv ustun?
3.  # uniform(0.2, 0.5) oralig'i?
4.  # randint(5, 60) eng katta qiymati?
5.  # loguniform nima uchun?
6.  # cv_results_ dagi muhim ustunlar?
7.  # rank_test_score nima?
8.  # return_train_score nima beradi?
9.  # refit nima qiladi?
10. # amaliy n_iter?
11. # ikki bosqichli qidiruv?
12. # ichma-ich parallellik muammosi?

1.  GridSearchCV(m, {5 ta parametr x 5 qiymat})

2.  {"max_features": uniform(0.1, 0.8)}   # [0.1, 0.8] kerak

3.  print(q.best_params_); exit()

4.  print(f"Natija: {q.best_score_}")

5.  RandomizedSearchCV(RandomForestClassifier(n_jobs=-1), ..., n_jobs=-1)