18.9-dars: Metrika tanlash

SAVOL 1: model chiqishi qanday ishlatiladi?
  reyting tuzish (top-N ni ko'rib chiqish)  -> AUC, AP, precision@k
  ha/yo'q qaror                             -> F1, balanced accuracy
  ehtimollikning o'zi kerak                 -> log loss, Brier
  son bashorat qilish                       -> MAE, RMSE, MAPE

SAVOL 2: xatolar narxi teng emasmi?
  ha -> og'irlikli metrika yoki maxsus funksiya

SAVOL 3: resurs cheklovi bormi?
  ha -> precision@k, recall@k

SAVOL 4: sinflar nomutanosibmi?
  ha -> accuracy ni TASHLANG

REYTING metrikalari (chegaradan MUSTAQIL):
  ROC AUC   - tasodifiy musbat tasodifiy manfiydan
              yuqori ball olish ehtimoli
              nomutanosiblikka sezgir EMAS (bu ham kamchilik)
  PR AUC (average_precision)
            - nomutanosib vazifada informativroq
            - bazaviy daraja = musbat ulushi

QAROR metrikalari (chegaraga BOG'LIQ):
  accuracy, precision, recall, F1, balanced_accuracy

EHTIMOLLIK metrikalari:
  log_loss  - kalibrlashni ham baholaydi, chetga sezgir
  Brier     - kvadratik, talqini osonroq

QOIDA: sozlashda REYTING metrikasi, yakunda QAROR metrikasi

Chegarani metrikadan emas, NARXDAN oling:

  FN narxi = C_fn  (o'tkazib yuborilgan firibgarlik)
  FP narxi = C_fp  (behuda tekshiruv)

  Optimal chegara:  p* = C_fp / (C_fp + C_fn)

  C_fn = 500, C_fp = 20  ->  p* = 20/520 = 0.038

RESURS CHEKLOVI bo'lsa:
  chegara emas, TOP-k ni oling
  k = kunlik quvvat

AMALIYOT:
  1. modelni AUC/AP bo'yicha sozlang
  2. chegarani validatsiyada narx bo'yicha toping
  3. testda narxni o'lchang

Satr sifatida:
  "roc_auc", "average_precision", "f1", "f1_macro",
  "balanced_accuracy", "neg_log_loss", "neg_brier_score",
  "neg_mean_absolute_error", "r2"

DIQQAT: sklearn KATTA qiymatni yaxshi deb hisoblaydi
  shuning uchun xatolar "neg_" bilan

Lug'at sifatida (bir nechta):
  scoring={"auc": "roc_auc", "ap": "average_precision"}
  refit="auc"

Ko'p sinfli uchun:
  "f1_macro"    - sinflar teng muhim
  "f1_weighted" - hajmga qarab
  "roc_auc_ovr" - har sinf qolganlarga qarshi

from sklearn.metrics import make_scorer

def foyda(y_true, y_pred, C_fn=500, C_fp=20):
    fn = ((y_true == 1) & (y_pred == 0)).sum()
    fp = ((y_true == 0) & (y_pred == 1)).sum()
    return -(C_fn * fn + C_fp * fp)          # KATTA = yaxshi

ball = make_scorer(foyda)                     # y_pred kerak

def ap_k(y_true, y_score, k=200):
    tartib = np.argsort(-y_score)[:k]
    return y_true[tartib].mean()

ball_k = make_scorer(ap_k, response_method="predict_proba")

BITTA metrika bilan sozlang (refit), lekin BIR NECHTASINI kuzating:

  scoring = {"auc": "roc_auc",
             "ap": "average_precision",
             "logloss": "neg_log_loss",
             "p@200": ball_k}
  refit = "ap"

NIMA UCHUN:
  - metrikalar zid bo'lsa bilib olasiz
  - hisobotda to'liq rasm bo'ladi
  - keyinchalik maqsad o'zgarsa qayta hisoblash shart emas

OGOHLANTIRISH: bir nechta metrikaga QARAB tanlov qilish
  (bugun AUC, ertaga F1) - bu validatsiyaga overfitting

import numpy as np
from sklearn.metrics import get_scorer_names, make_scorer

# reyting (chegaradan mustaqil) - sozlash uchun
"roc_auc", "average_precision"
# qaror (chegaraga bog'liq) - yakuniy hisobot uchun
"f1", "balanced_accuracy", "precision", "recall"
# ehtimollik
"neg_log_loss", "neg_brier_score"
# regressiya
"neg_mean_absolute_error", "neg_root_mean_squared_error", "r2"

def p_at_k(y_true, y_score, k=200):
    return float(y_true[np.argsort(-y_score)[:k]].mean())

skor = make_scorer(p_at_k, response_method="predict_proba", k=200)
chegara = C_fp / (C_fp + C_fn)          # narxdan
QOIDA: chiqish bilan nima qilinadi? · reyting bilan sozla ·
       chegarani narxdan hisobla · metrikani o'zgartirma

Reyting: ROC AUC, average_precision
Qaror: F1, balanced_accuracy, narx
Ehtimollik: log_loss, Brier
Chegara: p* = C_fp / (C_fp + C_fn)
Resurs cheklovi: precision@k

"""Turli metrikalar turli modelni tanlaydi (real numpy/sklearn)."""

import numpy as np
from sklearn.datasets import make_classification
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (accuracy_score, average_precision_score,
                             balanced_accuracy_score, brier_score_loss,
                             f1_score, log_loss, roc_auc_score)
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import GaussianNB
from sklearn.neighbors import KNeighborsClassifier
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler


def main() -> None:
    X, y = make_classification(n_samples=12000, n_features=20,
                               n_informative=7, n_redundant=5,
                               weights=[0.96, 0.04], flip_y=0.05,
                               class_sep=0.9, random_state=0)
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.4, stratify=y,
                                          random_state=0)
    print("=== 1. Ma'lumot ===")
    print(f"  o'quv {len(ytr)}, test {len(yte)}")
    print(f"  musbat ulushi: {y.mean():.2%}")

    modellar = {
        "logistik": make_pipeline(StandardScaler(),
                                  LogisticRegression(max_iter=2000)),
        "logistik (balanced)": make_pipeline(
            StandardScaler(),
            LogisticRegression(max_iter=2000, class_weight="balanced")),
        "KNN k=25": make_pipeline(StandardScaler(),
                                  KNeighborsClassifier(n_neighbors=25)),
        "GaussianNB": make_pipeline(StandardScaler(), GaussianNB()),
        "boosting": HistGradientBoostingClassifier(max_iter=200,
                                                   early_stopping=False,
                                                   random_state=0),
    }

    print("\n=== 2. Barcha metrikalar ===")
    natija = {}
    for nom, m in modellar.items():
        m.fit(Xtr, ytr)
        p = m.predict_proba(Xte)[:, 1]
        q = (p >= 0.5).astype(int)
        natija[nom] = {
            "accuracy": accuracy_score(yte, q),
            "bal_acc": balanced_accuracy_score(yte, q),
            "f1": f1_score(yte, q, zero_division=0),
            "auc": roc_auc_score(yte, p),
            "ap": average_precision_score(yte, p),
            "logloss": log_loss(yte, p),
            "brier": brier_score_loss(yte, p),
        }
    ustunlar = ["accuracy", "bal_acc", "f1", "auc", "ap", "logloss", "brier"]
    print("  " + f"{'model':<22}" + "".join(f"{u:>10}" for u in ustunlar))
    for nom, d in natija.items():
        print(f"  {nom:<22}" + "".join(f"{d[u]:>10.4f}" for u in ustunlar))

    print("\n=== 3. Har metrika qaysi modelni tanlaydi ===")
    print(f"  {'metrika':<12} {'eng yaxshi model':<24} {'qiymat':>9}")
    for u in ustunlar:
        kichik_yaxshi = u in ("logloss", "brier")
        eng = (min if kichik_yaxshi else max)(natija,
                                              key=lambda k: natija[k][u])
        print(f"  {u:<12} {eng:<24} {natija[eng][u]:>9.4f}")

    print("\n=== 4. accuracy nima uchun yaroqsiz ===")
    hamma_manfiy = float((yte == 0).mean())
    print(f"  'hech kim musbat emas' modeli aniqligi: {hamma_manfiy:.4f}")
    eng_acc = max(natija, key=lambda k: natija[k]["accuracy"])
    print(f"  eng yaxshi haqiqiy model aniqligi: "
          f"{natija[eng_acc]['accuracy']:.4f}")
    print(f"  farq: {natija[eng_acc]['accuracy'] - hamma_manfiy:+.4f}")
    print(f"  ammo AP bo'yicha farq: "
          f"{max(d['ap'] for d in natija.values()) - y.mean():+.4f}")
    print("  ⭐ Nomutanosib vazifada accuracy hech narsa aytmaydi")


if __name__ == "__main__":
    main()

=== 1. Ma'lumot ===
  o'quv 7200, test 4800
  musbat ulushi: 6.27%

=== 2. Barcha metrikalar ===
  model                   accuracy   bal_acc        f1       auc        ap   logloss     brier
  logistik                  0.9375    0.5017    0.0066    0.6688    0.2845    0.2169    0.0545
  logistik (balanced)       0.6179    0.6334    0.1761    0.6780    0.2047    0.6480    0.2285
  KNN k=25                  0.9406    0.5281    0.1066    0.7763    0.3792    0.4710    0.0496
  GaussianNB                0.9402    0.5388    0.1433    0.6713    0.2629    0.2145    0.0530
  boosting                  0.9527    0.6338    0.4165    0.8007    0.5405    0.2032    0.0416

=== 3. Har metrika qaysi modelni tanlaydi ===
  metrika      eng yaxshi model            qiymat
  accuracy     boosting                    0.9527
  bal_acc      boosting                    0.6338
  f1           boosting                    0.4165
  auc          boosting                    0.8007
  ap           boosting                    0.5405
  logloss      boosting                    0.2032
  brier        boosting                    0.0416

=== 4. accuracy nima uchun yaroqsiz ===
  'hech kim musbat emas' modeli aniqligi: 0.9373
  eng yaxshi haqiqiy model aniqligi: 0.9527
  farq: +0.0154
  ammo AP bo'yicha farq: +0.4779
  ⭐ Nomutanosib vazifada accuracy hech narsa aytmaydi

"""Optimal chegara va uning nazariy qiymati (real numpy/sklearn)."""

import numpy as np
from sklearn.calibration import CalibratedClassifierCV
from sklearn.datasets import make_classification
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.model_selection import train_test_split


def main() -> None:
    X, y = make_classification(n_samples=24000, n_features=20,
                               n_informative=7, n_redundant=5,
                               weights=[0.95, 0.05], flip_y=0.06,
                               class_sep=0.9, random_state=0)
    Xtr, Xqol, ytr, yqol = train_test_split(X, y, test_size=0.5,
                                            stratify=y, random_state=0)
    Xval, Xte, yval, yte = train_test_split(Xqol, yqol, test_size=0.5,
                                            stratify=yqol, random_state=0)
    model = CalibratedClassifierCV(
        HistGradientBoostingClassifier(max_iter=200, early_stopping=False,
                                       random_state=0),
        method="isotonic", cv=3).fit(Xtr, ytr)
    p_val = model.predict_proba(Xval)[:, 1]
    p_te = model.predict_proba(Xte)[:, 1]

    C_fn, C_fp = 500.0, 20.0
    print("=== 1. Narxlar ===")
    print(f"  o'tkazib yuborilgan musbat (FN): {C_fn:.0f}")
    print(f"  behuda tekshiruv (FP): {C_fp:.0f}")
    nazariy = C_fp / (C_fp + C_fn)
    print(f"  nazariy optimal chegara: {nazariy:.4f}")

    def narx(y_true, p, chegara):
        q = (p >= chegara).astype(int)
        fn = int(((y_true == 1) & (q == 0)).sum())
        fp = int(((y_true == 0) & (q == 1)).sum())
        return C_fn * fn + C_fp * fp, fn, fp

    print("\n=== 2. Validatsiyada chegara qidiruvi ===")
    chegaralar = np.linspace(0.005, 0.5, 100)
    narxlar = [narx(yval, p_val, c)[0] for c in chegaralar]
    eng_c = float(chegaralar[int(np.argmin(narxlar))])
    print(f"  {'chegara':>9} {'narx':>12} {'FN':>6} {'FP':>7}")
    for c in [0.02, 0.038, 0.1, 0.3, 0.5]:
        n, fn, fp = narx(yval, p_val, c)
        print(f"  {c:>9.3f} {n:>12.0f} {fn:>6} {fp:>7}")
    print(f"  empirik eng yaxshi chegara: {eng_c:.4f}")
    print(f"  nazariy: {nazariy:.4f}")

    print("\n=== 3. Testda tekshirish ===")
    print(f"  {'chegara':<22} {'narx':>12} {'FN':>6} {'FP':>7}")
    for nom, c in [("sukut 0.5", 0.5), ("nazariy", nazariy),
                   ("validatsiyadan", eng_c)]:
        n, fn, fp = narx(yte, p_te, c)
        print(f"  {nom:<22} {n:>12.0f} {fn:>6} {fp:>7}")
    n05 = narx(yte, p_te, 0.5)[0]
    nopt = narx(yte, p_te, eng_c)[0]
    print(f"  tejash: {n05 - nopt:.0f} ({(n05 - nopt) / n05:.1%})")

    print("\n=== 4. Narxlar o'zgarsa chegara ham o'zgaradi ===")
    print(f"  {'C_fn':>7} {'C_fp':>6} {'nazariy chegara':>17} "
          f"{'testdagi FN':>12} {'testdagi FP':>12}")
    for c_fn, c_fp in [(100, 20), (500, 20), (2000, 20), (500, 100)]:
        ch = c_fp / (c_fp + c_fn)
        q = (p_te >= ch).astype(int)
        fn = int(((yte == 1) & (q == 0)).sum())
        fp = int(((yte == 0) & (q == 1)).sum())
        print(f"  {c_fn:>7} {c_fp:>6} {ch:>17.4f} {fn:>12} {fp:>12}")
    print("  ⭐ Chegara - biznes qarori, model parametri emas")


if __name__ == "__main__":
    main()

=== 1. Narxlar ===
  o'tkazib yuborilgan musbat (FN): 500
  behuda tekshiruv (FP): 20
  nazariy optimal chegara: 0.0385

=== 2. Validatsiyada chegara qidiruvi ===
    chegara         narx     FN      FP
      0.020       109220      1    5436
      0.038        96380     69    3094
      0.100        94340    175     342
      0.300       118000    233      75
      0.500       138460    276      23
  empirik eng yaxshi chegara: 0.0750
  nazariy: 0.0385

=== 3. Testda tekshirish ===
  chegara                        narx     FN      FP
  sukut 0.5                    150820    301      16
  nazariy                       99600    102    2430
  validatsiyadan               102840    186     492
  tejash: 47980 (31.8%)

=== 4. Narxlar o'zgarsa chegara ham o'zgaradi ===
     C_fn   C_fp   nazariy chegara  testdagi FN  testdagi FP
      100     20            0.1667          230          153
      500     20            0.0385          102         2430
     2000     20            0.0099            0         5528
      500    100            0.1667          230          153
  ⭐ Chegara - biznes qarori, model parametri emas

"""make_scorer va precision@k (real numpy/sklearn)."""

import numpy as np
from sklearn.datasets import make_classification
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import make_scorer, roc_auc_score
from sklearn.model_selection import (RandomizedSearchCV, StratifiedKFold,
                                     cross_val_score)
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from scipy.stats import loguniform, randint


def p_at_k(y_true, y_score, k: int = 200) -> float:
    """Eng yuqori k ta ball orasida musbatlar ulushi."""
    k = min(k, len(y_true))
    tartib = np.argsort(-y_score)[:k]
    return float(np.asarray(y_true)[tartib].mean())


def topilgan_ulush(y_true, y_score, k: int = 200) -> float:
    """Eng yuqori k ta ichida topilgan musbatlarning umumiy ulushi."""
    y_true = np.asarray(y_true)
    k = min(k, len(y_true))
    tartib = np.argsort(-y_score)[:k]
    jami = max(int(y_true.sum()), 1)
    return float(y_true[tartib].sum() / jami)


def main() -> None:
    X, y = make_classification(n_samples=6000, n_features=22,
                               n_informative=7, n_redundant=5,
                               weights=[0.94, 0.06], flip_y=0.07,
                               class_sep=0.9, random_state=0)
    cv = StratifiedKFold(4, shuffle=True, random_state=0)
    print("=== 1. Sozlama ===")
    print(f"  {len(y)} qator, musbat ulushi {y.mean():.2%}")
    print(f"  kunlik tekshiruv quvvati: 200 ta")

    skor_p200 = make_scorer(p_at_k, response_method="predict_proba", k=200)
    skor_topilgan = make_scorer(topilgan_ulush,
                                response_method="predict_proba", k=200)

    print("\n=== 2. Modellarni uch metrikada taqqoslash ===")
    modellar = {
        "logistik": make_pipeline(StandardScaler(),
                                  LogisticRegression(max_iter=2000)),
        "boosting sodda": HistGradientBoostingClassifier(
            max_leaf_nodes=8, max_iter=200, early_stopping=False,
            random_state=0),
        "boosting murakkab": HistGradientBoostingClassifier(
            max_leaf_nodes=63, min_samples_leaf=5, max_iter=300,
            early_stopping=False, random_state=0),
    }
    print(f"  {'model':<22} {'AUC':>9} {'p@200':>9} {'topilgan':>10}")
    ballar = {}
    for nom, m in modellar.items():
        auc = cross_val_score(m, X, y, cv=cv, scoring="roc_auc").mean()
        p200 = cross_val_score(m, X, y, cv=cv, scoring=skor_p200).mean()
        top = cross_val_score(m, X, y, cv=cv, scoring=skor_topilgan).mean()
        ballar[nom] = (auc, p200, top)
        print(f"  {nom:<22} {auc:>9.4f} {p200:>9.4f} {top:>10.4f}")

    print("\n=== 3. Metrikalar turli modelni tanlaydimi ===")
    for i, nom_metrika in enumerate(["AUC", "p@200", "topilgan ulush"]):
        eng = max(ballar, key=lambda k: ballar[k][i])
        print(f"  {nom_metrika:<16} -> {eng}")

    print("\n=== 4. Sozlash metrikasi natijaga ta'siri ===")
    taqsimot = {"learning_rate": loguniform(0.02, 0.4),
                "max_leaf_nodes": randint(4, 64),
                "min_samples_leaf": randint(5, 100)}
    print(f"  {'sozlash metrikasi':<20} {'AUC':>9} {'p@200':>9} "
          f"{'topilgan':>10}")
    for nom, skor in [("roc_auc", "roc_auc"), ("p@200", skor_p200)]:
        q = RandomizedSearchCV(
            HistGradientBoostingClassifier(max_iter=200,
                                           early_stopping=False,
                                           random_state=0),
            taqsimot, n_iter=12, cv=cv, scoring=skor, random_state=0,
            n_jobs=1).fit(X, y)
        m = q.best_estimator_
        auc = cross_val_score(m, X, y, cv=cv, scoring="roc_auc").mean()
        p200 = cross_val_score(m, X, y, cv=cv, scoring=skor_p200).mean()
        top = cross_val_score(m, X, y, cv=cv, scoring=skor_topilgan).mean()
        print(f"  {nom:<20} {auc:>9.4f} {p200:>9.4f} {top:>10.4f}")
    print("  ⭐ Qaysi metrikani sozlasangiz, shuni olasiz")


if __name__ == "__main__":
    main()

=== 1. Sozlama ===
  6000 qator, musbat ulushi 9.23%
  kunlik tekshiruv quvvati: 200 ta

=== 2. Modellarni uch metrikada taqqoslash ===
  model                        AUC     p@200   topilgan
  logistik                  0.6915    0.2000     0.2888
  boosting sodda            0.7406    0.3337     0.4819
  boosting murakkab         0.7605    0.3675     0.5307

=== 3. Metrikalar turli modelni tanlaydimi ===
  AUC              -> boosting murakkab
  p@200            -> boosting murakkab
  topilgan ulush   -> boosting murakkab

=== 4. Sozlash metrikasi natijaga ta'siri ===
  sozlash metrikasi          AUC     p@200   topilgan
  roc_auc                 0.7627    0.3588     0.5180
  p@200                   0.7627    0.3588     0.5180
  ⭐ Qaysi metrikani sozlasangiz, shuni olasiz

"""Bitta sozlash, ko'p kuzatuv (real numpy/pandas/sklearn)."""

import numpy as np
import pandas as pd
from scipy.stats import loguniform, randint
from sklearn.datasets import make_classification
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.metrics import make_scorer
from sklearn.model_selection import RandomizedSearchCV, StratifiedKFold


def p_at_k(y_true, y_score, k: int = 150) -> float:
    k = min(k, len(y_true))
    tartib = np.argsort(-y_score)[:k]
    return float(np.asarray(y_true)[tartib].mean())


def main() -> None:
    X, y = make_classification(n_samples=7000, n_features=22,
                               n_informative=7, n_redundant=5,
                               weights=[0.93, 0.07], flip_y=0.08,
                               class_sep=0.9, random_state=0)
    cv = StratifiedKFold(4, shuffle=True, random_state=0)
    skorlar = {
        "auc": "roc_auc",
        "ap": "average_precision",
        "logloss": "neg_log_loss",
        "brier": "neg_brier_score",
        "p150": make_scorer(p_at_k, response_method="predict_proba", k=150),
    }
    taqsimot = {"learning_rate": loguniform(0.02, 0.4),
                "max_leaf_nodes": randint(4, 64),
                "min_samples_leaf": randint(5, 120)}
    q = RandomizedSearchCV(
        HistGradientBoostingClassifier(max_iter=200, early_stopping=False,
                                       random_state=0),
        taqsimot, n_iter=20, cv=cv, scoring=skorlar, refit="ap",
        random_state=0, n_jobs=1).fit(X, y)

    natija = pd.DataFrame(q.cv_results_)
    print("=== 1. AP bo'yicha eng yaxshi 5 nomzod ===")
    top = natija.nsmallest(5, "rank_test_ap")
    print(f"  {'o_rin':>6} {'ap':>8} {'auc':>8} {'logloss':>9} "
          f"{'brier':>9} {'p150':>8}")
    for _, r in top.iterrows():
        print(f"  {int(r['rank_test_ap']):>6} {r['mean_test_ap']:>8.4f} "
              f"{r['mean_test_auc']:>8.4f} {-r['mean_test_logloss']:>9.4f} "
              f"{-r['mean_test_brier']:>9.4f} {r['mean_test_p150']:>8.4f}")

    print("\n=== 2. Har metrika bo'yicha g'olib ===")
    print(f"  {'metrika':<10} {'g_olib indeksi':>16} {'AP o_rni':>11}")
    for nom in skorlar:
        idx = int(natija[f"rank_test_{nom}"].idxmin())
        print(f"  {nom:<10} {idx:>16} "
              f"{int(natija.loc[idx, 'rank_test_ap']):>11}")

    print("\n=== 3. Metrikalar orasidagi korrelyatsiya ===")
    nomlar = list(skorlar)
    print("  " + " " * 10 + "".join(f"{n:>10}" for n in nomlar))
    for a in nomlar:
        qator = f"  {a:<10}"
        for b in nomlar:
            k = float(natija[f"mean_test_{a}"].corr(natija[f"mean_test_{b}"]))
            qator += f"{k:>10.3f}"
        print(qator)

    print("\n=== 4. Zidlik bormi ===")
    ap_eng = int(natija["rank_test_ap"].idxmin())
    ll_eng = int(natija["rank_test_logloss"].idxmin())
    print(f"  AP g'olibi: {natija.loc[ap_eng, 'params']}")
    print(f"  logloss g'olibi: {natija.loc[ll_eng, 'params']}")
    print(f"  bir xilmi: {ap_eng == ll_eng}")
    print(f"  AP g'olibining logloss o'rni: "
          f"{int(natija.loc[ap_eng, 'rank_test_logloss'])}/{len(natija)}")
    print(f"  logloss g'olibining AP o'rni: "
          f"{int(natija.loc[ll_eng, 'rank_test_ap'])}/{len(natija)}")
    print("  reyting metrikasi va kalibrlash metrikasi mos kelmasligi mumkin")
    print("  ⭐ Bittasi bilan sozlang, hammasini hisobot qiling")


if __name__ == "__main__":
    main()

=== 1. AP bo'yicha eng yaxshi 5 nomzod ===
   o_rin       ap      auc   logloss     brier     p150
       1   0.5330   0.7874    0.2591    0.0652   0.5517
       2   0.5320   0.7772    0.5983    0.0698   0.5583
       3   0.5313   0.7823    0.3873    0.0687   0.5517
       4   0.5306   0.7796    0.7114    0.0707   0.5500
       5   0.5300   0.7756    0.7493    0.0706   0.5483

=== 2. Har metrika bo'yicha g'olib ===
  metrika      g_olib indeksi    AP o_rni
  auc                       8           9
  ap                       15           1
  logloss                   8           9
  brier                    15           1
  p150                     10           6

=== 3. Metrikalar orasidagi korrelyatsiya ===
                   auc        ap   logloss     brier      p150
  auc            1.000     0.760     0.261     0.648     0.658
  ap             0.760     1.000    -0.328     0.185     0.931
  logloss        0.261    -0.328     1.000     0.809    -0.301
  brier          0.648     0.185     0.809     1.000     0.190
  p150           0.658     0.931    -0.301     0.190     1.000

=== 4. Zidlik bormi ===
  AP g'olibi: {'learning_rate': np.float64(0.03495054590177218), 'max_leaf_nodes': 54, 'min_samples_leaf': 60}
  logloss g'olibi: {'learning_rate': np.float64(0.02124881606617806), 'max_leaf_nodes': 55, 'min_samples_leaf': 74}
  bir xilmi: False
  AP g'olibining logloss o'rni: 10/20
  logloss g'olibining AP o'rni: 9/20
  reyting metrikasi va kalibrlash metrikasi mos kelmasligi mumkin
  ⭐ Bittasi bilan sozlang, hammasini hisobot qiling

cross_val_score(m, X, y, scoring="accuracy")    # 2% musbat      # ⚠️
cross_val_score(m, X, y, scoring="average_precision")            # ✅

chegara = eng_yaxshi_f1_chegarasi                                # ⚠️
chegara = C_fp / (C_fp + C_fn)                                   # ✅

cross_val_score(m, X, y, scoring="mean_absolute_error")  # xato  # ⚠️
-cross_val_score(m, X, y, scoring="neg_mean_absolute_error")     # ✅

cross_val_score(m, X, y, scoring="f1")     # ikkilik uchun       # ⚠️
cross_val_score(m, X, y, scoring="f1_macro")                     # ✅

make_scorer(p_at_k)                 # predict beradi (0/1)       # ⚠️
make_scorer(p_at_k, response_method="predict_proba")             # ✅

# AUC yomon chiqdi -> F1 ga o'tamiz                              # ⚠️
# metrikani boshida tanlang va o'zgartirmang                     # ✅

chegara = eng_yaxshi(yte, p_te)                                  # ⚠️
chegara = eng_yaxshi(yval, p_val)   # validatsiyada              # ✅

1.  # metrika tanlashning birinchi savoli?
2.  # reyting metrikalari?
3.  # qaror metrikalari?
4.  # nomutanosibda qaysi biri?
5.  # optimal chegara formulasi?
6.  # C_fn=500, C_fp=20 da chegara?
7.  # resurs cheklovida qaysi metrika?
8.  # neg_ nima uchun?
9.  # ko'p sinfda f1 qanday?
10. # make_scorer da response_method?
11. # refit nima qiladi?
12. # metrikani qachon tanlash?

1.  cross_val_score(m, X, y, scoring="accuracy")   # 2% musbat

2.  chegara = eng_yaxshi_f1_chegarasi

3.  cross_val_score(m, X, y, scoring="mean_absolute_error")

4.  cross_val_score(m, X, y, scoring="f1")   # 5 sinf

5.  chegara = eng_yaxshi(yte, p_te)