IlmHamroh
Data Science va sun'iy intellekt/Regressiya10/12-dars19 daqiqa
Mundarija (22)

13.10-dars: Logistik regressiya

13-QISM — REGRESSIYA · 10-dars

1. Kirish va motivatsiya

Maqsad son emas, ha/yo'q bo'lsa (mijoz ketadimi, tranzaksiya firibgarmi, bemor kasalmi), chiziqli regressiya to'g'ridan-to'g'ri ishlamaydi: u 0 dan kichik va 1 dan katta "ehtimol" chiqaradi. Logistik regressiya shu muammoni hal qiladi: chiziqli kombinatsiyani sigmoid orqali (0, 1) oralig'iga siqadi va ehtimol beradi.

Bu darsda: sigmoid va log-odds, nega kvadratik emas log-loss, koeffitsiyentlarni odds ratio sifatida talqin qilish, regularizatsiya (C parametri), ko'p sinfli variantlar, sklearn amaliyoti va chegara 12.7-bob bilan bog'liqlik.

Real vaziyat. Mikromoliya tashkiloti kredit qaytarilishini bashorat qilmoqchi. Daraxt ansambli AUC 0.79 berdi, logistik regressiya 0.77 — lekin regulyator har rad etish uchun sabab talab qiladi. Logistik regressiya tanlandi: "daromadingiz oyiga 1 mln ga oshsa, tasdiqlanish shansi 1.6 barobar ortadi" degan tushuntirish mumkin. 0.02 AUC talqin evaziga berildi.

Bu darsda logistik regressiyani o'rganamiz.

Bu darsda:

  • Sigmoid va log-odds
  • Log-loss
  • Odds ratio talqini
  • Regularizatsiya va C
  • Ko'p sinfli variantlar
  • Kalibrlash va chegara
  • Tuzoqlar
  • Amaliy: kredit modeli

ℹ Misollar real numpy/pandas/sklearn bilan (Python 3.14).

2. Nazariya — chuqur tushuntirish

2.1. Sigmoid va log-odds

text
z = w0 + w1x1 + ... + wpxp          (chiziqli qism — "log-odds")
p = sigmoid(z) = 1 / (1 + e^(-z))   (ehtimol, (0,1) oralig'ida)

Teskari aloqa:
  odds = p / (1 - p) = e^z
  log-odds (logit) = log(p / (1 - p)) = z      ← MODEL SHU YERDA CHIZIQLI

z = 0  → p = 0.5
z = 2  → p = 0.88
z = -2 → p = 0.12

Logistik regressiya — log-odds shkalasida chiziqli model. Bu muhim: koeffitsiyentlar ehtimolga emas, log-oddsga ta'sir qiladi, shuning uchun bir xil koeffitsiyent turli nuqtalarda ehtimolni turlicha o'zgartiradi (0.5 atrofida ko'p, chekkalarda kam). Bu — chiziqli regressiyadan asosiy talqin farqi.

2.2. Log-loss

text
Nega kvadratik yo'qotish emas:
  · sigmoid bilan kvadratik yo'qotish QAVARIQ EMAS — lokal minimumlar
  · noto'g'ri ishonchli bashoratni yetarlicha jazolamaydi

Log-loss (cross-entropy):
  L = -(1/n) · sum [ y·log(p) + (1-y)·log(1-p) ]

  y=1 va p=0.99 → yo'qotish ≈ 0.01
  y=1 va p=0.01 → yo'qotish ≈ 4.6     ← ishonchli xato qattiq jazolanadi

Log-loss qavariq → gradient tushish yagona yechimga keladi (13.6)

Log-loss — logistik regressiyaning yo'qotishi va u maksimal ishonchlilik (maximum likelihood) tamoyilidan kelib chiqadi. Uning asosiy xossasi: model ishonchli va noto'g'ri bo'lsa, jarima keskin oshadi — shuning uchun logistik regressiya yaxshi kalibrlangan ehtimollar beradi 9.9-bob.

2.3. Odds ratio talqini

text
Koeffitsiyent w_j uchun:
  exp(w_j) = ODDS RATIO — "x_j bir birlikka oshsa, odds necha barobar o'zgaradi"

  w = 0.7  → exp(0.7) = 2.01 → odds 2 barobar oshadi
  w = -0.5 → exp(-0.5) = 0.61 → odds 39% kamayadi
  w = 0    → exp(0) = 1 → ta'sir yo'q

DIQQAT: odds ≠ ehtimol
  p = 0.10 → odds = 0.111;  odds 2 barobar oshsa → p = 0.182 (+8 punkt)
  p = 0.50 → odds = 1.0;    odds 2 barobar oshsa → p = 0.667 (+17 punkt)

Odds ratio — logistik regressiya talqinining standart tili (ayniqsa tibbiyot va moliyada). Lekin uni ehtimol o'zgarishi bilan chalkashtirmaslik kerak: bir xil odds ratio turli boshlang'ich ehtimolda turli punktli o'zgarish beradi. Biznes hisobotida ko'pincha ehtimol tilida gapirish tushunarliroq.

2.4. Regularizatsiya va C

python
LogisticRegression(penalty="l2", C=1.0, max_iter=1000)

C = 1 / alpha       ← TESKARI! (13.7 dagi alpha bilan aralashtirmang)
  C kichik → kuchli regularizatsiya
  C katta  → zaif regularizatsiya

penalty: "l2" (standart), "l1" (siyraklik — 13.8), "elasticnet" (solver="saga")
Masshtablash — MAJBURIY (13.7)

sklearn da logistik regressiya standart holda regularizatsiyalangan (C=1.0) — bu chiziqli regressiyadan farq qiladi va ko'pincha unutiladi. C — alpha ning teskarisi, uni logspace da CV bilan tanlang. To'liq ajraladigan ma'lumotda (perfect separation) regularizatsiyasiz koeffitsiyentlar cheksizga ketadi — regularizatsiya buni ham hal qiladi.

2.5. Ko'p sinfli variantlar

text
multinomial (softmax) — barcha sinflar uchun bitta model, ehtimollar yig'indisi 1
  sklearn da standart (multi_class="multinomial" yoki "auto")

one-vs-rest (OvR) — har sinf uchun alohida binar model
  ehtimollar normallashtiriladi, ba'zan noizchil

Odatda multinomial afzal; OvR — juda ko'p sinfda tezroq

Ko'p sinfda softmax (multinomial) tabiiy kengaytma: z_k har sinf uchun hisoblanadi va softmax ularni ehtimollarga aylantiradi. Bu neyron tarmoqlardagi chiqish qatlami bilan aynan bir xil (22-qism) — logistik regressiya aslida bir qatlamli tarmoq.

2.6. Kalibrlash va chegara

text
Logistik regressiya odatda YAXSHI KALIBRLANGAN (log-loss shuni ta'minlaydi)
  → predict_proba natijalarini ehtimol sifatida ishlatish mumkin 9.9-bob

Lekin: class_weight yoki resampling ishlatilsa — kalibrlash BUZILADI

Chegara: predict() 0.5 ishlatadi — bu kamdan-kam to'g'ri 12.7-bob
  narx/byudjetdan tanlang, validatsiyada

Logistik regressiyaning katta afzalligi — kalibrlangan ehtimollar: predict_proba natijasini to'g'ridan-to'g'ri qaror formulalarida ishlatish mumkin (9.9, 12.7). Daraxt ansambllari bunday kafolat bermaydi. Lekin class_weight="balanced" ishlatsangiz, ehtimollar siljiydi — kalibrlashni qayta tekshiring.

2.7. Tuzoqlar

Asosiy tuzoqlar: C ni alpha deb o'ylash (teskari); masshtablamaslik; koeffitsiyentni ehtimol o'zgarishi deb o'qish (u odds ratio); predict() ning 0.5 chegarasi 12.7-bob; max_iter yetmasligi (konvergensiya ogohlantirishi); to'liq ajralishda regularizatsiyasiz o'qitish; nomutanosib sinfda accuracy 12.7-bob; class_weight dan keyin ehtimollarni kalibrlangan deb hisoblash; ko'p sinfda average ni ko'rsatmaslik.

2.8. Chiziqli model — ehtimol uchun

Logistik regressiya chiziqli kombinatsiyani sigmoid orqali ehtimolga aylantiradi; model log-odds shkalasida chiziqli. Yo'qotish — log-loss (maksimal ishonchlilikdan): u qavariq va ishonchli xatoni qattiq jazolaydi, natijada ehtimollar kalibrlangan bo'ladi 9.9-bob. Koeffitsiyentlar odds ratio (exp(w)) sifatida o'qiladi — ehtimol o'zgarishi bilan chalkashtirmang. sklearn da model standart holda regularizatsiyalangan (C = 1/alpha), masshtablash majburiy, chegara esa alohida qaror 12.7-bob. Keyingi dars — robust va kvantil regressiya.

3. Tez ma'lumotnoma

python
import numpy as np
from sklearn.linear_model import LogisticRegression, LogisticRegressionCV
from sklearn.metrics import average_precision_score, log_loss, roc_auc_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

quvur = Pipeline([("sc", StandardScaler()),
                  ("m", LogisticRegression(C=1.0, max_iter=1000))]).fit(X_tr, y_tr)
p = quvur.predict_proba(X_te)[:, 1]

# talqin
w = quvur.named_steps["m"].coef_[0]
odds_ratio = np.exp(w)                      # "odds necha barobar"

# C ni tanlash
LogisticRegressionCV(Cs=np.logspace(-4, 4, 20), cv=5, max_iter=2000,
                     scoring="neg_log_loss")

# chegara — alohida qaror (12.7)
y_hat = (p >= chegara).astype(int)
QOIDA: masshtabla · C = 1/alpha · exp(w) = odds ratio · chegarani o'zing tanla

Logistik regressiya xulosasi

z = Xw · p = 1/(1+e^-z) · logit(p) = z (log-odds chiziqli)
Log-loss = -mean[y·log p + (1-y)·log(1-p)] — qavariq, kalibrlaydi
exp(w) = odds ratio ≠ ehtimol o'zgarishi
C = 1/alpha (teskari!) · masshtablash majburiy · chegara 0.5 emas

4. Batafsil misollar

Misollar real numpy/pandas/sklearn bilan (Python 3.14). Misollar bitta kredit ma'lumotidan foydalanadi.

Misol 1 — Sigmoid, log-odds va chiziqli regressiya bilan farq

python
"""Nega chiziqli regressiya klassifikatsiya uchun mos emas (real numpy/sklearn)."""

import numpy as np
from sklearn.linear_model import LinearRegression, LogisticRegression
from sklearn.metrics import log_loss, roc_auc_score


def main() -> None:
    rng = np.random.default_rng(3)
    n = 2000
    daromad = rng.lognormal(np.log(4_000_000), 0.5, n)
    z = -3.2 + 1.1 * (np.log(daromad) - np.log(4_000_000))
    y = (rng.random(n) < 1 / (1 + np.exp(-z))).astype(int)
    X = np.log(daromad).reshape(-1, 1) - np.log(4_000_000)

    print("=== 1. Ma'lumot ===")
    print(f"  {n} ariza, qaytarmaganlar ulushi: {y.mean():.2%}")

    print("\n=== 2. Chiziqli regressiya bilan ===")
    lin = LinearRegression().fit(X, y)
    b = lin.predict(X)
    print(f"  bashorat oralig'i: [{b.min():.3f}, {b.max():.3f}]")
    print(f"  0 dan kichik: {(b < 0).sum()} ta, 1 dan katta: {(b > 1).sum()} ta")
    print("  (bular ehtimol bo'la olmaydi)")

    print("\n=== 3. Logistik regressiya ===")
    log = LogisticRegression(max_iter=1000).fit(X, y)
    p = log.predict_proba(X)[:, 1]
    print(f"  bashorat oralig'i: [{p.min():.4f}, {p.max():.4f}]")
    print(f"  koeffitsiyent = {log.coef_[0][0]:.3f} (haqiqiy 1.1), "
          f"kesma = {log.intercept_[0]:.3f} (haqiqiy -3.2)")

    print("\n=== 4. Sigmoid jadvali ===")
    for z_val in [-3, -2, -1, 0, 1, 2, 3]:
        pr = 1 / (1 + np.exp(-z_val))
        print(f"  z = {z_val:+d}: p = {pr:.4f}, odds = {pr / (1 - pr):7.3f}, "
              f"logit = {np.log(pr / (1 - pr)):+.3f}")
    print(f"\n  AUC: chiziqli {roc_auc_score(y, b):.4f}, "
          f"logistik {roc_auc_score(y, p):.4f}")
    print(f"  log-loss: logistik {log_loss(y, p):.4f}")
    print("  ⭐ Reyting o'xshash, lekin faqat logistik ehtimol beradi")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Ma'lumot ===
  2000 ariza, qaytarmaganlar ulushi: 5.25%

=== 2. Chiziqli regressiya bilan ===
  bashorat oralig'i: [-0.057, 0.178]
  0 dan kichik: 96 ta, 1 dan katta: 0 ta
  (bular ehtimol bo'la olmaydi)

=== 3. Logistik regressiya ===
  bashorat oralig'i: [0.0053, 0.3576]
  koeffitsiyent = 1.288 (haqiqiy 1.1), kesma = -3.089 (haqiqiy -3.2)

=== 4. Sigmoid jadvali ===
  z = -3: p = 0.0474, odds =   0.050, logit = -3.000
  z = -2: p = 0.1192, odds =   0.135, logit = -2.000
  z = -1: p = 0.2689, odds =   0.368, logit = -1.000
  z = +0: p = 0.5000, odds =   1.000, logit = +0.000
  z = +1: p = 0.7311, odds =   2.718, logit = +1.000
  z = +2: p = 0.8808, odds =   7.389, logit = +2.000
  z = +3: p = 0.9526, odds =  20.086, logit = +3.000

  AUC: chiziqli 0.6762, logistik 0.6762
  log-loss: logistik 0.1952
  ⭐ Reyting o'xshash, lekin faqat logistik ehtimol beradi

Nima ko'rsatdi: 2.1, 2.2-bo'limlar.

Misol 2 — Odds ratio talqini

python
"""Koeffitsiyentlarni odds va ehtimol tilida o'qish (real pandas/sklearn)."""

import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler


def yarat(seed: int = 7, n: int = 6000) -> pd.DataFrame:
    rng = np.random.default_rng(seed)
    daromad = rng.lognormal(np.log(4_000_000), 0.45, n)
    yosh = rng.integers(21, 60, n).astype(float)
    kechikish = rng.poisson(0.4, n).astype(float)        # oldingi kechikishlar
    maqsad = rng.choice(["biznes", "iste'mol", "ta'lim"], n, p=[0.4, 0.45, 0.15])
    maqsad_qosh = pd.Series(maqsad).map({"biznes": 0.0, "iste'mol": 0.55,
                                         "ta'lim": -0.35}).to_numpy()
    z = (-2.4 - 0.9 * (np.log(daromad) - np.log(4_000_000)) - 0.02 * (yosh - 35)
         + 0.85 * kechikish + maqsad_qosh)
    y = (rng.random(n) < 1 / (1 + np.exp(-z))).astype(int)
    return pd.DataFrame({"daromad": daromad, "yosh": yosh, "kechikish": kechikish,
                         "maqsad": maqsad, "qaytarmadi": y})


def main() -> None:
    df = yarat()
    y = df["qaytarmadi"]
    son = ["daromad", "yosh", "kechikish"]
    X = df[son + ["maqsad"]].copy()
    X["daromad"] = np.log(X["daromad"])

    quvur = Pipeline([
        ("t", ColumnTransformer([
            ("son", StandardScaler(), son),
            ("kat", OneHotEncoder(handle_unknown="ignore", drop="first"), ["maqsad"]),
        ])),
        ("m", LogisticRegression(C=1e6, max_iter=2000)),      # deyarli jarimasiz
    ]).fit(X, y)

    print("=== 1. Sinf ulushi ===")
    print(f"  qaytarmaganlar: {y.mean():.2%}")

    print("\n=== 2. Koeffitsiyentlar va odds ratio ===")
    nomlar = quvur.named_steps["t"].get_feature_names_out()
    w = quvur.named_steps["m"].coef_[0]
    print(f"  {'belgi':<16} {'koef':>8} {'exp(koef)':>10}  talqin")
    for nom, v in zip(nomlar, w):
        talqin = ("odds {:.2f} barobar".format(np.exp(v)) if v > 0
                  else "odds {:.0%} kamayadi".format(1 - np.exp(v)))
        print(f"  {nom:<16} {v:>8.3f} {np.exp(v):>10.3f}  {talqin}")
    print("  (sonli belgilar standartlashtirilgan: 'bir SD o'zgarish')")

    print("\n=== 3. Odds ratio va ehtimol farqi ===")
    orat = 2.0
    for p0 in [0.02, 0.10, 0.30, 0.50]:
        odds0 = p0 / (1 - p0)
        p1 = (odds0 * orat) / (1 + odds0 * orat)
        print(f"  boshlang'ich p = {p0:.2f}: odds {orat}x → p = {p1:.3f} "
              f"(+{(p1 - p0) * 100:.1f} punkt)")
    print("  bir xil odds ratio — turli punktli o'zgarish")

    print("\n=== 4. Bitta ariza uchun tushuntirish ===")
    ariza = X.iloc[[0]]
    hissa = quvur.named_steps["t"].transform(ariza)[0] * w
    z = float(hissa.sum() + quvur.named_steps["m"].intercept_[0])
    print(f"  kesma: {quvur.named_steps['m'].intercept_[0]:+.3f}")
    for nom, h in zip(nomlar, hissa):
        if abs(h) > 0.01:
            print(f"  {nom:<16}: {h:+.3f}")
    print(f"  jami log-odds = {z:+.3f} → ehtimol = {1 / (1 + np.exp(-z)):.4f}")
    print(f"  model bashorati = {quvur.predict_proba(ariza)[0, 1]:.4f}")
    print("  ⭐ Har qaror uchun sababni ko'rsatish mumkin")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Sinf ulushi ===
  qaytarmaganlar: 14.15%

=== 2. Koeffitsiyentlar va odds ratio ===
  belgi                koef  exp(koef)  talqin
  son__daromad       -0.425      0.654  odds 35% kamayadi
  son__yosh          -0.223      0.800  odds 20% kamayadi
  son__kechikish      0.586      1.797  odds 1.80 barobar
  kat__maqsad_iste'mol    0.522      1.686  odds 1.69 barobar
  kat__maqsad_ta'lim   -0.456      0.634  odds 37% kamayadi
  (sonli belgilar standartlashtirilgan: 'bir SD o'zgarish')

=== 3. Odds ratio va ehtimol farqi ===
  boshlang'ich p = 0.02: odds 2.0x → p = 0.039 (+1.9 punkt)
  boshlang'ich p = 0.10: odds 2.0x → p = 0.182 (+8.2 punkt)
  boshlang'ich p = 0.30: odds 2.0x → p = 0.462 (+16.2 punkt)
  boshlang'ich p = 0.50: odds 2.0x → p = 0.667 (+16.7 punkt)
  bir xil odds ratio — turli punktli o'zgarish

=== 4. Bitta ariza uchun tushuntirish ===
  kesma: -2.200
  son__yosh       : +0.040
  son__kechikish  : +1.475
  jami log-odds = -0.691 → ehtimol = 0.3339
  model bashorati = 0.3339
  ⭐ Har qaror uchun sababni ko'rsatish mumkin

Nima ko'rsatdi: 2.3-bo'lim.

Misol 3 — Regularizatsiya, C va kalibrlash

python
"""C parametri, to'liq ajralish va ehtimol sifati (real numpy/sklearn)."""

import warnings

import numpy as np
from sklearn.calibration import calibration_curve
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import brier_score_loss, log_loss, roc_auc_score
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler


def main() -> None:
    rng = np.random.default_rng(11)
    n, p = 1200, 40
    X = rng.normal(0, 1, (n, p))
    haqiqiy = np.zeros(p)
    haqiqiy[:8] = rng.normal(0, 1.2, 8)
    z = -1.0 + X @ haqiqiy
    y = (rng.random(n) < 1 / (1 + np.exp(-z))).astype(int)
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0,
                                          stratify=y)

    print("=== 1. C ning ta'siri ===")
    print(f"  {'C':>8} {'test log-loss':>14} {'AUC':>7} {'||w||':>8}")
    for C in [0.001, 0.01, 0.1, 1, 10, 1000]:
        m = Pipeline([("sc", StandardScaler()),
                      ("m", LogisticRegression(C=C, max_iter=3000))]).fit(Xtr, ytr)
        pr = m.predict_proba(Xte)[:, 1]
        print(f"  {C:>8} {log_loss(yte, pr):>14.4f} {roc_auc_score(yte, pr):>7.3f} "
              f"{np.linalg.norm(m.named_steps['m'].coef_):>8.3f}")
    print("  (C kichik — kuchli jarima; C = 1/alpha)")

    print("\n=== 2. To'liq ajralish (perfect separation) ===")
    xs = np.array([[-2.0], [-1.0], [-0.5], [0.5], [1.0], [2.0]])
    ys = np.array([0, 0, 0, 1, 1, 1])
    with warnings.catch_warnings():
        warnings.simplefilter("ignore")
        zaif = LogisticRegression(C=1e10, max_iter=10_000).fit(xs, ys)
    kuchli = LogisticRegression(C=1.0, max_iter=10_000).fit(xs, ys)
    print(f"  jarimasiz koeffitsiyent: {zaif.coef_[0][0]:10.2f}  (cheksizga intiladi)")
    print(f"  C = 1 bilan:             {kuchli.coef_[0][0]:10.2f}")

    print("\n=== 3. Kalibrlash sifati ===")
    m = Pipeline([("sc", StandardScaler()),
                  ("m", LogisticRegression(C=1.0, max_iter=3000))]).fit(Xtr, ytr)
    pr = m.predict_proba(Xte)[:, 1]
    print(f"  Brier ball = {brier_score_loss(yte, pr):.4f} (past — yaxshi)")
    haqiqiy_u, bashorat_u = calibration_curve(yte, pr, n_bins=5, strategy="quantile")
    for b, h in zip(bashorat_u, haqiqiy_u):
        print(f"  bashorat {b:.3f} → haqiqiy ulush {h:.3f}")

    print("\n=== 4. class_weight kalibrlashni buzadi ===")
    mb = Pipeline([("sc", StandardScaler()),
                   ("m", LogisticRegression(C=1.0, max_iter=3000,
                                            class_weight="balanced"))]).fit(Xtr, ytr)
    prb = mb.predict_proba(Xte)[:, 1]
    print(f"  oddiy   : o'rtacha bashorat {pr.mean():.3f}, "
          f"haqiqiy ulush {yte.mean():.3f}, Brier {brier_score_loss(yte, pr):.4f}")
    print(f"  balanced: o'rtacha bashorat {prb.mean():.3f}, "
          f"AUC {roc_auc_score(yte, prb):.3f}, Brier {brier_score_loss(yte, prb):.4f}")
    print("  ⭐ Reyting saqlanadi, lekin ehtimollar siljiydi 9.9-bob")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. C ning ta'siri ===
         C  test log-loss     AUC    ||w||
     0.001         0.6026   0.900    0.237
      0.01         0.4527   0.906    1.017
       0.1         0.3842   0.907    2.078
         1         0.3817   0.906    2.577
        10         0.3824   0.906    2.658
      1000         0.3825   0.906    2.669
  (C kichik — kuchli jarima; C = 1/alpha)

=== 2. To'liq ajralish (perfect separation) ===
  jarimasiz koeffitsiyent:      15.26  (cheksizga intiladi)
  C = 1 bilan:                   1.18

=== 3. Kalibrlash sifati ===
  Brier ball = 0.1192 (past — yaxshi)
  bashorat 0.015 → haqiqiy ulush 0.014
  bashorat 0.086 → haqiqiy ulush 0.111
  bashorat 0.263 → haqiqiy ulush 0.278
  bashorat 0.621 → haqiqiy ulush 0.681
  bashorat 0.910 → haqiqiy ulush 0.917

=== 4. class_weight kalibrlashni buzadi ===
  oddiy   : o'rtacha bashorat 0.379, haqiqiy ulush 0.400, Brier 0.1192
  balanced: o'rtacha bashorat 0.426, AUC 0.906, Brier 0.1194
  ⭐ Reyting saqlanadi, lekin ehtimollar siljiydi (9.9)

Nima ko'rsatdi: 2.4, 2.6-bo'limlar.

Misol 4 — Ko'p sinfli logistik regressiya

python
"""Softmax va OvR (real numpy/sklearn)."""

import numpy as np
from sklearn.datasets import load_digits
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, f1_score, log_loss
from sklearn.model_selection import train_test_split
from sklearn.multiclass import OneVsRestClassifier
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler


def main() -> None:
    X, y = load_digits(return_X_y=True)
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0,
                                          stratify=y)

    print("=== 1. Vazifa ===")
    print(f"  {len(X)} rasm, {X.shape[1]} piksel, {len(np.unique(y))} sinf")

    print("\n=== 2. Multinomial (softmax) ===")
    soft = Pipeline([("sc", StandardScaler()),
                     ("m", LogisticRegression(max_iter=5000, C=0.1))]).fit(Xtr, ytr)
    ps = soft.predict_proba(Xte)
    print(f"  aniqlik = {accuracy_score(yte, soft.predict(Xte)):.4f}")
    print(f"  F1 macro = {f1_score(yte, soft.predict(Xte), average='macro'):.4f}")
    print(f"  log-loss = {log_loss(yte, ps):.4f}")
    print(f"  ehtimollar yig'indisi (birinchi 3 namuna): {ps[:3].sum(axis=1).round(6)}")

    print("\n=== 3. One-vs-Rest ===")
    ovr = OneVsRestClassifier(
        Pipeline([("sc", StandardScaler()),
                  ("m", LogisticRegression(max_iter=5000, C=0.1))])).fit(Xtr, ytr)
    po = ovr.predict_proba(Xte)
    print(f"  aniqlik = {accuracy_score(yte, ovr.predict(Xte)):.4f}")
    print(f"  F1 macro = {f1_score(yte, ovr.predict(Xte), average='macro'):.4f}")
    print(f"  log-loss = {log_loss(yte, po):.4f}")

    print("\n=== 4. Koeffitsiyentlar tuzilishi ===")
    w = soft.named_steps["m"].coef_
    print(f"  softmax koeffitsiyentlar shakli: {w.shape} (sinf × belgi)")
    print(f"  har sinf uchun o'z chiziqli funksiyasi z_k, keyin softmax")
    eng = np.abs(w).sum(axis=1)
    tartib = np.argsort(eng)[::-1][:3]
    for k in tartib:
        print(f"  sinf {k}: ||w|| = {np.linalg.norm(w[k]):.2f}")
    print("  ⭐ Softmax — neyron tarmoq chiqish qatlami bilan bir xil (22-qism)")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Vazifa ===
  1797 rasm, 64 piksel, 10 sinf

=== 2. Multinomial (softmax) ===
  aniqlik = 0.9722
  F1 macro = 0.9725
  log-loss = 0.1637
  ehtimollar yig'indisi (birinchi 3 namuna): [1. 1. 1.]

=== 3. One-vs-Rest ===
  aniqlik = 0.9648
  F1 macro = 0.9652
  log-loss = 0.2280

=== 4. Koeffitsiyentlar tuzilishi ===
  softmax koeffitsiyentlar shakli: (10, 64) (sinf × belgi)
  har sinf uchun o'z chiziqli funksiyasi z_k, keyin softmax
  sinf 4: ||w|| = 1.70
  sinf 2: ||w|| = 1.83
  sinf 1: ||w|| = 1.84
  ⭐ Softmax — neyron tarmoq chiqish qatlami bilan bir xil (22-qism)

Nima ko'rsatdi: 2.5-bo'lim.

5. To'g'ri va noto'g'ri tushunishlar

Noto'g'ri fikr To'g'risi
"Logistik regressiya — regressiya" Klassifikatsiya
"Koeffitsiyent — ehtimol o'zgarishi" Odds ratio
"C — regularizatsiya kuchi" Teskarisi
"sklearn da jarima yo'q" C=1.0 standart
"Masshtablash kerak emas" Majburiy
"predict() yetarli" Chegara — alohida qaror
"Odds = ehtimol" Har xil
"Kalibrlash har doim yaxshi" class_weight buzadi

6. Keng tarqalgan xatolar va yechimlari

1. Masshtablamaslik

python
LogisticRegression().fit(X, y)                                    # ⚠️
Pipeline([("sc", StandardScaler()), ("m", LogisticRegression())]) # ✅

2. C ni alpha deb tushunish

python
LogisticRegression(C=100)   # "kuchli jarima" deb o'ylab           # ⚠️
LogisticRegression(C=0.01)  # kuchli jarima                        # ✅

3. Koeffitsiyentni ehtimol deb o'qish

python
# "daromad koeffitsiyenti 0.7 — ehtimol 70% oshadi"               # ⚠️
# "odds exp(0.7) = 2.01 barobar oshadi"                           # ✅

4. 0.5 chegarasi

python
y_hat = model.predict(X_te)                                       # ⚠️
y_hat = (proba >= chegara).astype(int)   # 12.7                   # ✅

5. max_iter yetmasligi

python
LogisticRegression()                   # ConvergenceWarning        # ⚠️
LogisticRegression(max_iter=2000)                                 # ✅

6. To'liq ajralishda jarimasiz

python
LogisticRegression(penalty=None)                                  # ⚠️
LogisticRegression(C=1.0)                                         # ✅

7. class_weight dan keyin ehtimolga ishonish

python
proba = balanced_model.predict_proba(X)[:, 1]   # narx formulasida # ⚠️
# kalibrlashni qayta tekshiring 9.9-bob yoki chegarani moslang       # ✅

7. Integratsiya — bu bilim qayerda kerak bo'ladi

  • 9.9-dars (o'tilgan): Kalibrlash va qaror
  • 12.7-dars (o'tilgan): Klassifikatsiya metrikalari va chegara
  • 13.6-dars (o'tilgan): Gradient tushish
  • 13.7-13.8-darslar (o'tilgan): Regularizatsiya
  • Klassifikatsiya qismi: Boshqa algoritmlar bilan solishtirish
  • 22-qism: Neyron tarmoqlar (softmax)

8. Eng yaxshi amaliyotlar

  1. Pipeline + StandardScaler.

  2. C ni logspace'da CV bilan tanlang.

  3. Koeffitsiyentni odds ratio sifatida bering.

  4. Biznes hisobotida ehtimol tilida gapiring.

  5. Chegarani alohida tanlang.

  6. Kalibrlashni tekshiring.

  7. max_iter ni yetarli qo'ying.

  8. Ko'p sinfda macro metrikalarni bering.

9. Amaliy topshiriq

Vazifa 1: Bashorat qiling

python
1.  # sigmoid formulasi?
2.  # logit nima?
3.  # model qaysi shkalada chiziqli?
4.  # nega log-loss?
5.  # exp(w) nima?
6.  # odds va ehtimol farqi?
7.  # C nima?
8.  # C kichik bo'lsa?
9.  # masshtablash shartmi?
10. # to'liq ajralishda nima bo'ladi?
11. # multinomial va OvR farqi?
12. # class_weight nimani buzadi?
Javoblar
  1. 1/(1+e^(-z))
  2. log(p/(1-p))
  3. Log-odds
  4. Qavariq va kalibrlaydi
  5. Odds ratio
  6. odds = p/(1-p)
  7. 1/alpha
  8. Kuchli regularizatsiya
  9. Ha
  10. Koeffitsiyentlar cheksizga intiladi
  11. Bitta softmax / har sinfga binar
  12. Kalibrlashni

Vazifa 2: Xatolarni tuzating

python
1.  LogisticRegression().fit(X, y)   # masshtablanmagan

2.  LogisticRegression(C=0.001)   # "zaif jarima" deb

3.  # "koeffitsiyent 0.5 — ehtimol 50% oshadi"

4.  y_hat = model.predict(X_te)   # 2% musbat sinf

5.  LogisticRegression(penalty=None).fit(X_kichik, y_ajralgan)
Javoblar
python
1.  Pipeline([("sc", StandardScaler()), ("m", LogisticRegression())])

2.  LogisticRegression(C=1000)   # zaif jarima

3.  # "odds exp(0.5) = 1.65 barobar oshadi"

4.  y_hat = (proba >= chegara).astype(int)

5.  LogisticRegression(C=1.0)

Vazifa 3: Sigmoid

Modellang:

  1. Chiziqli va logistik
  2. Bashorat oralig'i
  3. Sigmoid jadvali
  4. AUC

Vazifa 4: Odds ratio

Modellang:

  1. Koeffitsiyentlar
  2. exp(w)
  3. Ehtimol o'zgarishi
  4. Bitta qaror tushuntirishi

Vazifa 5: C va kalibrlash

Modellang:

  1. C setkasi
  2. Log-loss
  3. Kalibrlash egri chizig'i
  4. class_weight ta'siri

Vazifa 6: Ko'p sinf

Modellang:

  1. Softmax va OvR
  2. Metrikalar
  3. Koeffitsiyentlar shakli
  4. Xulosa

Vazifa 7: O'ylash

Logistik regressiya 1950-yillardan beri ishlatiladi va bugungi kunda ham banklarda, tibbiyotda va sug'urtada asosiy model bo'lib qolmoqda — gradient boosting aniqroq bo'lsa ham. Bu nafaqat talqin masalasi. Yana qanday sabablar bor?

Javob

Qisqa javob: talqindan tashqari, logistik regressiya kalibrlangan ehtimol, barqarorlik, audit qilinishi va arzon qo'llab-quvvatlash beradi — bular tartibga solinadigan sohalarda aniqlikdan muhimroq bo'lishi mumkin.

1. Sabablar ro'yxati

Sabab Izoh
Kalibrlangan ehtimol Narx va zaxira formulalariga to'g'ridan-to'g'ri kiradi (9.9)
Monotonlik kafolati "Daromad oshsa, xavf kamayadi" — koeffitsiyent ishorasi kafolatlaydi
Barqarorlik Ma'lumot biroz o'zgarsa natija keskin o'zgarmaydi
Audit Har qaror qo'lda qayta hisoblanadi
Adolat tekshiruvi Koeffitsiyent darajasida nazorat qilish oson
Arzonlik Millisekundlarda qayta o'qitiladi, oddiy infratuzilma

2. Qachon boosting afzal

  • Aniqlik to'g'ridan-to'g'ri pulga aylanadi (reklama, tavsiya)
  • Ko'p nochiziqlik va o'zaro ta'sirlar bor
  • Tartibga solish talabi yo'q

3. Amaliy strategiya

  1. Ikkalasini ham o'qiting
  2. Farqni noaniqlik bilan o'lchang (11.1)
  3. Farq kichik bo'lsa — logistik regressiyani oling
  4. Boosting yutsa — SHAP bilan topilgan naqshlarni belgilarga aylantirib, logistik modelga qo'shing

4. Gibrid yondashuvlar

  • Boosting bilan belgi topish + logistik model bilan qaror
  • Logistik model + monotonlik cheklovlari
  • Segment bo'yicha alohida logistik modellar

5. Xulosa

  1. Aniqlik — yagona mezon emas
  2. Kalibrlash va barqarorlik ko'p sohada hal qiluvchi
  3. Talqin — qonuniy talab bo'lishi mumkin
  4. Ikki modelni solishtirib, ongli tanlov qiling

Nimani mustahkamlaydi: 2.3, 2.6-bo'limlar.

Xulosa

Bu darsda logistik regressiyani o'rgandik.

Eng muhim uch fikr:

  1. Log-odds shkalasida chiziqli. z = Xw chiziqli kombinatsiya sigmoid orqali (0, 1) ga siqiladi: p = 1/(1+e^(-z)), teskarisi esa logit(p) = log(p/(1-p)) = z. Chiziqli regressiya bu yerda ishlamaydi — u 0 dan kichik va 1 dan katta "ehtimollar" beradi.

  2. Log-loss va odds ratio. Yo'qotish log-loss (maksimal ishonchlilikdan): u qavariq (yagona yechim) va ishonchli xatoni qattiq jazolaydi, natijada ehtimollar kalibrlangan bo'ladi 9.9-bob. Koeffitsiyentlar exp(w) — odds ratio sifatida o'qiladi; uni ehtimol o'zgarishi bilan chalkashtirmang: bir xil odds ratio p = 0.02 va p = 0.5 da butunlay boshqa punktli o'zgarish beradi.

  3. Amaliy tafsilotlar. sklearn da model standart holda regularizatsiyalangan: C = 1/alpha (teskari!), masshtablash majburiy, max_iter ni yetarli qo'ying. To'liq ajralishda regularizatsiya koeffitsiyentlarning cheksizga ketishini to'xtatadi. predict() ning 0.5 chegarasi kamdan-kam to'g'ri — chegarani narx/byudjetdan tanlang 12.7-bob; class_weight="balanced" reytingni saqlaydi, lekin ehtimollarni siljitadi.

Keyingi darsda robust va kvantil regressiyani o'rganamiz: outlierlarga chidamli usullar, Huber, RANSAC va kvantil regressiya bilan bashorat intervallari.

Ulashish:Telegram'da

Izohlar (0)

Izoh yozish uchun kiring.

  • Hozircha izoh yo'q. Birinchi bo'ling!
13.10-dars: Logistik regressiya — IlmHamroh