IlmHamroh
Data Science va sun'iy intellekt/ML asoslari8/10-dars19 daqiqa
Mundarija (22)

12.8-dars: Regressiya metrikalari

12-QISM — MACHINE LEARNING ASOSLARI · 8-dars

1. Kirish va motivatsiya

Regressiyada "model qanchalik yaxshi?" degan savolga bitta son bilan javob berib bo'lmaydi. MAE o'rtacha xatoni beradi, RMSE katta xatolarni jazolaydi, R^2 bazaga nisbatan yaxshilanishni ko'rsatadi, MAPE foizda o'lchaydi. Ular turli savollarga javob beradi va ba'zan turli modelni tanlaydi.

Bu darsda: metrikalar oilasi va ularning xossalari, qaysi biri qachon (outlierlar, nol qiymatlar, nolga yaqin qiymatlar), R^2 tuzoqlari, baza bilan solishtirish, kvantil va narxga asoslangan metrikalar (asimmetrik xato), qoldiqlar tahlili 10.6-bob va natijani biznes tilida berish.

Real vaziyat. Logistika kompaniyasi yetkazib berish vaqtini bashorat qiladi. Model A: MAE 12 daqiqa, model B: MAE 11 daqiqa — B tanlandi. Keyin ma'lum bo'ldi: B ba'zan 3 soatga yanglishadi (RMSE 48 vs A da 19), va aynan bu holatlar mijoz shikoyatiga olib keladi. Bundan tashqari kech qolish erta yetkazishdan 5 marta qimmat — asimmetrik metrika kerak edi. Yakuniy tanlov: kvantil regressiya (q=0.8) va kech qolish jarimasi bilan baholash.

Bu darsda regressiya metrikalarini o'rganamiz.

Bu darsda:

  • MAE, RMSE, MedAE — farqlari
  • R^2 va uning tuzoqlari
  • MAPE, sMAPE, MSLE — nisbiy xatolar
  • Baza bilan solishtirish
  • Asimmetrik va kvantil metrikalar
  • Qoldiqlar tahlili
  • Metrika tuzoqlari
  • Amaliy: yetkazib berish vaqti modeli

ℹ Misollar real numpy/sklearn bilan (Python 3.14).

2. Nazariya — chuqur tushuntirish

2.1. MAE, RMSE, MedAE

text
MAE   = mean(|y - p|)          — o'rtacha absolyut xato; y birligida; outlierlarga chidamli
RMSE  = sqrt(mean((y - p)^2))  — katta xatolarni kuchli jazolaydi; y birligida
MSE   = mean((y - p)^2)        — optimallashtirish uchun (birligi — y^2)
MedAE = median(|y - p|)        — eng chidamli; "odatiy xato"

Har doim: RMSE >= MAE. Ularning nisbati — xato taqsimotining qiyshiqligi ko'rsatkichi
RMSE / MAE ~ 1.25  — normalga yaqin;  > 2 — bir necha katta xato bor

MAE — "o'rtacha necha birlik yanglishamiz" (talqin oson, biznes uchun qulay). RMSE — kvadrat tufayli katta xatolarni og'irroq jazolaydi: agar bitta katta xato qimmat bo'lsa (yetkazib berish, zaxira), RMSE to'g'riroq. MedAE — outlierlardan butunlay himoyalangan "odatiy" xato. Uchtasini birga berish eng ma'lumotli: RMSE/MAE nisbati xato taqsimoti haqida gapiradi.

2.2. R^2 va tuzoqlari

text
R^2 = 1 - SS_res / SS_tot,   SS_tot = sum((y - y_ort)^2)
    = "model o'rtacha bilan bashorat qilishdan qancha yaxshi"

R^2 = 1     mukammal
R^2 = 0     o'rtacha bilan bir xil
R^2 < 0     o'rtachadan YOMON (test to'plamida mumkin!)

Tuzoqlar:
  · y ning dispersiyasiga bog'liq — turli to'plamlar orasida solishtirib bo'lmaydi
  · kichik/bir xil segmentda past R^2 — model yomon degani emas
  · o'quvda belgilar qo'shilsa har doim o'sadi (adjusted R^2 — 13-qism)
  · vaqt qatorlarida aldamchi yuqori (trend)

R^2 — nisbiy metrika: bazaviy model (o'rtacha) bilan taqqoslash. Uning kuchi — birliksiz va "qancha dispersiya tushuntirildi" degan talqin. Kamchiligi: u ma'lumot tarqoqligiga bog'liq — bir xil model bir to'plamda R^2 = 0.9, boshqasida 0.3 berishi mumkin. Shuning uchun R^2 ni yakka emas, MAE/RMSE bilan birga bering.

2.3. Nisbiy xatolar: MAPE, sMAPE, MSLE

text
MAPE  = mean(|y - p| / |y|) × 100%     — foizda; y ~ 0 bo'lsa portlaydi; past bashoratni afzal ko'radi
sMAPE = simmetrik variant                — biroz yaxshiroq, lekin baribir nozik
MSLE  = mean((log1p(y) - log1p(p))^2)   — nisbiy xato; y >= 0; katta qiymatlarda yumshoq
WAPE  = sum|y - p| / sum|y|              — "vaznlangan MAPE"; nollarga chidamli, amaliyotda afzal

MAPE — biznesda mashhur ("10% xato"), lekin xavfli: y nolga yaqin bo'lsa cheksizga intiladi va u past bashoratlarni sistematik afzal ko'radi (kam bashorat maksimal 100% xato beradi, ortiq bashorat cheksiz). Talab bashoratida WAPE yoki MAE ni afzal ko'ring. MSLE — narx/talab kabi o'ng qiyshiq, musbat qiymatlarda foydali (nisbiy xatoni o'lchaydi).

2.4. Baza bilan solishtirish

text
Regressiya bazalari:
  · o'rtacha / median (DummyRegressor)
  · oxirgi qiymat (vaqt qatorida — "naive")
  · mavsumiy naive (bir yil oldingi shu kun)
  · oddiy qoida (narx = maydon × o'rtacha kvadrat metr narxi)

Yaxshilanish = (baza xatosi - model xatosi) / baza xatosi

Baza (12.1, 12.6) bo'lmasa metrika ma'nosiz: "MAE 12 daqiqa" yaxshimi? Agar oddiy o'rtacha 13 daqiqa bersa — model deyarli befoyda; 40 daqiqa bersa — zo'r. Vaqt qatorlarida naive (oxirgi qiymat) baza odatda kuchli va ko'p "murakkab" modellar undan yaxshi emas.

2.5. Asimmetrik va kvantil metrikalar

text
Kech qolish erta qolishdan a marta qimmat bo'lsa:
  narx = a × sum(max(0, y - p)) + sum(max(0, p - y))     # y > p — kam bashorat

Pinball (kvantil) yo'qotish, kvantil q uchun:
  L_q = mean( max(q(y - p), (q - 1)(y - p)) )
  q = 0.5 → MAE ga proporsional;  q = 0.9 → ortiqroq bashoratni rag'batlantiradi

sklearn: QuantileRegressor, GradientBoostingRegressor(loss="quantile", alpha=q),
         mean_pinball_loss

Ko'p vazifalarda xato narxi asimmetrik: zaxira yetmasligi ortiqcha zaxiradan qimmatroq (9.10 newsvendor), kech yetkazish erta yetkazishdan yomonroq. Bunday holda simmetrik MAE/RMSE noto'g'ri modelni tanlaydi. Yechim: pinball loss va kvantil regressiya — model o'rtachani emas, kerakli kvantilni bashorat qiladi.

2.6. Qoldiqlar tahlili

text
Bitta son yetarli emas — qoldiqlarga qarang (10.6, 12.6):
  1. qoldiq va bashorat grafigi — tuzilma bormi? (nochiziqlik)
  2. qoldiq taqsimoti — qiyshiqmi, og'ir dumlimi?
  3. segmentlar bo'yicha xato — qayerda yomon?
  4. y diapazoni bo'yicha xato — katta qiymatlarda o'sadimi? (geteroskedastiklik)
  5. eng katta 10 xato — nima umumiy?

Qoldiqlar tahlili — metrikadan ko'ra ko'proq ma'lumot beradi: xato qayerda to'planganini ko'rsatadi. Tipik topilmalar: model katta qiymatlarda tizimli kam bashorat qiladi (log-transformatsiya kerak), ma'lum segmentda yomon (alohida belgi yoki model kerak), qoldiqlarda egrilik bor (nochiziqli belgi kerak).

2.7. Metrika tuzoqlari

Asosiy tuzoqlar: R^2 ni yakka ishlatish; MAPE ni nolga yaqin qiymatlarda; bazani ko'rsatmaslik; simmetrik metrika asimmetrik narxda; o'quv metrikasini hisobot qilish 12.4-bob; metrikani log fazoda o'lchab, natijani asl birlikda e'lon qilish; RMSE va MAE ni birligini yozmaslik; noaniqlikni (CV SD yoki bootstrap CI — 11.8) bermaslik; outlier larni jim tashlab yuborish.

2.8. Metrika — vazifaning aksi

Regressiya metrikalari: MAE (o'rtacha xato, chidamli), RMSE (katta xatolarni jazolaydi), MedAE (odatiy xato), R^2 (bazaga nisbatan, lekin dispersiyaga bog'liq), MAPE/WAPE (foizda — nol qiymatlarda ehtiyot), MSLE (nisbiy, musbat qiymatlar). Xato narxi asimmetrik bo'lsa — pinball/kvantil yoki to'g'ridan-to'g'ri narx metrikasi. Har doim: baza, birlik, noaniqlik va qoldiqlar tahlili. Keyingi dars — pipeline va leakage: tayyorlashni to'g'ri qilish.

3. Tez ma'lumotnoma

python
import numpy as np
from sklearn.metrics import (mean_absolute_error, mean_absolute_percentage_error,
                             mean_pinball_loss, mean_squared_error, median_absolute_error,
                             r2_score)

mean_absolute_error(y, p)                    # MAE — y birligida
np.sqrt(mean_squared_error(y, p))            # RMSE
median_absolute_error(y, p)                  # MedAE
r2_score(y, p)                               # R^2 (test'da manfiy bo'lishi mumkin)
mean_absolute_percentage_error(y, p)         # MAPE (y ~ 0 da xavfli)
np.abs(y - p).sum() / np.abs(y).sum()        # WAPE — amaliy alternativa
mean_pinball_loss(y, p, alpha=0.8)           # asimmetrik

# baza
from sklearn.dummy import DummyRegressor
baza = DummyRegressor(strategy="mean").fit(X_tr, y_tr)

# asimmetrik narx
narx = 5 * np.maximum(0, y - p).sum() + np.maximum(0, p - y).sum()
QOIDA: MAE + RMSE + baza · birlikni yoz · asimmetriyani tekshir · qoldiqlarga qara

Metrikalar xulosasi

MAE — o'rtacha xato (chidamli) · RMSE — katta xatolar · MedAE — odatiy xato
R^2 — bazaga nisbatan (dispersiyaga bog'liq) · MAPE — foiz (nolda xavfli) · WAPE — afzal
Pinball/kvantil — asimmetrik narx · RMSE/MAE nisbati — qiyshiqlik ko'rsatkichi
Har doim: baza + birlik + noaniqlik + qoldiqlar

4. Batafsil misollar

Misollar real numpy/sklearn bilan (Python 3.14). Misollar bir xil yarat() generatoridan foydalanadi.

Misol 1 — MAE, RMSE, MedAE: bir xil model, turli baho

python
"""Metrikalarning outlierlarga munosabati (real numpy/sklearn)."""

import numpy as np
from sklearn.metrics import (mean_absolute_error, mean_squared_error,
                             median_absolute_error, r2_score)


def main() -> None:
    rng = np.random.default_rng(3)
    n = 2000
    y = rng.normal(45, 12, n)                       # yetkazib berish vaqti (daqiqa)

    print("=== 1. Ikki model: bir xil o'rtacha, turli xato taqsimoti ===")
    # A: bir tekis kichik xato
    a = y + rng.normal(0, 10, n)
    # B: ko'pincha aniqroq, lekin 3% holatda juda katta xato
    b = y + rng.normal(0, 6, n)
    katta = rng.choice(n, int(0.03 * n), replace=False)
    b[katta] += rng.normal(0, 90, len(katta))

    for nom, p in [("A (bir tekis)", a), ("B (ba'zan katta)", b)]:
        mae = mean_absolute_error(y, p)
        rmse = np.sqrt(mean_squared_error(y, p))
        med = median_absolute_error(y, p)
        print(f"  {nom:<17}: MAE {mae:5.2f}, RMSE {rmse:5.2f}, MedAE {med:5.2f}, "
              f"RMSE/MAE {rmse / mae:.2f}")

    print("\n=== 2. Qaysi model 'yaxshi'? — metrikaga bog'liq ===")
    print(f"  MAE bo'yicha:  {'B' if mean_absolute_error(y, b) < mean_absolute_error(y, a) else 'A'}")
    r_a = np.sqrt(mean_squared_error(y, a))
    r_b = np.sqrt(mean_squared_error(y, b))
    print(f"  RMSE bo'yicha: {'B' if r_b < r_a else 'A'}")
    print("  ⭐ Katta xato qimmat bo'lsa — RMSE to'g'ri tanlaydi")

    print("\n=== 3. Eng katta xatolar ===")
    for nom, p in [("A", a), ("B", b)]:
        xato = np.abs(y - p)
        print(f"  {nom}: median {np.median(xato):5.2f}, "
              f"95-p {np.percentile(xato, 95):6.2f}, maks {xato.max():6.2f}")

    print("\n=== 4. R^2 ===")
    print(f"  A: {r2_score(y, a):.3f}, B: {r2_score(y, b):.3f}")
    print("  (R^2 ham kvadratga asoslangan — RMSE bilan bir xil tartib beradi)")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Ikki model: bir xil o'rtacha, turli xato taqsimoti ===
  A (bir tekis)    : MAE  8.12, RMSE 10.10, MedAE  6.97, RMSE/MAE 1.24
  B (ba'zan katta) : MAE  7.12, RMSE 18.22, MedAE  4.33, RMSE/MAE 2.56

=== 2. Qaysi model 'yaxshi'? — metrikaga bog'liq ===
  MAE bo'yicha:  B
  RMSE bo'yicha: A
  ⭐ Katta xato qimmat bo'lsa — RMSE to'g'ri tanlaydi

=== 3. Eng katta xatolar ===
  A: median  6.97, 95-p  19.88, maks  32.94
  B: median  4.33, 95-p  13.64, maks 311.81

=== 4. R^2 ===
  A: 0.280, B: -1.341
  (R^2 ham kvadratga asoslangan — RMSE bilan bir xil tartib beradi)

Nima ko'rsatdi: 2.1-bo'lim.

Misol 2 — R^2 tuzoqlari va baza

python
"""R^2 ning dispersiyaga bog'liqligi va baza bilan solishtirish (real numpy/sklearn)."""

import numpy as np
from sklearn.dummy import DummyRegressor
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, r2_score
from sklearn.model_selection import train_test_split


def yarat(seed: int = 4, n: int = 3000):
    """Yetkazib berish vaqti: masofa, tirbandlik, buyurtmalar soni."""
    rng = np.random.default_rng(seed)
    masofa = rng.gamma(4, 1.6, n)
    tirband = rng.beta(2, 3, n)
    buyurtma = rng.poisson(3, n).astype(float)
    vaqt = 8 + 3.1 * masofa + 22 * tirband + 1.8 * buyurtma + rng.normal(0, 4.5, n)
    X = np.column_stack([masofa, tirband, buyurtma])
    return X, vaqt


def main() -> None:
    X, y = yarat()
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0)
    model = LinearRegression().fit(Xtr, ytr)
    p = model.predict(Xte)

    print("=== 1. To'liq to'plam ===")
    print(f"  R^2 = {r2_score(yte, p):.3f}, MAE = {mean_absolute_error(yte, p):.2f} daqiqa")

    print("\n=== 2. Tor segment (faqat qisqa masofa) ===")
    tor = Xte[:, 0] < np.percentile(X[:, 0], 25)
    print(f"  segment hajmi: {tor.sum()}")
    print(f"  R^2 = {r2_score(yte[tor], p[tor]):.3f}  ← keskin past")
    print(f"  MAE = {mean_absolute_error(yte[tor], p[tor]):.2f} daqiqa  ← deyarli o'zgarmadi")
    print("  ⭐ R^2 ma'lumot tarqoqligiga bog'liq, MAE esa emas")

    print("\n=== 3. Baza bilan solishtirish ===")
    baza = DummyRegressor(strategy="mean").fit(Xtr, ytr)
    pb = baza.predict(Xte)
    mae_b, mae_m = mean_absolute_error(yte, pb), mean_absolute_error(yte, p)
    print(f"  baza (o'rtacha): MAE {mae_b:.2f}, R^2 {r2_score(yte, pb):.3f}")
    print(f"  model          : MAE {mae_m:.2f}, R^2 {r2_score(yte, p):.3f}")
    print(f"  yaxshilanish: {(mae_b - mae_m) / mae_b:.1%}")

    print("\n=== 4. Manfiy R^2 ===")
    yomon = np.full_like(yte, ytr.mean() + 15)      # noto'g'ri siljigan model
    print(f"  siljigan model: R^2 = {r2_score(yte, yomon):.3f} (< 0 — o'rtachadan yomon)")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. To'liq to'plam ===
  R^2 = 0.861, MAE = 3.61 daqiqa

=== 2. Tor segment (faqat qisqa masofa) ===
  segment hajmi: 232
  R^2 = 0.507  ← keskin past
  MAE = 4.01 daqiqa  ← deyarli o'zgarmadi
  ⭐ R^2 ma'lumot tarqoqligiga bog'liq, MAE esa emas

=== 3. Baza bilan solishtirish ===
  baza (o'rtacha): MAE 9.60, R^2 -0.000
  model          : MAE 3.61, R^2 0.861
  yaxshilanish: 62.4%

=== 4. Manfiy R^2 ===
  siljigan model: R^2 = -1.520 (< 0 — o'rtachadan yomon)

Nima ko'rsatdi: 2.2, 2.4-bo'limlar.

Misol 3 — MAPE tuzog'i va WAPE

python
"""Nisbiy metrikalar: MAPE nega aldaydi (real numpy/sklearn)."""

import numpy as np
from sklearn.metrics import mean_absolute_error, mean_absolute_percentage_error


def main() -> None:
    rng = np.random.default_rng(11)

    print("=== 1. MAPE nolga yaqin qiymatlarda ===")
    y = np.array([0.5, 2.0, 50.0, 120.0, 800.0])
    p = y + np.array([2.0, 2.0, 2.0, 2.0, 2.0])     # har joyda bir xil xato: 2
    for yi, pi in zip(y, p):
        print(f"  haqiqiy {yi:6.1f}, bashorat {pi:6.1f}: "
              f"absolyut xato 2.0, foiz xato {2.0 / yi:7.1%}")
    print(f"  MAPE = {mean_absolute_percentage_error(y, p):.1%}  ← bitta kichik qiymat hukmron")
    print(f"  WAPE = {np.abs(y - p).sum() / np.abs(y).sum():.1%}  ← barqaror")

    print("\n=== 2. MAPE past bashoratni afzal ko'radi ===")
    y2 = np.array([100.0])
    print(f"  bashorat  50: MAPE {mean_absolute_percentage_error(y2, [50.0]):.0%}")
    print(f"  bashorat 150: MAPE {mean_absolute_percentage_error(y2, [150.0]):.0%}")
    print(f"  bashorat   0: MAPE {mean_absolute_percentage_error(y2, [0.0]):.0%}  "
          f"← kam bashorat maksimum 100%")
    print("  bashorat 500: MAPE 400% — ortiq bashorat cheksiz jazolanadi")

    print("\n=== 3. Real talab: bir xil MAE, turli MAPE ===")
    talab = rng.poisson(rng.gamma(5.0, 4.0, 500)).astype(float)
    kam = talab - 3                                 # tizimli kam bashorat
    ortiq = talab + 3                               # tizimli ortiq bashorat
    musbat = talab > 0
    print(f"  nol kunlar: {(talab == 0).mean():.1%} (MAPE ular uchun aniqlanmagan); "
          f"median talab {np.median(talab):.0f}")
    for nom, p3 in [("kam bashorat", kam), ("ortiq bashorat", ortiq)]:
        mape = np.mean(np.abs(talab[musbat] - p3[musbat]) / talab[musbat])
        print(f"  {nom:<15}: MAE {mean_absolute_error(talab, p3):5.2f}, "
              f"WAPE {np.abs(talab - p3).sum() / talab.sum():6.1%}, "
              f"MAPE {mape:8.1%}")
    print("  (MAE va WAPE bir xil, MAPE esa kam bashoratni afzal ko'radi)")
    print("  ⭐ Nollar bor joyda MAPE ishlatilmaydi — WAPE yoki MAE")

    print("\n=== 4. Log fazodagi metrika ===")
    y4 = rng.lognormal(4, 1.0, 1000)
    p4 = y4 * rng.lognormal(0, 0.3, 1000)
    print(f"  MAE = {mean_absolute_error(y4, p4):7.2f} (katta qiymatlar hukmron)")
    print(f"  MSLE = {np.mean((np.log1p(y4) - np.log1p(p4)) ** 2):.4f} (nisbiy xato)")
    print("  ⭐ O'ng qiyshiq y uchun nisbiy metrika adolatliroq")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. MAPE nolga yaqin qiymatlarda ===
  haqiqiy    0.5, bashorat    2.5: absolyut xato 2.0, foiz xato  400.0%
  haqiqiy    2.0, bashorat    4.0: absolyut xato 2.0, foiz xato  100.0%
  haqiqiy   50.0, bashorat   52.0: absolyut xato 2.0, foiz xato    4.0%
  haqiqiy  120.0, bashorat  122.0: absolyut xato 2.0, foiz xato    1.7%
  haqiqiy  800.0, bashorat  802.0: absolyut xato 2.0, foiz xato    0.2%
  MAPE = 101.2%  ← bitta kichik qiymat hukmron
  WAPE = 1.0%  ← barqaror

=== 2. MAPE past bashoratni afzal ko'radi ===
  bashorat  50: MAPE 50%
  bashorat 150: MAPE 50%
  bashorat   0: MAPE 100%  ← kam bashorat maksimum 100%
  bashorat 500: MAPE 400% — ortiq bashorat cheksiz jazolanadi

=== 3. Real talab: bir xil MAE, turli MAPE ===
  nol kunlar: 0.0% (MAPE ular uchun aniqlanmagan); median talab 20
  kam bashorat   : MAE  3.00, WAPE  14.7%, MAPE    20.7%
  ortiq bashorat : MAE  3.00, WAPE  14.7%, MAPE    20.7%
  (MAE va WAPE bir xil, MAPE esa kam bashoratni afzal ko'radi)
  ⭐ Nollar bor joyda MAPE ishlatilmaydi — WAPE yoki MAE

=== 4. Log fazodagi metrika ===
  MAE =   22.86 (katta qiymatlar hukmron)
  MSLE = 0.0815 (nisbiy xato)
  ⭐ O'ng qiyshiq y uchun nisbiy metrika adolatliroq

Nima ko'rsatdi: 2.3-bo'lim.

Misol 4 — Asimmetrik narx va kvantil regressiya

python
"""Kech qolish qimmat bo'lganda: pinball loss va kvantil model (real numpy/sklearn)."""

import numpy as np
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_pinball_loss
from sklearn.model_selection import train_test_split


def yarat(seed: int = 4, n: int = 3000):
    rng = np.random.default_rng(seed)
    masofa = rng.gamma(4, 1.6, n)
    tirband = rng.beta(2, 3, n)
    buyurtma = rng.poisson(3, n).astype(float)
    vaqt = 8 + 3.1 * masofa + 22 * tirband + 1.8 * buyurtma + rng.normal(0, 4.5, n)
    X = np.column_stack([masofa, tirband, buyurtma])
    return X, vaqt


def narx(y, p, jarima: float = 5.0) -> float:
    """Kech qolish (y > p) jarima marta qimmat."""
    return float(jarima * np.maximum(0, y - p).sum() + np.maximum(0, p - y).sum())


def main() -> None:
    X, y = yarat()
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0)

    print("=== 1. Simmetrik model (o'rtacha) ===")
    m = LinearRegression().fit(Xtr, ytr)
    p = m.predict(Xte)
    print(f"  MAE = {mean_absolute_error(yte, p):.2f}, "
          f"kech qolish ulushi = {(yte > p).mean():.1%}")
    print(f"  asimmetrik narx = {narx(yte, p):,.0f}")

    print("\n=== 2. Kvantil modellar ===")
    natijalar = {}
    for q in [0.5, 0.7, 0.83, 0.9]:
        qm = GradientBoostingRegressor(loss="quantile", alpha=q, random_state=0,
                                       n_estimators=200, max_depth=3).fit(Xtr, ytr)
        pq = qm.predict(Xte)
        natijalar[q] = narx(yte, pq)
        print(f"  q={q:.2f}: MAE {mean_absolute_error(yte, pq):5.2f}, "
              f"kech {(yte > pq).mean():5.1%}, narx {natijalar[q]:>9,.0f}")

    eng = min(natijalar, key=natijalar.get)
    print(f"\n=== 3. Eng arzon kvantil: q = {eng} ===")
    print(f"  nazariy optimal: jarima/(jarima+1) = {5 / 6:.3f}  (9.10 newsvendor)")
    print(f"  tejash (simmetrikka nisbatan): "
          f"{(narx(yte, p) - natijalar[eng]) / narx(yte, p):.1%}")

    print("\n=== 4. Pinball loss ===")
    for q in [0.5, 0.83]:
        qm = GradientBoostingRegressor(loss="quantile", alpha=q, random_state=0,
                                       n_estimators=200, max_depth=3).fit(Xtr, ytr)
        pq = qm.predict(Xte)
        print(f"  q={q}: pinball(0.83) = {mean_pinball_loss(yte, pq, alpha=0.83):.3f}")
    print("  ⭐ Metrika narxni aks ettirsa — model to'g'ri o'rganadi")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Simmetrik model (o'rtacha) ===
  MAE = 3.61, kech qolish ulushi = 51.0%
  asimmetrik narx = 9,811

=== 2. Kvantil modellar ===
  q=0.50: MAE  3.73, kech 51.3%, narx    10,233
  q=0.70: MAE  4.18, kech 31.6%, narx     7,346
  q=0.83: MAE  4.96, kech 20.9%, narx     6,293
  q=0.90: MAE  6.21, kech 10.8%, narx     6,534

=== 3. Eng arzon kvantil: q = 0.83 ===
  nazariy optimal: jarima/(jarima+1) = 0.833  (9.10 newsvendor)
  tejash (simmetrikka nisbatan): 35.9%

=== 4. Pinball loss ===
  q=0.5: pinball(0.83) = 1.895
  q=0.83: pinball(0.83) = 1.179
  ⭐ Metrika narxni aks ettirsa — model to'g'ri o'rganadi

Nima ko'rsatdi: 2.5-bo'lim.

5. To'g'ri va noto'g'ri tushunishlar

Noto'g'ri fikr To'g'risi
"R^2 0.9 — har doim zo'r" Dispersiyaga bog'liq
"R^2 manfiy bo'lolmaydi" Testda bo'ladi
"MAE va RMSE bir xil tartib beradi" Ko'pincha emas
"MAPE — universal foiz metrikasi" Nolda portlaydi
"Metrika — texnik tanlov" Narxdan kelib chiqadi
"Bitta son yetarli" Qoldiqlar tahlili kerak
"Baza kerak emas" Baza — o'lchov
"Xato simmetrik" Ko'pincha emas

6. Keng tarqalgan xatolar va yechimlari

1. R^2 yakka

python
print("R2:", r2_score(y_te, p))                                   # ⚠️
print(f"MAE {mae:.2f} daq, RMSE {rmse:.2f} daq, R2 {r2:.3f}")     # ✅

2. Bazasiz hisobot

python
print("MAE = 12")                                                 # ⚠️
print(f"MAE {mae:.1f} (baza {mae_b:.1f}, yaxshilanish {y:.0%})")  # ✅

3. MAPE nollar bilan

python
mean_absolute_percentage_error(talab, p)     # talabda nollar     # ⚠️
np.abs(talab - p).sum() / talab.sum()        # WAPE               # ✅

4. Asimmetrik narxda MAE

python
model = LinearRegression()                                        # ⚠️
GradientBoostingRegressor(loss="quantile", alpha=0.83)            # ✅

5. Birliksiz son

python
print("RMSE:", rmse)                                              # ⚠️
print(f"RMSE: {rmse:.1f} daqiqa")                                 # ✅

6. Noaniqliksiz

python
print(f"MAE {mae:.2f}")                                           # ⚠️
print(f"MAE {mae:.2f} ± {sd:.2f} (CV, 5 fold)")                   # ✅

7. Log fazoda o'lchab, asl birlikda e'lon qilish

python
# log(y) bo'yicha R2 = 0.95 → "model 95% tushuntiradi"            # ⚠️
# asl birlikda ham hisoblang: expm1 dan keyin MAE                 # ✅

7. Integratsiya — bu bilim qayerda kerak bo'ladi

  • 9.10-dars (o'tilgan): Asimmetrik narx, newsvendor
  • 10.6-dars (o'tilgan): Qoldiqlar tahlili
  • 11.8-dars (o'tilgan): Bootstrap CI
  • 12.7-dars (o'tilgan): Klassifikatsiya metrikalari
  • Regressiya qismi: Modellar va kengaytirilgan baholash

8. Eng yaxshi amaliyotlar

  1. MAE, RMSE va MedAE ni birga bering.

  2. Har doim bazani ko'rsating.

  3. Birlikni yozing.

  4. Noaniqlikni bering.

  5. Nollar bo'lsa MAPE ishlatmang.

  6. Narx asimmetrik bo'lsa — kvantil.

  7. Qoldiqlarga qarang.

  8. Metrikani oldindan kelishing.

9. Amaliy topshiriq

Vazifa 1: Bashorat qiling

python
1.  # MAE formulasi?
2.  # RMSE MAE dan kichik bo'la oladimi?
3.  # RMSE/MAE = 3 nimani anglatadi?
4.  # R^2 = 0 nima?
5.  # R^2 < 0 mumkinmi?
6.  # MAPE qachon portlaydi?
7.  # WAPE formulasi?
8.  # MSLE qachon?
9.  # pinball loss q=0.5 da?
10. # kech qolish 5x qimmat — qaysi kvantil?
11. # regressiya bazalari?
12. # metrikadan tashqari nima ko'rish kerak?
Javoblar
  1. mean(|y - p|)
  2. Yo'q
  3. Bir necha juda katta xato
  4. O'rtacha bilan bir xil
  5. Ha, testda
  6. y nolga yaqin
  7. sum|y-p| / sum|y|
  8. Musbat, o'ng qiyshiq y
  9. MAE ga proporsional
  10. q = 5/6 ≈ 0.83
  11. O'rtacha, median, naive, oddiy qoida
  12. Qoldiqlar va segmentlar

Vazifa 2: Xatolarni tuzating

python
1.  print("R2 =", r2_score(y, p))   # yolg'iz

2.  print("MAE =", mae)   # bazasiz, birliksiz

3.  mean_absolute_percentage_error(talab, p)   # talabda nollar

4.  # kech qolish 8x qimmat, LinearRegression bilan MAE optimallashtirilgan

5.  # log(y) da R2 = 0.96 deb hisobot
Javoblar
python
1.  print(f"MAE {mae:.2f}, RMSE {rmse:.2f}, R2 {r2:.3f}")

2.  print(f"MAE {mae:.1f} daqiqa (baza {mae_b:.1f})")

3.  np.abs(talab - p).sum() / talab.sum()

4.  GradientBoostingRegressor(loss="quantile", alpha=8 / 9)

5.  # asl birlikda MAE/RMSE ni ham bering

Vazifa 3: Metrikalar taqqoslash

Modellang:

  1. Outlierli ma'lumot
  2. MAE, RMSE, MedAE
  3. Turli model tanlovi
  4. Xulosa

Vazifa 4: Asimmetrik narx

Modellang:

  1. Jarima nisbatlari
  2. Kvantil modellar
  3. Optimal kvantil
  4. Tejash

Vazifa 5: Qoldiqlar

Modellang:

  1. Segmentlar bo'yicha xato
  2. y diapazoni bo'yicha
  3. Eng katta 10 xato
  4. Yaxshilash g'oyasi

Vazifa 6: Integratsiya

Modellang:

  1. Pipeline (12.6)
  2. CV va SD
  3. Bootstrap CI (11.8)
  4. Hisobot

Vazifa 7: O'ylash

Regressiya modelining sifati ko'pincha bitta son bilan ("MAE 12 daqiqa") e'lon qilinadi, lekin real qaror ko'pincha bashorat intervali talab qiladi ("95% ehtimol bilan 30-60 daqiqa"). Nuqtaviy bashorat va intervalli bashorat orasidagi farq nimada va qachon qaysi biri kerak?

Javob

Qisqa javob: nuqtaviy bashorat — "eng ehtimoliy qiymat", intervalli bashorat — "qanchalik noaniqmiz". Qaror noaniqlikka bog'liq bo'lsa (zaxira, va'da qilingan vaqt, byudjet), interval majburiy; shunchaki reyting yoki o'rtacha rejalashtirish uchun nuqtaviy yetarli.

1. Farqi

Nuqtaviy Intervalli
p = 45 daqiqa 80% interval: 32-61 daqiqa
MAE/RMSE bilan baholanadi qamrov (coverage) va kenglik bilan
Bitta model kvantil modellar yoki konformal bashorat

2. Qachon interval kerak

  • Mijozga vaqt va'da qilish (kech qolish jarimasi)
  • Zaxira va quvvat rejalashtirish (9.10)
  • Xavf boshqaruvi va moliya
  • Model ishonchsiz segmentlarni ajratish (past ishonch → inson qarori)

3. Qanday olinadi

  • Kvantil regressiya (q=0.1 va q=0.9) — 2.5
  • Konformal bashorat (kafolatlangan qamrov)
  • Bootstrap 11.8-bob yoki ansambl dispersiyasi

4. Qanday baholanadi

  • Qamrov: 80% interval haqiqatan 80% holatni qamraydimi
  • Kenglik: tor interval yaxshi, lekin qamrov buzilmasligi kerak
  • Pinball loss — ikkalasini birga o'lchaydi

5. Xulosa

  1. Nuqtaviy bashorat — noaniqlikni yashiradi
  2. Qaror noaniqlikka bog'liq bo'lsa — interval
  3. Interval sifati qamrov bilan o'lchanadi
  4. Kvantil modellar — eng oddiy yo'l

Nimani mustahkamlaydi: 2.5, 2.6-bo'limlar.

Xulosa

Bu darsda regressiya metrikalarini o'rgandik.

Eng muhim uch fikr:

  1. Metrikalar turli savollarga javob beradi. MAE — o'rtacha xato (outlierlarga chidamli), RMSE — katta xatolarni og'ir jazolaydi, MedAE — odatiy xato. RMSE/MAE nisbati xato taqsimotining qiyshiqligini ko'rsatadi. Uchtasini birga bering — ular ba'zan turli modelni tanlaydi.

  2. R^2 — nisbiy va aldamchi. U "o'rtacha bilan bashorat qilishdan qancha yaxshi" degani, ma'lumot tarqoqligiga bog'liq: tor segmentda past R^2 model yomon degani emas. Testda manfiy bo'lishi mumkin. MAPE nolga yaqin qiymatlarda portlaydi va past bashoratni afzal ko'radi — WAPE yoki MAE ishlating.

  3. Metrika narxni aks ettirsin. Xato asimmetrik bo'lsa (kech qolish erta qolishdan a marta qimmat), simmetrik MAE noto'g'ri modelni tanlaydi: kvantil regressiya (q = a/(a+1), 9.10 newsvendor) yoki to'g'ridan-to'g'ri narx metrikasi kerak. Har doim: baza, birlik, noaniqlik (CV SD yoki bootstrap CI) va qoldiqlar tahlili.

Keyingi darsda pipeline va ma'lumot sizib chiqishini o'rganamiz: tayyorlashni CV ichida to'g'ri qilish, ColumnTransformer, custom transformer va leakage manbalari.

Ulashish:Telegram'da

Izohlar (0)

Izoh yozish uchun kiring.

  • Hozircha izoh yo'q. Birinchi bo'ling!
12.8-dars: Regressiya metrikalari — IlmHamroh