13.11-dars: Robust va kvantil regressiya

Kichik xatolarda kvadratik, katta xatolarda CHIZIQLI:

  L(r) = 0.5 · r^2                  agar |r| <= d
       = d · (|r| - 0.5 · d)        agar |r| > d

d (epsilon) — chegara; sklearn da standart 1.35 (standartlashtirilgan qoldiqda)
  d katta → OLS ga yaqin
  d kichik → MAE ga yaqin (robustroq)

HuberRegressor(epsilon=1.35, alpha=0.0001)   ← masshtablash kerak

Pinball (kvantil) yo'qotish, q uchun:
  L_q(r) = q · r          agar r >= 0   (kam bashorat qilindi)
         = (q - 1) · r    agar r < 0    (ortiq bashorat qilindi)

q = 0.5 → median regressiyasi (MAE) — robust
q = 0.9 → 90-persentil chizig'i

QuantileRegressor(quantile=0.5, alpha=0.0)
GradientBoostingRegressor(loss="quantile", alpha=q)   ← nochiziqli variant

RANSAC (RANdom SAmple Consensus):
  1. tasodifiy minimal namuna tanlab model qurish
  2. unga mos keladigan ("inlier") nuqtalarni sanash
  3. eng ko'p inlier bergan modelni saqlash, takrorlash
  → 50% gacha outlier bo'lsa ham ishlaydi; lekin tasodifiy (random_state kerak)

Theil-Sen:
  barcha nuqta juftliklari nishabining MEDIANASI
  → juda robust (29% buzilish nuqtasi), lekin p katta bo'lsa sekin

Buzilish nuqtasi (breakdown point): OLS 0%, Huber ~0%, Theil-Sen 29%, RANSAC ~50%

OLS         — outlier yo'q yoki juda kam; eng samarali
HUBER       — 1-10% outlier; standart robust tanlov
RANSAC      — 20-50% outlier; qattiq buzilgan ma'lumot
THEIL-SEN   — kam belgi, yuqori ishonchlilik talabi
KVANTIL q=0.5 — median bashorat kerak; robust va talqin oson
KVANTIL q≠0.5 — asimmetrik narx yoki interval kerak

Avval: outlierlar QAYERDAN kelgan? (o'lchov xatosi / haqiqiy holat) — 13.4

Kvantil regressiya bilan:
  q = 0.05 va q = 0.95 modellarini alohida o'qitish → 90% interval

Baholash:
  QAMROV (coverage): interval haqiqatan 90% holatni qamraydimi
  KENGLIK: tor interval yaxshi, lekin qamrov buzilmasligi kerak

Muqobil: konformal bashorat (kafolatlangan qamrov), bootstrap 11.8-bob
Diqqat: kvantil modellari KESISHIB ketishi mumkin (q=0.9 < q=0.5)

Robust usullar bepul emas:
  · outlier YO'Q bo'lsa, OLS dan biroz kam samarali (kengroq CI)
  · Huber/kvantil — iterativ, sekinroq
  · RANSAC — tasodifiy, takrorlanuvchanlik uchun random_state
  · Theil-Sen — O(n^2) yoki undan yomon
  · statistik xulosa (p, CI) murakkabroq — bootstrap kerak 11.8-bob

Lekin: outlier bo'lsa — narx arzimas

import numpy as np
from sklearn.linear_model import (HuberRegressor, LinearRegression,
                                  QuantileRegressor, RANSACRegressor, TheilSenRegressor)
from sklearn.metrics import mean_pinball_loss
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

Pipeline([("sc", StandardScaler()), ("m", HuberRegressor(epsilon=1.35))])
RANSACRegressor(random_state=0).fit(X, y).inlier_mask_
TheilSenRegressor(random_state=0, max_subpopulation=10_000)

# kvantil interval
past = QuantileRegressor(quantile=0.05, alpha=0).fit(X, y)
yuqori = QuantileRegressor(quantile=0.95, alpha=0).fit(X, y)
qamrov = ((y_te >= past.predict(X_te)) & (y_te <= yuqori.predict(X_te))).mean()
QOIDA: avval outlier sababini top · Huber standart · RANSAC ko'p buzilishda · qamrovni o'lcha

Huber — kvadratik+chiziqli (epsilon=1.35) · RANSAC — inlier konsensusi
Theil-Sen — juftlik nishablari medianasi · Buzilish nuqtasi: OLS 0%, TS 29%, RANSAC 50%
Kvantil: q=0.5 median (robust), q≠0.5 interval va asimmetrik narx
Narx: outliersiz OLS dan biroz kam samarali — arzimas

"""OLS, Huber, RANSAC va Theil-Sen ni solishtirish (real numpy/sklearn)."""

import numpy as np
from sklearn.linear_model import (HuberRegressor, LinearRegression,
                                  RANSACRegressor, TheilSenRegressor)


def yarat(buzuq_ulush: float, seed: int = 5, n: int = 400):
    rng = np.random.default_rng(seed)
    x = rng.uniform(0, 20, n)
    y = 10 + 2.5 * x + rng.normal(0, 2.0, n)
    k = int(buzuq_ulush * n)
    if k:
        idx = rng.choice(n, k, replace=False)
        y[idx] = y[idx] + rng.normal(60, 15, k)         # GPS xatosi kabi
    return x.reshape(-1, 1), y


def nishablar(X, y) -> dict:
    return {
        "OLS": LinearRegression().fit(X, y).coef_[0],
        "Huber": HuberRegressor(epsilon=1.35).fit(X, y).coef_[0],
        "RANSAC": RANSACRegressor(random_state=0).fit(X, y).estimator_.coef_[0],
        "Theil-Sen": TheilSenRegressor(random_state=0,
                                       max_subpopulation=5000).fit(X, y).coef_[0],
    }


def main() -> None:
    print("=== 1. Toza ma'lumot (haqiqiy nishab 2.5) ===")
    X, y = yarat(0.0)
    for nom, w in nishablar(X, y).items():
        print(f"  {nom:<10}: {w:.4f}")

    print("\n=== 2. 5% buzilgan ===")
    X, y = yarat(0.05)
    for nom, w in nishablar(X, y).items():
        print(f"  {nom:<10}: {w:.4f}  (xato {abs(w - 2.5) / 2.5:+.1%})")

    print("\n=== 3. 25% buzilgan ===")
    X, y = yarat(0.25)
    for nom, w in nishablar(X, y).items():
        print(f"  {nom:<10}: {w:.4f}  (xato {abs(w - 2.5) / 2.5:+.1%})")

    print("\n=== 4. Buzilish ulushi bo'yicha ===")
    print(f"  {'ulush':>6} {'OLS':>8} {'Huber':>8} {'RANSAC':>8} {'Theil-Sen':>10}")
    for u in [0.0, 0.05, 0.15, 0.30, 0.45]:
        X, y = yarat(u)
        n = nishablar(X, y)
        print(f"  {u:>6.0%} {n['OLS']:>8.3f} {n['Huber']:>8.3f} "
              f"{n['RANSAC']:>8.3f} {n['Theil-Sen']:>10.3f}")
    print("  ⭐ Buzilish ulushi ortsa, RANSAC eng barqaror qoladi")


if __name__ == "__main__":
    main()

=== 1. Toza ma'lumot (haqiqiy nishab 2.5) ===
  OLS       : 2.4858
  Huber     : 2.4903
  RANSAC    : 2.4906
  Theil-Sen : 2.4847

=== 2. 5% buzilgan ===
  OLS       : 2.6286  (xato +5.1%)
  Huber     : 2.5017  (xato +0.1%)
  RANSAC    : 2.4893  (xato +0.4%)
  Theil-Sen : 2.5154  (xato +0.6%)

=== 3. 25% buzilgan ===
  OLS       : 2.3554  (xato +5.8%)
  Huber     : 2.4995  (xato +0.0%)
  RANSAC    : 2.5069  (xato +0.3%)
  Theil-Sen : 2.5432  (xato +1.7%)

=== 4. Buzilish ulushi bo'yicha ===
   ulush      OLS    Huber   RANSAC  Theil-Sen
      0%    2.486    2.490    2.491      2.485
      5%    2.629    2.502    2.489      2.515
     15%    2.720    2.494    2.475      2.511
     30%    2.475    2.504    2.533      2.576
     45%    2.343    2.347    2.510      2.707
  ⭐ Buzilish ulushi ortsa, RANSAC eng barqaror qoladi

"""Robustlik bepul emas, lekin arzon (real numpy/sklearn)."""

import numpy as np
from sklearn.linear_model import HuberRegressor, LinearRegression


def main() -> None:
    rng = np.random.default_rng(9)
    HAQIQIY = 2.5

    def tajriba(buzuq: float, eps: float, B: int = 200):
        ols, hub = [], []
        for _ in range(B):
            n = 200
            x = rng.uniform(0, 20, n)
            y = 10 + HAQIQIY * x + rng.normal(0, 2.0, n)
            k = int(buzuq * n)
            if k:
                idx = rng.choice(n, k, replace=False)
                y[idx] += rng.normal(60, 15, k)
            X = x.reshape(-1, 1)
            ols.append(LinearRegression().fit(X, y).coef_[0])
            hub.append(HuberRegressor(epsilon=eps, max_iter=500).fit(X, y).coef_[0])
        return np.array(ols), np.array(hub)

    print("=== 1. Toza ma'lumotda samaradorlik narxi ===")
    ols, hub = tajriba(0.0, 1.35)
    print(f"  OLS  : o'rtacha {ols.mean():.4f}, SD {ols.std():.4f}")
    print(f"  Huber: o'rtacha {hub.mean():.4f}, SD {hub.std():.4f}")
    print(f"  Huber ning SD si OLS dan {hub.std() / ols.std():.3f} barobar katta "
          f"(narx ~{(hub.std() / ols.std() - 1) * 100:.1f}%)")

    print("\n=== 2. 8% buzilgan ma'lumotda ===")
    ols, hub = tajriba(0.08, 1.35)
    print(f"  OLS  : o'rtacha {ols.mean():.4f} (siljish "
          f"{ols.mean() - HAQIQIY:+.4f}), SD {ols.std():.4f}")
    print(f"  Huber: o'rtacha {hub.mean():.4f} (siljish "
          f"{hub.mean() - HAQIQIY:+.4f}), SD {hub.std():.4f}")

    print("\n=== 3. epsilon ning ta'siri (8% buzilgan) ===")
    ols8, _ = tajriba(0.08, 1.35, B=100)
    for eps in [1.05, 1.35, 2.0, 5.0, 50.0]:
        _, hub = tajriba(0.08, eps, B=100)
        print(f"  epsilon {eps:>5}: nishab SD {hub.std():.4f}, "
              f"siljish {hub.mean() - HAQIQIY:+.4f}")
    print(f"  taqqoslash uchun OLS nishab SD: {ols8.std():.4f}")
    print("  (bu yerda outlierlar x bo'ylab bir tekis — ular nishabni siljitmaydi,")
    print("   lekin uning TARQOQLIGINI keskin oshiradi; Huber aynan shuni to'xtatadi)")

    print("\n=== 4. O'rtacha kvadratik xato (MSE) taqqoslash ===")
    for buzuq in [0.0, 0.03, 0.10]:
        ols, hub = tajriba(buzuq, 1.35, B=100)
        mse_o = np.mean((ols - HAQIQIY) ** 2)
        mse_h = np.mean((hub - HAQIQIY) ** 2)
        print(f"  {buzuq:>4.0%} buzilgan: OLS MSE {mse_o:.5f}, Huber MSE {mse_h:.5f}, "
              f"nisbat {mse_o / mse_h:6.1f}x")
    print("  ⭐ Toza ma'lumotda narx kichik, buzilganda foyda katta")


if __name__ == "__main__":
    main()

=== 1. Toza ma'lumotda samaradorlik narxi ===
  OLS  : o'rtacha 2.4996, SD 0.0240
  Huber: o'rtacha 2.4996, SD 0.0248
  Huber ning SD si OLS dan 1.034 barobar katta (narx ~3.4%)

=== 2. 8% buzilgan ma'lumotda ===
  OLS  : o'rtacha 2.4885 (siljish -0.0115), SD 0.2066
  Huber: o'rtacha 2.4964 (siljish -0.0036), SD 0.0280

=== 3. epsilon ning ta'siri (8% buzilgan) ===
  epsilon  1.05: nishab SD 0.0313, siljish +0.0016
  epsilon  1.35: nishab SD 0.0305, siljish -0.0057
  epsilon   2.0: nishab SD 0.0304, siljish -0.0051
  epsilon   5.0: nishab SD 0.1842, siljish +0.0023
  epsilon  50.0: nishab SD 0.2011, siljish +0.0138
  taqqoslash uchun OLS nishab SD: 0.1982
  (bu yerda outlierlar x bo'ylab bir tekis — ular nishabni siljitmaydi,
   lekin uning TARQOQLIGINI keskin oshiradi; Huber aynan shuni to'xtatadi)

=== 4. O'rtacha kvadratik xato (MSE) taqqoslash ===
    0% buzilgan: OLS MSE 0.00057, Huber MSE 0.00066, nisbat    0.9x
    3% buzilgan: OLS MSE 0.01746, Huber MSE 0.00087, nisbat   20.1x
   10% buzilgan: OLS MSE 0.05673, Huber MSE 0.00091, nisbat   62.4x
  ⭐ Toza ma'lumotda narx kichik, buzilganda foyda katta

"""Shartli kvantillar va bashorat intervali (real numpy/sklearn)."""

import numpy as np
from sklearn.linear_model import QuantileRegressor
from sklearn.metrics import mean_pinball_loss
from sklearn.model_selection import train_test_split


def main() -> None:
    rng = np.random.default_rng(13)
    n = 1500
    masofa = rng.uniform(1, 25, n)
    # tarqoqlik masofa bilan o'sadi (geteroskedastiklik)
    vaqt = 8 + 2.4 * masofa + rng.normal(0, 1.0 + 0.35 * masofa, n)
    X = masofa.reshape(-1, 1)
    Xtr, Xte, ytr, yte = train_test_split(X, vaqt, test_size=0.3, random_state=0)

    print("=== 1. Kvantil chiziqlari ===")
    modellar = {}
    for q in [0.05, 0.25, 0.5, 0.75, 0.95]:
        m = QuantileRegressor(quantile=q, alpha=0.0, solver="highs").fit(Xtr, ytr)
        modellar[q] = m
        print(f"  q = {q:.2f}: kesma {m.intercept_:6.2f}, nishab {m.coef_[0]:.3f}")
    print("  (nishablar farqi — tarqoqlik masofa bilan o'sishini ko'rsatadi)")

    print("\n=== 2. Bashorat intervali (90%) ===")
    past = modellar[0.05].predict(Xte)
    yuqori = modellar[0.95].predict(Xte)
    qamrov = ((yte >= past) & (yte <= yuqori)).mean()
    print(f"  qamrov = {qamrov:.1%} (kerak 90%)")
    print(f"  o'rtacha kenglik = {(yuqori - past).mean():.2f} daqiqa")

    print("\n=== 3. Interval masofa bo'yicha ===")
    for lo, hi in [(1, 6), (6, 14), (14, 25)]:
        mos = (Xte[:, 0] >= lo) & (Xte[:, 0] < hi)
        print(f"  masofa {lo:>2}-{hi:>2} km: kenglik "
              f"{(yuqori[mos] - past[mos]).mean():5.2f}, "
              f"qamrov {((yte[mos] >= past[mos]) & (yte[mos] <= yuqori[mos])).mean():.1%}")

    print("\n=== 4. Pinball loss bilan baholash ===")
    for q in [0.5, 0.9]:
        m = QuantileRegressor(quantile=q, alpha=0.0, solver="highs").fit(Xtr, ytr)
        p_q = m.predict(Xte)
        p_med = modellar[0.5].predict(Xte)
        print(f"  q = {q}: o'z modeli {mean_pinball_loss(yte, p_q, alpha=q):.4f}, "
              f"median modeli {mean_pinball_loss(yte, p_med, alpha=q):.4f}")
    print("  ⭐ Har kvantil o'z modelini talab qiladi")


if __name__ == "__main__":
    main()

=== 1. Kvantil chiziqlari ===
  q = 0.05: kesma   6.57, nishab 1.815
  q = 0.25: kesma   7.38, nishab 2.176
  q = 0.50: kesma   8.08, nishab 2.406
  q = 0.75: kesma   8.49, nishab 2.703
  q = 0.95: kesma   9.35, nishab 2.987
  (nishablar farqi — tarqoqlik masofa bilan o'sishini ko'rsatadi)

=== 2. Bashorat intervali (90%) ===
  qamrov = 87.6% (kerak 90%)
  o'rtacha kenglik = 17.54 daqiqa

=== 3. Interval masofa bo'yicha ===
  masofa  1- 6 km: kenglik  6.54, qamrov 82.3%
  masofa  6-14 km: kenglik 14.92, qamrov 90.3%
  masofa 14-25 km: kenglik 24.89, qamrov 87.9%

=== 4. Pinball loss bilan baholash ===
  q = 0.5: o'z modeli 2.1576, median modeli 2.1576
  q = 0.9: o'z modeli 0.9447, median modeli 2.0532
  ⭐ Har kvantil o'z modelini talab qiladi

"""Buzilgan ma'lumot + asimmetrik narx (real numpy/sklearn)."""

import numpy as np
from sklearn.linear_model import HuberRegressor, LinearRegression, QuantileRegressor
from sklearn.metrics import mean_absolute_error
from sklearn.model_selection import train_test_split


def yarat(seed: int = 17, n: int = 2000):
    rng = np.random.default_rng(seed)
    masofa = rng.uniform(1, 25, n)
    tirband = rng.beta(2, 3, n)
    vaqt = 8 + 2.2 * masofa + 14 * tirband + rng.normal(0, 3.0, n)
    toza = vaqt.copy()
    # 3% GPS xatosi: vaqt bir necha barobar oshib ketadi
    buzuq = rng.random(n) < 0.03
    vaqt[buzuq] += rng.uniform(120, 300, buzuq.sum())
    X = np.column_stack([masofa, tirband])
    return X, vaqt, toza, buzuq


def main() -> None:
    X, y, toza, buzuq = yarat()
    idx = np.arange(len(X))
    Xtr, Xte, ytr, yte, itr, ite = train_test_split(X, y, idx, test_size=0.3,
                                                    random_state=0)
    toza_te = toza[ite]
    buzuq_te = buzuq[ite]

    print("=== 1. Ma'lumot ===")
    print(f"  {len(X)} safar, buzilgan yozuvlar: {buzuq.sum()} ({buzuq.mean():.1%})")
    print(f"  test'da buzilgan: {buzuq_te.sum()} ta")

    print("\n=== 2. Koeffitsiyentlar (haqiqiy: masofa 2.2, tirband 14) ===")
    ols = LinearRegression().fit(Xtr, ytr)
    hub = HuberRegressor(epsilon=1.35, max_iter=1000).fit(Xtr, ytr)
    print(f"  OLS  : masofa {ols.coef_[0]:.3f}, tirband {ols.coef_[1]:6.2f}")
    print(f"  Huber: masofa {hub.coef_[0]:.3f}, tirband {hub.coef_[1]:6.2f}")

    print("\n=== 3. Toza yozuvlarda aniqlik ===")
    sof = ~buzuq_te
    for nom, m in [("OLS", ols), ("Huber", hub)]:
        print(f"  {nom:<6}: buzilmagan test yozuvlarida MAE "
              f"{mean_absolute_error(toza_te[sof], m.predict(Xte[sof])):.3f} daqiqa")

    print("\n=== 4. Mijozga va'da: 80-persentil ===")
    kv = QuantileRegressor(quantile=0.8, alpha=0.0, solver="highs").fit(Xtr, ytr)
    med = QuantileRegressor(quantile=0.5, alpha=0.0, solver="highs").fit(Xtr, ytr)
    for nom, m in [("median (q=0.5)", med), ("va'da (q=0.8)", kv)]:
        b = m.predict(Xte)
        kech = (toza_te > b).mean()
        print(f"  {nom:<15}: o'rtacha va'da {b.mean():5.1f} daqiqa, "
              f"kech qolish {kech:.1%}")
    print("  ⭐ Robust baho + kvantil va'da = ishonchli xizmat")


if __name__ == "__main__":
    main()

=== 1. Ma'lumot ===
  2000 safar, buzilgan yozuvlar: 63 (3.1%)
  test'da buzilgan: 13 ta

=== 2. Koeffitsiyentlar (haqiqiy: masofa 2.2, tirband 14) ===
  OLS  : masofa 2.191, tirband   5.69
  Huber: masofa 2.199, tirband  13.39

=== 3. Toza yozuvlarda aniqlik ===
  OLS   : buzilmagan test yozuvlarida MAE 7.911 daqiqa
  Huber : buzilmagan test yozuvlarida MAE 2.273 daqiqa

=== 4. Mijozga va'da: 80-persentil ===
  median (q=0.5) : o'rtacha va'da  42.3 daqiqa, kech qolish 48.8%
  va'da (q=0.8)  : o'rtacha va'da  45.1 daqiqa, kech qolish 15.2%
  ⭐ Robust baho + kvantil va'da = ishonchli xizmat

df = df[np.abs(z) < 3]                                            # ⚠️
# avval tekshiring: o'lchov xatosimi yoki haqiqiy holat 13.4-bob    # ✅

HuberRegressor().fit(X, y)                                        # ⚠️
Pipeline([("sc", StandardScaler()), ("m", HuberRegressor())])     # ✅

RANSACRegressor().fit(X, y)            # har safar boshqa natija  # ⚠️
RANSACRegressor(random_state=0)                                   # ✅

past, yuqori = q05.predict(X), q95.predict(X)                     # ⚠️
yuqori = np.maximum(yuqori, past)      # yoki monoton usul        # ✅

print(f"interval: {past:.1f}-{yuqori:.1f}")                       # ⚠️
print(f"... (test qamrovi {qamrov:.1%})")                         # ✅

# "o'rtacha yetkazib berish vaqti 34 daqiqa" (q=0.5 modeli)       # ⚠️
# "tipik (median) vaqt 34 daqiqa"                                 # ✅

TheilSenRegressor().fit(X, y)          # n = 200000               # ⚠️
TheilSenRegressor(max_subpopulation=10_000)  # yoki Huber         # ✅

1.  # Huber yo'qotishi qanday?
2.  # epsilon nima?
3.  # epsilon katta bo'lsa?
4.  # RANSAC qanday ishlaydi?
5.  # RANSAC buzilish nuqtasi?
6.  # Theil-Sen nima?
7.  # pinball loss formulasi?
8.  # q = 0.5 nima beradi?
9.  # interval qanday quriladi?
10. # qamrov nima?
11. # robustlikning narxi?
12. # birinchi savol nima?

1.  df = df[np.abs(z) < 3]   # tekshirmasdan

2.  HuberRegressor().fit(X, y)   # masshtablanmagan

3.  RANSACRegressor().fit(X, y)

4.  # "90% interval: 20-60 daqiqa" (qamrov o'lchanmagan)

5.  # q=0.5 modelini "o'rtacha vaqt" deb hisobot qilish