Data Science va sun'iy intellekt12.4-dars: Overfitting va underfitting
Data Science va sun'iy intellekt · ML asoslari

12.4-dars: Overfitting va underfitting

17 daqiqa4/10-dars

12-QISM — MACHINE LEARNING ASOSLARI · 4-dars


1. Kirish va motivatsiya

12.1-darsda ko'rdik: 1-qo'shni modeli o'quv ma'lumotida mukammal (R^2 = 1.000), testda esa zaif. Bu — overfitting (haddan tashqari moslashish): model ma'lumotdagi shovqinni ham naqsh deb o'rgangan. Teskari holat — underfitting: model juda sodda, hatto o'quv ma'lumotidagi haqiqiy naqshni ham ushlay olmaydi.

Har bir ML loyihasi shu ikki chekka orasida muvozanat qidiradi. Ularni tashxis qilish oson: o'quv va validatsiya natijalarini solishtirish yetarli. Davolash esa vaziyatga bog'liq: overfitting da — ko'proq ma'lumot, kamroq belgi, regularizatsiya (13-qism), soddaroq model; underfitting da — murakkabroq model, boyroq belgilar, kamroq regularizatsiya.

Real vaziyat. Jamoa uy narxi modelini quradi: qaror daraxti, chuqurlik cheklanmagan. O'quv R^2 = 1.00, test R^2 = 0.62. Jamoa "ma'lumot kam" deb o'ylab, yana 20 ta belgi qo'shadi — test R^2 0.58 ga tushadi. Data Scientist tashxis qo'yadi: bu overfitting, belgi qo'shish uni kuchaytiradi. Yechim: daraxt chuqurligini cheklash (max_depth=6) va bargdagi minimal kuzatuvlar sonini belgilash → test R^2 = 0.81. Murakkablikni kamaytirish aniqlikni oshirdi.

Bu darsda overfitting va underfittingni o'rganamiz.

Bu darsda:

  • Overfitting va underfitting: ta'rif va tashxis
  • Model murakkabligi va umumlashtirish
  • O'quv va validatsiya egri chiziqlari
  • Ma'lumot hajmi egri chizig'i (learning curve)
  • Regularizatsiya bilan boshqarish
  • Erta to'xtatish va boshqa usullar
  • Overfitting tuzoqlari
  • Amaliy: tashxis va davolash

ℹ Misollar real numpy/sklearn bilan (Python 3.14).


2. Nazariya — chuqur tushuntirish

2.1. Ta'rif va tashxis

text
                    O'QUV xatosi    VALIDATSIYA xatosi    TASHXIS
  juda sodda model      katta             katta           UNDERFITTING
  yaxshi model          kichik            kichik          MUVOZANAT
  juda murakkab         juda kichik       katta           OVERFITTING

Tashxis qoidasi:
  ikkalasi ham yomon        → underfitting (model yetarli emas)
  o'quv yaxshi, test yomon  → overfitting (yodlash)
  ikkalasi ham yomon va teng → belgilar yetarli emas yoki vazifa qiyin

Overfitting — model o'quv ma'lumotining tasodifiy xususiyatlarini (shovqinni) o'rganib olgan: u yodlaydi, lekin umumlashtirmaydi. Underfitting — model juda sodda: hatto mavjud naqshni ham ushlay olmaydi. Tashxis uchun ikkita raqam kerak: o'quv va validatsiya natijasi. Faqat bittasiga qarash — eng keng tarqalgan xato 12.1-bob.

2.2. Model murakkabligi

text
Murakkablik nimalar bilan oshadi:
  polinom darajasi, daraxt chuqurligi, neyron qatlamlari, belgilar soni,
  KNN da k ning KICHIKligi, regularizatsiya kuchining KICHIKligi

  xato
   │      \                    /
   │       \   validatsiya    /
   │        \_______________ /
   │         \             /
   │          \___________/   ← optimal murakkablik
   │              o'quv    \___
   └──────────────────────────────→ murakkablik

O'quv xatosi murakkablik bilan doim kamayadi; validatsiya xatosi — U shaklida

Murakkablik — modelning ma'lumotga moslashish qobiliyati. Kichik murakkablikda model haqiqiy naqshni ham ushlay olmaydi (underfitting); haddan tashqari murakkablikda shovqinni ham o'rganadi (overfitting). Optimal nuqta — validatsiya xatosi minimal bo'lgan joy. Uni faqat validatsiya 12.3-bob orqali topish mumkin — o'quv xatosi bu haqda hech narsa aytmaydi.

2.3. O'quv va validatsiya egri chiziqlari

text
VALIDATION CURVE — giperparametr bo'yicha:
  sklearn.model_selection.validation_curve(model, X, y, param_name=..., param_range=..., cv=5)
  → har qiymat uchun o'quv va validatsiya balli

O'qish tartibi:
  ikki egri ham past       → underfitting (murakkablikni oshiring)
  egrilar orasida katta tafovut → overfitting (murakkablikni kamaytiring)
  egrilar yaqin va yuqori  → yaxshi

Validation curve — giperparametrni tanlashning vizual usuli: murakkablik oshgani sari o'quv balli o'sadi, validatsiya balli esa avval o'sib, keyin tushadi. Tafovut (gap) kattaligi — overfitting darajasi. Bu grafik giperparametr qiymatini "ko'z bilan" tanlashga yordam beradi (keyinroq — avtomatik qidiruv, 12.10 va model baholash qismi).

2.4. Ma'lumot hajmi egri chizig'i

text
LEARNING CURVE — o'quv hajmi bo'yicha:
  sklearn.model_selection.learning_curve(model, X, y, train_sizes=..., cv=5)

  Katta tafovut, validatsiya o'smoqda  → KO'PROQ MA'LUMOT yordam beradi
  Egrilar birlashgan, ikkalasi ham past → ma'lumot emas, MODEL/BELGILAR muammosi

Learning curve muhim amaliy savolga javob beradi: "Ko'proq ma'lumot yig'ish foyda beradimi?". Agar validatsiya egri chizig'i hali o'sayotgan bo'lsa — ha; agar u tekislanib, o'quv egri chizig'i bilan birlashgan bo'lsa — yo'q (belgilar yoki model kerak). Bu ko'p resurs tejaydi: ma'lumot yig'ish qimmat.

2.5. Regularizatsiya

text
G'oya: modelga "murakkablik jarimasi" qo'shish

  Ridge (L2):  min ||y - Xw||^2 + alpha ||w||^2     — koeffitsiyentlarni kichraytiradi (13-qism)
  Lasso (L1):  ... + alpha ||w||_1                  — ba'zilarini aniq 0 ga (belgi tanlash)
  Daraxtlar:   max_depth, min_samples_leaf, ccp_alpha
  KNN:         k ni oshirish
  Neyron:      weight decay, dropout, erta to'xtatish

alpha (yoki C = 1/alpha) — giperparametr: validatsiyada tanlanadi

Regularizatsiya — overfittingga qarshi asosiy vosita: model erkinligini cheklaydi. Chiziqli modellarda — koeffitsiyentlarni kichraytirish (13-qism); daraxtlarda — o'sishni cheklash; neyron tarmoqlarda — dropout va erta to'xtatish. Muhim: regularizatsiyadan oldin standartlashtirish 12.9-bob, aks holda jarima birliklarga bog'liq bo'ladi.

2.6. Boshqa usullar

Overfittingni kamaytirishning boshqa yo'llari: ko'proq ma'lumot (eng ishonchli, lekin qimmat); belgilarni kamaytirish (ortiqcha va shovqinli belgilarni olib tashlash); erta to'xtatish (iterativ modellarda — validatsiya yomonlashganda to'xtatish); ansambllar (bagging — Random Forest, dispersiyani kamaytiradi); ma'lumotni boyitish (augmentation — rasm va matnda); kross-validatsiya bilan halol tanlov 12.3-bob. Underfittingga qarshi: murakkabroq model, ko'proq/boyroq belgilar, regularizatsiyani kamaytirish, uzoqroq o'qitish.

2.7. Overfitting tuzoqlari

Asosiy tuzoqlar: faqat o'quv ballini ko'rish; test bilan murakkablik tanlash 12.3-bob; leakage ni overfitting deb o'ylash (leakage da test ham yaxshi — 12.2); kichik validatsiya (baho beqaror); belgi qo'shish orqali "tuzatish" (ko'pincha kuchaytiradi); regularizatsiyasiz murakkab model; standartlashtirishsiz regularizatsiya; CV natijasiga cheksiz moslashish 11.9-bob; "murakkab model — zamonaviy" degan qarash.

2.8. Overfitting — murakkablik muvozanati

Overfitting — o'quvda yaxshi, validatsiyada yomon (shovqinni yodlash); underfitting — ikkalasida ham yomon (model yetarli emas). Tashxis — ikki ballni solishtirish; validation curve murakkablik bo'yicha, learning curve ma'lumot hajmi bo'yicha yo'l ko'rsatadi. Davolash: overfittingda — regularizatsiya, soddaroq model, kamroq belgi, ko'proq ma'lumot, erta to'xtatish, ansambl; underfittingda — murakkabroq model, boyroq belgilar. Keyingi dars — bu muvozanatning nazariy tushuntirishi: bias-variance.


3. Tez ma'lumotnoma

python
import numpy as np
from sklearn.model_selection import learning_curve, validation_curve, cross_validate

# tashxis: o'quv va validatsiya birga
res = cross_validate(model, X, y, cv=5, return_train_score=True)
res["train_score"].mean(), res["test_score"].mean()        # tafovut — overfitting belgisi

# validation curve (giperparametr)
tr, va = validation_curve(Ridge(), X, y, param_name="alpha",
                          param_range=np.logspace(-3, 3, 7), cv=5)

# learning curve (ma'lumot hajmi)
n, tr, va = learning_curve(model, X, y, train_sizes=np.linspace(0.1, 1.0, 6), cv=5)

# regularizatsiya
Ridge(alpha=1.0); Lasso(alpha=0.1)
DecisionTreeRegressor(max_depth=6, min_samples_leaf=20)
KNeighborsRegressor(n_neighbors=15)
QOIDA: ikki ballni birga ko'r · murakkablikni validatsiyada tanla · standartlashtir

Overfitting xulosasi

Overfitting — o'quv yaxshi, validatsiya yomon (shovqinni yodlash)
Underfitting — ikkalasi ham yomon (model sodda)
Tashxis — train va validation ballari orasidagi tafovut
Validation curve — murakkablik; learning curve — ma'lumot hajmi
Davolash: regularizatsiya, soddaroq model, kamroq belgi, ko'proq ma'lumot
Underfitting: murakkabroq model, boyroq belgilar

4. Batafsil misollar

Misollar real numpy/sklearn bilan (Python 3.14).

Misol 1 — Polinom darajasi: underfittingdan overfittinggacha

python
"""Polinom darajasi bo'yicha o'quv va validatsiya xatosi (real numpy/sklearn)."""

import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures, StandardScaler


def main() -> None:
    rng = np.random.default_rng(0)
    n = 120
    x = rng.uniform(-3, 3, n)
    y = 0.5 * x ** 3 - 2 * x + 1 + rng.normal(0, 2.5, n)          # haqiqiy: 3-daraja
    X = x.reshape(-1, 1)
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.35, random_state=0)

    print("=== 1. Daraja bo'yicha RMSE ===")
    print(f"  {'daraja':>7} {"o'quv":>9} {'test':>9}  tashxis")
    for d in [1, 2, 3, 5, 9, 15]:
        model = make_pipeline(PolynomialFeatures(d), StandardScaler(), LinearRegression())
        model.fit(Xtr, ytr)
        rtr = np.sqrt(mean_squared_error(ytr, model.predict(Xtr)))
        rte = np.sqrt(mean_squared_error(yte, model.predict(Xte)))
        if rtr > 3.0 and rte > 3.0:
            tashxis = "underfitting"
        elif rte > 1.15 * rtr:
            tashxis = "overfitting"
        else:
            tashxis = "muvozanat"
        print(f"  {d:>7} {rtr:>9.2f} {rte:>9.2f}  {tashxis}")

    print("\n=== 2. Shovqin darajasi (haqiqiy SD = 2.5) ===")
    print("  test RMSE shovqin darajasidan pastga tusha olmaydi (nazariy chegara)")

    print("\n=== 3. 15-darajali modelning koeffitsiyentlari ===")
    m15 = make_pipeline(PolynomialFeatures(15), StandardScaler(), LinearRegression()).fit(Xtr, ytr)
    koef = m15[-1].coef_
    print(f"  eng katta |koef| = {np.abs(koef).max():,.0f} — juda katta qiymatlar")
    print("  ⭐ O'quv xatosi tushadi, test xatosi ko'tariladi")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Daraja bo'yicha RMSE ===
   daraja     o'quv      test  tashxis
        1      3.53      3.70  underfitting
        2      3.52      3.77  underfitting
        3      2.52      2.66  muvozanat
        5      2.42      2.70  muvozanat
        9      2.38      2.75  overfitting
       15      2.34      2.95  overfitting

=== 2. Shovqin darajasi (haqiqiy SD = 2.5) ===
  test RMSE shovqin darajasidan pastga tusha olmaydi (nazariy chegara)

=== 3. 15-darajali modelning koeffitsiyentlari ===
  eng katta |koef| = 2,917 — juda katta qiymatlar
  ⭐ O'quv xatosi tushadi, test xatosi ko'tariladi

Nima ko'rsatdi: 2.1, 2.2-bo'limlar.

Misol 2 — Validation curve: daraxt chuqurligi

python
"""Daraxt chuqurligi bo'yicha validation curve (real numpy/sklearn)."""

import numpy as np
from sklearn.model_selection import validation_curve
from sklearn.tree import DecisionTreeRegressor


def main() -> None:
    rng = np.random.default_rng(1)
    n = 1200
    X = rng.normal(size=(n, 6))
    y = (2 * X[:, 0] + X[:, 1] ** 2 - 1.5 * X[:, 2] * X[:, 3]
         + rng.normal(0, 1.0, n))

    chuqurliklar = [1, 2, 3, 5, 8, 12, 20, None]
    param = [d if d is not None else 50 for d in chuqurliklar]

    tr, va = validation_curve(
        DecisionTreeRegressor(random_state=0), X, y,
        param_name="max_depth", param_range=param, cv=5, scoring="r2")

    print("=== 1. Chuqurlik bo'yicha R^2 ===")
    print(f"  {'chuqurlik':>10} {"o'quv":>8} {'CV':>8} {'tafovut':>9}")
    for d, a, b in zip(chuqurliklar, tr.mean(axis=1), va.mean(axis=1)):
        nom = "cheksiz" if d is None else str(d)
        print(f"  {nom:>10} {a:>8.3f} {b:>8.3f} {a - b:>9.3f}")

    eng = int(np.argmax(va.mean(axis=1)))
    print(f"\n=== 2. Eng yaxshi CV: chuqurlik = "
          f"{chuqurliklar[eng] if chuqurliklar[eng] else 'cheksiz'} "
          f"(CV R^2 = {va.mean(axis=1)[eng]:.3f})")

    print("\n=== 3. Boshqa cheklovlar ===")
    for leaf in [1, 5, 20, 50]:
        m = DecisionTreeRegressor(min_samples_leaf=leaf, random_state=0)
        from sklearn.model_selection import cross_validate
        r = cross_validate(m, X, y, cv=5, return_train_score=True, scoring="r2")
        print(f"  min_samples_leaf={leaf:>3}: o'quv {r['train_score'].mean():.3f}, "
              f"CV {r['test_score'].mean():.3f}")
    print("  ⭐ Murakkablikni cheklash — CV natijasini yaxshilaydi")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Chuqurlik bo'yicha R^2 ===
   chuqurlik    o'quv       CV   tafovut
           1    0.334    0.326     0.008
           2    0.431    0.375     0.056
           3    0.521    0.456     0.065
           5    0.650    0.469     0.182
           8    0.775    0.380     0.395
          12    0.892    0.311     0.581
          20    0.984    0.226     0.758
     cheksiz    1.000    0.231     0.769

=== 2. Eng yaxshi CV: chuqurlik = 5 (CV R^2 = 0.469)

=== 3. Boshqa cheklovlar ===
  min_samples_leaf=  1: o'quv 1.000, CV 0.231
  min_samples_leaf=  5: o'quv 0.821, CV 0.477
  min_samples_leaf= 20: o'quv 0.630, CV 0.493
  min_samples_leaf= 50: o'quv 0.532, CV 0.460
  ⭐ Murakkablikni cheklash — CV natijasini yaxshilaydi

Nima ko'rsatdi: 2.3, 2.5-bo'limlar.

Misol 3 — Learning curve: ko'proq ma'lumot yordam beradimi?

python
"""Learning curve: ma'lumot hajmi va model murakkabligi (real numpy/sklearn)."""

import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import learning_curve
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures, StandardScaler


def main() -> None:
    rng = np.random.default_rng(2)
    n = 2000
    X = rng.normal(size=(n, 4))
    y = (X[:, 0] ** 2 + 2 * X[:, 1] - X[:, 2] * X[:, 3] + rng.normal(0, 1.0, n))

    olchamlar = np.linspace(0.05, 1.0, 6)

    print("=== 1. Sodda model (chiziqli) ===")
    n1, tr1, va1 = learning_curve(LinearRegression(), X, y, train_sizes=olchamlar,
                                  cv=5, scoring="r2", random_state=0)
    for k, a, b in zip(n1, tr1.mean(axis=1), va1.mean(axis=1)):
        print(f"  n={k:>5}: o'quv {a:.3f}, CV {b:.3f}")

    print("\n=== 2. Murakkab model (3-darajali polinom) ===")
    model = make_pipeline(PolynomialFeatures(3), StandardScaler(), LinearRegression())
    n2, tr2, va2 = learning_curve(model, X, y, train_sizes=olchamlar,
                                  cv=5, scoring="r2", random_state=0)
    for k, a, b in zip(n2, tr2.mean(axis=1), va2.mean(axis=1)):
        print(f"  n={k:>5}: o'quv {a:.3f}, CV {b:.3f}, tafovut {a - b:.3f}")

    print("\n=== 3. Xulosa ===")
    print("  chiziqli model: egrilar birlashgan va past → ma'lumot emas, MODEL muammosi")
    print("  polinom: ma'lumot oshgani sari tafovut kamayadi va CV o'sadi")
    print("  ⭐ Learning curve — 'ko'proq ma'lumot kerakmi?' savoliga javob")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Sodda model (chiziqli) ===
  n=   80: o'quv 0.511, CV 0.421
  n=  384: o'quv 0.456, CV 0.470
  n=  688: o'quv 0.450, CV 0.468
  n=  992: o'quv 0.470, CV 0.470
  n= 1296: o'quv 0.468, CV 0.470
  n= 1600: o'quv 0.475, CV 0.470

=== 2. Murakkab model (3-darajali polinom) ===
  n=   80: o'quv 0.937, CV 0.571, tafovut 0.366
  n=  384: o'quv 0.886, CV 0.852, tafovut 0.034
  n=  688: o'quv 0.876, CV 0.859, tafovut 0.016
  n=  992: o'quv 0.868, CV 0.859, tafovut 0.009
  n= 1296: o'quv 0.863, CV 0.861, tafovut 0.002
  n= 1600: o'quv 0.869, CV 0.861, tafovut 0.007

=== 3. Xulosa ===
  chiziqli model: egrilar birlashgan va past → ma'lumot emas, MODEL muammosi
  polinom: ma'lumot oshgani sari tafovut kamayadi va CV o'sadi
  ⭐ Learning curve — 'ko'proq ma'lumot kerakmi?' savoliga javob

Nima ko'rsatdi: 2.4-bo'lim.

Misol 4 — Regularizatsiya bilan davolash

python
"""Ko'p belgi va kam kuzatuv: regularizatsiya overfittingni kamaytiradi (real numpy/sklearn)."""

import numpy as np
from sklearn.linear_model import Lasso, LinearRegression, Ridge
from sklearn.model_selection import cross_validate, train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler


def main() -> None:
    rng = np.random.default_rng(3)
    n, p = 120, 60                                    # p ~ n/2 — overfitting xavfi
    X = rng.normal(size=(n, p))
    w = np.zeros(p); w[:5] = [3, -2, 1.5, 1, -1]       # faqat 5 ta belgi muhim
    y = X @ w + rng.normal(0, 1.0, n)
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0)

    print("=== 1. Regularizatsiyasiz ===")
    lin = LinearRegression().fit(Xtr, ytr)
    print(f"  o'quv R^2 = {lin.score(Xtr, ytr):.3f}, test R^2 = {lin.score(Xte, yte):.3f}")

    print("\n=== 2. Ridge (alpha bo'yicha) ===")
    for alpha in [0.1, 1, 10, 100]:
        m = make_pipeline(StandardScaler(), Ridge(alpha=alpha)).fit(Xtr, ytr)
        print(f"  alpha={alpha:>5}: o'quv {m.score(Xtr, ytr):.3f}, test {m.score(Xte, yte):.3f}")

    print("\n=== 3. Lasso (belgi tanlash) ===")
    for alpha in [0.01, 0.1, 0.3]:
        m = make_pipeline(StandardScaler(), Lasso(alpha=alpha, max_iter=5000)).fit(Xtr, ytr)
        nol_emas = int((np.abs(m[-1].coef_) > 1e-8).sum())
        print(f"  alpha={alpha:<5}: test {m.score(Xte, yte):.3f}, "
              f"nolmas koeffitsiyentlar {nol_emas}/{p}")

    print("\n=== 4. CV bilan halol tanlov ===")
    for nom, m in [("chiziqli", LinearRegression()),
                   ("Ridge(10)", make_pipeline(StandardScaler(), Ridge(alpha=10))),
                   ("Lasso(0.1)", make_pipeline(StandardScaler(), Lasso(alpha=0.1, max_iter=5000)))]:
        r = cross_validate(m, Xtr, ytr, cv=5, return_train_score=True, scoring="r2")
        print(f"  {nom:<11}: o'quv {r['train_score'].mean():6.3f}, "
              f"CV {r['test_score'].mean():6.3f}")
    print("  ⭐ Regularizatsiya — erkinlikni cheklab, umumlashtirishni yaxshilaydi")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Regularizatsiyasiz ===
  o'quv R^2 = 0.988, test R^2 = 0.778

=== 2. Ridge (alpha bo'yicha) ===
  alpha=  0.1: o'quv 0.988, test 0.788
  alpha=    1: o'quv 0.987, test 0.840
  alpha=   10: o'quv 0.972, test 0.848
  alpha=  100: o'quv 0.784, test 0.534

=== 3. Lasso (belgi tanlash) ===
  alpha=0.01 : test 0.883, nolmas koeffitsiyentlar 56/60
  alpha=0.1  : test 0.945, nolmas koeffitsiyentlar 26/60
  alpha=0.3  : test 0.928, nolmas koeffitsiyentlar 5/60

=== 4. CV bilan halol tanlov ===
  chiziqli   : o'quv  0.995, CV  0.492
  Ridge(10)  : o'quv  0.972, CV  0.758
  Lasso(0.1) : o'quv  0.970, CV  0.904
  ⭐ Regularizatsiya — erkinlikni cheklab, umumlashtirishni yaxshilaydi

Nima ko'rsatdi: 2.5, 2.6-bo'limlar.


5. To'g'ri va noto'g'ri tushunishlar

Noto'g'ri fikr To'g'risi
"O'quv xatosi kichik — yaxshi model" Validatsiyaga qarang
"Overfitting — ko'p ma'lumot yo'qligi" Murakkablik ham sabab
"Belgi qo'shish yordam beradi" Ko'pincha kuchaytiradi
"Murakkab model — zamonaviy" Oddiy ko'pincha yaxshiroq
"Regularizatsiya aniqlikni pasaytiradi" Test aniqligini oshiradi
"Test bilan chuqurlik tanlash mumkin" Validatsiya/CV (12.3)
"Yuqori test balli — overfitting yo'q" Leakage bo'lishi mumkin
"Underfitting kam uchraydi" Juda oddiy modellarda tez-tez

6. Keng tarqalgan xatolar va yechimlari

1. Faqat o'quv balli

python
model.fit(X, y); print(model.score(X, y))                         # ⚠️
cross_validate(model, X, y, cv=5, return_train_score=True)        # ✅

2. Murakkablikni test bilan tanlash

python
for d in [3, 5, 10]: print(tree(d).fit(Xtr, ytr).score(Xte, yte)) # ⚠️
validation_curve(DecisionTreeRegressor(), X, y, param_name="max_depth", ...)  # ✅

3. Belgi qo'shib "tuzatish"

python
X = np.hstack([X, yangi_20_belgi])     # overfitting da                  # ⚠️
# regularizatsiya yoki belgilarni kamaytirish                            # ✅

4. Standartlashtirishsiz regularizatsiya

python
Ridge(alpha=1.0).fit(X_xom, y)                                    # ⚠️
make_pipeline(StandardScaler(), Ridge(alpha=1.0)).fit(X, y)       # ✅

5. Cheksiz daraxt

python
DecisionTreeRegressor().fit(Xtr, ytr)                             # ⚠️
DecisionTreeRegressor(max_depth=6, min_samples_leaf=20, random_state=0)  # ✅

6. Kichik validatsiya

python
train_test_split(X, y, test_size=0.05)     # n_val = 20           # ⚠️
cross_val_score(model, X, y, cv=5)                                # ✅

7. Leakage ni overfitting deb

python
# o'quv 0.99, test 0.98 — "overfitting yo'q" deb xulosa            # ⚠️
# belgilarni leakage uchun tekshiring 12.2-bob                       # ✅

7. Integratsiya — bu bilim qayerda kerak bo'ladi

  • 12.1-12.3-darslar (o'tilgan): Umumlashtirish, validatsiya
  • 13-qism (keyingi): Ridge va Lasso matematikasi
  • 12.5-dars: Bias-variance yoyilmasi
  • 12.10-dars: To'liq loyiha
  • Keyingi qismlar: Ansambllar, model sozlash

8. Eng yaxshi amaliyotlar

  1. Har doim ikki ballni birga ko'ring.

  2. Murakkablikni validatsiyada tanlang.

  3. Regularizatsiyadan foydalaning.

  4. Standartlashtiring (pipeline ichida).

  5. Learning curve bilan "ma'lumot kerakmi?" ni tekshiring.

  6. Daraxtlarni cheklang.

  7. Oddiy modeldan boshlang.

  8. Juda yaxshi natijada leakage ni tekshiring.


9. Amaliy topshiriq

Vazifa 1: Bashorat qiling

python
1.  # overfitting ta'rifi?
2.  # underfitting ta'rifi?
3.  # tashxis uchun nechta ball kerak?
4.  # o'quv xatosi murakkablik bilan?
5.  # validatsiya xatosi?
6.  # optimal nuqta qayerda?
7.  # validation curve nima ko'rsatadi?
8.  # learning curve nima ko'rsatadi?
9.  # Ridge nima qiladi?
10. # Lasso qo'shimcha nima qiladi?
11. # daraxtda cheklov parametrlari?
12. # o'quv 0.99, test 0.98 — ?
Javoblar
  1. O'quvda yaxshi, validatsiyada yomon
  2. Ikkalasida ham yomon
  3. Ikkita (o'quv va validatsiya)
  4. Kamayadi
  5. U shaklida (avval kamayadi, keyin o'sadi)
  6. Validatsiya minimumida
  7. Giperparametr ta'siri
  8. Ma'lumot hajmi ta'siri
  9. Koeffitsiyentlarni kichraytiradi
  10. Ba'zilarini 0 ga (belgi tanlash)
  11. max_depth, min_samples_leaf, ccp_alpha
  12. Leakage tekshirish

Vazifa 2: Xatolarni tuzating

python
1.  print("R^2:", model.fit(X, y).score(X, y))

2.  for d in [3, 6, 12]: print(d, tree(d).fit(Xtr, ytr).score(Xte, yte))

3.  Ridge(alpha=1.0).fit(X, y)   # belgilar turli birlikda

4.  DecisionTreeClassifier().fit(Xtr, ytr)   # 200 kuzatuv

5.  # overfitting: yana 30 ta belgi qo'shamiz
Javoblar
python
1.  cross_validate(model, X, y, cv=5, return_train_score=True)

2.  validation_curve(DecisionTreeRegressor(), Xtr, ytr, param_name="max_depth", ...)

3.  make_pipeline(StandardScaler(), Ridge(alpha=1.0)).fit(X, y)

4.  DecisionTreeClassifier(max_depth=4, min_samples_leaf=10, random_state=0)

5.  # regularizatsiya yoki belgilarni kamaytirish

Vazifa 3: Tashxis

Modellang:

  1. Uch model (sodda, o'rta, murakkab)
  2. O'quv va CV ballari
  3. Tashxis
  4. Davolash rejasi

Vazifa 4: Validation curve

Modellang:

  1. KNN uchun k = 1..50
  2. O'quv va CV egri chiziqlari
  3. Optimal k
  4. Grafik (matplotlib)

Vazifa 5: Learning curve

Modellang:

  1. Ikki model
  2. Turli o'quv hajmlari
  3. "Ma'lumot kerakmi?" xulosasi
  4. Tavsiya

Vazifa 6: Integratsiya

Modellang:

  1. CV (12.3)
  2. Ridge/Lasso (13-qism)
  3. Pipeline
  4. Baza model (12.1)

Vazifa 7: O'ylash

Zamonaviy chuqur o'rganish modellari milliardlab parametrga ega — ular o'quv ma'lumotini butunlay yodlab olishi mumkin, lekin baribir yaxshi umumlashtiradi. Bu klassik "murakkablik → overfitting" qarashiga zid ko'rinadi. Bu qanday tushuntiriladi va amaliy xulosa nima?

Javob

Qisqa javob: klassik qarash to'liq noto'g'ri emas, lekin u parametrlar sonini murakkablik o'lchovi deb oladi. Zamonaviy modellarda haqiqiy "samarali murakkablik" regularizatsiya, optimizatsiya usuli va ma'lumot hajmi bilan boshqariladi. Bundan tashqari, juda katta modellarda "ikki marta tushish" (double descent) hodisasi kuzatiladi: parametrlar soni kuzatuvlar sonidan ancha oshganda test xatosi yana kamayadi.

1. Nega klassik rasm to'liq emas

  • Parametrlar soni ≠ samarali murakkablik
  • Gradient tushish o'zi regularizatsiya kabi ishlaydi (implicit regularization)
  • Katta ma'lumot + augmentation + dropout + erta to'xtatish
  • Arxitektura ma'lumot tuzilishiga mos (konvolyutsiya, attention)

2. Double descent

Rejim Xato
Kam parametr Underfitting
Parametr ≈ kuzatuv soni Eng yomon (interpolatsiya chegarasi)
Parametr >> kuzatuv Yana yaxshilanadi

3. Amaliy xulosa

  1. Tashxis qoidasi o'zgarmaydi: o'quv va validatsiya ballarini solishtiring
  2. Katta modellarda ham validatsiya kerak (va u hal qiluvchi)
  3. Regularizatsiya turlari o'zgardi, lekin kerak bo'lib qolmoqda
  4. Kichik/o'rta jadval ma'lumotida oddiy modellar hali ham yutadi

4. Data Scientist qanday

  • Modelni ma'lumot hajmiga va turiga qarab tanlaydi
  • Validatsiyaga tayanadi, nazariy taxminlarga emas
  • Katta modellarda hisoblash narxini ham hisobga oladi

5. Xulosa

  1. "Ko'p parametr — albatta overfitting" — soddalashtirish
  2. Samarali murakkablik — parametrlar soni emas
  3. Validatsiya har doim hal qiluvchi
  4. Jadval ma'lumotida oddiy modellar — kuchli raqobatchi

Nimani mustahkamlaydi: 2.2, 2.5-bo'limlar.


Xulosa

Bu darsda overfitting va underfittingni o'rgandik.

Eng muhim uch fikr:

  1. Tashxis — ikki ball. Overfitting: o'quvda yaxshi, validatsiyada yomon (shovqinni yodlash). Underfitting: ikkalasida ham yomon. Faqat o'quv ballini ko'rish — eng keng tarqalgan xato; faqat test ballini ko'rish esa uni "ishlatib yuboradi" 12.3-bob.

  2. Murakkablik muvozanati. O'quv xatosi murakkablik bilan doim kamayadi, validatsiya xatosi — U shaklida. Optimal nuqtani validation curve (giperparametr bo'yicha) bilan topamiz; learning curve esa "ko'proq ma'lumot yordam beradimi?" savoliga javob beradi.

  3. Davolash. Overfitting: regularizatsiya (Ridge/Lasso — standartlashtirish bilan; daraxtlarda max_depth va min_samples_leaf), soddaroq model, kamroq belgi, ko'proq ma'lumot, erta to'xtatish, ansambllar. Underfitting: murakkabroq model, boyroq belgilar, kamroq regularizatsiya.

Keyingi darsda bias-variance yoyilmasini o'rganamiz: xatoning uch qismi (siljish, dispersiya, qaytarilmas shovqin) va ularning murakkablik bilan bog'liqligi.

ML asoslari · 4-dars yakunlandi

12.4-dars: Overfitting va underfitting

17 daqvaqt
0belgi
Keyingi dars: 12.5-dars: Bias-variance yoyilmasi
Telegram'da
Darsda xato koʻrdingizmi?

Izohlar · 0