12.4-dars: Overfitting va underfitting
12-QISM — MACHINE LEARNING ASOSLARI · 4-dars
1. Kirish va motivatsiya
12.1-darsda ko'rdik: 1-qo'shni modeli o'quv ma'lumotida mukammal (R^2 = 1.000), testda esa zaif. Bu — overfitting (haddan tashqari moslashish): model ma'lumotdagi shovqinni ham naqsh deb o'rgangan. Teskari holat — underfitting: model juda sodda, hatto o'quv ma'lumotidagi haqiqiy naqshni ham ushlay olmaydi.
Har bir ML loyihasi shu ikki chekka orasida muvozanat qidiradi. Ularni tashxis qilish oson: o'quv va validatsiya natijalarini solishtirish yetarli. Davolash esa vaziyatga bog'liq: overfitting da — ko'proq ma'lumot, kamroq belgi, regularizatsiya (13-qism), soddaroq model; underfitting da — murakkabroq model, boyroq belgilar, kamroq regularizatsiya.
Real vaziyat. Jamoa uy narxi modelini quradi: qaror daraxti, chuqurlik cheklanmagan. O'quv R^2 = 1.00, test R^2 = 0.62. Jamoa "ma'lumot kam" deb o'ylab, yana 20 ta belgi qo'shadi — test R^2 0.58 ga tushadi. Data Scientist tashxis qo'yadi: bu overfitting, belgi qo'shish uni kuchaytiradi. Yechim: daraxt chuqurligini cheklash (max_depth=6) va bargdagi minimal kuzatuvlar sonini belgilash → test R^2 = 0.81. Murakkablikni kamaytirish aniqlikni oshirdi.
Bu darsda overfitting va underfittingni o'rganamiz.
Bu darsda:
- Overfitting va underfitting: ta'rif va tashxis
- Model murakkabligi va umumlashtirish
- O'quv va validatsiya egri chiziqlari
- Ma'lumot hajmi egri chizig'i (learning curve)
- Regularizatsiya bilan boshqarish
- Erta to'xtatish va boshqa usullar
- Overfitting tuzoqlari
- Amaliy: tashxis va davolash
ℹ Misollar real numpy/sklearn bilan (Python 3.14).
2. Nazariya — chuqur tushuntirish
2.1. Ta'rif va tashxis
O'QUV xatosi VALIDATSIYA xatosi TASHXIS
juda sodda model katta katta UNDERFITTING
yaxshi model kichik kichik MUVOZANAT
juda murakkab juda kichik katta OVERFITTING
Tashxis qoidasi:
ikkalasi ham yomon → underfitting (model yetarli emas)
o'quv yaxshi, test yomon → overfitting (yodlash)
ikkalasi ham yomon va teng → belgilar yetarli emas yoki vazifa qiyinOverfitting — model o'quv ma'lumotining tasodifiy xususiyatlarini (shovqinni) o'rganib olgan: u yodlaydi, lekin umumlashtirmaydi. Underfitting — model juda sodda: hatto mavjud naqshni ham ushlay olmaydi. Tashxis uchun ikkita raqam kerak: o'quv va validatsiya natijasi. Faqat bittasiga qarash — eng keng tarqalgan xato 12.1-bob.
2.2. Model murakkabligi
Murakkablik nimalar bilan oshadi:
polinom darajasi, daraxt chuqurligi, neyron qatlamlari, belgilar soni,
KNN da k ning KICHIKligi, regularizatsiya kuchining KICHIKligi
xato
│ \ /
│ \ validatsiya /
│ \_______________ /
│ \ /
│ \___________/ ← optimal murakkablik
│ o'quv \___
└──────────────────────────────→ murakkablik
O'quv xatosi murakkablik bilan doim kamayadi; validatsiya xatosi — U shaklidaMurakkablik — modelning ma'lumotga moslashish qobiliyati. Kichik murakkablikda model haqiqiy naqshni ham ushlay olmaydi (underfitting); haddan tashqari murakkablikda shovqinni ham o'rganadi (overfitting). Optimal nuqta — validatsiya xatosi minimal bo'lgan joy. Uni faqat validatsiya 12.3-bob orqali topish mumkin — o'quv xatosi bu haqda hech narsa aytmaydi.
2.3. O'quv va validatsiya egri chiziqlari
VALIDATION CURVE — giperparametr bo'yicha:
sklearn.model_selection.validation_curve(model, X, y, param_name=..., param_range=..., cv=5)
→ har qiymat uchun o'quv va validatsiya balli
O'qish tartibi:
ikki egri ham past → underfitting (murakkablikni oshiring)
egrilar orasida katta tafovut → overfitting (murakkablikni kamaytiring)
egrilar yaqin va yuqori → yaxshiValidation curve — giperparametrni tanlashning vizual usuli: murakkablik oshgani sari o'quv balli o'sadi, validatsiya balli esa avval o'sib, keyin tushadi. Tafovut (gap) kattaligi — overfitting darajasi. Bu grafik giperparametr qiymatini "ko'z bilan" tanlashga yordam beradi (keyinroq — avtomatik qidiruv, 12.10 va model baholash qismi).
2.4. Ma'lumot hajmi egri chizig'i
LEARNING CURVE — o'quv hajmi bo'yicha:
sklearn.model_selection.learning_curve(model, X, y, train_sizes=..., cv=5)
Katta tafovut, validatsiya o'smoqda → KO'PROQ MA'LUMOT yordam beradi
Egrilar birlashgan, ikkalasi ham past → ma'lumot emas, MODEL/BELGILAR muammosiLearning curve muhim amaliy savolga javob beradi: "Ko'proq ma'lumot yig'ish foyda beradimi?". Agar validatsiya egri chizig'i hali o'sayotgan bo'lsa — ha; agar u tekislanib, o'quv egri chizig'i bilan birlashgan bo'lsa — yo'q (belgilar yoki model kerak). Bu ko'p resurs tejaydi: ma'lumot yig'ish qimmat.
2.5. Regularizatsiya
G'oya: modelga "murakkablik jarimasi" qo'shish
Ridge (L2): min ||y - Xw||^2 + alpha ||w||^2 — koeffitsiyentlarni kichraytiradi (13-qism)
Lasso (L1): ... + alpha ||w||_1 — ba'zilarini aniq 0 ga (belgi tanlash)
Daraxtlar: max_depth, min_samples_leaf, ccp_alpha
KNN: k ni oshirish
Neyron: weight decay, dropout, erta to'xtatish
alpha (yoki C = 1/alpha) — giperparametr: validatsiyada tanlanadiRegularizatsiya — overfittingga qarshi asosiy vosita: model erkinligini cheklaydi. Chiziqli modellarda — koeffitsiyentlarni kichraytirish (13-qism); daraxtlarda — o'sishni cheklash; neyron tarmoqlarda — dropout va erta to'xtatish. Muhim: regularizatsiyadan oldin standartlashtirish 12.9-bob, aks holda jarima birliklarga bog'liq bo'ladi.
2.6. Boshqa usullar
Overfittingni kamaytirishning boshqa yo'llari: ko'proq ma'lumot (eng ishonchli, lekin qimmat); belgilarni kamaytirish (ortiqcha va shovqinli belgilarni olib tashlash); erta to'xtatish (iterativ modellarda — validatsiya yomonlashganda to'xtatish); ansambllar (bagging — Random Forest, dispersiyani kamaytiradi); ma'lumotni boyitish (augmentation — rasm va matnda); kross-validatsiya bilan halol tanlov 12.3-bob. Underfittingga qarshi: murakkabroq model, ko'proq/boyroq belgilar, regularizatsiyani kamaytirish, uzoqroq o'qitish.
2.7. Overfitting tuzoqlari
Asosiy tuzoqlar: faqat o'quv ballini ko'rish; test bilan murakkablik tanlash 12.3-bob; leakage ni overfitting deb o'ylash (leakage da test ham yaxshi — 12.2); kichik validatsiya (baho beqaror); belgi qo'shish orqali "tuzatish" (ko'pincha kuchaytiradi); regularizatsiyasiz murakkab model; standartlashtirishsiz regularizatsiya; CV natijasiga cheksiz moslashish 11.9-bob; "murakkab model — zamonaviy" degan qarash.
2.8. Overfitting — murakkablik muvozanati
Overfitting — o'quvda yaxshi, validatsiyada yomon (shovqinni yodlash); underfitting — ikkalasida ham yomon (model yetarli emas). Tashxis — ikki ballni solishtirish; validation curve murakkablik bo'yicha, learning curve ma'lumot hajmi bo'yicha yo'l ko'rsatadi. Davolash: overfittingda — regularizatsiya, soddaroq model, kamroq belgi, ko'proq ma'lumot, erta to'xtatish, ansambl; underfittingda — murakkabroq model, boyroq belgilar. Keyingi dars — bu muvozanatning nazariy tushuntirishi: bias-variance.
3. Tez ma'lumotnoma
import numpy as np
from sklearn.model_selection import learning_curve, validation_curve, cross_validate
# tashxis: o'quv va validatsiya birga
res = cross_validate(model, X, y, cv=5, return_train_score=True)
res["train_score"].mean(), res["test_score"].mean() # tafovut — overfitting belgisi
# validation curve (giperparametr)
tr, va = validation_curve(Ridge(), X, y, param_name="alpha",
param_range=np.logspace(-3, 3, 7), cv=5)
# learning curve (ma'lumot hajmi)
n, tr, va = learning_curve(model, X, y, train_sizes=np.linspace(0.1, 1.0, 6), cv=5)
# regularizatsiya
Ridge(alpha=1.0); Lasso(alpha=0.1)
DecisionTreeRegressor(max_depth=6, min_samples_leaf=20)
KNeighborsRegressor(n_neighbors=15)
QOIDA: ikki ballni birga ko'r · murakkablikni validatsiyada tanla · standartlashtirOverfitting xulosasi
Overfitting — o'quv yaxshi, validatsiya yomon (shovqinni yodlash)
Underfitting — ikkalasi ham yomon (model sodda)
Tashxis — train va validation ballari orasidagi tafovut
Validation curve — murakkablik; learning curve — ma'lumot hajmi
Davolash: regularizatsiya, soddaroq model, kamroq belgi, ko'proq ma'lumot
Underfitting: murakkabroq model, boyroq belgilar4. Batafsil misollar
Misollar real numpy/sklearn bilan (Python 3.14).
Misol 1 — Polinom darajasi: underfittingdan overfittinggacha
"""Polinom darajasi bo'yicha o'quv va validatsiya xatosi (real numpy/sklearn)."""
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures, StandardScaler
def main() -> None:
rng = np.random.default_rng(0)
n = 120
x = rng.uniform(-3, 3, n)
y = 0.5 * x ** 3 - 2 * x + 1 + rng.normal(0, 2.5, n) # haqiqiy: 3-daraja
X = x.reshape(-1, 1)
Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.35, random_state=0)
print("=== 1. Daraja bo'yicha RMSE ===")
print(f" {'daraja':>7} {"o'quv":>9} {'test':>9} tashxis")
for d in [1, 2, 3, 5, 9, 15]:
model = make_pipeline(PolynomialFeatures(d), StandardScaler(), LinearRegression())
model.fit(Xtr, ytr)
rtr = np.sqrt(mean_squared_error(ytr, model.predict(Xtr)))
rte = np.sqrt(mean_squared_error(yte, model.predict(Xte)))
if rtr > 3.0 and rte > 3.0:
tashxis = "underfitting"
elif rte > 1.15 * rtr:
tashxis = "overfitting"
else:
tashxis = "muvozanat"
print(f" {d:>7} {rtr:>9.2f} {rte:>9.2f} {tashxis}")
print("\n=== 2. Shovqin darajasi (haqiqiy SD = 2.5) ===")
print(" test RMSE shovqin darajasidan pastga tusha olmaydi (nazariy chegara)")
print("\n=== 3. 15-darajali modelning koeffitsiyentlari ===")
m15 = make_pipeline(PolynomialFeatures(15), StandardScaler(), LinearRegression()).fit(Xtr, ytr)
koef = m15[-1].coef_
print(f" eng katta |koef| = {np.abs(koef).max():,.0f} — juda katta qiymatlar")
print(" ⭐ O'quv xatosi tushadi, test xatosi ko'tariladi")
if __name__ == "__main__":
main()Natijaning muhim qismi:
=== 1. Daraja bo'yicha RMSE ===
daraja o'quv test tashxis
1 3.53 3.70 underfitting
2 3.52 3.77 underfitting
3 2.52 2.66 muvozanat
5 2.42 2.70 muvozanat
9 2.38 2.75 overfitting
15 2.34 2.95 overfitting
=== 2. Shovqin darajasi (haqiqiy SD = 2.5) ===
test RMSE shovqin darajasidan pastga tusha olmaydi (nazariy chegara)
=== 3. 15-darajali modelning koeffitsiyentlari ===
eng katta |koef| = 2,917 — juda katta qiymatlar
⭐ O'quv xatosi tushadi, test xatosi ko'tariladiNima ko'rsatdi: 2.1, 2.2-bo'limlar.
Misol 2 — Validation curve: daraxt chuqurligi
"""Daraxt chuqurligi bo'yicha validation curve (real numpy/sklearn)."""
import numpy as np
from sklearn.model_selection import validation_curve
from sklearn.tree import DecisionTreeRegressor
def main() -> None:
rng = np.random.default_rng(1)
n = 1200
X = rng.normal(size=(n, 6))
y = (2 * X[:, 0] + X[:, 1] ** 2 - 1.5 * X[:, 2] * X[:, 3]
+ rng.normal(0, 1.0, n))
chuqurliklar = [1, 2, 3, 5, 8, 12, 20, None]
param = [d if d is not None else 50 for d in chuqurliklar]
tr, va = validation_curve(
DecisionTreeRegressor(random_state=0), X, y,
param_name="max_depth", param_range=param, cv=5, scoring="r2")
print("=== 1. Chuqurlik bo'yicha R^2 ===")
print(f" {'chuqurlik':>10} {"o'quv":>8} {'CV':>8} {'tafovut':>9}")
for d, a, b in zip(chuqurliklar, tr.mean(axis=1), va.mean(axis=1)):
nom = "cheksiz" if d is None else str(d)
print(f" {nom:>10} {a:>8.3f} {b:>8.3f} {a - b:>9.3f}")
eng = int(np.argmax(va.mean(axis=1)))
print(f"\n=== 2. Eng yaxshi CV: chuqurlik = "
f"{chuqurliklar[eng] if chuqurliklar[eng] else 'cheksiz'} "
f"(CV R^2 = {va.mean(axis=1)[eng]:.3f})")
print("\n=== 3. Boshqa cheklovlar ===")
for leaf in [1, 5, 20, 50]:
m = DecisionTreeRegressor(min_samples_leaf=leaf, random_state=0)
from sklearn.model_selection import cross_validate
r = cross_validate(m, X, y, cv=5, return_train_score=True, scoring="r2")
print(f" min_samples_leaf={leaf:>3}: o'quv {r['train_score'].mean():.3f}, "
f"CV {r['test_score'].mean():.3f}")
print(" ⭐ Murakkablikni cheklash — CV natijasini yaxshilaydi")
if __name__ == "__main__":
main()Natijaning muhim qismi:
=== 1. Chuqurlik bo'yicha R^2 ===
chuqurlik o'quv CV tafovut
1 0.334 0.326 0.008
2 0.431 0.375 0.056
3 0.521 0.456 0.065
5 0.650 0.469 0.182
8 0.775 0.380 0.395
12 0.892 0.311 0.581
20 0.984 0.226 0.758
cheksiz 1.000 0.231 0.769
=== 2. Eng yaxshi CV: chuqurlik = 5 (CV R^2 = 0.469)
=== 3. Boshqa cheklovlar ===
min_samples_leaf= 1: o'quv 1.000, CV 0.231
min_samples_leaf= 5: o'quv 0.821, CV 0.477
min_samples_leaf= 20: o'quv 0.630, CV 0.493
min_samples_leaf= 50: o'quv 0.532, CV 0.460
⭐ Murakkablikni cheklash — CV natijasini yaxshilaydiNima ko'rsatdi: 2.3, 2.5-bo'limlar.
Misol 3 — Learning curve: ko'proq ma'lumot yordam beradimi?
"""Learning curve: ma'lumot hajmi va model murakkabligi (real numpy/sklearn)."""
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import learning_curve
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures, StandardScaler
def main() -> None:
rng = np.random.default_rng(2)
n = 2000
X = rng.normal(size=(n, 4))
y = (X[:, 0] ** 2 + 2 * X[:, 1] - X[:, 2] * X[:, 3] + rng.normal(0, 1.0, n))
olchamlar = np.linspace(0.05, 1.0, 6)
print("=== 1. Sodda model (chiziqli) ===")
n1, tr1, va1 = learning_curve(LinearRegression(), X, y, train_sizes=olchamlar,
cv=5, scoring="r2", random_state=0)
for k, a, b in zip(n1, tr1.mean(axis=1), va1.mean(axis=1)):
print(f" n={k:>5}: o'quv {a:.3f}, CV {b:.3f}")
print("\n=== 2. Murakkab model (3-darajali polinom) ===")
model = make_pipeline(PolynomialFeatures(3), StandardScaler(), LinearRegression())
n2, tr2, va2 = learning_curve(model, X, y, train_sizes=olchamlar,
cv=5, scoring="r2", random_state=0)
for k, a, b in zip(n2, tr2.mean(axis=1), va2.mean(axis=1)):
print(f" n={k:>5}: o'quv {a:.3f}, CV {b:.3f}, tafovut {a - b:.3f}")
print("\n=== 3. Xulosa ===")
print(" chiziqli model: egrilar birlashgan va past → ma'lumot emas, MODEL muammosi")
print(" polinom: ma'lumot oshgani sari tafovut kamayadi va CV o'sadi")
print(" ⭐ Learning curve — 'ko'proq ma'lumot kerakmi?' savoliga javob")
if __name__ == "__main__":
main()Natijaning muhim qismi:
=== 1. Sodda model (chiziqli) ===
n= 80: o'quv 0.511, CV 0.421
n= 384: o'quv 0.456, CV 0.470
n= 688: o'quv 0.450, CV 0.468
n= 992: o'quv 0.470, CV 0.470
n= 1296: o'quv 0.468, CV 0.470
n= 1600: o'quv 0.475, CV 0.470
=== 2. Murakkab model (3-darajali polinom) ===
n= 80: o'quv 0.937, CV 0.571, tafovut 0.366
n= 384: o'quv 0.886, CV 0.852, tafovut 0.034
n= 688: o'quv 0.876, CV 0.859, tafovut 0.016
n= 992: o'quv 0.868, CV 0.859, tafovut 0.009
n= 1296: o'quv 0.863, CV 0.861, tafovut 0.002
n= 1600: o'quv 0.869, CV 0.861, tafovut 0.007
=== 3. Xulosa ===
chiziqli model: egrilar birlashgan va past → ma'lumot emas, MODEL muammosi
polinom: ma'lumot oshgani sari tafovut kamayadi va CV o'sadi
⭐ Learning curve — 'ko'proq ma'lumot kerakmi?' savoliga javobNima ko'rsatdi: 2.4-bo'lim.
Misol 4 — Regularizatsiya bilan davolash
"""Ko'p belgi va kam kuzatuv: regularizatsiya overfittingni kamaytiradi (real numpy/sklearn)."""
import numpy as np
from sklearn.linear_model import Lasso, LinearRegression, Ridge
from sklearn.model_selection import cross_validate, train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
def main() -> None:
rng = np.random.default_rng(3)
n, p = 120, 60 # p ~ n/2 — overfitting xavfi
X = rng.normal(size=(n, p))
w = np.zeros(p); w[:5] = [3, -2, 1.5, 1, -1] # faqat 5 ta belgi muhim
y = X @ w + rng.normal(0, 1.0, n)
Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0)
print("=== 1. Regularizatsiyasiz ===")
lin = LinearRegression().fit(Xtr, ytr)
print(f" o'quv R^2 = {lin.score(Xtr, ytr):.3f}, test R^2 = {lin.score(Xte, yte):.3f}")
print("\n=== 2. Ridge (alpha bo'yicha) ===")
for alpha in [0.1, 1, 10, 100]:
m = make_pipeline(StandardScaler(), Ridge(alpha=alpha)).fit(Xtr, ytr)
print(f" alpha={alpha:>5}: o'quv {m.score(Xtr, ytr):.3f}, test {m.score(Xte, yte):.3f}")
print("\n=== 3. Lasso (belgi tanlash) ===")
for alpha in [0.01, 0.1, 0.3]:
m = make_pipeline(StandardScaler(), Lasso(alpha=alpha, max_iter=5000)).fit(Xtr, ytr)
nol_emas = int((np.abs(m[-1].coef_) > 1e-8).sum())
print(f" alpha={alpha:<5}: test {m.score(Xte, yte):.3f}, "
f"nolmas koeffitsiyentlar {nol_emas}/{p}")
print("\n=== 4. CV bilan halol tanlov ===")
for nom, m in [("chiziqli", LinearRegression()),
("Ridge(10)", make_pipeline(StandardScaler(), Ridge(alpha=10))),
("Lasso(0.1)", make_pipeline(StandardScaler(), Lasso(alpha=0.1, max_iter=5000)))]:
r = cross_validate(m, Xtr, ytr, cv=5, return_train_score=True, scoring="r2")
print(f" {nom:<11}: o'quv {r['train_score'].mean():6.3f}, "
f"CV {r['test_score'].mean():6.3f}")
print(" ⭐ Regularizatsiya — erkinlikni cheklab, umumlashtirishni yaxshilaydi")
if __name__ == "__main__":
main()Natijaning muhim qismi:
=== 1. Regularizatsiyasiz ===
o'quv R^2 = 0.988, test R^2 = 0.778
=== 2. Ridge (alpha bo'yicha) ===
alpha= 0.1: o'quv 0.988, test 0.788
alpha= 1: o'quv 0.987, test 0.840
alpha= 10: o'quv 0.972, test 0.848
alpha= 100: o'quv 0.784, test 0.534
=== 3. Lasso (belgi tanlash) ===
alpha=0.01 : test 0.883, nolmas koeffitsiyentlar 56/60
alpha=0.1 : test 0.945, nolmas koeffitsiyentlar 26/60
alpha=0.3 : test 0.928, nolmas koeffitsiyentlar 5/60
=== 4. CV bilan halol tanlov ===
chiziqli : o'quv 0.995, CV 0.492
Ridge(10) : o'quv 0.972, CV 0.758
Lasso(0.1) : o'quv 0.970, CV 0.904
⭐ Regularizatsiya — erkinlikni cheklab, umumlashtirishni yaxshilaydiNima ko'rsatdi: 2.5, 2.6-bo'limlar.
5. To'g'ri va noto'g'ri tushunishlar
| Noto'g'ri fikr | To'g'risi |
|---|---|
| "O'quv xatosi kichik — yaxshi model" | Validatsiyaga qarang |
| "Overfitting — ko'p ma'lumot yo'qligi" | Murakkablik ham sabab |
| "Belgi qo'shish yordam beradi" | Ko'pincha kuchaytiradi |
| "Murakkab model — zamonaviy" | Oddiy ko'pincha yaxshiroq |
| "Regularizatsiya aniqlikni pasaytiradi" | Test aniqligini oshiradi |
| "Test bilan chuqurlik tanlash mumkin" | Validatsiya/CV (12.3) |
| "Yuqori test balli — overfitting yo'q" | Leakage bo'lishi mumkin |
| "Underfitting kam uchraydi" | Juda oddiy modellarda tez-tez |
6. Keng tarqalgan xatolar va yechimlari
1. Faqat o'quv balli
model.fit(X, y); print(model.score(X, y)) # ⚠️
cross_validate(model, X, y, cv=5, return_train_score=True) # ✅2. Murakkablikni test bilan tanlash
for d in [3, 5, 10]: print(tree(d).fit(Xtr, ytr).score(Xte, yte)) # ⚠️
validation_curve(DecisionTreeRegressor(), X, y, param_name="max_depth", ...) # ✅3. Belgi qo'shib "tuzatish"
X = np.hstack([X, yangi_20_belgi]) # overfitting da # ⚠️
# regularizatsiya yoki belgilarni kamaytirish # ✅4. Standartlashtirishsiz regularizatsiya
Ridge(alpha=1.0).fit(X_xom, y) # ⚠️
make_pipeline(StandardScaler(), Ridge(alpha=1.0)).fit(X, y) # ✅5. Cheksiz daraxt
DecisionTreeRegressor().fit(Xtr, ytr) # ⚠️
DecisionTreeRegressor(max_depth=6, min_samples_leaf=20, random_state=0) # ✅6. Kichik validatsiya
train_test_split(X, y, test_size=0.05) # n_val = 20 # ⚠️
cross_val_score(model, X, y, cv=5) # ✅7. Leakage ni overfitting deb
# o'quv 0.99, test 0.98 — "overfitting yo'q" deb xulosa # ⚠️
# belgilarni leakage uchun tekshiring 12.2-bob # ✅7. Integratsiya — bu bilim qayerda kerak bo'ladi
- 12.1-12.3-darslar (o'tilgan): Umumlashtirish, validatsiya
- 13-qism (keyingi): Ridge va Lasso matematikasi
- 12.5-dars: Bias-variance yoyilmasi
- 12.10-dars: To'liq loyiha
- Keyingi qismlar: Ansambllar, model sozlash
8. Eng yaxshi amaliyotlar
Har doim ikki ballni birga ko'ring.
Murakkablikni validatsiyada tanlang.
Regularizatsiyadan foydalaning.
Standartlashtiring (pipeline ichida).
Learning curve bilan "ma'lumot kerakmi?" ni tekshiring.
Daraxtlarni cheklang.
Oddiy modeldan boshlang.
Juda yaxshi natijada leakage ni tekshiring.
9. Amaliy topshiriq
Vazifa 1: Bashorat qiling
1. # overfitting ta'rifi?
2. # underfitting ta'rifi?
3. # tashxis uchun nechta ball kerak?
4. # o'quv xatosi murakkablik bilan?
5. # validatsiya xatosi?
6. # optimal nuqta qayerda?
7. # validation curve nima ko'rsatadi?
8. # learning curve nima ko'rsatadi?
9. # Ridge nima qiladi?
10. # Lasso qo'shimcha nima qiladi?
11. # daraxtda cheklov parametrlari?
12. # o'quv 0.99, test 0.98 — ?Javoblar
- O'quvda yaxshi, validatsiyada yomon
- Ikkalasida ham yomon
- Ikkita (o'quv va validatsiya)
- Kamayadi
- U shaklida (avval kamayadi, keyin o'sadi)
- Validatsiya minimumida
- Giperparametr ta'siri
- Ma'lumot hajmi ta'siri
- Koeffitsiyentlarni kichraytiradi
- Ba'zilarini 0 ga (belgi tanlash)
- max_depth, min_samples_leaf, ccp_alpha
- Leakage tekshirish
Vazifa 2: Xatolarni tuzating
1. print("R^2:", model.fit(X, y).score(X, y))
2. for d in [3, 6, 12]: print(d, tree(d).fit(Xtr, ytr).score(Xte, yte))
3. Ridge(alpha=1.0).fit(X, y) # belgilar turli birlikda
4. DecisionTreeClassifier().fit(Xtr, ytr) # 200 kuzatuv
5. # overfitting: yana 30 ta belgi qo'shamizJavoblar
1. cross_validate(model, X, y, cv=5, return_train_score=True)
2. validation_curve(DecisionTreeRegressor(), Xtr, ytr, param_name="max_depth", ...)
3. make_pipeline(StandardScaler(), Ridge(alpha=1.0)).fit(X, y)
4. DecisionTreeClassifier(max_depth=4, min_samples_leaf=10, random_state=0)
5. # regularizatsiya yoki belgilarni kamaytirishVazifa 3: Tashxis
Modellang:
- Uch model (sodda, o'rta, murakkab)
- O'quv va CV ballari
- Tashxis
- Davolash rejasi
Vazifa 4: Validation curve
Modellang:
- KNN uchun k = 1..50
- O'quv va CV egri chiziqlari
- Optimal k
- Grafik (matplotlib)
Vazifa 5: Learning curve
Modellang:
- Ikki model
- Turli o'quv hajmlari
- "Ma'lumot kerakmi?" xulosasi
- Tavsiya
Vazifa 6: Integratsiya
Modellang:
- CV (12.3)
- Ridge/Lasso (13-qism)
- Pipeline
- Baza model (12.1)
Vazifa 7: O'ylash
Zamonaviy chuqur o'rganish modellari milliardlab parametrga ega — ular o'quv ma'lumotini butunlay yodlab olishi mumkin, lekin baribir yaxshi umumlashtiradi. Bu klassik "murakkablik → overfitting" qarashiga zid ko'rinadi. Bu qanday tushuntiriladi va amaliy xulosa nima?
Javob
Qisqa javob: klassik qarash to'liq noto'g'ri emas, lekin u parametrlar sonini murakkablik o'lchovi deb oladi. Zamonaviy modellarda haqiqiy "samarali murakkablik" regularizatsiya, optimizatsiya usuli va ma'lumot hajmi bilan boshqariladi. Bundan tashqari, juda katta modellarda "ikki marta tushish" (double descent) hodisasi kuzatiladi: parametrlar soni kuzatuvlar sonidan ancha oshganda test xatosi yana kamayadi.
1. Nega klassik rasm to'liq emas
- Parametrlar soni ≠ samarali murakkablik
- Gradient tushish o'zi regularizatsiya kabi ishlaydi (implicit regularization)
- Katta ma'lumot + augmentation + dropout + erta to'xtatish
- Arxitektura ma'lumot tuzilishiga mos (konvolyutsiya, attention)
2. Double descent
| Rejim | Xato |
|---|---|
| Kam parametr | Underfitting |
| Parametr ≈ kuzatuv soni | Eng yomon (interpolatsiya chegarasi) |
| Parametr >> kuzatuv | Yana yaxshilanadi |
3. Amaliy xulosa
- Tashxis qoidasi o'zgarmaydi: o'quv va validatsiya ballarini solishtiring
- Katta modellarda ham validatsiya kerak (va u hal qiluvchi)
- Regularizatsiya turlari o'zgardi, lekin kerak bo'lib qolmoqda
- Kichik/o'rta jadval ma'lumotida oddiy modellar hali ham yutadi
4. Data Scientist qanday
- Modelni ma'lumot hajmiga va turiga qarab tanlaydi
- Validatsiyaga tayanadi, nazariy taxminlarga emas
- Katta modellarda hisoblash narxini ham hisobga oladi
5. Xulosa
- "Ko'p parametr — albatta overfitting" — soddalashtirish
- Samarali murakkablik — parametrlar soni emas
- Validatsiya har doim hal qiluvchi
- Jadval ma'lumotida oddiy modellar — kuchli raqobatchi
Nimani mustahkamlaydi: 2.2, 2.5-bo'limlar.
Xulosa
Bu darsda overfitting va underfittingni o'rgandik.
Eng muhim uch fikr:
Tashxis — ikki ball. Overfitting: o'quvda yaxshi, validatsiyada yomon (shovqinni yodlash). Underfitting: ikkalasida ham yomon. Faqat o'quv ballini ko'rish — eng keng tarqalgan xato; faqat test ballini ko'rish esa uni "ishlatib yuboradi" 12.3-bob.
Murakkablik muvozanati. O'quv xatosi murakkablik bilan doim kamayadi, validatsiya xatosi — U shaklida. Optimal nuqtani validation curve (giperparametr bo'yicha) bilan topamiz; learning curve esa "ko'proq ma'lumot yordam beradimi?" savoliga javob beradi.
Davolash. Overfitting: regularizatsiya (Ridge/Lasso — standartlashtirish bilan; daraxtlarda max_depth va min_samples_leaf), soddaroq model, kamroq belgi, ko'proq ma'lumot, erta to'xtatish, ansambllar. Underfitting: murakkabroq model, boyroq belgilar, kamroq regularizatsiya.
Keyingi darsda bias-variance yoyilmasini o'rganamiz: xatoning uch qismi (siljish, dispersiya, qaytarilmas shovqin) va ularning murakkablik bilan bog'liqligi.
Izohlar · 0
Izoh yozish uchun kiring.