IlmHamroh
Python kursi/ML va AI4/12-dars15 daqiqa
Mundarija (22)

25.4-dars: Modelni baholash

25-QISM — ML VA AI · 4-dars

1. Kirish va motivatsiya

25.2 da aniqlik (accuracy) bilan modelni baholadik. Lekin aniqlik yetmaydi — hatto chalg'itishi mumkin. Masalan: 1000 xatdan 10 tasi spam. "Hech qanday spam yo'q" deb aytgan model 99% aniq (990/1000) — lekin foydasiz (bironta spamni topmaydi). Nomutanosib ma'lumotda (kam spam) aniqlik yolg'on. Modelni to'g'ri baholash — ML ning muhim mahorati.

Model baholash — model qancha yaxshi ekanini to'g'ri o'lchash: klassifikatsiyada precision (bashorat qilganlaringdan qanchasi to'g'ri), recall (haqiqiy musbatlardan qanchasini topding), F1 (ikkalasi muvozanati), chalkashlik matritsasi (qayerda xato). Regressiyada MAE (o'rtacha xato), R2 (tushuntirilgan o'zgaruvchanlik). Cross-validation — ishonchli baho (bir bo'lish emas, ko'p marta). To'g'ri o'lchov — model qancha ishonchli ekanini ko'rsatadi.

Real vaziyat. Bir kasallik aniqlash modeli 95% aniq edi — "ajoyib!". Lekin recall 40% edi — kasallarning 60% ni o'tkazib yubordi (sog'lom deb aytdi). Kasallik aniqlashda recall muhim (kasalni o'tkazib yubormaslik). Aniqlik chalg'itdi, recall haqiqatni ko'rsatdi. To'g'ri o'lchov — model maqsadga bog'liq.

Bu darsda modelni to'g'ri baholashni o'rganamiz.

Bu darsda:

  • Nega aniqlik yetmaydi
  • Precision va recall
  • F1-ball
  • Chalkashlik matritsasi
  • Regressiya o'lchovlari (MAE, R2)
  • Cross-validation
  • O'lchov tanlovi (maqsadga qarab)
  • Amaliy: to'liq baholash

ℹ Misollarda scikit-learn bilan sinaladi.

2. Nazariya — chuqur tushuntirish

2.1. Nega aniqlik yetmaydi

Nomutanosib ma'lumotda aniqlik chalg'itadi:

1000 xat: 990 oddiy, 10 spam
Model: "hammasi oddiy"
Aniqlik: 990/1000 = 99%  (ajoyib ko'rinadi!)
Lekin: 0 spam topdi (foydasiz)

Aniqlik (accuracy) — to'g'ri bashorat ulushi — lekin nomutanosib ma'lumotda (kam musbat) chalg'itadi: "hammasi oddiy" model 99% aniq (990/1000), lekin bironta spamni topmaydi (foydasiz). Aniqlik barcha sinfni teng ko'radi. Kam sinf (spam, kasallik) muhim bo'lganda — aniqlik yetmaydi. Boshqa o'lchovlar (precision, recall) kerak.

2.2. Precision va recall

Ikki muhim o'lchov:

Precision: bashorat qilgan musbatlardan qanchasi to'g'ri?
   (spam dedik — qanchasi haqiqatan spam)
Recall: haqiqiy musbatlardan qanchasini topding?
   (haqiqiy spamning qanchasini topding)

Precision (aniqlik) — bashorat qilgan musbatlardan qanchasi to'g'ri (spam dedik — necha % haqiqatan spam; noto'g'ri ayblashdan qochish). Recall (to'liqlik) — haqiqiy musbatlardan qanchasini topding (haqiqiy spamning necha % topding; o'tkazib yubormaslik). Farq: precision — "ayblaganim to'g'rimi", recall — "hammasini topdimmi". Muammoga qarab muhim (spam — precision; kasallik — recall).

2.3. F1-ball

Precision va recall muvozanati:

python
from sklearn.metrics import f1_score
f1_score(y_true, y_pred)
# F1 = 2 * (precision * recall) / (precision + recall)
# ikkalasi yuqori bo'lganda yuqori

F1-ball — precision va recall garmonik o'rtachasi (muvozanat): ikkalasi yuqori bo'lganda yuqori, bittasi past bo'lsa F1 ham past. Sabab: precision va recall o'zaro (birini oshirsang, ikkinchi tushishi mumkin) — F1 muvozanatni o'lchaydi. Qachan: precision va recall ikkalasi muhim bo'lganda (bir umumiy o'lchov). classification_report — hammasini ko'rsatadi (precision, recall, F1 har sinf uchun).

2.4. Chalkashlik matritsasi

Qayerda xato:

python
from sklearn.metrics import confusion_matrix
confusion_matrix(y_true, y_pred)
# [[TN, FP],     TN: to'g'ri manfiy
#  [FN, TP]]     FP: yolg'on musbat (I tur)
#                FN: yolg'on manfiy (II tur)
#                TP: to'g'ri musbat

Chalkashlik matritsasi (confusion matrix) — bashoratlar taqsimoti: TP (to'g'ri musbat — spamni spam dedik), TN (to'g'ri manfiy), FP (yolg'on musbat — oddiyni spam dedik, I tur xato — 24.17), FN (yolg'on manfiy — spamni o'tkazib yubordik, II tur). Bu qayerda xato ekanini ko'rsatadi (FP ko'pmi yoki FN?). Precision, recall shundan hisoblanadi. Model xatosini tushunish.

2.5. Regressiya o'lchovlari (MAE, R2)

Son bashorat baholash:

python
from sklearn.metrics import mean_absolute_error, r2_score
mean_absolute_error(y_true, y_pred)   # o'rtacha mutlaq xato
r2_score(y_true, y_pred)              # tushuntirilgan o'zgaruvchanlik (0-1)
# RMSE: root mean squared error

Regressiya o'lchovlari (son bashorat — 25.1): MAE (Mean Absolute Error — o'rtacha mutlaq xato: bashorat qancha chetga, real birlikda — narx 5 mln xato), RMSE (kvadratik — katta xatoni ko'proq jazolaydi), R2 (0–1 — model o'zgaruvchanlikning qanchasini tushuntiradi; 1 = mukammal, 0 = o'rtacha kabi). MAE — tushunarli (real birlik), R2 — umumiy sifat. Klassifikatsiyada accuracy, regressiyada MAE/R2.

2.6. Cross-validation

Ishonchli baho (ko'p bo'lish):

python
from sklearn.model_selection import cross_val_score
scores = cross_val_score(model, X, y, cv=5)
# ma'lumotni 5 qismga bo'lib, 5 marta bahola
scores.mean()     # o'rtacha (ishonchli)

Cross-validation (o'zaro tekshirish) — ma'lumotni cv=5 qismga bo'lib, 5 marta o'qit/test (har qism bir marta test): cross_val_score(model, X, y, cv=5) → 5 ball, .mean() (o'rtacha). Sabab: bir train_test_split tasodifga bog'liq (baxtli/baxtsiz bo'lish) — cross-validation ko'p marta (ishonchli, barqaror baho). Bu 25.2 (bir bo'lish) ni yaxshilaydi. Model baholashning ishonchli usuli.

2.7. O'lchov tanlovi (maqsadga qarab)

Vaziyat Muhim o'lchov
Spam (noto'g'ri ayblash yomon) Precision
Kasallik (o'tkazib yuborish yomon) Recall
Muvozanat kerak F1
Nomutanosib ma'lumot Precision/recall/F1 (aniqlik emas)

O'lchov tanlovi maqsadga bog'liq: precision (yolg'on musbat yomon — spam filtri: oddiy xatni spamga yubormaslik), recall (yolg'on manfiy yomon — kasallik: kasalni o'tkazib yubormaslik), F1 (muvozanat), accuracy (mutanosib ma'lumot). "Xato narxi" muammoga qarab (spam vs kasallik — har xil xato muhim). To'g'ri o'lchov — model maqsadini aks ettiradi.

2.8. Halol baholash

Halol baholash — modelning haqiqiy unumdorligini ko'rsatish: test'da (train emas — 25.2), to'g'ri o'lchov (aniqlik yetmasa precision/recall), cross-validation (bir bo'lish tasodif), maqsadga mos (xato narxi). Muhim: bir raqam (accuracy) modelni to'liq tavsiflamaydi — kontekst kerak (nomutanosibmi? qaysi xato yomon?). "Model qancha yaxshi?" — "nima uchun?" savoliga bog'liq. Halol baholash — modelni to'g'ri tushunish (chalg'itmaslik, 24.16 statistika kabi).

3. Tez ma'lumotnoma

python
from sklearn.metrics import (accuracy_score, precision_score, recall_score,
                             f1_score, confusion_matrix, classification_report,
                             mean_absolute_error, r2_score)
from sklearn.model_selection import cross_val_score

# klassifikatsiya:
accuracy_score(y_true, y_pred)       # to'g'ri ulush
precision_score(y_true, y_pred)      # bashorat to'g'ri?
recall_score(y_true, y_pred)         # hammasini topdi?
f1_score(y_true, y_pred)             # muvozanat
confusion_matrix(y_true, y_pred)     # qayerda xato
classification_report(y_true, y_pred)  # hammasi

# regressiya:
mean_absolute_error(y_true, y_pred)  # o'rtacha xato
r2_score(y_true, y_pred)             # 0-1 (sifat)

# cross-validation (ishonchli):
cross_val_score(model, X, y, cv=5).mean()

Baholash xulosasi

Aniqlik yetmaydi (nomutanosib) · precision (bashorat to'g'ri) / recall (hammasi topdi)
F1 (muvozanat) · confusion (qayerda xato) · MAE/R2 (regressiya)
Cross-validation (ishonchli) · o'lchov maqsadga qarab

4. Batafsil misollar

Misollarda scikit-learn bilan sinaladi.

Misol 1 — Precision, recall, F1

python
"""precision (bashorat to'g'ri?), recall (hammasini topdi?), f1 (muvozanat)."""

import warnings
warnings.filterwarnings("ignore")

from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score
from sklearn.model_selection import train_test_split


def main() -> None:
    X, y = load_breast_cancer(return_X_y=True)
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
    model = LogisticRegression(max_iter=5000, random_state=42)
    model.fit(X_train, y_train)
    pred = model.predict(X_test)

    print("=== 1. Aniqlik (accuracy) ===")
    print(f"  {round(accuracy_score(y_test, pred), 3)}")

    print("\n=== 2. Precision (bashorat to'g'rimi?) ===")
    print(f"  {round(precision_score(y_test, pred), 3)}")

    print("\n=== 3. Recall (hammasini topdimmi?) ===")
    print(f"  {round(recall_score(y_test, pred), 3)}")

    print("\n=== 4. F1 (muvozanat) ===")
    print(f"  {round(f1_score(y_test, pred), 3)}")
    print("  ⭐ precision/recall/f1 — aniqlikdan boy o'lchovlar")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Aniqlik (accuracy) ===
  0.977

=== 2. Precision (bashorat to'g'rimi?) ===
  0.981

=== 3. Recall (hammasini topdimmi?) ===
  0.981

=== 4. F1 (muvozanat) ===
  0.981
  ⭐ precision/recall/f1 — aniqlikdan boy o'lchovlar

Nima ko'rsatdi: 2.2, 2.3-bo'limlar.

Misol 2 — Chalkashlik matritsasi

python
"""confusion_matrix: TP/TN/FP/FN — qayerda xato; classification_report."""

import warnings
warnings.filterwarnings("ignore")

from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix
from sklearn.model_selection import train_test_split


def main() -> None:
    X, y = load_breast_cancer(return_X_y=True)
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
    model = LogisticRegression(max_iter=5000, random_state=42)
    model.fit(X_train, y_train)
    pred = model.predict(X_test)

    print("=== 1. Chalkashlik matritsasi ===")
    cm = confusion_matrix(y_test, pred)
    print(f"  {cm.tolist()}")

    print("\n=== 2. Talqin ===")
    tn, fp, fn, tp = cm.ravel()
    print(f"  TN (to'g'ri manfiy): {tn}")
    print(f"  FP (yolg'on musbat, I tur): {fp}")
    print(f"  FN (yolg'on manfiy, II tur): {fn}")
    print(f"  TP (to'g'ri musbat): {tp}")

    print("\n=== 3. O'lchovlar matritsadan ===")
    print(f"  precision = TP/(TP+FP) = {round(tp / (tp + fp), 3)}")
    print(f"  recall = TP/(TP+FN) = {round(tp / (tp + fn), 3)}")

    print("\n=== 4. Qayerda xato ===")
    print(f"  {fp} oddiy → xato musbat, {fn} musbat → o'tkazib yuborildi")
    print("  ⭐ confusion matrix — model xatosini ko'rsatadi")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Chalkashlik matritsasi ===
  [[61, 2], [2, 106]]

=== 2. Talqin ===
  TN (to'g'ri manfiy): 61
  FP (yolg'on musbat, I tur): 2
  FN (yolg'on manfiy, II tur): 2
  TP (to'g'ri musbat): 106

=== 3. O'lchovlar matritsadan ===
  precision = TP/(TP+FP) = 0.981
  recall = TP/(TP+FN) = 0.981

=== 4. Qayerda xato ===
  2 oddiy → xato musbat, 2 musbat → o'tkazib yuborildi
  ⭐ confusion matrix — model xatosini ko'rsatadi

Nima ko'rsatdi: 2.4-bo'lim.

Misol 3 — Regressiya o'lchovlari va cross-validation

python
"""regressiya: MAE (o'rtacha xato), R2 (sifat); cross_val_score (ishonchli baho)."""

import warnings
warnings.filterwarnings("ignore")

import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LinearRegression, LogisticRegression
from sklearn.metrics import mean_absolute_error, r2_score
from sklearn.model_selection import cross_val_score


def main() -> None:
    print("=== 1. Regressiya o'lchovlari (MAE, R2) ===")
    X = np.array([[1], [2], [3], [4], [5]])
    y = np.array([2.1, 3.9, 6.1, 7.9, 10.1])
    lr = LinearRegression()
    lr.fit(X, y)
    pred = lr.predict(X)
    print(f"  MAE (o'rtacha xato): {round(mean_absolute_error(y, pred), 3)}")
    print(f"  R2 (sifat 0-1): {round(r2_score(y, pred), 3)}")

    print("\n=== 2. Cross-validation (5 marta) ===")
    Xc, yc = load_breast_cancer(return_X_y=True)
    model = LogisticRegression(max_iter=5000, random_state=42)
    scores = cross_val_score(model, Xc, yc, cv=5)
    print(f"  5 ball: {np.round(scores, 3).tolist()}")

    print("\n=== 3. O'rtacha (ishonchli baho) ===")
    print(f"  o'rtacha: {round(scores.mean(), 3)}")
    print(f"  std (barqarorlik): {round(scores.std(), 3)}")

    print("\n=== 4. Nega cross-validation ===")
    print("  bir bo'lish — tasodif (baxtli/baxtsiz)")
    print("  5 marta — ishonchli, barqaror baho")
    print("  ⭐ MAE/R2 (regressiya), cross-validation (ishonchli)")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Regressiya o'lchovlari (MAE, R2) ===
  MAE (o'rtacha xato): 0.096
  R2 (sifat 0-1): 0.999

=== 2. Cross-validation (5 marta) ===
  5 ball: [0.939, 0.947, 0.982, 0.93, 0.956]

=== 3. O'rtacha (ishonchli baho) ===
  o'rtacha: 0.951
  std (barqarorlik): 0.018

=== 4. Nega cross-validation ===
  bir bo'lish — tasodif (baxtli/baxtsiz)
  5 marta — ishonchli, barqaror baho
  ⭐ MAE/R2 (regressiya), cross-validation (ishonchli)

Nima ko'rsatdi: 2.5, 2.6-bo'limlar.

Misol 4 — Amaliy: to'liq model baholash

Real ML: modelni ko'p o'lchov bilan baholash — aniqlik, precision, recall, F1, cross-validation. Bu — to'g'ri model baholashning namunasi.

python
"""to'liq baholash: accuracy/precision/recall/f1, confusion, cross-validation — bir model."""

import warnings
warnings.filterwarnings("ignore")

import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score
from sklearn.model_selection import cross_val_score, train_test_split


def main() -> None:
    X, y = load_breast_cancer(return_X_y=True)
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
    model = RandomForestClassifier(random_state=42)
    model.fit(X_train, y_train)
    pred = model.predict(X_test)

    print("=== 1. Barcha klassifikatsiya o'lchovlari ===")
    print(f"  aniqlik: {round(accuracy_score(y_test, pred), 3)}")
    print(f"  precision: {round(precision_score(y_test, pred), 3)}")
    print(f"  recall: {round(recall_score(y_test, pred), 3)}")
    print(f"  F1: {round(f1_score(y_test, pred), 3)}")

    print("\n=== 2. Cross-validation (ishonchli) ===")
    cv = cross_val_score(RandomForestClassifier(random_state=42), X, y, cv=5)
    print(f"  o'rtacha: {round(cv.mean(), 3)} (± {round(cv.std(), 3)})")

    print("\n=== 3. O'lchov tanlovi ===")
    print("  kasallik aniqlash → recall muhim (kasalni o'tkazmaslik)")
    print(f"  recall: {round(recall_score(y_test, pred), 3)}")

    print("\n=== 4. Xulosa ===")
    print(f"  model ishonchli (cross-val {round(cv.mean(), 3)})")
    print(f"  recall yuqori (kasallik aniqlash uchun mos)")
    print("  ⭐ to'liq baholash — ko'p o'lchov, maqsadga qarab")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Barcha klassifikatsiya o'lchovlari ===
  aniqlik: 0.971
  precision: 0.964
  recall: 0.991
  F1: 0.977

=== 2. Cross-validation (ishonchli) ===
  o'rtacha: 0.956 (± 0.023)

=== 3. O'lchov tanlovi ===
  kasallik aniqlash → recall muhim (kasalni o'tkazmaslik)
  recall: 0.991

=== 4. Xulosa ===
  model ishonchli (cross-val 0.956)
  recall yuqori (kasallik aniqlash uchun mos)
  ⭐ to'liq baholash — ko'p o'lchov, maqsadga qarab

Nima ko'rsatdi: 2.1–2.8-bo'limlar.

5. To'g'ri va noto'g'ri tushunishlar

Noto'g'ri fikr To'g'risi
"Aniqlik yetarli" Nomutanosibda chalg'itadi
"Yuqori aniqlik — yaxshi model" Precision/recall ham ko'r
"Precision = recall" Har xil (ayblash vs topish)
"Bir bo'lish yetarli" Cross-validation (ishonchli)
"Har muammoga bir o'lchov" Maqsadga qarab
"F1 — o'rtacha" Garmonik (ikkalasi yuqori)
"R2 — klassifikatsiya" Regressiya (accuracy — klass)
"Bir raqam — to'liq baho" Kontekst kerak

6. Keng tarqalgan xatolar va yechimlari

1. Faqat aniqlik (nomutanosibda)

python
accuracy_score(...)   # 99% (kam spam)          # ⚠️ chalg'itadi
precision, recall, f1   # to'liq rasm            # ✅

2. Bir bo'lish (tasodif)

python
train_test_split; score   # bir marta            # ⚠️
cross_val_score(cv=5).mean()   # ishonchli        # ✅

3. Noto'g'ri o'lchov (maqsadga)

python
# kasallikda precision (recall kerak)             # ⚠️
recall_score(...)   # kasalni o'tkazmaslik         # ✅

4. Regressiyada accuracy

python
accuracy_score(narx_true, narx_pred)   # xato     # ⚠️
r2_score(...), mean_absolute_error(...)            # ✅

5. Train'da baholash

python
model.score(X_train, y_train)   # aldanish         # ⚠️
model.score(X_test, y_test)                        # ✅

6. F1'ni tushunmaslik

python
# precision 99%, recall 10% — "yaxshi"?           # ⚠️
# F1 past (muvozanat yo'q)                          # ✅

7. Kontekstsiz raqam

python
# "aniqlik 95%" (yolg'iz)                          # ⚠️
# nomutanosibmi? qaysi xato yomon?                  # ✅

7. Integratsiya — bu bilim qayerda kerak bo'ladi

  • 25.2-dars (o'tilgan): Model — baholash manbai
  • 25.3-dars (o'tilgan): Tayyorlash — cross-val Pipeline bilan
  • 25.6-dars: Ansambl — baholash
  • 24.17-dars (o'tilgan): Statistika — I/II tur xato
  • 25.12-dars: Deploy — model monitoring

8. Eng yaxshi amaliyotlar

  1. Aniqlikdan boy o'lchov (precision/recall/F1).

  2. Nomutanosibda — precision/recall (aniqlik emas).

  3. Cross-validation (bir bo'lish emas).

  4. O'lchov maqsadga qarab (xato narxi).

  5. Confusion matrix (qayerda xato).

  6. Regressiya — MAE/R2 (accuracy emas).

  7. Test'da bahola (train emas).

  8. Kontekst bilan (yolg'iz raqam emas).

9. Amaliy topshiriq

Vazifa 1: Bashorat qiling

python
1.  # nega aniqlik yetmaydi?
2.  # precision nima?
3.  # recall nima?
4.  # F1 nima?
5.  # confusion matrix nima?
6.  # FP nima?
7.  # FN nima?
8.  # MAE nima?
9.  # R2 nima?
10. # cross-validation nima?
11. # o'lchov qanday tanlanadi?
12. # spam uchun qaysi o'lchov?
Javoblar
  1. Nomutanosibda chalg'itadi (kam sinf)
  2. Bashorat qilgan musbatlar to'g'riligi
  3. Haqiqiy musbatlarning topilgan ulushi
  4. Precision va recall muvozanati
  5. Bashoratlar taqsimoti (TP/TN/FP/FN)
  6. Yolg'on musbat (I tur)
  7. Yolg'on manfiy (II tur)
  8. O'rtacha mutlaq xato (regressiya)
  9. Tushuntirilgan o'zgaruvchanlik (0-1)
  10. Ko'p marta bo'lib baholash
  11. Maqsadga qarab (xato narxi)
  12. Precision (noto'g'ri ayblamaslik)

Vazifa 2: Xatolarni tuzating

python
1.  accuracy_score(...)   # kam spam        # precision/recall

2.  train_test_split; score   # bir marta   # cross_val

3.  # kasallikda precision                   # recall

4.  accuracy_score(narx...)                  # r2_score

5.  model.score(X_train, y_train)            # test
Javoblar
python
1.  precision_score(...), recall_score(...), f1_score(...)

2.  cross_val_score(model, X, y, cv=5).mean()

3.  recall_score(...)

4.  r2_score(...), mean_absolute_error(...)

5.  model.score(X_test, y_test)

Vazifa 3: Precision/recall

O'lchov:

  1. precision_score
  2. recall_score
  3. f1_score
  4. Talqin

Vazifa 4: Confusion

Matritsa:

  1. confusion_matrix
  2. TP/TN/FP/FN
  3. O'lchovlar
  4. Qayerda xato

Vazifa 5: Regressiya/cross-val

Baholash:

  1. MAE, R2
  2. cross_val_score
  3. O'rtacha
  4. Barqarorlik

Vazifa 6: To'liq

Baholash:

  1. Barcha o'lchov
  2. Cross-validation
  3. O'lchov tanlovi
  4. Xulosa

Vazifa 7: O'ylash

Aniqlik (accuracy) — oddiy, lekin nomutanosib ma'lumotda (kam spam, kam kasal) chalg'itadi: "hammasi oddiy" model 99% aniq, lekin foydasiz. To'g'ri o'lchov (precision, recall, F1) maqsadga bog'liq. Nima uchun "bir raqam modelni to'liq tavsiflamaydi", va nega o'lchov tanlovi muammo kontekstiga (spam vs kasallik — har xil xato yomon) bog'liq — "to'g'ri narsani o'lchash" nima uchun muhim?

Javob

Qisqa javob: Bir raqam (accuracy) modelni to'liq tavsiflamaydi, chunki: nomutanosib ma'lumotda (kam musbat — spam, kasal) aniqlik chalg'itadi ("hammasi oddiy" model 99% aniq, lekin bironta spamni topmaydi — foydasiz). To'g'ri o'lchov maqsadga bog'liq: spam filtri — precision muhim (oddiy xatni spamga yubormaslik — yolg'on musbat yomon); kasallik — recall muhim (kasalni o'tkazib yubormaslik — yolg'on manfiy yomon). "Har xil xato — har xil narx": spamda noto'g'ri ayblash yomon, kasallikda o'tkazib yuborish yomon. "To'g'ri narsani o'lchash" muhim, chunki noto'g'ri o'lchov (kasallikda accuracy) — noto'g'ri model (kasalni o'tkazib yuboradigan). O'lchov modelning maqsadini aks ettirishi kerak. "Nimani o'lchasang, shuni olasan" — noto'g'ri o'lchov noto'g'ri modelga olib keladi.

1. Aniqlik chalg'itadi

Nomutanosib (990 oddiy, 10 spam): "hammasi oddiy" 99% aniq, lekin foydasiz. Aniqlik kam sinfni yashiradi.

2. O'lchov maqsadga bog'liq

Muammo Muhim o'lchov Nega
Spam Precision Oddiyni spam yomon
Kasallik Recall Kasalni o'tkazish yomon
Muvozanat F1 Ikkalasi

3. Xato narxi

Har xil xato — har xil narx: spamda FP (noto'g'ri ayblash — muhim xat yo'qoldi), kasallikda FN (o'tkazib yuborish — kasal davolamadi). Muammo xato narxini belgilaydi.

4. "To'g'ri narsani o'lchash"

Noto'g'ri o'lchov (kasallikda accuracy) → noto'g'ri model (kasalni o'tkazib yuboradi, lekin "aniq"). O'lchov maqsadni aks ettirishi kerak — "nimani o'lchasang, shuni optimallashtirasan".

5. Muhandislik saboqlari

  1. Bir raqam yetmaydi (kontekst)
  2. Aniqlik nomutanosibda chalg'itadi
  3. O'lchov maqsadga qarab (xato narxi)
  4. To'g'ri narsani o'lcha

6. Xulosa

  1. Aniqlik chalg'itishi mumkin
  2. O'lchov muammoga qarab (precision/recall)
  3. Xato narxi — o'lchovni belgilaydi
  4. To'g'ri o'lchov — to'g'ri model

Nimani mustahkamlaydi: 2.1–2.8-bo'limlar.

Xulosa

Bu darsda modelni to'g'ri baholashni o'rgandik.

Eng muhim uch fikr:

  1. Aniqlik yetmaydi — precision, recall, F1. Aniqlik (accuracy — to'g'ri bashorat ulushi) nomutanosib ma'lumotda (kam musbat) chalg'itadi: "hammasi oddiy" model 99% aniq, lekin foydasiz. Precision — bashorat qilgan musbatlardan qanchasi to'g'ri (spam dedik — necha % haqiqatan spam; noto'g'ri ayblamaslik). Recall — haqiqiy musbatlardan qanchasini topding (haqiqiy spamning necha % topding; o'tkazib yubormaslik). F1 — precision va recall garmonik o'rtachasi (muvozanat — ikkalasi yuqori bo'lganda yuqori). Farq: precision — "ayblaganim to'g'rimi", recall — "hammasini topdimmi".

  2. Confusion matrix, regressiya, cross-validation. Chalkashlik matritsasi (confusion_matrix): TP (to'g'ri musbat), TN (to'g'ri manfiy), FP (yolg'on musbat, I tur), FN (yolg'on manfiy, II tur — 24.17) — qayerda xato. Regressiya o'lchovlari: MAE (o'rtacha mutlaq xato — real birlik), R2 (0–1 — model o'zgaruvchanlikning qanchasini tushuntiradi). Cross-validation — ma'lumotni cv=5 qismga bo'lib 5 marta baholash (.mean()) — bir train_test_split tasodifga bog'liq, cross-validation ishonchli (barqaror baho).

  3. O'lchov maqsadga qarab — halol baholash. O'lchov tanlovi maqsadga bog'liq: precision (yolg'on musbat yomon — spam), recall (yolg'on manfiy yomon — kasallik), F1 (muvozanat), accuracy (mutanosib ma'lumot). "Har xil xato — har xil narx" (spamda noto'g'ri ayblash, kasallikda o'tkazib yuborish). Bir raqam modelni to'liq tavsiflamaydi — kontekst kerak (nomutanosibmi? qaysi xato yomon?). Halol baholash: test'da (train emas), to'g'ri o'lchov, cross-validation, maqsadga mos. "Nimani o'lchasang, shuni optimallashtirasan" — to'g'ri o'lchov to'g'ri modelga olib keladi (24.16 chalg'itmaslik kabi).

Keyingi darsda feature engineering ni o'rganamiz: mavjud ma'lumotdan yangi, foydali xususiyatlar yaratish — ko'pincha model tanlovidan ko'proq ta'sir qiladigan mahorat.

Ulashish:Telegram'da

Izohlar (0)

Izoh yozish uchun kiring.

  • Hozircha izoh yo'q. Birinchi bo'ling!