Mundarija (22)
- 1. Kirish va motivatsiya
- 2. Nazariya — chuqur tushuntirish
- 2.1. Xato xabarlarini o'qish
- 2.2. Nosozlikni lokalizatsiya qilish
- 2.3. set_config
- 2.4. cross_validate bilan diagnostika
- 2.5. error_score
- 2.6. Ogohlantirishlarni boshqarish
- 2.7. Tuzoqlar
- 2.8. To'rt qadam
- 3. Tez ma'lumotnoma
- 4. Batafsil misollar
- Misol 1 — Xato xabarlarini o'qish
- Misol 2 — Nosozlikni lokalizatsiya qilish
- Misol 3 — error_score va cross_validate
- Misol 4 — set_config va ogohlantirishlar
- 5. To'g'ri va noto'g'ri tushunishlar
- 6. Keng tarqalgan xatolar va yechimlari
- 7. Integratsiya — bu bilim qayerda kerak bo'ladi
- 8. Eng yaxshi amaliyotlar
- 9. Amaliy topshiriq
- Xulosa
19.8-dars: Diagnostika va tekshirish
19-QISM — SCIKIT-LEARN TO'LIQ · 8-dars
1. Kirish va motivatsiya
Quvur uzun bo'lgani sari xato qayerda ekanini topish qiyinlashadi. ColumnTransformer ichidagi Pipeline ichidagi transformer xato bersa, stek izi o'nlab qatorga cho'ziladi va aslida muhim bo'lgan bir qator o'rtada yashirinib qoladi.
Yaxshi xabar shuki, sklearn diagnostika uchun juda boy: obyektlarni ko'rish (set_config(display=...)), xato xabarlaridagi aniq ko'rsatkichlar, check_estimator, check_is_fitted, NotFittedError, InconsistentVersionWarning va cross_validate ning batafsil chiqishi.
Bu darsda: xato xabarlarini o'qish, uzun quvurda nosozlikni lokalizatsiya qilish, set_config imkoniyatlari, cross_validate bilan diagnostika, error_score va ogohlantirishlarni boshqarish.
Real vaziyat. GridSearchCV "All the 120 fits failed" deb xato berdi. Jamoa ikki soat kodni qaraydi. Aslida xabarning oxirida sabab yozilgan edi: ValueError: Input contains NaN. error_score="raise" qo'yilganda xato birinchi foldda darhol va to'liq stek izi bilan chiqdi — muammo besh daqiqada topildi.
Bu darsda diagnostikani o'rganamiz.
Bu darsda:
- Xato xabarlarini o'qish
- Nosozlikni lokalizatsiya qilish
- set_config
- cross_validate bilan diagnostika
- error_score
- Ogohlantirishlarni boshqarish
- Tuzoqlar
- Amaliy: nosozlikni topish
ℹ Misollar real numpy/pandas/sklearn bilan (Python 3.14, sklearn 1.9).
2. Nazariya — chuqur tushuntirish
2.1. Xato xabarlarini o'qish
NotFittedError
"This X instance is not fitted yet"
-> fit chaqirilmagan yoki clone dan keyin fit unutilgan
ValueError: Input contains NaN
-> imputatsiya yo'q yoki NaN qabul qilmaydigan model
ValueError: X has N features, but M is expecting K features
-> transform da ustunlar soni mos emas (reset=False tekshiruvi)
ValueError: The feature names should match those that were passed
-> ustun NOMLARI yoki TARTIBI o'zgargan
ValueError: could not convert string to float
-> kategoriyali ustun kodlanmagan
ValueError: Found unknown categories
-> OneHotEncoder(handle_unknown="ignore") qo'ying
ValueError: Found input variables with inconsistent numbers of samples
-> X va y uzunligi farq qiladiXabarning oxirgi qatorini o'qing — sklearn xatolarida eng aniq ma'lumot odatda oxirida bo'ladi.
2.2. Nosozlikni lokalizatsiya qilish
# 1. QADAMLARNI BIRIN-KETIN bajaring
for i in range(len(quvur)):
try:
chiqish = quvur[:i + 1].fit_transform(X, y)
print(i, quvur.steps[i][0], "OK", chiqish.shape)
except Exception as xato:
print(i, quvur.steps[i][0], "XATO", type(xato).__name__)
break
# 2. KIRISHNI tekshiring
print(X.isna().sum()) # NaN
print(X.dtypes) # turlar
print(np.isinf(X.select_dtypes(np.number)).sum()) # cheksizlik
# 3. ORALIQ natijani ko'ring
print(quvur[:-1].transform(X)[:3])
# 4. FOLD darajasida
for tr, te in cv.split(X, y):
... # qaysi foldda buzilishini topingQadamlarni birin-ketin bajarish — uzun quvurda nosozlikni topishning eng tez usuli.
2.3. set_config
from sklearn import set_config, get_config, config_context
set_config(display="diagram") # Jupyter da HTML sxema
set_config(display="text") # matnli repr
set_config(transform_output="pandas") # GLOBAL set_output
set_config(print_changed_only=False) # HAMMA parametrni ko'rsatish
set_config(assume_finite=True) # NaN/inf tekshiruvini O'CHIRISH (tez)
set_config(enable_metadata_routing=True)
VAQTINCHA (bitta blok uchun):
with config_context(assume_finite=True):
model.fit(X, y) # faqat shu blokda
DIQQAT: assume_finite=True tezlashtiradi, lekin NaN ni
JIM o'tkazib yuboradi - xato keyinroq, chuqurroq va
tushunarsizroq joyda chiqadi (masalan LinAlgError) print_changed_only=False sukut qiymatlarni ham ko'rsatadi — modelning haqiqiy konfiguratsiyasini tekshirishda foydali.
2.4. cross_validate bilan diagnostika
from sklearn.model_selection import cross_validate
natija = cross_validate(
quvur, X, y, cv=cv,
scoring={"auc": "roc_auc", "ap": "average_precision"},
return_train_score=True, # overfitting
return_estimator=True, # har foldning modeli
return_indices=True, # fold indekslari
error_score="raise", # xatoni YASHIRMA
)
natija kalitlari:
fit_time, score_time
test_auc, train_auc, test_ap, train_ap
estimator (har foldning fit qilingan nusxasi)
indices (train/test indekslari)
FOYDASI:
qaysi fold yomon ishladi
har foldda qanday parametr/koeffitsiyent chiqdi
o'quv va test farqi (overfitting) return_estimator=True har foldning modelini beradi — foldlar orasidagi beqarorlikni ko'rishning eng to'g'ridan-to'g'ri yo'li.
2.5. error_score
error_score=np.nan (SUKUT)
fit xato bersa - NaN yoziladi va ish DAVOM ETADI
+ bitta yomon nomzod butun qidiruvni to'xtatmaydi
- xato YASHIRINADI, "All fits failed" deb yig'ma xabar chiqadi
error_score="raise"
birinchi xatoda TO'XTAYDI va to'liq stek izini beradi
-> ISHLAB CHIQISH paytida SHUNI ISHLATING
error_score=0
xato bo'lgan nomzodga eng yomon ball beriladi
TAVSIYA:
ishlab chiqishda "raise", uzoq qidiruvda np.nan Ishlab chiqish paytida error_score="raise" — aks holda xatoning haqiqiy sababi yig'ma xabarda yo'qoladi.
2.6. Ogohlantirishlarni boshqarish
import warnings
from sklearn.exceptions import ConvergenceWarning
# 1. OGOHLANTIRISHNI KO'RISH (tavsiya)
warnings.simplefilter("always")
# 2. XATOGA AYLANTIRISH (testlarda)
warnings.simplefilter("error", ConvergenceWarning)
# 3. USHLAB OLISH
with warnings.catch_warnings(record=True) as ogohlar:
warnings.simplefilter("always")
model.fit(X, y)
for o in ogohlar:
print(o.category.__name__, o.message)
MUHIM OGOHLANTIRISHLAR:
ConvergenceWarning max_iter yetarli emas
InconsistentVersionWarning boshqa sklearn versiyasida saqlangan
UndefinedMetricWarning metrika aniqlanmagan - nol maxraj
DataConversionWarning y shakli noto'g'ri (ravel kerak) Ogohlantirishni o'chirmang, sababini tuzating — ConvergenceWarning model yaqinlashmaganini bildiradi va natija ishonchsiz bo'ladi.
2.7. Tuzoqlar
Asosiy tuzoqlar: xato xabarini oxirigacha o'qimaslik; error_score ni sukutda qoldirib "All fits failed" bilan qolish; ogohlantirishlarni filterwarnings("ignore") bilan o'chirish; assume_finite=True ni doimiy yoqish; check_estimator ni ishlatmaslik; return_train_score ni unutish; quvurni butunlay buzib qo'lda tekshirish.
2.8. To'rt qadam
Nosozlikni topishning to'rt qadami: xabarning oxirgi qatorini o'qing, kirishni tekshiring (NaN, dtype, inf), qadamlarni birin-ketin bajaring va error_score="raise" bilan haqiqiy stek izini oling. Bu to'rttasi sklearn xatolarining katta qismini bir necha daqiqada ochadi.
3. Tez ma'lumotnoma
import warnings
from sklearn import config_context, get_config, set_config
from sklearn.model_selection import cross_validate
from sklearn.utils.estimator_checks import check_estimator
set_config(display="text", print_changed_only=False)
# nosozlikni lokalizatsiya qilish
for i in range(len(quvur)):
try:
print(i, quvur.steps[i][0], quvur[:i + 1].fit_transform(X, y).shape)
except Exception as xato:
print(i, quvur.steps[i][0], type(xato).__name__, xato); break
natija = cross_validate(quvur, X, y, cv=cv, scoring={"auc": "roc_auc"},
return_train_score=True, return_estimator=True,
error_score="raise")
with warnings.catch_warnings(record=True) as ogohlar:
warnings.simplefilter("always"); model.fit(X, y)
QOIDA: oxirgi qatorni o'qi · kirishni tekshir · qadamma-qadam ·
error_score="raise"Diagnostika xulosasi
Xato xabari: oxirgi qator eng aniq
Lokalizatsiya: quvur[:i+1] bilan qadamma-qadam
set_config: display, transform_output, print_changed_only
cross_validate: return_train_score / estimator / indices
error_score="raise": ishlab chiqishda
Ogohlantirish: o'chirmang, tuzating4. Batafsil misollar
Misollar real numpy/pandas/sklearn bilan (Python 3.14).
Misol 1 — Xato xabarlarini o'qish
"""Eng ko'p uchraydigan xatolar va ularning ma'nosi (real pandas/sklearn)."""
import numpy as np
import pandas as pd
from sklearn.exceptions import NotFittedError
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import OneHotEncoder, StandardScaler
def sinab_kor(nom: str, amal) -> None:
try:
amal()
print(f" {nom:<34} xato chiqmadi")
except Exception as xato:
birinchi = str(xato).splitlines()[0]
print(f" {nom:<34} {type(xato).__name__}")
print(f" {'':<34} {birinchi[:62]}")
def main() -> None:
rng = np.random.default_rng(0)
df = pd.DataFrame({
"a": rng.normal(0, 1, 100),
"b": rng.normal(0, 1, 100),
"k": rng.choice(["x", "y"], 100),
})
y = rng.integers(0, 2, 100)
sonli = df[["a", "b"]]
print("=== 1. Etti xil xato ===")
sc = StandardScaler().fit(sonli)
sinab_kor("fit qilinmagan", lambda: StandardScaler().transform(sonli))
nan_bilan = sonli.copy()
nan_bilan.iloc[0, 0] = np.nan
sinab_kor("NaN bilan", lambda: LogisticRegression().fit(nan_bilan, y))
sinab_kor("ustunlar soni mos emas",
lambda: sc.transform(df[["a"]]))
boshqa_nom = sonli.rename(columns={"a": "A"})
sinab_kor("ustun nomi boshqa", lambda: sc.transform(boshqa_nom))
sinab_kor("kodlanmagan matn",
lambda: LogisticRegression().fit(df, y))
oh = OneHotEncoder(handle_unknown="error").fit(df[["k"]])
yangi = pd.DataFrame({"k": ["z"]})
sinab_kor("noma'lum kategoriya", lambda: oh.transform(yangi))
sinab_kor("X va y uzunligi farqli",
lambda: LogisticRegression().fit(sonli, y[:50]))
print("\n=== 2. Xabarning qaysi qismi muhim ===")
try:
sc.transform(df[["a"]])
except ValueError as xato:
qatorlar = str(xato).splitlines()
print(f" xabar {len(qatorlar)} qatordan iborat")
print(f" BIRINCHI qator: {qatorlar[0][:62]}")
print(f" OXIRGI qator: {qatorlar[-1][:62]}")
print("\n=== 3. Xato turi bo'yicha tashxis ===")
tashxis = {
"NotFittedError": "fit chaqirilmagan yoki clone dan keyin unutilgan",
"ValueError (NaN)": "imputatsiya yo'q",
"ValueError (features)": "ustunlar soni/nomi mos emas",
"ValueError (string)": "kategoriyali ustun kodlanmagan",
"ValueError (unknown)": "handle_unknown='ignore' qo'ying",
"ValueError (samples)": "X va y uzunligi farqli",
}
print(f" {'xato':<24} {'tashxis'}")
for xato_turi, sabab in tashxis.items():
print(f" {xato_turi:<24} {sabab}")
print("\n=== 4. NotFittedError qanday hosil bo'ladi ===")
from sklearn.base import clone
yangi_sc = clone(sc)
print(f" clone qilingan obyektda mean_ bormi: "
f"{hasattr(yangi_sc, 'mean_')}")
try:
yangi_sc.transform(sonli)
except NotFittedError:
print(" clone dan keyin fit chaqirish SHART")
print(" ⭐ Xabarning oxirgi qatorida eng aniq ma'lumot bo'ladi")
if __name__ == "__main__":
main()Natijaning muhim qismi:
=== 1. Etti xil xato ===
fit qilinmagan NotFittedError
This StandardScaler instance is not fitted yet. Call 'fit' wit
NaN bilan ValueError
Input X contains NaN.
ustunlar soni mos emas ValueError
The feature names should match those that were passed during f
ustun nomi boshqa ValueError
The feature names should match those that were passed during f
kodlanmagan matn ValueError
could not convert string to float: 'y'
noma'lum kategoriya ValueError
Found unknown categories ['z'] in column 0 during transform
X va y uzunligi farqli ValueError
Found input variables with inconsistent numbers of samples: [1
=== 2. Xabarning qaysi qismi muhim ===
xabar 3 qatordan iborat
BIRINCHI qator: The feature names should match those that were passed during f
OXIRGI qator: - b
=== 3. Xato turi bo'yicha tashxis ===
xato tashxis
NotFittedError fit chaqirilmagan yoki clone dan keyin unutilgan
ValueError (NaN) imputatsiya yo'q
ValueError (features) ustunlar soni/nomi mos emas
ValueError (string) kategoriyali ustun kodlanmagan
ValueError (unknown) handle_unknown='ignore' qo'ying
ValueError (samples) X va y uzunligi farqli
=== 4. NotFittedError qanday hosil bo'ladi ===
clone qilingan obyektda mean_ bormi: False
clone dan keyin fit chaqirish SHART
⭐ Xabarning oxirgi qatorida eng aniq ma'lumot bo'ladiNima ko'rsatdi: 2.1-bo'lim.
Misol 2 — Nosozlikni lokalizatsiya qilish
"""Uzun quvurda xato qayerda (real numpy/pandas/sklearn)."""
import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.decomposition import PCA
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
def yarat(n: int = 400, seed: int = 0, nan_bilan: bool = True):
rng = np.random.default_rng(seed)
df = pd.DataFrame({
"yosh": rng.integers(18, 70, n).astype(float),
"daromad": rng.lognormal(10, 0.5, n),
"ball": rng.normal(600, 80, n),
"hudud": rng.choice(["shimol", "janub"], n),
})
if nan_bilan:
df.loc[rng.choice(n, 30, replace=False), "ball"] = np.nan
y = (df["yosh"] + df["ball"].fillna(600) / 100 > 50).astype(int).to_numpy()
return df, y
def qadamma_qadam(quvur, X, y) -> None:
"""Har qadamni alohida bajarib, birinchi xatoni topadi."""
for i in range(len(quvur)):
nomi = quvur.steps[i][0]
try:
chiqish = quvur[:i + 1].fit_transform(X, y)
shakl = getattr(chiqish, "shape", "?")
print(f" {i}: {nomi:<12} OK {shakl}")
except Exception as xato:
print(f" {i}: {nomi:<12} XATO {type(xato).__name__}")
print(f" {str(xato).splitlines()[-1][:64]}")
return
print(" barcha qadamlar o'tdi")
def main() -> None:
df, y = yarat()
print("=== 1. Buzuq quvur: imputatsiya yo'q ===")
buzuq = Pipeline([
("t", ColumnTransformer([
("son", StandardScaler(), ["yosh", "daromad", "ball"]),
("kat", OneHotEncoder(sparse_output=False), ["hudud"])])),
("p", PCA(n_components=3, random_state=0)),
("m", LogisticRegression(max_iter=2000)),
])
qadamma_qadam(buzuq, df, y)
print("\n=== 2. Kirishni tekshirish ===")
print(f" {'ustun':<10} {'dtype':<10} {'NaN':>6} {'inf':>6}")
for ustun in df.columns:
seriya = df[ustun]
cheksiz = (int(np.isinf(seriya).sum())
if seriya.dtype.kind == "f" else 0)
print(f" {ustun:<10} {str(seriya.dtype):<10} "
f"{int(seriya.isna().sum()):>6} {cheksiz:>6}")
print("\n=== 3. Tuzatilgan quvur ===")
son_quvur = Pipeline([("imp", SimpleImputer(strategy="median")),
("sc", StandardScaler())])
toGri = Pipeline([
("t", ColumnTransformer([
("son", son_quvur, ["yosh", "daromad", "ball"]),
("kat", OneHotEncoder(sparse_output=False), ["hudud"])])),
("p", PCA(n_components=3, random_state=0)),
("m", LogisticRegression(max_iter=2000)),
])
qadamma_qadam(toGri, df, y)
print("\n=== 4. Ichki qadamlarni tekshirish ===")
toGri.fit(df, y)
ct = toGri["t"]
print(f" ColumnTransformer chiqishi: {ct.transform(df).shape}")
print(f" belgilar: {ct.get_feature_names_out().tolist()}")
imp = ct.named_transformers_["son"].named_steps["imp"]
print(f" imputatsiya qiymatlari: "
f"{np.round(imp.statistics_, 2).tolist()}")
print(f" PCA dan keyin: {toGri[:-1].transform(df).shape}")
print(f" model koeffitsiyentlari: "
f"{np.round(toGri['m'].coef_[0], 4).tolist()}")
print(" ⭐ quvur[:i+1] bilan qadamma-qadam - eng tez usul")
if __name__ == "__main__":
main()Natijaning muhim qismi:
=== 1. Buzuq quvur: imputatsiya yo'q ===
0: t OK (400, 5)
1: p XATO ValueError
PCA does not accept missing values encoded as NaN natively. For
=== 2. Kirishni tekshirish ===
ustun dtype NaN inf
yosh float64 0 0
daromad float64 0 0
ball float64 30 0
hudud str 0 0
=== 3. Tuzatilgan quvur ===
0: t OK (400, 5)
1: p OK (400, 3)
2: m XATO AttributeError
This 'Pipeline' has no attribute 'fit_transform'
=== 4. Ichki qadamlarni tekshirish ===
ColumnTransformer chiqishi: (400, 5)
belgilar: ['son__yosh', 'son__daromad', 'son__ball', 'kat__hudud_janub', 'kat__hudud_shimol']
imputatsiya qiymatlari: [46.0, 21025.19, 594.01]
PCA dan keyin: (400, 3)
model koeffitsiyentlari: [-1.9326, 4.336, 3.253]
⭐ quvur[:i+1] bilan qadamma-qadam - eng tez usulNima ko'rsatdi: 2.2-bo'lim.
Misol 3 — error_score va cross_validate
"""Yashirin xatoni ochish (real numpy/sklearn)."""
import warnings
import numpy as np
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import (GridSearchCV, StratifiedKFold,
cross_validate)
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
def main() -> None:
X, y = make_classification(n_samples=600, n_features=10,
n_informative=5, flip_y=0.2, random_state=0)
X_nan = X.copy()
X_nan[5, 2] = np.nan # bitta NaN
cv = StratifiedKFold(4, shuffle=True, random_state=0)
quvur = Pipeline([("sc", StandardScaler()),
("m", LogisticRegression(max_iter=2000))])
setka = {"m__C": [0.1, 1.0, 10.0]}
print("=== 1. error_score=np.nan (sukut) ===")
print(" HAMMA fit yiqilsa - yig'ma ValueError:")
try:
with warnings.catch_warnings():
# yo'l-yo'lakay chiqadigan ogohlantirishlarni jim qilamiz -
# bizni ValueError ning O'ZI qiziqtiradi
warnings.simplefilter("ignore")
GridSearchCV(quvur, setka, cv=cv, scoring="roc_auc", n_jobs=1,
error_score=np.nan).fit(X_nan, y)
print(" xato chiqmadi (kutilmagan)")
except ValueError as xato:
qatorlar = str(xato).splitlines()
print(f" birinchi qator: {qatorlar[0][:62]}")
print(f" xabar uzunligi: {len(qatorlar)} qator")
print(" haqiqiy sabab shu uzun matn ICHIDA yashirinadi")
print("\n=== 1b. BA'ZI fit yiqilsa - faqat ogohlantirish ===")
# ikkinchi nomzod xato beradi (manfiy C), qolganlari ishlaydi
yomon_setka = {"m__C": [1.0, -1.0]}
with warnings.catch_warnings(record=True) as ogohlar:
warnings.simplefilter("always")
g = GridSearchCV(quvur, yomon_setka, cv=cv, scoring="roc_auc",
n_jobs=1, error_score=np.nan).fit(X, y)
ballar = [None if np.isnan(b) else round(float(b), 4)
for b in g.cv_results_["mean_test_score"]]
print(f" ballar: {ballar}")
print(f" ogohlantirishlar soni: {len(ogohlar)}")
turlari = sorted({o.category.__name__ for o in ogohlar})
print(f" turlari: {turlari}")
print(" ish DAVOM ETDI, lekin bitta nomzod jim yo'qoldi")
print("\n=== 2. error_score='raise' ===")
try:
with warnings.catch_warnings():
warnings.simplefilter("ignore")
GridSearchCV(quvur, setka, cv=cv, scoring="roc_auc", n_jobs=1,
error_score="raise").fit(X_nan, y)
print(" xato chiqmadi (kutilmagan)")
except ValueError as xato:
print(f" ValueError darhol chiqdi:")
print(f" {str(xato).splitlines()[0][:66]}")
print(" sabab aniq va stek izi to'liq")
print("\n=== 3. cross_validate batafsil chiqishi ===")
natija = cross_validate(
quvur, X, y, cv=cv,
scoring={"auc": "roc_auc", "ap": "average_precision"},
return_train_score=True, return_estimator=True,
return_indices=True, error_score="raise")
print(f" kalitlar: {sorted(natija)}")
print(f" {'fold':>5} {'test auc':>10} {'train auc':>11} "
f"{'test ap':>9} {'farq':>8}")
for i in range(len(natija["test_auc"])):
farq = natija["train_auc"][i] - natija["test_auc"][i]
print(f" {i + 1:>5} {natija['test_auc'][i]:>10.4f} "
f"{natija['train_auc'][i]:>11.4f} "
f"{natija['test_ap'][i]:>9.4f} {farq:>8.4f}")
print("\n=== 4. Foldlar orasidagi beqarorlik ===")
koeflar = np.array([est["m"].coef_[0] for est in natija["estimator"]])
print(f" koeffitsiyentlar shakli: {koeflar.shape}")
print(f" {'belgi':>7} {'o_rtacha':>10} {'std':>9} {'nisbat':>9}")
for i in range(5):
ortacha = koeflar[:, i].mean()
std = koeflar[:, i].std()
nisbat = std / max(abs(ortacha), 1e-9)
print(f" {i:>7} {ortacha:>+10.4f} {std:>9.4f} {nisbat:>9.2f}")
beqaror = int((koeflar.std(axis=0)
/ np.abs(koeflar.mean(axis=0)).clip(1e-9) > 0.5).sum())
print(f" std/|o'rtacha| > 0.5 bo'lgan belgilar: {beqaror}/10")
print("\n=== 5. Fold indekslari ===")
for i in range(2):
tr = natija["indices"]["train"][i]
te = natija["indices"]["test"][i]
print(f" fold {i + 1}: o'quv {len(tr)}, test {len(te)}, "
f"kesishish {len(np.intersect1d(tr, te))}")
print(" ⭐ error_score='raise' sababni darhol ko'rsatadi")
if __name__ == "__main__":
main()Natijaning muhim qismi:
=== 1. error_score=np.nan (sukut) ===
HAMMA fit yiqilsa - yig'ma ValueError:
birinchi qator: Input X contains NaN.
xabar uzunligi: 2 qator
haqiqiy sabab shu uzun matn ICHIDA yashirinadi
=== 1b. BA'ZI fit yiqilsa - faqat ogohlantirish ===
ballar: [0.8718, None]
ogohlantirishlar soni: 2
turlari: ['FitFailedWarning', 'UserWarning']
ish DAVOM ETDI, lekin bitta nomzod jim yo'qoldi
=== 2. error_score='raise' ===
ValueError darhol chiqdi:
Input X contains NaN.
sabab aniq va stek izi to'liq
=== 3. cross_validate batafsil chiqishi ===
kalitlar: ['estimator', 'fit_time', 'indices', 'score_time', 'test_ap', 'test_auc', 'train_ap', 'train_auc']
fold test auc train auc test ap farq
1 0.8830 0.8758 0.8820 -0.0072
2 0.8915 0.8723 0.8919 -0.0192
3 0.8630 0.8832 0.8748 0.0202
4 0.8498 0.8867 0.8318 0.0369
=== 4. Foldlar orasidagi beqarorlik ===
koeffitsiyentlar shakli: (4, 10)
belgi o_rtacha std nisbat
0 -0.5643 0.0604 0.11
1 +0.2755 0.0386 0.14
2 +0.0625 0.0551 0.88
3 +0.1677 0.1142 0.68
4 -0.5149 0.0251 0.05
std/|o'rtacha| > 0.5 bo'lgan belgilar: 2/10
=== 5. Fold indekslari ===
fold 1: o'quv 450, test 150, kesishish 0
fold 2: o'quv 450, test 150, kesishish 0
⭐ error_score='raise' sababni darhol ko'rsatadiNima ko'rsatdi: 2.4, 2.5-bo'limlar.
Misol 4 — set_config va ogohlantirishlar
"""Konfiguratsiya va ogohlantirishlarni boshqarish (real sklearn)."""
import warnings
import numpy as np
import pandas as pd
from sklearn import config_context, get_config, set_config
from sklearn.datasets import make_classification
from sklearn.decomposition import PCA
from sklearn.exceptions import ConvergenceWarning
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
def main() -> None:
X, y = make_classification(n_samples=500, n_features=8,
n_informative=5, flip_y=0.25,
class_sep=0.5, random_state=0)
df = pd.DataFrame(X, columns=[f"x{i}" for i in range(X.shape[1])])
print("=== 1. Joriy konfiguratsiya ===")
joriy = get_config()
muhimlar = ["display", "print_changed_only", "transform_output",
"assume_finite"]
for kalit in muhimlar:
print(f" {kalit:<20} {joriy.get(kalit)}")
print("\n=== 2. print_changed_only ===")
model = LogisticRegression(C=0.5, max_iter=3000)
set_config(print_changed_only=True)
print(f" True: {repr(model)}")
set_config(print_changed_only=False)
toliq = repr(model)
print(f" False: {toliq[:70]}...")
print(f" to'liq uzunligi: {len(toliq)} belgi")
set_config(print_changed_only=True)
print("\n=== 3. transform_output global ===")
quvur = Pipeline([("sc", StandardScaler())])
oddiy = quvur.fit_transform(df)
print(f" sukut: {type(oddiy).__name__}")
with config_context(transform_output="pandas"):
ramka = Pipeline([("sc", StandardScaler())]).fit_transform(df)
print(f" config_context ichida: {type(ramka).__name__}, "
f"ustunlar {list(ramka.columns)[:3]}")
qayta = Pipeline([("sc", StandardScaler())]).fit_transform(df)
print(f" blokdan keyin: {type(qayta).__name__}")
print("\n=== 4. ConvergenceWarning ni ushlash ===")
with warnings.catch_warnings(record=True) as ogohlar:
warnings.simplefilter("always")
LogisticRegression(max_iter=2, solver="saga",
random_state=0).fit(X, y)
print(f" ogohlantirishlar: {len(ogohlar)}")
for o in ogohlar[:2]:
print(f" {o.category.__name__}: "
f"{str(o.message).splitlines()[0][:56]}")
print("\n=== 5. Tuzatilgandan keyin ===")
with warnings.catch_warnings(record=True) as ogohlar:
warnings.simplefilter("always")
m = LogisticRegression(max_iter=5000, solver="saga",
random_state=0).fit(X, y)
print(f" max_iter=5000 bilan ogohlantirishlar: {len(ogohlar)}")
print(f" haqiqiy iteratsiyalar: {int(m.n_iter_[0])}")
print(" ogohlantirishni O'CHIRMASDAN sababini tuzatdik")
print("\n=== 6. assume_finite ===")
print(f" sukut: {get_config()['assume_finite']}")
X_nan = X.copy()
X_nan[0, 0] = np.nan
# StandardScaler NaN ni O'ZI qabul qiladi - PCA esa yo'q
try:
PCA(n_components=2).fit(X_nan)
print(" PCA NaN bilan fit o'tdi (kutilmagan)")
except ValueError as xato:
print(f" NaN tekshiruvi ishladi: "
f"{str(xato).splitlines()[0][:48]}")
with config_context(assume_finite=True):
try:
pca = PCA(n_components=2).fit(X_nan)
print(f" assume_finite=True bilan: fit o'tdi")
print(f" natija NaN mi: "
f"{bool(np.isnan(pca.components_).any())}")
except Exception as xato:
print(f" {type(xato).__name__}: tekshiruv o'tkazildi, "
f"lekin hisob yiqildi")
print(" ⭐ assume_finite tezlashtiradi, lekin xatoni KECHIKTIRADI")
if __name__ == "__main__":
main()Natijaning muhim qismi:
=== 1. Joriy konfiguratsiya ===
display diagram
print_changed_only True
transform_output default
assume_finite False
=== 2. print_changed_only ===
True: LogisticRegression(C=0.5, max_iter=3000)
False: LogisticRegression(C=0.5, class_weight=None, dual=False, fit_intercept...
to'liq uzunligi: 294 belgi
=== 3. transform_output global ===
sukut: ndarray
config_context ichida: DataFrame, ustunlar ['x0', 'x1', 'x2']
blokdan keyin: ndarray
=== 4. ConvergenceWarning ni ushlash ===
ogohlantirishlar: 1
ConvergenceWarning: The max_iter was reached which means the coef_ did not c
=== 5. Tuzatilgandan keyin ===
max_iter=5000 bilan ogohlantirishlar: 0
haqiqiy iteratsiyalar: 17
ogohlantirishni O'CHIRMASDAN sababini tuzatdik
=== 6. assume_finite ===
sukut: False
NaN tekshiruvi ishladi: Input X contains NaN.
LinAlgError: tekshiruv o'tkazildi, lekin hisob yiqildi
⭐ assume_finite tezlashtiradi, lekin xatoni KECHIKTIRADINima ko'rsatdi: 2.3, 2.6-bo'limlar.
5. To'g'ri va noto'g'ri tushunishlar
| Noto'g'ri fikr | To'g'risi |
|---|---|
| "Xato xabari tushunarsiz" | Oxirgi qatorda aniq sabab |
"All fits failed — noma'lum xato" |
error_score="raise" sababni ochadi |
| "Ogohlantirishni o'chirsa bo'ladi" | Sababini tuzating |
"assume_finite=True tezlashtiradi" |
NaN ni jim o'tkazadi |
"cross_val_score yetarli" |
cross_validate ancha ko'p beradi |
| "Quvurni buzib tekshirish kerak" | quvur[:i+1] bilan |
"ConvergenceWarning zararsiz" |
Model yaqinlashmagan |
"print_changed_only ahamiyatsiz" |
Haqiqiy konfiguratsiyani ko'rsatadi |
6. Keng tarqalgan xatolar va yechimlari
1. Xatoni yashirish
GridSearchCV(quvur, setka, cv=cv) # error_score=np.nan # ⚠️
GridSearchCV(quvur, setka, cv=cv, error_score="raise") # ✅2. Ogohlantirishlarni o'chirish
warnings.filterwarnings("ignore") # ⚠️
LogisticRegression(max_iter=5000) # sababini tuzating # ✅3. Quvurni buzib tekshirish
X2 = quvur["sc"].fit_transform(X) # leakage xavfi # ⚠️
quvur[:1].fit_transform(X, y) # ✅4. assume_finite ni doimiy yoqish
set_config(assume_finite=True) # global # ⚠️
with config_context(assume_finite=True): ... # faqat kerakli joy # ✅5. Faqat cross_val_score
b = cross_val_score(quvur, X, y, cv=cv) # ⚠️
n = cross_validate(quvur, X, y, cv=cv, return_train_score=True,
return_estimator=True) # ✅6. Kirishni tekshirmaslik
quvur.fit(df, y) # xato chiqqach o'ylash # ⚠️
print(df.isna().sum(), df.dtypes) # oldin # ✅7. check_estimator ni o'tkazib yuborish
# o'z transformeringiz - qo'lda sinaymiz # ⚠️
check_estimator(MeningT()) # ✅7. Integratsiya — bu bilim qayerda kerak bo'ladi
- 19.1-dars (o'tilgan): Estimator API
- 19.4-dars (o'tilgan):
check_estimator - 19.6-dars (o'tilgan): Versiya ogohlantirishi
- 19.9-dars: Tezlik va xotira
- 19.10-dars: To'liq loyiha
8. Eng yaxshi amaliyotlar
Xabarni oxirigacha o'qing.
error_score="raise"ishlab chiqishda.Kirishni oldin tekshiring.
quvur[:i+1]bilan lokalizatsiya.cross_validateni ishlating.Ogohlantirishni tuzating, o'chirmang.
config_contextbilan vaqtincha.check_estimatorni o'z kodingizga.
9. Amaliy topshiriq
Vazifa 1: Bashorat qiling
1. # NotFittedError sababi?
2. # "Input contains NaN" sababi?
3. # "X has N features" sababi?
4. # "feature names should match" sababi?
5. # "could not convert string" sababi?
6. # error_score sukut qiymati?
7. # error_score="raise" nima beradi?
8. # cross_validate qanday kalitlar beradi?
9. # return_estimator nima uchun?
10. # assume_finite xavfi?
11. # config_context nima qiladi?
12. # ConvergenceWarning nimani bildiradi?Javoblar
fitchaqirilmagan- Imputatsiya yo'q
- Ustunlar soni mos emas
- Ustun nomi yoki tartibi o'zgargan
- Kategoriyali ustun kodlanmagan
np.nan- Darhol to'xtaydi va stek izi beradi
fit_time,test_*,train_*,estimator,indices- Har foldning modelini ko'rish
- NaN ni jim o'tkazadi - xato chuqurroq joyda chiqadi
- Vaqtincha konfiguratsiya
- Model yaqinlashmagan
Vazifa 2: Xatolarni tuzating
1. GridSearchCV(quvur, setka, cv=cv) # xato yashirinadi
2. warnings.filterwarnings("ignore")
3. X2 = quvur["sc"].fit_transform(X)
4. set_config(assume_finite=True)
5. b = cross_val_score(quvur, X, y, cv=cv)Javoblar
1. GridSearchCV(quvur, setka, cv=cv, error_score="raise")
2. LogisticRegression(max_iter=5000) # sababini tuzating
3. quvur[:1].fit_transform(X, y)
4. with config_context(assume_finite=True): ...
5. cross_validate(quvur, X, y, cv=cv, return_train_score=True,
return_estimator=True)Vazifa 3: Xatolar
Modellang:
- Etti xato
- Xabar qismlari
- Tashxis
NotFittedError
Vazifa 4: Lokalizatsiya
Modellang:
- Buzuq quvur
- Kirish
- Tuzatilgan
- Ichki qadamlar
Vazifa 5: error_score
Modellang:
- Sukut
raisecross_validate- Beqarorlik
Vazifa 6: Konfiguratsiya
Modellang:
- Joriy
print_changed_onlytransform_output- Ogohlantirishlar
Vazifa 7: O'ylash
GridSearchCV xabar berdi: "All the 500 fits failed. ... 500 fits failed with the following error: ...". Xabar juda uzun va terminalda kesilib qolgan. Qanday harakat qilasiz?
Javob
Qisqa javob: qidiruvni to'xtating va error_score="raise" bilan bitta nomzodni ishga tushiring — xato darhol to'liq stek izi bilan chiqadi.
1. Eng tez yo'l
g = GridSearchCV(quvur, setka, cv=cv, n_jobs=1, error_score="raise")
g.fit(X, y) # birinchi xatoda to'xtaydi, to'liq stek izin_jobs=1 ham muhim: parallel ishda stek izi protsesslar orasida uzilishi mumkin.
2. Undan ham tezroq: qidiruvsiz
from sklearn.base import clone
tr, te = next(cv.split(X, y))
clone(quvur).set_params(**{k: v[0] for k, v in setka.items()}) \
.fit(X[tr], y[tr])Bitta fold, bitta nomzod — xato o'sha zahoti chiqadi.
3. Kirishni tekshirish (ko'pincha shu yerda)
print(X.shape, y.shape)
print(X.isna().sum().sum() if hasattr(X, "isna") else np.isnan(X).sum())
print(X.dtypes.value_counts() if hasattr(X, "dtypes") else X.dtype)
print(np.unique(y, return_counts=True))"All fits failed" ning eng ko'p uchraydigan sabablari:
| Sabab | Belgisi |
|---|---|
| NaN va imputatsiya yo'q | Input contains NaN |
| Kategoriyali ustun kodlanmagan | could not convert string to float |
X va y uzunligi farqli |
inconsistent numbers of samples |
| Noto'g'ri parametr nomi | Invalid parameter |
| Foldda bitta sinf | Only one class present |
4. Parametr nomlarini tekshirish
noto_gri = set(setka) - set(quvur.get_params(deep=True))
print("mavjud bo'lmagan parametrlar:", noto_gri)Bu xato juda tez-tez uchraydi va uni oldindan tekshirish bir qator kod.
5. Xabarni to'liq o'qish (agar boshqa yo'l bo'lmasa)
import warnings
with warnings.catch_warnings(record=True) as ogohlar:
warnings.simplefilter("always")
g = GridSearchCV(quvur, setka, cv=cv, n_jobs=1).fit(X, y)
for o in ogohlar:
print(str(o.message)[:2000]) # kesilmagan matn6. Xulosa
error_score="raise"+n_jobs=1- Bitta fold, bitta nomzodni qo'lda sinang
- Kirishni tekshiring (NaN, dtype, uzunlik)
- Parametr nomlarini
get_paramsbilan solishtiring
Nimani mustahkamlaydi: 2.2, 2.5-bo'limlar.
Xulosa
Bu darsda diagnostikani o'rgandik.
Eng muhim uch fikr:
Xato xabarining oxirgi qatorini o'qing. sklearn xatolarida eng aniq ma'lumot odatda oxirida bo'ladi:
Input contains NaN,X has 3 features, but StandardScaler is expecting 4,could not convert string to float. Har bir xabar aniq bir tashxisga mos keladi va uni yodda saqlash diagnostika vaqtini bir necha barobar qisqartiradi.error_score="raise"ni ishlab chiqish paytida yoqing. Sukut bo'yichanp.nanxatoni yashiradi va oxirida "All the N fits failed" degan yig'ma xabar beradi."raise"esa birinchi xatoda to'xtaydi va to'liq stek izini ko'rsatadi — sabab besh daqiqada topiladi.Nosozlikni
quvur[:i+1]bilan lokalizatsiya qiling. Qadamlarni birin-ketin bajarib, qaysi qadamda buzilishini aniqlang — quvurni buzmasdan.cross_validateesareturn_train_score,return_estimatorvareturn_indicesbilan foldlar orasidagi beqarorlikni va overfitting ni ko'rsatadi. Ogohlantirishlarni esa o'chirmang, sababini tuzating.
Keyingi darsda tezlik va xotirani ko'ramiz: n_jobs, siyrak matritsalar, dtype, partial_fit va katta ma'lumot bilan ishlash.
Izohlar (0)
Izoh yozish uchun kiring.
- Hozircha izoh yo'q. Birinchi bo'ling!