Mundarija (22)
- 1. Kirish va motivatsiya
- 2. Nazariya — chuqur tushuntirish
- 2.1. Loyiha xaritasi
- 2.2. Ma'lumot quvuri
- 2.3. O'qitish, eksperiment jurnali va paket
- 2.4. Xizmat
- 2.5. Monitoring va qaror qoidalari
- 2.6. Dockerfile va CI/CT
- 2.7. Hisobot va model kartasi
- 2.8. Tuzoqlar
- 3. Tez ma'lumotnoma
- 4. Batafsil misollar
- Misol 1 — Ma'lumot manbai, validatsiya va idempotent yuklash
- Misol 2 — O'qitish, eksperiment jurnali, registr va paket
- Misol 3 — FastAPI xizmati: validatsiya, API kaliti va so'rov loglari
- Misol 4 — Hayot sikli: drift, trigger, shadow, darvoza va versiya almashuvi
- 5. To'g'ri va noto'g'ri tushunishlar
- 6. Keng tarqalgan xatolar va yechimlari
- 7. Integratsiya — bu bilim qayerda kerak bo'ladi
- 8. Eng yaxshi amaliyotlar
- 9. Amaliy topshiriq
- Xulosa
27.14-dars: Amaliyot — mijoz ketishini bashorat qilish xizmati
27-QISM — MLOPS VA DEPLOY · 14-dars
1. Kirish va motivatsiya
27-qismda modelni noutbukdan ishlab chiqarishga olib chiqishning barcha bo'laklarini ko'rdik: MLOps nima va nega kerak, reproduksiya va muhit, ma'lumot quvuri, eksperiment kuzatuvi, model versiyalash va registr, paketlash, FastAPI bilan API, uni mustahkamlash va testlash, Docker, bulut va deploy strategiyalari, monitoring, drift aniqlash, qayta o'qitish va A/B test. Endi ularni bitta tizimga yig'amiz va eng muhim savolga javob beramiz: bu bo'laklar birgalikda ishlaganda, dunyo o'zgarganini tizim o'zi sezadimi va o'zini xavfsiz yangilay oladimi?
Vazifa — telekom kompaniyasi uchun mijoz ketishini bashorat qilish xizmati. Har oy yangi mijozlar kesimi keladi: oylik to'lov, xizmat muddati, qo'ng'iroqlar, shikoyatlar, tarif va hudud. Mijoz ketdimi — bu belgi bir oy kechikib keladi. Xizmat har oy barcha mijozlar uchun ketish ehtimolini beradi, saqlash bo'limi eng xavfli mijozlarga taklif yuboradi.
Loyiha to'rt qadamdan iborat, har biri alohida misol:
- Ma'lumot manbai va quvur — oylik partiyalar, validatsiya, karantin, idempotent va atomik yuklash sqlite ga.
- O'qitish va registr — vaqt bo'yicha bo'lish, ikki nomzod, eksperiment jurnali, juftlashgan taqqoslash, paket, darvoza va Production.
- Xizmat — FastAPI:
/predict,/health,/version, validatsiya, API kaliti muhit o'zgaruvchisidan, so'rov loglari. - Hayot sikli — keyingi 8 oy: drift, kechikkan belgilar, sifat tushishi, trigger, challenger shadow rejimda, darvoza, registrda versiya almashuvi, xizmat yangi versiyani yuklaydi, rollback imkoniyati.
Ma'lumotda 9-oydan boshlab dunyo o'zgaradi: raqobatchi arzon tarif chiqaradi (endi A tarifi mijozlari ketadi, avval C tarifi ketardi), narxlar oshadi, shikoyatlar ko'payadi va ularning ta'siri kuchayadi. Bu — kovariat, konsept va belgilar driftining aralashmasi, xuddi haqiqiy hayotdagidek. Tizim buni oldindan bilmaydi — barcha qaror qoidalari 4-qadam boshida, natijalarni ko'rishdan oldin yoziladi.
Real vaziyat. Telekom kompaniyasi ketish modelini bir marta o'qitib, ishga tushirdi va unutdi. Model kvartal hisobotlarida "AUC 0.73" deb yozilaverdi — bu raqam o'qitish paytidagi validatsiyadan olingan edi. Yarim yildan keyin saqlash bo'limi taklif yuborilgan mijozlarning deyarli hech biri ketmoqchi emasligini, ketganlar esa taklif olmaganini payqadi. Model haqiqiy AUC ni hech kim o'lchamagan: belgilar boshqa tizimda edi, bashoratlar loglanmagan edi, qayta o'qitish uchun quvur yo'q edi. Bu darsdagi tizimda xuddi shu drift 11-oyda trigger bilan ushlanadi va 12-oyda yangi model ishga tushadi.
Bu darsda to'liq ML xizmatini quramiz: xom partiyadan tortib o'zini kuzatadigan va xavfsiz yangilanadigan xizmatgacha.
Bu darsda:
- Loyiha xaritasi va qaror qoidalari (oldindan)
- Oylik partiyalar, validatsiya, karantin, idempotent va atomik yuklash
- Vaqt bo'yicha bo'lish, eksperiment jurnali, juftlashgan taqqoslash
- Paket:
Pipeline+ sxema + metadata + reference profil + nazorat namunalari - Registr: bosqichlar, ruxsat etilgan o'tishlar, audit
- FastAPI xizmati: validatsiya, API kaliti, so'rov loglari,
/version - Monitoring: PSI, kechikkan belgilar, sifat, trigger
- Shadow, darvoza, versiya almashuvi va rollback
- Dockerfile va CI/CT konfiguratsiyasi
- Tuzoqlar
ℹ Misollar real numpy/pandas/sklearn/FastAPI bilan (Python 3.14). Hammasi bitta vaqtinchalik papkada va bitta jarayon ichida: server ishga tushirilmaydi, xizmat
TestClientorqali sinaladi.
2. Nazariya — chuqur tushuntirish
2.1. Loyiha xaritasi
MANBA: oylik partiyalar (CSV) ----------------------------- (1-misol)
|
v
QUVUR: validatsiya (sxema, diapazon, toifalar, takror, hajm)
|-- xato -> KARANTIN (fayl saqlanadi, sabab yoziladi)
v
OMBOR (sqlite): idempotent (xesh bo'yicha), atomik (tranzaksiya)
|
v
O'QITISH: vaqt bo'yicha bo'lish (1-5 oy o'quv, 6-oy validatsiya) (2-misol)
nomzodlar: LogReg < HistGB (soddalik tartibida)
EKSPERIMENT JURNALI (sqlite): yurish_id = xesh(model, param, ma'lumot, kod)
juftlashgan bootstrap -> eng sodda munosib model
|
v
PAKET (joblib): Pipeline + sxema + metadata + profil + nazorat namunalari
|
v
REGISTR: None -> Staging -> [darvoza] -> Production; audit
|
v
XIZMAT (FastAPI): /predict /health /version /admin/yangila (3-misol)
kalit env dan, Pydantic validatsiya, har so'rov LOG ga (versiya bilan)
|
v
MONITORING (har oy): (4-misol)
kirish PSI + bashorat PSI -> erta ogohlantirish
(oy-1) belgilari keldi -> haqiqiy AUC -> TRIGGER qoidasi
challenger (oxirgi 2 oy) -> Staging -> SHADOW -> darvoza
-> Production, eski -> Archived -> xizmat yangilanadi -> /version
rollback: Archived -> Production (bitta amal, auditda)Har bosqich — alohida, sinaladigan komponent; tizim ular orasidagi shartnomalar (sxema, paket formati, registr bosqichlari) bilan birlashadi.
2.2. Ma'lumot quvuri
VALIDATSIYA (partiya darajasida - hammasi yoki hech narsa):
sxema: ustunlar ro'yxati va tartibi aynan mos
hajm: >= 500 qator
bo'shlar: har ustunda <= 1%
takror: mijoz_id noyob
diapazon: oylik_tolov 1..2000 (ming so'm), muddat_oy >= 0
toifalar: tarif in {A,B,C}, hudud in {4 ta}
belgi: ketdi in {0,1}
YUKLASH:
xesh = sha256(kanonik CSV: saralangan, qat'iy format)
xesh o'zgarmagan -> o'tkazib yuboriladi (IDEMPOTENT)
xesh yangi -> BEGIN; DELETE oy; INSERT; jurnal; COMMIT (ATOMIK)
uzilish -> ROLLBACK: eski ma'lumot butun qoladiValidatsiyadan o'tmagan partiya o'chirilmaydi — u karantin papkasiga ko'chiriladi, sababi yoziladi. Manba tuzatilib qayta yetkazilsa, quvur uni oddiy partiya kabi yuklaydi. Birlik xatosi (so'm o'rniga ming so'm) — eng xavfli xato turlaridan biri: sxema to'g'ri, turlar to'g'ri, faqat qiymatlar 1000 marta katta. Uni faqat diapazon tekshiruvi ushlaydi. Model esa bunday partiyada jim ishlayveradi — va hamma mijozni "juda qimmat to'laydi, ketadi" deb baholaydi.
2.3. O'qitish, eksperiment jurnali va paket
BO'LISH - VAQT BO'YICHA (tasodifiy emas!):
o'quv 1-5 oy, validatsiya 6-oy ("kelajakni" bashorat qilish sinovi)
yakuniy model 1-6 oyda qayta o'qitiladi (xuddi shu konfig)
EKSPERIMENT JURNALI (MLflow g'oyasi, sqlite bilan):
yurish_id = sha256(model, parametrlar, ma'lumot xeshi, kod versiyasi)[:10]
bir xil yurish qayta yozilsa - TAKRORLANMAYDI (INSERT OR REPLACE)
TAQQOSLASH: 6-oyda juftlashgan bootstrap (300 marta)
farq = AUC_HistGB - AUC_LogReg, SE = bootstrap std
QOIDA: eng yaxshisidan sezilarli yomon bo'lmagan ENG SODDA
PAKET (joblib):
format - versiya, yuklashda tekshiriladi
pipeline - tayyorlash + model BITTA obyekt (train/serve farqi yo'q)
sxema - ustunlar va ruxsat etilgan toifalar
metadata - oylar, ma'lumot xeshi, kod versiyasi, val_auc
profil - reference taqsimotlar (monitoring uchun, 4-misol)
nazorat - 200 ta kirish + kutilgan chiqish (yuklashda tekshiriladi)Haqiqiy loyihada jurnal va registr uchun MLflow, ma'lumot versiyasi uchun DVC ishlatiladi — bu darsda xuddi shu g'oyalar bir necha o'nlab qator sqlite kodi bilan ko'rsatiladi, shunda ichida nima bo'layotgani ko'rinadi.
2.4. Xizmat
ENDPOINTLAR:
GET /health -> {"holat": "ok", "model_yuklangan": true}
GET /version -> {"nom", "versiya", ...} (qaysi model javob beryapti)
POST /predict -> X-API-Key sarlavhasi + {"mijozlar": [...]}
POST /admin/yangila -> registrdan Production ni qayta o'qish (kalit bilan)
HIMOYA:
kalit: os.environ["CHURN_API_KEY"] - kodda, image da, gitda YO'Q
o'rnatilmagan bo'lsa - xizmat ISHGA TUSHMAYDI
solishtirish: hmac.compare_digest (vaqt bo'yicha sizib chiqmaydi)
loglarda: faqat "********"
kirish: Pydantic, extra="forbid", diapazonlar, Literal toifalar -> 422
yuklash: paket nazorat namunalari mos kelmasa - yuklanmaydi
SO'ROV LOGI (monitoringning xom ashyosi):
oy, mijoz_id, versiya, p, shadow_versiya, shadow_pEslatma: bu darsda kalit tekshiruvi endpoint ichida yozilgan — qisqa bo'lishi uchun. 27.8-darsdagidek uni Depends bilan umumiy bog'liqlikka chiqarish tavsiya etiladi.
2.5. Monitoring va qaror qoidalari
Qoidalar 4-misolning boshida, natijalardan oldin konstantalar sifatida yozilgan:
ERTA OGOHLANTIRISH (belgisiz, har oy):
PSI(oylik_tolov), PSI(tarif), PSI(bashorat) > 0.10 -> "drift ogohl."
reference = JORIY Production modelning o'quv taqsimoti (paketdagi profil)
-> model almashsa, reference ham almashadi
SIFAT (belgilar 1 oy kechikadi):
oy boshida (oy-1) belgilari keladi -> so'rov logi bilan birlashtiriladi
AUC(oy-1) - xizmat HAQIQATAN bergan bashoratlar bo'yicha
TRIGGER:
AUC < reference_AUC - 0.03 IKKI oy ketma-ket (bitta oy - shovqin bo'lishi mumkin)
-> challenger: oxirgi 2 belgili oyda (27.13: to'satdan drift -> qisqa oyna)
-> registr: Staging -> xizmat uni SHADOW rejimda ishlatadi
DARVOZA (shadow oyi belgilari kelgach, BIR XIL so'rovlarda):
AUC farqi > 2*SE (bootstrap)
log loss juftlashgan farqi < -2*SE
har tarifda log loss yomonlashuvi <= 0.02
shadow xatolari = 0
-> Production; eski -> Archived; xizmat /admin/yangila
ROLLBACK: Archived -> Production ruxsat etilgan o'tish (auditda)Kechikish zanjiri halol hisoblanadi: drift 9-oyda boshlanadi, 9-oy belgilari 10-oy boshida keladi, qoida ikki oy talab qiladi — shuning uchun trigger eng erta 11-oyda, yangi model esa shadow oyidan keyin, eng erta 12-oyda ishga tushishi mumkin. Tizimni "tezroq" qilishning halol yo'llari 27.13 da ko'rilgan: proksi belgilar, erta ogohlantirishga tayyorgarlik, tarixiy so'rovlarda replay.
2.6. Dockerfile va CI/CT
Misollarda Docker va CI ishga tushirilmaydi, lekin loyiha ularsiz to'liq emas. 27.9-darsdagi qoidalar bo'yicha:
# Dockerfile - mijoz ketishi xizmati
FROM python:3.12.7-slim AS quruvchi
COPY requirements.lock .
RUN pip install --no-cache-dir --prefix=/install -r requirements.lock
FROM python:3.12.7-slim
RUN useradd --create-home --uid 10001 xizmat
COPY --from=quruvchi /install /usr/local
WORKDIR /app
COPY src/ /app/src/
# model artefakti image ga KIRMAYDI: registrdan ishga tushishda yuklanadi
# CHURN_API_KEY - faqat ishga tushirishda (secret), image da emas
ENV PYTHONUNBUFFERED=1
USER xizmat
EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=3s CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health')"
CMD ["uvicorn", "churn.xizmat:app", "--host", "0.0.0.0", "--port", "8000"]# .github/workflows/ci.yml - har push va PR da
name: ci
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install -r requirements.lock
- run: python -m pytest -q tests/ # quvur, paket, xizmat testlari
- run: python -m churn.paket --tekshir # nazorat namunalari
image:
needs: test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: docker build -t churn-xizmat:${{ github.sha }} .# .github/workflows/ct.yml - uzluksiz o'qitish (CT): jadval + qo'lda
name: ct
on:
schedule:
- cron: "0 3 2 * *" # har oyning 2-kuni (belgilar kelgandan keyin)
workflow_dispatch: {}
jobs:
monitoring-va-trigger:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: pip install -r requirements.lock
- run: python -m churn.monitoring --oy oxirgi # AUC, PSI, trigger
- run: python -m churn.orgatish --agar-trigger # challenger -> StagingMuhim nuqtalar: image teg sifatida commit xeshi oladi (:latest emas); model artefakti image ichida emas — xizmat uni registrdan oladi, shuning uchun model almashuvi yangi image talab qilmaydi (4-misoldagi /admin/yangila shu g'oya); kalit image ga yozilmaydi.
2.7. Hisobot va model kartasi
MODEL KARTASI (registr yozuviga biriktiriladi, 26.10 va 27.5):
vazifa, foydalanuvchi, cheklovlar
ma'lumot: oylar, xesh, ketish ulushi
metrika: val AUC (vaqt bo'yicha), kalibrovka
qaror: nomzodlar, juftlashgan farq, qoida
monitoring: reference profil, trigger va darvoza qoidalari
tarix: versiyalar, sabablar (audit)2.8. Tuzoqlar
Asosiy tuzoqlar: validatsiyasiz yuklash (birlik xatosi modelga jim kiradi); buzuq partiyani o'chirib yuborish (karantin o'rniga); qayta yuklashda takrorlanadigan qatorlar; tranzaksiyasiz yuklash; tasodifiy bo'lish bilan vaqtli ma'lumotda baholash; qarorni bitta AUC bilan qabul qilish; tayyorlash va modelni alohida saqlash (train/serve farqi); kalitni kodda yoki image da saqlash; bashoratlarni loglamaslik (keyin sifatni o'lchab bo'lmaydi); monitoring reference ni model almashganda yangilamaslik; bitta past oyda qayta o'qitish (shovqin); shadow siz almashtirish; eski artefaktni o'chirish.
3. Tez ma'lumotnoma
# 1) quvur
xatolar = validatsiya(df) # [] bo'lsa - toza
holat, xesh = yukla(db, oy, df) # idempotent + atomik
# 2) o'qitish va registr
pipe = logreg().fit(tr, tr.ketdi)
yid = yurish_yoz(db, "LogReg", param, metrika, malumot_xeshi(tr))
joblib.dump(paket_yasa(pipe, df.iloc[:200], metadata), yol)
bosqich_ozgartir(db, 1, "Staging", "yangi nomzod")
bosqich_ozgartir(db, 1, "Production", "darvoza OK")
# 3) xizmat
os.environ["CHURN_API_KEY"] # sir faqat muhitdan
mijoz = TestClient(ilova_yarat(Xizmat(papka)))
mijoz.post("/predict", json={"mijozlar": [...]}, headers={"X-API-Key": kalit})
# 4) monitoring va hayot sikli
psi(paket["profil"]["oylik_tolov"], df.oylik_tolov.to_numpy())
auc = roc_auc_score(j.ketdi, j.p) # j = so'rov logi + belgilar
if past_ketma_ket >= 2: # trigger -> challenger shadow
registr.otkaz(oy, 2, "Staging", "trigger: AUC 2 oy past")
if all(ok for ok, _ in darvoza_tekshir(j).values()):
registr.otkaz(oy, 2, "Production", "darvoza OK")
mijoz.post("/admin/yangila", headers=H) # /version -> 2Loyiha tuzilishi (haqiqiy repoda)
mijoz_ketishi/
requirements.lock pinlangan versiyalar 27.2-bob
Dockerfile .dockerignore 27.9-bob
konfig/qoidalar.yaml trigger, darvoza, PSI chegaralari (versiyalanadi!)
src/churn/
manba.py partiyalarni olish
quvur.py validatsiya, karantin, yuklash
orgatish.py nomzodlar, bo'lish, taqqoslash
kuzatuv.py eksperiment jurnali
registr.py bosqichlar, audit
paket.py yasash, yuklash, nazorat
xizmat.py FastAPI ilova
monitoring.py PSI, sifat, trigger, darvoza
tests/ har modul uchun 27.8-bob
.github/workflows/ci.yml, ct.ymlAmaliyot xulosasi
partiya -> validatsiya (karantin) -> ombor (idempotent, atomik)
vaqt bo'yicha bo'lish -> jurnal -> juftlashgan qaror -> paket (nazorat)
registr: Staging -> darvoza -> Production (audit)
xizmat: kalit env dan, validatsiya, log (versiya bilan)
monitoring: PSI (erta) + kechikkan belgilar (AUC) -> trigger
challenger -> shadow -> darvoza -> Production -> /version; rollback4. Batafsil misollar
Misollar real numpy/pandas/sklearn/FastAPI bilan (Python 3.14). Har misol mustaqil ishlaydi, shuning uchun umumiy kod (ma'lumot manbai, model, paket, registr) har birida takrorlanadi. Haqiqiy loyihada bu kod 3-bo'limdagi modullarda bir marta yoziladi.
Misol 1 — Ma'lumot manbai, validatsiya va idempotent yuklash
"""1-qadam: ma'lumot manbai (oylik partiyalar), validatsiya va idempotent yuklash."""
import hashlib
import io
import sqlite3
import tempfile
from pathlib import Path
import numpy as np
import pandas as pd
SON = ["oylik_tolov", "muddat_oy", "qongiroqlar", "shikoyatlar"]
KAT = ["tarif", "hudud"]
USTUNLAR = ["mijoz_id", "oy"] + SON + KAT + ["ketdi"]
TARIFLAR = ["A", "B", "C"]
HUDUDLAR = ["Toshkent", "Samarqand", "Buxoro", "Namangan"]
DRIFT_OYI = 9 # shu oydan dunyo o'zgaradi (27.12)
def oy_partiyasi(oy, n=1500):
"""Har oy - mijozlar kesimi; ketdi = keyingi oyda ketdimi (kechikib keladi)."""
rng = np.random.default_rng(1000 + oy)
d = oy >= DRIFT_OYI # raqobatchi arzon tarif chiqardi
tarif = rng.choice(TARIFLAR, n, p=[0.35, 0.30, 0.35] if d else [0.5, 0.3, 0.2])
hudud = rng.choice(HUDUDLAR, n, p=[0.4, 0.25, 0.2, 0.15])
tolov = rng.lognormal(np.log(120), 0.35, n) * (1.15 if d else 1.0)
muddat = rng.gamma(2.0, 12.0, n)
qongiroq = rng.poisson(25, n)
shikoyat = rng.poisson(0.8 if d else 0.5, n)
z = (np.log(tolov) - np.log(120)) / 0.35
f = (-1.8 + 1.0 * (muddat < 4) - 0.015 * muddat
+ 0.3 * (hudud == "Namangan")
+ (1.0 * (tarif == "A") - 0.3 * (tarif == "C") if d
else 1.0 * (tarif == "C"))
+ (0.9 if d else 0.5) * shikoyat + (0.3 if d else 0.7) * z)
ketdi = (rng.random(n) < 1 / (1 + np.exp(-f))).astype(int)
return pd.DataFrame({
"mijoz_id": [f"M{oy:02d}{i:05d}" for i in range(n)], "oy": oy,
"oylik_tolov": tolov.round(2), "muddat_oy": muddat.round(1),
"qongiroqlar": qongiroq, "shikoyatlar": shikoyat,
"tarif": tarif, "hudud": hudud, "ketdi": ketdi})
# ---------------- quvur: validatsiya ----------------
def validatsiya(df):
xatolar = []
if list(df.columns) != USTUNLAR:
return [f"sxema: kutilgan {len(USTUNLAR)} ustun, "
f"kelgan {sorted(set(df.columns) ^ set(USTUNLAR))} farq"]
if len(df) < 500:
xatolar.append(f"hajm: {len(df)} < 500")
if df.isna().mean().max() > 0.01:
xatolar.append("bo'sh qiymatlar > 1%")
if df["mijoz_id"].duplicated().any():
xatolar.append(f"takror id: {int(df['mijoz_id'].duplicated().sum())}")
if not df["oylik_tolov"].between(1, 2000).all():
xatolar.append(f"oylik_tolov diapazondan tashqari "
f"(maks {df['oylik_tolov'].max():.0f})")
if (df["muddat_oy"] < 0).any():
xatolar.append("muddat_oy < 0")
for c, ruxsat in [("tarif", TARIFLAR), ("hudud", HUDUDLAR)]:
yangi = sorted(set(df[c]) - set(ruxsat))
if yangi:
xatolar.append(f"{c}: noma'lum qiymatlar {yangi}")
if not df["ketdi"].isin([0, 1]).all():
xatolar.append("ketdi faqat 0/1")
return xatolar
def kanonik_xesh(df):
"""Tartib va format bir xil bo'lsa - xesh bir xil 27.5-bob."""
buf = io.StringIO()
df.sort_values("mijoz_id").to_csv(buf, index=False, float_format="%.4f",
lineterminator="\n")
return hashlib.sha256(buf.getvalue().encode()).hexdigest()[:12]
# ---------------- quvur: idempotent yuklash ----------------
def db_och(yol):
db = sqlite3.connect(yol)
db.executescript(
"CREATE TABLE IF NOT EXISTS mijozlar (mijoz_id TEXT, oy INT,"
" oylik_tolov REAL, muddat_oy REAL, qongiroqlar INT, shikoyatlar INT,"
" tarif TEXT, hudud TEXT, ketdi INT, PRIMARY KEY (oy, mijoz_id));"
"CREATE TABLE IF NOT EXISTS yuklashlar (oy INT PRIMARY KEY,"
" xesh TEXT, qatorlar INT, holat TEXT);")
return db
def yukla(db, oy, df, buzilish=False):
xesh = kanonik_xesh(df)
eski = db.execute("SELECT xesh FROM yuklashlar WHERE oy = ?",
(oy,)).fetchone()
if eski and eski[0] == xesh:
return "o'zgarmagan - o'tkazildi", xesh
with db: # tranzaksiya
db.execute("DELETE FROM mijozlar WHERE oy = ?", (oy,))
if buzilish:
raise RuntimeError("yuklash o'rtasida uzilish")
db.executemany("INSERT INTO mijozlar VALUES (?,?,?,?,?,?,?,?,?)",
df[USTUNLAR].itertuples(index=False, name=None))
db.execute("INSERT OR REPLACE INTO yuklashlar VALUES (?,?,?,?)",
(oy, xesh, len(df), "yuklandi"))
return ("almashtirildi" if eski else "yuklandi"), xesh
def quvur(db, manba, karantin):
natija = []
for fayl in sorted(manba.glob("oy_*.csv")):
oy = int(fayl.stem.split("_")[1])
df = pd.read_csv(fayl)
xatolar = validatsiya(df)
if xatolar:
(karantin / fayl.name).write_bytes(fayl.read_bytes())
natija.append((fayl.name, "KARANTIN", "; ".join(xatolar)))
continue
holat, xesh = yukla(db, oy, df)
natija.append((fayl.name, holat, xesh))
return natija
def main() -> None:
with tempfile.TemporaryDirectory() as tmp:
manba, karantin = Path(tmp, "manba"), Path(tmp, "karantin")
manba.mkdir()
karantin.mkdir()
db = db_och(Path(tmp, "ombor.sqlite"))
print("=== 1. Manba: 1-6 oy partiyalari (ikkitasi buzuq) ===")
for oy in range(1, 7):
df = oy_partiyasi(oy)
if oy == 4:
df["oylik_tolov"] *= 1000 # so'mda keldi (birlik xatosi)
if oy == 5:
df = df.drop(columns=["hudud"]) # ustun tushib qoldi
df.to_csv(manba / f"oy_{oy:02d}.csv", index=False)
print(f" fayllar: {sorted(p.name for p in manba.iterdir())}")
print("\n=== 2. Quvur: 1-yurish ===")
for nom, holat, izoh in quvur(db, manba, karantin):
print(f" {nom}: {holat:<24} {izoh}")
print("\n=== 3. Quvur: 2-yurish (xuddi shu fayllar) - idempotentlik ===")
oldin = db.execute("SELECT COUNT(*) FROM mijozlar").fetchone()[0]
for nom, holat, izoh in quvur(db, manba, karantin):
if holat != "KARANTIN":
print(f" {nom}: {holat}")
keyin = db.execute("SELECT COUNT(*) FROM mijozlar").fetchone()[0]
print(f" qatorlar: {oldin} -> {keyin} (takrorlanmadi: {oldin == keyin})")
print("\n=== 4. Manba tuzatildi: 4 va 5-oy qayta yetkazildi ===")
for oy in [4, 5]:
oy_partiyasi(oy).to_csv(manba / f"oy_{oy:02d}.csv", index=False)
(karantin / f"oy_{oy:02d}.csv").unlink()
for nom, holat, izoh in quvur(db, manba, karantin):
if holat != "o'zgarmagan - o'tkazildi":
print(f" {nom}: {holat:<24} {izoh}")
print("\n=== 5. Atomiklik: yuklash o'rtasida uzilish ===")
df2 = oy_partiyasi(2)
df2.loc[0, "shikoyatlar"] = 3 # tuzatilgan versiya
oldin = db.execute("SELECT COUNT(*) FROM mijozlar WHERE oy = 2"
).fetchone()[0]
try:
yukla(db, 2, df2, buzilish=True)
except RuntimeError as e:
print(f" xato: {e}")
keyin = db.execute("SELECT COUNT(*) FROM mijozlar WHERE oy = 2"
).fetchone()[0]
print(f" 2-oy qatorlari: {oldin} -> {keyin} (tranzaksiya qaytarildi)")
holat, xesh = yukla(db, 2, df2)
print(f" qayta urinish: {holat}, xesh {xesh}")
print("\n=== 6. Ombor holati ===")
jad = pd.read_sql("SELECT oy, COUNT(*) AS qator, AVG(ketdi) AS ketish,"
" AVG(oylik_tolov) AS tolov FROM mijozlar"
" GROUP BY oy ORDER BY oy", db)
print(jad.round(3).to_string(index=False))
print(f" yuklashlar jurnali: "
f"{db.execute('SELECT COUNT(*) FROM yuklashlar').fetchone()[0]}"
f" oy, karantinda: {len(list(karantin.iterdir()))} fayl")
db.close()
print(" ⭐ Validatsiya - darvoza, yuklash - idempotent va atomik")
if __name__ == "__main__":
main()Natijaning muhim qismi:
=== 1. Manba: 1-6 oy partiyalari (ikkitasi buzuq) ===
fayllar: ['oy_01.csv', 'oy_02.csv', 'oy_03.csv', 'oy_04.csv', 'oy_05.csv', 'oy_06.csv']
=== 2. Quvur: 1-yurish ===
oy_01.csv: yuklandi 610779f5e002
oy_02.csv: yuklandi d8a70865178a
oy_03.csv: yuklandi f1bff71f671e
oy_04.csv: KARANTIN oylik_tolov diapazondan tashqari (maks 316510)
oy_05.csv: KARANTIN sxema: kutilgan 9 ustun, kelgan ['hudud'] farq
oy_06.csv: yuklandi 1e64911df0ac
=== 3. Quvur: 2-yurish (xuddi shu fayllar) - idempotentlik ===
oy_01.csv: o'zgarmagan - o'tkazildi
oy_02.csv: o'zgarmagan - o'tkazildi
oy_03.csv: o'zgarmagan - o'tkazildi
oy_06.csv: o'zgarmagan - o'tkazildi
qatorlar: 6000 -> 6000 (takrorlanmadi: True)
=== 4. Manba tuzatildi: 4 va 5-oy qayta yetkazildi ===
oy_04.csv: yuklandi baf34efc64c9
oy_05.csv: yuklandi 78fcee6b69db
=== 5. Atomiklik: yuklash o'rtasida uzilish ===
xato: yuklash o'rtasida uzilish
2-oy qatorlari: 1500 -> 1500 (tranzaksiya qaytarildi)
qayta urinish: almashtirildi, xesh 098c77c05202
=== 6. Ombor holati ===
oy qator ketish tolov
1 1500 0.198 129.019
2 1500 0.214 126.245
3 1500 0.185 126.401
4 1500 0.193 126.462
5 1500 0.192 128.142
6 1500 0.213 128.860
yuklashlar jurnali: 6 oy, karantinda: 0 fayl
⭐ Validatsiya - darvoza, yuklash - idempotent va atomikNatija tahlili.
1-2-bo'limlar — manbadan olti oylik partiya keldi, ikkitasi buzuq. 4-oy partiyasida to'lov so'mda kelgan (birlik xatosi) — sxema va turlar to'g'ri, lekin diapazon tekshiruvi maksimum 316510 ni ushladi. 5-oyda hudud ustuni tushib qolgan — sxema tekshiruvi. Ikkala fayl karantinga ko'chirildi, qolgan to'rttasi yuklandi va har biri uchun kanonik xesh yozildi.
3-bo'lim — xuddi shu fayllar bilan quvur qayta ishga tushirildi (masalan, rejalashtiruvchi ikki marta ishga tushirib yubordi). Har partiya "o'zgarmagan — o'tkazildi", qatorlar soni 6000 -> 6000. Bu — idempotentlik: quvurni necha marta ishga tushirsangiz ham natija bir xil.
4-bo'lim — manba xatolarni tuzatib, 4 va 5-oyni qayta yetkazdi. Karantindagi nusxalar olib tashlandi, yangi fayllar validatsiyadan o'tib yuklandi.
5-bo'lim — atomiklik. 2-oyning tuzatilgan versiyasini yuklash DELETE dan keyin, INSERT dan oldin uzildi. Tranzaksiya qaytarildi — 2-oyda baribir 1500 qator, ya'ni eski ma'lumot butun. Qayta urinishda partiya "almashtirildi" — xesh o'zgargani uchun (bitta qatorda shikoyatlar tuzatilgan).
6-bo'lim — omborda olti oy, har birida 1500 qator; ketish ulushi 0.18-0.21 atrofida barqaror. Karantin bo'sh — hamma muammo hal qilingan.
Misol 2 — O'qitish, eksperiment jurnali, registr va paket
"""2-qadam: o'qitish, eksperiment kuzatuvi (sqlite), registr va paket (joblib)."""
import hashlib
import json
import sqlite3
import tempfile
import warnings
from pathlib import Path
import joblib
import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import log_loss, roc_auc_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder, StandardScaler
SON = ["oylik_tolov", "muddat_oy", "qongiroqlar", "shikoyatlar"]
KAT = ["tarif", "hudud"]
USTUNLAR = ["mijoz_id", "oy"] + SON + KAT + ["ketdi"]
TARIFLAR = ["A", "B", "C"]
HUDUDLAR = ["Toshkent", "Samarqand", "Buxoro", "Namangan"]
DRIFT_OYI = 9
KOD_VERSIYASI = "churn-1.0.0"
NOM = "mijoz_ketishi"
def oy_partiyasi(oy, n=1500):
"""Har oy - mijozlar kesimi; ketdi = keyingi oyda ketdimi (kechikib keladi)."""
rng = np.random.default_rng(1000 + oy)
d = oy >= DRIFT_OYI # raqobatchi arzon tarif chiqardi
tarif = rng.choice(TARIFLAR, n, p=[0.35, 0.30, 0.35] if d else [0.5, 0.3, 0.2])
hudud = rng.choice(HUDUDLAR, n, p=[0.4, 0.25, 0.2, 0.15])
tolov = rng.lognormal(np.log(120), 0.35, n) * (1.15 if d else 1.0)
muddat = rng.gamma(2.0, 12.0, n)
qongiroq = rng.poisson(25, n)
shikoyat = rng.poisson(0.8 if d else 0.5, n)
z = (np.log(tolov) - np.log(120)) / 0.35
f = (-1.8 + 1.0 * (muddat < 4) - 0.015 * muddat
+ 0.3 * (hudud == "Namangan")
+ (1.0 * (tarif == "A") - 0.3 * (tarif == "C") if d
else 1.0 * (tarif == "C"))
+ (0.9 if d else 0.5) * shikoyat + (0.3 if d else 0.7) * z)
ketdi = (rng.random(n) < 1 / (1 + np.exp(-f))).astype(int)
return pd.DataFrame({
"mijoz_id": [f"M{oy:02d}{i:05d}" for i in range(n)], "oy": oy,
"oylik_tolov": tolov.round(2), "muddat_oy": muddat.round(1),
"qongiroqlar": qongiroq, "shikoyatlar": shikoyat,
"tarif": tarif, "hudud": hudud, "ketdi": ketdi})
def ombor_qur(yol, oylar):
"""1-qadam natijasi: validatsiyadan o'tgan partiyalar sqlite da."""
db = sqlite3.connect(yol)
pd.concat([oy_partiyasi(o) for o in oylar]).to_sql("mijozlar", db,
index=False)
return db
def malumot_xeshi(df):
return hashlib.sha256(pd.util.hash_pandas_object(
df[USTUNLAR], index=False).values.tobytes()).hexdigest()[:12]
# ---------------- nomzodlar ----------------
def logreg():
return Pipeline([
("tayyor", ColumnTransformer([
("son", StandardScaler(), SON),
("kat", OneHotEncoder(handle_unknown="ignore"), KAT)])),
("model", LogisticRegression(max_iter=2000))])
def histgb():
return Pipeline([
("tayyor", ColumnTransformer([
("son", "passthrough", SON),
("kat", OrdinalEncoder(handle_unknown="use_encoded_value",
unknown_value=-1), KAT)])),
("model", HistGradientBoostingClassifier(
max_iter=150, learning_rate=0.05, categorical_features=[4, 5],
random_state=0))])
NOMZODLAR = {"LogReg": (logreg, {"C": 1.0}),
"HistGB": (histgb, {"max_iter": 150, "learning_rate": 0.05})}
SODDALIK = ["LogReg", "HistGB"]
# ---------------- eksperiment kuzatuvi (MLflow g'oyasi) ----------------
def kuzatuv_och(db):
db.execute("CREATE TABLE IF NOT EXISTS yurishlar (yurish_id TEXT PRIMARY"
" KEY, model TEXT, parametrlar TEXT, metrikalar TEXT,"
" malumot_xeshi TEXT, kod TEXT)")
def yurish_yoz(db, model, param, metrika, mx):
yid = hashlib.sha256(json.dumps([model, param, mx, KOD_VERSIYASI],
sort_keys=True).encode()).hexdigest()[:10]
with db:
db.execute("INSERT OR REPLACE INTO yurishlar VALUES (?,?,?,?,?,?)",
(yid, model, json.dumps(param, sort_keys=True),
json.dumps(metrika, sort_keys=True), mx, KOD_VERSIYASI))
return yid
# ---------------- registr 27.5-bob ----------------
RUXSAT = {("None", "Staging"), ("Staging", "Production"),
("Production", "Archived"), ("Staging", "Archived")}
def registr_och(db):
db.executescript(
"CREATE TABLE IF NOT EXISTS versiyalar (nom TEXT, versiya INT,"
" bosqich TEXT, yurish_id TEXT, artefakt TEXT, artefakt_xeshi TEXT,"
" metrikalar TEXT, PRIMARY KEY (nom, versiya));"
"CREATE TABLE IF NOT EXISTS audit (id INTEGER PRIMARY KEY"
" AUTOINCREMENT, nom TEXT, versiya INT, eski TEXT, yangi TEXT,"
" sabab TEXT);")
def bosqich_ozgartir(db, versiya, yangi, sabab):
eski = db.execute("SELECT bosqich FROM versiyalar WHERE nom=? AND"
" versiya=?", (NOM, versiya)).fetchone()[0]
if (eski, yangi) not in RUXSAT:
raise ValueError(f"ruxsat yo'q: {eski} -> {yangi}")
with db:
db.execute("UPDATE versiyalar SET bosqich=? WHERE nom=? AND"
" versiya=?", (yangi, NOM, versiya))
db.execute("INSERT INTO audit (nom, versiya, eski, yangi, sabab)"
" VALUES (?,?,?,?,?)", (NOM, versiya, eski, yangi, sabab))
def paket_yasa(pipe, df_nazorat, metadata):
return {"format": 1, "pipeline": pipe,
"sxema": {"son": SON, "kat": KAT,
"kategoriyalar": {"tarif": TARIFLAR, "hudud": HUDUDLAR}},
"metadata": metadata,
"nazorat": {"X": df_nazorat[SON + KAT].to_dict("list"),
"p": pipe.predict_proba(df_nazorat)[:, 1].tolist()}}
def paket_yukla(yol):
p = joblib.load(yol)
if p["format"] != 1:
raise ValueError("format mos emas")
X = pd.DataFrame(p["nazorat"]["X"])
farq = np.max(np.abs(p["pipeline"].predict_proba(X)[:, 1]
- np.array(p["nazorat"]["p"])))
return p, float(farq)
def main() -> None:
warnings.simplefilter("ignore")
with tempfile.TemporaryDirectory() as tmp:
ombor = ombor_qur(Path(tmp, "ombor.sqlite"), range(1, 7))
df = pd.read_sql("SELECT * FROM mijozlar ORDER BY oy, mijoz_id", ombor)
ombor.close()
tr, va = df[df.oy <= 5], df[df.oy == 6]
mx = malumot_xeshi(df)
print("=== 1. Dizayn: vaqt bo'yicha bo'lish ===")
print(f" o'quv: 1-5 oy ({len(tr)} qator), validatsiya: 6-oy "
f"({len(va)}), ketish ulushi {df.ketdi.mean():.3f}")
print(f" ma'lumot xeshi (1-6 oy): {mx}")
mlops = sqlite3.connect(Path(tmp, "mlops.sqlite"))
kuzatuv_och(mlops)
print("\n=== 2. Nomzodlar va eksperiment jurnali ===")
p_val, metrikalar = {}, {}
for nom, (yasa, param) in NOMZODLAR.items():
pipe = yasa().fit(tr, tr.ketdi)
p = pipe.predict_proba(va)[:, 1]
p_val[nom] = p
m = {"val_auc": round(float(roc_auc_score(va.ketdi, p)), 4),
"val_logloss": round(float(log_loss(va.ketdi, p)), 4),
"kalibrovka": round(float(abs(p.mean() - va.ketdi.mean())), 4)}
metrikalar[nom] = m
yid = yurish_yoz(mlops, nom, param, m, malumot_xeshi(tr))
print(f" {nom:<7} yurish {yid}: {m}")
yurish_yoz(mlops, "LogReg", NOMZODLAR["LogReg"][1], # xuddi shu
metrikalar["LogReg"], malumot_xeshi(tr)) # yurish yana
n = mlops.execute("SELECT COUNT(*) FROM yurishlar").fetchone()[0]
print(f" jurnalda {n} yurish (qayta yozilgan yurish takrorlanmadi - "
f"id = xesh)")
print("\n=== 3. Juftlashgan bootstrap: HistGB - LogReg (6-oy) ===")
rng = np.random.default_rng(0)
y = va.ketdi.to_numpy()
farqlar = []
for _ in range(300):
i = rng.integers(0, len(y), len(y))
farqlar.append(roc_auc_score(y[i], p_val["HistGB"][i])
- roc_auc_score(y[i], p_val["LogReg"][i]))
farqlar = np.array(farqlar)
d = roc_auc_score(y, p_val["HistGB"]) - roc_auc_score(y, p_val["LogReg"])
se = farqlar.std(ddof=1)
print(f" AUC farqi {d:+.4f}, bootstrap SE {se:.4f}, "
f"sezilarli: {abs(d) > 2 * se}")
eng = "HistGB" if d > 0 else "LogReg"
tanlov = "LogReg" if (eng == "LogReg" or d <= 2 * se) else "HistGB"
print(f" qoida: eng yaxshisidan sezilarli yomon bo'lmagan ENG SODDA "
f"-> {tanlov}")
print("\n=== 4. Yakuniy model (1-6 oy), paket va nazorat ===")
yakuniy = NOMZODLAR[tanlov][0]().fit(df, df.ketdi)
artefaktlar = Path(tmp, "artefaktlar")
artefaktlar.mkdir()
metadata = {"nom": NOM, "model": tanlov, "oylar": [1, 6],
"malumot_xeshi": mx, "kod": KOD_VERSIYASI,
"val_auc": round(float(roc_auc_score(y, p_val[tanlov])), 4),
"reference_ketish": round(float(df.ketdi.mean()), 4)}
paket = paket_yasa(yakuniy, df.iloc[:200], metadata)
yol = artefaktlar / f"{NOM}-v1.joblib"
joblib.dump(paket, yol)
artefakt_xeshi = hashlib.sha256(yol.read_bytes()).hexdigest()[:12]
_, farq = paket_yukla(yol)
print(f" paket kalitlari: {sorted(paket)}")
print(f" metadata: {metadata}")
print(f" qayta yuklangach nazorat namunalari maks farq: {farq:.2e}")
print("\n=== 5. Registr va darvoza ===")
registr_och(mlops)
yid = mlops.execute("SELECT yurish_id FROM yurishlar WHERE model=?",
(tanlov,)).fetchone()[0]
with mlops:
mlops.execute("INSERT INTO versiyalar VALUES (?,?,?,?,?,?,?)",
(NOM, 1, "None", yid, yol.name, artefakt_xeshi,
json.dumps({"val_auc": metadata["val_auc"]})))
bosqich_ozgartir(mlops, 1, "Staging", "yangi nomzod")
darvoza = {"val_auc >= 0.70": metadata["val_auc"] >= 0.70,
"nazorat farqi <= 1e-9": farq <= 1e-9,
"sxema to'liq": set(paket["sxema"]["son"]) == set(SON)}
for q, ok in darvoza.items():
print(f" {q:<24} {'OK' if ok else 'RAD'}")
if all(darvoza.values()):
bosqich_ozgartir(mlops, 1, "Production", "darvoza OK")
try:
bosqich_ozgartir(mlops, 1, "Staging", "qo'lda")
except ValueError as e:
print(f" noto'g'ri o'tish rad etildi: {e}")
print(pd.read_sql("SELECT nom, versiya, bosqich, yurish_id, artefakt"
" FROM versiyalar", mlops).to_string(index=False))
print(pd.read_sql("SELECT versiya, eski, yangi, sabab FROM audit",
mlops).to_string(index=False))
mlops.close()
print(" ⭐ Qaror jurnalda, model registrda, paket o'zini tekshiradi")
if __name__ == "__main__":
main()Natijaning muhim qismi:
=== 1. Dizayn: vaqt bo'yicha bo'lish ===
o'quv: 1-5 oy (7500 qator), validatsiya: 6-oy (1500), ketish ulushi 0.199
ma'lumot xeshi (1-6 oy): 53c21818a727
=== 2. Nomzodlar va eksperiment jurnali ===
LogReg yurish d9576f065a: {'val_auc': 0.7263, 'val_logloss': 0.4615, 'kalibrovka': 0.0156}
HistGB yurish 90978cf8c6: {'val_auc': 0.7198, 'val_logloss': 0.467, 'kalibrovka': 0.017}
jurnalda 2 yurish (qayta yozilgan yurish takrorlanmadi - id = xesh)
=== 3. Juftlashgan bootstrap: HistGB - LogReg (6-oy) ===
AUC farqi -0.0066, bootstrap SE 0.0090, sezilarli: False
qoida: eng yaxshisidan sezilarli yomon bo'lmagan ENG SODDA -> LogReg
=== 4. Yakuniy model (1-6 oy), paket va nazorat ===
paket kalitlari: ['format', 'metadata', 'nazorat', 'pipeline', 'sxema']
metadata: {'nom': 'mijoz_ketishi', 'model': 'LogReg', 'oylar': [1, 6], 'malumot_xeshi': '53c21818a727', 'kod': 'churn-1.0.0', 'val_auc': 0.7263, 'reference_ketish': 0.1992}
qayta yuklangach nazorat namunalari maks farq: 0.00e+00
=== 5. Registr va darvoza ===
val_auc >= 0.70 OK
nazorat farqi <= 1e-9 OK
sxema to'liq OK
noto'g'ri o'tish rad etildi: ruxsat yo'q: Production -> Staging
nom versiya bosqich yurish_id artefakt
mijoz_ketishi 1 Production d9576f065a mijoz_ketishi-v1.joblib
versiya eski yangi sabab
1 None Staging yangi nomzod
1 Staging Production darvoza OK
⭐ Qaror jurnalda, model registrda, paket o'zini tekshiradiNatija tahlili.
1-bo'lim — bo'lish vaqt bo'yicha: 1-5 oy o'quv, 6-oy validatsiya. Bu "kelajak oyni bashorat qilish" sinovi — ishlab chiqarishdagi vaziyatning aynan o'zi. Ma'lumot xeshi keyingi hamma yozuvlarga (yurishlar, paket metadata, registr) bog'lanadi.
2-bo'lim — ikki nomzod: LogReg AUC 0.7263, HistGB 0.7198. Ikkala yurish jurnalga yozildi; yurish identifikatori — model, parametrlar, ma'lumot va kod versiyasining xeshi, shuning uchun xuddi shu yurishni qayta yozish jurnalda takror hosil qilmadi (2 yurish).
3-bo'lim — 6-oyda juftlashgan bootstrap: HistGB - LogReg = -0.0066, SE 0.0090 — sezilarli emas. Qoida bo'yicha eng sodda — LogReg. Bu ma'lumotdagi bog'liqliklar asosan chiziqli (logit bo'yicha), shuning uchun boosting ustunlik bermadi — bu halol natija, va u xizmatni soddalashtiradi (tezroq, tushuntirish oson).
4-bo'lim — yakuniy model 1-6 oyda qayta o'qitildi va paketlandi: Pipeline (tayyorlash + model bitta obyekt), sxema, metadata (oylar, ma'lumot xeshi, kod versiyasi, val AUC, reference ketish ulushi) va 200 ta nazorat namunasi. Qayta yuklangach nazorat namunalari aynan mos (0.00e+00).
5-bo'lim — registr. Versiya None -> Staging ga o'tdi, uchta darvoza qoidasi tekshirildi (val_auc >= 0.70, nazorat, sxema) va Staging -> Production. Production -> Staging kabi ruxsat etilmagan o'tish rad etildi. Audit jadvalida har o'tish sababi bilan yozilgan.
Misol 3 — FastAPI xizmati: validatsiya, API kaliti va so'rov loglari
"""3-qadam: FastAPI xizmati - /predict, /health, /version, validatsiya, API kaliti, loglar."""
import hmac
import json
import logging
import os
import secrets
import sqlite3
import sys
import tempfile
import warnings
from pathlib import Path
from typing import Literal
import joblib
import numpy as np
import pandas as pd
from fastapi import FastAPI, Header, HTTPException
from pydantic import BaseModel, ConfigDict, Field
from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
with warnings.catch_warnings(): # TestClient importi ogohlantirish
warnings.simplefilter("ignore") # chiqaradi (stderr ni toza tutamiz)
from fastapi.testclient import TestClient
SON = ["oylik_tolov", "muddat_oy", "qongiroqlar", "shikoyatlar"]
KAT = ["tarif", "hudud"]
USTUNLAR = ["mijoz_id", "oy"] + SON + KAT + ["ketdi"]
TARIFLAR = ["A", "B", "C"]
HUDUDLAR = ["Toshkent", "Samarqand", "Buxoro", "Namangan"]
DRIFT_OYI = 9
NOM = "mijoz_ketishi"
logging.basicConfig(stream=sys.stdout, level=logging.INFO,
format=" LOG %(levelname)s %(message)s")
log = logging.getLogger("xizmat")
logging.getLogger("httpx").setLevel(logging.WARNING) # TestClient shovqini
def oy_partiyasi(oy, n=1500):
"""Har oy - mijozlar kesimi; ketdi = keyingi oyda ketdimi (kechikib keladi)."""
rng = np.random.default_rng(1000 + oy)
d = oy >= DRIFT_OYI # raqobatchi arzon tarif chiqardi
tarif = rng.choice(TARIFLAR, n, p=[0.35, 0.30, 0.35] if d else [0.5, 0.3, 0.2])
hudud = rng.choice(HUDUDLAR, n, p=[0.4, 0.25, 0.2, 0.15])
tolov = rng.lognormal(np.log(120), 0.35, n) * (1.15 if d else 1.0)
muddat = rng.gamma(2.0, 12.0, n)
qongiroq = rng.poisson(25, n)
shikoyat = rng.poisson(0.8 if d else 0.5, n)
z = (np.log(tolov) - np.log(120)) / 0.35
f = (-1.8 + 1.0 * (muddat < 4) - 0.015 * muddat
+ 0.3 * (hudud == "Namangan")
+ (1.0 * (tarif == "A") - 0.3 * (tarif == "C") if d
else 1.0 * (tarif == "C"))
+ (0.9 if d else 0.5) * shikoyat + (0.3 if d else 0.7) * z)
ketdi = (rng.random(n) < 1 / (1 + np.exp(-f))).astype(int)
return pd.DataFrame({
"mijoz_id": [f"M{oy:02d}{i:05d}" for i in range(n)], "oy": oy,
"oylik_tolov": tolov.round(2), "muddat_oy": muddat.round(1),
"qongiroqlar": qongiroq, "shikoyatlar": shikoyat,
"tarif": tarif, "hudud": hudud, "ketdi": ketdi})
# ---------------- 2-qadam natijasi: paket va registr ----------------
def logreg():
return Pipeline([
("tayyor", ColumnTransformer([
("son", StandardScaler(), SON),
("kat", OneHotEncoder(handle_unknown="ignore"), KAT)])),
("model", LogisticRegression(max_iter=2000))])
def tayyorla(papka):
df = pd.concat([oy_partiyasi(o) for o in range(1, 7)])
pipe = logreg().fit(df, df.ketdi)
nazorat = df.iloc[:200]
paket = {"format": 1, "pipeline": pipe,
"sxema": {"son": SON, "kat": KAT},
"metadata": {"nom": NOM, "model": "LogReg", "oylar": [1, 6]},
"nazorat": {"X": nazorat[SON + KAT].to_dict("list"),
"p": pipe.predict_proba(nazorat)[:, 1].tolist()}}
joblib.dump(paket, papka / f"{NOM}-v1.joblib")
db = sqlite3.connect(papka / "mlops.sqlite")
db.execute("CREATE TABLE versiyalar (nom TEXT, versiya INT, bosqich TEXT,"
" artefakt TEXT, PRIMARY KEY (nom, versiya))")
with db:
db.execute("INSERT INTO versiyalar VALUES (?,?,?,?)",
(NOM, 1, "Production", f"{NOM}-v1.joblib"))
db.close()
# ---------------- xizmat ----------------
class Mijoz(BaseModel):
model_config = ConfigDict(extra="forbid")
mijoz_id: str = Field(min_length=1, max_length=32)
oylik_tolov: float = Field(gt=0, lt=2000)
muddat_oy: float = Field(ge=0, le=600)
qongiroqlar: int = Field(ge=0, le=10_000)
shikoyatlar: int = Field(ge=0, le=100)
tarif: Literal["A", "B", "C"]
hudud: Literal["Toshkent", "Samarqand", "Buxoro", "Namangan"]
class Sorov(BaseModel):
mijozlar: list[Mijoz] = Field(min_length=1, max_length=5000)
class Xizmat:
"""Registrdagi Production versiyani yuklaydi; so'rovlarni loglaydi."""
def __init__(self, papka):
self.papka = papka
self.soat = 0 # virtual soat
self.log_db = sqlite3.connect(papka / "sorovlar.sqlite",
check_same_thread=False)
self.log_db.execute("CREATE TABLE IF NOT EXISTS sorovlar (id INTEGER"
" PRIMARY KEY AUTOINCREMENT, soat INT, versiya INT,"
" mijoz_id TEXT, kirish TEXT, p REAL)")
self.yangila()
def yangila(self):
db = sqlite3.connect(self.papka / "mlops.sqlite")
versiya, artefakt = db.execute(
"SELECT versiya, artefakt FROM versiyalar WHERE nom=? AND"
" bosqich='Production'", (NOM,)).fetchone()
db.close()
paket = joblib.load(self.papka / artefakt)
X = pd.DataFrame(paket["nazorat"]["X"])
farq = np.max(np.abs(paket["pipeline"].predict_proba(X)[:, 1]
- np.array(paket["nazorat"]["p"])))
if farq > 1e-9:
raise RuntimeError("nazorat namunalari mos emas")
self.versiya, self.paket = versiya, paket
log.info("yuklandi: %s v%d (nazorat OK)", NOM, versiya)
def bashorat(self, mijozlar):
df = pd.DataFrame([m.model_dump() for m in mijozlar])
p = self.paket["pipeline"].predict_proba(df)[:, 1]
self.soat += 1
with self.log_db:
self.log_db.executemany(
"INSERT INTO sorovlar (soat, versiya, mijoz_id, kirish, p)"
" VALUES (?,?,?,?,?)",
[(self.soat, self.versiya, m.mijoz_id,
json.dumps(m.model_dump(), sort_keys=True), float(q))
for m, q in zip(mijozlar, p)])
return p
def niqobla(kalit):
return "*" * 8 if kalit else "(yo'q)"
def ilova_yarat(xizmat):
kalit = os.environ.get("CHURN_API_KEY")
if not kalit: # 12-factor: sir env dan
raise RuntimeError("CHURN_API_KEY o'rnatilmagan")
log.info("API kaliti: %s (uzunligi %d)", niqobla(kalit), len(kalit))
app = FastAPI(title="Mijoz ketishi xizmati")
@app.get("/health")
def health():
return {"holat": "ok", "model_yuklangan": xizmat.paket is not None}
@app.get("/version")
def version():
m = xizmat.paket["metadata"]
return {"nom": NOM, "versiya": xizmat.versiya,
"algoritm": m["model"], "oylar": m["oylar"]}
@app.post("/predict")
def predict(sorov: Sorov, x_api_key: str | None = Header(default=None)):
if not hmac.compare_digest((x_api_key or "").encode(), kalit.encode()):
log.warning("rad etildi: kalit %s", niqobla(x_api_key))
raise HTTPException(status_code=401, detail="kalit noto'g'ri")
p = xizmat.bashorat(sorov.mijozlar)
return {"versiya": xizmat.versiya,
"bashoratlar": [{"mijoz_id": m.mijoz_id,
"ketish_ehtimoli": round(float(q), 4)}
for m, q in zip(sorov.mijozlar, p)]}
return app
def main() -> None:
warnings.simplefilter("ignore")
with tempfile.TemporaryDirectory() as tmp:
papka = Path(tmp)
tayyorla(papka)
print("=== 1. Kalitsiz ishga tushirish ===")
os.environ.pop("CHURN_API_KEY", None)
xizmat = Xizmat(papka)
try:
ilova_yarat(xizmat)
except RuntimeError as e:
print(f" rad etildi: {e}")
print("\n=== 2. Kalit muhit o'zgaruvchisidan ===")
os.environ["CHURN_API_KEY"] = secrets.token_hex(16) # sinov uchun
kalit = os.environ["CHURN_API_KEY"]
mijoz = TestClient(ilova_yarat(xizmat))
print(f" /health -> {mijoz.get('/health').json()}")
print(f" /version -> {mijoz.get('/version').json()}")
print("\n=== 3. /predict ===")
df = oy_partiyasi(7).iloc[:3]
yuk = {"mijozlar": df[["mijoz_id"] + SON + KAT].to_dict("records")}
r = mijoz.post("/predict", json=yuk)
print(f" kalitsiz: {r.status_code} {r.json()}")
r = mijoz.post("/predict", json=yuk, headers={"X-API-Key": "xato"})
print(f" noto'g'ri kalit: {r.status_code}")
r = mijoz.post("/predict", json=yuk, headers={"X-API-Key": kalit})
print(f" to'g'ri kalit: {r.status_code}")
for b in r.json()["bashoratlar"]:
print(f" {b}")
print("\n=== 4. Validatsiya xatolari (422) ===")
buzuq = [("manfiy to'lov", {"oylik_tolov": -5.0}),
("noma'lum tarif", {"tarif": "Z"}),
("ortiqcha maydon", {"yosh": 30}),
("matn o'rniga son", {"shikoyatlar": "ko'p"})]
for nom, ozgarish in buzuq:
m = dict(yuk["mijozlar"][0], **ozgarish)
r = mijoz.post("/predict", json={"mijozlar": [m]},
headers={"X-API-Key": kalit})
x = r.json()["detail"][0]
print(f" {nom:<17} {r.status_code} {x['loc'][-1]}: {x['type']}")
r = mijoz.post("/predict", json={"mijozlar": []},
headers={"X-API-Key": kalit})
print(f" {'bo_sh ro_yxat':<17} {r.status_code}")
print("\n=== 5. Partiya va so'rov loglari ===")
katta = oy_partiyasi(7).iloc[:500]
r = mijoz.post("/predict", json={"mijozlar": katta[
["mijoz_id"] + SON + KAT].to_dict("records")},
headers={"X-API-Key": kalit})
p = np.array([b["ketish_ehtimoli"] for b in r.json()["bashoratlar"]])
print(f" 500 mijoz: o'rtacha ehtimol {p.mean():.4f}, "
f"> 0.5: {int((p > 0.5).sum())}")
jad = pd.read_sql("SELECT soat, versiya, COUNT(*) AS n, AVG(p) AS p"
" FROM sorovlar GROUP BY soat", xizmat.log_db)
print(jad.round(4).to_string(index=False))
xizmat.log_db.close()
del os.environ["CHURN_API_KEY"]
print(" ⭐ Kalit env dan, loglarda niqoblangan; har so'rov versiya bilan")
if __name__ == "__main__":
main()Natijaning muhim qismi:
=== 1. Kalitsiz ishga tushirish ===
LOG INFO yuklandi: mijoz_ketishi v1 (nazorat OK)
rad etildi: CHURN_API_KEY o'rnatilmagan
=== 2. Kalit muhit o'zgaruvchisidan ===
LOG INFO API kaliti: ******** (uzunligi 32)
/health -> {'holat': 'ok', 'model_yuklangan': True}
/version -> {'nom': 'mijoz_ketishi', 'versiya': 1, 'algoritm': 'LogReg', 'oylar': [1, 6]}
=== 3. /predict ===
LOG WARNING rad etildi: kalit (yo'q)
kalitsiz: 401 {'detail': "kalit noto'g'ri"}
LOG WARNING rad etildi: kalit ********
noto'g'ri kalit: 401
to'g'ri kalit: 200
{'mijoz_id': 'M0700000', 'ketish_ehtimoli': 0.0856}
{'mijoz_id': 'M0700001', 'ketish_ehtimoli': 0.2437}
{'mijoz_id': 'M0700002', 'ketish_ehtimoli': 0.0565}
=== 4. Validatsiya xatolari (422) ===
manfiy to'lov 422 oylik_tolov: greater_than
noma'lum tarif 422 tarif: literal_error
ortiqcha maydon 422 yosh: extra_forbidden
matn o'rniga son 422 shikoyatlar: int_parsing
bo_sh ro_yxat 422
=== 5. Partiya va so'rov loglari ===
500 mijoz: o'rtacha ehtimol 0.2023, > 0.5: 29
soat versiya n p
1 1 3 0.1286
2 1 500 0.2023
⭐ Kalit env dan, loglarda niqoblangan; har so'rov versiya bilanNatija tahlili.
fastapi.testclient ni import qilish Starlette ning eskirganlik haqidagi ogohlantirishini stderr ga chiqaradi — shuning uchun import warnings.catch_warnings() ichida. httpx ning har so'rov haqidagi INFO loglari ham WARNING darajasiga tushirilgan, aks holda natija shovqinga to'lib ketadi.
1-bo'lim — CHURN_API_KEY o'rnatilmagan: model yuklandi (nazorat namunalari OK), lekin ilova yaratishdan bosh tortdi. Kalitsiz xizmatni ishga tushirishdan ko'ra, umuman ishga tushirmaslik xavfsizroq.
2-bo'lim — kalit muhit o'zgaruvchisidan olindi (sinovda secrets.token_hex(16) bilan yaratilgan, hech qayerga chop etilmagan). Logda faqat ******** va uzunligi. /version qaysi model javob berayotganini aytadi — monitoring va hodisalarni tekshirishda birinchi savol shu.
3-bo'lim — kalitsiz va noto'g'ri kalit bilan 401, to'g'ri kalit bilan 200 va har mijozga ehtimol.
4-bo'lim — Pydantic sxemasi to'rt xil buzuq kirishni 422 bilan rad etdi va har biri uchun qaysi maydon va qanday xato ekanini qaytardi: manfiy to'lov (greater_than), noma'lum tarif (literal_error), ortiqcha maydon (extra_forbidden — extra="forbid" tufayli), noto'g'ri tur (int_parsing). Bo'sh ro'yxat ham 422. Bu tekshiruvlar model ichiga "chiqindi" kirishining oldini oladi.
5-bo'lim — 500 mijozlik partiya. So'rov logida ikki yozuv guruhi: 3 ta va 500 ta mijoz, har biri versiya bilan. Aynan shu log 4-misolda belgilar bilan birlashtirilib, haqiqiy sifatni o'lchashga xizmat qiladi.
Misol 4 — Hayot sikli: drift, trigger, shadow, darvoza va versiya almashuvi
"""4-qadam: monitoring, drift, kechikkan belgilar, trigger, shadow, darvoza, versiya almashuvi."""
import json
import logging
import os
import secrets
import sqlite3
import sys
import tempfile
import warnings
from pathlib import Path
from typing import Literal
import joblib
import numpy as np
import pandas as pd
from fastapi import FastAPI, Header, HTTPException
from pydantic import BaseModel, ConfigDict, Field
from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
with warnings.catch_warnings():
warnings.simplefilter("ignore")
from fastapi.testclient import TestClient
SON = ["oylik_tolov", "muddat_oy", "qongiroqlar", "shikoyatlar"]
KAT = ["tarif", "hudud"]
USTUNLAR = ["mijoz_id", "oy"] + SON + KAT + ["ketdi"]
TARIFLAR = ["A", "B", "C"]
HUDUDLAR = ["Toshkent", "Samarqand", "Buxoro", "Namangan"]
DRIFT_OYI = 9
NOM = "mijoz_ketishi"
logging.basicConfig(stream=sys.stdout, level=logging.WARNING,
format=" LOG %(levelname)s %(message)s")
# ---------------- QAROR QOIDALARI (oldindan, natijani ko'rishdan oldin) ----------------
TRIGGER_AUC_TUSHISHI = 0.03 # AUC(oy) < reference - 0.03, 2 oy ketma-ket
PSI_OGOHLANTIRISH = 0.10 # erta ogohlantirish (qaror emas)
OYNA = 2 # challenger: oxirgi 2 belgili oy (27.13)
DARVOZA_SEGMENT = 0.02 # har tarifda log loss yomonlashuvi <= 0.02
def oy_partiyasi(oy, n=1500):
"""Har oy - mijozlar kesimi; ketdi = keyingi oyda ketdimi (kechikib keladi)."""
rng = np.random.default_rng(1000 + oy)
d = oy >= DRIFT_OYI # raqobatchi arzon tarif chiqardi
tarif = rng.choice(TARIFLAR, n, p=[0.35, 0.30, 0.35] if d else [0.5, 0.3, 0.2])
hudud = rng.choice(HUDUDLAR, n, p=[0.4, 0.25, 0.2, 0.15])
tolov = rng.lognormal(np.log(120), 0.35, n) * (1.15 if d else 1.0)
muddat = rng.gamma(2.0, 12.0, n)
qongiroq = rng.poisson(25, n)
shikoyat = rng.poisson(0.8 if d else 0.5, n)
z = (np.log(tolov) - np.log(120)) / 0.35
f = (-1.8 + 1.0 * (muddat < 4) - 0.015 * muddat
+ 0.3 * (hudud == "Namangan")
+ (1.0 * (tarif == "A") - 0.3 * (tarif == "C") if d
else 1.0 * (tarif == "C"))
+ (0.9 if d else 0.5) * shikoyat + (0.3 if d else 0.7) * z)
ketdi = (rng.random(n) < 1 / (1 + np.exp(-f))).astype(int)
return pd.DataFrame({
"mijoz_id": [f"M{oy:02d}{i:05d}" for i in range(n)], "oy": oy,
"oylik_tolov": tolov.round(2), "muddat_oy": muddat.round(1),
"qongiroqlar": qongiroq, "shikoyatlar": shikoyat,
"tarif": tarif, "hudud": hudud, "ketdi": ketdi})
# ---------------- model, profil, paket ----------------
def logreg():
return Pipeline([
("tayyor", ColumnTransformer([
("son", StandardScaler(), SON),
("kat", OneHotEncoder(handle_unknown="ignore"), KAT)])),
("model", LogisticRegression(max_iter=2000))])
def profil(x):
ichki = np.unique(np.quantile(x, np.linspace(0, 1, 11)[1:-1]))
ulush = np.bincount(np.searchsorted(ichki, x, side="right"),
minlength=len(ichki) + 1) / len(x)
return {"ichki": ichki.tolist(), "ulush": ulush.tolist()}
def psi(prof, x, eps=1e-4):
ichki = np.array(prof["ichki"])
q = np.bincount(np.searchsorted(ichki, x, side="right"),
minlength=len(ichki) + 1) / len(x)
p = np.clip(np.array(prof["ulush"]), eps, None)
q = np.clip(q, eps, None)
return float(np.sum((q - p) * np.log(q / p)))
def psi_kat(ref, cur, eps=1e-4):
p = np.clip([ref.get(t, 0) for t in TARIFLAR], eps, None)
q = np.clip([np.mean(cur == t) for t in TARIFLAR], eps, None)
return float(np.sum((q - p) * np.log(q / p)))
def paket_saqla(papka, versiya, df, oylar, ref_auc):
pipe = logreg().fit(df, df.ketdi)
p = pipe.predict_proba(df)[:, 1]
nazorat = df.iloc[:200]
paket = {"format": 1, "pipeline": pipe, "sxema": {"son": SON, "kat": KAT},
"metadata": {"nom": NOM, "versiya": versiya, "oylar": oylar,
"ref_auc": ref_auc},
"profil": {"oylik_tolov": profil(df.oylik_tolov.to_numpy()),
"shikoyatlar": profil(df.shikoyatlar.to_numpy()),
"tarif": {t: float(np.mean(df.tarif == t))
for t in TARIFLAR},
"p": profil(p)},
"nazorat": {"X": nazorat[SON + KAT].to_dict("list"),
"p": pipe.predict_proba(nazorat)[:, 1].tolist()}}
artefakt = f"{NOM}-v{versiya}.joblib"
joblib.dump(paket, papka / artefakt)
return artefakt
# ---------------- registr ----------------
RUXSAT = {("None", "Staging"), ("Staging", "Production"),
("Production", "Archived"), ("Staging", "Archived"),
("Archived", "Production")} # rollback
class Registr:
def __init__(self, yol):
self.db = sqlite3.connect(yol, check_same_thread=False)
self.db.executescript(
"CREATE TABLE versiyalar (nom TEXT, versiya INT, bosqich TEXT,"
" artefakt TEXT, PRIMARY KEY (nom, versiya));"
"CREATE TABLE audit (id INTEGER PRIMARY KEY AUTOINCREMENT,"
" oy INT, versiya INT, eski TEXT, yangi TEXT, sabab TEXT);")
def qosh(self, versiya, artefakt):
with self.db:
self.db.execute("INSERT INTO versiyalar VALUES (?,?,?,?)",
(NOM, versiya, "None", artefakt))
def otkaz(self, oy, versiya, yangi, sabab):
eski = self.db.execute("SELECT bosqich FROM versiyalar WHERE"
" versiya=?", (versiya,)).fetchone()[0]
if (eski, yangi) not in RUXSAT:
raise ValueError(f"{eski} -> {yangi}")
with self.db:
self.db.execute("UPDATE versiyalar SET bosqich=? WHERE versiya=?",
(yangi, versiya))
self.db.execute("INSERT INTO audit (oy, versiya, eski, yangi,"
" sabab) VALUES (?,?,?,?,?)",
(oy, versiya, eski, yangi, sabab))
def bosqichda(self, bosqich):
r = self.db.execute("SELECT versiya, artefakt FROM versiyalar WHERE"
" bosqich=?", (bosqich,)).fetchone()
return r
# ---------------- xizmat (3-qadam + shadow) ----------------
class Mijoz(BaseModel):
model_config = ConfigDict(extra="forbid")
mijoz_id: str = Field(min_length=1, max_length=32)
oylik_tolov: float = Field(gt=0, lt=2000)
muddat_oy: float = Field(ge=0, le=600)
qongiroqlar: int = Field(ge=0, le=10_000)
shikoyatlar: int = Field(ge=0, le=100)
tarif: Literal["A", "B", "C"]
hudud: Literal["Toshkent", "Samarqand", "Buxoro", "Namangan"]
class Sorov(BaseModel):
oy: int = Field(ge=1, le=99)
mijozlar: list[Mijoz] = Field(min_length=1, max_length=5000)
def paket_yukla(papka, artefakt):
paket = joblib.load(papka / artefakt)
X = pd.DataFrame(paket["nazorat"]["X"])
if np.max(np.abs(paket["pipeline"].predict_proba(X)[:, 1]
- np.array(paket["nazorat"]["p"]))) > 1e-9:
raise RuntimeError("nazorat namunalari mos emas")
return paket
class Xizmat:
def __init__(self, papka, registr):
self.papka, self.registr = papka, registr
self.db = sqlite3.connect(papka / "sorovlar.sqlite",
check_same_thread=False)
self.db.execute("CREATE TABLE sorovlar (oy INT, mijoz_id TEXT,"
" versiya INT, p REAL, shadow_versiya INT,"
" shadow_p REAL)")
self.yangila()
def yangila(self):
v, a = self.registr.bosqichda("Production")
self.versiya, self.paket = v, paket_yukla(self.papka, a)
s = self.registr.bosqichda("Staging")
self.shadow = (s[0], paket_yukla(self.papka, s[1])) if s else None
def bashorat(self, oy, mijozlar):
df = pd.DataFrame([m.model_dump() for m in mijozlar])
p = self.paket["pipeline"].predict_proba(df)[:, 1]
sv, sp = None, [None] * len(df)
if self.shadow: # javob foydalanuvchiga
try: # bormaydi, faqat log
sv = self.shadow[0]
sp = self.shadow[1]["pipeline"].predict_proba(df)[:, 1].tolist()
except Exception: # shadow xatosi xizmatni
sv, sp = None, [None] * len(df) # to'xtatmaydi
with self.db:
self.db.executemany("INSERT INTO sorovlar VALUES (?,?,?,?,?,?)",
zip([oy] * len(df), df.mijoz_id,
[self.versiya] * len(df), p.tolist(),
[sv] * len(df), sp))
return p
def ilova_yarat(xizmat):
kalit = os.environ["CHURN_API_KEY"]
app = FastAPI()
def tekshir(k):
if not secrets.compare_digest((k or "").encode(), kalit.encode()):
raise HTTPException(status_code=401)
@app.get("/version")
def version():
return {"nom": NOM, "versiya": xizmat.versiya,
"shadow": xizmat.shadow[0] if xizmat.shadow else None}
@app.post("/predict")
def predict(sorov: Sorov, x_api_key: str | None = Header(default=None)):
tekshir(x_api_key)
p = xizmat.bashorat(sorov.oy, sorov.mijozlar)
return {"versiya": xizmat.versiya, "p": [round(float(q), 4) for q in p]}
@app.post("/admin/yangila")
def yangila(x_api_key: str | None = Header(default=None)):
tekshir(x_api_key)
xizmat.yangila()
return {"versiya": xizmat.versiya}
return app
def ll(y, p):
p = np.clip(p, 1e-6, 1 - 1e-6)
return -(y * np.log(p) + (1 - y) * np.log(1 - p))
def darvoza_tekshir(j):
"""Shadow oyi: bir xil so'rovlarda champion (p) va challenger (shadow_p)."""
y = j.ketdi.to_numpy()
a, b = j.p.to_numpy(), j.shadow_p.to_numpy(dtype=float)
rng = np.random.default_rng(0)
farq = roc_auc_score(y, b) - roc_auc_score(y, a)
bs = []
for _ in range(300):
i = rng.integers(0, len(y), len(y))
bs.append(roc_auc_score(y[i], b[i]) - roc_auc_score(y[i], a[i]))
se = float(np.std(bs, ddof=1))
d = ll(y, b) - ll(y, a)
se_d = d.std(ddof=1) / np.sqrt(len(d))
seg = {t: float(d[j.tarif.to_numpy() == t].mean()) for t in TARIFLAR}
return {
"AUC sezilarli yaxshi (> 2*SE)": (farq > 2 * se,
f"{farq:+.4f}, SE {se:.4f}"),
"log loss sezilarli yaxshi": (d.mean() < -2 * se_d,
f"{d.mean():+.4f}, SE {se_d:.4f}"),
"har tarifda yomonlashuv <= 0.02": (
max(seg.values()) <= DARVOZA_SEGMENT,
str({k: round(v, 4) for k, v in seg.items()})),
"shadow xatolari = 0": (bool(np.isfinite(b).all()),
f"{int((~np.isfinite(b)).sum())} ta"),
}
def main() -> None:
warnings.simplefilter("ignore")
os.environ["CHURN_API_KEY"] = secrets.token_hex(16) # sinov kaliti
H = {"X-API-Key": os.environ["CHURN_API_KEY"]}
with tempfile.TemporaryDirectory() as tmp:
papka = Path(tmp)
oylar = {o: oy_partiyasi(o) for o in range(1, 15)} # manba (1-qadam)
registr = Registr(papka / "mlops.sqlite")
tr = pd.concat([oylar[o] for o in range(1, 6)])
ref_auc = roc_auc_score(oylar[6].ketdi, logreg().fit(
tr, tr.ketdi).predict_proba(oylar[6])[:, 1])
v1 = pd.concat([oylar[o] for o in range(1, 7)]) # 2-qadam
registr.qosh(1, paket_saqla(papka, 1, v1, [1, 6], ref_auc))
registr.otkaz(6, 1, "Staging", "yangi nomzod")
registr.otkaz(6, 1, "Production", "darvoza OK (2-qadam)")
xizmat = Xizmat(papka, registr)
mijoz = TestClient(ilova_yarat(xizmat)) # 3-qadam
print("=== 1. Boshlang'ich holat ===")
print(f" /version -> {mijoz.get('/version').json()}")
print(f" reference AUC (6-oy): {ref_auc:.4f}; trigger: AUC < "
f"{ref_auc - TRIGGER_AUC_TUSHISHI:.4f} 2 oy ketma-ket")
print("\n=== 2. Oylar: oy boshida (oy-1) belgilari, keyin so'rovlar ===")
print(f" {'oy':>3} {'belgi':>5} {'AUC':>6} {'xizmat':>6} "
f"{'PSI tolov':>9} {'PSI tarif':>9} {'PSI p':>6} harakat")
past_ketma_ket, oy_auc, darvoza = 0, {}, None
for oy in range(7, 16):
harakat, auc_txt = [], "-"
# A) (oy-1) belgilari keldi -> xizmat bergan bashoratlar bilan
log = pd.read_sql("SELECT mijoz_id, p, shadow_p FROM sorovlar"
" WHERE oy = ?", xizmat.db, params=(oy - 1,))
if len(log):
j = log.merge(oylar[oy - 1][["mijoz_id", "ketdi", "tarif"]],
on="mijoz_id")
oy_auc[oy - 1] = roc_auc_score(j.ketdi, j.p)
auc_txt = f"{oy_auc[oy - 1]:.4f}"
past = oy_auc[oy - 1] < ref_auc - TRIGGER_AUC_TUSHISHI
past_ketma_ket = past_ketma_ket + 1 if past else 0
if j.shadow_p.notna().all() and xizmat.shadow:
darvoza = darvoza_tekshir(j)
ok = all(v for v, _ in darvoza.values())
harakat.append(f"darvoza {'OK' if ok else 'RAD'}")
if ok:
registr.otkaz(oy, 1, "Archived", "v2 ga almashdi")
registr.otkaz(oy, 2, "Production", "darvoza OK")
mijoz.post("/admin/yangila", headers=H)
harakat.append("v2 -> Production")
elif past_ketma_ket >= 2 and xizmat.shadow is None \
and xizmat.versiya == 1:
oyna = list(range(oy - OYNA, oy))
yangi = pd.concat([oylar[o] for o in oyna])
registr.qosh(2, paket_saqla(papka, 2, yangi,
[oyna[0], oyna[-1]],
oy_auc[oy - 1]))
registr.otkaz(oy, 2, "Staging", "trigger: AUC 2 oy past")
mijoz.post("/admin/yangila", headers=H)
harakat.append(f"TRIGGER: v2 ({oyna[0]}-{oyna[-1]} oy)"
f" -> shadow")
# B) shu oy so'rovlari (15-oy so'rovlari bu simulyatsiyada yo'q)
if oy <= 14:
prof = xizmat.paket["profil"] # reference = JORIY model
df = oylar[oy]
r = mijoz.post("/predict", headers=H, json={
"oy": oy,
"mijozlar": df[["mijoz_id"] + SON + KAT].to_dict("records")})
p = np.array(r.json()["p"])
psi_t = psi(prof["oylik_tolov"], df.oylik_tolov.to_numpy())
psi_k = psi_kat(prof["tarif"], df.tarif.to_numpy())
psi_p = psi(prof["p"], p)
if max(psi_t, psi_k, psi_p) > PSI_OGOHLANTIRISH:
harakat.insert(0, "drift ogohl.")
ustunlar = (f"{'v' + str(r.json()['versiya']):>6} {psi_t:>9.3f}"
f" {psi_k:>9.3f} {psi_p:>6.3f}")
else:
ustunlar = f"{'-':>6} {'-':>9} {'-':>9} {'-':>6}"
print(f" {oy:>3} {oy - 1:>5} {auc_txt:>6} {ustunlar} "
f"{'; '.join(harakat)}")
print("\n=== 3. Darvoza tafsiloti (shadow oyi, bir xil so'rovlar) ===")
for q, (ok, izoh) in darvoza.items():
print(f" {q:<32} {'OK' if ok else 'RAD'} {izoh}")
print("\n=== 4. Almashtirish foydasi: v2 xizmat qilgan oylar ===")
v1_pipe = paket_yukla(papka, f"{NOM}-v1.joblib")["pipeline"]
for oy in [12, 13, 14]:
v1_auc = roc_auc_score(oylar[oy].ketdi,
v1_pipe.predict_proba(oylar[oy])[:, 1])
print(f" {oy}-oy AUC: xizmat (v2) {oy_auc[oy]:.4f}, "
f"agar v1 qolganida {v1_auc:.4f}")
print("\n=== 5. Rollback imkoniyati ===")
registr.otkaz(15, 2, "Archived", "rollback sinovi")
registr.otkaz(15, 1, "Production", "rollback sinovi")
mijoz.post("/admin/yangila", headers=H)
print(f" rollback -> /version {mijoz.get('/version').json()}")
registr.otkaz(15, 1, "Archived", "sinov tugadi")
registr.otkaz(15, 2, "Production", "sinov tugadi")
mijoz.post("/admin/yangila", headers=H)
print(f" qaytarildi -> /version {mijoz.get('/version').json()}")
print(pd.read_sql("SELECT oy, versiya, eski, yangi, sabab FROM audit",
registr.db).to_string(index=False))
xizmat.db.close()
registr.db.close()
del os.environ["CHURN_API_KEY"]
print(" ⭐ Qarorlar oldindan yozilgan qoidalar bilan, har qadam auditda")
if __name__ == "__main__":
main()Natijaning muhim qismi:
=== 1. Boshlang'ich holat ===
/version -> {'nom': 'mijoz_ketishi', 'versiya': 1, 'shadow': None}
reference AUC (6-oy): 0.7263; trigger: AUC < 0.6963 2 oy ketma-ket
=== 2. Oylar: oy boshida (oy-1) belgilari, keyin so'rovlar ===
oy belgi AUC xizmat PSI tolov PSI tarif PSI p harakat
7 6 - v1 0.005 0.002 0.004
8 7 0.7230 v1 0.004 0.002 0.005
9 8 0.7300 v1 0.145 0.131 0.270 drift ogohl.
10 9 0.6285 v1 0.171 0.173 0.363 drift ogohl.
11 10 0.6212 v1 0.178 0.142 0.331 drift ogohl.; TRIGGER: v2 (9-10 oy) -> shadow
12 11 0.6291 v2 0.013 0.001 0.012 darvoza OK; v2 -> Production
13 12 0.7779 v2 0.014 0.002 0.011
14 13 0.7791 v2 0.008 0.006 0.011
15 14 0.7436 - - - -
=== 3. Darvoza tafsiloti (shadow oyi, bir xil so'rovlar) ===
AUC sezilarli yaxshi (> 2*SE) OK +0.1395, SE 0.0164
log loss sezilarli yaxshi OK -0.1097, SE 0.0125
har tarifda yomonlashuv <= 0.02 OK {'A': -0.1483, 'B': -0.0408, 'C': -0.1336}
shadow xatolari = 0 OK 0 ta
=== 4. Almashtirish foydasi: v2 xizmat qilgan oylar ===
12-oy AUC: xizmat (v2) 0.7779, agar v1 qolganida 0.6262
13-oy AUC: xizmat (v2) 0.7791, agar v1 qolganida 0.6223
14-oy AUC: xizmat (v2) 0.7436, agar v1 qolganida 0.6332
=== 5. Rollback imkoniyati ===
rollback -> /version {'nom': 'mijoz_ketishi', 'versiya': 1, 'shadow': None}
qaytarildi -> /version {'nom': 'mijoz_ketishi', 'versiya': 2, 'shadow': None}
oy versiya eski yangi sabab
6 1 None Staging yangi nomzod
6 1 Staging Production darvoza OK (2-qadam)
11 2 None Staging trigger: AUC 2 oy past
12 1 Production Archived v2 ga almashdi
12 2 Staging Production darvoza OK
15 2 Production Archived rollback sinovi
15 1 Archived Production rollback sinovi
15 1 Production Archived sinov tugadi
15 2 Archived Production sinov tugadi
⭐ Qarorlar oldindan yozilgan qoidalar bilan, har qadam auditdaNatija tahlili.
Qaror qoidalari fayl boshida, natijadan oldin yozilgan: PSI ogohlantirishi 0.10, trigger — AUC reference dan 0.03 pastda ikki oy ketma-ket, challenger oynasi — oxirgi 2 belgili oy, segment chegarasi 0.02.
2-bo'lim — oyma-oy jadval. Har oy boshida oldingi oy belgilari keladi va xizmat haqiqatan bergan bashoratlar bilan birlashtiriladi, keyin shu oy so'rovlari keladi.
- 7-8 oy — hammasi tinch: PSI
0.002-0.005, AUC0.7230va0.7300(reference0.7263). - 9-oy — raqobatchi tarifi chiqdi. Belgilar hali yo'q, lekin erta ogohlantirish darhol yondi: to'lov PSI
0.145, tarif0.131, bashorat PSI0.270. Bu ogohlantirish, qaror emas — 27.12 dagi kabi kirish drifti har doim ham sifatni tushirmaydi. - 10-oy — 9-oy belgilari keldi: AUC
0.6285— chegaradan (0.6963) past. Birinchi past oy. - 11-oy — AUC
0.6212, ikkinchi past oy → trigger. Challenger (v2) 9-10 oylarda o'qitildi,Stagingga o'tdi, xizmat uni shadow rejimda yukladi. 11-oy so'rovlariga javobni hali ham v1 beradi, v2 ning bashoratlari faqat logga yoziladi. - 12-oy — 11-oy belgilari keldi: v1 AUC
0.6291. Bir xil so'rovlarda v2 bilan taqqoslash — darvoza OK. v2Productionga, v1Archivedga, xizmat/admin/yangilabilan yangi versiyani yukladi: 12-oy so'rovlariga v2 javob beryapti. Diqqat: PSI birdan0.001-0.013ga tushdi — reference endi v2 ning o'quv taqsimoti (9-10 oy), ya'ni "yangi normal". - 13-15 oy boshlari — v2 xizmat qilgan 12, 13 va 14-oylar belgilari keldi: AUC
0.7779,0.7791,0.7436— reference dan ham yuqori, chunki yangi rejimda shikoyat va tarif signali kuchliroq.
3-bo'lim — darvoza tafsiloti (11-oy, bir xil 1500 so'rov): AUC farqi +0.1395 (SE 0.0164), log loss -0.1097 (SE 0.0125), uchala tarifda ham yaxshilanish, shadow xatolari yo'q. Eng katta yaxshilanish A tarifida (-0.1483) — aynan u yerda dunyo o'zgargan edi.
4-bo'lim — foyda halol o'lchandi: agar v1 qolganida 12-14 oylarda AUC 0.62-0.63 bo'lardi; v2 0.74-0.78 berdi.
5-bo'lim — rollback: ikki registr o'tishi va /admin/yangila — /version darhol 1 ni ko'rsatdi; sinovdan keyin v2 qaytarildi. Audit jadvalida butun tarix bor: kim, qachon (oy), qaysi versiya, qaysi bosqichdan qaysi bosqichga va nima sababdan.
Halol cheklovlar: ma'lumot sintetik va drift keskin (to'satdan); haqiqiy hayotda drift aralash va shovqinli, trigger chegaralari tarixiy ma'lumotda kalibrlanadi 27.12-bob. Kechikish zanjiri tufayli drift boshlanganidan yangi model ishga tushguncha 3 oy o'tdi (9 → 12) — bu tizimning xatosi emas, kechikkan belgilarning tabiiy narxi.
5. To'g'ri va noto'g'ri tushunishlar
| Noto'g'ri fikr | To'g'risi |
|---|---|
| "Sxema to'g'ri bo'lsa — ma'lumot to'g'ri" | Birlik xatosini faqat diapazon ushlaydi |
| "Buzuq partiyani o'chirib yuboramiz" | Karantin: sabab bilan saqlanadi, tuzatilgach qayta yuklanadi |
| "Quvurni bir marta ishga tushiramiz" | Qayta ishga tushirish odatiy hol — idempotentlik shart |
| "Tasodifiy bo'lish yetarli" | Vaqtli ma'lumotda vaqt bo'yicha bo'lish |
| "Model faylini saqlasak yetadi" | Pipeline + sxema + metadata + profil + nazorat |
| "Kalitni konfig faylga yozamiz" | Faqat muhit o'zgaruvchisi / secret; loglarda niqob |
| "Validatsiya AUC — ishlab chiqarish sifati" | Haqiqiy sifat — log + kechikkan belgilar |
| "Drift ogohlantirishi — darhol qayta o'qitish" | Ogohlantirish; qaror — oldindan yozilgan trigger |
| "Yangi model yaxshiroq — darhol almashtiramiz" | Shadow, bir xil so'rovlar, darvoza |
| "Model almashsa monitoring o'zgarmaydi" | Reference profil ham yangi modelniki bo'ladi |
6. Keng tarqalgan xatolar va yechimlari
1. Validatsiyasiz yuklash
df.to_sql("mijozlar", db, if_exists="append") # ⚠️
if not validatsiya(df): yukla(db, oy, df) # ✅2. Idempotent bo'lmagan yuklash
db.executemany("INSERT INTO mijozlar ...", qatorlar) # ⚠️ har safar qo'shadi
if xesh != eski_xesh: yukla(db, oy, df) # DELETE + INSERT tranzaksiyada # ✅3. Tasodifiy bo'lish
train_test_split(df, test_size=0.2) # ⚠️ kelajak o'quvga tushadi
tr, va = df[df.oy <= 5], df[df.oy == 6] # ✅4. Kalit kodda
KALIT = "abc123" # ⚠️
kalit = os.environ["CHURN_API_KEY"] # ✅5. Bashoratni loglamaslik
javob = {"p": p} # ⚠️ sifatni o'lchab bo'lmaydi
log_yoz(oy, mijoz_id, versiya, p); javob = {"p": p} # ✅6. Eski reference
prof = v1_paket["profil"] # butun hayot davomida # ⚠️
prof = xizmat.paket["profil"] # joriy Production # ✅7. Bitta past oyda qayta o'qitish
if auc < chegara: qayta_orgat() # ⚠️ shovqin
if past_ketma_ket >= 2: qayta_orgat() # ✅ oldindan kelishilgan7. Integratsiya — bu bilim qayerda kerak bo'ladi
- 27.1-27.13-darslar (o'tilgan): Butun qism
- 19-qism (o'tilgan):
Pipeline,ColumnTransformer, joblib - 18-qism (o'tilgan): Juftlashgan taqqoslash va vaqt bo'yicha baholash
- 11-qism (o'tilgan): Gipoteza testlari va A/B — onlayn tajriba uchun
- Maxsus mavzular qismi: Yangi turdagi ma'lumot va vazifalarda xuddi shu skelet — baholash dizayni, paket, xizmat va monitoring
- Loyihalar va karyera qismi: Portfolio loyihasi sifatida — to'liq ishlaydigan ML xizmati
8. Eng yaxshi amaliyotlar
Validatsiya — darvoza; buzuq partiya — karantin.
Yuklash idempotent va atomik.
Vaqt bo'yicha bo'lish, juftlashgan qaror, eng sodda munosib model.
Paket o'zini tekshiradi; registrda audit.
Sir faqat muhitdan; loglarda niqob.
Har bashorat versiya bilan loglanadi.
Qaror qoidalari oldindan, kodda (yoki versiyalangan konfigda).
Shadow, darvoza, rollback — avtomatik va sinalgan.
9. Amaliy topshiriq
Vazifa 1: Bashorat qiling
1. # birlik xatosini qaysi tekshiruv ushlaydi?
2. # quvur ikki marta ishga tushsa, qatorlar soni?
3. # yuklash o'rtasida uzilsa?
4. # nega vaqt bo'yicha bo'lish?
5. # yurish_id nimadan hisoblanadi?
6. # paketda nechta qism va nima uchun nazorat namunalari?
7. # kalit o'rnatilmagan bo'lsa xizmat?
8. # extra="forbid" nima beradi?
9. # drift 9-oyda - trigger eng erta qachon?
10. # shadow oyida foydalanuvchi qaysi versiya javobini oladi?
11. # model almashgach PSI reference?
12. # rollback qanday bajariladi?Javoblar
- Diapazon (sxema va tur emas)
- O'zgarmaydi — idempotent
- Tranzaksiya qaytariladi, eski ma'lumot butun
- Ishlab chiqarishda kelajakni bashorat qilamiz; tasodifiy bo'lish optimistik
- Model, parametrlar, ma'lumot xeshi, kod versiyasi
- Beshta; yuklashda aynan shu chiqish qayta olinishini tekshirish uchun
- Ishga tushmaydi
- Noma'lum maydon — 422
- 11-oy (belgi 1 oy kechikadi + 2 oy qoidasi)
- Production (v1)
- Yangi modelning o'quv taqsimoti
- Registrda Archived -> Production +
/admin/yangila, auditda
Vazifa 2: Xatolarni tuzating
1. df.to_sql("mijozlar", db, if_exists="append")
2. tr, va = train_test_split(df, test_size=0.2)
3. KALIT = "abc123"
4. prof = v1_paket["profil"] # hamma oylar uchun
5. if auc < chegara: registr.production = orgat(oxirgi_oy)Javoblar
1. if not validatsiya(df): yukla(db, oy, df) # xesh + tranzaksiya
2. tr, va = df[df.oy <= 5], df[df.oy == 6]
3. kalit = os.environ["CHURN_API_KEY"]
4. prof = xizmat.paket["profil"] # joriy Production
5. if past_ketma_ket >= 2: registr.otkaz(oy, 2, "Staging", "trigger") # -> shadow -> darvozaVazifa 3: Quvur
Modellang (1-misol asosida):
- Yangi tekshiruv: har oyda ketish ulushi 0.05-0.60 oralig'ida bo'lsin
- Karantin sababini alohida
karantinjadvaliga yozing - Partiya kechikib kelsa (6-oydan keyin 4-oy) — quvur to'g'ri ishlaydimi?
- Xeshni ustun tartibiga bog'liq bo'lmaydigan qiling
Vazifa 4: O'qitish
Modellang (2-misol asosida):
- Vaqt bo'yicha 3 ta fold (1-3→4, 1-4→5, 1-5→6) bilan taqqoslang
- HistGB giperparametrlarini 3 xil qilib, jurnalda yurishlarni solishtiring
- Kalibrovkani darvozaga qo'shing
- Model kartasini JSON sifatida paketga qo'shing
Vazifa 5: Xizmat
Modellang (3-misol asosida):
- Kalit tekshiruvini
Dependsbilan bog'liqlikka chiqaring /predictjavobigaso_rov_idqo'shing va logga yozing- So'rov hajmi limiti (5000) oshsa nima bo'lishini sinang
- Ikki kalitni (eski va yangi) bir vaqtda qabul qilib, kalit almashtirishni (rotation) modellang
Vazifa 6: Hayot sikli
Modellang (4-misol asosida):
- Trigger qoidasini "bir oy" qiling — 7-8 oylarda yolg'on trigger bo'ladimi?
- Challenger oynasini 4 oy (7-10) qiling — darvoza natijasi qanday o'zgaradi?
- Challenger ni ataylab buzing (bir tarif ma'lumotini tashlab) — darvoza qaysi qoida bilan rad etadi?
- Proksi belgi qo'shing: 15 kunlik faollik — trigger necha oy erta yonadi?
Vazifa 7: O'ylash
Rahbar so'radi: "Tizim drift 9-oyda boshlanganini o'sha oyda ko'rgan ekan (PSI ogohlantirishi). Nega yangi model faqat 12-oyda ishga tushdi? Uch oy yo'qotdik. Keyingi safar ogohlantirish yonishi bilan darhol qayta o'qitib, almashtiraylik."
Javob
Qisqa javob: ogohlantirish bilan darhol almashtirish mumkin emas — o'sha paytda yangi rejim haqida belgili ma'lumot yo'q. Lekin tayyorgarlikni ertaroq boshlash va kechikishni qisqartirish mumkin.
1. 9-oyda nima ma'lum edi. Faqat kirishlar va bashoratlar: to'lov, tarif va bashorat taqsimoti siljigan (PSI 0.145, 0.131, 0.270). Model sifati tushdimi — noma'lum: 27.12 da ko'rdikki, kirish drifti ba'zan sifatga umuman ta'sir qilmaydi. Yangi modelni o'qitish uchun esa yangi rejimdagi belgilar kerak — 9-oy belgilari faqat 10-oy boshida keladi.
2. Agar 9-oyda darhol qayta o'qitsak. Yangi model 1-8 oylarda (hammasi eski rejim) o'qitilgan bo'lardi — u v1 dan deyarli farq qilmaydi. "Qayta o'qitildi" degan belgi qo'yiladi, lekin muammo hal bo'lmaydi.
3. Kechikish zanjiri.
# 9-oy: drift boshlandi, PSI ogohlantirishi <- tayyorgarlik boshlanishi mumkin
# 10-oy: 9-oy belgilari -> 1-past oy
# 11-oy: 10-oy belgilari -> 2-past oy -> TRIGGER, v2 (9-10 oy) -> shadow
# 12-oy: 11-oy belgilari -> darvoza OK -> v2 Production4. Qanday qisqartirish mumkin (halol yo'llar).
- Bir oylik qoida: trigger 10-oyda yonadi. Narxi — yolg'on triggerlar ko'payadi (bitta shovqinli oy); buni tarixiy ma'lumotda o'lchab qaror qilish kerak (Vazifa 6.1).
- Proksi belgi: 30 kunlik ketish o'rniga, masalan, 10 kunlik faollik pasayishi — sifat signali 2-3 hafta erta keladi.
- Shadow ni qisqartirish: shadow uchun to'liq oy emas, belgilari tez keladigan kichik namuna.
- Vaqtinchalik himoya: ogohlantirish paytida saqlash bo'limiga "A tarifi mijozlariga ham e'tibor bering" kabi qo'lda qoida — model almashguncha.
5. Nima qilmaslik kerak. Shadow va darvozani o'tkazib yuborish. 27.13-darsda ko'rdikki, quvur xatosi bilan o'qitilgan model umumiy metrikada yaxshi ko'rinib, bir segmentda halokatli bo'lishi mumkin; xizmat kodi xatosi esa faqat nazorat namunalari bilan ko'rinadi.
Rahbarga javob: "9-oyda biz faqat 'nimadir o'zgardi' degan signalni ko'rdik; yangi modelni o'qitish uchun yangi dunyodan belgilar kerak edi, ular 10-oyda kela boshladi. Keyingi safar uchun uchta yaxshilanish taklif qilaman: proksi belgi, tarixda kalibrlangan bir oylik trigger va ogohlantirish paytidagi vaqtinchalik biznes qoidasi. Bu kechikishni qisqartiradi, xavfsizlik tekshiruvlarini esa saqlab qoladi; qanchaga qisqarishini tarixiy ma'lumotda xuddi shu simulyatsiya bilan o'lchab, keyingi hisobotda raqam bilan beraman."
Nimani mustahkamlaydi: 2.1, 2.2, 2.5-bo'limlar.
Xulosa
Bu darsda to'liq ML xizmatini qurdik: xom oylik partiyadan tortib o'zini kuzatadigan va xavfsiz yangilanadigan xizmatgacha.
Eng muhim uch fikr:
Ma'lumot va model — shartnomalar bilan himoyalangan zanjir. 1-misolda validatsiya birlik xatosini (maksimum
316510) va tushib qolgan ustunni ushlab, karantinga yubordi; quvur ikki marta ishga tushganda qatorlar takrorlanmadi (6000 -> 6000), yuklash uzilganda tranzaksiya eski ma'lumotni saqlab qoldi. 2-misolda vaqt bo'yicha bo'lish va juftlashgan bootstrap LogReg ni tanladi (HistGB farqi-0.0066,SE 0.0090— sezilarli emas), paket esaPipeline, sxema, metadata, reference profil va nazorat namunalarini birga olib yuradi va yuklanganda o'zini tekshiradi.Xizmat — himoyalangan va kuzatiladigan. 3-misolda kalit faqat muhitdan olindi (kalitsiz xizmat ishga tushmadi), loglarda niqoblandi; Pydantic to'rt xil buzuq kirishni aniq xabar bilan
422qildi; har bashorat versiya bilan loglandi — bu log keyin haqiqiy sifatni o'lchashning yagona manbai bo'ldi.Hayot sikli oldindan yozilgan qoidalar bilan boshqarildi. 4-misolda 9-oyda PSI erta ogohlantirish berdi, kechikkan belgilar 10 va 11-oyda AUC tushishini (
0.6285,0.6212) ko'rsatdi, trigger v2 ni shadow ga chiqardi, bir xil so'rovlarda darvoza (+0.1395,SE 0.0164) uni tasdiqladi va registr orqali xizmat yangi versiyani yukladi (/version→ 2). v2 xizmat qilgan oylarda AUC0.74-0.78, v1 qolganida0.62-0.63bo'lardi. Rollback — ikki registr o'tishi va bitta chaqiruv.
27-qism xulosasi. Bu qismda model "ishlaydigan noutbuk" dan "ishlaydigan xizmat" ga aylandi. Reproduksiya va muhit natijani qayta olishni kafolatladi; ma'lumot quvuri — validatsiya, karantin, idempotent va atomik yuklashni; eksperiment kuzatuvi va registr — har qaror va har versiyaning tarixini; paketlash — tayyorlash va modelni bitta, o'zini tekshiradigan artefaktga birlashtirishni. FastAPI xizmati, uni mustahkamlash va testlash, Docker va deploy strategiyalari modelni foydalanuvchiga xavfsiz yetkazdi. Monitoring, drift aniqlash, qayta o'qitish va A/B test esa eng muhim haqiqatni ko'rsatdi: deploy — oxiri emas, boshlanish; dunyo o'zgaradi va tizim buni sezishi, o'lchashi va oldindan yozilgan qoidalar bilan xavfsiz javob berishi kerak. Butun qism davomida bitta tamoyil takrorlandi: qaror raqam bilan, oldindan belgilangan qoida bilan va halol — bu oldingi qismlardagi juftlashgan taqqoslash va "eng sodda munosib model" g'oyasining ishlab chiqarishdagi davomi.
Keyingi qism — Maxsus mavzular: kursning asosiy yo'lidan tashqarida qolgan, lekin amalda tez-tez uchraydigan vazifa va ma'lumot turlari. Ularning har birida shu kursda qurilgan skelet — baholash dizayni, juftlashgan taqqoslash, "eng sodda munosib model" qoidasi, paket, xizmat va monitoring — yangi sharoitga qo'llanadi.
Izohlar (0)
Izoh yozish uchun kiring.
- Hozircha izoh yo'q. Birinchi bo'ling!