27.11-dars: Monitoring

QATLAM          SAVOL                          MISOL METRIKALAR
--------------  -----------------------------  ------------------------------------
1. TIZIM        xizmat ishlayaptimi, tezmi?    so'rovlar/s, xato ulushi (5xx),
                                               kechikish p50/p95/p99, CPU, xotira,
                                               navbat uzunligi, replikalar soni
2. MA'LUMOT     kirish kutilganidek mi?        null ulushi, sxema buzilishi (tur,
                                               oraliq), kategoriyalar ulushi,
                                               taqsimot siljishi 27.12-bob
3. MODEL        bashoratlar oqilonami?         bashorat taqsimoti (o'rtacha, ulush
                                               > chegara), ishonch, model versiyasi;
                                               belgilar kelganda - AUC, xato
4. BIZNES       foyda keltiryaptimi?           konversiya, kelishuvlar ulushi,
                                               firibgarlik zarari, shikoyatlar

TIZIM USULLARI:
  RED (so'rovga asoslangan xizmatlar): Rate, Errors, Duration
  USE (resurslar):  Utilization, Saturation, Errors

VAZIFA                 BELGI QACHON MA'LUM         KECHIKISH
mijoz ketishi          keyingi oyda                 ~ 30 kun
kredit defolti         to'lov muddatlari o'tgach     oylar - yil
firibgarlik            chargeback kelganda          kunlar - haftalar
tavsiya (bosish)       darhol                       soniyalar
talab prognozi         davr tugagach                kun - hafta

BELGI YO'Q PAYTDA NIMA KUZATILADI (PROKSI):
  kirish ma'lumotlari: null, oraliq, taqsimot
  bashorat taqsimoti: o'rtacha ball, "ijobiy" ulushi, ishonch
  qisman belgilar: erta signal (1-to'lov o'tkazib yuborildi)
  biznes proksi: tasdiqlangan arizalar ulushi

MUHIM:
  proksi signal "nimadir o'zgardi" deydi, "model yomonlashdi" demaydi
  -> tekshiruv boshlanadi; sifat esa belgilar kelganda tasdiqlanadi
  belgilar kelgach - sifat metrikasi QAYTA hisoblanadi (orqaga qarab)

COUNTER (hisoblagich): faqat o'sadi
  api_sorovlar_jami{yol="/predict", status="500"} 90
  foydali narsa - TEZLIK: rate = (qiymat2 - qiymat1) / vaqt
  qayta ishga tushishda 0 ga tushadi -> "reset" ni hisobga olish kerak

GAUGE: istalgan tomonga o'zgaradi
  api_navbat_uzunligi 3, xotira_bayt, yuklangan_model_versiya

HISTOGRAM: bucketlar (kumulyativ)
  api_kechikish_soniya_bucket{le="0.1"}  12815   <- 100 ms gacha
  api_kechikish_soniya_bucket{le="+Inf"} 20000   <- hammasi
  api_kechikish_soniya_sum / _count               <- o'rtacha uchun
  persentil TAXMINI: kerakli bucketni topib, ichida chiziqli interpolyatsiya

YORLIQLAR (labels): yol, status, model_versiya
  DIQQAT: foydalanuvchi id, so'rov id kabi yuqori kardinallikli yorliqlar
  YO'Q - har qiymat alohida vaqt qatori, xotira portlaydi

PROMETHEUS MODELI (pull):
  ilova /metrics sahifasida joriy qiymatlarni matn sifatida beradi
  Prometheus har 15-30 soniyada "tortib oladi" (scrape) va saqlaydi
  so'rov tili (PromQL) misollari:
    rate(api_sorovlar_jami[5m])                            so'rov/s
    sum(rate(api_sorovlar_jami{status=~"5.."}[5m]))
      / sum(rate(api_sorovlar_jami[5m]))                   xato ulushi
    histogram_quantile(0.99,
      sum by (le) (rate(api_kechikish_soniya_bucket[5m]))) p99

# Haqiqiy loyihada: prometheus_client (bu darsda o'rnatilmagan, g'oya uchun)
from prometheus_client import Counter, Histogram

SOROVLAR = Counter("api_sorovlar_jami", "Jami so'rovlar", ["yol", "status"])
KECHIKISH = Histogram("api_kechikish_soniya", "Kechikish",
                      buckets=[0.02, 0.05, 0.1, 0.2, 0.3, 0.5, 1.0])

with KECHIKISH.time():
    natija = model_bashorat(kirish)
SOROVLAR.labels(yol="/predict", status="200").inc()

histogram_quantile bucket ICHIDAGI taqsimotni bilmaydi -> chiziqli taxmin
  xato <= bucket kengligi
  standart bucketlar (5ms, 10ms, 25ms, 50ms, 100ms, 250ms, 500ms, 1s, ...)
    100-250 ms oralig'ida bitta bucket -> p99 = 320 ms bo'lsa, xato katta
  SLA chegarasi atrofida ZICH bucketlar kerak (masalan, 250, 300, 400 ms)

OXIRGI CHEKLI BUCKETDAN KATTA persentil:
  Prometheus oxirgi chekli chegarani qaytaradi -> p99 "1 s" bo'lib qotib qoladi
  -> bucketlar kutilgan maksimumdan yuqorigacha borsin

ALTERNATIVA: summary (klientda persentil) - lekin replikalar bo'yicha
  QO'SHIB BO'LMAYDI; shuning uchun odatda histogram afzal

ODDIY LOG:        "2026-09-24 10:15 bashorat 0.83 mijoz uchun, 41ms"
                   -> odam o'qiydi, mashina qiynaladi (regex, format o'zgaradi)

STRUKTURALI LOG:  {"sorov_id": "70fd086adf5b", "hodisa": "bashorat",
                   "model_versiya": "churn-v3", "ms": 31.5, "ball": 0.83}
                   -> filtr, guruhlash, metrikaga aylantirish oson

SO'ROV ID (correlation / trace id):
  gateway da yaratiladi -> har xizmatga sarlavhada uzatiladi
  -> har log qatorida bor -> bitta so'rovning butun izi bitta filtr bilan
  (to'liq variant: distributed tracing - OpenTelemetry, span lar)

QOIDALAR:
  har log qatorida: vaqt, daraja, xizmat, sorov_id, model_versiya
  shaxsiy ma'lumot (ism, telefon, karta) - NIQOBLANADI yoki yozilmaydi
  sirlar - hech qachon 27.9-bob
  bashorat loglari = kelajakdagi o'quv/baholash ma'lumoti:
    kirish + bashorat + versiya saqlanadi -> belgi kelganda birlashtiriladi

STATIK CHEGARA:      xato ulushi > 2%
  + sodda, tushunarli    - so'rovlar kam paytda (tun) shovqin katta
                         - mavsumiylikni bilmaydi

"FOR" SHARTI:        xato ulushi > 2% ketma-ket 15 daqiqa
  + bir martalik sakrashni e'tiborsiz qoldiradi
  - qisqa insidentni butunlay o'tkazib yuboradi, kechikish qo'shadi

STATISTIK (dinamik): x > o'rtacha(oxirgi kun) + k * std
  + moslashadi           - std ichida tun/kunduz aralash, insidentlar ham
                           asosga "singib" ketadi

SO'ROVLAR SONIGA MOS (binomial z):
  z = (xato - n * p) / sqrt(n * p * (1 - p)),   p - tarixiy ulush
  + tunda n kichik -> maxraj kichik -> bitta xato "yong'in" emas
  - n juda kichik bo'lsa normal yaqinlashuv buziladi -> minimal n yoki
    ketma-ketlik sharti

BAHOLASH (ma'lum insidentlar tarixida):
  precision = haqiqiy ogohlantirishlar / barcha ogohlantirishlar
  recall    = topilgan insidentlar / barcha insidentlar
  aniqlash kechikishi, haftasiga yolg'on ogohlantirishlar

ALERT FATIGUE:
  haftasiga 20 yolg'on signal -> odamlar ogohlantirishni o'qimay qo'yadi
  -> haqiqiy insident ham e'tiborsiz qoladi
QOIDA: har ogohlantirish = odam HOZIR nimadir qilishi kerak
  harakat talab qilmasa - dashboard yoki kunlik hisobot, ogohlantirish emas
  PAGE (tunda uyg'otadi) vs TICKET (ish vaqtida ko'riladi)

SLI (indicator):  o'lchanadigan ko'rsatkich
                  "yaxshi so'rovlar / barcha so'rovlar"
                  yaxshi = status < 500 VA kechikish < 300 ms
SLO (objective):  maqsad: 30 kunda SLI >= 99.7%
SLA (agreement):  mijoz bilan shartnoma (odatda SLO dan yumshoqroq, jarima bilan)

XATO BYUDJETI = 1 - SLO = 0.3% so'rovlar "yomon" bo'lishi MUMKIN
  30 kunda 10 mln so'rov -> 30 000 yomon so'rovga ruxsat
  to'liq uzilish ekvivalenti: 0.003 * 30 * 24 * 60 = 130 daqiqa

BYUDJET SIYOSATI:
  byudjet bor  -> yangi funksiyalar, tajribalar, tez deploy
  byudjet tugadi -> "muzlatish": faqat ishonchlilik ishlari

BURN RATE = (joriy yomon ulush) / (byudjet ulushi)
  burn = 1  -> byudjet aynan 30 kunda tugaydi
  burn = 14.4 -> 1 soatda oylik byudjetning 2% i (o'rtacha trafikda)

KO'P OYNALI OGOHLANTIRISH (Google SRE Workbook g'oyasi):
  page:   burn(1 soat) > 14.4  VA burn(5 daq)  > 14.4
  page:   burn(6 soat) > 6     VA burn(30 daq) > 6
  ticket: burn(3 kun)  > 1     VA burn(6 soat) > 1
  uzun oyna - ahamiyatlilik, qisqa oyna - "hali davom etyaptimi"

BITTA XIZMAT UCHUN DASHBOARD (yuqoridan pastga - umumiydan tafsilotga):

+------------------------------------------------------------------+
| SLO: 99.72% (maqsad 99.7%)  | byudjet qoldi: 6%  | burn(1h): 0.4 |  <- holat
+------------------------------------------------------------------+
| so'rov/s (RED: Rate)        | xato ulushi (Errors) | p50/p95/p99 |  <- tizim
+------------------------------------------------------------------+
| null ulushi, sxema xatolari | kirish taqsimoti (asosiy belgilar) |  <- ma'lumot
+------------------------------------------------------------------+
| o'rtacha ball, ball > 0.5   | model versiyasi bo'yicha trafik    |  <- model
| AUC (belgilar kelgan kunlar, kechikish bilan belgilangan)        |
+------------------------------------------------------------------+
| biznes: konversiya, kelishuvlar ulushi                           |  <- biznes
+------------------------------------------------------------------+
| deploy belgilari (annotatsiya) - grafiklarda vertikal chiziq     |

QOIDALAR:
  har panelda: birlik, chegara chizig'i (SLO / ogohlantirish), vaqt oralig'i
  deploy va insidentlar annotatsiya bilan - "nima o'zgardi" darhol ko'rinadi
  dashboard - tahlil uchun; ogohlantirish - harakat uchun (2.6)

import json
import math


def rate(oldingi, joriy, soniya):
    """Counter dan tezlik; reset bo'lsa joriy qiymat 0 dan boshlangan."""
    farq = joriy - oldingi if joriy >= oldingi else joriy
    return farq / soniya


def binomial_z(xato, n, p):
    return (xato - n * p) / math.sqrt(n * p * (1 - p))


def burn_rate(yomon, jami, slo):
    return (yomon / jami) / (1 - slo)


def log_json(**maydonlar):
    print(json.dumps(maydonlar, ensure_ascii=False))


log_json(sorov_id="70fd086adf5b", hodisa="bashorat", ms=31.5, ball=0.83)

# Prometheus ogohlantirish qoidasi (g'oya)
groups:
  - name: churn-api
    rules:
      - alert: TezByudjetYonishi
        expr: |
          (sum(rate(api_yomon_jami[1h])) / sum(rate(api_sorovlar_jami[1h]))) / 0.003 > 14.4
          and
          (sum(rate(api_yomon_jami[5m])) / sum(rate(api_sorovlar_jami[5m]))) / 0.003 > 14.4
        labels: {severity: page}
        annotations:
          summary: "churn-api: xato byudjeti tez yonmoqda"

to'rt qatlam: tizim (RED/USE) -> ma'lumot -> model -> biznes
belgi kechikadi -> proksi: null, sxema, bashorat taqsimoti
counter (rate, reset), gauge, histogram (bucketlar SLA atrofida zich)
persentil: bucketlarni qo'shib, keyin histogram_quantile
loglar: JSON, sorov_id, model_versiya, shaxsiy ma'lumotsiz
ogohlantirish: n ga mos (binomial z) + ketma-ketlik; tarixda baholash
SLO -> byudjet -> burn rate (1h/5m, 6h/30m, 3d/6h)

"""Metrikalar noldan: Counter, Gauge, Histogram (Prometheus g'oyasi),
persentilni bucketlardan taxminlash va uning xatosi. Virtual vaqt."""

import bisect
import math

import numpy as np


class Counter:
    """Faqat o'sadi. Tezlik (rate) - ikki o'lchov orasidagi farqdan."""

    def __init__(self, nom, yordam):
        self.nom, self.yordam, self.qiymatlar = nom, yordam, {}

    def inc(self, n=1.0, **yorliq):
        k = tuple(sorted(yorliq.items()))
        self.qiymatlar[k] = self.qiymatlar.get(k, 0.0) + n


class Gauge:
    """Istalgan tomonga o'zgaradi: navbat uzunligi, xotira, yuklangan model."""

    def __init__(self, nom, yordam):
        self.nom, self.yordam, self.qiymatlar = nom, yordam, {}

    def set(self, v, **yorliq):
        self.qiymatlar[tuple(sorted(yorliq.items()))] = v


class Histogram:
    """Kumulyativ bucketlar: le=chegara -> shu chegaragacha nechta."""

    def __init__(self, nom, yordam, chegaralar):
        self.nom, self.yordam = nom, yordam
        self.chegaralar = list(chegaralar) + [math.inf]
        self.sanoq = [0] * len(self.chegaralar)
        self.yigindi = 0.0
        self.soni = 0

    def observe(self, v):
        self.sanoq[bisect.bisect_left(self.chegaralar, v)] += 1
        self.yigindi += v
        self.soni += 1

    def kumulyativ(self):
        return np.cumsum(self.sanoq)

    def birlashtir(self, boshqa):
        assert self.chegaralar == boshqa.chegaralar, "bucketlar bir xil bo'lsin"
        h = Histogram(self.nom, self.yordam, self.chegaralar[:-1])
        h.sanoq = [a + b for a, b in zip(self.sanoq, boshqa.sanoq)]
        h.yigindi = self.yigindi + boshqa.yigindi
        h.soni = self.soni + boshqa.soni
        return h


def histogram_quantile(q, h):
    """Prometheus dagi kabi: bucket ichida chiziqli interpolyatsiya."""
    kum = h.kumulyativ()
    nishon = q * kum[-1]
    i = int(np.searchsorted(kum, nishon))
    if h.chegaralar[i] == math.inf:           # oxirgi chekli chegara qaytadi
        return h.chegaralar[i - 1]
    past = 0.0 if i == 0 else h.chegaralar[i - 1]
    oldingi = 0 if i == 0 else kum[i - 1]
    ichida = kum[i] - oldingi
    return past + (h.chegaralar[i] - past) * (nishon - oldingi) / ichida


def ekspozitsiya(metrikalar):
    """/metrics sahifasining matn formati (qisqartirilgan)."""
    qatorlar = []
    for m in metrikalar:
        tur = type(m).__name__.lower()
        qatorlar += [f"# HELP {m.nom} {m.yordam}", f"# TYPE {m.nom} {tur}"]
        if isinstance(m, Histogram):
            for le, s in zip(m.chegaralar, m.kumulyativ()):
                le_s = "+Inf" if le == math.inf else f"{le:g}"
                qatorlar.append(f'{m.nom}_bucket{{le="{le_s}"}} {s}')
            qatorlar.append(f"{m.nom}_sum {m.yigindi:.3f}")
            qatorlar.append(f"{m.nom}_count {m.soni}")
        else:
            for k in sorted(m.qiymatlar):
                y = ",".join(f'{a}="{b}"' for a, b in k)
                qatorlar.append(f"{m.nom}{{{y}}} {m.qiymatlar[k]:g}")
    return qatorlar


STANDART = [0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10]
MOSLANGAN = [0.02, 0.04, 0.06, 0.08, 0.1, 0.12, 0.15, 0.2, 0.25, 0.3, 0.4,
             0.5, 0.75, 1.0, 1.5, 2.5]


def main() -> None:
    rng = np.random.default_rng(0)

    print("=== 1. Yig'ish: 20000 so'rov (virtual kechikishlar) ===")
    sorovlar = Counter("api_sorovlar_jami", "Jami so'rovlar")
    navbat = Gauge("api_navbat_uzunligi", "Navbatdagi so'rovlar")
    kech = Histogram("api_kechikish_soniya", "So'rov kechikishi", STANDART)
    kechikishlar = rng.lognormal(math.log(0.08), 0.6, 20_000)
    for v in kechikishlar:
        kech.observe(float(v))
    for yol, ulush in [("/predict", 0.9), ("/health", 0.1)]:
        for status, p in [("200", 0.985), ("422", 0.01), ("500", 0.005)]:
            sorovlar.inc(round(20_000 * ulush * p), yol=yol, status=status)
    navbat.set(3)
    matn = ekspozitsiya([sorovlar, navbat, kech])
    tanlangan = [q for q in matn if "predict" in q or "navbat" in q
                 or q.startswith(("api_kechikish_soniya_sum",
                                  "api_kechikish_soniya_count"))
                 or 'le="0.1"' in q or 'le="+Inf"' in q]
    for q in tanlangan:
        print(f"  {q}")
    print(f"  (jami {len(matn)} qator, {len(matn) - len(tanlangan)} tasi "
          f"ko'rsatilmadi)")

    print("\n=== 2. Persentilni bucketlardan taxminlash ===")
    moslangan = Histogram("k2", "", MOSLANGAN)
    for v in kechikishlar:
        moslangan.observe(float(v))
    print(f"  {'q':>5} {'haqiqiy':>9} {'standart':>9} {'xato':>7} "
          f"{'moslangan':>9} {'xato':>7}")
    for q in [0.5, 0.9, 0.95, 0.99]:
        hq = float(np.quantile(kechikishlar, q))
        a = histogram_quantile(q, kech)
        b = histogram_quantile(q, moslangan)
        print(f"  {q:>5} {hq * 1000:>7.1f}ms {a * 1000:>7.1f}ms "
              f"{(a - hq) / hq:>+7.1%} {b * 1000:>7.1f}ms {(b - hq) / hq:>+7.1%}")
    print(f"  bucketlar: standart {len(STANDART)}, moslangan {len(MOSLANGAN)}"
          f" - aniqlik bucket kengligiga bog'liq")

    print("\n=== 3. Counter dan tezlik: 15 soniyalik o'lchovlar, qayta ishga "
          "tushish bilan ===")
    haqiqiy_tezlik = 40.0                       # so'rov/s
    olchovlar, qiymat = [], 0.0
    for i in range(12):
        if i == 7:
            qiymat = 0.0                        # pod qayta ishga tushdi
        qiymat += haqiqiy_tezlik * 15 + rng.normal(0, 20)
        olchovlar.append(qiymat)
    sodda, tuzatilgan = [], []
    for a, b in zip(olchovlar, olchovlar[1:]):
        sodda.append((b - a) / 15)
        tuzatilgan.append((b - a if b >= a else b) / 15)   # reset -> 0 dan
    print(f"  sodda farq:     min {min(sodda):>8.1f}, o'rtacha "
          f"{np.mean(sodda):>6.1f} so'rov/s")
    print(f"  reset hisobga:  min {min(tuzatilgan):>8.1f}, o'rtacha "
          f"{np.mean(tuzatilgan):>6.1f} so'rov/s (haqiqiy {haqiqiy_tezlik})")

    print("\n=== 4. Replikalar bo'yicha birlashtirish ===")
    replikalar = []
    for i, (med, sig, n) in enumerate([(0.06, 0.4, 9000), (0.065, 0.4, 9000),
                                       (0.3, 0.5, 2000)]):
        h = Histogram("r", "", MOSLANGAN)
        x = rng.lognormal(math.log(med), sig, n)
        for v in x:
            h.observe(float(v))
        replikalar.append((h, x))
        print(f"  replika {i}: {n} so'rov, p99 {histogram_quantile(0.99, h) * 1000:.0f} ms")
    hammasi = np.concatenate([x for _, x in replikalar])
    haqiqiy = float(np.quantile(hammasi, 0.99))
    ortacha_p99 = np.mean([histogram_quantile(0.99, h) for h, _ in replikalar])
    birlashgan = replikalar[0][0].birlashtir(replikalar[1][0]) \
        .birlashtir(replikalar[2][0])
    b99 = histogram_quantile(0.99, birlashgan)
    print(f"  haqiqiy umumiy p99:             {haqiqiy * 1000:>6.0f} ms")
    print(f"  p99 larning o'rtachasi (XATO):   {ortacha_p99 * 1000:>6.0f} ms "
          f"({(ortacha_p99 - haqiqiy) / haqiqiy:+.0%})")
    print(f"  birlashgan gistogramma p99:     {b99 * 1000:>6.0f} ms "
          f"({(b99 - haqiqiy) / haqiqiy:+.0%})")
    print("  ⭐ Persentillarni o'rtachalamang - gistogrammalarni qo'shing")


if __name__ == "__main__":
    main()

=== 1. Yig'ish: 20000 so'rov (virtual kechikishlar) ===
  api_sorovlar_jami{status="200",yol="/predict"} 17730
  api_sorovlar_jami{status="422",yol="/predict"} 180
  api_sorovlar_jami{status="500",yol="/predict"} 90
  # HELP api_navbat_uzunligi Navbatdagi so'rovlar
  # TYPE api_navbat_uzunligi gauge
  api_navbat_uzunligi{} 3
  api_kechikish_soniya_bucket{le="0.1"} 12815
  api_kechikish_soniya_bucket{le="+Inf"} 20000
  api_kechikish_soniya_sum 1916.705
  api_kechikish_soniya_count 20000
  (jami 27 qator, 17 tasi ko'rsatilmadi)

=== 2. Persentilni bucketlardan taxminlash ===
      q   haqiqiy  standart    xato moslangan    xato
    0.5    79.7ms    83.5ms   +4.7%    79.8ms   +0.1%
    0.9   172.1ms   217.4ms  +26.3%   178.0ms   +3.5%
   0.95   213.1ms   240.1ms  +12.6%   217.9ms   +2.2%
   0.99   319.9ms   417.4ms  +30.5%   333.7ms   +4.3%
  bucketlar: standart 11, moslangan 16 - aniqlik bucket kengligiga bog'liq

=== 3. Counter dan tezlik: 15 soniyalik o'lchovlar, qayta ishga tushish bilan ===
  sodda farq:     min   -242.5, o'rtacha   14.5 so'rov/s
  reset hisobga:  min     38.2, o'rtacha   40.4 so'rov/s (haqiqiy 40.0)

=== 4. Replikalar bo'yicha birlashtirish ===
  replika 0: 9000 so'rov, p99 158 ms
  replika 1: 9000 so'rov, p99 180 ms
  replika 2: 2000 so'rov, p99 1075 ms
  haqiqiy umumiy p99:                569 ms
  p99 larning o'rtachasi (XATO):      471 ms (-17%)
  birlashgan gistogramma p99:        607 ms (+7%)
  ⭐ Persentillarni o'rtachalamang - gistogrammalarni qo'shing

"""Strukturali loglar, so'rov id, ma'lumot/model metrikalari va belgilar
kechikishi. 45 virtual kun; 25-kuni ma'lumot quvurida xato paydo bo'ladi."""

import hashlib
import json
from collections import defaultdict

import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score

KUNLAR = 45
KUNIGA = 400
BUZILISH_KUNI = 25
BELGI_KECHIKISHI = 14          # ketish belgisi 14 kundan keyin ma'lum (faraziy)
USTUNLAR = ["oylik_tolov", "muddat_oy", "murojaatlar"]


def mijozlar(rng, n):
    tolov = rng.lognormal(np.log(120_000), 0.4, n)          # so'm
    muddat = rng.integers(1, 60, n).astype(float)
    murojaat = rng.poisson(1.5, n).astype(float)
    logit = (-1.0 + 0.9 * np.log(tolov / 120_000) - 0.05 * muddat
             + 0.45 * murojaat)
    ketdi = rng.random(n) < 1 / (1 + np.exp(-logit))
    return np.column_stack([tolov, muddat, murojaat]), ketdi.astype(int)


def belgilar(X):
    """Modelga kirish: log(to'lov), muddat, murojaatlar; null -> median."""
    Z = X.copy()
    Z[:, 0] = np.log(np.where(np.isnan(Z[:, 0]), 120_000, Z[:, 0]))
    Z[:, 1] = np.where(np.isnan(Z[:, 1]), 30, Z[:, 1])
    return Z


def sorov_id(kun, i):
    return hashlib.sha256(f"{kun}-{i}".encode()).hexdigest()[:12]


def log(yozuvlar, **maydonlar):
    """Bitta hodisa = bitta JSON qator (vaqt belgisi o'rniga virtual kun)."""
    yozuvlar.append(json.dumps(maydonlar, ensure_ascii=False))


def main() -> None:
    rng = np.random.default_rng(3)
    Xo, yo = mijozlar(rng, 5000)
    model = LogisticRegression(max_iter=1000).fit(belgilar(Xo), yo)

    loglar, belgi_jadvali = [], {}
    for kun in range(KUNLAR):
        X, y = mijozlar(rng, KUNIGA)
        if kun >= BUZILISH_KUNI:
            # quvur xatosi: to'lovning bir qismi tiyinda (x100),
            # muddat esa ba'zan kelmaydi
            tiyin = rng.random(KUNIGA) < 0.6
            X[tiyin, 0] *= 100
            X[rng.random(KUNIGA) < 0.2, 1] = np.nan
        ball = model.predict_proba(belgilar(X))[:, 1]
        belgi_ms = rng.lognormal(np.log(8), 0.3, KUNIGA)
        kesh_otdi = rng.random(KUNIGA) < 0.03          # belgilar keshi o'tdi
        belgi_ms[kesh_otdi] += rng.uniform(120, 250, kesh_otdi.sum())
        model_ms = rng.lognormal(np.log(12), 0.3, KUNIGA)
        for i in range(KUNIGA):
            sid = sorov_id(kun, i)
            kirish = {u: (None if np.isnan(v) else round(float(v), 2))
                      for u, v in zip(USTUNLAR, X[i])}
            log(loglar, kun=kun, sorov_id=sid, hodisa="qabul", yol="/predict")
            log(loglar, kun=kun, sorov_id=sid, hodisa="belgilar",
                ms=round(float(belgi_ms[i]), 1), kesh=not kesh_otdi[i],
                kirish=kirish)
            log(loglar, kun=kun, sorov_id=sid, hodisa="bashorat",
                model_versiya="churn-v3", ms=round(float(model_ms[i]), 1),
                ball=round(float(ball[i]), 4))
            belgi_jadvali[sid] = (kun + BELGI_KECHIKISHI, int(y[i]))

    print("=== 1. Strukturali log qatorlari (JSON) ===")
    for q in loglar[:3]:
        print(f"  {q[:78]}".rstrip())
    print(f"  jami {len(loglar)} qator: har so'rovga 3 hodisa, bitta sorov_id")

    print("\n=== 2. So'rov id bo'yicha iz: eng sekin so'rov ===")
    yozuvlar = [json.loads(q) for q in loglar]
    izlar = defaultdict(list)
    for r in yozuvlar:
        izlar[r["sorov_id"]].append(r)
    jami_ms = {sid: sum(r.get("ms", 0) for r in rr) for sid, rr in izlar.items()}
    sekin = max(sorted(jami_ms), key=lambda s: jami_ms[s])
    for r in izlar[sekin]:
        qoshimcha = (f"{r['ms']:>6} ms" if "ms" in r else "")
        if r["hodisa"] == "belgilar":
            qoshimcha += f", kesh={r['kesh']}"
        qator = f"  {sekin} kun {r['kun']:>2} {r['hodisa']:<9} {qoshimcha}"
        print(qator.rstrip())
    otganlar = [r for r in yozuvlar if r["hodisa"] == "belgilar"]
    p99_kesh = np.percentile([r["ms"] for r in otganlar if r["kesh"]], 99)
    p99_yoq = np.percentile([r["ms"] for r in otganlar if not r["kesh"]], 99)
    print(f"  belgilar bosqichi p99: kesh bilan {p99_kesh:.0f} ms, "
          f"keshsiz {p99_yoq:.0f} ms -> sekinlik sababi topildi")

    print("\n=== 3. Loglardan kunlik metrikalar ===")
    kun_boyicha = defaultdict(list)
    for r in yozuvlar:
        if r["hodisa"] != "qabul":
            kun_boyicha[(r["kun"], r["hodisa"])].append(r)
    kunlik = {}
    for kun in range(KUNLAR):
        kir = [r["kirish"] for r in kun_boyicha[(kun, "belgilar")]]
        bash = kun_boyicha[(kun, "bashorat")]
        b = np.array([r["ball"] for r in bash])
        kunlik[kun] = {
            "null": np.mean([k["muddat_oy"] is None for k in kir]),
            "sxema": np.mean([(k["oylik_tolov"] or 0) > 2_000_000 for k in kir]),
            "ball": b.mean(), "yuqori": (b > 0.5).mean(),
            "sidlar": [r["sorov_id"] for r in bash], "ballar": b}
    bugun = KUNLAR - 1
    sarlavha = ["null muddat", "tolov>2mln", "o'rt ball", "ball>0.5",
                f"AUC ({bugun}-kun bilgani)"]
    print(f"  {'kun':>3} {sarlavha[0]:>11} {sarlavha[1]:>10} {sarlavha[2]:>9} "
          f"{sarlavha[3]:>8} {sarlavha[4]:>21}")
    for kun in [5, 20, 24, 25, 30, 31, 44]:
        m = kunlik[kun]
        if kun + BELGI_KECHIKISHI <= bugun:
            auc = roc_auc_score([belgi_jadvali[s][1] for s in m["sidlar"]],
                                m["ballar"])
            auc_s = f"{auc:.3f}"
        else:
            auc_s = "hali noma'lum"
        print(f"  {kun:>3} {m['null']:>11.1%} {m['sxema']:>10.1%} "
              f"{m['ball']:>9.3f} {m['yuqori']:>8.1%} {auc_s:>21}")

    print("\n=== 4. Qachon SEZILADI: proksi vs sifat metrikasi ===")
    ref = [kunlik[k]["ball"] for k in range(BUZILISH_KUNI)]
    ort_ref, std_ref = np.mean(ref), np.std(ref, ddof=1)
    birinchi = {}
    for kun in range(KUNLAR):
        m = kunlik[kun]
        if "null ulushi > 5%" not in birinchi and m["null"] > 0.05:
            birinchi["null ulushi > 5%"] = kun
        if "sxema buzilishi > 1%" not in birinchi and m["sxema"] > 0.01:
            birinchi["sxema buzilishi > 1%"] = kun
        if ("o'rtacha ball > 4 std" not in birinchi
                and abs(m["ball"] - ort_ref) > 4 * std_ref):
            birinchi["o'rtacha ball > 4 std"] = kun

    def oyna_auc(d):                            # [d-6, d] kunlar
        sid = [s for k in range(d - 6, d + 1) for s in kunlik[k]["sidlar"]]
        ball = np.concatenate([kunlik[k]["ballar"] for k in range(d - 6, d + 1)])
        return roc_auc_score([belgi_jadvali[s][1] for s in sid], ball)

    asos_auc = np.mean([oyna_auc(d) for d in range(6, BUZILISH_KUNI)])
    for d in range(6, bugun - BELGI_KECHIKISHI + 1):
        if oyna_auc(d) < asos_auc - 0.03:
            birinchi["7 kunlik AUC -0.03"] = d + BELGI_KECHIKISHI
            break
    print(f"  asos 7 kunlik AUC: {asos_auc:.3f}")
    for nom, kun in sorted(birinchi.items(), key=lambda x: x[1]):
        print(f"  {nom:<22} birinchi signal: {kun}-kun "
              f"(buzilishdan {kun - BUZILISH_KUNI} kun keyin)")
    print(f"  buzilish {BUZILISH_KUNI}-kuni; belgilar {BELGI_KECHIKISHI} kun "
          f"kechikadi")
    print("  ⭐ Belgilar kechiksa - ma'lumot va bashorat metrikalari (proksi) "
          "birinchi signal")


if __name__ == "__main__":
    main()

=== 1. Strukturali log qatorlari (JSON) ===
  {"kun": 0, "sorov_id": "22841ea360fc", "hodisa": "qabul", "yol": "/predict"}
  {"kun": 0, "sorov_id": "22841ea360fc", "hodisa": "belgilar", "ms": 8.5, "kesh"
  {"kun": 0, "sorov_id": "22841ea360fc", "hodisa": "bashorat", "model_versiya":
  jami 54000 qator: har so'rovga 3 hodisa, bitta sorov_id

=== 2. So'rov id bo'yicha iz: eng sekin so'rov ===
  70fd086adf5b kun 32 qabul
  70fd086adf5b kun 32 belgilar   260.8 ms, kesh=False
  70fd086adf5b kun 32 bashorat    31.5 ms
  belgilar bosqichi p99: kesh bilan 16 ms, keshsiz 256 ms -> sekinlik sababi topildi

=== 3. Loglardan kunlik metrikalar ===
  kun null muddat tolov>2mln o'rt ball ball>0.5  AUC (44-kun bilgani)
    5        0.0%       0.0%     0.180     3.8%                 0.703
   20        0.0%       0.0%     0.178     2.8%                 0.763
   24        0.0%       0.0%     0.177     3.8%                 0.768
   25       19.0%      59.8%     0.581    60.0%                 0.537
   30       19.2%      63.0%     0.611    64.8%                 0.620
   31       22.2%      58.2%     0.569    58.5%         hali noma'lum
   44       15.8%      60.5%     0.585    61.5%         hali noma'lum

=== 4. Qachon SEZILADI: proksi vs sifat metrikasi ===
  asos 7 kunlik AUC: 0.768
  null ulushi > 5%       birinchi signal: 25-kun (buzilishdan 0 kun keyin)
  sxema buzilishi > 1%   birinchi signal: 25-kun (buzilishdan 0 kun keyin)
  o'rtacha ball > 4 std  birinchi signal: 25-kun (buzilishdan 0 kun keyin)
  7 kunlik AUC -0.03     birinchi signal: 39-kun (buzilishdan 14 kun keyin)
  buzilish 25-kuni; belgilar 14 kun kechikadi
  ⭐ Belgilar kechiksa - ma'lumot va bashorat metrikalari (proksi) birinchi signal

"""Ogohlantirish qoidalari: statik chegara, statistik chegara va binomial z.
30 virtual kun, 5 daqiqalik oynalar, 8 ta ma'lum insident. Precision, recall,
aniqlash kechikishi va haftasiga yolg'on ogohlantirishlar (alert fatigue)."""

import numpy as np
import pandas as pd

OYNA_DAQ = 5
KUNGA = 24 * 60 // OYNA_DAQ          # 288 oyna
KUNLAR = 30
P0 = 0.005                           # odatiy xato ulushi

# (kun, soat, davomiylik daqiqa, qo'shimcha xato ulushi) - "haqiqat"
INSIDENTLAR = [(2, 14, 30, 0.05), (5, 3, 60, 0.03), (8, 10, 15, 0.015),
               (12, 20, 120, 0.008), (16, 12, 10, 0.10), (20, 4, 45, 0.02),
               (24, 15, 240, 0.005), (28, 9, 20, 0.04)]


def malumot(rng):
    t = np.arange(KUNLAR * KUNGA)
    soat = (t % KUNGA) * OYNA_DAQ / 60
    # tunda ~100, kunduzi ~3000 so'rov / 5 daqiqa
    kutilgan = 100 + 2900 * np.exp(-((soat - 14) / 4.5) ** 2)
    n = rng.poisson(kutilgan)
    p = np.full(len(t), P0)
    haqiqat = np.zeros(len(t), dtype=int)          # insident raqami (1..8)
    for i, (kun, s, davom, qosh) in enumerate(INSIDENTLAR, start=1):
        bosh = kun * KUNGA + s * 60 // OYNA_DAQ
        oxir = bosh + davom // OYNA_DAQ
        p[bosh:oxir] += qosh
        haqiqat[bosh:oxir] = i
    xato = rng.binomial(n, p)
    return pd.DataFrame({"n": n, "xato": xato, "insident": haqiqat})


def qoidalar(df):
    r = df["xato"] / df["n"].clip(lower=1)
    oldingi = r.shift(1)
    ort = oldingi.rolling(KUNGA, min_periods=KUNGA // 2).mean()
    std = oldingi.rolling(KUNGA, min_periods=KUNGA // 2).std()
    # kunlik tarixiy ulush (xatolar / so'rovlar), joriy oynasiz
    p_hat = (df["xato"].shift(1).rolling(KUNGA, min_periods=KUNGA // 2).sum()
             / df["n"].shift(1).rolling(KUNGA, min_periods=KUNGA // 2).sum())
    z = (df["xato"] - df["n"] * p_hat) / np.sqrt(df["n"] * p_hat * (1 - p_hat))

    def ketma_ket(shart, k):
        return shart.astype(int).rolling(k).sum() == k

    return {
        "statik r > 2%": r > 0.02,
        "statik r > 5%": r > 0.05,
        "statik r > 2%, 15 daq": ketma_ket(r > 0.02, 3),
        "o'rtacha + 3 std": r > ort + 3 * std,
        "o'rtacha + 5 std": r > ort + 5 * std,
        "binomial z > 4": z > 4,
        "binomial z > 4, 10 daq": ketma_ket(z > 4, 2),
    }


def baholash(signal, insident):
    """Ketma-ket yongan oynalar -> bitta ogohlantirish (hodisa)."""
    s = signal.fillna(False).to_numpy()
    boshlar = np.flatnonzero(s & ~np.r_[False, s[:-1]])
    haqiqiy_alert, topilgan, kechikish = 0, {}, []
    ins = insident.to_numpy()
    for b in boshlar:
        oxir = b
        while oxir + 1 < len(s) and s[oxir + 1]:
            oxir += 1
        tegishli = set(ins[b:oxir + 1]) - {0}
        if tegishli:
            haqiqiy_alert += 1
            for i in tegishli:
                topilgan.setdefault(i, b)
    for i, b in topilgan.items():
        ins_bosh = int(np.flatnonzero(ins == i)[0])
        kechikish.append((b - ins_bosh) * OYNA_DAQ)
    jami = len(boshlar)
    yolgon = jami - haqiqiy_alert
    return {"jami": jami, "yolgon_haftasiga": yolgon / (KUNLAR / 7),
            "precision": haqiqiy_alert / jami if jami else float("nan"),
            "recall": len(topilgan) / len(INSIDENTLAR),
            "kechikish": float(np.median(kechikish)) if kechikish
            else float("nan"), "topilgan": sorted(topilgan)}


def main() -> None:
    rng = np.random.default_rng(11)
    df = malumot(rng)

    print("=== 1. Ma'lumot: 30 kun x 288 oyna ===")
    r = df["xato"] / df["n"]
    kun = np.arange(len(df)) % KUNGA
    tun = (kun < 5 * 12) & (df["insident"] == 0)
    kunduz = (kun >= 12 * 12) & (kun < 16 * 12) & (df["insident"] == 0)
    print(f"  oynalar: {len(df)}, insidentli oynalar: "
          f"{(df['insident'] > 0).sum()}")
    print(f"  tungi oyna: o'rtacha {df['n'][tun].mean():.0f} so'rov, xato "
          f"ulushi std {r[tun].std():.4f}")
    print(f"  kunduzgi oyna: o'rtacha {df['n'][kunduz].mean():.0f} so'rov, "
          f"xato ulushi std {r[kunduz].std():.4f}")
    print("  bir xil P0 = 0.5%, lekin tunda shovqin ancha katta")

    print("\n=== 2. Qoidalar taqqoslash ===")
    ustun = "yolg'on/hafta"
    print(f"  {'qoida':<24} {'alert':>5} {ustun:>13} "
          f"{'precision':>9} {'recall':>6} {'kechikish':>9}")
    natija = {}
    for nom, signal in qoidalar(df).items():
        b = baholash(signal, df["insident"])
        natija[nom] = b
        kech = (f"{b['kechikish']:.0f} daq" if b["kechikish"] == b["kechikish"]
                else "-")
        print(f"  {nom:<24} {b['jami']:>5} {b['yolgon_haftasiga']:>13.1f} "
              f"{b['precision']:>9.2f} {b['recall']:>6.2f} {kech:>9}")

    print("\n=== 3. Qaysi insidentlar topildi ===")
    for nom in ["statik r > 5%", "statik r > 2%, 15 daq", "binomial z > 4, 10 daq"]:
        belgilar = "".join("X" if i in natija[nom]["topilgan"] else "-"
                           for i in range(1, len(INSIDENTLAR) + 1))
        print(f"  {nom:<24} {belgilar}  (1-8)")
    tavsif = [f"{i}: +{q:.1%} {d} daq" for i, (_, _, d, q) in
              enumerate(INSIDENTLAR, start=1)]
    print("  " + ", ".join(tavsif[:4]))
    print("  " + ", ".join(tavsif[4:]))

    print("\n=== 4. Qaror: yolg'on <= 1/hafta, eng yuqori recall ===")
    mos = {k: v for k, v in natija.items() if v["yolgon_haftasiga"] <= 1}
    if mos:
        eng = max(mos, key=lambda k: (mos[k]["recall"], -mos[k]["kechikish"]))
        print(f"  mos: {len(mos)} ta; tanlandi: {eng} (recall "
              f"{mos[eng]['recall']:.2f}, kechikish "
              f"{mos[eng]['kechikish']:.0f} daq)")
    shovqinli = max(natija, key=lambda k: natija[k]["yolgon_haftasiga"])
    print(f"  eng shovqinli: {shovqinli} - haftasiga "
          f"{natija[shovqinli]['yolgon_haftasiga']:.1f} yolg'on signal")
    print("  ⭐ Chegara so'rovlar soniga moslashsin - tungi shovqin 'yong'in' emas")


if __name__ == "__main__":
    main()

=== 1. Ma'lumot: 30 kun x 288 oyna ===
  oynalar: 8640, insidentli oynalar: 108
  tungi oyna: o'rtacha 110 so'rov, xato ulushi std 0.0069
  kunduzgi oyna: o'rtacha 2819 so'rov, xato ulushi std 0.0013
  bir xil P0 = 0.5%, lekin tunda shovqin ancha katta

=== 2. Qoidalar taqqoslash ===
  qoida                    alert yolg'on/hafta precision recall kechikish
  statik r > 2%               94          19.6      0.11   0.88     0 daq
  statik r > 5%                4           0.0      1.00   0.38     0 daq
  statik r > 2%, 15 daq        6           0.0      1.00   0.62    10 daq
  o'rtacha + 3 std           126          26.6      0.10   0.88     0 daq
  o'rtacha + 5 std            31           4.7      0.35   0.75     0 daq
  binomial z > 4              33           3.0      0.61   1.00     0 daq
  binomial z > 4, 10 daq      10           0.0      1.00   0.88     5 daq

=== 3. Qaysi insidentlar topildi ===
  statik r > 5%            XX--X---  (1-8)
  statik r > 2%, 15 daq    XXX--X-X  (1-8)
  binomial z > 4, 10 daq   XXXXXX-X  (1-8)
  1: +5.0% 30 daq, 2: +3.0% 60 daq, 3: +1.5% 15 daq, 4: +0.8% 120 daq
  5: +10.0% 10 daq, 6: +2.0% 45 daq, 7: +0.5% 240 daq, 8: +4.0% 20 daq

=== 4. Qaror: yolg'on <= 1/hafta, eng yuqori recall ===
  mos: 3 ta; tanlandi: binomial z > 4, 10 daq (recall 0.88, kechikish 5 daq)
  eng shovqinli: o'rtacha + 3 std - haftasiga 26.6 yolg'on signal
  ⭐ Chegara so'rovlar soniga moslashsin - tungi shovqin 'yong'in' emas

"""SLO/SLI, xato byudjeti va ko'p oynali burn-rate ogohlantirishlari.
30 virtual kun, 5 daqiqalik oynalar, 5 ta ma'lum insident."""

import numpy as np
import pandas as pd

OYNA = 5                       # daqiqa
KUNGA = 288
KUNLAR = 30
SLO = 0.997                    # 30 kunda so'rovlarning 99.7% "yaxshi"
BYUDJET = 1 - SLO
ASOS_YOMON = 0.001             # odatiy yomon ulush (5xx yoki > 300 ms)

# (nom, kun, soat, davomiylik daqiqa, qo'shimcha yomon ulush)
INSIDENTLAR = [("A katta xato", 3, 10, 60, 0.20),
               ("B tungi xato", 9, 2, 180, 0.03),
               ("C yarim uzilish", 14, 14, 20, 0.50),
               ("D sekin regressiya", 20, 0, 3 * 24 * 60, 0.006),
               ("E qisqa xato", 27, 13, 30, 0.10)]

# Google SRE Workbook g'oyasi: (nom, uzun oyna, qisqa oyna, burn chegarasi)
QOIDALAR = [("page: tez yonish", 12, 1, 14.4),       # 1 soat / 5 daq
            ("page: sekin yonish", 72, 6, 6.0),      # 6 soat / 30 daq
            ("ticket", 864, 72, 1.0)]                # 3 kun / 6 soat


def malumot(rng):
    t = np.arange(KUNLAR * KUNGA)
    soat = (t % KUNGA) * OYNA / 60
    n = rng.poisson(200 + 2800 * np.exp(-((soat - 14) / 4.5) ** 2))
    p = np.full(len(t), ASOS_YOMON)
    ins = np.full(len(t), -1)
    for i, (_, kun, s, davom, q) in enumerate(INSIDENTLAR):
        b = kun * KUNGA + s * 60 // OYNA
        p[b:b + davom // OYNA] += q
        ins[b:b + davom // OYNA] = i
    return pd.DataFrame({"n": n, "yomon": rng.binomial(n, p), "ins": ins})


def burn(df, oyna):
    y = df["yomon"].rolling(oyna, min_periods=1).sum()
    n = df["n"].rolling(oyna, min_periods=1).sum()
    return (y / n) / BYUDJET


def main() -> None:
    rng = np.random.default_rng(5)
    df = malumot(rng)

    print("=== 1. SLI va xato byudjeti (30 kun) ===")
    jami, yomon = df["n"].sum(), df["yomon"].sum()
    sli = 1 - yomon / jami
    byudjet = BYUDJET * jami
    print(f"  so'rovlar: {jami}, yomon: {yomon}")
    print(f"  SLI = {sli:.4%}, SLO = {SLO:.1%} -> "
          f"{'bajarildi' if sli >= SLO else 'BUZILDI'}")
    print(f"  byudjet: {byudjet:.0f} yomon so'rov, sarflandi "
          f"{yomon / byudjet:.1%}")
    print(f"  ruxsat etilgan to'liq uzilish: 30 kunda "
          f"{BYUDJET * 30 * 24 * 60:.0f} daqiqa")

    print("\n=== 2. Byudjet sarfi vaqt bo'yicha ===")
    kunlik = df.groupby(np.arange(len(df)) // KUNGA)[["n", "yomon"]].sum()
    kum = kunlik["yomon"].cumsum() / byudjet
    muzlatish = next((k for k in kum.index if kum[k] >= 1.0), None)
    for kun in [2, 3, 8, 9, 13, 14, 19, 22, 26, 27, 29]:
        belgi = " <- byudjet tugadi" if muzlatish == kun else ""
        print(f"  {kun:>2}-kun oxiri: {kum[kun]:>7.1%}{belgi}")
    if muzlatish is not None:
        print(f"  siyosat: {muzlatish}-kundan yangi funksiyalar chiqarish "
              f"to'xtatiladi, faqat ishonchlilik ishlari")
    else:
        qoldi = 1 - kum.iloc[-1]
        print(f"  byudjet tugamadi, qoldi {qoldi:.1%}")
        for i, (nom, *_) in enumerate(INSIDENTLAR):
            m = df["ins"] == i
            ulush = (df.loc[m, "yomon"].sum()
                     - ASOS_YOMON * df.loc[m, "n"].sum()) / byudjet
            if ulush > qoldi:
                print(f"  yana bitta '{nom}' ({ulush:.1%}) SLO ni buzardi")

    print("\n=== 3. Insidentlar: byudjetning qancha qismi ===")
    for i, (nom, _, _, davom, q) in enumerate(INSIDENTLAR):
        m = df["ins"] == i
        qosh = df.loc[m, "yomon"].sum() - ASOS_YOMON * df.loc[m, "n"].sum()
        print(f"  {nom:<19} {davom:>5} daq, +{q:>5.1%}: byudjetning "
              f"{qosh / byudjet:>6.1%} qismi")

    print("\n=== 4. Burn-rate ogohlantirishlari ===")
    signallar = {}
    for nom, uzun, qisqa, chegara in QOIDALAR:
        signallar[nom] = (burn(df, uzun) > chegara) & (burn(df, qisqa) > chegara)
    signallar[f"sodda: 5 daq > {BYUDJET:.1%}"] = df["yomon"] / df["n"] > BYUDJET
    ustun = "yolg'on"
    print(f"  {'qoida':<22} {'alertlar':>8} {ustun:>7}  "
          f"birinchi signal (daq, insident boshidan)")
    for nom, s in signallar.items():
        s = s.to_numpy()
        boshlar = np.flatnonzero(s & ~np.r_[False, s[:-1]])
        yaqin = []
        for b in boshlar:        # insident ichida yoki tugaganidan 6 soat ichida
            oyna = df["ins"].to_numpy()[max(0, b - 72):b + 1]
            yaqin.append(oyna.max() >= 0)
        yolgon = len(boshlar) - sum(yaqin)
        kech = []
        for i, (qisqa_nom, kun, soat, davom, _) in enumerate(INSIDENTLAR):
            b0 = kun * KUNGA + soat * 60 // OYNA
            ichida = boshlar[(boshlar >= b0) & (boshlar < b0 + davom // OYNA)]
            kech.append(f"{qisqa_nom[0]}:{(ichida[0] - b0) * OYNA}"
                        if len(ichida) else f"{qisqa_nom[0]}:-")
        print(f"  {nom:<22} {len(boshlar):>8} {yolgon:>7}  {' '.join(kech)}")
    print("\n=== 5. SLO tanlash: bir xil tizim, turli maqsad ===")
    for slo in [0.995, 0.997, 0.999]:
        b = 1 - slo
        sarf = yomon / (b * jami)
        holat = "bajarildi" if sli >= slo else "BUZILDI"
        print(f"  SLO {slo:.1%}: odatiy fon byudjetning "
              f"{ASOS_YOMON / b:>4.0%} ini yeydi, jami sarf {sarf:>6.1%} "
              f"-> {holat}")
    print("  ⭐ Byudjet - ishonchlilik va tezlik orasidagi kelishuv; "
          "ogohlantirish - byudjet yonish tezligiga")


if __name__ == "__main__":
    main()

=== 1. SLI va xato byudjeti (30 kun) ===
  so'rovlar: 9762495, yomon: 27471
  SLI = 99.7186%, SLO = 99.7% -> bajarildi
  byudjet: 29287 yomon so'rov, sarflandi 93.8%
  ruxsat etilgan to'liq uzilish: 30 kunda 130 daqiqa

=== 2. Byudjet sarfi vaqt bo'yicha ===
   2-kun oxiri:    3.3%
   3-kun oxiri:   18.4%
   8-kun oxiri:   23.9%
   9-kun oxiri:   25.8%
  13-kun oxiri:   30.0%
  14-kun oxiri:   51.5%
  19-kun oxiri:   57.1%
  22-kun oxiri:   80.3%
  26-kun oxiri:   84.7%
  27-kun oxiri:   91.5%
  29-kun oxiri:   93.8%
  byudjet tugamadi, qoldi 6.2%
  yana bitta 'A katta xato' (14.0%) SLO ni buzardi
  yana bitta 'C yarim uzilish' (20.4%) SLO ni buzardi
  yana bitta 'D sekin regressiya' (19.8%) SLO ni buzardi

=== 3. Insidentlar: byudjetning qancha qismi ===
  A katta xato           60 daq, +20.0%: byudjetning  14.0% qismi
  B tungi xato          180 daq, + 3.0%: byudjetning   0.8% qismi
  C yarim uzilish        20 daq, +50.0%: byudjetning  20.4% qismi
  D sekin regressiya   4320 daq, + 0.6%: byudjetning  19.8% qismi
  E qisqa xato           30 daq, +10.0%: byudjetning   5.8% qismi

=== 4. Burn-rate ogohlantirishlari ===
  qoida                  alertlar yolg'on  birinchi signal (daq, insident boshidan)
  page: tez yonish              3       0  A:10 B:- C:5 D:- E:25
  page: sekin yonish            2       0  A:10 B:- C:5 D:- E:-
  ticket                        3       0  A:30 B:- C:5 D:1285 E:-
  sodda: 5 daq > 0.3%         664     559  A:0 B:- C:0 D:5 E:0

=== 5. SLO tanlash: bir xil tizim, turli maqsad ===
  SLO 99.5%: odatiy fon byudjetning  20% ini yeydi, jami sarf  56.3% -> bajarildi
  SLO 99.7%: odatiy fon byudjetning  33% ini yeydi, jami sarf  93.8% -> bajarildi
  SLO 99.9%: odatiy fon byudjetning 100% ini yeydi, jami sarf 281.4% -> BUZILDI
  ⭐ Byudjet - ishonchlilik va tezlik orasidagi kelishuv; ogohlantirish - byudjet yonish tezligiga

dashboard: so'rov/s, xato %, p99, CPU                      # ⚠️
+ null ulushi, sxema xatolari, o'rtacha ball, ball > 0.5,  # ✅
  model_versiya bo'yicha trafik, biznes metrikasi

p99 = np.mean([replika.p99 for replika in replikalar])      # ⚠️
p99 = histogram_quantile(0.99, sum(bucketlar))              # ✅

SOROVLAR.labels(foydalanuvchi=uid).inc()                    # ⚠️ millionlab qator
SOROVLAR.labels(yol="/predict", status="200").inc()         # ✅ id - logda

print(f"bashorat {ball} mijoz {mijoz_ismi} uchun")          # ⚠️ matn + shaxsiy
log_json(sorov_id=sid, hodisa="bashorat", ball=ball,        # ✅
         model_versiya="churn-v3")

expr: xato_ulushi > 0.02                                    # ⚠️ tunda 3 xato = 2.7%
expr: binomial_z > 4 (ketma-ket 2 oyna, n >= minimal)       # ✅

expr: yomon_ulush_5m > 0.003                                # ⚠️ yuzlab alert
expr: burn_1h > 14.4 and burn_5m > 14.4                     # ✅ page

tezlik = (joriy - oldingi) / soniya                         # ⚠️ manfiy tezlik
tezlik = rate(oldingi, joriy, soniya)                       # ✅ reset hisobga

1.  ML monitoringining to'rt qatlami qaysilar?
2.  Belgilar kechikishi nima va unga qarshi nima qilinadi?
3.  Counter va gauge farqi?
4.  Nima uchun persentillarni o'rtachalab bo'lmaydi?
5.  histogram_quantile xatosi nimaga bog'liq?
6.  Nima uchun foydalanuvchi id yorliq bo'lmasligi kerak?
7.  So'rov id nima beradi?
8.  Tunda statik chegara nima uchun ko'p yolg'on signal beradi?
9.  "for: 15m" ning kamchiligi?
10. Xato byudjeti qanday hisoblanadi?
11. Burn rate = 1 nimani bildiradi?
12. Nima uchun ikki oyna (uzun va qisqa) ishlatiladi?

1. p99 = (replika1_p99 + replika2_p99 + replika3_p99) / 3
2. KECHIKISH = Histogram(..., buckets=[0.005, 0.01, 0.1, 1, 10])   # SLA 300 ms
3. SOROVLAR.labels(mijoz_id=mijoz.id, yol=yol).inc()
4. log.info(f"Mijoz {ism} ({telefon}) uchun ball {ball}")
5. alert: xato_ulushi_5m > 0.01   (tunda 100 so'rov/oyna)
6. SLO = 99.99%, fon xato ulushi 0.05%