29.1-dars: Data Science loyihasining hayot sikli

   +--------------------------+
   | 1. BIZNESNI TUSHUNISH    |<----------------------------+
   +--------------------------+                             |
         |            ^                                     |
         v            |                                     |
   +--------------------------+                             |
   | 2. MA'LUMOTNI TUSHUNISH  |                             |
   +--------------------------+                             |
         |                                                  |
         v                                                  |
   +--------------------------+     +--------------------+  |
   | 3. MA'LUMOT TAYYORLASH   |<--->| 4. MODELLASHTIRISH |  |
   +--------------------------+     +--------------------+  |
                                              |             |
                                              v             |
                                      +--------------+      |
                                      | 5. BAHOLASH  |------+  yo'q
                                      +--------------+
                                              |  ha (biznes mezoni bajarildi)
                                              v
                                      +--------------+
                                      | 6. JORIY     |
                                      |    ETISH     |
                                      +--------------+

1. QAROR NIMA?         qaysi mijozga qo'ng'iroq qilish (eslatma + restrukturizatsiya taklifi)
2. KIM QABUL QILADI?   qo'ng'iroq markazi rahbari; operatorlar ro'yxat bo'yicha ishlaydi
3. QACHON?             har oy oxirida (hisobvaraq chiqqanda), keyingi oy davomida qo'ng'iroq
4. QANDAY CHEKLOV?     oyiga K = 200 qo'ng'iroq (kuniga ~10 ta, ~20 ish kuni)
5. NIMA O'ZGARADI?     qo'ng'iroq qilingan xavfli mijozlarning bir qismi 60+ kunga tushmaydi
                       -> bank zarari kamayadi; qo'ng'iroq narxi -> xarajat

         |  tarjima
         v
TAHLIL BIRLIGI:   (mijoz, as-of oy) - oy oxiridagi holat
NOMZODLAR:        hozir kechikmagan (oxirgi hisobvaraq to'langan) faol mijozlar
AS-OF (vaqt nuqtasi): t-oy oxiri; belgilar FAQAT shu paytgacha ma'lum ma'lumotdan
GORIZONT:         keyingi 60 kun (t va t+1 hisobvaraqlari)
BELGI (label):    y60 = ikkala hisobvaraq bo'yicha minimal to'lov qilinmagan (60+ kun)
MODEL CHIQISHI:   p = P(y60 = 1 | t gacha ma'lumot) - kalibrlangan ehtimol
SIYOSAT:          har oy p * zarar bo'yicha top-200 ga qo'ng'iroq
BIZNES METRIKASI: tejalgan zarar - qo'ng'iroq xarajati (mln so'm/oy)

BIZNES METRIKASI     tejalgan zarar - xarajat (mln so'm/oy)     <- rahbariyat shu bilan qaror qiladi
      ^
QAROR METRIKASI      tejam@K, precision@K, recall@K              <- siyosat sifatini o'lchaydi
      ^
MODEL METRIKASI      AUC, log-loss, Brier, kalibrlash            <- model ichki sifati
      +
TO'SIQ (guardrail)   shikoyatlar, qayta qo'ng'iroqlar, segmentlar bo'yicha adolat (29.11)

                              qo'ng'iroq qilindi            qo'ng'iroq qilinmadi
mijoz DPD60 ga tushadi (TP/FN)  +0.30 * zarar - 0.02          0  (zarar to'liq qoladi)
mijoz tushmaydi       (FP/TN)   -0.02                         0

zarar = 0.45 * balans (faraziy LGD), 0.30 - qo'ng'iroq samarasi (faraziy), 0.02 = 20 ming so'm
tejam(S) = sum_{i in S} ( 0.30 * zarar_i * y60_i - 0.02 )
qo'ng'iroq foydali, agar  p_i * 0.30 * zarar_i > 0.02   (p_i kalibrlangan bo'lishi SHART)

QIYMAT ZINAPOYASI (oyiga tejam, har loyihada o'lchanadi):
  0. hech narsa qilmaslik              0          <- har doim variant!
  1. tasodifiy K ta qo'ng'iroq         ~0         <- reyting qiymati shu bilan o'lchanadi
  2. oddiy qoida (util eng yuqori)     ?          <- bitta ustun, 5 daqiqa
  3. ekspert ball-kartasi              ?          <- bo'lim tajribasi
  4. oddiy model (LogReg)              ?
  5. murakkab model (HistGB, tarmoq)   ?
  6. oracle (kelajakni biladi)         yuqori chegara - erishib bo'lmaydi

MODEL QIYMATI = model - ENG YAXSHI QOIDA   (tasodifiydan emas!)
MODEL XARAJATI = ishlab chiqish + ma'lumot quvuri + monitoring + qayta o'qitish + xatar

BOSQICH 0. RAMKA (1-2 hafta)
  loyiha hujjati, biznes metrikasi, xarajat parametrlari (faraziy -> moliya tasdiqlaydi)
  ma'lumot inventari, kirish huquqlari, oracle va qoida bazaviysi
  DARVOZA: oracle qiymati loyiha xarajatidan sezilarli katta?  yo'q -> TO'XTATISH

BOSQICH 1. TEZ PROTOTIP (2-4 hafta)
  bitta jadval, sizishsiz belgilar, oddiy qoida va LogReg, bitta vaqt bo'yicha val
  DARVOZA: model eng yaxshi qoidadan yaxshiroqmi yoki hech bo'lmaganda istiqbollimi?

BOSQICH 2. TO'LIQ BAHOLASH (3-6 hafta)
  sifat tekshiruvlari, vaqt bo'yicha CV, nomzodlar, kalibrlash, siyosat, segmentlar
  oldindan yozilgan baholash rejasi -> test BIR MARTA -> model kartasi
  DARVOZA: test da biznes mezoni bajarildimi?

BOSQICH 3. PILOT (dastlabki taxmin 2-3 oy; 29.4 da namuna hajmi bilan tekshiriladi)
  mijozlarning bir qismida A/B: model ro'yxati vs qoida ro'yxati vs qo'ng'iroqsiz
  DARVOZA: haqiqiy samara (0.30 faraziyi) tasdiqlandimi?

BOSQICH 4. ISHLAB CHIQARISH
  quvur, xizmat, monitoring, qayta o'qitish qoidasi, doimiy nazorat guruhi (27-qism)

# Loyiha hujjati: kredit karta kechikishini oldindan aniqlash

**Egasi:** chakana kredit risk bo'limi · **DS jamoasi:** 2 kishi · **Versiya:** 0.3 (namuna)

## 1. Biznes muammosi
60+ kunlik kechikishga tushgan kredit karta hisoblari bank uchun asosiy zarar manbai.
Qo'ng'iroq markazi oyiga ~200 profilaktik qo'ng'iroq qila oladi, lekin hozir ro'yxat
qo'lda tuziladi (limitdan foydalanish ulushi bo'yicha).

## 2. Qaror va foydalanuvchi
- Qaror: har oy oxirida qaysi 200 mijozga qo'ng'iroq qilish.
- Foydalanuvchi: qo'ng'iroq markazi rahbari; ro'yxat CRM ga tushadi.
- Harakat: eslatma + to'lov jadvalini qayta ko'rib chiqish taklifi.

## 3. DS vazifasi
- Tahlil birligi: (mijoz, as-of oy); nomzodlar - hozir kechikmagan faol mijozlar.
- Belgi: as-of dan keyingi 60 kunda 60+ kun kechikish (y60).
- Chiqish: kalibrlangan ehtimol p; siyosat: p * kutilgan zarar bo'yicha top-K.

## 4. Muvaffaqiyat mezoni
- Asosiy: oylik tejam = 0.30 * sum(zarar * y60) - 0.02 * K (mln so'm), test oylarida.
- Talab: joriy qoidadan juftlashgan farq > 2*SE.
- Yordamchi: precision@200, recall@200, AUC, kalibrlash (bashorat / haqiqiy ulush).
- To'siqlar: segmentlar (hudud, yangi mijozlar) bo'yicha keskin farq yo'q; shikoyatlar oshmaydi.

## 5. Bazaviylar
Hech narsa; tasodifiy 200; qoida "util eng yuqori"; ekspert ball-kartasi.

## 6. Ma'lumot
mijozlar, hisobvaraqlar, to'lovlar, tranzaksiyalar (2024-01 dan boshlab), omborga kirish bor.
As-of qoidasi: faqat sanasi <= as-of bo'lgan yozuvlar.

## 7. Farazlar va xatarlar
- Faraziy: qo'ng'iroq samarasi 30%, qo'ng'iroq narxi 20 ming so'm, zarar = 45% balans
  -> pilotda va moliya bo'limida tasdiqlanadi.
- Belgi yetilishi 2 oy; qaror halqasi -> 5% nazorat guruhi doimiy.
- Drift (makroiqtisodiy) -> oylik monitoring.

## 8. Reja va darvozalar
Ramka (2 hafta) -> prototip (3 hafta) -> baholash (4 hafta) -> pilot (muddat - namuna hajmi hisobidan) -> ishlab chiqarish.
To'xtatish mezoni: prototipda model qoidadan ustun emas va oracle qiymati past.

## 9. Doiradan tashqari
Kredit limitini o'zgartirish, undiruv (collection) strategiyasi, yangi mijozlarni skoring.

import numpy as np

# qo'ng'iroq qiymati (xarajat matritsasi): qo'ng'iroq qilinsa, qo'ng'iroqsizga nisbatan
qiymat = SAMARA * zarar * y60 - NARX          # TP: +0.30*z-0.02, FP: -0.02

# har oy top-K siyosatining tejami (biznes metrikasi)
def tejam_k(qiymat, ball, guruhlar, k=200):
    return sum(qiymat[g[np.argsort(-ball[g])[:k]]].sum() for g in guruhlar) / len(guruhlar)

# qiymat zinapoyasi
tejam_k(qiymat, rng.random(n), guruhlar)       # tasodifiy ~0
tejam_k(qiymat, df.util.to_numpy(), guruhlar)  # oddiy qoida
tejam_k(qiymat, p * zarar, guruhlar)           # model: kutilgan zarar bo'yicha
tejam_k(qiymat, y60 * zarar, guruhlar)         # oracle - yuqori chegara

# qaror qoidasi: eng yaxshisidan sezilarli yomon bo'lmagan eng sodda
farq = baho[nom] - baho[eng]
sezilarli_yomon = farq < -2 * se               # se - juftlashgan bootstrap yoki oylar bo'yicha

istak -> qaror (kim, qachon, sig'im) -> DS vazifasi (birlik, as-of, gorizont, belgi)
metrikalar: biznes > qaror (top-K) > model (AUC) + to'siqlar
qiymat zinapoyasi: hech narsa, tasodifiy, qoida, model, oracle
xatarlar: mavjudlik, sizish, belgi ta'rifi va yetilishi, qaror halqasi, drift
reja: ramka -> prototip -> baholash -> pilot -> ishlab chiqarish (darvozalar bilan)

"""Biznes savolidan DS vazifasiga: tahlil birligi, xarajat matritsasi, qiymat chegaralari."""

import numpy as np
import pandas as pd

HUDUDLAR = ["Toshkent", "Samarqand", "Farg'ona", "Andijon", "Buxoro", "Qashqadaryo"]
OYLAR = 24                               # 1 = 2024-01, ..., 24 = 2025-12
BELGILAR = ["yosh", "daromad", "limit", "avto_tolov", "maosh_loyihasi", "staj",
            "util", "util_3", "util_ozg", "kechikish_6", "tolov_nisbati_3",
            "tolov_kuni_3", "naqd_soni_3", "naqd_summa_3", "xarid_soni_3"]
K, SAMARA, NARX = 200, 0.30, 0.02        # oyiga qo'ng'iroq, oldini olish, mln so'm


def sigmoid(z):
    return 1 / (1 + np.exp(-z))


def oy_nomi(m):
    return f"{2024 + (m - 1) // 12}-{(m - 1) % 12 + 1:02d}"


def yarat_bank(seed=29, n=6000):
    """Kredit karta portfeli: mijozlar, hisobvaraqlar, tolovlar, tranzaksiyalar."""
    rng = np.random.default_rng(seed)
    ids = np.arange(1, n + 1)
    yosh = rng.integers(21, 66, n)
    maosh = rng.random(n) < 0.45
    daromad = np.round(np.exp(rng.normal(np.log(6.0), 0.45, n)), 2)   # mln so'm
    ochilgan = np.where(rng.random(n) < 0.7, rng.integers(-40, 1, n),
                        rng.integers(1, 20, n))
    xavf = (rng.normal(0, 1, n) - 0.5 * maosh - 0.8 * np.log(daromad / 6)
            + 0.015 * (35 - yosh))                     # yashirin: moliyaviy xavf
    unut = rng.normal(0, 1, n) + 0.03 * (35 - yosh)     # yashirin: unutuvchanlik
    avto = (rng.random(n) < sigmoid(-0.2 - 0.9 * unut)).astype(int)
    limit = np.round(daromad * rng.uniform(1.5, 3.5, n), 1)
    hudud = rng.choice(HUDUDLAR, n, p=[0.34, 0.15, 0.14, 0.13, 0.12, 0.12])
    mj = pd.DataFrame({"mijoz_id": ids, "yosh": yosh, "hudud": hudud,
                       "daromad": daromad, "limit": limit, "avto_tolov": avto,
                       "maosh_loyihasi": maosh.astype(int), "ochilgan_oy": ochilgan})
    s, oldingi, ketma = rng.normal(0, 1, n), np.zeros(n, int), np.zeros(n, int)
    hv, tl, tr = [], [], []
    for m in range(1, OYLAR + 1):
        faol = (ochilgan <= m) & (ketma < 3)            # 3 ketma-ket kechikish - defolt
        s = 0.75 * s + 0.66 * rng.normal(0, 1, n)
        d = 0.8 * xavf + 0.7 * s + 0.15 * (m % 12 == 1) + 0.08 * max(m - 18, 0)
        util = np.clip(sigmoid(-0.4 + 0.55 * d + rng.normal(0, 0.5, n)), 0.01, 1)
        balans = np.round(util * limit, 3)
        min_t = np.round(np.maximum(0.05 * balans, 0.05), 3)
        for turi, lam in [("xarid", np.exp(1.2 + 0.15 * np.log(daromad / 6) - 0.1 * d)),
                          ("naqd", np.exp(-1.6 + 0.6 * d)),
                          ("onlayn", np.exp(0.6 - 0.02 * (yosh - 35)))]:
            i = np.repeat(np.arange(n), rng.poisson(lam) * faol)
            summa = (rng.uniform(0.2, 1.5, i.size) * (1 + 0.3 * d[i].clip(0))
                     if turi == "naqd" else np.exp(rng.normal(np.log(0.25), 0.8, i.size)))
            tr.append(pd.DataFrame({"mijoz_id": ids[i], "oy": m,
                                    "kun": rng.integers(1, 29, i.size), "turi": turi,
                                    "summa": np.round(summa, 3)}))
        q = rng.random(n) < sigmoid(-3.7 + 1.3 * d + 1.2 * (util > 0.85)
                                    + 3.8 * (oldingi == 2))          # qiyinchilik
        u = ~q & (rng.random(n) < sigmoid(-3.4 + 1.1 * unut - 1.8 * avto)
                  * (oldingi != 1))                                  # unutish
        miss = (q | u) & faol
        ulush = sigmoid(0.3 - 1.2 * d + rng.normal(0, 0.5, n))
        summa = np.where(miss, min_t * rng.uniform(0, 0.6, n),
                         min_t + (balans - min_t).clip(0) * ulush)
        tolaydi = faol & (~miss | (rng.random(n) < 0.4))
        kun = np.clip(np.round(6 + 14 * sigmoid(unut + 0.5 * d)
                               + rng.normal(0, 3, n)), 1, 25)
        hv.append(pd.DataFrame({"mijoz_id": ids[faol], "oy": m,
                                "balans": balans[faol], "min_tolov": min_t[faol]}))
        if m < OYLAR:                    # m-oy hisobvarag'i (m+1)-oyda to'lanadi
            tl.append(pd.DataFrame({"mijoz_id": ids[tolaydi], "hisob_oy": m,
                                    "tolov_oy": m + 1,
                                    "tolov_kuni": kun[tolaydi].astype(int),
                                    "summa": np.round(summa[tolaydi], 3)}))
        ketma = np.where(faol, np.where(miss, ketma + 1, 0), ketma)
        oldingi = np.where(miss, np.where(q, 2, 1), 0)
    return {"mijozlar": mj, "hisobvaraqlar": pd.concat(hv, ignore_index=True),
            "tolovlar": pd.concat(tl, ignore_index=True),
            "tranzaksiyalar": pd.concat(tr, ignore_index=True)}


def orta(a):
    """Qator bo'yicha NaN siz o'rtacha (bo'sh qatorda NaN, ogohlantirishsiz)."""
    k = (~np.isnan(a)).sum(axis=1)
    return np.where(k > 0, np.nansum(a, axis=1) / np.maximum(k, 1), np.nan)


def belgilar_jadvali(bank, oylar=range(7, 23)):
    """Har as-of oy t (oy oxiri): faqat t gacha ma'lum belgilar + t, t+1 natijasi."""
    mj = bank["mijozlar"]
    ids, kol = mj["mijoz_id"].to_numpy(), np.arange(1, OYLAR + 1)

    def mat(df, qiymat, ustun="oy"):
        return (df.pivot(index="mijoz_id", columns=ustun, values=qiymat)
                .reindex(index=ids, columns=kol).to_numpy(dtype=float))

    hv, tl = bank["hisobvaraqlar"], bank["tolovlar"]
    bal, mint = mat(hv, "balans"), mat(hv, "min_tolov")
    tol, kun = mat(tl, "summa", "hisob_oy"), mat(tl, "tolov_kuni", "hisob_oy")
    bor = ~np.isnan(bal)
    miss = bor & ~(np.nan_to_num(tol) >= mint - 1e-9)    # minimal to'lov qilinmagan
    g = (bank["tranzaksiyalar"].groupby(["turi", "mijoz_id", "oy"])["summa"]
         .agg(["size", "sum"]))

    def tr_mat(turi, ust):
        return (g.loc[turi, ust].unstack("oy").reindex(index=ids, columns=kol)
                .fillna(0).to_numpy())

    naqd_n, naqd_s = tr_mat("naqd", "size"), tr_mat("naqd", "sum")
    xarid_n = tr_mat("xarid", "size")
    util = bal / mj["limit"].to_numpy()[:, None]
    och = mj["ochilgan_oy"].to_numpy()
    qismlar = []
    for t in oylar:
        c = t - 1                                        # t-oy ustuni (0 dan)
        ok = (och <= t - 1) & bor[:, c] & bor[:, c - 1] & ~miss[:, c - 1]
        u3 = util[:, c - 3]
        f = mj.drop(columns="ochilgan_oy").assign(
            oy=t, staj=t - och, util=util[:, c], util_3=orta(util[:, c - 2:c + 1]),
            util_ozg=util[:, c] - np.where(np.isnan(u3), util[:, c], u3),
            kechikish_6=miss[:, max(c - 6, 0):c].sum(axis=1),
            tolov_nisbati_3=orta(np.nan_to_num(tol[:, c - 3:c]) / bal[:, c - 3:c]),
            tolov_kuni_3=orta(kun[:, c - 3:c]),
            naqd_soni_3=naqd_n[:, c - 2:c + 1].sum(axis=1),
            naqd_summa_3=naqd_s[:, c - 2:c + 1].sum(axis=1),
            xarid_soni_3=xarid_n[:, c - 2:c + 1].sum(axis=1),
            balans=bal[:, c], y30=(miss[:, c] | miss[:, c + 1]).astype(int),
            y60=(miss[:, c] & miss[:, c + 1]).astype(int))
        qismlar.append(f[ok])
    df = pd.concat(qismlar, ignore_index=True)
    df["tolov_kuni_3"] = df["tolov_kuni_3"].fillna(25.0)
    df["tolov_nisbati_3"] = df["tolov_nisbati_3"].fillna(0.0)
    df["zarar"] = 0.45 * df["balans"]              # DPD60 bo'lsa kutilgan zarar
    df["qiymat"] = SAMARA * df["zarar"] * df["y60"] - NARX   # qo'ng'iroq qiymati
    return df


def top_k_qiymat(df, ball, k=K):
    """Har as-of oyda ball bo'yicha top-k ga qo'ng'iroq; oylik tejam (mln so'm)."""
    natija = []
    for _, g in df.assign(_b=ball).groupby("oy"):
        natija.append(g.nlargest(k, "_b")["qiymat"].sum())
    return np.array(natija)


def main() -> None:
    bank = yarat_bank()
    print("=== 1. Manba jadvallari (xom ma'lumot) ===")
    for nom in ["mijozlar", "hisobvaraqlar", "tolovlar", "tranzaksiyalar"]:
        print(f"  {nom:<15} {len(bank[nom]):>8} qator")
    print(f"  davr: {oy_nomi(1)} - {oy_nomi(OYLAR)}")

    df = belgilar_jadvali(bank)
    oylik = df.groupby("oy").agg(n=("y60", "size"), y30=("y30", "mean"),
                                 y60=("y60", "mean"), dpd60=("y60", "sum"))
    print("\n=== 2. Tahlil birligi: (mijoz, as-of oy) ===")
    print(f"  qatorlar: {len(df)}, as-of oylar: {oy_nomi(df.oy.min())} - "
          f"{oy_nomi(df.oy.max())} ({df.oy.nunique()} oy)")
    print(f"  oyiga nomzod mijozlar (hozir kechikmagan): {oylik.n.min()} - "
          f"{oylik.n.max()}")
    print(f"  60 kun ichida 30+ kun kechikish (y30): {df.y30.mean():.3f}")
    print(f"  60 kun ichida 60+ kun kechikish (y60): {df.y60.mean():.3f}")
    print(f"  oyiga DPD60 holatlari: o'rtacha {oylik.dpd60.mean():.1f} "
          f"(min {oylik.dpd60.min()}, max {oylik.dpd60.max()})")
    print(f"  qo'ng'iroq sig'imi: oyiga K = {K} ta = nomzodlarning "
          f"{K / oylik.n.mean():.1%} i")

    print("\n=== 3. Bitta qo'ng'iroqning qiymati (xarajat matritsasi, mln so'm) ===")
    z60 = df.loc[df.y60 == 1, "zarar"]
    print(f"  DPD60 bo'lgan mijozning kutilgan zarari: o'rtacha {z60.mean():.2f}, "
          f"mediana {z60.median():.2f}")
    print("  qo'ng'iroq qilinsa (qo'ng'iroqsiz holatga nisbatan):")
    print(f"    mijoz DPD60 ga tushadigan bo'lsa (TP):  +{SAMARA} * zarar - {NARX}")
    print(f"    mijoz DPD60 ga tushmaydigan bo'lsa (FP): -{NARX}")
    print("  qo'ng'iroq qilinmasa: 0 (FN da zarar to'liq qoladi, TN - hech narsa)")
    ulush = z60.sort_values(ascending=False)
    top20 = ulush.iloc[: int(0.2 * len(ulush))].sum() / ulush.sum()
    print(f"  zarar konsentratsiyasi: eng katta 20% DPD60 zararning {top20:.1%} i")
    p_chegara = NARX / (SAMARA * z60.median())
    print(f"  zararsizlik chegarasi (mediana zararda): p > {p_chegara:.3f}; "
          f"bazaviy ulush {df.y60.mean():.3f}")

    print("\n=== 4. Qiymat chegaralari: val oylari 16-18, oyiga tejam (mln so'm) ===")
    va = df[df.oy.between(16, 18)].copy()
    rng = np.random.default_rng(0)
    tasodifiy = np.mean([top_k_qiymat(va, rng.random(len(va))).mean()
                         for _ in range(50)])
    chegaralar = {
        "hech kimga qo'ng'iroq yo'q": 0.0,
        "tasodifiy 200 ta (50 marta)": tasodifiy,
        "qoida: util eng yuqori": top_k_qiymat(va, va.util).mean(),
        "oracle (kelajakni biladi)": top_k_qiymat(va, va.y60 * va.zarar).mean(),
    }
    for nom, v in chegaralar.items():
        print(f"  {nom:<30} {v:>+8.2f}")
    toliq = (SAMARA * va.zarar * va.y60).groupby(va.oy).sum().mean()
    print(f"  oldini olish mumkin bo'lgan jami zarar (hammaga qo'ng'iroq): {toliq:.1f}")

    print("\n=== 5. Xulosa (natijadan hisoblangan) ===")
    qoida = chegaralar["qoida: util eng yuqori"]
    oracle = chegaralar["oracle (kelajakni biladi)"]
    if abs(tasodifiy) < 0.05 * qoida:
        print(f"  tasodifiy qo'ng'iroq deyarli nol ({tasodifiy:+.2f}) -> reyting shart")
    print(f"  oddiy qoida oracle qiymatining {qoida / oracle:.0%} ini beradi")
    print(f"  model uchun o'yin maydoni: oyiga ko'pi bilan {oracle - qoida:.1f} mln so'm "
          f"(amalda ancha kam)")
    if oylik.dpd60.mean() < K:
        print("  sig'im DPD60 holatlaridan katta -> muammo reytingda, sig'imda emas")


if __name__ == "__main__":
    main()

=== 1. Manba jadvallari (xom ma'lumot) ===
  mijozlar            6000 qator
  hisobvaraqlar     111058 qator
  tolovlar          100780 qator
  tranzaksiyalar    586885 qator
  davr: 2024-01 - 2025-12

=== 2. Tahlil birligi: (mijoz, as-of oy) ===
  qatorlar: 68532, as-of oylar: 2024-07 - 2025-10 (16 oy)
  oyiga nomzod mijozlar (hozir kechikmagan): 4064 - 4475
  60 kun ichida 30+ kun kechikish (y30): 0.109
  60 kun ichida 60+ kun kechikish (y60): 0.020
  oyiga DPD60 holatlari: o'rtacha 87.6 (min 66, max 116)
  qo'ng'iroq sig'imi: oyiga K = 200 ta = nomzodlarning 4.7% i

=== 3. Bitta qo'ng'iroqning qiymati (xarajat matritsasi, mln so'm) ===
  DPD60 bo'lgan mijozning kutilgan zarari: o'rtacha 3.64, mediana 3.12
  qo'ng'iroq qilinsa (qo'ng'iroqsiz holatga nisbatan):
    mijoz DPD60 ga tushadigan bo'lsa (TP):  +0.3 * zarar - 0.02
    mijoz DPD60 ga tushmaydigan bo'lsa (FP): -0.02
  qo'ng'iroq qilinmasa: 0 (FN da zarar to'liq qoladi, TN - hech narsa)
  zarar konsentratsiyasi: eng katta 20% DPD60 zararning 38.9% i
  zararsizlik chegarasi (mediana zararda): p > 0.021; bazaviy ulush 0.020

=== 4. Qiymat chegaralari: val oylari 16-18, oyiga tejam (mln so'm) ===
  hech kimga qo'ng'iroq yo'q        +0.00
  tasodifiy 200 ta (50 marta)       +0.01
  qoida: util eng yuqori           +24.26
  oracle (kelajakni biladi)        +78.57
  oldini olish mumkin bo'lgan jami zarar (hammaga qo'ng'iroq): 82.6

=== 5. Xulosa (natijadan hisoblangan) ===
  tasodifiy qo'ng'iroq deyarli nol (+0.01) -> reyting shart
  oddiy qoida oracle qiymatining 31% ini beradi
  model uchun o'yin maydoni: oyiga ko'pi bilan 54.3 mln so'm (amalda ancha kam)
  sig'im DPD60 holatlaridan katta -> muammo reytingda, sig'imda emas

"""Model metrikasi va biznes metrikasi: AUC va tejalgan zarar bir xil narsani o'lchamaydi."""

import numpy as np
import pandas as pd
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

HUDUDLAR = ["Toshkent", "Samarqand", "Farg'ona", "Andijon", "Buxoro", "Qashqadaryo"]
OYLAR = 24                               # 1 = 2024-01, ..., 24 = 2025-12
BELGILAR = ["yosh", "daromad", "limit", "avto_tolov", "maosh_loyihasi", "staj",
            "util", "util_3", "util_ozg", "kechikish_6", "tolov_nisbati_3",
            "tolov_kuni_3", "naqd_soni_3", "naqd_summa_3", "xarid_soni_3"]
K, SAMARA, NARX = 200, 0.30, 0.02        # oyiga qo'ng'iroq, oldini olish, mln so'm


def sigmoid(z):
    return 1 / (1 + np.exp(-z))


def oy_nomi(m):
    return f"{2024 + (m - 1) // 12}-{(m - 1) % 12 + 1:02d}"


def yarat_bank(seed=29, n=6000):
    """Kredit karta portfeli: mijozlar, hisobvaraqlar, tolovlar, tranzaksiyalar."""
    rng = np.random.default_rng(seed)
    ids = np.arange(1, n + 1)
    yosh = rng.integers(21, 66, n)
    maosh = rng.random(n) < 0.45
    daromad = np.round(np.exp(rng.normal(np.log(6.0), 0.45, n)), 2)   # mln so'm
    ochilgan = np.where(rng.random(n) < 0.7, rng.integers(-40, 1, n),
                        rng.integers(1, 20, n))
    xavf = (rng.normal(0, 1, n) - 0.5 * maosh - 0.8 * np.log(daromad / 6)
            + 0.015 * (35 - yosh))                     # yashirin: moliyaviy xavf
    unut = rng.normal(0, 1, n) + 0.03 * (35 - yosh)     # yashirin: unutuvchanlik
    avto = (rng.random(n) < sigmoid(-0.2 - 0.9 * unut)).astype(int)
    limit = np.round(daromad * rng.uniform(1.5, 3.5, n), 1)
    hudud = rng.choice(HUDUDLAR, n, p=[0.34, 0.15, 0.14, 0.13, 0.12, 0.12])
    mj = pd.DataFrame({"mijoz_id": ids, "yosh": yosh, "hudud": hudud,
                       "daromad": daromad, "limit": limit, "avto_tolov": avto,
                       "maosh_loyihasi": maosh.astype(int), "ochilgan_oy": ochilgan})
    s, oldingi, ketma = rng.normal(0, 1, n), np.zeros(n, int), np.zeros(n, int)
    hv, tl, tr = [], [], []
    for m in range(1, OYLAR + 1):
        faol = (ochilgan <= m) & (ketma < 3)            # 3 ketma-ket kechikish - defolt
        s = 0.75 * s + 0.66 * rng.normal(0, 1, n)
        d = 0.8 * xavf + 0.7 * s + 0.15 * (m % 12 == 1) + 0.08 * max(m - 18, 0)
        util = np.clip(sigmoid(-0.4 + 0.55 * d + rng.normal(0, 0.5, n)), 0.01, 1)
        balans = np.round(util * limit, 3)
        min_t = np.round(np.maximum(0.05 * balans, 0.05), 3)
        for turi, lam in [("xarid", np.exp(1.2 + 0.15 * np.log(daromad / 6) - 0.1 * d)),
                          ("naqd", np.exp(-1.6 + 0.6 * d)),
                          ("onlayn", np.exp(0.6 - 0.02 * (yosh - 35)))]:
            i = np.repeat(np.arange(n), rng.poisson(lam) * faol)
            summa = (rng.uniform(0.2, 1.5, i.size) * (1 + 0.3 * d[i].clip(0))
                     if turi == "naqd" else np.exp(rng.normal(np.log(0.25), 0.8, i.size)))
            tr.append(pd.DataFrame({"mijoz_id": ids[i], "oy": m,
                                    "kun": rng.integers(1, 29, i.size), "turi": turi,
                                    "summa": np.round(summa, 3)}))
        q = rng.random(n) < sigmoid(-3.7 + 1.3 * d + 1.2 * (util > 0.85)
                                    + 3.8 * (oldingi == 2))          # qiyinchilik
        u = ~q & (rng.random(n) < sigmoid(-3.4 + 1.1 * unut - 1.8 * avto)
                  * (oldingi != 1))                                  # unutish
        miss = (q | u) & faol
        ulush = sigmoid(0.3 - 1.2 * d + rng.normal(0, 0.5, n))
        summa = np.where(miss, min_t * rng.uniform(0, 0.6, n),
                         min_t + (balans - min_t).clip(0) * ulush)
        tolaydi = faol & (~miss | (rng.random(n) < 0.4))
        kun = np.clip(np.round(6 + 14 * sigmoid(unut + 0.5 * d)
                               + rng.normal(0, 3, n)), 1, 25)
        hv.append(pd.DataFrame({"mijoz_id": ids[faol], "oy": m,
                                "balans": balans[faol], "min_tolov": min_t[faol]}))
        if m < OYLAR:                    # m-oy hisobvarag'i (m+1)-oyda to'lanadi
            tl.append(pd.DataFrame({"mijoz_id": ids[tolaydi], "hisob_oy": m,
                                    "tolov_oy": m + 1,
                                    "tolov_kuni": kun[tolaydi].astype(int),
                                    "summa": np.round(summa[tolaydi], 3)}))
        ketma = np.where(faol, np.where(miss, ketma + 1, 0), ketma)
        oldingi = np.where(miss, np.where(q, 2, 1), 0)
    return {"mijozlar": mj, "hisobvaraqlar": pd.concat(hv, ignore_index=True),
            "tolovlar": pd.concat(tl, ignore_index=True),
            "tranzaksiyalar": pd.concat(tr, ignore_index=True)}


def orta(a):
    """Qator bo'yicha NaN siz o'rtacha (bo'sh qatorda NaN, ogohlantirishsiz)."""
    k = (~np.isnan(a)).sum(axis=1)
    return np.where(k > 0, np.nansum(a, axis=1) / np.maximum(k, 1), np.nan)


def belgilar_jadvali(bank, oylar=range(7, 23)):
    """Har as-of oy t (oy oxiri): faqat t gacha ma'lum belgilar + t, t+1 natijasi."""
    mj = bank["mijozlar"]
    ids, kol = mj["mijoz_id"].to_numpy(), np.arange(1, OYLAR + 1)

    def mat(df, qiymat, ustun="oy"):
        return (df.pivot(index="mijoz_id", columns=ustun, values=qiymat)
                .reindex(index=ids, columns=kol).to_numpy(dtype=float))

    hv, tl = bank["hisobvaraqlar"], bank["tolovlar"]
    bal, mint = mat(hv, "balans"), mat(hv, "min_tolov")
    tol, kun = mat(tl, "summa", "hisob_oy"), mat(tl, "tolov_kuni", "hisob_oy")
    bor = ~np.isnan(bal)
    miss = bor & ~(np.nan_to_num(tol) >= mint - 1e-9)    # minimal to'lov qilinmagan
    g = (bank["tranzaksiyalar"].groupby(["turi", "mijoz_id", "oy"])["summa"]
         .agg(["size", "sum"]))

    def tr_mat(turi, ust):
        return (g.loc[turi, ust].unstack("oy").reindex(index=ids, columns=kol)
                .fillna(0).to_numpy())

    naqd_n, naqd_s = tr_mat("naqd", "size"), tr_mat("naqd", "sum")
    xarid_n = tr_mat("xarid", "size")
    util = bal / mj["limit"].to_numpy()[:, None]
    och = mj["ochilgan_oy"].to_numpy()
    qismlar = []
    for t in oylar:
        c = t - 1                                        # t-oy ustuni (0 dan)
        ok = (och <= t - 1) & bor[:, c] & bor[:, c - 1] & ~miss[:, c - 1]
        u3 = util[:, c - 3]
        f = mj.drop(columns="ochilgan_oy").assign(
            oy=t, staj=t - och, util=util[:, c], util_3=orta(util[:, c - 2:c + 1]),
            util_ozg=util[:, c] - np.where(np.isnan(u3), util[:, c], u3),
            kechikish_6=miss[:, max(c - 6, 0):c].sum(axis=1),
            tolov_nisbati_3=orta(np.nan_to_num(tol[:, c - 3:c]) / bal[:, c - 3:c]),
            tolov_kuni_3=orta(kun[:, c - 3:c]),
            naqd_soni_3=naqd_n[:, c - 2:c + 1].sum(axis=1),
            naqd_summa_3=naqd_s[:, c - 2:c + 1].sum(axis=1),
            xarid_soni_3=xarid_n[:, c - 2:c + 1].sum(axis=1),
            balans=bal[:, c], y30=(miss[:, c] | miss[:, c + 1]).astype(int),
            y60=(miss[:, c] & miss[:, c + 1]).astype(int))
        qismlar.append(f[ok])
    df = pd.concat(qismlar, ignore_index=True)
    df["tolov_kuni_3"] = df["tolov_kuni_3"].fillna(25.0)
    df["tolov_nisbati_3"] = df["tolov_nisbati_3"].fillna(0.0)
    df["zarar"] = 0.45 * df["balans"]              # DPD60 bo'lsa kutilgan zarar
    df["qiymat"] = SAMARA * df["zarar"] * df["y60"] - NARX   # qo'ng'iroq qiymati
    return df


def oy_guruhlari(df):
    oy = df["oy"].to_numpy()
    return [np.flatnonzero(oy == t) for t in np.unique(oy)]


def tejam_k(qiymat, ball, guruhlar, k=K):
    """Har oyda ball bo'yicha top-k; oylik o'rtacha tejam (mln so'm)."""
    jami = 0.0
    for g in guruhlar:
        jami += qiymat[g[np.argsort(-ball[g], kind="stable")[:k]]].sum()
    return jami / len(guruhlar)


def bootstrap(y, qiymat, ballar, guruhlar, takror=300, seed=1):
    """Oy ichida qayta tanlash: har ball uchun AUC va tejam namunalari (juftlashgan)."""
    rng = np.random.default_rng(seed)
    auc = {k: [] for k in ballar}
    tej = {k: [] for k in ballar}
    for _ in range(takror):
        gb = [rng.choice(g, len(g)) for g in guruhlar]
        idx = np.concatenate(gb)
        yangi = []
        bosh = 0
        for g in gb:
            yangi.append(np.arange(bosh, bosh + len(g)))
            bosh += len(g)
        for k, b in ballar.items():
            auc[k].append(roc_auc_score(y[idx], b[idx]))
            tej[k].append(tejam_k(qiymat[idx], b[idx], yangi))
    return ({k: np.array(v) for k, v in auc.items()},
            {k: np.array(v) for k, v in tej.items()})


def main() -> None:
    df = belgilar_jadvali(yarat_bank())
    tr, va = df[df.oy <= 14], df[df.oy.between(16, 18)].reset_index(drop=True)
    guruhlar = oy_guruhlari(va)
    y, qiymat, zarar = va["y60"].to_numpy(), va["qiymat"].to_numpy(), va["zarar"].to_numpy()

    lr = make_pipeline(StandardScaler(), LogisticRegression(max_iter=2000))
    p = lr.fit(tr[BELGILAR], tr["y60"]).predict_proba(va[BELGILAR])[:, 1]
    print("=== 1. LogReg (y60), o'quv: as-of 7-14, val: 16-18 ===")
    print(f"  AUC {roc_auc_score(y, p):.4f}, tejam (top-{K}, p bo'yicha) "
          f"{tejam_k(qiymat, p, guruhlar):.1f} mln so'm/oy")
    tp = fp = fn = 0
    for g in guruhlar:
        top = g[np.argsort(-p[g], kind="stable")[:K]]
        tp += y[top].sum()
        fp += K - y[top].sum()
        fn += y[g].sum() - y[top].sum()
    m = len(guruhlar)
    print(f"  oyiga: TP {tp / m:.1f}, FP {fp / m:.1f}, FN {fn / m:.1f}; "
          f"precision@{K} {tp / (tp + fp):.3f}, recall@{K} {tp / (tp + fn):.3f}")

    print("\n=== 2. Bir xil model, ikki reyting: p va p * zarar ===")
    pz = p * zarar
    ballar = {"p": p, "p*zarar": pz}
    for k, b in ballar.items():
        print(f"  {k:<8} AUC {roc_auc_score(y, b):.4f}   tejam "
              f"{tejam_k(qiymat, b, guruhlar):6.1f}")

    print("\n=== 3. AUC oshadi, tejam o'zgarmaydi: faqat top-K dan tashqari tartib ===")
    yangi = p.copy()
    for g in guruhlar:
        tartib = g[np.argsort(-p[g], kind="stable")]
        top, qolgan = tartib[:K], tartib[K:]
        kuchaytir = p[qolgan] * (1 + 4 * y[qolgan])      # musbatlarni yuqoriga
        daraja = np.argsort(np.argsort(kuchaytir)) / len(qolgan)
        yangi[qolgan] = p[top].min() * 0.999 * daraja
    ballar["p (tashqari tuzatilgan)"] = yangi
    for k in ["p", "p (tashqari tuzatilgan)"]:
        print(f"  {k:<24} AUC {roc_auc_score(y, ballar[k]):.4f}   tejam "
              f"{tejam_k(qiymat, ballar[k], guruhlar):6.1f}")

    print("\n=== 4. Juftlashgan bootstrap (oy ichida, 300 takror): AUC va tejam farqi ===")
    hgb = HistGradientBoostingClassifier(max_iter=200, learning_rate=0.05,
                                         max_leaf_nodes=15, min_samples_leaf=100,
                                         random_state=0)
    ph = hgb.fit(tr[BELGILAR], tr["y60"]).predict_proba(va[BELGILAR])[:, 1]
    ballar = {"p": p, "p*zarar": pz, "HGB p*zarar": ph * zarar}
    auc_b, tej_b = bootstrap(y, qiymat, ballar, guruhlar)
    xulosa = {}
    for a, b in [("p*zarar", "p"), ("HGB p*zarar", "p*zarar")]:
        da, dt = auc_b[a] - auc_b[b], tej_b[a] - tej_b[b]
        ta = roc_auc_score(y, ballar[a]) - roc_auc_score(y, ballar[b])
        tt = tejam_k(qiymat, ballar[a], guruhlar) - tejam_k(qiymat, ballar[b], guruhlar)
        print(f"  {a} - {b}:")
        print(f"    AUC farqi   {ta:+.4f} (SE {da.std(ddof=1):.4f}) "
              f"sezilarli: {abs(ta) > 2 * da.std(ddof=1)}")
        print(f"    tejam farqi {tt:+7.2f} (SE {dt.std(ddof=1):.2f}) "
              f"sezilarli: {abs(tt) > 2 * dt.std(ddof=1)}")
        xulosa[a] = (ta, abs(ta) > 2 * da.std(ddof=1), tt, abs(tt) > 2 * dt.std(ddof=1))

    print("\n=== 5. Xulosa (natijadan hisoblangan) ===")
    for nom, (ta, sa, tt, st) in xulosa.items():
        if sa and not st:
            print(f"  {nom}: AUC farqi sezilarli ({ta:+.4f}), tejam farqi yo'q "
                  f"({tt:+.2f}) -> AUC farqi qiymatga o'tmadi")
    a_p = roc_auc_score(y, p)
    a_y = roc_auc_score(y, yangi)
    if abs(tejam_k(qiymat, yangi, guruhlar) - tejam_k(qiymat, p, guruhlar)) < 1e-9:
        print(f"  top-K tashqarisi: AUC {a_y - a_p:+.4f}, tejam aynan o'zgarmadi")
    print("  qaror top-K ga tayansa, metrika ham top-K qiymatini o'lchasin")


if __name__ == "__main__":
    main()

=== 1. LogReg (y60), o'quv: as-of 7-14, val: 16-18 ===
  AUC 0.8482, tejam (top-200, p bo'yicha) 26.3 mln so'm/oy
  oyiga: TP 24.0, FP 176.0, FN 52.7; precision@200 0.120, recall@200 0.313

=== 2. Bir xil model, ikki reyting: p va p * zarar ===
  p        AUC 0.8482   tejam   26.3
  p*zarar  AUC 0.8314   tejam   28.8

=== 3. AUC oshadi, tejam o'zgarmaydi: faqat top-K dan tashqari tartib ===
  p                        AUC 0.8482   tejam   26.3
  p (tashqari tuzatilgan)  AUC 0.9536   tejam   26.3

=== 4. Juftlashgan bootstrap (oy ichida, 300 takror): AUC va tejam farqi ===
  p*zarar - p:
    AUC farqi   -0.0169 (SE 0.0046) sezilarli: True
    tejam farqi   +2.53 (SE 2.58) sezilarli: False
  HGB p*zarar - p*zarar:
    AUC farqi   -0.0127 (SE 0.0039) sezilarli: True
    tejam farqi   +0.18 (SE 1.94) sezilarli: False

=== 5. Xulosa (natijadan hisoblangan) ===
  p*zarar: AUC farqi sezilarli (-0.0169), tejam farqi yo'q (+2.53) -> AUC farqi qiymatga o'tmadi
  HGB p*zarar: AUC farqi sezilarli (-0.0127), tejam farqi yo'q (+0.18) -> AUC farqi qiymatga o'tmadi
  top-K tashqarisi: AUC +0.1054, tejam aynan o'zgarmadi
  qaror top-K ga tayansa, metrika ham top-K qiymatini o'lchasin

"""Bazaviylar va "model kerakmi?": qoidalar, ekspert ball-kartasi va LogReg."""

import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

HUDUDLAR = ["Toshkent", "Samarqand", "Farg'ona", "Andijon", "Buxoro", "Qashqadaryo"]
OYLAR = 24                               # 1 = 2024-01, ..., 24 = 2025-12
BELGILAR = ["yosh", "daromad", "limit", "avto_tolov", "maosh_loyihasi", "staj",
            "util", "util_3", "util_ozg", "kechikish_6", "tolov_nisbati_3",
            "tolov_kuni_3", "naqd_soni_3", "naqd_summa_3", "xarid_soni_3"]
K, SAMARA, NARX = 200, 0.30, 0.02        # oyiga qo'ng'iroq, oldini olish, mln so'm


def sigmoid(z):
    return 1 / (1 + np.exp(-z))


def oy_nomi(m):
    return f"{2024 + (m - 1) // 12}-{(m - 1) % 12 + 1:02d}"


def yarat_bank(seed=29, n=6000):
    """Kredit karta portfeli: mijozlar, hisobvaraqlar, tolovlar, tranzaksiyalar."""
    rng = np.random.default_rng(seed)
    ids = np.arange(1, n + 1)
    yosh = rng.integers(21, 66, n)
    maosh = rng.random(n) < 0.45
    daromad = np.round(np.exp(rng.normal(np.log(6.0), 0.45, n)), 2)   # mln so'm
    ochilgan = np.where(rng.random(n) < 0.7, rng.integers(-40, 1, n),
                        rng.integers(1, 20, n))
    xavf = (rng.normal(0, 1, n) - 0.5 * maosh - 0.8 * np.log(daromad / 6)
            + 0.015 * (35 - yosh))                     # yashirin: moliyaviy xavf
    unut = rng.normal(0, 1, n) + 0.03 * (35 - yosh)     # yashirin: unutuvchanlik
    avto = (rng.random(n) < sigmoid(-0.2 - 0.9 * unut)).astype(int)
    limit = np.round(daromad * rng.uniform(1.5, 3.5, n), 1)
    hudud = rng.choice(HUDUDLAR, n, p=[0.34, 0.15, 0.14, 0.13, 0.12, 0.12])
    mj = pd.DataFrame({"mijoz_id": ids, "yosh": yosh, "hudud": hudud,
                       "daromad": daromad, "limit": limit, "avto_tolov": avto,
                       "maosh_loyihasi": maosh.astype(int), "ochilgan_oy": ochilgan})
    s, oldingi, ketma = rng.normal(0, 1, n), np.zeros(n, int), np.zeros(n, int)
    hv, tl, tr = [], [], []
    for m in range(1, OYLAR + 1):
        faol = (ochilgan <= m) & (ketma < 3)            # 3 ketma-ket kechikish - defolt
        s = 0.75 * s + 0.66 * rng.normal(0, 1, n)
        d = 0.8 * xavf + 0.7 * s + 0.15 * (m % 12 == 1) + 0.08 * max(m - 18, 0)
        util = np.clip(sigmoid(-0.4 + 0.55 * d + rng.normal(0, 0.5, n)), 0.01, 1)
        balans = np.round(util * limit, 3)
        min_t = np.round(np.maximum(0.05 * balans, 0.05), 3)
        for turi, lam in [("xarid", np.exp(1.2 + 0.15 * np.log(daromad / 6) - 0.1 * d)),
                          ("naqd", np.exp(-1.6 + 0.6 * d)),
                          ("onlayn", np.exp(0.6 - 0.02 * (yosh - 35)))]:
            i = np.repeat(np.arange(n), rng.poisson(lam) * faol)
            summa = (rng.uniform(0.2, 1.5, i.size) * (1 + 0.3 * d[i].clip(0))
                     if turi == "naqd" else np.exp(rng.normal(np.log(0.25), 0.8, i.size)))
            tr.append(pd.DataFrame({"mijoz_id": ids[i], "oy": m,
                                    "kun": rng.integers(1, 29, i.size), "turi": turi,
                                    "summa": np.round(summa, 3)}))
        q = rng.random(n) < sigmoid(-3.7 + 1.3 * d + 1.2 * (util > 0.85)
                                    + 3.8 * (oldingi == 2))          # qiyinchilik
        u = ~q & (rng.random(n) < sigmoid(-3.4 + 1.1 * unut - 1.8 * avto)
                  * (oldingi != 1))                                  # unutish
        miss = (q | u) & faol
        ulush = sigmoid(0.3 - 1.2 * d + rng.normal(0, 0.5, n))
        summa = np.where(miss, min_t * rng.uniform(0, 0.6, n),
                         min_t + (balans - min_t).clip(0) * ulush)
        tolaydi = faol & (~miss | (rng.random(n) < 0.4))
        kun = np.clip(np.round(6 + 14 * sigmoid(unut + 0.5 * d)
                               + rng.normal(0, 3, n)), 1, 25)
        hv.append(pd.DataFrame({"mijoz_id": ids[faol], "oy": m,
                                "balans": balans[faol], "min_tolov": min_t[faol]}))
        if m < OYLAR:                    # m-oy hisobvarag'i (m+1)-oyda to'lanadi
            tl.append(pd.DataFrame({"mijoz_id": ids[tolaydi], "hisob_oy": m,
                                    "tolov_oy": m + 1,
                                    "tolov_kuni": kun[tolaydi].astype(int),
                                    "summa": np.round(summa[tolaydi], 3)}))
        ketma = np.where(faol, np.where(miss, ketma + 1, 0), ketma)
        oldingi = np.where(miss, np.where(q, 2, 1), 0)
    return {"mijozlar": mj, "hisobvaraqlar": pd.concat(hv, ignore_index=True),
            "tolovlar": pd.concat(tl, ignore_index=True),
            "tranzaksiyalar": pd.concat(tr, ignore_index=True)}


def orta(a):
    """Qator bo'yicha NaN siz o'rtacha (bo'sh qatorda NaN, ogohlantirishsiz)."""
    k = (~np.isnan(a)).sum(axis=1)
    return np.where(k > 0, np.nansum(a, axis=1) / np.maximum(k, 1), np.nan)


def belgilar_jadvali(bank, oylar=range(7, 23)):
    """Har as-of oy t (oy oxiri): faqat t gacha ma'lum belgilar + t, t+1 natijasi."""
    mj = bank["mijozlar"]
    ids, kol = mj["mijoz_id"].to_numpy(), np.arange(1, OYLAR + 1)

    def mat(df, qiymat, ustun="oy"):
        return (df.pivot(index="mijoz_id", columns=ustun, values=qiymat)
                .reindex(index=ids, columns=kol).to_numpy(dtype=float))

    hv, tl = bank["hisobvaraqlar"], bank["tolovlar"]
    bal, mint = mat(hv, "balans"), mat(hv, "min_tolov")
    tol, kun = mat(tl, "summa", "hisob_oy"), mat(tl, "tolov_kuni", "hisob_oy")
    bor = ~np.isnan(bal)
    miss = bor & ~(np.nan_to_num(tol) >= mint - 1e-9)    # minimal to'lov qilinmagan
    g = (bank["tranzaksiyalar"].groupby(["turi", "mijoz_id", "oy"])["summa"]
         .agg(["size", "sum"]))

    def tr_mat(turi, ust):
        return (g.loc[turi, ust].unstack("oy").reindex(index=ids, columns=kol)
                .fillna(0).to_numpy())

    naqd_n, naqd_s = tr_mat("naqd", "size"), tr_mat("naqd", "sum")
    xarid_n = tr_mat("xarid", "size")
    util = bal / mj["limit"].to_numpy()[:, None]
    och = mj["ochilgan_oy"].to_numpy()
    qismlar = []
    for t in oylar:
        c = t - 1                                        # t-oy ustuni (0 dan)
        ok = (och <= t - 1) & bor[:, c] & bor[:, c - 1] & ~miss[:, c - 1]
        u3 = util[:, c - 3]
        f = mj.drop(columns="ochilgan_oy").assign(
            oy=t, staj=t - och, util=util[:, c], util_3=orta(util[:, c - 2:c + 1]),
            util_ozg=util[:, c] - np.where(np.isnan(u3), util[:, c], u3),
            kechikish_6=miss[:, max(c - 6, 0):c].sum(axis=1),
            tolov_nisbati_3=orta(np.nan_to_num(tol[:, c - 3:c]) / bal[:, c - 3:c]),
            tolov_kuni_3=orta(kun[:, c - 3:c]),
            naqd_soni_3=naqd_n[:, c - 2:c + 1].sum(axis=1),
            naqd_summa_3=naqd_s[:, c - 2:c + 1].sum(axis=1),
            xarid_soni_3=xarid_n[:, c - 2:c + 1].sum(axis=1),
            balans=bal[:, c], y30=(miss[:, c] | miss[:, c + 1]).astype(int),
            y60=(miss[:, c] & miss[:, c + 1]).astype(int))
        qismlar.append(f[ok])
    df = pd.concat(qismlar, ignore_index=True)
    df["tolov_kuni_3"] = df["tolov_kuni_3"].fillna(25.0)
    df["tolov_nisbati_3"] = df["tolov_nisbati_3"].fillna(0.0)
    df["zarar"] = 0.45 * df["balans"]              # DPD60 bo'lsa kutilgan zarar
    df["qiymat"] = SAMARA * df["zarar"] * df["y60"] - NARX   # qo'ng'iroq qiymati
    return df


def oy_guruhlari(df):
    oy = df["oy"].to_numpy()
    return [np.flatnonzero(oy == t) for t in np.unique(oy)]


def tejam_k(qiymat, ball, guruhlar, k=K):
    jami = 0.0
    for g in guruhlar:
        jami += qiymat[g[np.argsort(-ball[g], kind="stable")[:k]]].sum()
    return jami / len(guruhlar)


def ekspert_ball(df):
    """Kredit bo'limi tajribasidan: oddiy ball-karta, teng ballda katta balans oldin."""
    ball = (2 * (df["util"] > 0.8) + 1 * (df["naqd_soni_3"] >= 2)
            + 2 * (df["tolov_nisbati_3"] < 0.15) + 1 * (df["util_ozg"] > 0.15))
    return (ball + df["zarar"] / 100).to_numpy()


def main() -> None:
    df = belgilar_jadvali(yarat_bank())
    tr, va = df[df.oy <= 14], df[df.oy.between(16, 18)].reset_index(drop=True)
    guruhlar = oy_guruhlari(va)
    qiymat = va["qiymat"].to_numpy()
    lr = make_pipeline(StandardScaler(), LogisticRegression(max_iter=2000))
    p = lr.fit(tr[BELGILAR], tr["y60"]).predict_proba(va[BELGILAR])[:, 1]
    rng = np.random.default_rng(0)
    nomzodlar = {                       # soddalik tartibida
        "0 hech kim": None,
        "1 tasodifiy": rng.random(len(va)),
        "2 qoida: util": va["util"].to_numpy(),
        "3 qoida: util*zarar": (va["util"] * va["zarar"]).to_numpy(),
        "4 ekspert ball-karta": ekspert_ball(va),
        "5 LogReg p*zarar": p * va["zarar"].to_numpy(),
    }

    print("=== 1. Val oylari 16-18: oylik tejam (mln so'm) ===")
    baho = {}
    for nom, b in nomzodlar.items():
        baho[nom] = 0.0 if b is None else tejam_k(qiymat, b, guruhlar)
        print(f"  {nom:<22} {baho[nom]:>7.2f}")

    print("\n=== 2. Juftlashgan bootstrap (oy ichida, 300): eng yaxshisidan farq ===")
    eng = max(baho, key=baho.get)
    bs = {nom: [] for nom in nomzodlar}
    rng = np.random.default_rng(1)
    for _ in range(300):
        gb = [rng.choice(g, len(g)) for g in guruhlar]
        idx = np.concatenate(gb)
        chegara = np.cumsum([0] + [len(g) for g in gb])
        yangi = [np.arange(chegara[i], chegara[i + 1]) for i in range(len(gb))]
        for nom, b in nomzodlar.items():
            bs[nom].append(0.0 if b is None else tejam_k(qiymat[idx], b[idx], yangi))
    tanlov = None
    print(f"  eng yaxshisi: {eng}")
    print(f"  {'nomzod':<22} {'farq':>7} {'SE':>6} {'sezilarli yomon':>16}")
    for nom in nomzodlar:
        d = np.array(bs[nom]) - np.array(bs[eng])
        farq, se = baho[nom] - baho[eng], d.std(ddof=1)
        yomon = farq < -2 * se
        if not yomon and tanlov is None:
            tanlov = nom
        print(f"  {nom:<22} {farq:>+7.2f} {se:>6.2f} {str(yomon):>16}")
    print(f"  qaror qoidasi -> eng sodda munosib: {tanlov}")

    print("\n=== 3. \"Model kerakmi?\" - pul bilan ===")
    qoidalar = [n for n in nomzodlar if n[0] in "234"]
    eng_qoida = max(qoidalar, key=baho.get)
    d = np.array(bs["5 LogReg p*zarar"]) - np.array(bs[eng_qoida])
    farq, se = baho["5 LogReg p*zarar"] - baho[eng_qoida], d.std(ddof=1)
    lo, hi = 12 * (farq - 2 * se), 12 * (farq + 2 * se)
    print(f"  eng yaxshi qoida: {eng_qoida}")
    print(f"  model - qoida: oyiga {farq:+.2f} (SE {se:.2f}) -> yiliga {12 * farq:+.0f} "
          f"mln so'm, +-2*SE: [{lo:+.0f}, {hi:+.0f}]")
    print(f"  qoida model qiymatining {baho[eng_qoida] / baho['5 LogReg p*zarar']:.0%} "
          f"ini beradi")

    print("\n=== 4. Sig'im o'zgarsa: qoida va model farqi (oyiga, mln so'm) ===")
    print(f"  {'K':>5} {'qoida':>8} {'model':>8} {'farq':>7} {'model/qoida':>12}")
    for k in [50, 200, 800, 2000]:
        a = tejam_k(qiymat, nomzodlar[eng_qoida], guruhlar, k)
        b = tejam_k(qiymat, nomzodlar["5 LogReg p*zarar"], guruhlar, k)
        print(f"  {k:>5} {a:>8.2f} {b:>8.2f} {b - a:>+7.2f} {b / a:>12.2f}")

    print("\n=== 5. Xulosa (natijadan hisoblangan) ===")
    if tanlov == "5 LogReg p*zarar":
        print("  eng sodda munosib - model: qoidalar sezilarli yomon")
    else:
        print(f"  eng sodda munosib - {tanlov}: model sezilarli ustun emas")
    if lo <= 0:
        print("  3 oylik val da model foydasi noaniq (interval 0 ni qamraydi):")
        print("  prototip qarori - qoida bazaviy va zaxira sifatida saqlanadi, model")
        print("  ko'proq oyda (vaqt bo'yicha CV) tekshiriladi")
    else:
        print("  model foydasi 2*SE dan katta - loyiha keyingi bosqichga o'tadi")


if __name__ == "__main__":
    main()

=== 1. Val oylari 16-18: oylik tejam (mln so'm) ===
  0 hech kim                0.00
  1 tasodifiy              -0.32
  2 qoida: util            24.26
  3 qoida: util*zarar      23.04
  4 ekspert ball-karta     19.36
  5 LogReg p*zarar         28.85

=== 2. Juftlashgan bootstrap (oy ichida, 300): eng yaxshisidan farq ===
  eng yaxshisi: 5 LogReg p*zarar
  nomzod                    farq     SE  sezilarli yomon
  0 hech kim              -28.85   4.10             True
  1 tasodifiy             -29.17   4.09             True
  2 qoida: util            -4.59   2.93            False
  3 qoida: util*zarar      -5.81   2.98            False
  4 ekspert ball-karta     -9.49   3.09             True
  5 LogReg p*zarar         +0.00   0.00            False
  qaror qoidasi -> eng sodda munosib: 2 qoida: util

=== 3. "Model kerakmi?" - pul bilan ===
  eng yaxshi qoida: 2 qoida: util
  model - qoida: oyiga +4.59 (SE 2.93) -> yiliga +55 mln so'm, +-2*SE: [-15, +125]
  qoida model qiymatining 84% ini beradi

=== 4. Sig'im o'zgarsa: qoida va model farqi (oyiga, mln so'm) ===
      K    qoida    model    farq  model/qoida
     50     9.68    14.29   +4.61         1.48
    200    24.26    28.85   +4.59         1.19
    800    38.98    48.44   +9.46         1.24
   2000    34.09    38.32   +4.24         1.12

=== 5. Xulosa (natijadan hisoblangan) ===
  eng sodda munosib - 2 qoida: util: model sezilarli ustun emas
  3 oylik val da model foydasi noaniq (interval 0 ni qamraydi):
  prototip qarori - qoida bazaviy va zaxira sifatida saqlanadi, model
  ko'proq oyda (vaqt bo'yicha CV) tekshiriladi

"""Loyiha xatarlari: belgi ta'rifi (30 yoki 60 kun) va ma'lumot mavjudligi."""

import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

HUDUDLAR = ["Toshkent", "Samarqand", "Farg'ona", "Andijon", "Buxoro", "Qashqadaryo"]
OYLAR = 24                               # 1 = 2024-01, ..., 24 = 2025-12
BELGILAR = ["yosh", "daromad", "limit", "avto_tolov", "maosh_loyihasi", "staj",
            "util", "util_3", "util_ozg", "kechikish_6", "tolov_nisbati_3",
            "tolov_kuni_3", "naqd_soni_3", "naqd_summa_3", "xarid_soni_3"]
K, SAMARA, NARX = 200, 0.30, 0.02        # oyiga qo'ng'iroq, oldini olish, mln so'm


def sigmoid(z):
    return 1 / (1 + np.exp(-z))


def oy_nomi(m):
    return f"{2024 + (m - 1) // 12}-{(m - 1) % 12 + 1:02d}"


def yarat_bank(seed=29, n=6000):
    """Kredit karta portfeli: mijozlar, hisobvaraqlar, tolovlar, tranzaksiyalar."""
    rng = np.random.default_rng(seed)
    ids = np.arange(1, n + 1)
    yosh = rng.integers(21, 66, n)
    maosh = rng.random(n) < 0.45
    daromad = np.round(np.exp(rng.normal(np.log(6.0), 0.45, n)), 2)   # mln so'm
    ochilgan = np.where(rng.random(n) < 0.7, rng.integers(-40, 1, n),
                        rng.integers(1, 20, n))
    xavf = (rng.normal(0, 1, n) - 0.5 * maosh - 0.8 * np.log(daromad / 6)
            + 0.015 * (35 - yosh))                     # yashirin: moliyaviy xavf
    unut = rng.normal(0, 1, n) + 0.03 * (35 - yosh)     # yashirin: unutuvchanlik
    avto = (rng.random(n) < sigmoid(-0.2 - 0.9 * unut)).astype(int)
    limit = np.round(daromad * rng.uniform(1.5, 3.5, n), 1)
    hudud = rng.choice(HUDUDLAR, n, p=[0.34, 0.15, 0.14, 0.13, 0.12, 0.12])
    mj = pd.DataFrame({"mijoz_id": ids, "yosh": yosh, "hudud": hudud,
                       "daromad": daromad, "limit": limit, "avto_tolov": avto,
                       "maosh_loyihasi": maosh.astype(int), "ochilgan_oy": ochilgan})
    s, oldingi, ketma = rng.normal(0, 1, n), np.zeros(n, int), np.zeros(n, int)
    hv, tl, tr = [], [], []
    for m in range(1, OYLAR + 1):
        faol = (ochilgan <= m) & (ketma < 3)            # 3 ketma-ket kechikish - defolt
        s = 0.75 * s + 0.66 * rng.normal(0, 1, n)
        d = 0.8 * xavf + 0.7 * s + 0.15 * (m % 12 == 1) + 0.08 * max(m - 18, 0)
        util = np.clip(sigmoid(-0.4 + 0.55 * d + rng.normal(0, 0.5, n)), 0.01, 1)
        balans = np.round(util * limit, 3)
        min_t = np.round(np.maximum(0.05 * balans, 0.05), 3)
        for turi, lam in [("xarid", np.exp(1.2 + 0.15 * np.log(daromad / 6) - 0.1 * d)),
                          ("naqd", np.exp(-1.6 + 0.6 * d)),
                          ("onlayn", np.exp(0.6 - 0.02 * (yosh - 35)))]:
            i = np.repeat(np.arange(n), rng.poisson(lam) * faol)
            summa = (rng.uniform(0.2, 1.5, i.size) * (1 + 0.3 * d[i].clip(0))
                     if turi == "naqd" else np.exp(rng.normal(np.log(0.25), 0.8, i.size)))
            tr.append(pd.DataFrame({"mijoz_id": ids[i], "oy": m,
                                    "kun": rng.integers(1, 29, i.size), "turi": turi,
                                    "summa": np.round(summa, 3)}))
        q = rng.random(n) < sigmoid(-3.7 + 1.3 * d + 1.2 * (util > 0.85)
                                    + 3.8 * (oldingi == 2))          # qiyinchilik
        u = ~q & (rng.random(n) < sigmoid(-3.4 + 1.1 * unut - 1.8 * avto)
                  * (oldingi != 1))                                  # unutish
        miss = (q | u) & faol
        ulush = sigmoid(0.3 - 1.2 * d + rng.normal(0, 0.5, n))
        summa = np.where(miss, min_t * rng.uniform(0, 0.6, n),
                         min_t + (balans - min_t).clip(0) * ulush)
        tolaydi = faol & (~miss | (rng.random(n) < 0.4))
        kun = np.clip(np.round(6 + 14 * sigmoid(unut + 0.5 * d)
                               + rng.normal(0, 3, n)), 1, 25)
        hv.append(pd.DataFrame({"mijoz_id": ids[faol], "oy": m,
                                "balans": balans[faol], "min_tolov": min_t[faol]}))
        if m < OYLAR:                    # m-oy hisobvarag'i (m+1)-oyda to'lanadi
            tl.append(pd.DataFrame({"mijoz_id": ids[tolaydi], "hisob_oy": m,
                                    "tolov_oy": m + 1,
                                    "tolov_kuni": kun[tolaydi].astype(int),
                                    "summa": np.round(summa[tolaydi], 3)}))
        ketma = np.where(faol, np.where(miss, ketma + 1, 0), ketma)
        oldingi = np.where(miss, np.where(q, 2, 1), 0)
    return {"mijozlar": mj, "hisobvaraqlar": pd.concat(hv, ignore_index=True),
            "tolovlar": pd.concat(tl, ignore_index=True),
            "tranzaksiyalar": pd.concat(tr, ignore_index=True)}


def orta(a):
    """Qator bo'yicha NaN siz o'rtacha (bo'sh qatorda NaN, ogohlantirishsiz)."""
    k = (~np.isnan(a)).sum(axis=1)
    return np.where(k > 0, np.nansum(a, axis=1) / np.maximum(k, 1), np.nan)


def belgilar_jadvali(bank, oylar=range(7, 23)):
    """Har as-of oy t (oy oxiri): faqat t gacha ma'lum belgilar + t, t+1 natijasi."""
    mj = bank["mijozlar"]
    ids, kol = mj["mijoz_id"].to_numpy(), np.arange(1, OYLAR + 1)

    def mat(df, qiymat, ustun="oy"):
        return (df.pivot(index="mijoz_id", columns=ustun, values=qiymat)
                .reindex(index=ids, columns=kol).to_numpy(dtype=float))

    hv, tl = bank["hisobvaraqlar"], bank["tolovlar"]
    bal, mint = mat(hv, "balans"), mat(hv, "min_tolov")
    tol, kun = mat(tl, "summa", "hisob_oy"), mat(tl, "tolov_kuni", "hisob_oy")
    bor = ~np.isnan(bal)
    miss = bor & ~(np.nan_to_num(tol) >= mint - 1e-9)    # minimal to'lov qilinmagan
    g = (bank["tranzaksiyalar"].groupby(["turi", "mijoz_id", "oy"])["summa"]
         .agg(["size", "sum"]))

    def tr_mat(turi, ust):
        return (g.loc[turi, ust].unstack("oy").reindex(index=ids, columns=kol)
                .fillna(0).to_numpy())

    naqd_n, naqd_s = tr_mat("naqd", "size"), tr_mat("naqd", "sum")
    xarid_n = tr_mat("xarid", "size")
    util = bal / mj["limit"].to_numpy()[:, None]
    och = mj["ochilgan_oy"].to_numpy()
    qismlar = []
    for t in oylar:
        c = t - 1                                        # t-oy ustuni (0 dan)
        ok = (och <= t - 1) & bor[:, c] & bor[:, c - 1] & ~miss[:, c - 1]
        u3 = util[:, c - 3]
        f = mj.drop(columns="ochilgan_oy").assign(
            oy=t, staj=t - och, util=util[:, c], util_3=orta(util[:, c - 2:c + 1]),
            util_ozg=util[:, c] - np.where(np.isnan(u3), util[:, c], u3),
            kechikish_6=miss[:, max(c - 6, 0):c].sum(axis=1),
            tolov_nisbati_3=orta(np.nan_to_num(tol[:, c - 3:c]) / bal[:, c - 3:c]),
            tolov_kuni_3=orta(kun[:, c - 3:c]),
            naqd_soni_3=naqd_n[:, c - 2:c + 1].sum(axis=1),
            naqd_summa_3=naqd_s[:, c - 2:c + 1].sum(axis=1),
            xarid_soni_3=xarid_n[:, c - 2:c + 1].sum(axis=1),
            balans=bal[:, c], y30=(miss[:, c] | miss[:, c + 1]).astype(int),
            y60=(miss[:, c] & miss[:, c + 1]).astype(int))
        qismlar.append(f[ok])
    df = pd.concat(qismlar, ignore_index=True)
    df["tolov_kuni_3"] = df["tolov_kuni_3"].fillna(25.0)
    df["tolov_nisbati_3"] = df["tolov_nisbati_3"].fillna(0.0)
    df["zarar"] = 0.45 * df["balans"]              # DPD60 bo'lsa kutilgan zarar
    df["qiymat"] = SAMARA * df["zarar"] * df["y60"] - NARX   # qo'ng'iroq qiymati
    return df


def oy_guruhlari(df):
    oy = df["oy"].to_numpy()
    return [np.flatnonzero(oy == t) for t in np.unique(oy)]


def top_k(ball, guruhlar, k=K):
    return np.concatenate([g[np.argsort(-ball[g], kind="stable")[:k]] for g in guruhlar])


def lr_ball(tr, va, belgi):
    m = make_pipeline(StandardScaler(), LogisticRegression(max_iter=2000))
    return m.fit(tr[BELGILAR], tr[belgi]).predict_proba(va[BELGILAR])[:, 1]


def main() -> None:
    df = belgilar_jadvali(yarat_bank())
    print("=== 1. Ikki ta'rif (as-of 7-22, barcha qatorlar) ===")
    print(f"  y30 (60 kun ichida 30+ kun kechikish): {df.y30.mean():.3f}")
    print(f"  y60 (60 kun ichida 60+ kun kechikish): {df.y60.mean():.3f}")
    print(f"  y60 = 1 bo'lganlar ichida y30 = 1: {df.loc[df.y60 == 1, 'y30'].mean():.3f}")
    ozi = ((df.y30 == 1) & (df.y60 == 0)).sum() / df.y30.sum()
    print(f"  y30 = 1 lar ichida o'zi to'lagan (y60 = 0): {ozi:.1%}")
    print(f"  oldingi 6 oyda kechikkan (kechikish_6 > 0) ulushi: "
          f"y60 da {(df.loc[df.y60 == 1, 'kechikish_6'] > 0).mean():.3f}, "
          f"faqat y30 da {(df.loc[(df.y30 == 1) & (df.y60 == 0), 'kechikish_6'] > 0).mean():.3f}")
    print(f"  avto_tolov ulushi: y60 da {df.loc[df.y60 == 1, 'avto_tolov'].mean():.3f}, "
          f"faqat y30 da {df.loc[(df.y30 == 1) & (df.y60 == 0), 'avto_tolov'].mean():.3f}")

    print("\n=== 2. Qaysi belgida o'qitamiz? o'quv 7-14, val 16-18 ===")
    tr, va = df[df.oy <= 14], df[df.oy.between(16, 18)].reset_index(drop=True)
    guruhlar = oy_guruhlari(va)
    qiymat, zarar = va["qiymat"].to_numpy(), va["zarar"].to_numpy()
    ballar = {b: lr_ball(tr, va, b) for b in ["y30", "y60"]}
    print(f"  {'model':<10} {'AUC y30':>8} {'AUC y60':>8} {'tejam/oy':>9} "
          f"{'y60':>6} {'faqat y30':>10} {'toza':>6}")
    tejamlar = {}
    for b, p in ballar.items():
        top = top_k(p * zarar, guruhlar)
        tejamlar[b] = qiymat[top].sum() / len(guruhlar)
        y60, y30 = va["y60"].to_numpy()[top], va["y30"].to_numpy()[top]
        m = len(guruhlar)
        print(f"  {b + ' da':<10} {roc_auc_score(va.y30, p):>8.4f} "
              f"{roc_auc_score(va.y60, p):>8.4f} {tejamlar[b]:>9.2f} "
              f"{y60.sum() / m:>6.1f} {((y30 == 1) & (y60 == 0)).sum() / m:>10.1f} "
              f"{(y30 == 0).sum() / m:>6.1f}")
    print("  ko'proq dalil: har oy t = 12-18 uchun o'quv - as-of <= t-2 (7 juft):")
    farqlar = []
    for t in range(12, 19):
        o, s = df[df.oy <= t - 2], df[df.oy == t]
        q, z = s["qiymat"].to_numpy(), s["zarar"].to_numpy()
        tt = {b: q[np.argsort(-lr_ball(o, s, b) * z, kind="stable")[:K]].sum()
              for b in ["y30", "y60"]}
        farqlar.append(tt["y60"] - tt["y30"])
    farqlar = np.array(farqlar)
    farq, se = farqlar.mean(), farqlar.std(ddof=1) / np.sqrt(len(farqlar))
    print(f"  oylik farqlar (y60 - y30): {np.round(farqlar, 1).tolist()}")
    print(f"  o'rtacha {farq:+.2f} mln so'm/oy (SE {se:.2f}), sezilarli: "
          f"{abs(farq) > 2 * se}, y60 yaxshi oylar: {(farqlar > 0).sum()}/{len(farqlar)}")

    print("\n=== 3. Ma'lumot mavjudligi: belgi qachon yetiladi? ===")
    print("  as-of t uchun y30/y60 (t va t+1 hisobvaraqlari) (t+2)-oy oxirida ma'lum")
    for nom, ufq in [("y60 (60 kun)", 2), ("y90 (90 kun, faraziy)", 3)]:
        oxirgi = 16 - ufq
        print(f"  {nom:<22} 16-oy oxirida o'qitish uchun oxirgi as-of: {oxirgi} "
              f"-> {oxirgi - 6} oy tarix")
    sarlavha = "o'quv oylari"
    print(f"  {sarlavha:<14} {'qator':>6} {'y60 soni':>9} {'AUC':>7} {'tejam/oy':>9}")
    natija = {}
    for bosh in [13, 11, 7]:
        t2 = df[df.oy.between(bosh, 14)]
        p = lr_ball(t2, va, "y60")
        natija[bosh] = qiymat[top_k(p * zarar, guruhlar)].sum() / len(guruhlar)
        print(f"  {f'{bosh}-14':<14} {len(t2):>6} {t2.y60.sum():>9} "
              f"{roc_auc_score(va.y60, p):>7.4f} {natija[bosh]:>9.2f}")

    print("\n=== 4. Xulosa (natijadan hisoblangan) ===")
    if farq > 2 * se:
        print(f"  y30 da o'qitilgan model oyiga {farq:.1f} mln so'm kam tejaydi: u o'zi "
              f"to'laydigan 'unutuvchan'larni ham tanlaydi")
    print(f"  y30 lar ichida {ozi:.0%} i o'zi to'laydi -> ta'rif qaror bilan mos bo'lishi shart")
    qiymatlar = np.array(list(natija.values()))
    if qiymatlar.max() - qiymatlar.min() < 0.1 * qiymatlar.mean():
        print(f"  2, 4 va 8 oylik o'quv yaqin ({qiymatlar.min():.2f}-{qiymatlar.max():.2f})"
              f" -> bu vazifada tarix hajmi asosiy cheklov emas")


if __name__ == "__main__":
    main()

=== 1. Ikki ta'rif (as-of 7-22, barcha qatorlar) ===
  y30 (60 kun ichida 30+ kun kechikish): 0.109
  y60 (60 kun ichida 60+ kun kechikish): 0.020
  y60 = 1 bo'lganlar ichida y30 = 1: 1.000
  y30 = 1 lar ichida o'zi to'lagan (y60 = 0): 81.3%
  oldingi 6 oyda kechikkan (kechikish_6 > 0) ulushi: y60 da 0.223, faqat y30 da 0.323
  avto_tolov ulushi: y60 da 0.481, faqat y30 da 0.258

=== 2. Qaysi belgida o'qitamiz? o'quv 7-14, val 16-18 ===
  model       AUC y30  AUC y60  tejam/oy    y60  faqat y30   toza
  y30 da       0.7302   0.7369     22.41   16.3       47.3  136.3
  y60 da       0.6561   0.8482     28.85   20.7       35.0  144.3
  ko'proq dalil: har oy t = 12-18 uchun o'quv - as-of <= t-2 (7 juft):
  oylik farqlar (y60 - y30): [2.8, 6.3, 9.4, 8.5, 7.4, 6.8, 7.9]
  o'rtacha +7.00 mln so'm/oy (SE 0.81), sezilarli: True, y60 yaxshi oylar: 7/7

=== 3. Ma'lumot mavjudligi: belgi qachon yetiladi? ===
  as-of t uchun y30/y60 (t va t+1 hisobvaraqlari) (t+2)-oy oxirida ma'lum
  y60 (60 kun)           16-oy oxirida o'qitish uchun oxirgi as-of: 14 -> 8 oy tarix
  y90 (90 kun, faraziy)  16-oy oxirida o'qitish uchun oxirgi as-of: 13 -> 7 oy tarix
  o'quv oylari    qator  y60 soni     AUC  tejam/oy
  13-14            8488       192  0.8434     28.56
  11-14           16918       373  0.8405     30.47
  7-14            33358       697  0.8482     28.85

=== 4. Xulosa (natijadan hisoblangan) ===
  y30 da o'qitilgan model oyiga 7.0 mln so'm kam tejaydi: u o'zi to'laydigan 'unutuvchan'larni ham tanlaydi
  y30 lar ichida 81% i o'zi to'laydi -> ta'rif qaror bilan mos bo'lishi shart
  2, 4 va 8 oylik o'quv yaqin (28.56-30.47) -> bu vazifada tarix hajmi asosiy cheklov emas

vazifa = "kechikishni bashorat qiluvchi model"                           # ⚠️
vazifa = {"qaror": "oyiga 200 qo'ng'iroq", "belgi": "y60", "metrika": "tejam/oy"}  # ✅

print(roc_auc_score(y, p))                                              # ⚠️
print(tejam_k(qiymat, p * zarar, guruhlar), roc_auc_score(y, p))        # ✅ ikkalasi

ro_yxat = df.nlargest(200, "p")                                         # ⚠️
ro_yxat = df.assign(k=df.p * df.zarar).nlargest(200, "k")               # ✅

print("model tasodifiydan 30 mln ko'p tejaydi")                         # ⚠️
print("model eng yaxshi qoidadan +4.6 (SE 2.9) ko'p tejaydi")           # ✅

y = df.y30          # musbatlar ko'p, o'qitish oson                     # ⚠️
y = df.y60          # qaror maqsadi: zararli kechikish                  # ✅

oquv = df[df.oy <= joriy_oy]                                            # ⚠️ belgi hali yo'q
oquv = df[df.oy <= joriy_oy - 2]                                        # ✅ 60 kunlik gorizont

qongiroq = top_k(hammasi)                                               # ⚠️ samara o'lchanmaydi
nazorat = rng.random(n) < 0.05; qongiroq = top_k(hammasi[~nazorat])     # ✅

1.  # oyiga 87.6 DPD60, sig'im 200 - sig'im cheklovmi?
2.  # zararsizlik chegarasi 0.021, bazaviy ulush 0.020 - tasodifiy qo'ng'iroq qiymati?
3.  # p o'rniga p * zarar bo'yicha tartib - AUC va tejam qayoqqa?
4.  # top-K dan tashqaridagi tartibni yaxshilasak - AUC va tejam?
5.  # 3 oy val da qoida "util" modeldan -4.59 (SE 2.93) - qaysi tanlanadi?
6.  # K = 50 va K = 800 - model/qoida nisbati qayerda katta?
7.  # K = 2000 - tejam K = 800 dan ko'pmi?
8.  # y30 lar ichida o'zi to'laganlar ulushi taxminan?
9.  # y30 da o'qitilgan model - AUC y30 bo'yicha yaxshimi, tejam bo'yicha-chi?
10. # 90 kunlik belgi - o'quv uchun oxirgi as-of qanday o'zgaradi?

1.  hisobot = f"Model AUC {auc:.2f} - loyiha muvaffaqiyatli"

2.  royxat = df[df.oy == t].nlargest(200, "p")

3.  foyda = tejam_k(qiymat, p * zarar, g) - tejam_k(qiymat, rng.random(n), g)
    print(f"model foydasi {foyda:.1f}")

4.  oquv = df[df.oy <= t]       # t - joriy as-of oy

5.  y = df["y30"]               # "kechikish" = 30 kun, ko'p musbat