29.3-dars: To'liq loyiha: modellashtirish va baholash

BAHOLASH REJASI (natijadan oldin yoziladi):
  1. MAQSAD VA METRIKA   oylik tejam, top-200, ball = p * zarar   (29.1 dan)
  2. YORDAMCHI           AUC, Brier, p/haqiqiy (kalibrlash)
  3. SXEMA               har oy t = 12-18: o'quv as-of <= t-2, baho t-oyda (7 juft)
  4. NOMZODLAR           soddalik tartibida: qoida -> LogReg -> HistGB -> MLP
  5. QAROR QOIDASI       eng yaxshisidan farqi > -2*SE bo'lgan ENG SODDA nomzod
  6. KEYINGI QADAMLAR    kalibrlash, siyosat, segmentlar - faqat val da
  7. TEST                20-22, bir marta, hamma narsa muhrlangandan keyin
MUHR: sha256(json(reja)) -> hisobotga va model kartasiga yoziladi

NOMZOD                 NEGA                                 NARXI
qoida: util            29.1 dagi bazaviy, darhol ishlaydi   0 (allaqachon bor)
LogReg                 chiziqli, kalibrlangan, tushunarli   past
HistGB                 chiziqsizlik va o'zaro ta'sir        o'rta (sozlash, tushuntirish)
MLP (kichik tarmoq)    moslashuvchan, katta ma'lumotda      yuqori (sozlash, barqarorlik)

as-of:  7  8  9 10 11 12 13 14 15 16 17 18
fold 1: [o'quv 7-10 ]  .  B              <- baho: 12  (11 - bo'shliq, belgi yetilmagan)
fold 2: [o'quv 7-11    ]  .  B           <- 13
 ...
fold 7: [o'quv 7-16               ]  .  B <- 18
         kengayuvchi oyna (expanding window), bo'shliq 1 oy

HAR FOLD - BITTA OY -> har nomzodning oylik tejami
JUFTLASHGAN FARQ: d_t = tejam(nomzod, t) - tejam(eng yaxshi, t)
  farq = mean(d),  SE = std(d) / sqrt(7)
  "sezilarli yomon", agar farq < -2*SE
JUFTLASH NEGA: oylar bir-biridan keskin farq qiladi (bazaviy ulush 0.015-0.026),
  lekin bir oyda hamma nomzod bir xil mijozlarda baholanadi -> oy shovqini ayiriladi

QAROR:         qo'ng'iroq, agar  EV = p * 0.30 * zarar - 0.02 > 0
KUTILGAN TEJAM: sum(EV) - rahbariyatga beriladigan prognoz
IKKALASI HAM p ning O'ZIGA bog'liq, faqat tartibiga emas

p 19 barobar oshirilsa (class_weight="balanced"):
  AUC        o'zgarmaydi 0.8483-bob
  EV > 0     deyarli hamma mijoz -> oyiga 4105 qo'ng'iroq
  prognoz    1308.8 mln so'm, haqiqat 0.47

TUZATISH (kalibrlash ma'lumoti o'quvdan ALOHIDA bo'lishi shart):
  Platt:     logit(p) ustida bir o'lchamli logistik regressiya (2 parametr)
  izotonik:  monoton pog'onali funksiya - ko'p ma'lumot kerak
  bizda:     7-12 da o'qitish, 13-14 da Platt, 16-18 da baholash
O'LCHOVLAR:  Brier, log-loss, ECE (kvantil bo'laklarda |p - ulush|), p/haqiqiy,
             ishonchlilik jadvali (bo'laklar bo'yicha bashorat/haqiqiy)

1. TOP-K (sig'im cheklovi)    har oy ball bo'yicha eng yuqori K ta
2. CHEGARA (sig'imsiz)         EV > 0 bo'lgan hammaga
3. ARALASH                     top-K, lekin faqat EV > 0 bo'lsa (ortiqcha qo'ng'iroq yo'q)

BALL: p emas, p * zarar (kutilgan tejam tartibi)

SIG'IM EGRI CHIZIG'I: K = 100, 200, ..., har biri uchun kutilgan tejam
  CHEGARAVIY QIYMAT: (tejam(K2) - tejam(K1)) / (K2 - K1)
  -> "yana 100 qo'ng'iroq oyiga qancha beradi?"
  -> qo'shimcha operator xarajati bilan solishtiriladi
  optimal K: chegaraviy qiymat 0 ga tushgan joy (EV > 0 lar soni)

TESTDAN OLDIN MUHRLANADI:  model turi, belgilar, o'quv oylari, siyosat, metrika
TEST:                       bitta ishga tushirish, natija jurnalga yoziladi
QULF:                       jurnal mavjud bo'lsa, qayta ochish rad etiladi
TEST YOMON CHIQSA:          natija O'SHANDAY yoziladi; tuzatish -> yangi reja,
                            yangi test ma'lumoti (keyingi oylar) bilan
TEST YAXSHI CHIQSA:         CV bilan farq tushuntiriladi (drift, tasodif), bayram emas

# Model kartasi: kredit karta kechikishi - profilaktik qo'ng'iroq (v1)

## Vazifa
Har oy oxirida hozir kechikmagan mijozlar orasidan top-200 ni tanlash (ball = p * kutilgan zarar).
Belgi: 60 kun ichida 60+ kun kechikish (y60). Foydalanuvchi: qo'ng'iroq markazi.

## Model
LogReg + StandardScaler, 15 belgi (utilizatsiya, to'lov xulqi, naqd yechish, staj ...).
O'quv: as-of 2024-07 - 2025-06. Reja muhri: <sha256>.

## Baholash
- CV (7 oy, kengayuvchi oyna): tejam, qoidadan farq (SE), AUC
- Test (3 oy, bir marta): tejam, qoidadan farq (SE), AUC, kalibrlash
- Segmentlar: yangi/eski, maosh loyihasi, avto to'lov, hudud - kalibrlash va recall

## Qaror siyosati
top-200, p * zarar; sig'im egri chizig'i: +100 qo'ng'iroqning chegaraviy qiymati.

## Cheklovlar va xatarlar
- Qo'ng'iroq samarasi 0.30-bob va zarar (0.45 * balans) faraziy -> pilot (29.4)
- Makroiqtisodiy drift: bashorat darajasi siljiydi -> oylik monitoring, qayta kalibrlash
- Qaror halqasi: 5% nazorat guruhi doimiy

## Foydalanmang
Kredit berish, limit o'zgartirish, mijozni jazolash uchun.

import hashlib
import json

import numpy as np

# reja muhri
muhr = hashlib.sha256(json.dumps(REJA, sort_keys=True).encode()).hexdigest()[:12]

# vaqt bo'yicha CV (kengayuvchi oyna, bo'shliq 1 oy)
for t in range(12, 19):
    tr, te = df[df.oy <= t - 2], df[df.oy == t]
    p = model.fit(tr[BELGILAR], tr.y60).predict_proba(te[BELGILAR])[:, 1]
    tejam[nom].append(tejam_oy(te, p * te.zarar.to_numpy()))

# juftlashgan qaror qoidasi
d = np.array(tejam[nom]) - np.array(tejam[eng])
sezilarli_yomon = d.mean() < -2 * d.std(ddof=1) / np.sqrt(len(d))

# kalibrlash va siyosat
ev = p * SAMARA * zarar - NARX                    # kutilgan qiymat
tanla = np.argsort(-(p * zarar))[:K]              # top-K
kutilgan_tejam = ev[tanla].sum()

# Platt (alohida kalibrlash oylarida)
lg = np.log(p_kal / (1 - p_kal))[:, None]
platt = LogisticRegression(C=1e6).fit(lg, y_kal)

reja (muhr) -> vaqt bo'yicha CV (7 oy) -> juftlashgan farq -> eng sodda munosib
kalibrlash: EV = p * 0.30 * zarar - 0.02 -> p ning o'zi muhim; balanced -> Platt
siyosat: top-K (p * zarar); sig'im egri chizig'i -> +100 qo'ng'iroq qiymati
segmentlar: p/haqiqiy (SE bilan), recall -> 29.11
test: bir marta, qulf, drift halol yoziladi -> model kartasi

"""Oldindan yozilgan baholash rejasi va vaqt bo'yicha CV: qoida, LogReg, HistGB, kichik tarmoq."""

import hashlib
import json
import warnings

import numpy as np
import pandas as pd
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.exceptions import ConvergenceWarning
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
from sklearn.neural_network import MLPClassifier
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

HUDUDLAR = ["Toshkent", "Samarqand", "Farg'ona", "Andijon", "Buxoro", "Qashqadaryo"]
OYLAR = 24                               # 1 = 2024-01, ..., 24 = 2025-12
BELGILAR = ["yosh", "daromad", "limit", "avto_tolov", "maosh_loyihasi", "staj",
            "util", "util_3", "util_ozg", "kechikish_6", "tolov_nisbati_3",
            "tolov_kuni_3", "naqd_soni_3", "naqd_summa_3", "xarid_soni_3"]
K, SAMARA, NARX = 200, 0.30, 0.02        # oyiga qo'ng'iroq, oldini olish, mln so'm


def sigmoid(z):
    return 1 / (1 + np.exp(-z))


def oy_nomi(m):
    return f"{2024 + (m - 1) // 12}-{(m - 1) % 12 + 1:02d}"


def yarat_bank(seed=29, n=6000):
    """Kredit karta portfeli: mijozlar, hisobvaraqlar, tolovlar, tranzaksiyalar."""
    rng = np.random.default_rng(seed)
    ids = np.arange(1, n + 1)
    yosh = rng.integers(21, 66, n)
    maosh = rng.random(n) < 0.45
    daromad = np.round(np.exp(rng.normal(np.log(6.0), 0.45, n)), 2)   # mln so'm
    ochilgan = np.where(rng.random(n) < 0.7, rng.integers(-40, 1, n),
                        rng.integers(1, 20, n))
    xavf = (rng.normal(0, 1, n) - 0.5 * maosh - 0.8 * np.log(daromad / 6)
            + 0.015 * (35 - yosh))                     # yashirin: moliyaviy xavf
    unut = rng.normal(0, 1, n) + 0.03 * (35 - yosh)     # yashirin: unutuvchanlik
    avto = (rng.random(n) < sigmoid(-0.2 - 0.9 * unut)).astype(int)
    limit = np.round(daromad * rng.uniform(1.5, 3.5, n), 1)
    hudud = rng.choice(HUDUDLAR, n, p=[0.34, 0.15, 0.14, 0.13, 0.12, 0.12])
    mj = pd.DataFrame({"mijoz_id": ids, "yosh": yosh, "hudud": hudud,
                       "daromad": daromad, "limit": limit, "avto_tolov": avto,
                       "maosh_loyihasi": maosh.astype(int), "ochilgan_oy": ochilgan})
    s, oldingi, ketma = rng.normal(0, 1, n), np.zeros(n, int), np.zeros(n, int)
    hv, tl, tr = [], [], []
    for m in range(1, OYLAR + 1):
        faol = (ochilgan <= m) & (ketma < 3)            # 3 ketma-ket kechikish - defolt
        s = 0.75 * s + 0.66 * rng.normal(0, 1, n)
        d = 0.8 * xavf + 0.7 * s + 0.15 * (m % 12 == 1) + 0.08 * max(m - 18, 0)
        util = np.clip(sigmoid(-0.4 + 0.55 * d + rng.normal(0, 0.5, n)), 0.01, 1)
        balans = np.round(util * limit, 3)
        min_t = np.round(np.maximum(0.05 * balans, 0.05), 3)
        for turi, lam in [("xarid", np.exp(1.2 + 0.15 * np.log(daromad / 6) - 0.1 * d)),
                          ("naqd", np.exp(-1.6 + 0.6 * d)),
                          ("onlayn", np.exp(0.6 - 0.02 * (yosh - 35)))]:
            i = np.repeat(np.arange(n), rng.poisson(lam) * faol)
            summa = (rng.uniform(0.2, 1.5, i.size) * (1 + 0.3 * d[i].clip(0))
                     if turi == "naqd" else np.exp(rng.normal(np.log(0.25), 0.8, i.size)))
            tr.append(pd.DataFrame({"mijoz_id": ids[i], "oy": m,
                                    "kun": rng.integers(1, 29, i.size), "turi": turi,
                                    "summa": np.round(summa, 3)}))
        q = rng.random(n) < sigmoid(-3.7 + 1.3 * d + 1.2 * (util > 0.85)
                                    + 3.8 * (oldingi == 2))          # qiyinchilik
        u = ~q & (rng.random(n) < sigmoid(-3.4 + 1.1 * unut - 1.8 * avto)
                  * (oldingi != 1))                                  # unutish
        miss = (q | u) & faol
        ulush = sigmoid(0.3 - 1.2 * d + rng.normal(0, 0.5, n))
        summa = np.where(miss, min_t * rng.uniform(0, 0.6, n),
                         min_t + (balans - min_t).clip(0) * ulush)
        tolaydi = faol & (~miss | (rng.random(n) < 0.4))
        kun = np.clip(np.round(6 + 14 * sigmoid(unut + 0.5 * d)
                               + rng.normal(0, 3, n)), 1, 25)
        hv.append(pd.DataFrame({"mijoz_id": ids[faol], "oy": m,
                                "balans": balans[faol], "min_tolov": min_t[faol]}))
        if m < OYLAR:                    # m-oy hisobvarag'i (m+1)-oyda to'lanadi
            tl.append(pd.DataFrame({"mijoz_id": ids[tolaydi], "hisob_oy": m,
                                    "tolov_oy": m + 1,
                                    "tolov_kuni": kun[tolaydi].astype(int),
                                    "summa": np.round(summa[tolaydi], 3)}))
        ketma = np.where(faol, np.where(miss, ketma + 1, 0), ketma)
        oldingi = np.where(miss, np.where(q, 2, 1), 0)
    return {"mijozlar": mj, "hisobvaraqlar": pd.concat(hv, ignore_index=True),
            "tolovlar": pd.concat(tl, ignore_index=True),
            "tranzaksiyalar": pd.concat(tr, ignore_index=True)}


def orta(a):
    """Qator bo'yicha NaN siz o'rtacha (bo'sh qatorda NaN, ogohlantirishsiz)."""
    k = (~np.isnan(a)).sum(axis=1)
    return np.where(k > 0, np.nansum(a, axis=1) / np.maximum(k, 1), np.nan)


def belgilar_jadvali(bank, oylar=range(7, 23)):
    """Har as-of oy t (oy oxiri): faqat t gacha ma'lum belgilar + t, t+1 natijasi."""
    mj = bank["mijozlar"]
    ids, kol = mj["mijoz_id"].to_numpy(), np.arange(1, OYLAR + 1)

    def mat(df, qiymat, ustun="oy"):
        return (df.pivot(index="mijoz_id", columns=ustun, values=qiymat)
                .reindex(index=ids, columns=kol).to_numpy(dtype=float))

    hv, tl = bank["hisobvaraqlar"], bank["tolovlar"]
    bal, mint = mat(hv, "balans"), mat(hv, "min_tolov")
    tol, kun = mat(tl, "summa", "hisob_oy"), mat(tl, "tolov_kuni", "hisob_oy")
    bor = ~np.isnan(bal)
    miss = bor & ~(np.nan_to_num(tol) >= mint - 1e-9)    # minimal to'lov qilinmagan
    g = (bank["tranzaksiyalar"].groupby(["turi", "mijoz_id", "oy"])["summa"]
         .agg(["size", "sum"]))

    def tr_mat(turi, ust):
        return (g.loc[turi, ust].unstack("oy").reindex(index=ids, columns=kol)
                .fillna(0).to_numpy())

    naqd_n, naqd_s = tr_mat("naqd", "size"), tr_mat("naqd", "sum")
    xarid_n = tr_mat("xarid", "size")
    util = bal / mj["limit"].to_numpy()[:, None]
    och = mj["ochilgan_oy"].to_numpy()
    qismlar = []
    for t in oylar:
        c = t - 1                                        # t-oy ustuni (0 dan)
        ok = (och <= t - 1) & bor[:, c] & bor[:, c - 1] & ~miss[:, c - 1]
        u3 = util[:, c - 3]
        f = mj.drop(columns="ochilgan_oy").assign(
            oy=t, staj=t - och, util=util[:, c], util_3=orta(util[:, c - 2:c + 1]),
            util_ozg=util[:, c] - np.where(np.isnan(u3), util[:, c], u3),
            kechikish_6=miss[:, max(c - 6, 0):c].sum(axis=1),
            tolov_nisbati_3=orta(np.nan_to_num(tol[:, c - 3:c]) / bal[:, c - 3:c]),
            tolov_kuni_3=orta(kun[:, c - 3:c]),
            naqd_soni_3=naqd_n[:, c - 2:c + 1].sum(axis=1),
            naqd_summa_3=naqd_s[:, c - 2:c + 1].sum(axis=1),
            xarid_soni_3=xarid_n[:, c - 2:c + 1].sum(axis=1),
            balans=bal[:, c], y30=(miss[:, c] | miss[:, c + 1]).astype(int),
            y60=(miss[:, c] & miss[:, c + 1]).astype(int))
        qismlar.append(f[ok])
    df = pd.concat(qismlar, ignore_index=True)
    df["tolov_kuni_3"] = df["tolov_kuni_3"].fillna(25.0)
    df["tolov_nisbati_3"] = df["tolov_nisbati_3"].fillna(0.0)
    df["zarar"] = 0.45 * df["balans"]              # DPD60 bo'lsa kutilgan zarar
    df["qiymat"] = SAMARA * df["zarar"] * df["y60"] - NARX   # qo'ng'iroq qiymati
    return df

REJA = {
    "maqsad": "oyiga K=200 qo'ng'iroq, tejam = 0.30*zarar*y60 - 0.02",
    "asosiy_metrika": "oylik tejam (top-200, ball = p * zarar)",
    "yordamchi": ["AUC", "Brier", "p/haqiqiy"],
    "cv": "har oy t = 12-18: o'quv as-of <= t-2, baho t-oyda (7 juft)",
    "nomzodlar_soddalik_tartibida": ["qoida: util", "LogReg", "HistGB", "MLP (16)"],
    "qaror_qoidasi": "eng yaxshisidan farqi > -2*SE bo'lgan eng sodda nomzod",
    "test": "20-22, bir marta, tanlov va siyosat muhrlangandan keyin",
}


def nomzodlar():
    return {
        "LogReg": make_pipeline(StandardScaler(), LogisticRegression(max_iter=2000)),
        "HistGB": HistGradientBoostingClassifier(
            max_iter=200, learning_rate=0.05, max_leaf_nodes=15, min_samples_leaf=100,
            l2_regularization=1.0, random_state=0),
        "MLP (16)": make_pipeline(StandardScaler(), MLPClassifier(
            hidden_layer_sizes=(16,), alpha=1e-2, batch_size=1024, learning_rate_init=3e-3,
            max_iter=40, random_state=0)),
    }


def tejam_oy(g, ball, k=K):
    return g["qiymat"].to_numpy()[np.argsort(-ball, kind="stable")[:k]].sum()


def main() -> None:
    df = belgilar_jadvali(yarat_bank())
    muhr = hashlib.sha256(json.dumps(REJA, sort_keys=True).encode()).hexdigest()[:12]
    print("=== 1. Baholash rejasi (natijadan OLDIN yozildi va muhrlandi) ===")
    for k, v in REJA.items():
        print(f"  {k}: {v}")
    print(f"  reja muhri (sha256): {muhr}")

    print("\n=== 2. Vaqt bo'yicha CV: oylik tejam (mln so'm) ===")
    nomlar = REJA["nomzodlar_soddalik_tartibida"]
    tejam = {n: [] for n in nomlar}
    auc = {n: [] for n in nomlar}
    for t in range(12, 19):
        tr, te = df[df.oy <= t - 2], df[df.oy == t]
        z = te["zarar"].to_numpy()
        ballar = {"qoida: util": te["util"].to_numpy()}
        for nom, m in nomzodlar().items():
            with warnings.catch_warnings():       # MLP ataylab 40 epoxa: ogohlantirish
                warnings.simplefilter("ignore", ConvergenceWarning)
                m.fit(tr[BELGILAR], tr["y60"])
            ballar[nom] = m.predict_proba(te[BELGILAR])[:, 1]
        for nom in nomlar:
            b = ballar[nom] if nom == "qoida: util" else ballar[nom] * z
            tejam[nom].append(tejam_oy(te, b))
            auc[nom].append(roc_auc_score(te["y60"], ballar[nom]))
    print(f"  {'oy':<8}" + "".join(f"{n:>13}" for n in nomlar))
    for i, t in enumerate(range(12, 19)):
        print(f"  {oy_nomi(t):<8}" + "".join(f"{tejam[n][i]:>13.2f}" for n in nomlar))
    orta_nom = "o'rtacha"
    print(f"  {orta_nom:<8}" + "".join(f"{np.mean(tejam[n]):>13.2f}" for n in nomlar))
    print(f"  {'AUC':<8}" + "".join(f"{np.mean(auc[n]):>13.4f}" for n in nomlar))

    print("\n=== 3. Juftlashgan taqqoslash (oylar bo'yicha) va qaror qoidasi ===")
    eng = max(nomlar, key=lambda n: np.mean(tejam[n]))
    print(f"  eng yaxshi o'rtacha: {eng}")
    tanlov = None
    for nom in nomlar:
        d = np.array(tejam[nom]) - np.array(tejam[eng])
        farq, se = d.mean(), d.std(ddof=1) / np.sqrt(len(d))
        yomon = farq < -2 * se
        yutgan = int((d > 0).sum())
        if not yomon and tanlov is None:
            tanlov = nom
        print(f"  {nom:<12} farq {farq:+6.2f}  SE {se:5.2f}  sezilarli yomon: {str(yomon):<5}"
              f"  yutgan oylar {yutgan}/7")
    print(f"  TANLOV (eng sodda munosib): {tanlov}")

    print("\n=== 4. Xulosa (natijadan hisoblangan) ===")
    d = np.array(tejam["LogReg"]) - np.array(tejam["qoida: util"])
    farq, se = d.mean(), d.std(ddof=1) / np.sqrt(len(d))
    if farq > 2 * se:
        print(f"  29.1 dagi ochiq savol: model qoidadan oyiga {farq:+.2f} (SE {se:.2f}) "
              f"ko'p tejaydi -> sezilarli ({int((d > 0).sum())}/7 oy)")
    else:
        print(f"  model qoidadan sezilarli yaxshi emas ({farq:+.2f}, SE {se:.2f})")
    if tanlov != eng:
        print(f"  {eng} o'rtachada yuqori, lekin farq shovqin ichida -> {tanlov} tanlandi")


if __name__ == "__main__":
    main()

=== 1. Baholash rejasi (natijadan OLDIN yozildi va muhrlandi) ===
  maqsad: oyiga K=200 qo'ng'iroq, tejam = 0.30*zarar*y60 - 0.02
  asosiy_metrika: oylik tejam (top-200, ball = p * zarar)
  yordamchi: ['AUC', 'Brier', 'p/haqiqiy']
  cv: har oy t = 12-18: o'quv as-of <= t-2, baho t-oyda (7 juft)
  nomzodlar_soddalik_tartibida: ['qoida: util', 'LogReg', 'HistGB', 'MLP (16)']
  qaror_qoidasi: eng yaxshisidan farqi > -2*SE bo'lgan eng sodda nomzod
  test: 20-22, bir marta, tanlov va siyosat muhrlangandan keyin
  reja muhri (sha256): af71c572eab7

=== 2. Vaqt bo'yicha CV: oylik tejam (mln so'm) ===
  oy        qoida: util       LogReg       HistGB     MLP (16)
  2024-12         27.17        33.09        30.09        31.39
  2025-01         28.08        36.98        37.03        27.90
  2025-02         21.08        32.54        35.86        31.94
  2025-03         22.94        26.53        25.01        23.81
  2025-04         21.62        25.45        27.54        26.63
  2025-05         32.25        33.06        34.11        31.17
  2025-06         18.89        29.22        33.53        27.18
  o'rtacha        24.58        30.98        31.88        28.58
  AUC            0.8008       0.8380       0.8300       0.8167

=== 3. Juftlashgan taqqoslash (oylar bo'yicha) va qaror qoidasi ===
  eng yaxshi o'rtacha: HistGB
  qoida: util  farq  -7.30  SE  2.13  sezilarli yomon: True   yutgan oylar 0/7
  LogReg       farq  -0.90  SE  0.98  sezilarli yomon: False  yutgan oylar 2/7
  HistGB       farq  +0.00  SE  0.00  sezilarli yomon: False  yutgan oylar 0/7
  MLP (16)     farq  -3.31  SE  1.34  sezilarli yomon: True   yutgan oylar 1/7
  TANLOV (eng sodda munosib): LogReg

=== 4. Xulosa (natijadan hisoblangan) ===
  29.1 dagi ochiq savol: model qoidadan oyiga +6.40 (SE 1.49) ko'p tejaydi -> sezilarli (7/7 oy)
  HistGB o'rtachada yuqori, lekin farq shovqin ichida -> LogReg tanlandi

"""Kalibrlash: ehtimollar qaror uchun kerak (Brier, ECE, "p * zarar > narx" siyosati)."""

import numpy as np
import pandas as pd
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import brier_score_loss, log_loss, roc_auc_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

HUDUDLAR = ["Toshkent", "Samarqand", "Farg'ona", "Andijon", "Buxoro", "Qashqadaryo"]
OYLAR = 24                               # 1 = 2024-01, ..., 24 = 2025-12
BELGILAR = ["yosh", "daromad", "limit", "avto_tolov", "maosh_loyihasi", "staj",
            "util", "util_3", "util_ozg", "kechikish_6", "tolov_nisbati_3",
            "tolov_kuni_3", "naqd_soni_3", "naqd_summa_3", "xarid_soni_3"]
K, SAMARA, NARX = 200, 0.30, 0.02        # oyiga qo'ng'iroq, oldini olish, mln so'm


def sigmoid(z):
    return 1 / (1 + np.exp(-z))


def oy_nomi(m):
    return f"{2024 + (m - 1) // 12}-{(m - 1) % 12 + 1:02d}"


def yarat_bank(seed=29, n=6000):
    """Kredit karta portfeli: mijozlar, hisobvaraqlar, tolovlar, tranzaksiyalar."""
    rng = np.random.default_rng(seed)
    ids = np.arange(1, n + 1)
    yosh = rng.integers(21, 66, n)
    maosh = rng.random(n) < 0.45
    daromad = np.round(np.exp(rng.normal(np.log(6.0), 0.45, n)), 2)   # mln so'm
    ochilgan = np.where(rng.random(n) < 0.7, rng.integers(-40, 1, n),
                        rng.integers(1, 20, n))
    xavf = (rng.normal(0, 1, n) - 0.5 * maosh - 0.8 * np.log(daromad / 6)
            + 0.015 * (35 - yosh))                     # yashirin: moliyaviy xavf
    unut = rng.normal(0, 1, n) + 0.03 * (35 - yosh)     # yashirin: unutuvchanlik
    avto = (rng.random(n) < sigmoid(-0.2 - 0.9 * unut)).astype(int)
    limit = np.round(daromad * rng.uniform(1.5, 3.5, n), 1)
    hudud = rng.choice(HUDUDLAR, n, p=[0.34, 0.15, 0.14, 0.13, 0.12, 0.12])
    mj = pd.DataFrame({"mijoz_id": ids, "yosh": yosh, "hudud": hudud,
                       "daromad": daromad, "limit": limit, "avto_tolov": avto,
                       "maosh_loyihasi": maosh.astype(int), "ochilgan_oy": ochilgan})
    s, oldingi, ketma = rng.normal(0, 1, n), np.zeros(n, int), np.zeros(n, int)
    hv, tl, tr = [], [], []
    for m in range(1, OYLAR + 1):
        faol = (ochilgan <= m) & (ketma < 3)            # 3 ketma-ket kechikish - defolt
        s = 0.75 * s + 0.66 * rng.normal(0, 1, n)
        d = 0.8 * xavf + 0.7 * s + 0.15 * (m % 12 == 1) + 0.08 * max(m - 18, 0)
        util = np.clip(sigmoid(-0.4 + 0.55 * d + rng.normal(0, 0.5, n)), 0.01, 1)
        balans = np.round(util * limit, 3)
        min_t = np.round(np.maximum(0.05 * balans, 0.05), 3)
        for turi, lam in [("xarid", np.exp(1.2 + 0.15 * np.log(daromad / 6) - 0.1 * d)),
                          ("naqd", np.exp(-1.6 + 0.6 * d)),
                          ("onlayn", np.exp(0.6 - 0.02 * (yosh - 35)))]:
            i = np.repeat(np.arange(n), rng.poisson(lam) * faol)
            summa = (rng.uniform(0.2, 1.5, i.size) * (1 + 0.3 * d[i].clip(0))
                     if turi == "naqd" else np.exp(rng.normal(np.log(0.25), 0.8, i.size)))
            tr.append(pd.DataFrame({"mijoz_id": ids[i], "oy": m,
                                    "kun": rng.integers(1, 29, i.size), "turi": turi,
                                    "summa": np.round(summa, 3)}))
        q = rng.random(n) < sigmoid(-3.7 + 1.3 * d + 1.2 * (util > 0.85)
                                    + 3.8 * (oldingi == 2))          # qiyinchilik
        u = ~q & (rng.random(n) < sigmoid(-3.4 + 1.1 * unut - 1.8 * avto)
                  * (oldingi != 1))                                  # unutish
        miss = (q | u) & faol
        ulush = sigmoid(0.3 - 1.2 * d + rng.normal(0, 0.5, n))
        summa = np.where(miss, min_t * rng.uniform(0, 0.6, n),
                         min_t + (balans - min_t).clip(0) * ulush)
        tolaydi = faol & (~miss | (rng.random(n) < 0.4))
        kun = np.clip(np.round(6 + 14 * sigmoid(unut + 0.5 * d)
                               + rng.normal(0, 3, n)), 1, 25)
        hv.append(pd.DataFrame({"mijoz_id": ids[faol], "oy": m,
                                "balans": balans[faol], "min_tolov": min_t[faol]}))
        if m < OYLAR:                    # m-oy hisobvarag'i (m+1)-oyda to'lanadi
            tl.append(pd.DataFrame({"mijoz_id": ids[tolaydi], "hisob_oy": m,
                                    "tolov_oy": m + 1,
                                    "tolov_kuni": kun[tolaydi].astype(int),
                                    "summa": np.round(summa[tolaydi], 3)}))
        ketma = np.where(faol, np.where(miss, ketma + 1, 0), ketma)
        oldingi = np.where(miss, np.where(q, 2, 1), 0)
    return {"mijozlar": mj, "hisobvaraqlar": pd.concat(hv, ignore_index=True),
            "tolovlar": pd.concat(tl, ignore_index=True),
            "tranzaksiyalar": pd.concat(tr, ignore_index=True)}


def orta(a):
    """Qator bo'yicha NaN siz o'rtacha (bo'sh qatorda NaN, ogohlantirishsiz)."""
    k = (~np.isnan(a)).sum(axis=1)
    return np.where(k > 0, np.nansum(a, axis=1) / np.maximum(k, 1), np.nan)


def belgilar_jadvali(bank, oylar=range(7, 23)):
    """Har as-of oy t (oy oxiri): faqat t gacha ma'lum belgilar + t, t+1 natijasi."""
    mj = bank["mijozlar"]
    ids, kol = mj["mijoz_id"].to_numpy(), np.arange(1, OYLAR + 1)

    def mat(df, qiymat, ustun="oy"):
        return (df.pivot(index="mijoz_id", columns=ustun, values=qiymat)
                .reindex(index=ids, columns=kol).to_numpy(dtype=float))

    hv, tl = bank["hisobvaraqlar"], bank["tolovlar"]
    bal, mint = mat(hv, "balans"), mat(hv, "min_tolov")
    tol, kun = mat(tl, "summa", "hisob_oy"), mat(tl, "tolov_kuni", "hisob_oy")
    bor = ~np.isnan(bal)
    miss = bor & ~(np.nan_to_num(tol) >= mint - 1e-9)    # minimal to'lov qilinmagan
    g = (bank["tranzaksiyalar"].groupby(["turi", "mijoz_id", "oy"])["summa"]
         .agg(["size", "sum"]))

    def tr_mat(turi, ust):
        return (g.loc[turi, ust].unstack("oy").reindex(index=ids, columns=kol)
                .fillna(0).to_numpy())

    naqd_n, naqd_s = tr_mat("naqd", "size"), tr_mat("naqd", "sum")
    xarid_n = tr_mat("xarid", "size")
    util = bal / mj["limit"].to_numpy()[:, None]
    och = mj["ochilgan_oy"].to_numpy()
    qismlar = []
    for t in oylar:
        c = t - 1                                        # t-oy ustuni (0 dan)
        ok = (och <= t - 1) & bor[:, c] & bor[:, c - 1] & ~miss[:, c - 1]
        u3 = util[:, c - 3]
        f = mj.drop(columns="ochilgan_oy").assign(
            oy=t, staj=t - och, util=util[:, c], util_3=orta(util[:, c - 2:c + 1]),
            util_ozg=util[:, c] - np.where(np.isnan(u3), util[:, c], u3),
            kechikish_6=miss[:, max(c - 6, 0):c].sum(axis=1),
            tolov_nisbati_3=orta(np.nan_to_num(tol[:, c - 3:c]) / bal[:, c - 3:c]),
            tolov_kuni_3=orta(kun[:, c - 3:c]),
            naqd_soni_3=naqd_n[:, c - 2:c + 1].sum(axis=1),
            naqd_summa_3=naqd_s[:, c - 2:c + 1].sum(axis=1),
            xarid_soni_3=xarid_n[:, c - 2:c + 1].sum(axis=1),
            balans=bal[:, c], y30=(miss[:, c] | miss[:, c + 1]).astype(int),
            y60=(miss[:, c] & miss[:, c + 1]).astype(int))
        qismlar.append(f[ok])
    df = pd.concat(qismlar, ignore_index=True)
    df["tolov_kuni_3"] = df["tolov_kuni_3"].fillna(25.0)
    df["tolov_nisbati_3"] = df["tolov_nisbati_3"].fillna(0.0)
    df["zarar"] = 0.45 * df["balans"]              # DPD60 bo'lsa kutilgan zarar
    df["qiymat"] = SAMARA * df["zarar"] * df["y60"] - NARX   # qo'ng'iroq qiymati
    return df


def ece(y, p, bolak=10):
    """Kutilgan kalibrlash xatosi: kvantil bo'laklarda |o'rtacha p - haqiqiy ulush|."""
    chegara = np.quantile(p, np.linspace(0, 1, bolak + 1))
    idx = np.clip(np.searchsorted(chegara, p, side="right") - 1, 0, bolak - 1)
    return sum(abs(p[idx == b].mean() - y[idx == b].mean()) * (idx == b).mean()
               for b in range(bolak) if (idx == b).any())


def platt(p_kal, y_kal):
    """Platt: logit(p) ustida bir o'lchamli logistik regressiya."""
    lg = np.log(p_kal / (1 - p_kal))[:, None]
    m = LogisticRegression(C=1e6, max_iter=1000).fit(lg, y_kal)
    return lambda p: m.predict_proba(np.log(p / (1 - p))[:, None])[:, 1]


def siyosat_tejam(va, p):
    """Sig'imsiz siyosat: kutilgan qiymat musbat bo'lsa qo'ng'iroq."""
    ev = p * SAMARA * va["zarar"].to_numpy() - NARX
    tanlandi = ev > 0
    m = va["oy"].nunique()
    return (tanlandi.sum() / m, va["qiymat"].to_numpy()[tanlandi].sum() / m,
            ev[tanlandi].sum() / m)


def main() -> None:
    df = belgilar_jadvali(yarat_bank())
    tr, va = df[df.oy <= 14], df[df.oy.between(16, 18)].reset_index(drop=True)
    y = va["y60"].to_numpy()

    def lr(**kw):
        return make_pipeline(StandardScaler(), LogisticRegression(max_iter=2000, **kw))

    modellar = {}
    modellar["LogReg"] = lr().fit(tr[BELGILAR], tr.y60).predict_proba(va[BELGILAR])[:, 1]
    hgb = HistGradientBoostingClassifier(max_iter=200, learning_rate=0.05, max_leaf_nodes=15,
                                         min_samples_leaf=100, l2_regularization=1.0,
                                         random_state=0)
    modellar["HistGB"] = hgb.fit(tr[BELGILAR], tr.y60).predict_proba(va[BELGILAR])[:, 1]
    bal = lr(class_weight="balanced").fit(tr[BELGILAR], tr.y60)
    modellar["LogReg balanced"] = bal.predict_proba(va[BELGILAR])[:, 1]
    # tuzatish: 7-12 da o'qitib, 13-14 da Platt (kalibrlash ma'lumoti o'quvdan ajratilgan)
    a, b = tr[tr.oy <= 12], tr[tr.oy >= 13]
    bal2 = lr(class_weight="balanced").fit(a[BELGILAR], a.y60)
    f = platt(bal2.predict_proba(b[BELGILAR])[:, 1], b.y60.to_numpy())
    modellar["balanced + Platt"] = f(bal2.predict_proba(va[BELGILAR])[:, 1])

    print("=== 1. Kalibrlash o'lchovlari (o'quv 7-14, val 16-18) ===")
    print(f"  haqiqiy y60 ulushi val da: {y.mean():.4f}")
    print(f"  {'model':<18} {'AUC':>7} {'Brier':>8} {'log-loss':>9} {'ECE':>8} {'p/haqiqiy':>10}")
    for nom, p in modellar.items():
        print(f"  {nom:<18} {roc_auc_score(y, p):>7.4f} {brier_score_loss(y, p):>8.5f} "
              f"{log_loss(y, p):>9.4f} {ece(y, p):>8.4f} {p.mean() / y.mean():>10.2f}")

    print("\n=== 2. Ishonchlilik jadvali (LogReg va LogReg balanced, 5 kvantil) ===")
    for nom in ["LogReg", "LogReg balanced"]:
        p = modellar[nom]
        q = pd.qcut(p, 5, labels=False)
        qator = [f"{p[q == i].mean():.3f}/{y[q == i].mean():.3f}" for i in range(5)]
        print(f"  {nom:<16} (bashorat/haqiqiy): " + "  ".join(qator))

    print("\n=== 3. Qaror: \"p * 0.30 * zarar > 0.02\" bo'lsa qo'ng'iroq (sig'imsiz) ===")
    sarlavha = "qo'ng'iroq/oy"
    print(f"  {'model':<18} {sarlavha:>13} {'haqiqiy tejam':>14} {'kutilgan':>9}")
    natija = {}
    for nom, p in modellar.items():
        natija[nom] = siyosat_tejam(va, p)
        n, h, k = natija[nom]
        print(f"  {nom:<18} {n:>13.0f} {h:>14.2f} {k:>9.2f}")

    print("\n=== 4. Top-200 (p * zarar) - sig'im bilan ===")
    for nom, p in modellar.items():
        t = np.mean([va["qiymat"].to_numpy()[g][np.argsort(-(p * va["zarar"].to_numpy())[g],
                                                           kind="stable")[:K]].sum()
                     for g in [np.flatnonzero(va.oy.to_numpy() == m) for m in (16, 17, 18)]])
        print(f"  {nom:<18} tejam {t:6.2f} mln so'm/oy")

    print("\n=== 5. Xulosa (natijadan hisoblangan) ===")
    for nom, (n, h, k) in natija.items():
        if abs(k - h) > 0.25 * abs(h) + 1:
            print(f"  {nom}: oyiga {n:.0f} qo'ng'iroq, kutilgan {k:.1f}, haqiqiy {h:.2f} "
                  f"-> kalibrlanmagan ehtimol bilan qaror va prognoz noto'g'ri")
        else:
            print(f"  {nom}: kutilgan {k:.1f} va haqiqiy {h:.1f} - yaqin")
    print(f"  sig'imsiz optimal: oyiga ~{natija['LogReg'][0]:.0f} qo'ng'iroq; sig'im {K} "
          f"-> cheklov bog'lovchi (3-misol)")

if __name__ == "__main__":
    main()

=== 1. Kalibrlash o'lchovlari (o'quv 7-14, val 16-18) ===
  haqiqiy y60 ulushi val da: 0.0174
  model                  AUC    Brier  log-loss      ECE  p/haqiqiy
  LogReg              0.8482  0.01616    0.0722   0.0022       1.09
  HistGB              0.8319  0.01618    0.0730   0.0022       1.08
  LogReg balanced     0.8483  0.16114    0.4895   0.3160      19.14
  balanced + Platt    0.8486  0.01621    0.0723   0.0031       1.18

=== 2. Ishonchlilik jadvali (LogReg va LogReg balanced, 5 kvantil) ===
  LogReg           (bashorat/haqiqiy): 0.002/0.000  0.004/0.002  0.008/0.007  0.017/0.018  0.063/0.060
  LogReg balanced  (bashorat/haqiqiy): 0.076/0.000  0.157/0.003  0.272/0.007  0.444/0.016  0.718/0.061

=== 3. Qaror: "p * 0.30 * zarar > 0.02" bo'lsa qo'ng'iroq (sig'imsiz) ===
  model              qo'ng'iroq/oy  haqiqiy tejam  kutilgan
  LogReg                      1015          49.12     51.13
  HistGB                      1002          46.27     50.18
  LogReg balanced             4105           0.47   1308.80
  balanced + Platt            1060          49.54     57.67

=== 4. Top-200 (p * zarar) - sig'im bilan ===
  LogReg             tejam  28.85 mln so'm/oy
  HistGB             tejam  29.17 mln so'm/oy
  LogReg balanced    tejam  28.05 mln so'm/oy
  balanced + Platt   tejam  29.34 mln so'm/oy

=== 5. Xulosa (natijadan hisoblangan) ===
  LogReg: kutilgan 51.1 va haqiqiy 49.1 - yaqin
  HistGB: kutilgan 50.2 va haqiqiy 46.3 - yaqin
  LogReg balanced: oyiga 4105 qo'ng'iroq, kutilgan 1308.8, haqiqiy 0.47 -> kalibrlanmagan ehtimol bilan qaror va prognoz noto'g'ri
  balanced + Platt: kutilgan 57.7 va haqiqiy 49.5 - yaqin
  sig'imsiz optimal: oyiga ~1015 qo'ng'iroq; sig'im 200 -> cheklov bog'lovchi (3-misol)

"""Siyosat va sig'im: top-K, kutilgan tejam, qo'shimcha operator qiymati; segmentlar bo'yicha xato."""

import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

HUDUDLAR = ["Toshkent", "Samarqand", "Farg'ona", "Andijon", "Buxoro", "Qashqadaryo"]
OYLAR = 24                               # 1 = 2024-01, ..., 24 = 2025-12
BELGILAR = ["yosh", "daromad", "limit", "avto_tolov", "maosh_loyihasi", "staj",
            "util", "util_3", "util_ozg", "kechikish_6", "tolov_nisbati_3",
            "tolov_kuni_3", "naqd_soni_3", "naqd_summa_3", "xarid_soni_3"]
K, SAMARA, NARX = 200, 0.30, 0.02        # oyiga qo'ng'iroq, oldini olish, mln so'm


def sigmoid(z):
    return 1 / (1 + np.exp(-z))


def oy_nomi(m):
    return f"{2024 + (m - 1) // 12}-{(m - 1) % 12 + 1:02d}"


def yarat_bank(seed=29, n=6000):
    """Kredit karta portfeli: mijozlar, hisobvaraqlar, tolovlar, tranzaksiyalar."""
    rng = np.random.default_rng(seed)
    ids = np.arange(1, n + 1)
    yosh = rng.integers(21, 66, n)
    maosh = rng.random(n) < 0.45
    daromad = np.round(np.exp(rng.normal(np.log(6.0), 0.45, n)), 2)   # mln so'm
    ochilgan = np.where(rng.random(n) < 0.7, rng.integers(-40, 1, n),
                        rng.integers(1, 20, n))
    xavf = (rng.normal(0, 1, n) - 0.5 * maosh - 0.8 * np.log(daromad / 6)
            + 0.015 * (35 - yosh))                     # yashirin: moliyaviy xavf
    unut = rng.normal(0, 1, n) + 0.03 * (35 - yosh)     # yashirin: unutuvchanlik
    avto = (rng.random(n) < sigmoid(-0.2 - 0.9 * unut)).astype(int)
    limit = np.round(daromad * rng.uniform(1.5, 3.5, n), 1)
    hudud = rng.choice(HUDUDLAR, n, p=[0.34, 0.15, 0.14, 0.13, 0.12, 0.12])
    mj = pd.DataFrame({"mijoz_id": ids, "yosh": yosh, "hudud": hudud,
                       "daromad": daromad, "limit": limit, "avto_tolov": avto,
                       "maosh_loyihasi": maosh.astype(int), "ochilgan_oy": ochilgan})
    s, oldingi, ketma = rng.normal(0, 1, n), np.zeros(n, int), np.zeros(n, int)
    hv, tl, tr = [], [], []
    for m in range(1, OYLAR + 1):
        faol = (ochilgan <= m) & (ketma < 3)            # 3 ketma-ket kechikish - defolt
        s = 0.75 * s + 0.66 * rng.normal(0, 1, n)
        d = 0.8 * xavf + 0.7 * s + 0.15 * (m % 12 == 1) + 0.08 * max(m - 18, 0)
        util = np.clip(sigmoid(-0.4 + 0.55 * d + rng.normal(0, 0.5, n)), 0.01, 1)
        balans = np.round(util * limit, 3)
        min_t = np.round(np.maximum(0.05 * balans, 0.05), 3)
        for turi, lam in [("xarid", np.exp(1.2 + 0.15 * np.log(daromad / 6) - 0.1 * d)),
                          ("naqd", np.exp(-1.6 + 0.6 * d)),
                          ("onlayn", np.exp(0.6 - 0.02 * (yosh - 35)))]:
            i = np.repeat(np.arange(n), rng.poisson(lam) * faol)
            summa = (rng.uniform(0.2, 1.5, i.size) * (1 + 0.3 * d[i].clip(0))
                     if turi == "naqd" else np.exp(rng.normal(np.log(0.25), 0.8, i.size)))
            tr.append(pd.DataFrame({"mijoz_id": ids[i], "oy": m,
                                    "kun": rng.integers(1, 29, i.size), "turi": turi,
                                    "summa": np.round(summa, 3)}))
        q = rng.random(n) < sigmoid(-3.7 + 1.3 * d + 1.2 * (util > 0.85)
                                    + 3.8 * (oldingi == 2))          # qiyinchilik
        u = ~q & (rng.random(n) < sigmoid(-3.4 + 1.1 * unut - 1.8 * avto)
                  * (oldingi != 1))                                  # unutish
        miss = (q | u) & faol
        ulush = sigmoid(0.3 - 1.2 * d + rng.normal(0, 0.5, n))
        summa = np.where(miss, min_t * rng.uniform(0, 0.6, n),
                         min_t + (balans - min_t).clip(0) * ulush)
        tolaydi = faol & (~miss | (rng.random(n) < 0.4))
        kun = np.clip(np.round(6 + 14 * sigmoid(unut + 0.5 * d)
                               + rng.normal(0, 3, n)), 1, 25)
        hv.append(pd.DataFrame({"mijoz_id": ids[faol], "oy": m,
                                "balans": balans[faol], "min_tolov": min_t[faol]}))
        if m < OYLAR:                    # m-oy hisobvarag'i (m+1)-oyda to'lanadi
            tl.append(pd.DataFrame({"mijoz_id": ids[tolaydi], "hisob_oy": m,
                                    "tolov_oy": m + 1,
                                    "tolov_kuni": kun[tolaydi].astype(int),
                                    "summa": np.round(summa[tolaydi], 3)}))
        ketma = np.where(faol, np.where(miss, ketma + 1, 0), ketma)
        oldingi = np.where(miss, np.where(q, 2, 1), 0)
    return {"mijozlar": mj, "hisobvaraqlar": pd.concat(hv, ignore_index=True),
            "tolovlar": pd.concat(tl, ignore_index=True),
            "tranzaksiyalar": pd.concat(tr, ignore_index=True)}


def orta(a):
    """Qator bo'yicha NaN siz o'rtacha (bo'sh qatorda NaN, ogohlantirishsiz)."""
    k = (~np.isnan(a)).sum(axis=1)
    return np.where(k > 0, np.nansum(a, axis=1) / np.maximum(k, 1), np.nan)


def belgilar_jadvali(bank, oylar=range(7, 23)):
    """Har as-of oy t (oy oxiri): faqat t gacha ma'lum belgilar + t, t+1 natijasi."""
    mj = bank["mijozlar"]
    ids, kol = mj["mijoz_id"].to_numpy(), np.arange(1, OYLAR + 1)

    def mat(df, qiymat, ustun="oy"):
        return (df.pivot(index="mijoz_id", columns=ustun, values=qiymat)
                .reindex(index=ids, columns=kol).to_numpy(dtype=float))

    hv, tl = bank["hisobvaraqlar"], bank["tolovlar"]
    bal, mint = mat(hv, "balans"), mat(hv, "min_tolov")
    tol, kun = mat(tl, "summa", "hisob_oy"), mat(tl, "tolov_kuni", "hisob_oy")
    bor = ~np.isnan(bal)
    miss = bor & ~(np.nan_to_num(tol) >= mint - 1e-9)    # minimal to'lov qilinmagan
    g = (bank["tranzaksiyalar"].groupby(["turi", "mijoz_id", "oy"])["summa"]
         .agg(["size", "sum"]))

    def tr_mat(turi, ust):
        return (g.loc[turi, ust].unstack("oy").reindex(index=ids, columns=kol)
                .fillna(0).to_numpy())

    naqd_n, naqd_s = tr_mat("naqd", "size"), tr_mat("naqd", "sum")
    xarid_n = tr_mat("xarid", "size")
    util = bal / mj["limit"].to_numpy()[:, None]
    och = mj["ochilgan_oy"].to_numpy()
    qismlar = []
    for t in oylar:
        c = t - 1                                        # t-oy ustuni (0 dan)
        ok = (och <= t - 1) & bor[:, c] & bor[:, c - 1] & ~miss[:, c - 1]
        u3 = util[:, c - 3]
        f = mj.drop(columns="ochilgan_oy").assign(
            oy=t, staj=t - och, util=util[:, c], util_3=orta(util[:, c - 2:c + 1]),
            util_ozg=util[:, c] - np.where(np.isnan(u3), util[:, c], u3),
            kechikish_6=miss[:, max(c - 6, 0):c].sum(axis=1),
            tolov_nisbati_3=orta(np.nan_to_num(tol[:, c - 3:c]) / bal[:, c - 3:c]),
            tolov_kuni_3=orta(kun[:, c - 3:c]),
            naqd_soni_3=naqd_n[:, c - 2:c + 1].sum(axis=1),
            naqd_summa_3=naqd_s[:, c - 2:c + 1].sum(axis=1),
            xarid_soni_3=xarid_n[:, c - 2:c + 1].sum(axis=1),
            balans=bal[:, c], y30=(miss[:, c] | miss[:, c + 1]).astype(int),
            y60=(miss[:, c] & miss[:, c + 1]).astype(int))
        qismlar.append(f[ok])
    df = pd.concat(qismlar, ignore_index=True)
    df["tolov_kuni_3"] = df["tolov_kuni_3"].fillna(25.0)
    df["tolov_nisbati_3"] = df["tolov_nisbati_3"].fillna(0.0)
    df["zarar"] = 0.45 * df["balans"]              # DPD60 bo'lsa kutilgan zarar
    df["qiymat"] = SAMARA * df["zarar"] * df["y60"] - NARX   # qo'ng'iroq qiymati
    return df


def tanla(va, ball, k, faqat_musbat_ev=False, p=None):
    """Har oyda ball bo'yicha top-k indekslari (ixtiyoriy: kutilgan qiymat > 0 shart)."""
    oy = va["oy"].to_numpy()
    tanlangan = []
    for m in np.unique(oy):
        g = np.flatnonzero(oy == m)
        g = g[np.argsort(-ball[g], kind="stable")[:k]]
        if faqat_musbat_ev:
            g = g[p[g] * SAMARA * va["zarar"].to_numpy()[g] > NARX]
        tanlangan.append(g)
    return np.concatenate(tanlangan)


def main() -> None:
    df = belgilar_jadvali(yarat_bank())
    tr, va = df[df.oy <= 14], df[df.oy.between(16, 18)].reset_index(drop=True)
    m = make_pipeline(StandardScaler(), LogisticRegression(max_iter=2000))
    p = m.fit(tr[BELGILAR], tr["y60"]).predict_proba(va[BELGILAR])[:, 1]
    z, q, y = va["zarar"].to_numpy(), va["qiymat"].to_numpy(), va["y60"].to_numpy()
    ev = p * SAMARA * z - NARX                     # har qo'ng'iroqning kutilgan qiymati
    oylar = va["oy"].nunique()

    print("=== 1. Siyosatlar (K = 200, val 16-18, LogReg) ===")
    siyosatlar = {
        "top-200: p": tanla(va, p, K),
        "top-200: p * zarar": tanla(va, p * z, K),
        "top-200: p * zarar, EV > 0": tanla(va, p * z, K, True, p),
    }
    for nom, s in siyosatlar.items():
        print(f"  {nom:<28} qo'ng'iroq/oy {len(s) / oylar:6.1f}  tejam {q[s].sum() / oylar:6.2f}"
              f"  kutilgan {ev[s].sum() / oylar:6.2f}")

    print("\n=== 2. Sig'im egri chizig'i: K oshsa (oyiga, mln so'm) ===")
    sarlavha = "+100 qo'ng'iroq"
    print(f"  {'K':>5} {'tejam':>7} {'kutilgan':>9} {sarlavha:>16}")
    oldingi = None
    egri = {}
    for k in [100, 200, 300, 400, 600, 800, 1000, 1500]:
        s = tanla(va, p * z, k)
        egri[k] = (q[s].sum() / oylar, ev[s].sum() / oylar)
        chegara = ("" if oldingi is None
                   else f"{(egri[k][1] - oldingi[1]) / (k - oldingi[0]) * 100:+.2f}")
        print(f"  {k:>5} {egri[k][0]:>7.2f} {egri[k][1]:>9.2f} {chegara:>16}")
        oldingi = (k, egri[k][1])
    eng_k = max(egri, key=lambda k: egri[k][1])
    print(f"  kutilgan tejam maksimumi: K = {eng_k} atrofida")

    print("\n=== 3. Segmentlar bo'yicha xato tahlili (K = 200, p * zarar) ===")
    tan = np.zeros(len(va), bool)
    tan[siyosatlar["top-200: p * zarar"]] = True
    segmentlar = {
        "yangi (staj<6)": va.staj < 6, "eski": va.staj >= 6,
        "maosh loyihasi": va.maosh_loyihasi == 1, "maoshsiz": va.maosh_loyihasi == 0,
        "avto to'lov": va.avto_tolov == 1, "avto yo'q": va.avto_tolov == 0,
    }
    for h in sorted(va.hudud.unique()):
        segmentlar[h] = va.hudud == h
    print(f"  {'segment':<16} {'n':>6} {'y60':>5} {'p/haqiqiy':>10} {'SE':>6} "
          f"{'tanlangan':>10} {'recall':>7}")
    shubhali = []
    for nom, s in segmentlar.items():
        s = s.to_numpy()
        n1 = y[s].sum()
        kal = p[s].sum() / max(n1, 1)
        se = kal / np.sqrt(max(n1, 1))                 # Puasson: nisbiy xato ~ 1/sqrt(n)
        recall = (tan & s & (y == 1)).sum() / max(n1, 1)
        print(f"  {nom:<16} {s.sum():>6} {n1:>5} {kal:>10.2f} {se:>6.2f} "
              f"{tan[s].mean():>10.3f} {recall:>7.3f}")
        if abs(kal - 1) > 2 * se:
            shubhali.append(nom)

    print("\n=== 4. Xulosa (natijadan hisoblangan) ===")
    a, b = egri[200][1], egri[300][1]
    print(f"  sig'im 200 -> 300: sof kutilgan tejam {b - a:+.2f} mln so'm/oy "
          f"(qo'ng'iroq narxidan keyin, har qo'shimcha qo'ng'iroq ~{(b - a) / 100 * 1000:.0f} ming)")
    print(f"  qo'shimcha sig'imning boshqa xarajatlari (ish o'rni, o'qitish) oyiga "
          f"{b - a:.1f} mln so'mdan kam bo'lsa - foydali")
    if shubhali:
        print(f"  kalibrlash 2*SE dan ko'p siljigan segmentlar: {shubhali}")
    else:
        print("  hech bir segmentda kalibrlash 2*SE dan ko'p siljimagan")


if __name__ == "__main__":
    main()

=== 1. Siyosatlar (K = 200, val 16-18, LogReg) ===
  top-200: p                   qo'ng'iroq/oy  200.0  tejam  26.32  kutilgan  30.49
  top-200: p * zarar           qo'ng'iroq/oy  200.0  tejam  28.85  kutilgan  33.66
  top-200: p * zarar, EV > 0   qo'ng'iroq/oy  200.0  tejam  28.85  kutilgan  33.66

=== 2. Sig'im egri chizig'i: K oshsa (oyiga, mln so'm) ===
      K   tejam  kutilgan  +100 qo'ng'iroq
    100   23.27     24.46
    200   28.85     33.66            +9.20
    300   35.83     39.45            +5.80
    400   38.55     43.33            +3.88
    600   48.28     48.10            +2.39
    800   48.44     50.42            +1.16
   1000   48.76     51.11            +0.35
   1500   45.62     48.90            -0.44
  kutilgan tejam maksimumi: K = 1000 atrofida

=== 3. Segmentlar bo'yicha xato tahlili (K = 200, p * zarar) ===
  segment               n   y60  p/haqiqiy     SE  tanlangan  recall
  yangi (staj<6)     1369    30       1.19   0.22      0.073   0.400
  eski              11834   200       1.07   0.08      0.042   0.250
  maosh loyihasi     6383    82       1.20   0.13      0.028   0.195
  maoshsiz           6820   148       1.03   0.08      0.062   0.311
  avto to'lov        6848   111       1.04   0.10      0.037   0.234
  avto yo'q          6355   119       1.13   0.10      0.055   0.303
  Andijon            1773    35       1.04   0.18      0.049   0.286
  Buxoro             1543    25       1.18   0.24      0.047   0.240
  Farg'ona           1883    31       1.07   0.19      0.040   0.323
  Qashqadaryo        1505    31       0.97   0.17      0.049   0.258
  Samarqand          2004    34       1.14   0.20      0.054   0.235
  Toshkent           4495    74       1.11   0.13      0.041   0.270

=== 4. Xulosa (natijadan hisoblangan) ===
  sig'im 200 -> 300: sof kutilgan tejam +5.80 mln so'm/oy (qo'ng'iroq narxidan keyin, har qo'shimcha qo'ng'iroq ~58 ming)
  qo'shimcha sig'imning boshqa xarajatlari (ish o'rni, o'qitish) oyiga 5.8 mln so'mdan kam bo'lsa - foydali
  hech bir segmentda kalibrlash 2*SE dan ko'p siljimagan

"""Test bir marta: yakuniy model, test qulfi, qoida bilan juftlashgan natija va model kartasi."""

import hashlib
import json
import pathlib
import tempfile

import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

HUDUDLAR = ["Toshkent", "Samarqand", "Farg'ona", "Andijon", "Buxoro", "Qashqadaryo"]
OYLAR = 24                               # 1 = 2024-01, ..., 24 = 2025-12
BELGILAR = ["yosh", "daromad", "limit", "avto_tolov", "maosh_loyihasi", "staj",
            "util", "util_3", "util_ozg", "kechikish_6", "tolov_nisbati_3",
            "tolov_kuni_3", "naqd_soni_3", "naqd_summa_3", "xarid_soni_3"]
K, SAMARA, NARX = 200, 0.30, 0.02        # oyiga qo'ng'iroq, oldini olish, mln so'm


def sigmoid(z):
    return 1 / (1 + np.exp(-z))


def oy_nomi(m):
    return f"{2024 + (m - 1) // 12}-{(m - 1) % 12 + 1:02d}"


def yarat_bank(seed=29, n=6000):
    """Kredit karta portfeli: mijozlar, hisobvaraqlar, tolovlar, tranzaksiyalar."""
    rng = np.random.default_rng(seed)
    ids = np.arange(1, n + 1)
    yosh = rng.integers(21, 66, n)
    maosh = rng.random(n) < 0.45
    daromad = np.round(np.exp(rng.normal(np.log(6.0), 0.45, n)), 2)   # mln so'm
    ochilgan = np.where(rng.random(n) < 0.7, rng.integers(-40, 1, n),
                        rng.integers(1, 20, n))
    xavf = (rng.normal(0, 1, n) - 0.5 * maosh - 0.8 * np.log(daromad / 6)
            + 0.015 * (35 - yosh))                     # yashirin: moliyaviy xavf
    unut = rng.normal(0, 1, n) + 0.03 * (35 - yosh)     # yashirin: unutuvchanlik
    avto = (rng.random(n) < sigmoid(-0.2 - 0.9 * unut)).astype(int)
    limit = np.round(daromad * rng.uniform(1.5, 3.5, n), 1)
    hudud = rng.choice(HUDUDLAR, n, p=[0.34, 0.15, 0.14, 0.13, 0.12, 0.12])
    mj = pd.DataFrame({"mijoz_id": ids, "yosh": yosh, "hudud": hudud,
                       "daromad": daromad, "limit": limit, "avto_tolov": avto,
                       "maosh_loyihasi": maosh.astype(int), "ochilgan_oy": ochilgan})
    s, oldingi, ketma = rng.normal(0, 1, n), np.zeros(n, int), np.zeros(n, int)
    hv, tl, tr = [], [], []
    for m in range(1, OYLAR + 1):
        faol = (ochilgan <= m) & (ketma < 3)            # 3 ketma-ket kechikish - defolt
        s = 0.75 * s + 0.66 * rng.normal(0, 1, n)
        d = 0.8 * xavf + 0.7 * s + 0.15 * (m % 12 == 1) + 0.08 * max(m - 18, 0)
        util = np.clip(sigmoid(-0.4 + 0.55 * d + rng.normal(0, 0.5, n)), 0.01, 1)
        balans = np.round(util * limit, 3)
        min_t = np.round(np.maximum(0.05 * balans, 0.05), 3)
        for turi, lam in [("xarid", np.exp(1.2 + 0.15 * np.log(daromad / 6) - 0.1 * d)),
                          ("naqd", np.exp(-1.6 + 0.6 * d)),
                          ("onlayn", np.exp(0.6 - 0.02 * (yosh - 35)))]:
            i = np.repeat(np.arange(n), rng.poisson(lam) * faol)
            summa = (rng.uniform(0.2, 1.5, i.size) * (1 + 0.3 * d[i].clip(0))
                     if turi == "naqd" else np.exp(rng.normal(np.log(0.25), 0.8, i.size)))
            tr.append(pd.DataFrame({"mijoz_id": ids[i], "oy": m,
                                    "kun": rng.integers(1, 29, i.size), "turi": turi,
                                    "summa": np.round(summa, 3)}))
        q = rng.random(n) < sigmoid(-3.7 + 1.3 * d + 1.2 * (util > 0.85)
                                    + 3.8 * (oldingi == 2))          # qiyinchilik
        u = ~q & (rng.random(n) < sigmoid(-3.4 + 1.1 * unut - 1.8 * avto)
                  * (oldingi != 1))                                  # unutish
        miss = (q | u) & faol
        ulush = sigmoid(0.3 - 1.2 * d + rng.normal(0, 0.5, n))
        summa = np.where(miss, min_t * rng.uniform(0, 0.6, n),
                         min_t + (balans - min_t).clip(0) * ulush)
        tolaydi = faol & (~miss | (rng.random(n) < 0.4))
        kun = np.clip(np.round(6 + 14 * sigmoid(unut + 0.5 * d)
                               + rng.normal(0, 3, n)), 1, 25)
        hv.append(pd.DataFrame({"mijoz_id": ids[faol], "oy": m,
                                "balans": balans[faol], "min_tolov": min_t[faol]}))
        if m < OYLAR:                    # m-oy hisobvarag'i (m+1)-oyda to'lanadi
            tl.append(pd.DataFrame({"mijoz_id": ids[tolaydi], "hisob_oy": m,
                                    "tolov_oy": m + 1,
                                    "tolov_kuni": kun[tolaydi].astype(int),
                                    "summa": np.round(summa[tolaydi], 3)}))
        ketma = np.where(faol, np.where(miss, ketma + 1, 0), ketma)
        oldingi = np.where(miss, np.where(q, 2, 1), 0)
    return {"mijozlar": mj, "hisobvaraqlar": pd.concat(hv, ignore_index=True),
            "tolovlar": pd.concat(tl, ignore_index=True),
            "tranzaksiyalar": pd.concat(tr, ignore_index=True)}


def orta(a):
    """Qator bo'yicha NaN siz o'rtacha (bo'sh qatorda NaN, ogohlantirishsiz)."""
    k = (~np.isnan(a)).sum(axis=1)
    return np.where(k > 0, np.nansum(a, axis=1) / np.maximum(k, 1), np.nan)


def belgilar_jadvali(bank, oylar=range(7, 23)):
    """Har as-of oy t (oy oxiri): faqat t gacha ma'lum belgilar + t, t+1 natijasi."""
    mj = bank["mijozlar"]
    ids, kol = mj["mijoz_id"].to_numpy(), np.arange(1, OYLAR + 1)

    def mat(df, qiymat, ustun="oy"):
        return (df.pivot(index="mijoz_id", columns=ustun, values=qiymat)
                .reindex(index=ids, columns=kol).to_numpy(dtype=float))

    hv, tl = bank["hisobvaraqlar"], bank["tolovlar"]
    bal, mint = mat(hv, "balans"), mat(hv, "min_tolov")
    tol, kun = mat(tl, "summa", "hisob_oy"), mat(tl, "tolov_kuni", "hisob_oy")
    bor = ~np.isnan(bal)
    miss = bor & ~(np.nan_to_num(tol) >= mint - 1e-9)    # minimal to'lov qilinmagan
    g = (bank["tranzaksiyalar"].groupby(["turi", "mijoz_id", "oy"])["summa"]
         .agg(["size", "sum"]))

    def tr_mat(turi, ust):
        return (g.loc[turi, ust].unstack("oy").reindex(index=ids, columns=kol)
                .fillna(0).to_numpy())

    naqd_n, naqd_s = tr_mat("naqd", "size"), tr_mat("naqd", "sum")
    xarid_n = tr_mat("xarid", "size")
    util = bal / mj["limit"].to_numpy()[:, None]
    och = mj["ochilgan_oy"].to_numpy()
    qismlar = []
    for t in oylar:
        c = t - 1                                        # t-oy ustuni (0 dan)
        ok = (och <= t - 1) & bor[:, c] & bor[:, c - 1] & ~miss[:, c - 1]
        u3 = util[:, c - 3]
        f = mj.drop(columns="ochilgan_oy").assign(
            oy=t, staj=t - och, util=util[:, c], util_3=orta(util[:, c - 2:c + 1]),
            util_ozg=util[:, c] - np.where(np.isnan(u3), util[:, c], u3),
            kechikish_6=miss[:, max(c - 6, 0):c].sum(axis=1),
            tolov_nisbati_3=orta(np.nan_to_num(tol[:, c - 3:c]) / bal[:, c - 3:c]),
            tolov_kuni_3=orta(kun[:, c - 3:c]),
            naqd_soni_3=naqd_n[:, c - 2:c + 1].sum(axis=1),
            naqd_summa_3=naqd_s[:, c - 2:c + 1].sum(axis=1),
            xarid_soni_3=xarid_n[:, c - 2:c + 1].sum(axis=1),
            balans=bal[:, c], y30=(miss[:, c] | miss[:, c + 1]).astype(int),
            y60=(miss[:, c] & miss[:, c + 1]).astype(int))
        qismlar.append(f[ok])
    df = pd.concat(qismlar, ignore_index=True)
    df["tolov_kuni_3"] = df["tolov_kuni_3"].fillna(25.0)
    df["tolov_nisbati_3"] = df["tolov_nisbati_3"].fillna(0.0)
    df["zarar"] = 0.45 * df["balans"]              # DPD60 bo'lsa kutilgan zarar
    df["qiymat"] = SAMARA * df["zarar"] * df["y60"] - NARX   # qo'ng'iroq qiymati
    return df


def lr():
    return make_pipeline(StandardScaler(), LogisticRegression(max_iter=2000))


def oylik_tejam(d, ball, k=K):
    oy = d["oy"].to_numpy()
    q = d["qiymat"].to_numpy()
    return np.array([q[g][np.argsort(-ball[g], kind="stable")[:k]].sum()
                     for g in [np.flatnonzero(oy == m) for m in np.unique(oy)]])


def testni_och(papka, yozuv):
    """Test belgilariga faqat bir marta ruxsat: jurnal fayli qulf vazifasini bajaradi."""
    jurnal = pathlib.Path(papka) / "test_jurnali.json"
    if jurnal.exists():
        eski = json.loads(jurnal.read_text(encoding="utf8"))
        raise RuntimeError(f"test allaqachon ochilgan (reja {eski['reja_muhri']})")
    jurnal.write_text(json.dumps(yozuv, ensure_ascii=False), encoding="utf8")


def main() -> None:
    df = belgilar_jadvali(yarat_bank())
    reja = {"model": "LogReg", "siyosat": "top-200, p * zarar", "oquv": "as-of 7-18",
            "test": "as-of 20-22", "metrika": "oylik tejam; qoida bilan juftlashgan"}
    muhr = hashlib.sha256(json.dumps(reja, sort_keys=True).encode()).hexdigest()[:12]

    print("=== 1. Testdan oldin: CV bahosi (7 oy, 12-18) ===")
    cv_m, cv_q = [], []
    for t in range(12, 19):
        tr, te = df[df.oy <= t - 2], df[df.oy == t]
        p = lr().fit(tr[BELGILAR], tr.y60).predict_proba(te[BELGILAR])[:, 1]
        cv_m.append(oylik_tejam(te, p * te.zarar.to_numpy())[0])
        cv_q.append(oylik_tejam(te, te.util.to_numpy())[0])
    d = np.array(cv_m) - np.array(cv_q)
    print(f"  model {np.mean(cv_m):.2f}, qoida {np.mean(cv_q):.2f}, farq {d.mean():+.2f} "
          f"(SE {d.std(ddof=1) / np.sqrt(len(d)):.2f}) mln so'm/oy")
    print(f"  reja muhri: {muhr} -> yakuniy model as-of 7-18 da o'qitiladi (19 - bo'shliq)")

    oquv = df[df.oy <= 18]
    yakuniy = lr().fit(oquv[BELGILAR], oquv.y60)
    test = df[df.oy.between(20, 22)].reset_index(drop=True)
    with tempfile.TemporaryDirectory() as papka:
        testni_och(papka, {"reja_muhri": muhr, **reja})
        print("\n=== 2. TEST OCHILDI (bir marta): as-of 20-22 ===")
        p = yakuniy.predict_proba(test[BELGILAR])[:, 1]
        z = test.zarar.to_numpy()
        tm, tq = oylik_tejam(test, p * z), oylik_tejam(test, test.util.to_numpy())
        for i, m in enumerate(range(20, 23)):
            g = test.oy == m
            print(f"  {oy_nomi(m)}: model {tm[i]:6.2f}, qoida {tq[i]:6.2f}, "
                  f"y60 {test.y60[g].mean():.4f}, bashorat {p[g].mean():.4f}")
        rng = np.random.default_rng(3)
        guruh = [np.flatnonzero(test.oy.to_numpy() == m) for m in range(20, 23)]
        farqlar = []
        for _ in range(300):
            idx = [rng.choice(g, len(g)) for g in guruh]
            sub = test.iloc[np.concatenate(idx)].reset_index(drop=True)
            pp = p[np.concatenate(idx)]
            farqlar.append(oylik_tejam(sub, pp * sub.zarar.to_numpy()).mean()
                           - oylik_tejam(sub, sub.util.to_numpy()).mean())
        farq, se = tm.mean() - tq.mean(), np.std(farqlar, ddof=1)
        print(f"  o'rtacha: model {tm.mean():.2f}, qoida {tq.mean():.2f}, farq {farq:+.2f} "
              f"(bootstrap SE {se:.2f}), sezilarli: {farq > 2 * se}")
        auc = roc_auc_score(test.y60, p)
        kal = p.mean() / test.y60.mean()
        print(f"  AUC {auc:.4f}; bashorat / haqiqiy ulush: {kal:.3f}")

        print("\n=== 3. Testni qayta ochishga urinish ===")
        try:
            testni_och(papka, {"reja_muhri": "yangi", **reja})
        except RuntimeError as xato:
            print(f"  rad etildi: {xato}")

    print("\n=== 4. Model kartasi (raqamlar natijadan) ===")
    karta = [
        "## Model kartasi: kredit karta kechikishi - profilaktik qo'ng'iroq (v1)",
        "- Vazifa: har oy top-200 mijoz (p * kutilgan zarar); belgi y60 (60 kunda 60+ kun)",
        f"- Model: LogReg + StandardScaler, {len(BELGILAR)} belgi; o'quv as-of 7-18 "
        f"({len(oquv)} qator)",
        f"- CV (12-18): tejam {np.mean(cv_m):.2f} mln so'm/oy, qoidadan {d.mean():+.2f}",
        f"- Test (20-22): tejam {tm.mean():.2f} mln so'm/oy, qoidadan {farq:+.2f} "
        f"(SE {se:.2f}); AUC {auc:.4f}",
        f"- Kalibrlash testda: bashorat/haqiqiy {kal:.3f} -> oylik monitoring va qayta "
        f"kalibrlash",
        "- Cheklovlar: samara 0.30 va zarar 0.45*balans faraziy; pilot kerak 29.4-bob",
        "- Foydalanmang: kredit berish yoki limit qarori uchun",
        f"- Reja muhri: {muhr}",
    ]
    for q in karta:
        print("  " + q)

    print("\n=== 5. Xulosa (natijadan hisoblangan) ===")
    if farq > 2 * se:
        print(f"  testda model qoidadan sezilarli yaxshi: {farq:+.2f} mln so'm/oy")
    else:
        print(f"  testda farq sezilarli emas ({farq:+.2f}, SE {se:.2f}) - halol yoziladi")
    print(f"  CV bahosi {np.mean(cv_m):.2f}, test {tm.mean():.2f}: "
          f"{'test yuqori' if tm.mean() > np.mean(cv_m) else 'test past'}")
    if kal < 0.9:
        print("  testda model xavfni kam baholadi (drift) -> kalibrlash monitoringi shart")


if __name__ == "__main__":
    main()

=== 1. Testdan oldin: CV bahosi (7 oy, 12-18) ===
  model 30.98, qoida 24.58, farq +6.40 (SE 1.49) mln so'm/oy
  reja muhri: dbfafe034225 -> yakuniy model as-of 7-18 da o'qitiladi (19 - bo'shliq)

=== 2. TEST OCHILDI (bir marta): as-of 20-22 ===
  2025-08: model  47.69, qoida  32.88, y60 0.0260, bashorat 0.0198
  2025-09: model  43.71, qoida  30.55, y60 0.0227, bashorat 0.0203
  2025-10: model  35.58, qoida  29.79, y60 0.0212, bashorat 0.0215
  o'rtacha: model 42.33, qoida 31.08, farq +11.25 (bootstrap SE 3.98), sezilarli: True
  AUC 0.8296; bashorat / haqiqiy ulush: 0.880

=== 3. Testni qayta ochishga urinish ===
  rad etildi: test allaqachon ochilgan (reja dbfafe034225)

=== 4. Model kartasi (raqamlar natijadan) ===
  ## Model kartasi: kredit karta kechikishi - profilaktik qo'ng'iroq (v1)
  - Vazifa: har oy top-200 mijoz (p * kutilgan zarar); belgi y60 (60 kunda 60+ kun)
  - Model: LogReg + StandardScaler, 15 belgi; o'quv as-of 7-18 (50894 qator)
  - CV (12-18): tejam 30.98 mln so'm/oy, qoidadan +6.40
  - Test (20-22): tejam 42.33 mln so'm/oy, qoidadan +11.25 (SE 3.98); AUC 0.8296
  - Kalibrlash testda: bashorat/haqiqiy 0.880 -> oylik monitoring va qayta kalibrlash
  - Cheklovlar: samara 0.30 va zarar 0.45*balans faraziy; pilot kerak 29.4-bob
  - Foydalanmang: kredit berish yoki limit qarori uchun
  - Reja muhri: dbfafe034225

=== 5. Xulosa (natijadan hisoblangan) ===
  testda model qoidadan sezilarli yaxshi: +11.25 mln so'm/oy
  CV bahosi 30.98, test 42.33: test yuqori
  testda model xavfni kam baholadi (drift) -> kalibrlash monitoringi shart

eng = max(variantlar, key=lambda v: val_auc(v))                         # ⚠️ 50 variant
REJA = {...}; muhr = sha256(REJA); eng = qaror_qoidasi(cv_natija)       # ✅

print(np.mean(tejam_a), np.mean(tejam_b))                               # ⚠️
d = np.array(tejam_a) - np.array(tejam_b); se = d.std(ddof=1) / np.sqrt(len(d))  # ✅

m = LogisticRegression(class_weight="balanced"); qongiroq = p * 0.3 * z > 0.02  # ⚠️
p = platt(p_kal, y_kal)(p)  # alohida oylarda kalibrlash, keyin qaror           # ✅

platt(model.predict_proba(X_oquv)[:, 1], y_oquv)                        # ⚠️ optimistik
platt(model.predict_proba(X_kal)[:, 1], y_kal)       # 13-14 oylar       # ✅

tanla = np.argsort(-p)[:K]                                              # ⚠️
tanla = np.argsort(-(p * zarar))[:K]                                    # ✅

for v in variantlar: print(test_tejam(v))                               # ⚠️
testni_och(papka, {"reja_muhri": muhr, ...})   # ikkinchi marta - RuntimeError  # ✅

print(f"model {tm.mean():.1f} mln tejaydi")                            # ⚠️
print(f"CV {cv:.1f}, test {tm.mean():.1f} (y60 {y_test:.4f} va {y_oquv:.4f}); p/haqiqiy {kal:.2f}")  # ✅

1.  # 7 oylik CV da LogReg qoidadan necha oyda yutdi?
2.  # HistGB o'rtachada eng yaxshi - nega tanlanmadi?
3.  # class_weight="balanced" AUC ni qanchaga o'zgartiradi? p/haqiqiy?
4.  # balanced model bilan "EV > 0" siyosati oyiga nechta qo'ng'iroq?
5.  # top-200 da kalibrlash xatosi tejamni keskin o'zgartiradimi?
6.  # sig'im 200 -> 300: chegaraviy kutilgan tejam?
7.  # kutilgan tejam qaysi K da maksimal?
8.  # Buxoro segmentida p/haqiqiy 1.18 (SE 0.24) - muammomi?
9.  # test tejami CV dan yuqori chiqdi - model yaxshilanganmi?
10. # testda bashorat/haqiqiy 0.880 - nima qilish kerak?

1.  for c in [0.01, 0.1, 1, 10]:
        for d in [3, 5, 8]:
            natija[(c, d)] = test_tejam(model(c, d))
    eng = max(natija, key=natija.get)

2.  m = LogisticRegression(class_weight="balanced").fit(X, y)
    qongiroq = m.predict_proba(X_yangi)[:, 1] * 0.3 * zarar > 0.02

3.  print("HistGB tanlandi: o'rtacha tejam eng yuqori 31.88-bob")

4.  segment = "Buxoro"; print("Buxoroda model xavfni 18% oshirib baholaydi - tuzatish kerak")