28.11-dars: Sababiy xulosa (causal inference)

KUZATILGAN:  kupon olganlar 60 ming so'm ko'p xarid qildi
SAVOL:       kupon BERILSA, xarid qanchaga oshadi?

Farq manbalari:
  1. SABAB         kupon -> xarid                       (biz izlayotgan)
  2. CHALKASHTIRISH sodiqlik -> kupon, sodiqlik -> xarid (yashirin umumiy sabab)
  3. TESKARI        xarid -> kupon (ko'p xarid qilganga kupon yuboriladi)
  4. TANLASH        faqat "sharh qoldirganlar" tahlil qilindi (kollayder)
  5. TASODIF        kichik namuna, ko'p taqqoslash 11.9-bob

Korrelyatsiya = 1 + 2 + 3 + 4 + 5 aralashmasi.
Sababiy xulosa = 2-5 ni ajratib, faqat 1 ni baholash.

Har mijoz i uchun IKKI potentsial natija:
  Y_i(1) - kupon berilsa, keyingi oydagi xarid
  Y_i(0) - kupon berilmasa
Individual ta'sir:  tau_i = Y_i(1) - Y_i(0)

ASOSIY MUAMMO: har mijozda faqat BITTASI kuzatiladi
  Y_i = T_i * Y_i(1) + (1 - T_i) * Y_i(0)
  ikkinchisi - "kontrfaktual" (bo'lmagan olam)

O'RTACHA TA'SIRLAR (estimand - nimani baholaymiz):
  ATE  = E[Y(1) - Y(0)]            butun populyatsiya
  ATT  = E[Y(1) - Y(0) | T = 1]    kupon OLGANLAR orasida
  ATC  = E[Y(1) - Y(0) | T = 0]    olmaganlar orasida
  CATE = E[Y(1) - Y(0) | X = x]    belgilar bo'yicha (uplift)

NAIV FARQ nima beradi:
  E[Y | T=1] - E[Y | T=0]
    = ATT  +  ( E[Y(0) | T=1] - E[Y(0) | T=0] )
               \_____ tanlash siljishi (selection bias) _____/
  sodiqlar kupon olsa: E[Y(0)|T=1] > E[Y(0)|T=0] -> naiv farq OSHIRIB ko'rsatadi

1. IGNORABILITY (chalkashtiruvchi qolmagan):  (Y(1), Y(0)) mustaqil T | X
   X ni hisobga olgach, kupon olish "tasodifiy" - hamma umumiy sabablar X da
   TEKSHIRIB BO'LMAYDI - faqat domen bilimi bilan asoslanadi
2. POSITIVITY (overlap):  0 < P(T=1 | X) < 1  har X uchun
   har turdagi mijozda kupon olgan ham, olmagan ham bor
   TEKSHIRSA BO'LADI - propensity taqsimoti bilan
3. SUTVA / izchillik: bir mijozning kuponi boshqasiga ta'sir qilmaydi,
   "kupon" bitta aniq muolaja (20% lik, bir xil muddat)

CHALKASHTIRUVCHI (fork)        MEDIATOR (chain)            KOLLAYDER
      sodiqlik                 kupon -> tashrif -> xarid    kupon      xarid
      /      \                  \_________________/^           \      /
     v        v                    (bevosita)                   v    v
  kupon ---> xarid                                             sharh

NAZORAT QILISH KERAK         UMUMIY ta'sir uchun NAZORAT    NAZORAT QILMASLIK
(orqa eshikni yopadi)        QILMASLIK kerak; nazorat       kerak! Kollayder bo'yicha
                             qilsangiz - faqat BEVOSITA     filtr/nazorat mustaqil
                             ta'sir qoladi                  kupon va xarid orasida
                                                            SOXTA bog'lanish yaratadi

ORQA ESHIK MEZONI (backdoor criterion):
  T dan Y ga ta'sirni baholash uchun Z to'plami yetarli, agar:
    1. Z hech bir elementi T ning avlodi (natijasi) emas
    2. Z T <- ... -> Y ko'rinishidagi barcha "orqa eshik" yo'llarni to'sadi
  shunda  ATE = sum_z ( E[Y | T=1, Z=z] - E[Y | T=0, Z=z] ) * P(Z=z)

"YOMON NAZORAT" (bad controls):
  mediator     - ta'sirning bir qismini yeb qo'yadi
  kollayder    - soxta bog'lanish ochadi
  T dan KEYIN o'lchangan har narsa (T ning natijasi bo'lishi mumkin)
QOIDA: "hamma belgini modelga qo'shaman" - sababiy tahlilda XATO

                  kuponsiz   kupon    farq
yangi mijozlar     5.0%       8.0%    +3.0   <- har segmentda kupon YORDAM beradi
doimiy mijozlar   30.0%      33.0%    +3.0
UMUMIY            ~20%       ~13%     MANFIY <- kuponlar asosan yangi mijozlarga
                                                (past bazaviy konversiya) berilgan
Qaysi javob to'g'ri? Segment kuponga SABAB bo'lgani uchun (segment -> kupon)
-> segment ichidagi farq va segment ulushlari bilan standartlash.
Agar segment kuponning NATIJASI bo'lsa (mediator) - umumiy farq to'g'ri bo'lardi.

1. REGRESSIYA BILAN TUZATISH
   Y ~ T + X  -> T koeffitsienti
   ta'sir X ga bog'liq bo'lsa - bu ATE EMAS, balki variansga qarab
   vaznlangan o'rtacha; model shakli noto'g'ri bo'lsa - siljish
   G-HISOBLASH (standartlash): mu_1(x) = E[Y | T=1, X=x] va mu_0(x) alohida
   ATE = mean_i ( mu_1(x_i) - mu_0(x_i) )

2. PROPENSITY SCORE:  e(x) = P(T = 1 | X = x)   (logistik regressiya, 13.10)
   Rosenbaum-Rubin: X bo'yicha ignorability bo'lsa, e(X) bo'yicha ham bor
   -> ko'p o'lchovli X o'rniga BITTA son bo'yicha tenglashtirish

3. MOSLASHTIRISH (matching):  har kupon olgan mijozga e(x) si eng yaqin
   kupon OLMAGAN mijoz; farqlar o'rtachasi -> ATT
   logit(e) bo'yicha, "caliper" (masalan 0.2 * std) - uzoq juftlar tashlanadi
   qaytarib (with replacement) - bitta nazorat bir necha marta ishlatiladi

4. IPW (inverse probability weighting):
   w = T / e  +  (1 - T) / (1 - e)
   g'oya: kupon olish ehtimoli past bo'lgan kupon oluvchi - "kamyob",
          u o'ziga o'xshash ko'p odam uchun "gapiradi"
   Hajek (normallangan):  sum(w*T*Y)/sum(w*T) - sum(w*(1-T)*Y)/sum(w*(1-T))
   ATT uchun: kupon olganlar w = 1, olmaganlar w = e / (1 - e)

OVERLAP TEKSHIRUVI:
  e(x) taqsimotini guruhlar bo'yicha ko'ring (min, kvantillar, max)
  e ~ 0 yoki e ~ 1 bo'lgan hudud - u yerda taqqoslash uchun "egizak" yo'q
  maks og'irlik, samarali namuna hajmi: ESS = (sum w)^2 / sum w^2

BALANS (SMD - standartlashgan o'rtachalar farqi):
  SMD = (orta_1 - orta_0) / sqrt((var_1 + var_0) / 2)
  vaznlashdan (yoki moslashtirishdan) KEYIN |SMD| < 0.1 - odatiy mezon
  balans yo'q -> propensity modeli yetarli emas (belgilar, nochiziqlik)

KESISH (trimming/clipping):
  e ni [0.05, 0.95] ga kesish yoki o'sha hududdagi qatorlarni tashlash
  dispersiya keskin kamayadi, lekin BAHOLANADIGAN kattalik o'zgaradi:
  endi "overlap hududidagi mijozlar uchun ta'sir"  - buni hisobotda ayting

AIPW (augmented IPW):
  psi_i = mu_1(x_i) - mu_0(x_i)
          + T_i * (Y_i - mu_1(x_i)) / e(x_i)
          - (1 - T_i) * (Y_i - mu_0(x_i)) / (1 - e(x_i))
  ATE = mean(psi_i)

NEGA "IKKI KARRA":
  natija modeli (mu) TO'G'RI bo'lsa - tuzatish hadlari o'rtachada 0 -> to'g'ri
  propensity (e) TO'G'RI bo'lsa     - tuzatish mu xatosini aynan yo'qotadi
  IKKALASI noto'g'ri bo'lsa         - kafolat yo'q
BONUS: ikkalasi yaxshi bo'lsa - dispersiya IPW dan ancha kichik
AMALDA: mu va e ni ML modellari (HistGB) bilan, cross-fitting bilan
        (bir qismda o'qitib, boshqasida baholash) - "double ML" g'oyasi

Barcha yuqoridagi usullar BITTA farazga tayanadi: hamma chalkashtiruvchilar X da.
Agar U (masalan, mijoz daromadi) kupon olishga ham, xaridga ham ta'sir qilsa
va ma'lumotda YO'Q bo'lsa:
  regressiya, moslashtirish, IPW, AIPW - HAMMASI siljigan
  va hech biri buni o'zi "sezmaydi" (balans X bo'yicha mukammal bo'lishi mumkin)

NIMA QILISH MUMKIN:
  sezgirlik tahlili - "U qanchalik kuchli bo'lsa, xulosa o'zgaradi?"
    (E-qiymat: kuzatilgan ta'sirni yo'qotish uchun U ning ham T, ham Y bilan
     bog'lanishi qanchalik kuchli bo'lishi kerak)
  boshqa dizayn: instrumental o'zgaruvchi (IV), regressiya uzilishi (RDD),
    farqlar farqi (DiD, 2.11), tabiiy tajriba
  eng yaxshisi - RCT

TO'RT SEGMENT (xarid ehtimoli bo'yicha):
                        kuponsiz OLMAYDI      kuponsiz OLADI
  kupon bilan OLADI     ISHONTIRILADIGAN      "BARIBIR OLADI"
                        (persuadables) +      (sure things) 0 - kupon isrof
  kupon bilan OLMAYDI   "YO'QOTILGAN"         "BEZOVTA QILMA"
                        (lost causes) 0       (sleeping dogs) - zarar!

JAVOB MODELI (xato!): P(xarid | X) yuqorilarga kupon -> asosan "baribir oladi"

META-LEARNER LAR:
  S-learner: bitta model f(X, T);   uplift = f(x, 1) - f(x, 0)
             kamchilik: model T ni "kuchsiz" belgi sifatida e'tiborsiz
             qoldirishi mumkin -> uplift nolga siqiladi
  T-learner: ikki model f1 (T=1 da), f0 (T=0 da); uplift = f1(x) - f0(x)
             kamchilik: ikki modelning xatolari qo'shiladi
  X-learner, DR-learner - ikkalasining yaxshi tomonlarini birlashtiradi

BAHOLASH - faqat RCT (yoki ishonchli tuzatilgan) ma'lumotda:
  QINI EGRI CHIZIG'I: mijozlarni bashorat qilingan uplift bo'yicha saralab,
  birinchi k ta ichida:
    Qini(k) = Y_1(k) - Y_0(k) * N_1(k) / N_0(k)
    (kupon olganlar xaridlari - kuponsizlar xaridlari, hajmga moslab)
  = "shu k ta mijozga kupon berilsa, qo'shimcha xaridlar soni"
  tasodifiy tartib - to'g'ri chiziq; maydon farqi - Qini koeffitsienti

                oldin (1-26 hafta)   keyin (27-52 hafta)   farq
Samarqand           A_oldin              A_keyin            A_keyin - A_oldin
Toshkent            B_oldin              B_keyin            B_keyin - B_oldin
DiD = (A_keyin - A_oldin) - (B_keyin - B_oldin)

NAIV 1: A_keyin - A_oldin      - trend va mavsumni ta'sir deb oladi
NAIV 2: A_keyin - B_keyin      - shaharlar orasidagi doimiy farqni ta'sir deb oladi
DiD: ikkalasini ham yo'qotadi, AGAR:

PARALLEL TRENDLAR FARAZI: aksiya bo'lmaganda Samarqand va Toshkent
  bir xil DINAMIKADA o'zgarardi (darajalari har xil bo'lishi mumkin)
  TEKSHIRUV: aksiyadan oldingi davrda "soxta aksiya" (placebo) DiD ~ 0 bo'lishi
  kerak; oldingi haftalar bo'yicha farqlar grafigi (event study)
  BUZILSA: Samarqand tezroq o'sayotgan bo'lsa - DiD o'sishni ta'sirga qo'shadi
SE: do'kon bo'yicha (klaster) - bir do'konning haftalari mustaqil emas

import numpy as np
from sklearn.linear_model import LinearRegression, LogisticRegression

# propensity score
e = LogisticRegression(C=1e6, max_iter=1000).fit(X, T).predict_proba(X)[:, 1]

# overlap va balans
w = T / e + (1 - T) / (1 - e)
ess = w[T == 1].sum() ** 2 / (w[T == 1] ** 2).sum()


def smd(x, T, w):
    m1 = np.average(x[T == 1], weights=w[T == 1])
    m0 = np.average(x[T == 0], weights=w[T == 0])
    return (m1 - m0) / np.sqrt((x[T == 1].var() + x[T == 0].var()) / 2)


# g-hisoblash (T-learner) va AIPW
mu1 = LinearRegression().fit(X[T == 1], Y[T == 1]).predict(X)
mu0 = LinearRegression().fit(X[T == 0], Y[T == 0]).predict(X)
ate_g = np.mean(mu1 - mu0)
ate_ipw = (np.sum(T * Y / e) / np.sum(T / e)
           - np.sum((1 - T) * Y / (1 - e)) / np.sum((1 - T) / (1 - e)))
psi = mu1 - mu0 + T * (Y - mu1) / e - (1 - T) * (Y - mu0) / (1 - e)
ate_aipw = psi.mean()

# DiD (do'kon darajasida)
farq = y[:, keyin].mean(1) - y[:, oldin].mean(1)
did = farq[muolaja].mean() - farq[~muolaja].mean()
se = np.sqrt(farq[muolaja].var(ddof=1) / muolaja.sum()
             + farq[~muolaja].var(ddof=1) / (~muolaja).sum())

# DoWhy - DAG bilan savolni qo'yish, baholash va "rad etish" testlari
from dowhy import CausalModel

model = CausalModel(data=df, treatment="kupon", outcome="xarid",
                    common_causes=["sodiqlik", "oldingi", "faollik"])
estimand = model.identify_effect()
baho = model.estimate_effect(estimand,
                             method_name="backdoor.propensity_score_weighting")
rad = model.refute_estimate(estimand, baho, method_name="placebo_treatment_refuter")

# EconML - CATE va double ML
from econml.dml import LinearDML
from econml.metalearners import TLearner
from sklearn.ensemble import HistGradientBoostingRegressor

dml = LinearDML(model_y=HistGradientBoostingRegressor(),
                model_t=HistGradientBoostingRegressor(), discrete_treatment=True)
dml.fit(Y, T, X=X_het, W=X_boshqa)
cate = dml.effect(X_het)

savol -> estimand (ATE/ATT/CATE) -> DAG -> nazorat to'plami (orqa eshik)
mediator va kollayderni NAZORAT QILMA
kuzatuv: overlap -> balans (SMD) -> bir necha usul (reg, IPW, AIPW) + bootstrap CI
AIPW: bittasi to'g'ri bo'lsa yetadi; o'lchanmagan U -> hammasi yiqiladi
uplift: javob modeli emas; Qini - RCT da
DiD: parallel trendlar -> placebo tekshiruv, do'kon bo'yicha SE

"""DAG lar simulyatsiyada: nimani nazorat qilish kerak va nimani - yo'q."""

import numpy as np
import statsmodels.api as sm


def koef(y, *ustunlar):
    """OLS: birinchi regressor koeffitsienti va SE si."""
    r = sm.OLS(y, sm.add_constant(np.column_stack(ustunlar))).fit()
    return float(r.params[1]), float(r.bse[1])


def sigmoid(z):
    return 1 / (1 + np.exp(-z))


def main() -> None:
    rng = np.random.default_rng(0)
    n = 20_000
    xulosa = []

    print("=== 1. Chalkashtiruvchi: sodiqlik -> kupon, sodiqlik -> xarid ===")
    sod = rng.normal(0, 1, n)
    kupon = (rng.random(n) < sigmoid(1.5 * sod - 0.5)).astype(float)
    xarid = 200 + 60 * sod + 30 * kupon + rng.normal(0, 40, n)
    naiv = xarid[kupon == 1].mean() - xarid[kupon == 0].mean()
    b1, _ = koef(xarid, kupon)
    b2, s2 = koef(xarid, kupon, sod)
    print("  haqiqiy ta'sir: 30.0 ming so'm")
    print(f"  sodiqlik o'rtachasi: kupon {sod[kupon == 1].mean():+.2f}, "
          f"kuponsiz {sod[kupon == 0].mean():+.2f}")
    print(f"  naiv farq               {naiv:6.1f}")
    print(f"  xarid ~ kupon + sodiq   {b2:6.1f} (SE {s2:.2f})")
    xulosa.append(("chalkashtiruvchi", naiv, b2, s2, 30.0))

    print("\n=== 2. Mediator: kupon -> tashriflar -> xarid (kupon tasodifiy) ===")
    kupon = rng.integers(0, 2, n).astype(float)
    tashrif = 2 + 1.5 * kupon + rng.normal(0, 1, n)
    xarid = 100 + 20 * tashrif + 10 * kupon + rng.normal(0, 40, n)
    b1, s1 = koef(xarid, kupon)
    b2, s2 = koef(xarid, kupon, tashrif)
    print("  haqiqiy: umumiy ta'sir 10 + 20 * 1.5 = 40.0, bevosita 10.0")
    print(f"  xarid ~ kupon            {b1:6.1f} (SE {s1:.2f})  <- umumiy")
    print(f"  xarid ~ kupon + tashrif  {b2:6.1f} (SE {s2:.2f})  <- faqat bevosita")
    xulosa.append(("mediator (umumiy)", b2, b1, s1, 40.0))

    print("\n=== 3. Kollayder: kupon -> sharh <- xarid (kupon ta'sir QILMAYDI) ===")
    kupon = rng.integers(0, 2, n).astype(float)
    xarid = 200 + rng.normal(0, 50, n)
    sharh = rng.random(n) < sigmoid(-2 + 1.5 * kupon + 0.04 * (xarid - 200))
    b1, s1 = koef(xarid, kupon)
    b2, s2 = koef(xarid, kupon, sharh.astype(float))
    ichida = (xarid[sharh & (kupon == 1)].mean()
              - xarid[sharh & (kupon == 0)].mean())
    print("  haqiqiy ta'sir: 0.0")
    print(f"  hamma mijozlar: xarid ~ kupon          {b1:6.1f} (SE {s1:.2f})")
    print(f"  xarid ~ kupon + sharh                  {b2:6.1f} (SE {s2:.2f})")
    print(f"  faqat sharh qoldirganlar ({sharh.mean():.1%}): farq {ichida:6.1f}")
    xulosa.append(("kollayder", b2, b1, s1, 0.0))

    print("\n=== 4. Simpson paradoksi: segment -> kupon, segment -> konversiya ===")
    n2 = 40_000
    doimiy = rng.random(n2) < 0.4
    kupon = rng.random(n2) < np.where(doimiy, 0.2, 0.7)
    p = np.where(doimiy, 0.30, 0.05) + 0.03 * kupon
    oldi = rng.random(n2) < p
    print(f"  {'segment':<10} {'kuponsiz':>9} {'kupon':>7} {'farq':>7} "
          f"{'kupon ulushi':>13}")
    farqlar, ulushlar = [], []
    for nom, s in [("yangi", ~doimiy), ("doimiy", doimiy)]:
        k0, k1 = oldi[s & ~kupon].mean(), oldi[s & kupon].mean()
        farqlar.append(k1 - k0)
        ulushlar.append(s.mean())
        print(f"  {nom:<10} {k0:>9.3f} {k1:>7.3f} {k1 - k0:>+7.3f} "
              f"{kupon[s].mean():>13.2f}")
    u0, u1 = oldi[~kupon].mean(), oldi[kupon].mean()
    print(f"  {'UMUMIY':<10} {u0:>9.3f} {u1:>7.3f} {u1 - u0:>+7.3f} "
          f"{kupon.mean():>13.2f}")
    standart = float(np.dot(farqlar, ulushlar))
    print(f"  standartlangan (segment ulushlari bilan): {standart:+.3f} "
          f"(haqiqiy +0.030)")

    print("\n=== 5. Xulosa (natijadan hisoblangan) ===")
    for nom, yomon, yaxshi, se, haq in xulosa:
        print(f"  {nom:<18} noto'g'ri tanlov {yomon:6.1f}, to'g'ri {yaxshi:6.1f}"
              f" (haqiqiy {haq:.1f}); to'g'ri 2*SE ichida: "
              f"{abs(yaxshi - haq) <= 2 * se}")
    print("  ⭐ Nazorat to'plamini DAG hal qiladi, ma'lumot emas")


if __name__ == "__main__":
    main()

=== 1. Chalkashtiruvchi: sodiqlik -> kupon, sodiqlik -> xarid ===
  haqiqiy ta'sir: 30.0 ming so'm
  sodiqlik o'rtachasi: kupon +0.63, kuponsiz -0.43
  naiv farq                 94.4
  xarid ~ kupon + sodiq     31.1 (SE 0.67)

=== 2. Mediator: kupon -> tashriflar -> xarid (kupon tasodifiy) ===
  haqiqiy: umumiy ta'sir 10 + 20 * 1.5 = 40.0, bevosita 10.0
  xarid ~ kupon              39.2 (SE 0.63)  <- umumiy
  xarid ~ kupon + tashrif     9.1 (SE 0.70)  <- faqat bevosita

=== 3. Kollayder: kupon -> sharh <- xarid (kupon ta'sir QILMAYDI) ===
  haqiqiy ta'sir: 0.0
  hamma mijozlar: xarid ~ kupon             0.7 (SE 0.71)
  xarid ~ kupon + sharh                   -11.9 (SE 0.59)
  faqat sharh qoldirganlar (32.6%): farq  -14.4

=== 4. Simpson paradoksi: segment -> kupon, segment -> konversiya ===
  segment     kuponsiz   kupon    farq  kupon ulushi
  yangi          0.052   0.081  +0.029          0.70
  doimiy         0.301   0.335  +0.034          0.20
  UMUMIY         0.211   0.121  -0.090          0.50
  standartlangan (segment ulushlari bilan): +0.031 (haqiqiy +0.030)

=== 5. Xulosa (natijadan hisoblangan) ===
  chalkashtiruvchi   noto'g'ri tanlov   94.4, to'g'ri   31.1 (haqiqiy 30.0); to'g'ri 2*SE ichida: True
  mediator (umumiy)  noto'g'ri tanlov    9.1, to'g'ri   39.2 (haqiqiy 40.0); to'g'ri 2*SE ichida: True
  kollayder          noto'g'ri tanlov  -11.9, to'g'ri    0.7 (haqiqiy 0.0); to'g'ri 2*SE ichida: True
  ⭐ Nazorat to'plamini DAG hal qiladi, ma'lumot emas

"""Kupon ta'siri kuzatuv ma'lumotida: usullarni haqiqiy ATE/ATT bilan solishtirish."""

import numpy as np
from sklearn.linear_model import LinearRegression, LogisticRegression
from sklearn.neighbors import NearestNeighbors

USTUNLAR = ["sodiqlik", "log_oldingi", "faollik", "toshkent"]
MAQSAD = {"naiv farq": "ATE", "regressiya T+X": "ATE", "g-hisoblash": "ATE",
          "moslashtirish": "ATT", "IPW": "ATE", "IPW ATT": "ATT",
          "IPW kesilgan": "ATE", "AIPW": "ATE"}


def yarat(seed, n=4000):
    """Sodiq mijozlar kuponni ko'proq oladi va kuponsiz ham ko'p xarid qiladi."""
    rng = np.random.default_rng(seed)
    sod = rng.normal(0, 1, n)
    oldingi = rng.poisson(np.exp(1.0 + 0.5 * sod))
    faol = rng.normal(0, 1, n)
    toshkent = (rng.random(n) < 0.45).astype(float)
    X = np.column_stack([sod, np.log1p(oldingi), faol, toshkent])
    logit = -0.6 + 1.3 * sod + 0.4 * np.log1p(oldingi) + 0.5 * faol - 0.3 * toshkent
    T = (rng.random(n) < 1 / (1 + np.exp(-logit))).astype(int)
    tau = 40 - 15 * sod + 8 * faol                   # sodiqlarga kam ta'sir
    y0 = 180 + 55 * sod + 20 * np.log1p(oldingi) + 15 * faol + 20 * toshkent
    Y = y0 + tau * T + rng.normal(0, 60, n)          # ming so'm
    return X, T, Y, tau


def propensity(X, T):
    return LogisticRegression(C=1e6, max_iter=1000).fit(X, T).predict_proba(X)[:, 1]


def usullar(X, T, Y):
    e = propensity(X, T)
    b = {"naiv farq": Y[T == 1].mean() - Y[T == 0].mean()}
    b["regressiya T+X"] = LinearRegression().fit(
        np.column_stack([T, X]), Y).coef_[0]
    mu1 = LinearRegression().fit(X[T == 1], Y[T == 1]).predict(X)
    mu0 = LinearRegression().fit(X[T == 0], Y[T == 0]).predict(X)
    b["g-hisoblash"] = np.mean(mu1 - mu0)
    lg = np.log(e / (1 - e))
    nn = NearestNeighbors(n_neighbors=1).fit(lg[T == 0, None])
    d, j = nn.kneighbors(lg[T == 1, None])
    ok = d[:, 0] <= 0.2 * lg.std()
    b["moslashtirish"] = np.mean(Y[T == 1][ok] - Y[T == 0][j[ok, 0]])

    def hajek(e_):
        w1, w0 = T / e_, (1 - T) / (1 - e_)
        return np.sum(w1 * Y) / np.sum(w1) - np.sum(w0 * Y) / np.sum(w0)

    b["IPW"] = hajek(e)
    w0 = (1 - T) * e / (1 - e)
    b["IPW ATT"] = Y[T == 1].mean() - np.sum(w0 * Y) / np.sum(w0)
    b["IPW kesilgan"] = hajek(np.clip(e, 0.05, 0.95))
    psi = mu1 - mu0 + T * (Y - mu1) / e - (1 - T) * (Y - mu0) / (1 - e)
    b["AIPW"] = psi.mean()
    return {k: float(v) for k, v in b.items()}, e, float(ok.mean())


def smd(x, T, w):
    m1 = np.average(x[T == 1], weights=w[T == 1])
    m0 = np.average(x[T == 0], weights=w[T == 0])
    return (m1 - m0) / np.sqrt((x[T == 1].var() + x[T == 0].var()) / 2)


def main() -> None:
    X, T, Y, tau = yarat(0)
    ate, att = tau.mean(), tau[T == 1].mean()
    haq = {"ATE": ate, "ATT": att}

    print("=== 1. Ma'lumot (urug' 0): kim kupon oldi? ===")
    print(f"  n={len(T)}, kupon olganlar ulushi {T.mean():.3f}")
    print(f"  haqiqiy ATE {ate:.2f}, ATT {att:.2f} ming so'm "
          f"(sodiqlarga ta'sir kam -> ATT < ATE)")

    print("\n=== 2. Overlap va balans ===")
    b, e, moslandi = usullar(X, T, Y)
    for g, nom in [(1, "kupon"), (0, "kuponsiz")]:
        q = np.quantile(e[T == g], [0, 0.05, 0.5, 0.95, 1])
        print(f"  {nom:<8} e: min {q[0]:.3f} 5% {q[1]:.3f} med {q[2]:.3f} "
              f"95% {q[3]:.3f} max {q[4]:.3f}")
    print(f"  e < 0.05 yoki e > 0.95: {np.mean((e < 0.05) | (e > 0.95)):.3f} ulush")
    w = T / e + (1 - T) / (1 - e)
    for g, nom in [(1, "kupon"), (0, "kuponsiz")]:
        wg = w[T == g]
        print(f"  {nom:<8} maks og'irlik {wg.max():6.1f}, ESS "
              f"{wg.sum() ** 2 / (wg ** 2).sum():6.0f} / {len(wg)}")
    print(f"  moslashtirish: caliper ichida {moslandi:.3f} kupon oluvchi")
    print(f"  {'belgi':<12} {'SMD xom':>8} {'SMD IPW':>8}")
    bir = np.ones(len(T))
    for j, u in enumerate(USTUNLAR):
        print(f"  {u:<12} {smd(X[:, j], T, bir):>+8.3f} "
              f"{smd(X[:, j], T, w):>+8.3f}")

    print("\n=== 3. Usullar (urug' 0) va bootstrap 95% CI (200 takror) ===")
    rng = np.random.default_rng(123)
    bs = {k: [] for k in b}
    for _ in range(200):
        i = rng.integers(0, len(T), len(T))
        bi, _, _ = usullar(X[i], T[i], Y[i])
        for k in bi:
            bs[k].append(bi[k])
    print(f"  {'usul':<15} {'maqsad':>6} {'baho':>7} {'95% CI':>17} "
          f"{'haqiqiy':>8} {'qamradi':>8}")
    for k, v in b.items():
        lo, hi = np.percentile(bs[k], [2.5, 97.5])
        h = haq[MAQSAD[k]]
        print(f"  {k:<15} {MAQSAD[k]:>6} {v:>7.2f} [{lo:>6.2f}, {hi:>6.2f}] "
              f"{h:>8.2f} {str(lo <= h <= hi):>8}")

    print("\n=== 4. 30 urug': xato = baho - haqiqiy (o'z maqsadiga nisbatan) ===")
    xato = {k: [] for k in MAQSAD}
    for s in range(1, 31):
        X, T, Y, tau = yarat(s)
        bi, _, _ = usullar(X, T, Y)
        h = {"ATE": tau.mean(), "ATT": tau[T == 1].mean()}
        for k in bi:
            xato[k].append(bi[k] - h[MAQSAD[k]])
    print(f"  {'usul':<15} {'o_rt xato':>9} {'SE':>6} {'RMSE':>7} {'siljigan':>9}")
    siljimagan = []
    for k in MAQSAD:
        x = np.array(xato[k])
        se = x.std(ddof=1) / np.sqrt(len(x))
        rmse = float(np.sqrt(np.mean(x ** 2)))
        silj = abs(x.mean()) > 2 * se
        if not silj:
            siljimagan.append((rmse, k))
        print(f"  {k:<15} {x.mean():>+9.2f} {se:>6.2f} {rmse:>7.2f} "
              f"{str(silj):>9}")
    siljimagan.sort()
    print(f"  siljimaganlar ichida eng kichik RMSE: {siljimagan[0][1]} "
          f"({siljimagan[0][0]:.2f})")
    print("  ⭐ Naiv farq - chalkashtirish; tuzatish - faqat o'lchangan X bo'yicha")


if __name__ == "__main__":
    main()

=== 1. Ma'lumot (urug' 0): kim kupon oldi? ===
  n=4000, kupon olganlar ulushi 0.452
  haqiqiy ATE 40.22, ATT 33.17 ming so'm (sodiqlarga ta'sir kam -> ATT < ATE)

=== 2. Overlap va balans ===
  kupon    e: min 0.014 5% 0.187 med 0.647 95% 0.943 max 0.992
  kuponsiz e: min 0.003 5% 0.043 med 0.267 95% 0.759 max 0.970
  e < 0.05 yoki e > 0.95: 0.056 ulush
  kupon    maks og'irlik   69.1, ESS    753 / 1808
  kuponsiz maks og'irlik   32.8, ESS   1251 / 2192
  moslashtirish: caliper ichida 0.992 kupon oluvchi
  belgi         SMD xom  SMD IPW
  sodiqlik       +1.187   -0.024
  log_oldingi    +0.762   -0.009
  faollik        +0.311   +0.031
  toshkent       -0.071   +0.017

=== 3. Usullar (urug' 0) va bootstrap 95% CI (200 takror) ===
  usul            maqsad    baho            95% CI  haqiqiy  qamradi
  naiv farq          ATE  102.44 [ 98.01, 107.43]    40.22    False
  regressiya T+X     ATE   40.01 [ 35.84,  44.25]    40.22     True
  g-hisoblash        ATE   40.89 [ 36.78,  45.32]    40.22     True
  moslashtirish      ATT   32.95 [ 28.23,  42.67]    33.17     True
  IPW                ATE   38.05 [ 29.59,  47.06]    40.22     True
  IPW ATT            ATT   35.01 [ 28.03,  43.73]    33.17     True
  IPW kesilgan       ATE   41.52 [ 35.23,  48.12]    40.22     True
  AIPW               ATE   38.32 [ 32.30,  44.09]    40.22     True

=== 4. 30 urug': xato = baho - haqiqiy (o'z maqsadiga nisbatan) ===
  usul            o_rt xato     SE    RMSE  siljigan
  naiv farq          +62.57   0.39   62.61      True
  regressiya T+X      -0.57   0.37    2.07     False
  g-hisoblash         +0.41   0.38    2.07     False
  moslashtirish       +0.51   0.72    3.91     False
  IPW                 +0.49   0.55    3.00     False
  IPW ATT             +1.11   0.81    4.50     False
  IPW kesilgan        +3.69   0.51    4.59      True
  AIPW                +0.38   0.45    2.44     False
  siljimaganlar ichida eng kichik RMSE: g-hisoblash 2.07-bob
  ⭐ Naiv farq - chalkashtirish; tuzatish - faqat o'lchangan X bo'yicha

"""Ikki karra mustahkamlik, o'lchanmagan chalkashtiruvchi va overlap - 30 urug'da."""

import numpy as np
from sklearn.linear_model import LinearRegression, LogisticRegression

HAQIQIY = 20.0
USULLAR = ["regressiya", "IPW", "IPW kesilgan", "AIPW"]


def yarat(rng, n, kuch=1.0, u_kuch=0.0):
    """x1 ning kvadrati ham tanlashga, ham natijaga ta'sir qiladi; u - yashirin."""
    x1, x2, u = rng.normal(0, 1, (3, n))
    logit = kuch * (-0.8 + 0.8 * x1 + 0.7 * (x1 ** 2 - 1) + 0.5 * x2) + u_kuch * u
    T = (rng.random(n) < 1 / (1 + np.exp(-logit))).astype(int)
    Y = (100 + 20 * x1 + 25 * (x1 ** 2 - 1) + 10 * x2 + 25 * u_kuch * u
         + HAQIQIY * T + rng.normal(0, 30, n))
    return x1, x2, T, Y


def baholar(Xn, Xp, T, Y):
    """Xn - natija modeli belgilari, Xp - propensity modeli belgilari."""
    e = LogisticRegression(C=1e6, max_iter=1000).fit(Xp, T).predict_proba(Xp)[:, 1]
    e = np.clip(e, 1e-6, 1 - 1e-6)          # faqat 0 ga bo'lishdan saqlanish
    mu1 = LinearRegression().fit(Xn[T == 1], Y[T == 1]).predict(Xn)
    mu0 = LinearRegression().fit(Xn[T == 0], Y[T == 0]).predict(Xn)

    def hajek(e_):
        w1, w0 = T / e_, (1 - T) / (1 - e_)
        return np.sum(w1 * Y) / np.sum(w1) - np.sum(w0 * Y) / np.sum(w0)

    psi = mu1 - mu0 + T * (Y - mu1) / e - (1 - T) * (Y - mu0) / (1 - e)
    w = T / e + (1 - T) / (1 - e)
    return ({"regressiya": np.mean(mu1 - mu0), "IPW": hajek(e),
             "IPW kesilgan": hajek(np.clip(e, 0.05, 0.95)), "AIPW": psi.mean()},
            float(w.max()), float(w.sum() ** 2 / (w ** 2).sum() / len(w)))


def tajriba(kuch, u_kuch, natija_togri, ps_togri, urug=30, n=3000):
    xatolar = {k: [] for k in USULLAR}
    maks_w, ess = [], []
    for s in range(urug):
        rng = np.random.default_rng(s)
        x1, x2, T, Y = yarat(rng, n, kuch, u_kuch)
        togri = np.column_stack([x1, x1 ** 2, x2])
        notogri = np.column_stack([x1, x2])
        b, mw, es = baholar(togri if natija_togri else notogri,
                            togri if ps_togri else notogri, T, Y)
        for k in USULLAR:
            xatolar[k].append(b[k] - HAQIQIY)
        maks_w.append(mw)
        ess.append(es)
    return ({k: np.array(v) for k, v in xatolar.items()},
            float(np.median(maks_w)), float(np.mean(ess)))


def katak(x):
    se = x.std(ddof=1) / np.sqrt(len(x))
    belgi = "*" if abs(x.mean()) > 2 * se else " "
    return f"{x.mean():+6.1f}{belgi}({x.std(ddof=1):4.1f})"


def main() -> None:
    print("=== 1. Ikki karra mustahkamlik: 30 urug', haqiqiy ATE = 20 ===")
    print("  katak: o'rtacha xato, * - sezilarli siljish (> 2 SE), (std)")
    ssenariylar = [
        ("ikkala model to'g'ri", 1.0, 0.0, True, True),
        ("natija modeli xato", 1.0, 0.0, False, True),
        ("propensity xato", 1.0, 0.0, True, False),
        ("ikkalasi xato", 1.0, 0.0, False, False),
    ]
    print(f"  {'ssenariy':<22}" + "".join(f"{k:>15}" for k in USULLAR))
    for nom, *arg in ssenariylar:
        x, _, _ = tajriba(*arg)
        print(f"  {nom:<22}" + "".join(f"{katak(x[k]):>15}" for k in USULLAR))

    print("\n=== 2. Kuchsiz overlap: tanlash 3 barobar kuchli (modellar to'g'ri) ===")
    for nom, kuch in [("oddiy tanlash", 1.0), ("kuchli tanlash", 3.0)]:
        x, mw, ess = tajriba(kuch, 0.0, True, True)
        print(f"  {nom:<16} maks og'irlik (mediana) {mw:8.1f}, ESS ulushi "
              f"{ess:.3f}")
        print("    " + "  ".join(f"{k}: {katak(x[k])}" for k in USULLAR[1:]))

    print("\n=== 3. O'lchanmagan chalkashtiruvchi u (modellar x bo'yicha to'g'ri) ===")
    print(f"  {'u kuchi':>8}" + "".join(f"{k:>15}" for k in USULLAR))
    for u_kuch in [0.0, 0.25, 0.5, 1.0]:
        x, _, _ = tajriba(1.0, u_kuch, True, True)
        print(f"  {u_kuch:>8}" + "".join(f"{katak(x[k]):>15}" for k in USULLAR))
    x, _, _ = tajriba(1.0, 1.0, True, True)
    hammasi = all(abs(x[k].mean()) > 2 * x[k].std(ddof=1) / np.sqrt(30)
                  for k in USULLAR)
    print(f"  u kuchi 1.0 da hamma usul sezilarli siljigan: {hammasi}")
    print("  ⭐ AIPW: bittasi to'g'ri bo'lsa yetadi; yashirin u ga qarshi - ojiz")


if __name__ == "__main__":
    main()

=== 1. Ikki karra mustahkamlik: 30 urug', haqiqiy ATE = 20 ===
  katak: o'rtacha xato, * - sezilarli siljish (> 2 SE), (std)
  ssenariy                   regressiya            IPW   IPW kesilgan           AIPW
  ikkala model to'g'ri      +0.2 ( 1.7)    +1.2 ( 4.2)    +3.1*( 2.1)    +0.2 ( 2.0)
  natija modeli xato       +27.4*( 2.1)    +1.2 ( 4.2)    +3.1*( 2.1)    +1.2 ( 4.2)
  propensity xato           +0.2 ( 1.7)   +34.4*( 3.9)   +32.1*( 2.4)    +0.3 ( 1.8)
  ikkalasi xato            +27.4*( 2.1)   +34.4*( 3.9)   +32.1*( 2.4)   +47.8*( 6.4)

=== 2. Kuchsiz overlap: tanlash 3 barobar kuchli (modellar to'g'ri) ===
  oddiy tanlash    maks og'irlik (mediana)     30.0, ESS ulushi 0.533
    IPW:   +1.2 ( 4.2)  IPW kesilgan:   +3.1*( 2.1)  AIPW:   +0.2 ( 2.0)
  kuchli tanlash   maks og'irlik (mediana)    385.9, ESS ulushi 0.047
    IPW:   +7.2*11.7-bob  IPW kesilgan:  +29.5*( 3.1)  AIPW:   +2.2 10.4-bob

=== 3. O'lchanmagan chalkashtiruvchi u (modellar x bo'yicha to'g'ri) ===
   u kuchi     regressiya            IPW   IPW kesilgan           AIPW
       0.0    +0.2 ( 1.7)    +1.2 ( 4.2)    +3.1*( 2.1)    +0.2 ( 2.0)
      0.25    +1.6*( 1.8)    +2.8*( 3.8)    +4.5*( 2.0)    +1.6*( 2.0)
       0.5    +5.9*( 1.8)    +7.0*( 3.8)    +8.7*( 2.1)    +5.9*( 2.0)
       1.0   +20.8*( 1.8)   +22.5*( 2.4)   +23.2*( 1.8)   +21.0*( 1.8)
  u kuchi 1.0 da hamma usul sezilarli siljigan: True
  ⭐ AIPW: bittasi to'g'ri bo'lsa yetadi; yashirin u ga qarshi - ojiz

"""Kimga kupon berish (uplift, Qini) va aksiya ta'siri farqlar farqi bilan."""

import numpy as np
from sklearn.ensemble import HistGradientBoostingClassifier


def sigmoid(z):
    return 1 / (1 + np.exp(-z))


def uplift_malumot(rng, n):
    """RCT: kupon 50% tasodifiy. Ta'sir narxga sezgir va yangi mijozlarda katta."""
    sod = rng.normal(0, 1, n)
    sezgir = rng.normal(0, 1, n)
    faol = rng.normal(0, 1, n)
    yangi = (rng.random(n) < 0.3).astype(float)
    X = np.column_stack([sod, sezgir, faol, yangi])
    z0 = -1.2 + 1.0 * sod + 0.5 * faol - 0.3 * yangi
    delta = -0.1 + 0.8 * sezgir + 0.7 * yangi - 0.5 * sod
    T = rng.integers(0, 2, n)
    p0, p1 = sigmoid(z0), sigmoid(z0 + delta)
    Y = (rng.random(n) < np.where(T == 1, p1, p0)).astype(int)
    return X, T, Y, p1 - p0


def model():
    return HistGradientBoostingClassifier(max_iter=150, learning_rate=0.05,
                                          max_leaf_nodes=15, min_samples_leaf=50,
                                          random_state=0)


def qini(uplift, T, Y):
    """Qini egri chizig'i: top-k ga kupon berilsa qo'shimcha xaridlar."""
    t = np.argsort(-uplift, kind="stable")
    T, Y = T[t], Y[t]
    n1, n0 = np.cumsum(T), np.cumsum(1 - T)
    q = np.cumsum(Y * T) - np.cumsum(Y * (1 - T)) * n1 / np.maximum(n0, 1)
    k = np.arange(1, len(T) + 1)
    return float(np.mean(q - q[-1] * k / len(T))), q


def did(y, muolaja, oldin, keyin):
    """Do'kon darajasida DiD va SE (do'kon bo'yicha klaster)."""
    farq = y[:, keyin].mean(1) - y[:, oldin].mean(1)
    a, b = farq[muolaja], farq[~muolaja]
    return (a.mean() - b.mean(),
            np.sqrt(a.var(ddof=1) / len(a) + b.var(ddof=1) / len(b)))


def did_malumot(rng, parallel):
    hafta = np.arange(1, 53)
    muolaja = np.arange(24) < 12                     # 12 Samarqand do'koni
    daraja = np.where(muolaja, 80.0, 120.0) + rng.normal(0, 8, 24)
    y = (daraja[:, None] + 0.3 * hafta + 8 * np.sin(2 * np.pi * hafta / 52)
         + 12.0 * (muolaja[:, None] & (hafta >= 27))
         + rng.normal(0, 4, (24, 52)))
    if not parallel:
        y += 0.35 * hafta * muolaja[:, None]        # Samarqand tezroq o'sadi
    return y, muolaja, hafta


def main() -> None:
    rng = np.random.default_rng(0)
    Xtr, Ttr, Ytr, _ = uplift_malumot(rng, 20_000)
    Xte, Tte, Yte, tau = uplift_malumot(rng, 20_000)

    print("=== 1. Uplift: RCT ma'lumoti (o'quv 20 000, test 20 000) ===")
    ate = Yte[Tte == 1].mean() - Yte[Tte == 0].mean()
    print(f"  o'rtacha uplift: haqiqiy {tau.mean():+.4f}, test RCT bahosi "
          f"{ate:+.4f}")
    print(f"  ishontiriladigan (tau > 0.05) {np.mean(tau > 0.05):.3f}, "
          f"'bezovta qilma' (tau < -0.02) {np.mean(tau < -0.02):.3f}")
    javob = model().fit(Xtr, Ytr).predict_proba(Xte)[:, 1]
    f1 = model().fit(Xtr[Ttr == 1], Ytr[Ttr == 1]).predict_proba(Xte)[:, 1]
    f0 = model().fit(Xtr[Ttr == 0], Ytr[Ttr == 0]).predict_proba(Xte)[:, 1]
    s = model().fit(np.column_stack([Xtr, Ttr]), Ytr)
    s_up = (s.predict_proba(np.column_stack([Xte, np.ones(len(Xte))]))[:, 1]
            - s.predict_proba(np.column_stack([Xte, np.zeros(len(Xte))]))[:, 1])
    bashorat = {"tasodifiy": rng.random(len(Tte)), "javob modeli": javob,
                "S-learner": s_up, "T-learner": f1 - f0, "oracle (haqiqiy)": tau}

    print("\n=== 2. Qini: test to'plamida ===")
    k30 = int(0.3 * len(Tte))
    print(f"  {'model':<17} {'Qini maydoni':>12} {'top-30% qo_sh.':>15} "
          f"{'corr(tau)':>10}")
    for nom, u in bashorat.items():
        maydon, q = qini(u, Tte, Yte)
        print(f"  {nom:<17} {maydon:>12.1f} {q[k30 - 1]:>15.1f} "
              f"{np.corrcoef(u, tau)[0, 1]:>10.3f}")
    print(f"  hammaga kupon ({len(Tte):,} ta): qo'shimcha xaridlar "
          f"{qini(tau, Tte, Yte)[1][-1]:.1f}")

    print("\n=== 3. Juftlashgan bootstrap (200): Qini maydoni farqi ===")
    b_rng = np.random.default_rng(1)
    juftlar = [("T-learner", "S-learner"), ("T-learner", "javob modeli")]
    farqlar = {j: [] for j in juftlar}
    for _ in range(200):
        i = b_rng.integers(0, len(Tte), len(Tte))
        m = {nom: qini(bashorat[nom][i], Tte[i], Yte[i])[0]
             for nom in ["T-learner", "S-learner", "javob modeli"]}
        for a, b in juftlar:
            farqlar[(a, b)].append(m[a] - m[b])
    for (a, b), v in farqlar.items():
        v = np.array(v)
        print(f"  {a} - {b}: {v.mean():+.1f} (SE {v.std(ddof=1):.1f}), "
              f"sezilarli: {abs(v.mean()) > 2 * v.std(ddof=1)}")
    ts = np.array(farqlar[("T-learner", "S-learner")])
    if ts.mean() > 2 * ts.std(ddof=1):
        print("  tanlov: T-learner (S-learner sezilarli yomon)")
    elif ts.mean() < -2 * ts.std(ddof=1):
        print("  tanlov: S-learner (sezilarli yaxshi va soddaroq - bitta model)")
    else:
        print("  tanlov: S-learner (sodda, T dan sezilarli yomon emas)")

    print("\n=== 4. Farqlar farqi: aksiya Samarqandda 27-haftadan, haqiqiy +12 ===")
    for nom, parallel in [("parallel trendlar", True), ("trendlar farqli", False)]:
        y, muolaja, hafta = did_malumot(np.random.default_rng(5), parallel)
        oldin, keyin = hafta < 27, hafta >= 27
        oldin_keyin = (y[muolaja][:, keyin].mean()
                       - y[muolaja][:, oldin].mean())
        keyin_farq = y[muolaja][:, keyin].mean() - y[~muolaja][:, keyin].mean()
        d, se = did(y, muolaja, oldin, keyin)
        p, pse = did(y[:, :26], muolaja, hafta[:26] < 14, hafta[:26] >= 14)
        print(f"  [{nom}]")
        print(f"    naiv oldin-keyin {oldin_keyin:+6.1f}, naiv shaharlar farqi "
              f"{keyin_farq:+6.1f}")
        print(f"    DiD {d:+6.2f} (SE {se:.2f}); placebo (1-26 hafta, soxta "
              f"14-hafta) {p:+5.2f} (SE {pse:.2f})")
        print(f"    parallel trend tekshiruvi o'tdi: {abs(p) <= 2 * pse}; "
              f"DiD 2*SE ichida +12 ga: {abs(d - 12) <= 2 * se}")
    print("  ⭐ Uplift - RCT da baholanadi; DiD - placebo bilan tekshiriladi")


if __name__ == "__main__":
    main()

=== 1. Uplift: RCT ma'lumoti (o'quv 20 000, test 20 000) ===
  o'rtacha uplift: haqiqiy +0.0150, test RCT bahosi +0.0134
  ishontiriladigan (tau > 0.05) 0.391, 'bezovta qilma' (tau < -0.02) 0.395

=== 2. Qini: test to'plamida ===
  model             Qini maydoni  top-30% qo_sh.  corr(tau)
  tasodifiy                -20.2             8.3     -0.016
  javob modeli             -19.6            15.4     -0.023
  S-learner                464.0           593.7      0.952
  T-learner                447.0           562.6      0.917
  oracle (haqiqiy)         476.6           647.9      1.000
  hammaga kupon (20,000 ta): qo'shimcha xaridlar 135.1

=== 3. Juftlashgan bootstrap (200): Qini maydoni farqi ===
  T-learner - S-learner: -16.8 (SE 8.0), sezilarli: True
  T-learner - javob modeli: +466.2 (SE 27.4), sezilarli: True
  tanlov: S-learner (sezilarli yaxshi va soddaroq - bitta model)

=== 4. Farqlar farqi: aksiya Samarqandda 27-haftadan, haqiqiy +12 ===
  [parallel trendlar]
    naiv oldin-keyin  +10.0, naiv shaharlar farqi  -25.5
    DiD +12.77 (SE 0.52); placebo (1-26 hafta, soxta 14-hafta) -0.65 (SE 0.61)
    parallel trend tekshiruvi o'tdi: True; DiD 2*SE ichida +12 ga: True
  [trendlar farqli]
    naiv oldin-keyin  +19.1, naiv shaharlar farqi  -11.7
    DiD +21.87 (SE 0.52); placebo (1-26 hafta, soxta 14-hafta) +3.90 (SE 0.61)
    parallel trend tekshiruvi o'tdi: False; DiD 2*SE ichida +12 ga: False
  ⭐ Uplift - RCT da baholanadi; DiD - placebo bilan tekshiriladi

tasir = df[df.kupon == 1].xarid.mean() - df[df.kupon == 0].xarid.mean()   # ⚠️
# DAG -> nazorat to'plami -> g-hisoblash / IPW / AIPW + bootstrap CI       # ✅

LinearRegression().fit(df[["kupon", "tashriflar", "sodiqlik"]], y)        # ⚠️ mediator
LinearRegression().fit(df[["kupon", "sodiqlik"]], y)                      # ✅ faqat oldingi belgilar

w = T / e + (1 - T) / (1 - e); ate = ...                                  # ⚠️
# avval: e taqsimoti guruhlar bo'yicha, maks og'irlik, ESS, SMD            # ✅

e = propensity(X, T)
for _ in range(200): i = ...; ate_i = ipw(T[i], Y[i], e[i])               # ⚠️
for _ in range(200): i = ...; ate_i = ipw(T[i], Y[i], propensity(X[i], T[i]))  # ✅

e = np.clip(e, 0.05, 0.95)   # va hisobotda "ATE"                         # ⚠️
# "overlap hududi (0.05 < e < 0.95) uchun ta'sir" deb yozing              # ✅

maqsad = model.fit(X, y).predict_proba(X)[:, 1].argsort()[::-1][:k]       # ⚠️
uplift = f1.predict_proba(X)[:, 1] - f0.predict_proba(X)[:, 1]            # ✅ + Qini (RCT da)

did = (a_keyin - a_oldin) - (b_keyin - b_oldin)                           # ⚠️
# + placebo (oldingi davrda soxta sana), SE do'kon bo'yicha klaster        # ✅

1.  # sodiqlar kupon ko'proq olsa, naiv farq ta'sirni oshiradimi yoki kamaytiradimi?
2.  # mediatorni nazorat qilsak nima baholanadi?
3.  # faqat sharh qoldirganlarni tahlil qilish - qanday xato?
4.  # Simpson: har segmentda +3 punkt, umumiy -9 - qaysi javob to'g'ri?
5.  # kupon sodiqlarga ko'proq berilsa va ularga kam ta'sir qilsa: ATT va ATE?
6.  # IPW dan keyin |SMD| < 0.1 bo'lsa, chalkashtirish yo'qmi?
7.  # e ni [0.05, 0.95] ga kesish nimani o'zgartiradi?
8.  # AIPW: natija modeli xato, propensity to'g'ri - siljish?
9.  # AIPW: ikkalasi xato - siljish?
10. # yashirin u kuchaysa AIPW nima qiladi?
11. # javob modeli bilan tanlangan top-30% - uplift?
12. # placebo DiD 0 dan sezilarli farq qilsa?

1.  tasir = y[kupon == 1].mean() - y[kupon == 0].mean()   # kuzatuv ma'lumoti

2.  X = df[["kupon", "tashriflar", "sharh_qoldirdi", "sodiqlik"]]

3.  e = propensity(X, T)
    ates = [ipw(T[i], Y[i], e[i]) for i in bootstrap_indekslar]

4.  maqsad = df.sort_values("xarid_ehtimoli", ascending=False).head(k)

5.  did = (sam_keyin - sam_oldin) - (tosh_keyin - tosh_oldin)   # tekshiruvsiz