16.6-dars: Gauss aralashmasi (GMM)

Ma'lumot K ta Gauss taqsimotidan hosil bo'lgan deb faraz qilinadi:

  p(x) = sum_k pi_k * N(x | mu_k, Sigma_k)

  pi_k    - aralashma og'irligi (sum pi_k = 1)
  mu_k    - k-komponentning markazi
  Sigma_k - k-komponentning kovariatsiya matritsasi

Har nuqta uchun MAS'ULIYAT (responsibility):

  gamma_ik = pi_k * N(x_i | mu_k, Sigma_k) / p(x_i)

  -> har komponentga tegishlilik EHTIMOLLIGI (yig'indisi 1)

Expectation-Maximization - ikki qadam navbatma-navbat:

E-qadam (Expectation):
  joriy parametrlar bilan har nuqta uchun gamma_ik hisoblanadi

M-qadam (Maximization):
  gamma ga qarab parametrlar yangilanadi:
    pi_k    = o'rtacha gamma_ik
    mu_k    = gamma bilan og'irlangan o'rtacha
    Sigma_k = gamma bilan og'irlangan kovariatsiya

Har qadamda LOG-EHTIMOLLIK oshadi (yoki o'zgarmaydi)
  -> yaqinlashish kafolatlangan, lekin LOKAL maksimumga

K-means = GMM ning chegaraviy holati (qattiq biriktirish + sferik teng kovariatsiya)

covariance_type:

full      - har komponent o'z to'liq kovariatsiya matritsasiga ega
            eng moslashuvchan; k * p(p+1)/2 parametr
tied      - barcha komponentlar BIR XIL kovariatsiyaga ega
            p(p+1)/2 parametr; kam ma'lumotda barqaror
diag      - diagonal (belgilar o'zaro bog'liq emas)
            k * p parametr; tez, yuqori o'lchamda amaliy
spherical - har komponent sferik (bitta dispersiya)
            k parametr; K-means ga eng yaqin

Kam ma'lumot yoki ko'p belgi -> soddaroq turni tanlang
reg_covar=1e-6 - singulyar matritsadan himoya

GMM - ehtimollikka asoslangan model, shuning uchun
klassik model tanlash mezonlarini ishlatish mumkin:

  AIC = -2*logL + 2*m
  BIC = -2*logL + m*log(n)          m = parametrlar soni

  ikkalasi ham MINIMALLASHTIRILADI

BIC ko'proq jarima soladi -> soddaroq model tanlaydi
AIC bashorat sifatiga, BIC "haqiqiy" modelni topishga moyil

sklearn: model.aic(X), model.bic(X)

Bu - klasterlar sonini tanlashning YAGONA statistik usuli
  (silhouette va elbow evristik)

K-means - GMM ning maxsus holati:
  covariance_type = "spherical"
  barcha komponentlar teng dispersiyaga ega
  dispersiya -> 0 (qattiq biriktirish)
  aralashma og'irliklari teng

Shuning uchun:
  GMM K-means dan UMUMIYROQ
  GMM cho'zinchoq va burchakli klasterlarni topa oladi
  GMM sekinroq va ko'proq parametrga ega
  GMM ehtimollik beradi

sklearn: GaussianMixture(init_params="k-means++") - standart

g = GaussianMixture(n_components=4, random_state=0).fit(Xs)

# zichlik (log-ehtimollik) -> anomaliya
ball = g.score_samples(Xs)
anomaliya = ball < np.quantile(ball, 0.02)

# yangi namuna generatsiya qilish
Xyangi, yorliq = g.sample(500)

# yumshoq biriktirish
ehtimollik = g.predict_proba(Xs)
ishonch = ehtimollik.max(axis=1)      # past -> chegaradagi nuqta

import numpy as np
from sklearn.mixture import BayesianGaussianMixture, GaussianMixture

g = GaussianMixture(n_components=4, covariance_type="full", n_init=10,
                    reg_covar=1e-6, random_state=0).fit(Xs)

g.predict(Xs), g.predict_proba(Xs)         # qattiq va yumshoq
g.weights_, g.means_, g.covariances_
g.bic(Xs), g.aic(Xs), g.score(Xs)          # model tanlash
g.score_samples(Xs)                        # log-zichlik -> anomaliya
Xyangi, yorliq = g.sample(500)             # generatsiya

# komponentlar sonini avtomatik kamaytirish
BayesianGaussianMixture(n_components=10, weight_concentration_prior=0.01)
QOIDA: masshtabla · n_init oshir · covariance_type ni o'lchamga mosla ·
       BIC bilan k tanla

p(x) = sum pi_k N(x | mu_k, Sigma_k); EM bilan o'qitiladi
Mas'uliyat = tegishlilik ehtimolligi (yumshoq biriktirish)
covariance_type: full / tied / diag / spherical
BIC - k ni tanlashning yagona statistik usuli

"""Cho'zinchoq klasterlarda farq (real numpy/sklearn)."""

import numpy as np
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
from sklearn.metrics import adjusted_rand_score
from sklearn.mixture import GaussianMixture
from sklearn.preprocessing import StandardScaler


def cho_zinchoq(seed: int = 0, n: int = 900):
    X, y = make_blobs(n_samples=n, centers=3, cluster_std=0.7,
                      random_state=seed)
    return X @ np.array([[2.6, -0.9], [-0.4, 0.5]]), y


def turli_olcham(seed: int = 0):
    rng = np.random.default_rng(seed)
    a = rng.normal([0, 0], 0.6, (600, 2))
    b = rng.normal([5, 0], 0.6, (100, 2))
    c = rng.normal([2.5, 4.5], 0.6, (100, 2))
    return np.vstack([a, b, c]), np.array([0] * 600 + [1] * 100 + [2] * 100)


def main() -> None:
    vazifalar = {
        "sferik": make_blobs(n_samples=900, centers=3, cluster_std=1.0,
                             random_state=0),
        "cho'zinchoq": cho_zinchoq(),
        "turli o'lcham": turli_olcham(),
    }

    print("=== 1. K-means va GMM ===")
    print(f"  {'vazifa':<16} {'KMeans ARI':>12} {'GMM ARI':>10} "
          f"{'farq':>9}")
    for nom, (X, y) in vazifalar.items():
        Xs = StandardScaler().fit_transform(X)
        k = len(np.unique(y))
        km = KMeans(k, n_init=10, random_state=0).fit_predict(Xs)
        gm = GaussianMixture(k, covariance_type="full", n_init=10,
                             random_state=0).fit_predict(Xs)
        a1 = adjusted_rand_score(y, km)
        a2 = adjusted_rand_score(y, gm)
        print(f"  {nom:<16} {a1:>12.4f} {a2:>10.4f} {a2 - a1:>+9.4f}")

    print("\n=== 2. Yumshoq biriktirish ===")
    X, y = cho_zinchoq()
    Xs = StandardScaler().fit_transform(X)
    g = GaussianMixture(3, covariance_type="full", n_init=10,
                        random_state=0).fit(Xs)
    p = g.predict_proba(Xs)
    ishonch = p.max(axis=1)
    print(f"  {'ishonch':>14} {'nuqtalar':>10} {'ulush':>8} {'xato %':>9}")
    yorliq = g.predict(Xs)
    from scipy.stats import mode
    # har komponentni eng ko'p mos keladigan haqiqiy sinfga bog'lash
    moslik = {}
    for k in range(3):
        moslik[k] = int(mode(y[yorliq == k], keepdims=False).mode)
    togri = np.array([moslik[c] for c in yorliq]) == y
    for past, yuqori in [(0.33, 0.5), (0.5, 0.7), (0.7, 0.9), (0.9, 1.01)]:
        m = (ishonch >= past) & (ishonch < yuqori)
        if not m.any():
            continue
        print(f"  {past:.2f} - {yuqori:.2f} {int(m.sum()):>10} "
              f"{m.mean():>7.1%} {(~togri[m]).mean():>8.1%}")
    print("  (past ishonchli nuqtalarda xato ko'proq)")

    print("\n=== 3. Komponent parametrlari ===")
    print(f"  {'komponent':>10} {'og_irlik':>10} {'markaz':>22} "
          f"{'kovariatsiya izi':>18}")
    for k in range(3):
        markaz = np.round(g.means_[k], 3)
        iz = float(np.trace(g.covariances_[k]))
        print(f"  {k:>10} {g.weights_[k]:>10.4f} {str(markaz):>22} "
              f"{iz:>18.4f}")

    print("\n=== 4. Turli o'lchamdagi klasterlarda ===")
    X, y = turli_olcham()
    Xs = StandardScaler().fit_transform(X)
    print(f"  haqiqiy o'lchamlar: {np.bincount(y).tolist()}")
    km = KMeans(3, n_init=10, random_state=0).fit_predict(Xs)
    g2 = GaussianMixture(3, covariance_type="full", n_init=10,
                         random_state=0).fit(Xs)
    print(f"  KMeans topdi:  {np.bincount(km).tolist()}")
    print(f"  GMM topdi:     {np.bincount(g2.predict(Xs)).tolist()}")
    print(f"  GMM og'irliklari: {np.round(g2.weights_, 4).tolist()}")
    print("  ⭐ GMM komponent og'irligini ham o'rganadi")


if __name__ == "__main__":
    main()

=== 1. K-means va GMM ===
  vazifa             KMeans ARI    GMM ARI      farq
  sferik                 0.8046     0.8016   -0.0030
  cho'zinchoq            0.7823     0.9509   +0.1685
  turli o'lcham          1.0000     1.0000   +0.0000

=== 2. Yumshoq biriktirish ===
         ishonch   nuqtalar    ulush    xato %
  0.50 - 0.70         16    1.8%    37.5%
  0.70 - 0.90         28    3.1%    25.0%
  0.90 - 1.01        856   95.1%     0.2%
  (past ishonchli nuqtalarda xato ko'proq)

=== 3. Komponent parametrlari ===
   komponent   og_irlik                 markaz   kovariatsiya izi
           0     0.3330          [0.086 0.223]             0.2921
           1     0.3379        [ 1.068 -1.221]             0.2662
           2     0.3291        [-1.184  1.029]             0.2876

=== 4. Turli o'lchamdagi klasterlarda ===
  haqiqiy o'lchamlar: [600, 100, 100]
  KMeans topdi:  [600, 100, 100]
  GMM topdi:     [600, 100, 100]
  GMM og'irliklari: [0.75, 0.125, 0.125]
  ⭐ GMM komponent og'irligini ham o'rganadi

"""Model murakkabligini tanlash (real numpy/sklearn)."""

import numpy as np
from sklearn.datasets import make_blobs
from sklearn.metrics import adjusted_rand_score
from sklearn.mixture import GaussianMixture
from sklearn.preprocessing import StandardScaler


def yarat(seed: int = 4, n: int = 1500, p: int = 2):
    """Uch komponent, turli shakl va yo'nalish."""
    rng = np.random.default_rng(seed)
    komponent = rng.choice(3, n, p=[0.45, 0.35, 0.2])
    markazlar = np.zeros((3, p))
    markazlar[:, 0] = [0.0, 5.0, 2.5]
    markazlar[:, 1] = [0.0, 0.5, 5.0]
    X = np.zeros((n, p))
    burchaklar = [0.0, np.pi / 4, -np.pi / 6]
    kengliklar = [(2.0, 0.4), (1.5, 0.5), (0.9, 0.9)]
    for k in range(3):
        m = komponent == k
        nuqta = rng.normal(0, 1, (int(m.sum()), p))
        nuqta[:, 0] *= kengliklar[k][0]
        nuqta[:, 1] *= kengliklar[k][1]
        c, s = np.cos(burchaklar[k]), np.sin(burchaklar[k])
        R = np.eye(p)
        R[:2, :2] = [[c, -s], [s, c]]
        X[m] = nuqta @ R.T + markazlar[k]
    return X, komponent


def main() -> None:
    X, y = yarat()
    Xs = StandardScaler().fit_transform(X)

    print("=== 1. Kovariatsiya turlari (k = 3) ===")
    print(f"  {'tur':<12} {'parametrlar':>12} {'logL':>10} {'BIC':>11} "
          f"{'ARI':>9}")
    for tur in ["spherical", "diag", "tied", "full"]:
        g = GaussianMixture(3, covariance_type=tur, n_init=10,
                            random_state=0).fit(Xs)
        m = int(g._n_parameters())
        print(f"  {tur:<12} {m:>12} {g.score(Xs) * len(Xs):>10.1f} "
              f"{g.bic(Xs):>11.1f} "
              f"{adjusted_rand_score(y, g.predict(Xs)):>9.4f}")

    print("\n=== 2. BIC bo'yicha komponentlar sonini tanlash ===")
    print(f"  {'k':>3} {'spherical':>11} {'diag':>11} {'tied':>11} "
          f"{'full':>11}")
    natijalar = {}
    for k in range(1, 9):
        qator = []
        for tur in ["spherical", "diag", "tied", "full"]:
            g = GaussianMixture(k, covariance_type=tur, n_init=5,
                                random_state=0).fit(Xs)
            natijalar[(k, tur)] = g.bic(Xs)
            qator.append(g.bic(Xs))
        print(f"  {k:>3} " + "".join(f"{v:>11.1f}" for v in qator))
    eng = min(natijalar, key=natijalar.get)
    print(f"  eng past BIC: k = {eng[0]}, tur = {eng[1]} "
          f"({natijalar[eng]:.1f})")

    print("\n=== 3. AIC va BIC farqi ===")
    print(f"  {'k':>3} {'AIC':>11} {'BIC':>11} {'AIC tanlovi':>13} "
          f"{'BIC tanlovi':>13}")
    aiclar, biclar = [], []
    for k in range(1, 9):
        g = GaussianMixture(k, covariance_type="full", n_init=5,
                            random_state=0).fit(Xs)
        aiclar.append(g.aic(Xs))
        biclar.append(g.bic(Xs))
    for i, k in enumerate(range(1, 9)):
        a = "<-" if i == int(np.argmin(aiclar)) else ""
        b = "<-" if i == int(np.argmin(biclar)) else ""
        print(f"  {k:>3} {aiclar[i]:>11.1f} {biclar[i]:>11.1f} "
              f"{a:>13} {b:>13}")

    print("\n=== 4. Belgilar soni ortganda ===")
    print(f"  {'p':>3} {'spherical':>11} {'diag':>11} {'full':>11} "
          f"{'eng yaxshi':>12}")
    for p in [2, 5, 15, 40]:
        Xp, yp = yarat(n=1500, p=max(p, 2))
        if p > 2:
            rng = np.random.default_rng(1)
            Xp = np.column_stack([Xp[:, :2],
                                  rng.normal(0, 1, (len(Xp), p - 2))])
        Xps = StandardScaler().fit_transform(Xp)
        ballar = {}
        for tur in ["spherical", "diag", "full"]:
            g = GaussianMixture(3, covariance_type=tur, n_init=3,
                                reg_covar=1e-4, random_state=0).fit(Xps)
            ballar[tur] = g.bic(Xps)
        eng_tur = min(ballar, key=ballar.get)
        print(f"  {p:>3} " + "".join(f"{ballar[t]:>11.1f}"
                                     for t in ["spherical", "diag", "full"])
              + f"{eng_tur:>12}")
    print("  ⭐ Yuqori o'lchamda soddaroq kovariatsiya afzal")


if __name__ == "__main__":
    main()

=== 1. Kovariatsiya turlari (k = 3) ===
  tur           parametrlar       logL         BIC       ARI
  spherical              11    -3345.1      6770.6    0.7706
  diag                   14    -3003.9      6110.2    0.8959
  tied                   11    -3235.0      6550.5    0.7648
  full                   17    -2756.6      5637.5    0.9201

=== 2. BIC bo'yicha komponentlar sonini tanlash ===
    k   spherical        diag        tied        full
    1      8535.6     8542.9     8486.1     8486.1
    2      7924.5     6950.0     8188.7     6735.4
    3      6770.6     6110.2     6550.5     5637.5
    4      6489.3     5866.5     6344.7     5679.9
    5      6178.1     5903.1     6169.9     5726.1
    6      6051.1     5812.0     6060.3     5764.4
    7      5940.9     5835.7     6003.5     5808.1
    8      5920.5     5811.0     5930.3     5848.6
  eng past BIC: k = 3, tur = full 5637.5-bob

=== 3. AIC va BIC farqi ===
    k         AIC         BIC   AIC tanlovi   BIC tanlovi
    1      8459.6      8486.1
    2      6677.0      6735.4
    3      5547.2      5637.5            <-            <-
    4      5557.7      5679.9
    5      5572.0      5726.1
    6      5578.4      5764.4
    7      5590.3      5808.1
    8      5598.9      5848.6

=== 4. Belgilar soni ortganda ===
    p   spherical        diag        full   eng yaxshi
    2      6770.6     6110.1     5637.5        full
    5     21091.4    18910.1    18570.3        full
   15     64070.5    61984.0    63505.6        diag
   40    171000.0   169379.6   183658.0        diag
  ⭐ Yuqori o'lchamda soddaroq kovariatsiya afzal

"""E va M qadamlarini noldan qurish (real numpy/sklearn)."""

import numpy as np
from sklearn.mixture import GaussianMixture
from sklearn.preprocessing import StandardScaler


def yarat(seed: int = 6, n: int = 1200):
    rng = np.random.default_rng(seed)
    k = rng.choice(2, n, p=[0.65, 0.35])
    X = np.where(k[:, None] == 0,
                 rng.normal([0.0, 0.0], [1.0, 0.5], (n, 2)),
                 rng.normal([3.5, 2.0], [0.6, 1.2], (n, 2)))
    return X, k


def gauss_zichlik(X, mu, Sigma):
    """Ko'p o'lchamli Gauss zichligi."""
    p = X.shape[1]
    ayirma = X - mu
    teskari = np.linalg.inv(Sigma)
    det = np.linalg.det(Sigma)
    daraja = -0.5 * np.einsum("ij,jk,ik->i", ayirma, teskari, ayirma)
    return np.exp(daraja) / np.sqrt((2 * np.pi) ** p * det)


def main() -> None:
    X, haqiqiy = yarat()
    Xs = StandardScaler().fit_transform(X)
    n, p = Xs.shape
    K = 2
    rng = np.random.default_rng(0)

    print("=== 1. Boshlang'ich parametrlar ===")
    pi = np.full(K, 1.0 / K)
    mu = Xs[rng.choice(n, K, replace=False)]
    Sigma = np.array([np.cov(Xs.T) for _ in range(K)])
    print(f"  og'irliklar: {np.round(pi, 4).tolist()}")
    print(f"  markazlar: {np.round(mu, 3).tolist()}")

    print("\n=== 2. EM iteratsiyalari ===")
    print(f"  {'qadam':>6} {'log-ehtimollik':>16} {'o_sish':>11} "
          f"{'og_irliklar':>22}")
    oldingi = None
    for qadam in range(50):
        # E-qadam
        zichlik = np.column_stack([pi[k] * gauss_zichlik(Xs, mu[k], Sigma[k])
                                   for k in range(K)])
        jami = zichlik.sum(axis=1)
        gamma = zichlik / jami[:, None]
        logL = float(np.log(jami).sum())
        # M-qadam
        Nk = gamma.sum(axis=0)
        pi = Nk / n
        mu = (gamma.T @ Xs) / Nk[:, None]
        Sigma = np.array([
            (gamma[:, k, None] * (Xs - mu[k])).T @ (Xs - mu[k]) / Nk[k]
            + 1e-6 * np.eye(p) for k in range(K)])
        osish = "-" if oldingi is None else f"{logL - oldingi:.6f}"
        if qadam in [0, 1, 4, 9, 24, 49]:
            print(f"  {qadam + 1:>6} {logL:>16.4f} {osish:>11} "
                  f"{str(np.round(pi, 4).tolist()):>22}")
        oldingi = logL

    print("\n=== 3. sklearn bilan solishtirish ===")
    g = GaussianMixture(2, covariance_type="full", n_init=10,
                        random_state=0).fit(Xs)
    print(f"  qo'lda log-ehtimollik:  {oldingi:.4f}")
    print(f"  sklearn:                {g.score(Xs) * n:.4f}")
    print(f"  qo'lda og'irliklar:  {np.round(np.sort(pi), 4).tolist()}")
    print(f"  sklearn og'irliklar: {np.round(np.sort(g.weights_), 4).tolist()}")
    print(f"  haqiqiy og'irliklar: "
          f"{np.round(np.sort(np.bincount(haqiqiy) / n), 4).tolist()}")

    print("\n=== 4. Log-ehtimollik monoton oshadimi ===")
    pi2 = np.full(K, 1.0 / K)
    mu2 = Xs[np.random.default_rng(5).choice(n, K, replace=False)]
    Sigma2 = np.array([np.cov(Xs.T) for _ in range(K)])
    tarix = []
    for _ in range(30):
        z = np.column_stack([pi2[k] * gauss_zichlik(Xs, mu2[k], Sigma2[k])
                             for k in range(K)])
        jami = z.sum(axis=1)
        gamma = z / jami[:, None]
        tarix.append(float(np.log(jami).sum()))
        Nk = gamma.sum(axis=0)
        pi2 = Nk / n
        mu2 = (gamma.T @ Xs) / Nk[:, None]
        Sigma2 = np.array([
            (gamma[:, k, None] * (Xs - mu2[k])).T @ (Xs - mu2[k]) / Nk[k]
            + 1e-6 * np.eye(p) for k in range(K)])
    farqlar = np.diff(tarix)
    print(f"  30 qadamda log-ehtimollik: {tarix[0]:.2f} -> {tarix[-1]:.2f}")
    print(f"  manfiy o'sishlar soni: {int((farqlar < -1e-8).sum())} / "
          f"{len(farqlar)}")
    print(f"  eng kichik o'sish: {farqlar.min():.8f}")
    print("  ⭐ EM log-ehtimollikni monoton oshiradi")


if __name__ == "__main__":
    main()

=== 1. Boshlang'ich parametrlar ===
  og'irliklar: [0.5, 0.5]
  markazlar: [[-0.776, -0.672], [-0.903, -0.343]]

=== 2. EM iteratsiyalari ===
   qadam   log-ehtimollik      o_sish            og_irliklar
       1       -3477.4009           -       [0.5337, 0.4663]
       2       -3029.0155  448.385428       [0.5366, 0.4634]
       5       -2902.8523   50.077182       [0.5882, 0.4118]
      10       -2600.5738   74.545991       [0.6495, 0.3505]
      25       -2442.2944    0.000000       [0.6527, 0.3473]
      50       -2442.2944   -0.000000       [0.6527, 0.3473]

=== 3. sklearn bilan solishtirish ===
  qo'lda log-ehtimollik:  -2442.2944
  sklearn:                -2442.3346
  qo'lda og'irliklar:  [0.3473, 0.6527]
  sklearn og'irliklar: [0.346, 0.654]
  haqiqiy og'irliklar: [0.3475, 0.6525]

=== 4. Log-ehtimollik monoton oshadimi ===
  30 qadamda log-ehtimollik: -3353.65 -> -2442.29
  manfiy o'sishlar soni: 0 / 29
  eng kichik o'sish: -0.00000000
  ⭐ EM log-ehtimollikni monoton oshiradi

"""GMM ning generativ imkoniyatlari (real numpy/pandas/sklearn)."""

import numpy as np
import pandas as pd
from sklearn.metrics import roc_auc_score
from sklearn.mixture import BayesianGaussianMixture, GaussianMixture
from sklearn.preprocessing import StandardScaler


def yarat(seed: int = 12, n: int = 3000, anomaliya: float = 0.02):
    rng = np.random.default_rng(seed)
    rejim = rng.choice(3, n, p=[0.5, 0.3, 0.2])
    markazlar = np.array([[20.0, 1.2], [45.0, 2.8], [70.0, 1.6]])
    kengliklar = np.array([[3.0, 0.25], [5.0, 0.4], [4.0, 0.3]])
    X = markazlar[rejim] + rng.normal(0, 1, (n, 2)) * kengliklar[rejim]
    buzuq = rng.random(n) < anomaliya
    X[buzuq] += rng.normal(0, 1, (int(buzuq.sum()), 2)) * np.array([25.0, 3.0])
    return X, rejim, buzuq


def main() -> None:
    X, rejim, buzuq = yarat()
    sc = StandardScaler()
    Xs = sc.fit_transform(X)

    print("=== 1. Ma'lumot ===")
    print(f"  {len(X)} o'lchov, 3 ta rejim")
    print(f"  rejim ulushlari: "
          f"{np.round(np.bincount(rejim) / len(X), 3).tolist()}")
    print(f"  anomaliyalar: {int(buzuq.sum())} ({buzuq.mean():.1%})")

    print("\n=== 2. Zichlikka asoslangan anomaliya ===")
    g = GaussianMixture(3, covariance_type="full", n_init=10,
                        random_state=0).fit(Xs)
    ball = g.score_samples(Xs)
    print(f"  log-zichlik: min {ball.min():.2f}, mediana "
          f"{np.median(ball):.2f}, max {ball.max():.2f}")
    print(f"  ROC AUC (anomaliyani aniqlash): "
          f"{roc_auc_score(buzuq, -ball):.4f}")
    print(f"  {'chegara':>10} {'belgilangan':>13} {'topildi':>10} "
          f"{'aniqlik':>10}")
    for q in [0.01, 0.02, 0.05, 0.10]:
        chegara = np.quantile(ball, q)
        bashorat = ball < chegara
        topildi = int((bashorat & buzuq).sum())
        print(f"  {q:>10.0%} {int(bashorat.sum()):>13} "
              f"{topildi:>10} {topildi / max(bashorat.sum(), 1):>10.1%}")

    print("\n=== 3. Yumshoq segmentatsiya ===")
    p = g.predict_proba(Xs)
    ishonch = p.max(axis=1)
    print(f"  {'ishonch':>14} {'nuqtalar':>10} {'ulush':>8}")
    for past, yuqori in [(0.34, 0.6), (0.6, 0.8), (0.8, 0.95), (0.95, 1.01)]:
        m = (ishonch >= past) & (ishonch < yuqori)
        print(f"  {past:.2f} - {yuqori:.2f} {int(m.sum()):>10} "
              f"{m.mean():>7.1%}")
    oraliq = ishonch < 0.8
    print(f"  o'tish davridagi nuqtalar (ishonch < 0.8): "
          f"{int(oraliq.sum())} ({oraliq.mean():.1%})")
    print(f"  ulardagi anomaliya ulushi: {buzuq[oraliq].mean():.1%} "
          f"(umumiy {buzuq.mean():.1%})")

    print("\n=== 4. Generatsiya va BayesianGaussianMixture ===")
    Xyangi, yorliq = g.sample(2000)
    asl = pd.DataFrame(sc.inverse_transform(Xs), columns=["harorat", "tebranish"])
    yangi = pd.DataFrame(sc.inverse_transform(Xyangi),
                         columns=["harorat", "tebranish"])
    print(f"  {'belgi':<12} {'asl o_rtacha':>14} {'yangi o_rtacha':>16} "
          f"{'asl std':>10} {'yangi std':>11}")
    for nom in asl.columns:
        print(f"  {nom:<12} {asl[nom].mean():>14.3f} {yangi[nom].mean():>16.3f} "
              f"{asl[nom].std():>10.3f} {yangi[nom].std():>11.3f}")
    b = BayesianGaussianMixture(n_components=10, covariance_type="full",
                                weight_concentration_prior=0.01, n_init=5,
                                random_state=0, max_iter=300).fit(Xs)
    faol = int((b.weights_ > 0.02).sum())
    print(f"  BayesianGaussianMixture: 10 ta komponentdan {faol} tasi faol")
    print(f"  og'irliklar: {np.round(np.sort(b.weights_)[::-1][:6], 4).tolist()}")
    print("  ⭐ GMM - to'liq generativ model")


if __name__ == "__main__":
    main()

=== 1. Ma'lumot ===
  3000 o'lchov, 3 ta rejim
  rejim ulushlari: [0.501, 0.303, 0.196]
  anomaliyalar: 49 (1.6%)

=== 2. Zichlikka asoslangan anomaliya ===
  log-zichlik: min -12.01, mediana -1.71, max 0.59
  ROC AUC (anomaliyani aniqlash): 0.9496
     chegara   belgilangan    topildi    aniqlik
          1%            30         30     100.0%
          2%            60         39      65.0%
          5%           150         44      29.3%
         10%           300         44      14.7%

=== 3. Yumshoq segmentatsiya ===
         ishonch   nuqtalar    ulush
  0.34 - 0.60          4    0.1%
  0.60 - 0.80          4    0.1%
  0.80 - 0.95         35    1.2%
  0.95 - 1.01       2957   98.6%
  o'tish davridagi nuqtalar (ishonch < 0.8): 8 (0.3%)
  ulardagi anomaliya ulushi: 50.0% (umumiy 1.6%)

=== 4. Generatsiya va BayesianGaussianMixture ===
  belgi          asl o_rtacha   yangi o_rtacha    asl std   yangi std
  harorat              37.581           37.709     20.169      20.207
  tebranish             1.765            1.753      0.865       0.854
  BayesianGaussianMixture: 10 ta komponentdan 3 tasi faol
  og'irliklar: [0.4935, 0.2989, 0.1924, 0.0152, 0.0, 0.0]
  ⭐ GMM - to'liq generativ model

GaussianMixture(4).fit(X)                                        # ⚠️
GaussianMixture(4).fit(StandardScaler().fit_transform(X))        # ✅

GaussianMixture(4)               # standart n_init=1              # ⚠️
GaussianMixture(4, n_init=10, random_state=0)                    # ✅

GaussianMixture(5, covariance_type="full").fit(X_100belgi)       # ⚠️
GaussianMixture(5, covariance_type="diag").fit(X_100belgi)       # ✅

GaussianMixture(20).fit(X_kichik)   # ValueError                 # ⚠️
GaussianMixture(20, reg_covar=1e-4).fit(X_kichik)                # ✅

yorliq = g.predict(Xs)                                           # ⚠️
p = g.predict_proba(Xs); ishonch = p.max(axis=1)                 # ✅

k = min(range(1, 20), key=lambda k: GaussianMixture(k).fit(Xs).bic(Xs)) # ⚠️
# BIC + barqarorlik + mazmun 16.3-bob                              # ✅

GaussianMixture(2).fit(yarim_oy_malumot)                         # ⚠️
DBSCAN(eps=0.3).fit(yarim_oy_malumot)                            # ✅

1.  # GMM modeli formulasi?
2.  # mas'uliyat (responsibility) nima?
3.  # EM ning ikki qadami?
4.  # EM nimani oshiradi?
5.  # yaqinlashish kafolatlanganmi?
6.  # kovariatsiya turlari?
7.  # yuqori o'lchamda qaysi tur?
8.  # BIC formulasi?
9.  # AIC va BIC farqi?
10. # K-means GMM ning qanday holati?
11. # score_samples nima beradi?
12. # sample() nima qiladi?

1.  GaussianMixture(4).fit(X)   # masshtablanmagan

2.  GaussianMixture(4)   # n_init standart

3.  GaussianMixture(5, covariance_type="full").fit(X_100belgi)

4.  yorliq = g.predict(Xs)   # ishonch kerak edi

5.  GaussianMixture(2).fit(yarim_oy_malumot)