20.3-dars: Qatlamlar va arxitektura

TO'LIQ BOG'LANGAN QATLAM (Dense / Linear):
  kirish:  (n_namuna, n_kirish)
  W:       (n_kirish, n_chiqish)
  b:       (n_chiqish,)
  chiqish: (n_namuna, n_chiqish)

  chiqish = f(kirish @ W + b)

PARAMETRLAR SONI = n_kirish * n_chiqish + n_chiqish
                 = n_chiqish * (n_kirish + 1)

MISOL (20 -> 64 -> 64 -> 3):
  1-qatlam: 64 * (20 + 1) = 1344
  2-qatlam: 64 * (64 + 1) = 4160
  3-qatlam:  3 * (64 + 1) =  195
  JAMI: 5699

QOIDA: har qatlamning n_kirish i oldingi qatlamning n_chiqish iga teng

(n, 20) @ (20, 64) -> (n, 64)   OK
(n, 64) @ (64, 64) -> (n, 64)   OK
(n, 64) @ (64,  3) -> (n,  3)   OK

(n, 64) @ (20, 64) -> XATO: 64 != 20

BIRINCHI QATLAM n_kirish = belgilar soni
OXIRGI QATLAM n_chiqish:
  regressiya         -> 1
  binar              -> 1 (sigmoid) yoki 2 (softmax)
  K sinfli           -> K
  ko'p yorliqli      -> yorliqlar soni

KENGLIK (bir qatlamdagi neyronlar soni):
  + har qatlam ko'proq XUSUSIYAT o'rgana oladi
  + parallel hisoblashga qulay
  - parametrlar KVADRATIK o'sadi
  - juda keng => yod olish

CHUQURLIK (qatlamlar soni):
  + IERARXIYA: oddiy xususiyatlardan murakkabiga
  + bir xil ifoda kuchi uchun KAMROQ parametr
  - gradient muammolari 20.2-bob
  - o'rgatish qiyinroq

NAZARIY FAKT:
  ba'zi funksiyalar uchun CHUQUR tarmoq eksponensial kam neyron talab
  qiladi (Telgarsky 2016), lekin buni AMALDA foydalanish qiyin

AMALIY BOSHLANG'ICH NUQTA:
  tabular ma'lumot: 2-3 qatlam, 32-256 neyron
  rasm:             CNN (24-qism)
  matn:             transformer (26-qism)

QO'POL MO'LJAL:
  parametrlar soni <= namunalar soni / 10   (regularizatsiyasiz)

MISOL: 5000 namuna
  byudjet ~ 500 parametr
  (20 -> 16 -> 8 -> 3):  16*21 + 8*17 + 3*9 = 336 + 136 + 27 = 499

BU QAT'IY QOIDA EMAS:
  dropout, weight decay, erta to'xtash byudjetni kengaytiradi
  lekin YO'NALISH to'g'ri: kam ma'lumot -> kichik tarmoq

TEKSHIRUV: o'quv va validatsiya balli orasidagi farq
  farq katta -> model juda katta yoki regularizatsiya kam

1. BAZAVIY: chiziqli model (logistik / Ridge)
2. KICHIK tarmoq: (32,) - bazaviydan yaxshimi?
3. KENGAYTIRISH: (64,) -> (128,) - to'xtaguncha
4. CHUQURLASHTIRISH: (64, 64) -> (64, 64, 64)
5. REGULARIZATSIYA: dropout, weight decay, erta to'xtash
6. Har qadamda CV bilan tekshiring va SE ni hisoblang (18-qism)

MUHIM: har qadamda FAQAT BITTA narsani o'zgartiring

(784 -> 512 -> 256 -> 10) tarmoqda:
  1-qatlam: 512 * 785 = 401 920   (78%)
  2-qatlam: 256 * 513 = 131 328   (26%)
  3-qatlam:  10 * 257 =   2 570   (0.5%)

BIRINCHI QATLAM odatda eng katta, chunki kirish o'lchami katta

XULOSA: kirish o'lchamini kamaytirish (belgi tanlash, PCA)
        parametrlarni KESKIN kamaytiradi

import numpy as np

def parametrlar(olchamlar):
    """olchamlar = [kirish, yashirin1, ..., chiqish]"""
    return sum(olchamlar[i + 1] * (olchamlar[i] + 1)
               for i in range(len(olchamlar) - 1))

parametrlar([20, 64, 64, 3])          # 5699

# qatlamlarni yaratish
rng = np.random.default_rng(0)
Ws, bs = [], []
for i in range(len(olchamlar) - 1):
    n_in, n_out = olchamlar[i], olchamlar[i + 1]
    Ws.append(rng.normal(0, np.sqrt(2 / n_in), (n_in, n_out)))
    bs.append(np.zeros(n_out))

# sklearn da
MLPClassifier(hidden_layer_sizes=(64, 64))   # kirish/chiqish avtomatik

parametr = n_chiqish * (n_kirish + 1)
kenglik x2 -> parametr x4
byudjet  ~ namunalar / 10
tartib:  chiziqli -> kichik -> keng -> chuqur -> regularizatsiya

"""Arxitektura raqamlarda (real numpy)."""

import numpy as np


def qatlam_parametrlari(n_kirish, n_chiqish):
    return n_chiqish * n_kirish, n_chiqish


def jami_parametrlar(olchamlar):
    return sum(o * (i + 1) for i, o in zip(olchamlar[:-1], olchamlar[1:]))


def main() -> None:
    print("=== 1. Bitta qatlam ===")
    print(f"  {'kirish':>8} {'chiqish':>8} {'W':>10} {'b':>6} {'jami':>8}")
    for n_in, n_out in [(20, 64), (64, 64), (64, 3), (784, 512)]:
        w, b = qatlam_parametrlari(n_in, n_out)
        print(f"  {n_in:>8} {n_out:>8} {w:>10} {b:>6} {w + b:>8}")

    print("\n=== 2. To'liq tarmoq ===")
    olchamlar = [20, 64, 64, 3]
    print(f"  arxitektura: {' -> '.join(map(str, olchamlar))}")
    print(f"  {'qatlam':>7} {'shakl':>12} {'parametr':>10} {'ulush':>8}")
    jami = jami_parametrlar(olchamlar)
    for i in range(len(olchamlar) - 1):
        n_in, n_out = olchamlar[i], olchamlar[i + 1]
        p = n_out * (n_in + 1)
        print(f"  {i + 1:>7} {f'({n_in}, {n_out})':>12} {p:>10} "
              f"{p / jami:>8.1%}")
    print(f"  {'JAMI':>7} {'':>12} {jami:>10}")

    print("\n=== 3. Kenglikni oshirish ===")
    print(f"  {'kenglik':>8} {'parametr':>10} {'nisbat':>9}")
    asos = jami_parametrlar([20, 32, 32, 3])
    for k in [32, 64, 128, 256, 512]:
        p = jami_parametrlar([20, k, k, 3])
        print(f"  {k:>8} {p:>10} {p / asos:>9.1f}x")
    print("  kenglik 2x -> parametr ~4x (KVADRATIK)")

    print("\n=== 4. Chuqurlikni oshirish ===")
    print(f"  {'qatlamlar':>10} {'parametr':>10} {'nisbat':>9}")
    asos = jami_parametrlar([20, 64, 3])
    for n in range(1, 7):
        olch = [20] + [64] * n + [3]
        p = jami_parametrlar(olch)
        print(f"  {n:>10} {p:>10} {p / asos:>9.1f}x")
    print("  chuqurlik 2x -> parametr ~2x (CHIZIQLI)")

    print("\n=== 5. Bir xil byudjet, boshqa shakl ===")
    variantlar = [[20, 256, 3], [20, 96, 96, 3], [20, 64, 64, 64, 3],
                  [20, 48, 48, 48, 48, 3]]
    print(f"  {'arxitektura':<28} {'parametr':>10} {'chuqurlik':>10}")
    for olch in variantlar:
        p = jami_parametrlar(olch)
        print(f"  {' -> '.join(map(str, olch)):<28} {p:>10} "
              f"{len(olch) - 2:>10}")
    print("  ⭐ Bir xil byudjetni turlicha taqsimlash mumkin")

    print("\n=== 6. Birinchi qatlam ulushi ===")
    print(f"  {'kirish':>8} {'1-qatlam ulushi':>17}")
    for kirish in [10, 50, 200, 784, 3000]:
        olch = [kirish, 512, 256, 10]
        p1 = 512 * (kirish + 1)
        print(f"  {kirish:>8} {p1 / jami_parametrlar(olch):>17.1%}")
    print("  kirish kattalashsa - 1-qatlam hamma narsani yutadi")


if __name__ == "__main__":
    main()

=== 1. Bitta qatlam ===
    kirish  chiqish          W      b     jami
        20       64       1280     64     1344
        64       64       4096     64     4160
        64        3        192      3      195
       784      512     401408    512   401920

=== 2. To'liq tarmoq ===
  arxitektura: 20 -> 64 -> 64 -> 3
   qatlam        shakl   parametr    ulush
        1     (20, 64)       1344    23.6%
        2     (64, 64)       4160    73.0%
        3      (64, 3)        195     3.4%
     JAMI                    5699

=== 3. Kenglikni oshirish ===
   kenglik   parametr    nisbat
        32       1827       1.0x
        64       5699       3.1x
       128      19587      10.7x
       256      71939      39.4x
       512     274947     150.5x
  kenglik 2x -> parametr ~4x (KVADRATIK)

=== 4. Chuqurlikni oshirish ===
   qatlamlar   parametr    nisbat
           1       1539       1.0x
           2       5699       3.7x
           3       9859       6.4x
           4      14019       9.1x
           5      18179      11.8x
           6      22339      14.5x
  chuqurlik 2x -> parametr ~2x (CHIZIQLI)

=== 5. Bir xil byudjet, boshqa shakl ===
  arxitektura                    parametr  chuqurlik
  20 -> 256 -> 3                     6147          1
  20 -> 96 -> 96 -> 3               11619          2
  20 -> 64 -> 64 -> 64 -> 3          9859          3
  20 -> 48 -> 48 -> 48 -> 48 -> 3       8211          4
  ⭐ Bir xil byudjetni turlicha taqsimlash mumkin

=== 6. Birinchi qatlam ulushi ===
    kirish   1-qatlam ulushi
        10              4.0%
        50             16.3%
       200             43.5%
       784             75.0%
      3000             92.0%
  kirish kattalashsa - 1-qatlam hamma narsani yutadi

"""Shakllar mosligi va oldinga o'tish (real numpy)."""

import numpy as np


def tarmoq_yarat(olchamlar, seed=0):
    rng = np.random.default_rng(seed)
    Ws, bs = [], []
    for n_in, n_out in zip(olchamlar[:-1], olchamlar[1:]):
        Ws.append(rng.normal(0, np.sqrt(2.0 / n_in), (n_in, n_out)))
        bs.append(np.zeros(n_out))
    return Ws, bs


def oldinga(X, Ws, bs):
    """Har qatlam chiqishini qaytaradi."""
    a = X
    chiqishlar = [a]
    for i, (W, b) in enumerate(zip(Ws, bs)):
        z = a @ W + b
        a = np.maximum(0, z) if i < len(Ws) - 1 else z
        chiqishlar.append(a)
    return chiqishlar


def main() -> None:
    rng = np.random.default_rng(1)
    X = rng.normal(0, 1, (8, 20))
    olchamlar = [20, 64, 32, 3]

    print("=== 1. Og'irliklar shakli ===")
    Ws, bs = tarmoq_yarat(olchamlar)
    print(f"  {'qatlam':>7} {'W shakli':>12} {'b shakli':>10} "
          f"{'W std':>9}")
    for i, (W, b) in enumerate(zip(Ws, bs)):
        print(f"  {i + 1:>7} {str(W.shape):>12} {str(b.shape):>10} "
              f"{W.std():>9.4f}")

    print("\n=== 2. Oldinga o'tish: shakllar zanjiri ===")
    chiqishlar = oldinga(X, Ws, bs)
    print(f"  {'bosqich':<18} {'shakl':>12} {'o_rtacha':>10} {'std':>9}")
    nomlar = ["kirish"] + [f"qatlam {i + 1}" for i in range(len(Ws))]
    for nom, a in zip(nomlar, chiqishlar):
        print(f"  {nom:<18} {str(a.shape):>12} {a.mean():>10.4f} "
              f"{a.std():>9.4f}")

    print("\n=== 3. Shakl xatosi ===")
    try:
        X @ Ws[1]                       # (8,20) @ (64,32)
        print("  xato chiqmadi (kutilmagan)")
    except ValueError as xato:
        print(f"  ValueError: {str(xato)[:64]}")
    print("  shakl xatosi DARHOL va ANIQ chiqadi")

    print("\n=== 4. ReLU dan keyingi siyraklik ===")
    print(f"  {'qatlam':>7} {'nol ulushi':>12} {'faol neyron':>13}")
    for i in range(1, len(chiqishlar) - 1):
        a = chiqishlar[i]
        print(f"  {i:>7} {(a == 0).mean():>12.1%} "
              f"{int((a.max(axis=0) > 0).sum()):>13}")
    print("  ReLU chiqishning ~yarmini nolga aylantiradi")

    print("\n=== 5. Chiqish o'lchami vazifaga qarab ===")
    vazifalar = [("regressiya", 1), ("binar (sigmoid)", 1),
                 ("3 sinf (softmax)", 3), ("10 sinf", 10),
                 ("5 yorliq", 5)]
    print(f"  {'vazifa':<20} {'oxirgi qatlam':>14} {'parametr':>10}")
    for nom, k in vazifalar:
        print(f"  {nom:<20} {k:>14} {k * (32 + 1):>10}")

    print("\n=== 6. Kenglik profillari ===")
    profillar = {
        "teng": [20, 64, 64, 64, 3],
        "toraygan": [20, 128, 64, 32, 3],
        "kengaygan": [20, 32, 64, 128, 3],
        "shisha soat": [20, 128, 32, 128, 3],
    }
    print(f"  {'profil':<14} {'shakl':<26} {'parametr':>10}")
    for nom, olch in profillar.items():
        p = sum(o * (i + 1) for i, o in zip(olch[:-1], olch[1:]))
        print(f"  {nom:<14} {'-'.join(map(str, olch)):<26} {p:>10}")
    print("  ⭐ Toraygan profil eng keng tarqalgan")


if __name__ == "__main__":
    main()

=== 1. Og'irliklar shakli ===
   qatlam     W shakli   b shakli     W std
        1     (20, 64)      (64,)    0.3105
        2     (64, 32)      (32,)    0.1792
        3      (32, 3)       (3,)    0.2843

=== 2. Oldinga o'tish: shakllar zanjiri ===
  bosqich                   shakl   o_rtacha       std
  kirish                  (8, 20)    -0.0989    0.8899
  qatlam 1                (8, 64)     0.4701    0.7132
  qatlam 2                (8, 32)     0.4956    0.7232
  qatlam 3                 (8, 3)    -0.0856    1.5503

=== 3. Shakl xatosi ===
  ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0,
  shakl xatosi DARHOL va ANIQ chiqadi

=== 4. ReLU dan keyingi siyraklik ===
   qatlam   nol ulushi   faol neyron
        1        53.5%            63
        2        49.6%            30
  ReLU chiqishning ~yarmini nolga aylantiradi

=== 5. Chiqish o'lchami vazifaga qarab ===
  vazifa                oxirgi qatlam   parametr
  regressiya                        1         33
  binar (sigmoid)                   1         33
  3 sinf (softmax)                  3         99
  10 sinf                          10        330
  5 yorliq                          5        165

=== 6. Kenglik profillari ===
  profil         shakl                        parametr
  teng           20-64-64-64-3                    9859
  toraygan       20-128-64-32-3                  13123
  kengaygan      20-32-64-128-3                  11491
  shisha soat    20-128-32-128-3                 11427
  ⭐ Toraygan profil eng keng tarqalgan

"""Arxitekturani CV bilan tanlash (real sklearn)."""

import warnings

import numpy as np
from sklearn.datasets import make_classification
from sklearn.exceptions import ConvergenceWarning
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold, cross_val_score
from sklearn.neural_network import MLPClassifier
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler


def param_soni(kirish, yashirin, chiqish):
    olch = [kirish] + list(yashirin) + [chiqish]
    return sum(o * (i + 1) for i, o in zip(olch[:-1], olch[1:]))


def quvur(yashirin):
    return make_pipeline(
        StandardScaler(),
        MLPClassifier(hidden_layer_sizes=yashirin, max_iter=120,
                      learning_rate_init=0.02, random_state=0,
                      early_stopping=False))


def baho(model, X, y, cv):
    """Davr byudjeti ATAYLAB cheklangan - ogohlantirishni jim qilamiz."""
    with warnings.catch_warnings():
        warnings.simplefilter("ignore", ConvergenceWarning)
        return cross_val_score(model, X, y, cv=cv, scoring="accuracy")


def main() -> None:
    X, y = make_classification(n_samples=1500, n_features=20,
                               n_informative=8, n_redundant=6,
                               n_classes=3, flip_y=0.12, class_sep=0.9,
                               random_state=0)
    cv = StratifiedKFold(3, shuffle=True, random_state=0)
    print(f"  ma'lumot: {X.shape}, sinflar: {len(np.unique(y))}")

    print("\n=== 1. Bazaviy chiziqli model ===")
    asos = baho(make_pipeline(StandardScaler(),
                              LogisticRegression(max_iter=2000)),
                X, y, cv)
    se = float(asos.std(ddof=1) / np.sqrt(len(asos)))
    print(f"  LogisticRegression CV: {asos.mean():.4f} (SE {se:.4f})")
    print(f"  qaror chegarasi (2*SE): {2 * se:.4f}")

    print("\n=== 2. Kenglikni oshiramiz (1 qatlam) ===")
    print(f"  {'kenglik':>8} {'parametr':>10} {'CV':>8} {'std':>8} "
          f"{'asosdan':>9}")
    for k in [4, 16, 64, 256]:
        b = baho(quvur((k,)), X, y, cv)
        print(f"  {k:>8} {param_soni(20, (k,), 3):>10} {b.mean():>8.4f} "
              f"{b.std():>8.4f} {b.mean() - asos.mean():>+9.4f}")

    print("\n=== 3. Chuqurlikni oshiramiz (kenglik 64) ===")
    print(f"  {'qatlam':>7} {'parametr':>10} {'CV':>8} {'std':>8} "
          f"{'asosdan':>9}")
    for n in [1, 2, 3]:
        yashirin = (64,) * n
        b = baho(quvur(yashirin), X, y, cv)
        print(f"  {n:>7} {param_soni(20, yashirin, 3):>10} "
              f"{b.mean():>8.4f} {b.std():>8.4f} "
              f"{b.mean() - asos.mean():>+9.4f}")

    print("\n=== 4. Bir xil byudjet, boshqa shakl ===")
    variantlar = [(256,), (96, 96), (64, 64, 64)]
    print(f"  {'shakl':<20} {'parametr':>10} {'CV':>8} {'std':>8}")
    for yashirin in variantlar:
        b = baho(quvur(yashirin), X, y, cv)
        print(f"  {str(yashirin):<20} {param_soni(20, yashirin, 3):>10} "
              f"{b.mean():>8.4f} {b.std():>8.4f}")
    print("  bir xil byudjetda farq odatda SE ichida qoladi")

    print("\n=== 5. Ma'lumot kam bo'lsa ===")
    print(f"  {'namuna':>8} {'shakl':<16} {'parametr/namuna':>16} "
          f"{'CV':>8}")
    for n in [300, 1500]:
        for yashirin in [(16,), (256, 256)]:
            p = param_soni(20, yashirin, 3)
            b = baho(quvur(yashirin), X[:n], y[:n], cv)
            print(f"  {n:>8} {str(yashirin):<16} {p / n:>16.2f} "
                  f"{b.mean():>8.4f}")
    print("  \u2b50 Kam ma'lumotda katta tarmoq foyda bermaydi")


if __name__ == "__main__":
    main()

  ma'lumot: (1500, 20), sinflar: 3

=== 1. Bazaviy chiziqli model ===
  LogisticRegression CV: 0.6160 (SE 0.0125)
  qaror chegarasi (2*SE): 0.0250

=== 2. Kenglikni oshiramiz (1 qatlam) ===
   kenglik   parametr       CV      std   asosdan
         4         99   0.6860   0.0199   +0.0700
        16        387   0.7267   0.0096   +0.1107
        64       1539   0.7233   0.0255   +0.1073
       256       6147   0.7360   0.0263   +0.1200

=== 3. Chuqurlikni oshiramiz (kenglik 64) ===
   qatlam   parametr       CV      std   asosdan
        1       1539   0.7233   0.0255   +0.1073
        2       5699   0.7500   0.0201   +0.1340
        3       9859   0.7413   0.0137   +0.1253

=== 4. Bir xil byudjet, boshqa shakl ===
  shakl                  parametr       CV      std
  (256,)                     6147   0.7360   0.0263
  (96, 96)                  11619   0.7333   0.0217
  (64, 64, 64)               9859   0.7413   0.0137
  bir xil byudjetda farq odatda SE ichida qoladi

=== 5. Ma'lumot kam bo'lsa ===
    namuna shakl             parametr/namuna       CV
       300 (16,)                        1.29   0.6833
       300 (256, 256)                 239.80   0.6767
      1500 (16,)                        0.26   0.7267
      1500 (256, 256)                  47.96   0.7513
  ⭐ Kam ma'lumotda katta tarmoq foyda bermaydi

"""O'quv va validatsiya farqi arxitektura hajmiga qarab."""

import numpy as np
from sklearn.datasets import make_classification
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPClassifier
from sklearn.preprocessing import StandardScaler


def param_soni(kirish, yashirin, chiqish):
    olch = [kirish] + list(yashirin) + [chiqish]
    return sum(o * (i + 1) for i, o in zip(olch[:-1], olch[1:]))


def main() -> None:
    X, y = make_classification(n_samples=900, n_features=20,
                               n_informative=6, n_redundant=4,
                               n_classes=2, flip_y=0.15, class_sep=0.8,
                               random_state=0)
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.35,
                                          random_state=0, stratify=y)
    sc = StandardScaler().fit(Xtr)
    Xtr, Xte = sc.transform(Xtr), sc.transform(Xte)
    print(f"  o'quv: {Xtr.shape}, test: {Xte.shape}")

    print("\n=== 1. Hajm ortganda o'quv/test farqi ===")
    print(f"  {'shakl':<18} {'parametr':>9} {'p/n':>7} {'o_quv':>8} "
          f"{'test':>8} {'farq':>8}")
    natijalar = []
    for yashirin in [(4,), (16,), (64,), (256,), (256, 256)]:
        m = MLPClassifier(hidden_layer_sizes=yashirin, max_iter=800,
                          learning_rate_init=0.01, random_state=0,
                          early_stopping=False).fit(Xtr, ytr)
        a_tr = accuracy_score(ytr, m.predict(Xtr))
        a_te = accuracy_score(yte, m.predict(Xte))
        p = param_soni(20, yashirin, 1)
        natijalar.append((yashirin, p, a_tr, a_te))
        print(f"  {str(yashirin):<18} {p:>9} {p / len(Xtr):>7.2f} "
              f"{a_tr:>8.4f} {a_te:>8.4f} {a_tr - a_te:>8.4f}")

    print("\n=== 2. Eng yaxshi test balli ===")
    eng = max(natijalar, key=lambda t: t[3])
    print(f"  shakl: {eng[0]}, parametr: {eng[1]}")
    print(f"  test: {eng[3]:.4f}, o'quv: {eng[2]:.4f}")
    eng_katta = max(natijalar, key=lambda t: t[1])
    print(f"  eng katta model testi: {eng_katta[3]:.4f} "
          f"({eng_katta[1]} parametr)")
    print(f"  ⭐ Eng katta model eng yaxshisi EMAS")

    print("\n=== 3. Regularizatsiya byudjetni kengaytiradi ===")
    print(f"  {'alpha':>9} {'o_quv':>8} {'test':>8} {'farq':>8}")
    for alpha in [1e-6, 1e-3, 1e-1, 1.0, 10.0]:
        m = MLPClassifier(hidden_layer_sizes=(256, 256), alpha=alpha,
                          max_iter=800, learning_rate_init=0.01,
                          random_state=0,
                          early_stopping=False).fit(Xtr, ytr)
        a_tr = accuracy_score(ytr, m.predict(Xtr))
        a_te = accuracy_score(yte, m.predict(Xte))
        print(f"  {alpha:>9.0e} {a_tr:>8.4f} {a_te:>8.4f} "
              f"{a_tr - a_te:>8.4f}")
    print("  katta model + regularizatsiya = ishlaydigan variant")

    print("\n=== 4. Erta to'xtash ===")
    print(f"  {'early_stopping':>15} {'davrlar':>9} {'o_quv':>8} "
          f"{'test':>8}")
    for erta in [False, True]:
        m = MLPClassifier(hidden_layer_sizes=(256, 256), max_iter=800,
                          learning_rate_init=0.01, random_state=0,
                          early_stopping=erta,
                          n_iter_no_change=10).fit(Xtr, ytr)
        print(f"  {str(erta):>15} {m.n_iter_:>9} "
              f"{accuracy_score(ytr, m.predict(Xtr)):>8.4f} "
              f"{accuracy_score(yte, m.predict(Xte)):>8.4f}")

    print("\n=== 5. Tavsiya ===")
    tavsiya = [
        ("n < 1000", "(16,) yoki (32,)"),
        ("1000 - 10000", "(64,) yoki (64, 64)"),
        ("10000 - 100000", "(128, 128) yoki (256, 128)"),
        ("n > 100000", "chuqurroq + regularizatsiya"),
    ]
    print(f"  {'namunalar':<18} {'boshlang_ich shakl'}")
    for a, b in tavsiya:
        print(f"  {a:<18} {b}")
    print("  ⭐ Bu boshlang'ich nuqta, yakuniy javob emas")


if __name__ == "__main__":
    main()

  o'quv: (585, 20), test: (315, 20)

=== 1. Hajm ortganda o'quv/test farqi ===
  shakl               parametr     p/n    o_quv     test     farq
  (4,)                      89    0.15   0.8650   0.7810   0.0840
  (16,)                    353    0.60   1.0000   0.7111   0.2889
  (64,)                   1409    2.41   1.0000   0.7365   0.2635
  (256,)                  5633    9.63   1.0000   0.7619   0.2381
  (256, 256)             71425  122.09   1.0000   0.7746   0.2254

=== 2. Eng yaxshi test balli ===
  shakl: (4,), parametr: 89
  test: 0.7810, o'quv: 0.8650
  eng katta model testi: 0.7746 (71425 parametr)
  ⭐ Eng katta model eng yaxshisi EMAS

=== 3. Regularizatsiya byudjetni kengaytiradi ===
      alpha    o_quv     test     farq
      1e-06   0.9966   0.7683   0.2283
      1e-03   0.9949   0.7619   0.2330
      1e-01   0.9692   0.7651   0.2042
      1e+00   0.9949   0.7619   0.2330
      1e+01   0.7812   0.7714   0.0098
  katta model + regularizatsiya = ishlaydigan variant

=== 4. Erta to'xtash ===
   early_stopping   davrlar    o_quv     test
            False        53   1.0000   0.7746
             True        17   0.9009   0.8159

=== 5. Tavsiya ===
  namunalar          boshlang_ich shakl
  n < 1000           (16,) yoki (32,)
  1000 - 10000       (64,) yoki (64, 64)
  10000 - 100000     (128, 128) yoki (256, 128)
  n > 100000         chuqurroq + regularizatsiya
  ⭐ Bu boshlang'ich nuqta, yakuniy javob emas

MLPClassifier(hidden_layer_sizes=(512, 512, 512))     # ⚠️
MLPClassifier(hidden_layer_sizes=(32,))               # ✅ keyin oshiring

# shunchaki o'rgatish                                 # ⚠️
print(param_soni(n_belgi, yashirin, n_sinf) / len(X)) # ✅

# 3 sinf uchun 1 ta chiqish                           # ⚠️
# 3 sinf uchun 3 ta chiqish + softmax                 # ✅

MLPClassifier().fit(X, y)                             # ⚠️
make_pipeline(StandardScaler(), MLPClassifier())      # ✅

# (32,) -> (256, 256, 256) va lr ni ham o'zgartirish  # ⚠️
# bir qadamda FAQAT bitta o'zgarish                   # ✅

# faqat tarmoqlarni taqqoslash                        # ⚠️
# avval LogisticRegression / Ridge                    # ✅

print(test_ball)                                      # ⚠️
print(oquv_ball, test_ball, oquv_ball - test_ball)    # ✅

1.  # (20 -> 64) qatlamda necha parametr?
2.  # kenglik 2x bo'lsa parametr necha barobar?
3.  # chuqurlik 2x bo'lsa-chi?
4.  # oxirgi qatlam 3 sinf uchun?
5.  # parametr/namuna mo'ljali?
6.  # qaysi qatlam eng katta?
7.  # shakl xatosi qachon chiqadi?
8.  # tabularda necha qatlam?
9.  # bazaviy model nima?
10. # yod olish belgisi?
11. # regularizatsiya nima beradi?
12. # toraygan profil nima?

1.  MLPClassifier(hidden_layer_sizes=(512, 512, 512))

2.  MLPClassifier().fit(X, y)

3.  # 3 sinf uchun 1 ta chiqish

4.  print(test_ball)

5.  # faqat tarmoqlarni taqqoslash