22.4-dars: Pooling

OYNA k x k, QADAM s (odatda k = s = 2):

  MaxPool:  y[i, j] = max(oyna)
  AvgPool:  y[i, j] = mean(oyna)

MISOL (4x4 -> 2x2):
  1 3 | 2 0          max:  6 2       avg: 3.75 1.25
  5 6 | 1 2                7 9            2.50 6.00
  ----+----
  7 2 | 9 4
  0 1 | 3 8

XUSUSIYATLARI:
  parametrlar: 0 (hech narsa o'rganmaydi)
  har kanal ALOHIDA: (N, C, H, W) -> (N, C, H/2, W/2), C o'zgarmaydi

GRADIENT:
  max: faqat maksimum turgan joyga (qolganlari 0) - "g'olib oladi"
  avg: oyna bo'ylab teng (1 / k^2)

MA'NOSI:
  max - "shu sohada xususiyat BORMI" (eng kuchli javob)
  avg - "shu sohada xususiyat QANCHA" (silliqlaydi)

FORMULA - konvolyutsiya bilan bir xil (22.2-dars):
  H_chiq = floor((H + 2p - k) / s) + 1

  MaxPool2d(2):              32 -> 16,  7 -> 3  (oxirgi qator tashlanadi)
  MaxPool2d(2, ceil_mode=True): 7 -> 4
  MaxPool2d(3, 2, padding=1): 32 -> 16  (ustma-ust oynalar, ResNet boshida)

NIMA BERADI:
  hisob: keyingi qatlamlar 4 barobar arzon
  retseptiv maydon: jump 2 barobar -> keyingi 3x3 ikki barobar keng ko'radi
  kanallar odatda ko'paytiriladi: 32x32x16 -> 16x16x32
    ("qayerda" aniqligi -> "nima" boyligi)

TIPIK ZANJIR (32x32 kirish):
  conv 32x32x16 -> pool 16x16x16 -> conv 16x16x32 -> pool 8x8x32
  -> conv 8x8x64 -> pool 4x4x64 -> global pool 64 -> Linear

GAP: har kanal -> bitta son (butun xarita o'rtachasi)
  (N, C, H, W) -> (N, C, 1, 1) -> flatten -> (N, C)
  nn.AdaptiveAvgPool2d(1)  ==  x.mean(dim=(2, 3))
  global max: nn.AdaptiveMaxPool2d(1) == x.amax(dim=(2, 3))

BOSH (klassifikator) TAQQOSLASH, 1000 sinf:
                        Flatten + Linear     GAP + Linear
  512 x 7 x 7              25 089 000          513 000
  2048 x 7 x 7            100 353 000        2 049 000

AFZALLIKLARI:
  1. parametrlar H*W marta kam -> ortiqcha moslashish kam
  2. ISTALGAN kirish o'lchami ishlaydi
  3. siljishga invariant (ob'ekt chetdan chiqmaguncha)

ADAPTIVE POOLING: chiqish o'lchami beriladi, oyna o'zi tanlanadi
  AdaptiveAvgPool2d((2, 2)): 7x7, 10x10, 13x13 -> hammasi 2x2

NARXI: "qayerda" ma'lumoti butunlay yo'qoladi
  -> aniqlash va segmentatsiyada global pooling ishlatilmaydi

AFSONA: "MaxPool tarmoqni siljishga invariant qiladi"

HAQIQAT:
  MaxPool 2x2 - faqat oyna ICHIDAGI siljishni "yutadi"
    maksimum oynadan chiqib ketmasa - chiqish o'zgarmaydi
    oyna chegarasidan o'tsa - boshqa oynaga tushadi
  siljish oyna o'lchamiga yetganda foyda deyarli yo'qoladi

  juft siljish (2 piksel) va 2x2 pool:
    chiqish ANIQ 1 ga siljiydi - bu ekvivariantlik, invariantlik emas

O'LCHOV:
  nisbiy o'zgarish = ||f(siljit(x)) - f(x)|| / ||f(x)||
  0 - to'liq invariant, pooling siz qiymat bilan solishtiriladi

TO'LIQ INVARIANTLIK:
  faqat GLOBAL pooling (GAP/GMP) - ob'ekt xaritadan chiqmaguncha
  qo'shimcha: augmentatsiya (tasodifiy siljitish, 21-qism)

IKKI USUL O'LCHAMNI 2 BAROBAR KAMAYTIRADI:

                    MaxPool2d(2)         Conv2d(C, C, 3, stride=2, p=1)
  parametrlar       0                    C*C*9 + C
  o'rganadi         yo'q                 ha
  kanallarni        aralashtirmaydi      aralashtiradi
  qaysi arxitektura LeNet, VGG           ResNet (qisman), zamonaviy tarmoqlar

AMALDA:
  kichik vazifalarda farq ko'pincha shovqin ichida
  stride conv - ko'proq parametr, "qanday kichraytirishni" o'rganadi
  pooling - arzon, sodda, yaxshi boshlang'ich nuqta

QOIDA: bitta seed bilan emas, bir necha seed va SE bilan taqqoslang (18-qism)

HAR QATLAMDAN KEYIN SHAKLNI BILISH SHART:
  Conv2d(1, 16, 3, p=1):   (N, 1, 8, 8)   -> (N, 16, 8, 8)
  MaxPool2d(2):                           -> (N, 16, 4, 4)
  Conv2d(16, 32, 3, p=1):                 -> (N, 32, 4, 4)
  MaxPool2d(2):                           -> (N, 32, 2, 2)
  AdaptiveAvgPool2d(1):                   -> (N, 32, 1, 1)
  Flatten:                                -> (N, 32)
  Linear(32, 10):                         -> (N, 10)

TEKSHIRISH USULI:
  for q in model: x = q(x); print(type(q).__name__, x.shape)
  - bitta kichik batch bilan, o'rgatishdan OLDIN

CHEKLOV: 2x2 pool har safar o'lchamni ikkiga bo'ladi
  8x8 rasmda ikki marta - 2x2; uchinchisi 1x1
  kichik rasmda pooling sonini cheklang

import torch
import torch.nn as nn


class KichikCNN(nn.Module):
    def __init__(self, n_sinf=10):
        super().__init__()
        self.xususiyat = nn.Sequential(
            nn.Conv2d(1, 16, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),
            nn.Conv2d(16, 32, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),
            nn.Conv2d(32, 64, 3, padding=1), nn.ReLU())
        self.gap = nn.AdaptiveAvgPool2d(1)           # istalgan o'lcham
        self.bosh = nn.Linear(64, n_sinf)

    def forward(self, x):
        return self.bosh(self.gap(self.xususiyat(x)).flatten(1))


def shakllar(model, x):
    for q in model.xususiyat:
        x = q(x)
        print(f"{type(q).__name__:<12} {tuple(x.shape)}")

MaxPool2d(2): (N, C, H, W) -> (N, C, H/2, W/2), parametr 0
max - "bormi", avg - "qancha"; gradient: max -> bitta joyga
GAP: AdaptiveAvgPool2d(1) == mean(dim=(2, 3)); o'lchamdan ozod
mahalliy pooling - qisman chidamli; global pooling - invariant
stride=2 conv - o'rganiladigan alternativa, ko'proq parametr

"""MaxPool va AvgPool: qo'lda, torch bilan, o'lcham va gradient."""

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F


def pool_qolda(x, k=2, s=2, turi="max"):
    h_ch = (x.shape[0] - k) // s + 1
    w_ch = (x.shape[1] - k) // s + 1
    y = np.zeros((h_ch, w_ch))
    for i in range(h_ch):
        for j in range(w_ch):
            oyna = x[i * s:i * s + k, j * s:j * s + k]
            y[i, j] = oyna.max() if turi == "max" else oyna.mean()
    return y


def jadval(m):
    for qator in m:
        print("    " + " ".join(f"{v:>5.2f}" for v in qator))


def main() -> None:
    x = np.array([[1, 3, 2, 0],
                  [5, 6, 1, 2],
                  [7, 2, 9, 4],
                  [0, 1, 3, 8]], dtype=np.float64)
    xt = torch.tensor(x).view(1, 1, 4, 4)

    print("=== 1. 2x2 oynalar, stride=2 ===")
    jadval(x)
    for turi in ["max", "avg"]:
        q = pool_qolda(x, turi=turi)
        t = (F.max_pool2d(xt, 2) if turi == "max"
             else F.avg_pool2d(xt, 2))[0, 0].numpy()
        print(f"  {turi}_pool:")
        jadval(q)
        print(f"    torch bilan teng: {np.array_equal(q, t)}")

    print("\n=== 2. Parametrlar yo'q, kanallar alohida ===")
    mp = nn.MaxPool2d(2)
    print(f"  {mp}")
    print(f"  parametrlar soni: {sum(p.numel() for p in mp.parameters())}")
    torch.manual_seed(0)
    z = torch.randn(4, 16, 32, 32)
    print(f"  (4, 16, 32, 32) -> {tuple(mp(z).shape)}  "
          f"(kanallar soni o'zgarmaydi)")
    alohida = torch.stack([F.max_pool2d(z[:, c:c + 1], 2)[:, 0]
                           for c in range(16)], dim=1)
    print(f"  har kanalni alohida pool qilish bilan teng: "
          f"{torch.equal(alohida, mp(z))}")

    print("\n=== 3. O'lchamlar: kernel, stride, padding, ceil_mode ===")
    print(f"  {'H':>3} {'k':>3} {'s':>3} {'p':>3} {'ceil':>5} "
          f"{'formula':>8} {'torch':>6}")
    for h, k, s, p, ceil in [(32, 2, 2, 0, False), (7, 2, 2, 0, False),
                             (7, 2, 2, 0, True), (32, 3, 2, 1, False),
                             (28, 3, 1, 1, False), (13, 3, 2, 0, False)]:
        kasr = (h + 2 * p - k) / s + 1
        f = int(np.ceil(kasr - 1) + 1) if ceil else int(np.floor(kasr - 1) + 1)
        t = F.max_pool2d(torch.zeros(1, 1, h, h), k, s, p,
                         ceil_mode=ceil).shape[-1]
        print(f"  {h:>3} {k:>3} {s:>3} {p:>3} {str(ceil):>5} {f:>8} {t:>6}")
    print("  formula konvolyutsiya bilan bir xil: floor((H + 2p - k)/s) + 1")

    print("\n=== 4. Gradient qayerga boradi ===")
    for turi in ["max", "avg"]:
        xg = torch.tensor(x).view(1, 1, 4, 4).requires_grad_()
        y = F.max_pool2d(xg, 2) if turi == "max" else F.avg_pool2d(xg, 2)
        y.sum().backward()
        print(f"  {turi}_pool gradienti:")
        jadval(xg.grad[0, 0].numpy())
    print("  max: gradient faqat maksimumga (1), qolganlari 0")
    print("  avg: gradient teng bo'linadi (1/4)")

    print("\n=== 5. Max va avg nimani saqlaydi ===")
    tekis = torch.zeros(1, 1, 8, 8)
    tekis[0, 0, 2, 5] = 1.0                    # bitta yorqin nuqta
    print("  8x8 da bitta yorqin nuqta (qiymat 1):")
    print(f"    max_pool 4x4 -> maks {F.max_pool2d(tekis, 4).max():.4f}")
    print(f"    avg_pool 4x4 -> maks {F.avg_pool2d(tekis, 4).max():.4f}")
    print("  max - 'bor-yo'qligini' saqlaydi, avg - 'qanchaligini' (xiralaydi)")
    print("  ⭐ Pooling - parametrsiz, kanal bo'yicha alohida, o'lchamni kichraytiradi")


if __name__ == "__main__":
    main()

=== 1. 2x2 oynalar, stride=2 ===
     1.00  3.00  2.00  0.00
     5.00  6.00  1.00  2.00
     7.00  2.00  9.00  4.00
     0.00  1.00  3.00  8.00
  max_pool:
     6.00  2.00
     7.00  9.00
    torch bilan teng: True
  avg_pool:
     3.75  1.25
     2.50  6.00
    torch bilan teng: True

=== 2. Parametrlar yo'q, kanallar alohida ===
  MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  parametrlar soni: 0
  (4, 16, 32, 32) -> (4, 16, 16, 16)  (kanallar soni o'zgarmaydi)
  har kanalni alohida pool qilish bilan teng: True

=== 3. O'lchamlar: kernel, stride, padding, ceil_mode ===
    H   k   s   p  ceil  formula  torch
   32   2   2   0 False       16     16
    7   2   2   0 False        3      3
    7   2   2   0  True        4      4
   32   3   2   1 False       16     16
   28   3   1   1 False       28     28
   13   3   2   0 False        6      6
  formula konvolyutsiya bilan bir xil: floor((H + 2p - k)/s) + 1

=== 4. Gradient qayerga boradi ===
  max_pool gradienti:
     0.00  0.00  1.00  0.00
     0.00  1.00  0.00  0.00
     1.00  0.00  1.00  0.00
     0.00  0.00  0.00  0.00
  avg_pool gradienti:
     0.25  0.25  0.25  0.25
     0.25  0.25  0.25  0.25
     0.25  0.25  0.25  0.25
     0.25  0.25  0.25  0.25
  max: gradient faqat maksimumga (1), qolganlari 0
  avg: gradient teng bo'linadi (1/4)

=== 5. Max va avg nimani saqlaydi ===
  8x8 da bitta yorqin nuqta (qiymat 1):
    max_pool 4x4 -> maks 1.0000
    avg_pool 4x4 -> maks 0.0625
  max - 'bor-yo'qligini' saqlaydi, avg - 'qanchaligini' (xiralaydi)
  ⭐ Pooling - parametrsiz, kanal bo'yicha alohida, o'lchamni kichraytiradi

"""Global average pooling: mean ga tengligi, istalgan o'lcham, bosh parametrlari."""

import torch
import torch.nn as nn


class FlattenBosh(nn.Module):
    """Xususiyat xaritasini yoyib, Linear ga beradi - o'lcham qat'iy."""

    def __init__(self, c, h, w, n_sinf=10):
        super().__init__()
        self.lin = nn.Linear(c * h * w, n_sinf)

    def forward(self, f):
        return self.lin(f.flatten(1))


class GAPBosh(nn.Module):
    """Har kanalni bitta songa o'rtachalaydi - o'lcham erkin."""

    def __init__(self, c, n_sinf=10):
        super().__init__()
        self.gap = nn.AdaptiveAvgPool2d(1)
        self.lin = nn.Linear(c, n_sinf)

    def forward(self, f):
        return self.lin(self.gap(f).flatten(1))


def main() -> None:
    torch.manual_seed(0)
    f = torch.randn(4, 64, 7, 7)

    print("=== 1. AdaptiveAvgPool2d(1) = mean(dim=(2, 3)) ===")
    gap = nn.AdaptiveAvgPool2d(1)(f)
    print(f"  (4, 64, 7, 7) -> {tuple(gap.shape)}")
    print(f"  mean bilan teng: "
          f"{torch.allclose(gap.flatten(1), f.mean(dim=(2, 3)))}")
    gmp = nn.AdaptiveMaxPool2d(1)(f)
    print(f"  AdaptiveMaxPool2d(1) = amax(dim=(2, 3)): "
          f"{torch.equal(gmp.flatten(1), f.amax(dim=(2, 3)))}")

    print("\n=== 2. Istalgan kirish o'lchami ===")
    xususiyat = nn.Sequential(nn.Conv2d(3, 16, 3, padding=1), nn.ReLU(),
                              nn.MaxPool2d(2),
                              nn.Conv2d(16, 32, 3, padding=1), nn.ReLU(),
                              nn.MaxPool2d(2))
    gap_bosh = GAPBosh(32)
    flat_bosh = FlattenBosh(32, 8, 8)          # 32x32 kirish uchun
    print(f"  {'kirish':>10} {'xususiyat':>16} {'GAP bosh':>10} "
          f"{'Flatten bosh':>14}")
    for h in [32, 48, 64, 33]:
        x = torch.randn(2, 3, h, h)
        fx = xususiyat(x)
        g = tuple(gap_bosh(fx).shape)
        try:
            fl = str(tuple(flat_bosh(fx).shape))
        except RuntimeError:
            fl = "RuntimeError"
        print(f"  {f'{h}x{h}':>10} {str(tuple(fx.shape[1:])):>16} "
              f"{str(g):>10} {fl:>14}")

    print("\n=== 3. Bosh parametrlari ===")
    print(f"  {'xususiyat xaritasi':<20} {'Flatten+Linear':>15} "
          f"{'GAP+Linear':>11}")
    for c, h in [(32, 8), (64, 7), (512, 7), (2048, 7)]:
        fl = sum(p.numel() for p in FlattenBosh(c, h, h, 1000).parameters())
        gp = sum(p.numel() for p in GAPBosh(c, 1000).parameters())
        print(f"  {f'{c}x{h}x{h}':<20} {fl:>15,} {gp:>11,}")
    print("  GAP bosh parametrlari H*W marta kam (bias dan tashqari)")

    print("\n=== 4. Adaptive pooling - chiqish o'lchami beriladi ===")
    for chiq in [(1, 1), (2, 2), (3, 3)]:
        for h in [7, 10, 13]:
            y = nn.AdaptiveAvgPool2d(chiq)(torch.randn(1, 8, h, h))
            print(f"  AdaptiveAvgPool2d({chiq}) {h}x{h} -> "
                  f"{tuple(y.shape[2:])}")

    print("\n=== 5. GAP joyni unutadi ===")
    a = torch.zeros(1, 1, 8, 8)
    b = torch.zeros(1, 1, 8, 8)
    a[0, 0, 1, 1] = 1.0            # chap-yuqori burchakda
    b[0, 0, 6, 5] = 1.0            # o'ng-pastda
    ga = nn.AdaptiveAvgPool2d(1)(a).item()
    gb = nn.AdaptiveAvgPool2d(1)(b).item()
    print(f"  nuqta (1, 1) da: GAP {ga:.4f};  nuqta (6, 5) da: GAP {gb:.4f}")
    print(f"  teng: {ga == gb} - 'nima bor' qoladi, 'qayerda' yo'qoladi")
    print("  ⭐ GAP: har kanal -> bitta son; o'lchamdan ozod, parametr kam")


if __name__ == "__main__":
    main()

=== 1. AdaptiveAvgPool2d(1) = mean(dim=(2, 3)) ===
  (4, 64, 7, 7) -> (4, 64, 1, 1)
  mean bilan teng: True
  AdaptiveMaxPool2d(1) = amax(dim=(2, 3)): True

=== 2. Istalgan kirish o'lchami ===
      kirish        xususiyat   GAP bosh   Flatten bosh
       32x32       (32, 8, 8)    (2, 10)        (2, 10)
       48x48     (32, 12, 12)    (2, 10)   RuntimeError
       64x64     (32, 16, 16)    (2, 10)   RuntimeError
       33x33       (32, 8, 8)    (2, 10)        (2, 10)

=== 3. Bosh parametrlari ===
  xususiyat xaritasi    Flatten+Linear  GAP+Linear
  32x8x8                     2,049,000      33,000
  64x7x7                     3,137,000      65,000
  512x7x7                   25,089,000     513,000
  2048x7x7                 100,353,000   2,049,000
  GAP bosh parametrlari H*W marta kam (bias dan tashqari)

=== 4. Adaptive pooling - chiqish o'lchami beriladi ===
  AdaptiveAvgPool2d((1, 1)) 7x7 -> (1, 1)
  AdaptiveAvgPool2d((1, 1)) 10x10 -> (1, 1)
  AdaptiveAvgPool2d((1, 1)) 13x13 -> (1, 1)
  AdaptiveAvgPool2d((2, 2)) 7x7 -> (2, 2)
  AdaptiveAvgPool2d((2, 2)) 10x10 -> (2, 2)
  AdaptiveAvgPool2d((2, 2)) 13x13 -> (2, 2)
  AdaptiveAvgPool2d((3, 3)) 7x7 -> (3, 3)
  AdaptiveAvgPool2d((3, 3)) 10x10 -> (3, 3)
  AdaptiveAvgPool2d((3, 3)) 13x13 -> (3, 3)

=== 5. GAP joyni unutadi ===
  nuqta (1, 1) da: GAP 0.0156;  nuqta (6, 5) da: GAP 0.0156
  teng: True - 'nima bor' qoladi, 'qayerda' yo'qoladi
  ⭐ GAP: har kanal -> bitta son; o'lchamdan ozod, parametr kam

"""Siljishga kichik invariantlik: pooling dan oldin va keyin o'lchash."""

import torch
import torch.nn as nn
import torch.nn.functional as F
from sklearn.datasets import load_digits


def nisbiy_ozgarish(a, b):
    """Har rasm uchun ||a - b|| / ||a||, keyin o'rtacha."""
    a, b = a.flatten(1), b.flatten(1)
    return ((a - b).norm(dim=1) / a.norm(dim=1).clamp_min(1e-8)).mean().item()


def main() -> None:
    torch.manual_seed(0)
    d = load_digits()
    x = torch.tensor(d.images[:500], dtype=torch.float32).unsqueeze(1) / 16
    x = F.pad(x, (4, 4, 4, 4))                 # 16x16, raqam markazda
    print(f"  rasmlar: {tuple(x.shape)} (8x8 raqam 16x16 maydon markazida)")

    conv = nn.Conv2d(1, 8, 3, padding=1)
    with torch.no_grad():
        f0 = F.relu(conv(x))                   # xususiyat xaritalari

    bosqichlar = {
        "xarita (pooling yo'q)": lambda f: f,
        "MaxPool 2x2": lambda f: F.max_pool2d(f, 2),
        "AvgPool 2x2": lambda f: F.avg_pool2d(f, 2),
        "MaxPool 4x4": lambda f: F.max_pool2d(f, 4),
        "global max (GMP)": lambda f: f.amax(dim=(2, 3)),
        "global avg (GAP)": lambda f: f.mean(dim=(2, 3)),
    }
    siljishlar = [(0, 1), (1, 1), (0, 2), (2, 2), (3, 3)]

    print("\n=== 1. Nisbiy o'zgarish: ||f(siljit(x)) - f(x)|| / ||f(x)|| ===")
    sarlavha = " ".join(f"{f'({a},{b})':>7}" for a, b in siljishlar)
    print(f"  {'bosqich':<22} {sarlavha}")
    natija = {}
    with torch.no_grad():
        for nom, fn in bosqichlar.items():
            asl = fn(f0)
            qator = []
            for a, b in siljishlar:
                xs = torch.roll(x, shifts=(a, b), dims=(2, 3))
                qator.append(nisbiy_ozgarish(asl, fn(F.relu(conv(xs)))))
            natija[nom] = qator
            print(f"  {nom:<22} " + " ".join(f"{v:>7.3f}" for v in qator))

    print("\n=== 2. Pooling yo'q holatga nisbatan kamayish ===")
    xarita = natija["xarita (pooling yo'q)"]
    print(f"  {'bosqich':<22} {sarlavha}")
    for nom in list(bosqichlar)[1:]:
        kam = [1 - v / x0 for v, x0 in zip(natija[nom], xarita)]
        print(f"  {nom:<22} " + " ".join(f"{v:>7.0%}" for v in kam))
    mp2 = [1 - v / x0 for v, x0 in zip(natija["MaxPool 2x2"], xarita)]
    print(f"  MaxPool 2x2: 1 pikselda {mp2[0]:.0%}, 2 pikselda {mp2[2]:.0%}, "
          f"(3,3) da {mp2[4]:.0%}")
    if mp2[0] > mp2[2]:
        print("  siljish kattalashgan sari mahalliy pooling foydasi yo'qoladi")
    if max(natija["global avg (GAP)"]) < 1e-5:
        print("  GAP hamma siljishda o'zgarmas (raqam chetdan chiqmaguncha)")

    print("\n=== 3. Siljish grid bilan mos kelsa ===")
    with torch.no_grad():
        p0 = F.max_pool2d(f0, 2)
        xs = torch.roll(x, shifts=(2, 2), dims=(2, 3))
        p2 = F.max_pool2d(F.relu(conv(xs)), 2)
        p2_qaytarilgan = torch.roll(p2, shifts=(-1, -1), dims=(2, 3))
    print(f"  (2,2) siljish, MaxPool 2x2 chiqishini (1,1) orqaga surilsa: "
          f"nisbiy farq {nisbiy_ozgarish(p0, p2_qaytarilgan):.1e}")
    print("  juft siljish - chiqish ham aniq siljiydi (ekvivariant),")
    print(f"  lekin joyma-joy taqqoslasa o'zgarish "
          f"{natija['MaxPool 2x2'][3]:.3f} (invariant emas)")
    print("  ⭐ Pooling kichik siljishga qisman chidamli; to'liq - faqat global pooling")


if __name__ == "__main__":
    main()

  rasmlar: (500, 1, 16, 16) (8x8 raqam 16x16 maydon markazida)

=== 1. Nisbiy o'zgarish: ||f(siljit(x)) - f(x)|| / ||f(x)|| ===
  bosqich                  (0,1)   (1,1)   (0,2)   (2,2)   (3,3)
  xarita (pooling yo'q)    0.363   0.462   0.490   0.537   0.569
  MaxPool 2x2              0.317   0.401   0.473   0.532   0.587
  AvgPool 2x2              0.225   0.302   0.358   0.416   0.477
  MaxPool 4x4              0.238   0.343   0.367   0.476   0.589
  global max (GMP)         0.000   0.000   0.000   0.000   0.000
  global avg (GAP)         0.000   0.000   0.000   0.000   0.000

=== 2. Pooling yo'q holatga nisbatan kamayish ===
  bosqich                  (0,1)   (1,1)   (0,2)   (2,2)   (3,3)
  MaxPool 2x2                13%     13%      3%      1%     -3%
  AvgPool 2x2                38%     35%     27%     23%     16%
  MaxPool 4x4                34%     26%     25%     11%     -3%
  global max (GMP)          100%    100%    100%    100%    100%
  global avg (GAP)          100%    100%    100%    100%    100%
  MaxPool 2x2: 1 pikselda 13%, 2 pikselda 3%, (3,3) da -3%
  siljish kattalashgan sari mahalliy pooling foydasi yo'qoladi
  GAP hamma siljishda o'zgarmas (raqam chetdan chiqmaguncha)

=== 3. Siljish grid bilan mos kelsa ===
  (2,2) siljish, MaxPool 2x2 chiqishini (1,1) orqaga surilsa: nisbiy farq 0.0e+00
  juft siljish - chiqish ham aniq siljiydi (ekvivariant),
  lekin joyma-joy taqqoslasa o'zgarish 0.532 (invariant emas)
  ⭐ Pooling kichik siljishga qisman chidamli; to'liq - faqat global pooling

"""O'lchamlar zanjiri va MaxPool bilan stride=2 konvolyutsiyani taqqoslash."""

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split


def pool_tarmoq():
    return nn.Sequential(
        nn.Conv2d(1, 16, 3, padding=1), nn.ReLU(),
        nn.MaxPool2d(2),                                  # 8 -> 4
        nn.Conv2d(16, 32, 3, padding=1), nn.ReLU(),
        nn.MaxPool2d(2),                                  # 4 -> 2
        nn.Conv2d(32, 32, 3, padding=1), nn.ReLU(),
        nn.AdaptiveAvgPool2d(1), nn.Flatten(), nn.Linear(32, 10))


def stride_tarmoq():
    return nn.Sequential(
        nn.Conv2d(1, 16, 3, padding=1), nn.ReLU(),
        nn.Conv2d(16, 16, 3, stride=2, padding=1), nn.ReLU(),   # 8 -> 4
        nn.Conv2d(16, 32, 3, padding=1), nn.ReLU(),
        nn.Conv2d(32, 32, 3, stride=2, padding=1), nn.ReLU(),   # 4 -> 2
        nn.Conv2d(32, 32, 3, padding=1), nn.ReLU(),
        nn.AdaptiveAvgPool2d(1), nn.Flatten(), nn.Linear(32, 10))


def qisqa(q):
    if isinstance(q, nn.Conv2d):
        return (f"Conv2d({q.in_channels}, {q.out_channels}, "
                f"k={q.kernel_size[0]}, s={q.stride[0]})")
    if isinstance(q, nn.MaxPool2d):
        return f"MaxPool2d({q.kernel_size})"
    if isinstance(q, nn.Linear):
        return f"Linear({q.in_features}, {q.out_features})"
    return type(q).__name__


def zanjir(model, x):
    print(f"  {'qatlam':<26} {'chiqish shakli':>16} {'param':>7}")
    print(f"  {'kirish':<26} {str(tuple(x.shape)):>16} {'':>7}")
    for q in model:
        x = q(x)
        if isinstance(q, nn.ReLU):
            continue
        p = sum(t.numel() for t in q.parameters())
        print(f"  {qisqa(q):<26} {str(tuple(x.shape)):>16} {p:>7}")


def tayyorla():
    d = load_digits()
    x = torch.tensor(d.images, dtype=torch.float32).unsqueeze(1) / 16
    y = torch.tensor(d.target)
    idx_tr, idx_te = train_test_split(np.arange(len(y)), test_size=0.4,
                                      stratify=d.target, random_state=0)
    return x[idx_tr], y[idx_tr], x[idx_te], y[idx_te]


def orgat(model, x, y, davrlar=30, seed=0):
    opt = torch.optim.Adam(model.parameters(), lr=0.003)
    g = torch.Generator().manual_seed(seed)
    for _ in range(davrlar):
        model.train()
        tartib = torch.randperm(len(y), generator=g)
        for i in range(0, len(y), 64):
            idx = tartib[i:i + 64]
            opt.zero_grad()
            F.cross_entropy(model(x[idx]), y[idx]).backward()
            opt.step()
    return model


def aniqlik(model, x, y):
    model.eval()
    with torch.no_grad():
        return (model(x).argmax(1) == y).float().mean().item()


def main() -> None:
    x_tr, y_tr, x_te, y_te = tayyorla()
    print(f"  load_digits 8x8: o'quv {tuple(x_tr.shape)}, "
          f"test {len(y_te)}")

    print("\n=== 1. O'lchamlar zanjiri: MaxPool bilan ===")
    torch.manual_seed(0)
    zanjir(pool_tarmoq(), x_tr[:8])

    print("\n=== 2. O'lchamlar zanjiri: stride=2 konvolyutsiya bilan ===")
    zanjir(stride_tarmoq(), x_tr[:8])

    print("\n=== 3. O'rgatish (3 seed, 30 davr) ===")
    print(f"  {'seed':>5} {'MaxPool':>9} {'stride=2':>9}")
    a, b = [], []
    for seed in range(3):
        torch.manual_seed(seed)
        m1 = orgat(pool_tarmoq(), x_tr, y_tr, seed=seed)
        torch.manual_seed(seed)
        m2 = orgat(stride_tarmoq(), x_tr, y_tr, seed=seed)
        a.append(aniqlik(m1, x_te, y_te))
        b.append(aniqlik(m2, x_te, y_te))
        print(f"  {seed:>5} {a[-1]:>9.4f} {b[-1]:>9.4f}")
    a, b = np.array(a), np.array(b)
    p1 = sum(p.numel() for p in pool_tarmoq().parameters())
    p2 = sum(p.numel() for p in stride_tarmoq().parameters())
    print(f"  o'rtacha: MaxPool {a.mean():.4f}, stride=2 {b.mean():.4f}")
    print(f"  parametrlar: MaxPool {p1}, stride=2 {p2} (+{p2 - p1})")
    farq = b - a
    se = farq.std(ddof=1) / np.sqrt(len(farq))
    print(f"  farq (stride - MaxPool): {farq.mean():+.4f}, SE {se:.4f}")
    if abs(farq.mean()) <= 2 * se or abs(farq.mean()) < 0.005:
        print("  QAROR: farq shovqin ichida - bu vazifada ikkalasi teng")
    elif farq.mean() > 0:
        print("  QAROR: stride=2 konvolyutsiya bu vazifada yaxshiroq")
    else:
        print("  QAROR: MaxPool bu vazifada yaxshiroq")
    print("  ⭐ Pooling - bepul kichraytirish; stride=2 conv - o'rganiladigan")


if __name__ == "__main__":
    main()

  load_digits 8x8: o'quv (1078, 1, 8, 8), test 719

=== 1. O'lchamlar zanjiri: MaxPool bilan ===
  qatlam                       chiqish shakli   param
  kirish                         (8, 1, 8, 8)
  Conv2d(1, 16, k=3, s=1)       (8, 16, 8, 8)     160
  MaxPool2d(2)                  (8, 16, 4, 4)       0
  Conv2d(16, 32, k=3, s=1)      (8, 32, 4, 4)    4640
  MaxPool2d(2)                  (8, 32, 2, 2)       0
  Conv2d(32, 32, k=3, s=1)      (8, 32, 2, 2)    9248
  AdaptiveAvgPool2d             (8, 32, 1, 1)       0
  Flatten                             (8, 32)       0
  Linear(32, 10)                      (8, 10)     330

=== 2. O'lchamlar zanjiri: stride=2 konvolyutsiya bilan ===
  qatlam                       chiqish shakli   param
  kirish                         (8, 1, 8, 8)
  Conv2d(1, 16, k=3, s=1)       (8, 16, 8, 8)     160
  Conv2d(16, 16, k=3, s=2)      (8, 16, 4, 4)    2320
  Conv2d(16, 32, k=3, s=1)      (8, 32, 4, 4)    4640
  Conv2d(32, 32, k=3, s=2)      (8, 32, 2, 2)    9248
  Conv2d(32, 32, k=3, s=1)      (8, 32, 2, 2)    9248
  AdaptiveAvgPool2d             (8, 32, 1, 1)       0
  Flatten                             (8, 32)       0
  Linear(32, 10)                      (8, 10)     330

=== 3. O'rgatish (3 seed, 30 davr) ===
   seed   MaxPool  stride=2
      0    0.9722    0.9680
      1    0.9666    0.9722
      2    0.9750    0.9694
  o'rtacha: MaxPool 0.9713, stride=2 0.9699
  parametrlar: MaxPool 14378, stride=2 25946 (+11568)
  farq (stride - MaxPool): -0.0014, SE 0.0035
  QAROR: farq shovqin ichida - bu vazifada ikkalasi teng
  ⭐ Pooling - bepul kichraytirish; stride=2 conv - o'rganiladigan

nn.Sequential(..., nn.Flatten(), nn.Linear(32 * 8 * 8, 10))       # ⚠️ faqat 32x32
nn.Sequential(..., nn.AdaptiveAvgPool2d(1), nn.Flatten(), nn.Linear(32, 10))  # ✅

nn.MaxPool2d(2)(torch.rand(1, 8, 7, 7))       # -> 3x3, 7-qator tashlandi  # ⚠️
nn.MaxPool2d(2, ceil_mode=True)               # -> 4x4                    # ✅

# 8x8 kirish, 4 ta MaxPool2d(2): 8 -> 4 -> 2 -> 1 -> RuntimeError   # ⚠️
# 8x8 kirish: 2 ta pooling, keyin global pooling                     # ✅

model = cnn_maxpool_bilan    # "siljishga chidamli bo'ladi"          # ⚠️
# augmentatsiya (tasodifiy siljitish) + global pooling               # ✅

bbox = nn.Linear(64, 4)(gap(xarita).flatten(1))   # joy yo'qolgan    # ⚠️
# joy kerak bo'lsa - xaritani fazoviy saqlang (aniqlash mavzusida)   # ✅

f.mean(dim=(1, 2))              # kanallar ustidan - xato             # ⚠️
f.mean(dim=(2, 3))              # H, W ustidan - GAP                  # ✅

# "stride conv 0.3% yaxshi chiqdi" - bitta o'rgatish                 # ⚠️
# 3+ seed, juftlashgan farq va SE (18-qism)                          # ✅

1.  # MaxPool2d(2) parametrlari soni?
2.  # (4, 16, 32, 32) MaxPool2d(2) dan keyin?
3.  # MaxPool2d(2) 7x7 dan?
4.  # MaxPool2d(2, ceil_mode=True) 7x7 dan?
5.  # MaxPool2d(3, 2, padding=1) 32x32 dan?
6.  # [[1, 3], [5, 6]] max va avg?
7.  # MaxPool gradienti oynada qayerga boradi?
8.  # AdaptiveAvgPool2d(1) nimaga teng?
9.  # 512x7x7 dan 1000 sinf: Flatten+Linear parametrlari?
10. # xuddi shu, GAP+Linear?
11. # qaysi pooling to'liq siljish invariant?
12. # MaxPool2d(2) va Conv2d(C, C, 3, stride=2, padding=1) chiqish o'lchami?

1.  nn.Sequential(conv_qismi, nn.Flatten(), nn.Linear(32 * 8 * 8, 10))   # har xil o'lcham

2.  gap = f.mean(dim=(1, 2))

3.  # 8x8 kirish: [Conv, MaxPool2d(2)] x 4

4.  nn.MaxPool2d(2)(torch.rand(1, 8, 7, 7))   # 7-qator ham kerak

5.  bbox = nn.Linear(64, 4)(f.mean(dim=(2, 3)))   # ob'ekt joyi