IlmHamroh
Python kursi/ML va AI9/12-dars16 daqiqa
Mundarija (22)

25.9-dars: Modelni o'qitish

25-QISM — ML VA AI · 9-dars

1. Kirish va motivatsiya

25.8 da neyron tarmoq qurdik — lekin uning vaznlari tasodifiy (o'rganmagan). Model foydali bo'lishi uchun ma'lumotda o'qitish kerak: bashorat qiladi → xatosini o'lchaydi → vaznlarni yaxshilaydi → takrorlaydi. Bu — modelni o'qitish (training). scikit-learn'da fit bir buyruq edi 25.2-bob, PyTorch'da o'qitish siklli ni o'zimiz yozamiz (to'liq nazorat).

Modelni o'qitish — takrorlanuvchi jarayon (gradient descent): 1) bashorat (model(X)), 2) yo'qotish (loss — bashorat vs haqiqiy xato), 3) gradient (loss.backward() — 25.7), 4) yangilash (optimizer.step() — vaznlarni yaxshila), takror. Yo'qotish funksiyasi (MSE — regressiya, CrossEntropy — klassifikatsiya) xatoni o'lchaydi; optimizator (SGD, Adam) vaznlarni gradient bo'yicha yangilaydi. Har takror (epoch) da loss kamayadi — model o'rganadi. Bu chuqur o'qitishning yuragi.

Real vaziyat. Bir neyron tarmoq boshda tasodifiy edi (loss 165). O'qitish siklli yozildi: 200 epoch, har birida bashorat → loss → gradient → yangilash. Loss 165'dan 0.001'ga tushdi — model naqshni (y = 2x + 1) o'rgandi. Vaznlar tasodifiydan to'g'riga (w≈2, b≈1). O'qitish — model qanday "o'rganishi". Loss'ning tushishi — o'rganish belgisi.

Bu darsda modelni o'qitishni o'rganamiz.

Bu darsda:

  • O'qitish siklli (training loop)
  • Yo'qotish funksiyasi (loss)
  • Optimizator (SGD, Adam)
  • Epoch va gradient descent
  • Train/test o'qitish
  • O'rganish sur'ati (learning rate)
  • Klassifikatsiya o'qitish
  • Amaliy: to'liq o'qitish

ℹ Misollarda PyTorch (torch.manual_seed(0)) bilan sinaladi.

2. Nazariya — chuqur tushuntirish

2.1. O'qitish siklli

Takrorlanuvchi to'rt qadam:

python
for epoch in range(200):
    optimizer.zero_grad()        # 1. gradientni tozala
    pred = model(X)              # 2. bashorat
    loss = loss_fn(pred, y)      # 3. yo'qotish (xato)
    loss.backward()              # 4. gradient
    optimizer.step()             # 5. vaznlarni yangila

O'qitish siklli — beshta qadam, ko'p marta takror: zero_grad (gradientni tozala — 25.7 yig'iladi), bashorat (model(X)), loss (yo'qotish — xato), backward (gradient — 25.7), step (vaznlarni yangila). Har takror (epoch) da model biroz yaxshilanadi (loss kamayadi). Bu gradient descent — xatoni kamaytirish yo'nalishida vaznlarni o'zgartirish. scikit-learn fit shu siklli yashiradi; PyTorch — ochiq (nazorat).

2.2. Yo'qotish funksiyasi (loss)

Model xatosini o'lchash:

python
import torch.nn as nn
# regressiya (son):
loss_fn = nn.MSELoss()           # o'rtacha kvadratik xato
# klassifikatsiya (toifa):
loss_fn = nn.CrossEntropyLoss()  # sinf xatosi

Yo'qotish funksiyasi (loss function) — model bashorati haqiqiydan qancha chetda ekanini o'lchaydi (bir son — kichik = yaxshi): MSELoss (regressiya — o'rtacha kvadratik xato, 25.4 MAE kabi), CrossEntropyLoss (klassifikatsiya — sinf xatosi). O'qitishning maqsadi — loss'ni kamaytirish (0 ga yaqinlashtirish). Loss — model qancha "adashgan" ni ko'rsatadi; gradient loss'ni kamaytirish yo'nalishi. To'g'ri loss (muammoga qarab) muhim.

2.3. Optimizator (SGD, Adam)

Vaznlarni yangilovchi:

python
import torch.optim as optim
optimizer = optim.SGD(model.parameters(), lr=0.01)   # oddiy
optimizer = optim.Adam(model.parameters(), lr=0.01)  # kuchliroq
optimizer.step()     # gradient bo'yicha vaznlarni yangila

Optimizator — vaznlarni gradient bo'yicha yangilovchi (loss'ni kamaytirish): SGD (Stochastic Gradient Descent — oddiy: vazn = vazn - lr × gradient), Adam (adaptiv — ko'pincha tezroq, barqaror — zamonaviy standart). optimizer.step() — bir qadam (vaznlarni yangila), zero_grad() (gradientni tozala). lr (learning rate — qadam kattaligi). Optimizator — "qanday yangilash" (gradient — "qaysi yo'nalish"). Adam — yaxshi boshlangich tanlov.

2.4. Epoch va gradient descent

O'rganish takrori:

Epoch: butun ma'lumot bo'yicha bir o'tish
   epoch 1: loss 165 → epoch 100: loss 0.5 → epoch 200: loss 0.001
gradient descent: loss'ni kamaytirish (tepalikdan pastga)

Epoch — butun ma'lumot bo'yicha bir o'tish (bashorat + yangilash). Ko'p epoch — model ko'p marta o'rganadi (loss kamayadi). Gradient descent — loss'ni kamaytirish (tog' cho'qqisidan pastga — har qadam eng tik pastga): gradient yo'nalishni, lr qadamni belgilaydi. Loss egri chizig'i (epoch bo'yicha) — o'rganish belgisi (tushib borsa — yaxshi). Juda ko'p epoch — overfitting 25.2-bob. Epoch soni — muvozanat.

2.5. Train/test o'qitish

To'g'ri o'qitish:

python
# train'da o'qit:
for epoch in range(200):
    ...train siklli...
# test'da bahola (o'qitishda emas):
with torch.no_grad():
    test_loss = loss_fn(model(X_test), y_test)

Train/test (25.2 kabi): model train'da o'qitiladi (sikl), test'da baholanadi (ko'rmagan ma'lumot — halol). Test'da torch.no_grad() (gradient kerak emas — faqat bashorat, tez). Muhim: overfitting 25.2-bob — train loss tushsa ham, test loss oshsa (yodlash). Train va test loss kuzatiladi (yaqin bo'lsin). Bu 25.2 (scikit-learn train/test) ning PyTorch versiyasi. O'qitish halol baholash bilan.

2.6. O'rganish sur'ati (learning rate)

Qadam kattaligi:

python
optim.SGD(model.parameters(), lr=0.01)   # o'rta
# lr juda katta: sakraydi (loss oshadi/tebranadi)
# lr juda kichik: sekin o'rganadi

O'rganish sur'ati (learning rate, lr) — har qadamda vazn qancha o'zgaradi: juda katta (sakraydi — minimumdan o'tib ketadi, loss tebranadi yoki oshadi), juda kichik (sekin — ko'p epoch kerak), to'g'ri (barqaror tushadi). Eng muhim giperparametr — sinab topiladi (0.001, 0.01, 0.1). Adam lrga kamroq sezgir (adaptiv). Loss tebransa — lr kichraytir. "To'g'ri qadam — barqaror o'rganish".

2.7. Klassifikatsiya o'qitish

Toifa bashorat o'qitish:

python
model = nn.Sequential(nn.Linear(4, 16), nn.ReLU(), nn.Linear(16, 3))
loss_fn = nn.CrossEntropyLoss()      # klassifikatsiya
optimizer = optim.Adam(model.parameters(), lr=0.05)
# X: float32, y: long (sinf indeksi)

Klassifikatsiya o'qitish (regressiya kabi, farqlar): CrossEntropyLoss (loss), chiqish qatlami sinf soni (3 sinf → Linear(_, 3)), y — long tur (sinf indeksi 0/1/2). CrossEntropy softmax'ni ichida qiladi (chiqishga softmax qo'shmaydi — logits). Baholash: pred.argmax(1) (eng yuqori ehtimolli sinf) vs haqiqiy. O'qitish siklli bir xil (loss, gradient, step). Muammoga qarab loss va chiqish.

2.8. O'qitish — gradient descent

Modelni o'qitish — gradient descent (gradient bo'yicha pasayish): loss (xato) — "balandlik", gradient — "eng tik pastga yo'nalish", lr — qadam. Har epoch model pastga (kam loss) qadam tashlaydi — asta-sekin minimumga (yaxshi model). Bu barcha chuqur o'qitish (GPT ham) ning asosi — faqat ko'proq parametr, ma'lumot, epoch. PyTorch siklli ochiq (nazorat), scikit-learn fit yashirgan. Muhim: loss kuzat (tushsa — o'rganadi), overfitting (train vs test), lr (qadam). "Model xatoni kamaytirib o'rganadi".

3. Tez ma'lumotnoma

python
import torch
import torch.nn as nn
import torch.optim as optim

# komponentlar:
model = nn.Sequential(...)
loss_fn = nn.MSELoss()               # regressiya
loss_fn = nn.CrossEntropyLoss()      # klassifikatsiya
optimizer = optim.Adam(model.parameters(), lr=0.01)

# o'qitish sikli:
for epoch in range(200):
    optimizer.zero_grad()            # 1. gradient tozala
    pred = model(X)                  # 2. bashorat
    loss = loss_fn(pred, y)          # 3. xato
    loss.backward()                  # 4. gradient
    optimizer.step()                 # 5. yangila

# test (baholash):
with torch.no_grad():
    test_loss = loss_fn(model(X_test), y_test)

# klassifikatsiya baholash:
acc = (model(X).argmax(1) == y).float().mean()

O'qitish xulosasi

Sikl: zero_grad → bashorat → loss → backward → step (takror)
Loss: MSE (regressiya) / CrossEntropy (klassifikatsiya) · optimizator: SGD/Adam
Epoch: bir o'tish · lr: qadam · gradient descent (loss kamayadi)

4. Batafsil misollar

Misollarda PyTorch (torch.manual_seed(0)) bilan sinaladi.

Misol 1 — O'qitish siklli (regressiya)

python
"""o'qitish siklli: zero_grad → bashorat → loss → backward → step; loss kamayadi."""

import warnings
warnings.filterwarnings("ignore")

import torch
import torch.nn as nn


def main() -> None:
    torch.manual_seed(0)
    # ma'lumot: y = 2x + 1
    X = torch.arange(1, 11, dtype=torch.float32).reshape(-1, 1)
    y = 2 * X + 1

    print("=== 1. Model va komponentlar ===")
    model = nn.Linear(1, 1)
    loss_fn = nn.MSELoss()
    optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
    print("  model: Linear(1,1), loss: MSE, optim: SGD")

    print("\n=== 2. O'qitish siklli (200 epoch) ===")
    losses = []
    for epoch in range(200):
        optimizer.zero_grad()
        pred = model(X)
        loss = loss_fn(pred, y)
        loss.backward()
        optimizer.step()
        losses.append(float(loss))

    print(f"  boshlang'ich loss: {round(losses[0], 2)}")
    print(f"  oxirgi loss: {round(losses[-1], 4)}")

    print("\n=== 3. O'rganilgan parametrlar ===")
    print(f"  w = {round(float(model.weight), 3)} (haqiqiy: 2)")
    print(f"  b = {round(float(model.bias), 3)} (haqiqiy: 1)")

    print("\n=== 4. Loss kamaydi (o'rganish) ===")
    print(f"  loss tushdi: {losses[0] > losses[100] > losses[-1]}")
    print("  ⭐ o'qitish siklli — loss kamayadi, model o'rganadi")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Model va komponentlar ===
  model: Linear(1,1), loss: MSE, optim: SGD

=== 2. O'qitish siklli (200 epoch) ===
  boshlang'ich loss: 165.61
  oxirgi loss: 0.0012

=== 3. O'rganilgan parametrlar ===
  w = 2.011 (haqiqiy: 2)
  b = 0.926 (haqiqiy: 1)

=== 4. Loss kamaydi (o'rganish) ===
  loss tushdi: True
  ⭐ o'qitish siklli — loss kamayadi, model o'rganadi

Nima ko'rsatdi: 2.1, 2.2, 2.3-bo'limlar.

Misol 2 — Bashorat va gradient descent

python
"""o'qitilgan model bashorat; gradient descent (loss egri chizig'i — kamayadi)."""

import warnings
warnings.filterwarnings("ignore")

import torch
import torch.nn as nn


def main() -> None:
    torch.manual_seed(0)
    X = torch.arange(1, 11, dtype=torch.float32).reshape(-1, 1)
    y = 2 * X + 1
    model = nn.Linear(1, 1)
    loss_fn = nn.MSELoss()
    optimizer = torch.optim.SGD(model.parameters(), lr=0.01)

    losses = []
    for epoch in range(200):
        optimizer.zero_grad()
        loss = loss_fn(model(X), y)
        loss.backward()
        optimizer.step()
        losses.append(float(loss))

    print("=== 1. Loss egri chizig'i (gradient descent) ===")
    print(f"  epoch 0: {round(losses[0], 2)}")
    print(f"  epoch 50: {round(losses[50], 3)}")
    print(f"  epoch 100: {round(losses[100], 4)}")
    print(f"  epoch 199: {round(losses[199], 5)}")

    print("\n=== 2. Bashorat (o'qitilgan model) ===")
    with torch.no_grad():
        for x_val in [11.0, 15.0]:
            b = float(model(torch.tensor([[x_val]])))
            haqiqiy = 2 * x_val + 1
            print(f"  x={x_val}: bashorat {round(b, 2)} (haqiqiy: {haqiqiy})")

    print("\n=== 3. Model naqshni o'rgandi ===")
    print(f"  y = {round(float(model.weight), 2)}x + {round(float(model.bias), 2)}")

    print("\n=== 4. Gradient descent ===")
    print("  loss (xato) → pastga (kamayadi)")
    print("  har epoch model yaxshilanadi")
    print("  ⭐ gradient descent — loss'ni kamaytirib o'rganish")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Loss egri chizig'i (gradient descent) ===
  epoch 0: 165.61
  epoch 50: 0.004
  epoch 100: 0.0027
  epoch 199: 0.00118

=== 2. Bashorat (o'qitilgan model) ===
  x=11.0: bashorat 23.04 (haqiqiy: 23.0)
  x=15.0: bashorat 31.09 (haqiqiy: 31.0)

=== 3. Model naqshni o'rgandi ===
  y = 2.01x + 0.93

=== 4. Gradient descent ===
  loss (xato) → pastga (kamayadi)
  har epoch model yaxshilanadi
  ⭐ gradient descent — loss'ni kamaytirib o'rganish

Nima ko'rsatdi: 2.4, 2.6-bo'limlar.

Misol 3 — Klassifikatsiya o'qitish

python
"""klassifikatsiya: CrossEntropyLoss, Adam; iris ma'lumoti; aniqlik (argmax)."""

import warnings
warnings.filterwarnings("ignore")

import torch
import torch.nn as nn
from sklearn.datasets import load_iris


def main() -> None:
    torch.manual_seed(0)
    X_np, y_np = load_iris(return_X_y=True)
    X = torch.tensor(X_np, dtype=torch.float32)
    y = torch.tensor(y_np, dtype=torch.long)     # sinf indeksi

    print("=== 1. Klassifikatsiya modeli ===")
    model = nn.Sequential(nn.Linear(4, 16), nn.ReLU(), nn.Linear(16, 3))
    loss_fn = nn.CrossEntropyLoss()
    optimizer = torch.optim.Adam(model.parameters(), lr=0.05)
    print("  kirish 4 → 16 → chiqish 3 (sinf)")

    print("\n=== 2. O'qitish (100 epoch) ===")
    for epoch in range(100):
        optimizer.zero_grad()
        loss = loss_fn(model(X), y)
        loss.backward()
        optimizer.step()
    print(f"  oxirgi loss: {round(float(loss), 4)}")

    print("\n=== 3. Aniqlik (argmax) ===")
    with torch.no_grad():
        pred = model(X).argmax(1)      # eng yuqori ehtimolli sinf
        acc = (pred == y).float().mean()
    print(f"  aniqlik: {round(float(acc), 3)}")

    print("\n=== 4. Regressiya vs klassifikatsiya ===")
    print("  regressiya: MSELoss, son chiqish")
    print("  klassifikatsiya: CrossEntropyLoss, sinf chiqish")
    print("  ⭐ klassifikatsiya o'qitish — CrossEntropy, argmax")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Klassifikatsiya modeli ===
  kirish 4 → 16 → chiqish 3 (sinf)

=== 2. O'qitish (100 epoch) ===
  oxirgi loss: 0.0492

=== 3. Aniqlik (argmax) ===
  aniqlik: 0.98

=== 4. Regressiya vs klassifikatsiya ===
  regressiya: MSELoss, son chiqish
  klassifikatsiya: CrossEntropyLoss, sinf chiqish
  ⭐ klassifikatsiya o'qitish — CrossEntropy, argmax

Nima ko'rsatdi: 2.7-bo'lim.

Misol 4 — Amaliy: to'liq o'qitish (train/test)

Real chuqur o'qitish: train'da o'qitish, test'da baholash — halol jarayon. Bu — to'liq o'qitish sikllining namunasi.

python
"""to'liq o'qitish: train/test bo'lish, o'qitish siklli, test baholash (overfitting kuzat)."""

import warnings
warnings.filterwarnings("ignore")

import torch
import torch.nn as nn
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split


def main() -> None:
    torch.manual_seed(0)
    X_np, y_np = load_iris(return_X_y=True)
    Xtr, Xte, ytr, yte = train_test_split(X_np, y_np, test_size=0.3, random_state=42)
    Xtr = torch.tensor(Xtr, dtype=torch.float32)
    Xte = torch.tensor(Xte, dtype=torch.float32)
    ytr = torch.tensor(ytr, dtype=torch.long)
    yte = torch.tensor(yte, dtype=torch.long)

    print("=== 1. Model va o'qitish ===")
    model = nn.Sequential(nn.Linear(4, 16), nn.ReLU(), nn.Linear(16, 3))
    loss_fn = nn.CrossEntropyLoss()
    optimizer = torch.optim.Adam(model.parameters(), lr=0.05)

    for epoch in range(100):
        optimizer.zero_grad()
        loss = loss_fn(model(Xtr), ytr)     # TRAIN'da
        loss.backward()
        optimizer.step()
    print(f"  train loss: {round(float(loss), 4)}")

    print("\n=== 2. Train aniqligi ===")
    with torch.no_grad():
        train_acc = (model(Xtr).argmax(1) == ytr).float().mean()
    print(f"  train: {round(float(train_acc), 3)}")

    print("\n=== 3. Test aniqligi (ko'rmagan) ===")
    with torch.no_grad():
        test_acc = (model(Xte).argmax(1) == yte).float().mean()
    print(f"  test: {round(float(test_acc), 3)}")

    print("\n=== 4. Overfitting tekshiruvi ===")
    print(f"  train {round(float(train_acc), 3)} vs test {round(float(test_acc), 3)}")
    print(f"  yaqin → yaxshi umumlashtirgan")
    print("  ⭐ to'liq o'qitish — train o'qit, test bahola")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. Model va o'qitish ===
  train loss: 0.061

=== 2. Train aniqligi ===
  train: 0.981

=== 3. Test aniqligi (ko'rmagan) ===
  test: 1.0

=== 4. Overfitting tekshiruvi ===
  train 0.981 vs test 1.0
  yaqin → yaxshi umumlashtirgan
  ⭐ to'liq o'qitish — train o'qit, test bahola

Nima ko'rsatdi: 2.1–2.8-bo'limlar.

5. To'g'ri va noto'g'ri tushunishlar

Noto'g'ri fikr To'g'risi
"fit bir buyruq (PyTorch)" Siklli o'zimiz yozamiz
"zero_grad keraksiz" Gradient yig'iladi (tozala)
"Bir loss har muammoga" MSE/CrossEntropy (turi)
"lr muhim emas" Eng muhim giperparametr
"Ko'p epoch doim yaxshi" Overfitting
"Train loss yetarli" Test loss ham (overfitting)
"SGD = Adam" Adam adaptiv (ko'pincha yaxshi)
"Loss oshsa — normal" Muammo (lr, xato)

6. Keng tarqalgan xatolar va yechimlari

1. zero_grad unutish

python
loss.backward(); optimizer.step()   # ⚠️ gradient yig'iladi
optimizer.zero_grad(); ...           # ✅ avval tozala

2. Noto'g'ri loss (muammoga)

python
nn.MSELoss()   # klassifikatsiyada                 # ⚠️
nn.CrossEntropyLoss()   # klassifikatsiya           # ✅

3. lr juda katta (loss tebranadi)

python
optim.SGD(..., lr=10)   # sakraydi                  # ⚠️
optim.SGD(..., lr=0.01)   # barqaror                 # ✅

4. Test'da gradient (sekin)

python
model(X_test)   # gradient kuzatiladi               # ⚠️
with torch.no_grad(): model(X_test)                  # ✅

5. y tur noto'g'ri (klassifikatsiya)

python
y = torch.tensor([0, 1, 2], dtype=torch.float)   # ⚠️
y = torch.tensor([0, 1, 2], dtype=torch.long)     # ✅ (sinf)

6. Overfittingni sezmaslik

python
# faqat train loss                                  # ⚠️
# train vs test loss taqqosla                        # ✅

7. Juda kam/ko'p epoch

python
range(5)   # kam (o'rganmaydi)                       # ⚠️
range(100000)   # ko'p (overfitting, sekin)          # ⚠️
# loss kuzat (barqarorlashsa to'xta)                 # ✅

7. Integratsiya — bu bilim qayerda kerak bo'ladi

  • 25.7-dars (o'tilgan): Gradient — o'qitish asosi
  • 25.8-dars (o'tilgan): Neyron tarmoq — o'qitiladigan model
  • 25.4-dars (o'tilgan): Baholash — test loss
  • 25.2-dars (o'tilgan): Train/test — halol
  • 25.12-dars: Deploy — o'qitilgan model

8. Eng yaxshi amaliyotlar

  1. Siklli: zero_grad → bashorat → loss → backward → step.

  2. To'g'ri loss (MSE/CrossEntropy).

  3. Adam — yaxshi boshlangich optimizator.

  4. lr sinab topi (0.001, 0.01, 0.1).

  5. Test'da no_grad (tez).

  6. Train va test loss kuzat (overfitting).

  7. Loss kamaymasa — lr yoki model tekshir.

  8. Epoch — loss barqarorlashsa to'xta.

9. Amaliy topshiriq

Vazifa 1: Bashorat qiling

python
1.  # o'qitish siklli nima?
2.  # zero_grad nima?
3.  # loss nima?
4.  # backward nima?
5.  # step nima?
6.  # MSELoss nima?
7.  # CrossEntropyLoss nima?
8.  # optimizator nima?
9.  # SGD vs Adam?
10. # epoch nima?
11. # learning rate nima?
12. # gradient descent nima?
Javoblar
  1. Bashorat → loss → gradient → yangilash (takror)
  2. Gradientni tozalash (yig'iladi)
  3. Model xatosi (bashorat vs haqiqiy)
  4. Gradientni hisoblash
  5. Vaznlarni yangilash
  6. O'rtacha kvadratik xato (regressiya)
  7. Sinf xatosi (klassifikatsiya)
  8. Vaznlarni yangilovchi
  9. SGD oddiy, Adam adaptiv
  10. Butun ma'lumot bo'yicha bir o'tish
  11. Qadam kattaligi
  12. Loss'ni kamaytirish (pastga)

Vazifa 2: Xatolarni tuzating

python
1.  loss.backward(); step()   # zero_grad yo'q     # avval tozala

2.  nn.MSELoss()   # klassifikatsiyada             # CrossEntropy

3.  optim.SGD(..., lr=10)                          # 0.01

4.  model(X_test)   # gradient                     # no_grad

5.  y = torch.tensor([0,1], dtype=torch.float)     # long
Javoblar
python
1.  optimizer.zero_grad(); ...; step()

2.  nn.CrossEntropyLoss()

3.  optim.SGD(..., lr=0.01)

4.  with torch.no_grad(): model(X_test)

5.  y = torch.tensor([0,1], dtype=torch.long)

Vazifa 3: O'qitish siklli

Regressiya:

  1. Model, loss, optim
  2. Sikl (5 qadam)
  3. Loss kamayadi
  4. Parametrlar

Vazifa 4: Gradient descent

O'rganish:

  1. Loss egri
  2. Bashorat
  3. Naqsh
  4. Kamayish

Vazifa 5: Klassifikatsiya

O'qitish:

  1. CrossEntropy
  2. Adam
  3. argmax
  4. Aniqlik

Vazifa 6: Train/test

To'liq:

  1. Bo'lish
  2. Train o'qit
  3. Test bahola
  4. Overfitting

Vazifa 7: O'ylash

Modelni o'qitish — gradient descent: loss (xato) "balandlik", gradient "pastga yo'nalish", lr "qadam" — model asta-sekin minimumga (kam loss) tushadi. Bu oddiy chiziqli modeldan GPT'gacha barcha chuqur o'qitishning asosi (faqat ko'proq parametr, ma'lumot). Nima uchun "oddiy g'oya (xatoni kamaytir) katta miqyosda ishlaydi", va nega bu "iteratsiya orqali yaxshilash" (o'rganish — bir marta emas, takror) tamoyilini ko'rsatadi?

Javob

Qisqa javob: Gradient descent — oddiy g'oya: "xatoni o'lcha (loss), uni kamaytirish yo'nalishini top (gradient), kichik qadam tashla (lr), takrorla". Bu katta miqyosda ishlaydi, chunki: har qadam (epoch) modelni biroz yaxshilaydi — millionlab qadam (ko'p parametr, ma'lumot) bilan murakkab model (GPT) o'rganadi. "Iteratsiya orqali yaxshilash" tamoyili — o'rganish bir marta emas, takror: har epoch kichik yaxshilash, birga katta natija. Bu inson o'rganishiga o'xshaydi (mashq — takror, bir marta emas), muhandislik jarayoniga (prototip → yaxshilash → takror). "Kichik, takroriy qadamlar — katta natija": oddiy qoida (gradient bo'yicha pastga) + ko'p iteratsiya = murakkab o'rganish. Bu chuqur o'qitishning nafis kuchi — bitta oddiy tamoyil, katta miqyosda.

1. Oddiy g'oya

Gradient descent: loss → gradient → qadam → takror. Har qadam biroz yaxshilash. Oddiy, lekin kuchli.

2. Katta miqyosda

Har qadam kichik yaxshilash × millionlab qadam (ko'p parametr, ma'lumot) = murakkab model (GPT). Oddiy g'oya, katta natija.

3. Iteratsiya orqali yaxshilash

O'rganish — bir marta emas, takror:

  • Har epoch: kichik yaxshilash
  • Ko'p epoch: katta natija
  • Loss asta-sekin kamayadi

4. Boshqa joyda

Soha Iteratsiya
ML o'qitish Epoch (loss kamayadi)
Inson o'rganish Mashq (takror)
Muhandislik Prototip → yaxshilash
Evolyutsiya Avlod (moslashish)

5. Muhandislik saboqlari

  1. Oddiy g'oya (xatoni kamaytir) katta miqyosda
  2. Iteratsiya — takroriy yaxshilash
  3. Kichik qadamlar → katta natija
  4. Bitta tamoyil, ko'p iteratsiya

6. Xulosa

  1. Gradient descent — oddiy, kuchli
  2. Har qadam biroz yaxshilash
  3. Iteratsiya (takror) — o'rganish
  4. Oddiy g'oya katta miqyosda (GPT)

Nimani mustahkamlaydi: 2.1–2.8-bo'limlar.

Xulosa

Bu darsda modelni o'qitishni o'rgandik.

Eng muhim uch fikr:

  1. O'qitish siklli. O'qitish siklli — beshta qadam, ko'p marta takror: zero_grad (gradientni tozala — 25.7 yig'iladi), bashorat (model(X)), loss (yo'qotish — xato), backward (gradient — 25.7), step (vaznlarni yangila). Har takror (epoch) da loss kamayadi — model o'rganadi. scikit-learn fit shu siklli yashirgan 25.2-bob; PyTorch ochiq (to'liq nazorat). Bu gradient descent — loss (xato) ni kamaytirish yo'nalishida vaznlarni o'zgartirish.

  2. Loss, optimizator, lr. Yo'qotish funksiyasi (loss) — bashorat haqiqiydan qancha chetda (kichik = yaxshi): MSELoss (regressiya), CrossEntropyLoss (klassifikatsiya — y long tur, sinf indeksi). Optimizator — vaznlarni gradient bo'yicha yangilovchi: SGD (oddiy — vazn - lr × gradient), Adam (adaptiv — ko'pincha tezroq, zamonaviy standart). Learning rate (lr) — qadam kattaligi (eng muhim giperparametr): juda katta (tebranadi), juda kichik (sekin) — sinab topiladi. Epoch — butun ma'lumot bo'yicha bir o'tish.

  3. Train/test va gradient descent. Model train'da o'qitiladi, test'da baholanadi (25.2 — halol; torch.no_grad() — test'da gradient kerak emas). Overfitting kuzat (train loss tushsa ham test loss oshsa — yodlash; train vs test taqqosla). Klassifikatsiya baholash: pred.argmax(1) (eng yuqori ehtimolli sinf). Modelni o'qitish — gradient descent (loss "balandlik", gradient "pastga", lr "qadam" — minimumga tushadi). Bu oddiy chiziqli modeldan GPT'gacha barcha chuqur o'qitishning asosi (faqat ko'proq parametr, ma'lumot, epoch). "Iteratsiya orqali yaxshilash" — o'rganish bir marta emas, takror (kichik qadamlar → katta natija). Oddiy g'oya (xatoni kamaytir), katta miqyosda.

Keyingi darsda LLM API bilan ishlash ni o'rganamiz: katta til modellari (GPT, Claude) ni API orqali ishlatish — zamonaviy AI ilovalari (chatbot, matn generatsiya) qurish.

Ulashish:Telegram'da

Izohlar (0)

Izoh yozish uchun kiring.

  • Hozircha izoh yo'q. Birinchi bo'ling!