14.4-dars: LDA va QDA

Har sinf uchun ko'p o'lchovli normal taqsimot:
  P(x | y = k) = N(x; mu_k, Sigma_k)

Bayes qoidasi 14.1-bob:
  argmax_k  P(y=k) · N(x; mu_k, Sigma_k)

TAXMINLAR:
  LDA: barcha sinflarda BIR XIL kovariatsiya (Sigma_k = Sigma)  → chiziqli chegara
  QDA: har sinfda O'Z kovariatsiyasi (Sigma_k)                   → kvadratik chegara
  Naive Bayes: Sigma diagonal (belgilar mustaqil — 14.3)

p — belgilar soni, K — sinflar soni

Naive Bayes:  K·p (o'rtacha) + K·p (dispersiya)          → 2·K·p
LDA:          K·p (o'rtacha) + p(p+1)/2 (umumiy Sigma)
QDA:          K·p (o'rtacha) + K·p(p+1)/2 (har sinf uchun)

p = 20, K = 2:  NB 80,  LDA 250,  QDA 460
p = 50, K = 2:  NB 200, LDA 1375, QDA 2650   ← QDA uchun juda ko'p ma'lumot kerak

from sklearn.discriminant_analysis import LinearDiscriminantAnalysis

LinearDiscriminantAnalysis(solver="lsqr", shrinkage="auto")     # Ledoit-Wolf
LinearDiscriminantAnalysis(solver="lsqr", shrinkage=0.3)        # qo'lda

Shrinkage: Sigma_reg = (1 - a)·Sigma + a·(o'rtacha dispersiya)·I
  a = 0 → oddiy LDA;  a = 1 → diagonal (Naive Bayes ga yaqin)

QDA uchun: QuadraticDiscriminantAnalysis(reg_param=0.1)

LDA sinflarni eng yaxshi AJRATADIGAN yo'nalishlarni topadi
  → PCA 10.9-bob dispersiyani maksimallashtiradi, LDA esa sinflar orasidagi farqni

Maksimal komponentlar soni: K - 1
  (2 sinf → 1 o'lcham; 10 sinf → 9 o'lcham)

lda = LinearDiscriminantAnalysis(n_components=2).fit(X, y)
X_2d = lda.transform(X)          # vizualizatsiya uchun

LDA va logistik regressiya IKKALASI ham chiziqli chegara beradi, lekin:

  LDA:    generativ — P(x|y) modellaydi, normallik taxminiga tayanadi
          + kam ma'lumotda barqaror, ko'p sinfda tabiiy, tez
          - outlierlarga sezgir, taxmin buzilsa yomonlashadi

  LogReg: diskriminativ — P(y|x) ni to'g'ridan-to'g'ri
          + taxminlari kam, regularizatsiya oson, kalibrlangan 13.10-bob
          - kam ma'lumotda beqarorroq

Amalda: n katta bo'lsa natijalar juda yaqin

LDA:
  · kam namuna, ko'p belgi (shrinkage bilan)
  · belgilar taxminan normal, sinf kovariatsiyalari o'xshash
  · vizualizatsiya kerak (K-1 o'lchamga proyeksiya)
  · ko'p sinfli vazifa (tabiiy qo'llab-quvvatlash)

QDA:
  · sinflar TARQOQLIGI sezilarli farq qiladi
  · har sinfda yetarlicha ko'p namuna bor

Ikkalasi ham EMAS:
  · kuchli kategorik belgilar, qiyshiq taqsimotlar
  · juda murakkab chegara (kernel SVM, ansambllar yaxshiroq)

import numpy as np
from sklearn.discriminant_analysis import (LinearDiscriminantAnalysis,
                                           QuadraticDiscriminantAnalysis)
from sklearn.model_selection import GridSearchCV

LinearDiscriminantAnalysis()                                   # svd (standart)
LinearDiscriminantAnalysis(solver="lsqr", shrinkage="auto")    # regularizatsiyalangan
QuadraticDiscriminantAnalysis(reg_param=0.1)

# o'lchamni kamaytirish
lda = LinearDiscriminantAnalysis(n_components=2).fit(X, y)
X2 = lda.transform(X)                  # maksimal K-1 komponent

lda.means_, lda.priors_, lda.coef_, lda.explained_variance_ratio_
GridSearchCV(QuadraticDiscriminantAnalysis(), {"reg_param": [0, 0.01, 0.1, 0.5]}, cv=5)
QOIDA: p katta bo'lsa shrinkage · QDA uchun ko'p namuna · qiyshiqlikni tuzat

Har sinf — normal taqsimot; farq kovariatsiyada
NB (diagonal) < LDA (umumiy Sigma) < QDA (har sinfda Sigma_k)
Parametrlar: NB 2Kp, LDA Kp + p(p+1)/2, QDA Kp + Kp(p+1)/2
shrinkage — p katta bo'lganda majburiy · LDA proyeksiyasi: K-1 o'lcham

"""Chiziqli va kvadratik chegara qachon kerak (real numpy/sklearn)."""

import numpy as np
from sklearn.discriminant_analysis import (LinearDiscriminantAnalysis,
                                           QuadraticDiscriminantAnalysis)
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import GaussianNB


def yarat(tur: str, seed: int = 5, n: int = 4000):
    rng = np.random.default_rng(seed)
    y = rng.integers(0, 2, n)
    if tur == "teng":
        # ikkala sinfda bir xil kovariatsiya
        A = np.array([[1.4, 0.7], [0.0, 0.9]])
        X = rng.normal(0, 1, (n, 2)) @ A + np.where(y[:, None] == 1,
                                                    np.array([1.6, 0.8]), 0.0)
    else:
        # sinflar tarqoqligi keskin farq qiladi
        A0 = np.array([[0.6, 0.0], [0.0, 0.6]])
        A1 = np.array([[2.2, 1.3], [0.0, 1.8]])
        Z = rng.normal(0, 1, (n, 2))
        X = np.where(y[:, None] == 0, Z @ A0, Z @ A1 + np.array([0.4, 0.0]))
    return X, y


def main() -> None:
    modellar = {
        "GaussianNB": GaussianNB(),
        "LDA": LinearDiscriminantAnalysis(),
        "QDA": QuadraticDiscriminantAnalysis(),
    }

    print("=== 1. Teng kovariatsiya (LDA taxmini to'g'ri) ===")
    X, y = yarat("teng")
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0,
                                          stratify=y)
    for nom, m in modellar.items():
        m.fit(Xtr, ytr)
        print(f"  {nom:<11}: test aniqligi {(m.predict(Xte) == yte).mean():.4f}")

    print("\n=== 2. Farqli kovariatsiya (LDA taxmini buzilgan) ===")
    X, y = yarat("farqli")
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0,
                                          stratify=y)
    for nom, m in modellar.items():
        m.fit(Xtr, ytr)
        print(f"  {nom:<11}: test aniqligi {(m.predict(Xte) == yte).mean():.4f}")

    print("\n=== 3. Sinflarning kovariatsiyalari ===")
    for k in [0, 1]:
        S = np.cov(Xtr[ytr == k].T)
        print(f"  sinf {k}: dispersiyalar {np.diag(S).round(3)}, "
              f"korrelyatsiya {S[0, 1] / np.sqrt(S[0, 0] * S[1, 1]):+.3f}")

    print("\n=== 4. QDA chegarasi kvadratik ===")
    qda = QuadraticDiscriminantAnalysis().fit(Xtr, ytr)
    lda = LinearDiscriminantAnalysis().fit(Xtr, ytr)
    # chiziq bo'ylab bashoratlar qanday o'zgaradi
    chiziq = np.column_stack([np.linspace(-6, 6, 13), np.zeros(13)])
    print(f"  x2 = 0 chizig'i bo'ylab QDA bashoratlari:")
    print(f"    {qda.predict(chiziq)}")
    print(f"  LDA bashoratlari:")
    print(f"    {lda.predict(chiziq)}")
    print("  ⭐ QDA bitta chiziqda sinfni ikki marta almashtira oladi")


if __name__ == "__main__":
    main()

=== 1. Teng kovariatsiya (LDA taxmini to'g'ri) ===
  GaussianNB : test aniqligi 0.7125
  LDA        : test aniqligi 0.7208
  QDA        : test aniqligi 0.7200

=== 2. Farqli kovariatsiya (LDA taxmini buzilgan) ===
  GaussianNB : test aniqligi 0.8808
  LDA        : test aniqligi 0.6000
  QDA        : test aniqligi 0.8883

=== 3. Sinflarning kovariatsiyalari ===
  sinf 0: dispersiyalar [0.373 0.351], korrelyatsiya -0.045
  sinf 1: dispersiyalar [5.11  5.193], korrelyatsiya +0.577

=== 4. QDA chegarasi kvadratik ===
  x2 = 0 chizig'i bo'ylab QDA bashoratlari:
    [1 1 1 1 1 0 0 0 1 1 1 1 1]
  LDA bashoratlari:
    [0 0 0 0 0 0 0 1 1 1 1 1 1]
  ⭐ QDA bitta chiziqda sinfni ikki marta almashtira oladi

"""p katta, n kichik: regularizatsiya hal qiladi (real numpy/sklearn)."""

import warnings

import numpy as np
from sklearn.discriminant_analysis import (LinearDiscriminantAnalysis,
                                           QuadraticDiscriminantAnalysis)
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler


def aralashtirish(p: int) -> np.ndarray:
    """Yengil korrelyatsiya beruvchi QAT'IY matritsa (barcha namunada bir xil)."""
    rng = np.random.default_rng(0)
    return np.eye(p) + 0.25 * rng.normal(0, 1, (p, p)) / np.sqrt(p)


def yarat(n: int, p: int, seed: int = 3):
    """Taqsimot O'ZGARMAYDI — faqat namuna o'zgaradi (seed)."""
    rng = np.random.default_rng(seed)
    y = rng.integers(0, 2, n)
    siljish = np.zeros(p)
    siljish[:8] = [0.8, -0.7, 0.6, -0.5, 0.55, 0.45, -0.4, 0.5]
    X = rng.normal(0, 1, (n, p)) @ aralashtirish(p) + y[:, None] * siljish
    return X, y


def main() -> None:
    p = 100
    Xte, yte = yarat(6000, p, seed=99)

    print(f"=== 1. Belgilar soni p = {p}, turli o'quv hajmi ===")
    print(f"  {'n':>6} {'LDA':>8} {'LDA+shr':>9} {'QDA':>8} {'QDA+reg':>9} "
          f"{'LogReg':>8}")
    for n in [60, 120, 400, 2000]:
        Xtr, ytr = yarat(n, p, seed=n)
        modellar = {
            "LDA": LinearDiscriminantAnalysis(),
            "LDA+shr": LinearDiscriminantAnalysis(solver="lsqr",
                                                  shrinkage="auto"),
            "QDA": QuadraticDiscriminantAnalysis(),
            "QDA+reg": QuadraticDiscriminantAnalysis(reg_param=0.3),
            "LogReg": Pipeline([("sc", StandardScaler()),
                                ("m", LogisticRegression(max_iter=3000))]),
        }
        ballar = []
        with warnings.catch_warnings():
            warnings.simplefilter("ignore")      # kollinearlik ogohlantirishlari
            for m in modellar.values():
                try:
                    m.fit(Xtr, ytr)
                    ballar.append((m.predict(Xte) == yte).mean())
                except np.linalg.LinAlgError:
                    # kovariatsiya matritsasi to'liq rangli emas (n < p)
                    ballar.append(None)
        kengliklar = [8, 9, 8, 9, 8]
        print(f"  {n:>6} " + " ".join(
            (f"{b:>{w}.4f}" if b is not None else f"{'xato':>{w}}")
            for b, w in zip(ballar, kengliklar)))

    print("  ('xato' — kovariatsiya matritsasi to'liq rangli emas: sinfda n < p)")

    print("\n=== 2. Shrinkage darajasi (n = 100) ===")
    Xtr, ytr = yarat(100, p, seed=7)
    for a in [0.0, 0.1, 0.3, 0.6, 1.0, "auto"]:
        with warnings.catch_warnings():
            warnings.simplefilter("ignore")
            m = LinearDiscriminantAnalysis(solver="lsqr", shrinkage=a).fit(Xtr, ytr)
        nom = "auto" if a == "auto" else f"{a:.1f}"
        print(f"  shrinkage {nom:>5}: test aniqligi "
              f"{(m.predict(Xte) == yte).mean():.4f}")
    print("  (1.0 — kovariatsiya to'liq diagonal, Naive Bayes ga yaqin)")

    print("\n=== 3. Parametrlar soni ===")
    K = 2
    for pp in [10, 40, 100]:
        nb = 2 * K * pp
        lda = K * pp + pp * (pp + 1) // 2
        qda = K * pp + K * pp * (pp + 1) // 2
        print(f"  p = {pp:>3}: NB {nb:>6}, LDA {lda:>6}, QDA {qda:>7}")

    print("\n=== 4. Xulosa ===")
    print("  n kichik va p katta bo'lsa: LDA+shrinkage eng barqaror")
    print("  n o'sganda farq yo'qoladi")
    print("  ⭐ shrinkage — LDA uchun Ridge ning analogi 13.7-bob")


if __name__ == "__main__":
    main()

=== 1. Belgilar soni p = 100, turli o'quv hajmi ===
       n      LDA   LDA+shr      QDA   QDA+reg   LogReg
      60   0.5945    0.6480     xato      xato   0.6433
     120   0.5820    0.6517     xato      xato   0.6392
     400   0.7550    0.7532   0.5393    0.5955   0.7407
    2000   0.7905    0.7895   0.6407    0.6807   0.7885
  ('xato' — kovariatsiya matritsasi to'liq rangli emas: sinfda n < p)

=== 2. Shrinkage darajasi (n = 100) ===
  shrinkage   0.0: test aniqligi 0.4610
  shrinkage   0.1: test aniqligi 0.6650
  shrinkage   0.3: test aniqligi 0.6782
  shrinkage   0.6: test aniqligi 0.6887
  shrinkage   1.0: test aniqligi 0.6865
  shrinkage  auto: test aniqligi 0.6902
  (1.0 — kovariatsiya to'liq diagonal, Naive Bayes ga yaqin)

=== 3. Parametrlar soni ===
  p =  10: NB     40, LDA     75, QDA     130
  p =  40: NB    160, LDA    900, QDA    1720
  p = 100: NB    400, LDA   5250, QDA   10300

=== 4. Xulosa ===
  n kichik va p katta bo'lsa: LDA+shrinkage eng barqaror
  n o'sganda farq yo'qoladi
  ⭐ shrinkage — LDA uchun Ridge ning analogi (13.7)

"""Nazoratli proyeksiya: LDA va PCA (real numpy/sklearn)."""

import numpy as np
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler


def main() -> None:
    X, y = load_digits(return_X_y=True)
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0,
                                          stratify=y)

    print("=== 1. Vazifa ===")
    print(f"  {len(X)} rasm, {X.shape[1]} piksel, {len(np.unique(y))} sinf")
    print(f"  LDA uchun maksimal komponentlar: K - 1 = {len(np.unique(y)) - 1}")

    print("\n=== 2. LDA proyeksiyasi ===")
    lda = LinearDiscriminantAnalysis(n_components=9).fit(Xtr, ytr)
    print(f"  tushuntirilgan nisbat (birinchi 5): "
          f"{lda.explained_variance_ratio_[:5].round(3)}")
    print(f"  jami (9 komponent): {lda.explained_variance_ratio_.sum():.3f}")

    print("\n=== 3. Klassifikatsiya: LDA va PCA proyeksiyasida ===")
    print(f"  {'k':>4} {'LDA proyeksiya':>16} {'PCA proyeksiya':>16}")
    for k in [2, 4, 9]:
        lda_k = LinearDiscriminantAnalysis(n_components=k).fit(Xtr, ytr)
        pca_k = PCA(n_components=k, random_state=0).fit(Xtr)
        log_lda = LogisticRegression(max_iter=3000).fit(lda_k.transform(Xtr), ytr)
        log_pca = Pipeline([("sc", StandardScaler()),
                            ("m", LogisticRegression(max_iter=3000))]).fit(
            pca_k.transform(Xtr), ytr)
        print(f"  {k:>4} "
              f"{(log_lda.predict(lda_k.transform(Xte)) == yte).mean():>16.4f} "
              f"{(log_pca.predict(pca_k.transform(Xte)) == yte).mean():>16.4f}")
    print("  (LDA yorliqlarni biladi — kam o'lchamda ham sinflarni ajratadi)")

    print("\n=== 4. LDA ning o'zi klassifikator sifatida ===")
    print(f"  LDA (to'liq): test aniqligi {(lda.predict(Xte) == yte).mean():.4f}")
    toliq = Pipeline([("sc", StandardScaler()),
                      ("m", LogisticRegression(max_iter=3000))]).fit(Xtr, ytr)
    print(f"  LogReg (to'liq): test aniqligi "
          f"{(toliq.predict(Xte) == yte).mean():.4f}")
    # sinflar markazlari orasidagi masofa proyeksiyada
    Z = lda.transform(Xtr)
    markazlar = np.array([Z[ytr == k].mean(axis=0) for k in range(10)])
    masofalar = np.linalg.norm(markazlar[:, None] - markazlar[None], axis=2)
    np.fill_diagonal(masofalar, np.inf)
    a, b = np.unravel_index(masofalar.argmin(), masofalar.shape)
    print(f"  proyeksiyada eng yaqin sinflar: {a} va {b} "
          f"(masofa {masofalar[a, b]:.2f})")
    print("  ⭐ LDA — ham klassifikator, ham nazoratli o'lchamni kamaytiruvchi")


if __name__ == "__main__":
    main()

=== 1. Vazifa ===
  1797 rasm, 64 piksel, 10 sinf
  LDA uchun maksimal komponentlar: K - 1 = 9

=== 2. LDA proyeksiyasi ===
  tushuntirilgan nisbat (birinchi 5): [0.297 0.182 0.164 0.111 0.086]
  jami (9 komponent): 1.000

=== 3. Klassifikatsiya: LDA va PCA proyeksiyasida ===
     k   LDA proyeksiya   PCA proyeksiya
     2           0.7019           0.5833
     4           0.8870           0.7981
     9           0.9500           0.9370
  (LDA yorliqlarni biladi — kam o'lchamda ham sinflarni ajratadi)

=== 4. LDA ning o'zi klassifikator sifatida ===
  LDA (to'liq): test aniqligi 0.9593
  LogReg (to'liq): test aniqligi 0.9722
  proyeksiyada eng yaqin sinflar: 1 va 8 (masofa 4.57)
  ⭐ LDA — ham klassifikator, ham nazoratli o'lchamni kamaytiruvchi

"""Qiyshiqlik, outlier va kategoriyalarning ta'siri (real numpy/sklearn)."""

import numpy as np
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler


def main() -> None:
    rng = np.random.default_rng(13)
    n = 3000

    print("=== 1. Qiyshiq belgilar ===")
    y = rng.integers(0, 2, n)
    X_log = rng.lognormal(y[:, None] * 0.45, 1.0, (n, 6))
    Xtr, Xte, ytr, yte = train_test_split(X_log, y, test_size=0.3, random_state=0,
                                          stratify=y)
    lda = LinearDiscriminantAnalysis().fit(Xtr, ytr)
    log = Pipeline([("sc", StandardScaler()),
                    ("m", LogisticRegression(max_iter=2000))]).fit(Xtr, ytr)
    print(f"  xom belgilar: LDA {(lda.predict(Xte) == yte).mean():.4f}, "
          f"LogReg {(log.predict(Xte) == yte).mean():.4f}")
    lda2 = LinearDiscriminantAnalysis().fit(np.log(Xtr), ytr)
    log2 = Pipeline([("sc", StandardScaler()),
                     ("m", LogisticRegression(max_iter=2000))]).fit(np.log(Xtr), ytr)
    print(f"  log olingan: LDA {(lda2.predict(np.log(Xte)) == yte).mean():.4f}, "
          f"LogReg {(log2.predict(np.log(Xte)) == yte).mean():.4f}")

    print("\n=== 2. Outlierlar ta'siri ===")
    y2 = rng.integers(0, 2, n)
    X2 = rng.normal(y2[:, None] * 1.0, 1.0, (n, 4))
    Xtr, Xte, ytr, yte = train_test_split(X2, y2, test_size=0.3, random_state=0,
                                          stratify=y2)
    print(f"  {'buzilgan ulush':>16} {'LDA':>8} {'LogReg':>8}")
    for ulush in [0.0, 0.02, 0.05, 0.10]:
        Xb = Xtr.copy()
        k = int(ulush * len(Xb))
        if k:
            idx = rng.choice(len(Xb), k, replace=False)
            Xb[idx] += rng.normal(0, 25, (k, 4))
        l = LinearDiscriminantAnalysis().fit(Xb, ytr)
        g = Pipeline([("sc", StandardScaler()),
                      ("m", LogisticRegression(max_iter=2000))]).fit(Xb, ytr)
        print(f"  {ulush:>16.0%} {(l.predict(Xte) == yte).mean():>8.4f} "
              f"{(g.predict(Xte) == yte).mean():>8.4f}")
    print("  (LDA o'rtacha va kovariatsiyaga tayanadi — outlierlarga sezgirroq)")

    print("\n=== 3. Binar (kategorik) belgilar ===")
    y3 = rng.integers(0, 2, n)
    X3 = (rng.random((n, 8)) < np.where(y3[:, None] == 1, 0.65, 0.35)).astype(float)
    Xtr, Xte, ytr, yte = train_test_split(X3, y3, test_size=0.3, random_state=0,
                                          stratify=y3)
    from sklearn.naive_bayes import BernoulliNB
    for nom, m in [("LDA", LinearDiscriminantAnalysis()),
                   ("BernoulliNB", BernoulliNB()),
                   ("LogReg", Pipeline([("sc", StandardScaler()),
                                        ("m", LogisticRegression(max_iter=2000))]))]:
        m.fit(Xtr, ytr)
        print(f"  {nom:<12}: {(m.predict(Xte) == yte).mean():.4f}")
    print("  (binar belgilarda normallik taxmini buzilgan, lekin LDA baribir ishlaydi)")

    print("\n=== 4. Xulosa ===")
    print("  qiyshiqlik → transformatsiya yordam beradi")
    print("  outlier → LDA sezgirroq, tekshiring 13.4-bob")
    print("  ⭐ Taxminlar buzilsa LogReg ishonchliroq, lekin farq ko'pincha kichik")


if __name__ == "__main__":
    main()

=== 1. Qiyshiq belgilar ===
  xom belgilar: LDA 0.6944, LogReg 0.7022
  log olingan: LDA 0.7278, LogReg 0.7289

=== 2. Outlierlar ta'siri ===
    buzilgan ulush      LDA   LogReg
                0%   0.8267   0.8256
                2%   0.8211   0.8222
                5%   0.8267   0.8189
               10%   0.8156   0.8144
  (LDA o'rtacha va kovariatsiyaga tayanadi — outlierlarga sezgirroq)

=== 3. Binar (kategorik) belgilar ===
  LDA         : 0.7878
  BernoulliNB : 0.7844
  LogReg      : 0.7822
  (binar belgilarda normallik taxmini buzilgan, lekin LDA baribir ishlaydi)

=== 4. Xulosa ===
  qiyshiqlik → transformatsiya yordam beradi
  outlier → LDA sezgirroq, tekshiring 13.4-bob
  ⭐ Taxminlar buzilsa LogReg ishonchliroq, lekin farq ko'pincha kichik

LinearDiscriminantAnalysis().fit(X_200_belgi, y)                  # ⚠️
LinearDiscriminantAnalysis(solver="lsqr", shrinkage="auto")       # ✅

QuadraticDiscriminantAnalysis().fit(X, y)   # sinfda 20 namuna    # ⚠️
LinearDiscriminantAnalysis(solver="lsqr", shrinkage="auto")       # ✅

LinearDiscriminantAnalysis(shrinkage="auto")   # solver="svd"     # ⚠️
LinearDiscriminantAnalysis(solver="lsqr", shrinkage="auto")       # ✅

LinearDiscriminantAnalysis().fit(X_lognormal, y)                  # ⚠️
LinearDiscriminantAnalysis().fit(np.log(X_lognormal), y)          # ✅

LinearDiscriminantAnalysis(n_components=5).fit(X, y)   # 3 sinf   # ⚠️
LinearDiscriminantAnalysis(n_components=2)                        # ✅

LinearDiscriminantAnalysis().fit(X, y)                            # ⚠️
# avval outlierlarni tekshiring (13.4, 13.11)                     # ✅

QuadraticDiscriminantAnalysis()                                   # ⚠️
GridSearchCV(QDA(), {"reg_param": [0, 0.01, 0.1, 0.5]}, cv=5)     # ✅

1.  # LDA qanday model?
2.  # LDA va QDA farqi?
3.  # Naive Bayes ular bilan qanday bog'liq?
4.  # LDA chegarasi qanday?
5.  # QDA chegarasi?
6.  # QDA parametrlari soni?
7.  # shrinkage nima qiladi?
8.  # shrinkage = 1 nima?
9.  # LDA maksimal komponenti?
10. # LDA va PCA farqi?
11. # LDA outlierlarga qanday?
12. # qaysi solver shrinkage ni qo'llab-quvvatlaydi?

1.  LinearDiscriminantAnalysis().fit(X_500_belgi, y)   # n = 200

2.  QuadraticDiscriminantAnalysis().fit(X, y)   # sinfda 15 namuna

3.  LinearDiscriminantAnalysis(shrinkage=0.3)   # solver standart

4.  LinearDiscriminantAnalysis(n_components=6).fit(X, y)   # 3 sinf

5.  LinearDiscriminantAnalysis().fit(daromad_ustunlari, y)   # lognormal