8.5-dars: Korrelyatsiya tahlil

8-QISM — EDA (RAZVEDKA TAHLILI) · 5-dars

1. Kirish va motivatsiya

Bivariate 8.4-bob bir juftlikni ko'rdi — endi barcha juftliklarni birga. Korrelyatsiya matritsasi 4.9-bob barcha son ustunlar orasidagi aloqani bir jadvalda ko'rsatadi, heatmap 5.8-bob esa uni rangli qiladi. Bu EDA'ning kuchli vositasi: bir qarashda qaysi ustunlar bog'liq, qaysilari mustaqil. Ayniqsa ikki savol muhim: (1) qaysi xususiyatlar target (maqsad) bilan bog'liq (model uchun muhim feature — 19-qism), (2) qaysi xususiyatlar o'zaro bog'liq (multikollinearlik — model muammosi). Bu darsda korrelyatsiya matritsasi, heatmap, feature-target aloqasi, va multikollinearlik. Nega muhim? (1) Barcha juftlik — bir qarashda (matritsa/heatmap); (2) Feature — target bilan bog'liq (muhim); (3) Multikollinearlik — o'zaro bog'liq (muammo). Bu dars korrelyatsiya tahlilni o'rgatadi — barcha aloqalar birga.

Korrelyatsiya tahlil — barcha aloqalar birga: matritsa (df.corr() — barcha juftlik; 4.9), heatmap (5.8 — rangli; coolwarm, annot), feature-target (qaysi xususiyat target bilan — model muhim; 19-qism), multikollinearlik (xususiyatlar o'zaro bog'liq — model muammo), turli korrelyatsiya (Pearson chiziqli; Spearman tartib — 4.9). Foydalanish: barcha aloqa, feature tanlash, multikollinearlik. Bu 4.9 (korrelyatsiya), 5.8 (heatmap), 8.4 (bivariate), 19-qism (feature) bilan bog'liq. Korrelyatsiya tahlil — barcha aloqa (matritsa/heatmap). Korrelyatsiya. Matritsa.

Real vaziyat. Data Scientist uy narxi ma'lumotida 8 xususiyat (maydon, xonalar, yosh, joylashuv, ...) — qaysi narxga ta'sir? Korrelyatsiya matritsasi (df.corr()) + heatmap: (1) feature-target — narx (target) bilan: maydon 0.85 (kuchli musbat — muhim), xonalar 0.7 (kuchli), yosh -0.4 (o'rtacha manfiy — eski arzon), joylashuv id 0.02 (mustaqil — tashla), (2) multikollinearlik — maydon va xonalar o'zaro 0.9 (kuchli — ikkalasi bir narsa; modelga ikkalasi ortiqcha). Data Scientist muhim featurelarni tanladi (maydon, xonalar, yosh — narx bilan), multikollinearlik ko'rdi (maydon-xonalar — bitta), model uchun tayyorlandi. Korrelyatsiya tahlil — barcha aloqa.

Bu darsda korrelyatsiya tahlilni o'rganamiz.

Bu darsda:

• Korrelyatsiya matritsasi (barcha juftlik)
• Heatmap (rangli talqin)
• Feature-target aloqasi
• Multikollinearlik (o'zaro bog'liq)
• Pearson vs Spearman
• Korrelyatsiya amaliyoti
• Korrelyatsiya tuzoqlari
• Amaliy: korrelyatsiya tahlil

ℹ Misollar real pandas/numpy bilan (Python 3.14).

2. Nazariya — chuqur tushuntirish

2.1. Korrelyatsiya matritsasi (barcha juftlik)

Barcha son ustun:

import pandas as pd

# barcha son ustun korrelyatsiyasi (4.9)
korr = df.corr(numeric_only=True)
# n×n matritsa: har juftlik korrelyatsiyasi (-1..+1)

# target bilan (narx ustuni)
korr["narx"].sort_values(ascending=False)

Korrelyatsiya matritsasi (barcha juftlik) — n×n: df.corr(numeric_only=True) — barcha son ustun juftligi korrelyatsiyasi (n×n matritsa; -1..+1; 4.9); diagonal (o'zi bilan — 1.0), simmetrik (yuqori=pastki). Sabab: barcha juftlik (bivariate — bitta; matritsa — hammasi birga; bir qarashda); son ustun (numeric_only — kategoriya emas); target ustun (korr["narx"] — target bilan barcha; feature). corr (matritsa), diagonal 1.0-bob, target ustun (feature aloqasi). Korrelyatsiya matritsasi — n×n (barcha juftlik). Matritsa. corr.

2.2. Heatmap (rangli talqin)

Rangli ko'rinish:

import seaborn as sns

# heatmap — korrelyatsiya matritsasi rangli (5.8)
sns.heatmap(df.corr(numeric_only=True),
    annot=True,          # katakda son
    cmap="coolwarm",     # divergent (musbat/manfiy)
    vmin=-1, vmax=1, center=0)
# to'q qizil (kuchli musbat), to'q ko'k (manfiy), och (kuchsiz)

Heatmap (rangli talqin) — 5.8: sns.heatmap(df.corr(), annot=True, cmap="coolwarm", vmin=-1, vmax=1, center=0) — korrelyatsiya matritsasi rangli (5.8; to'q qizil kuchli musbat, to'q ko'k manfiy, och kuchsiz; annot son, coolwarm divergent, qat'iy masshtab). Sabab: matritsa raqam (8×8=64 son — jadval o'qish qiyin); heatmap — rang (naqsh bir qarashda; qaysi bog'liq — to'q; mustaqil — och; 5.8); talqin oson. heatmap (rangli), annot (son), coolwarm (divergent), naqsh (bir qarashda). Heatmap — rangli (5.8; talqin). Heatmap. coolwarm.

2.3. Feature-target aloqasi

Feature-target aloqasi — model uchun: target (maqsad — bashorat qilinadigan; narx) bilan xususiyatlar (feature) korrelyatsiyasi: kuchli (|r| yuqori — maydon 0.85; muhim feature — model uchun), kuchsiz (|r| past — joylashuv id 0.02; mustaqil — tashla yoki e'tibor), manfiy (yosh -0.4 — teskari, lekin muhim). Sabab: feature tanlash (target bilan bog'liq — model uchun muhim; 19-qism; kuchsiz — foydasiz); korr["narx"].sort_values (target bilan — tartiblab; muhim feature). Lekin: chiziqli (Pearson — nochiziqli o'tkazib yuboradi; 2.5), aloqa ≠ sabab 4.10-bob. Target ustun (feature aloqasi), kuchli (muhim feature), kuchsiz (tashla). Feature-target — model uchun (muhim feature). Feature. Target.

2.4. Multikollinearlik (o'zaro bog'liq)

Multikollinearlik (o'zaro bog'liq) — model muammosi: xususiyatlar o'zaro kuchli bog'liq (feature-feature — maydon va xonalar 0.9; bir narsani o'lchaydi); muammo (regressiya — 15-qism; koeffitsiyent beqaror/talqin qiyin; ortiqcha), hal (bittasini tashla yoki birlashtir — PCA 18-qism). Sabab: feature-feature (target emas — xususiyatlar o'zaro; multikollinearlik); model (chiziqli — koeffitsiyent buziladi; daraxt — kamroq); "ikki xususiyat bir narsa — ortiqcha". Feature-feature (o'zaro), kuchli (0.9) (multikollinearlik), hal (tashla/PCA). Multikollinearlik — o'zaro bog'liq (model muammo). Multikollinearlik. O'zaro.

2.5. Pearson vs Spearman

Pearson vs Spearman — korrelyatsiya turi: Pearson (corr() standart — chiziqli aloqa; son; normal; 4.9), Spearman (corr(method="spearman") — tartib (rank) aloqa; monoton (o'suvchan/kamayuvchan — chiziqli shart emas); outlierga chidamliroq; tartibli kategoriya), Kendall (tartib — kichik namuna). Sabab: aloqa turi (Pearson — chiziqli; nochiziqli lekin monoton (o'suvchan egri) — Spearman ko'rsatadi; Pearson past); outlier (Spearman chidamliroq — tartib). Pearson (chiziqli — son), Spearman (tartib — monoton, chidamli), tanlov (aloqa turi). Pearson vs Spearman — chiziqli vs tartib. Pearson. Spearman.

2.6. Korrelyatsiya amaliyoti

Korrelyatsiya amaliyoti: matritsa (df.corr(numeric_only=True) — barcha son juftlik); heatmap (sns.heatmap(annot, coolwarm, vmin/vmax, center) — rangli; 5.8); feature-target (korr["target"].sort_values — muhim feature; 19-qism); multikollinearlik (feature-feature kuchli — o'zaro; tashla); tur (Pearson chiziqli; Spearman monoton/chidamli); scatter (kuchli juftlik — shakl tekshir; Anscombe 5.4); aloqa ≠ sabab 4.10-bob. Tuzoqlar: kategoriya (numeric_only), sababiyat 4.10-bob, nochiziqli (Pearson past — scatter), diagonal (1.0 — ma'nosiz), heatmap masshtab (vmin/vmax). Amaliyot — matritsa, heatmap, feature, multikollinearlik. Korrelyatsiya. Aloqa.

2.7. Korrelyatsiya tuzoqlari

Korrelyatsiya asosiy tuzoqlari: kategoriya (corr() — son ustun; kategoriya (kod 1,2,3) — korrelyatsiya ma'nosiz (tartibsiz); numeric_only=True; kcategoriya — crosstab 8.4); sababiyat (korrelyatsiya — aloqa; sabab EMAS (4.10; to'q qizil ≠ sabab; yashirin o'zgaruvchi)); nochiziqli (Pearson — chiziqli; nochiziqli (U shakl) — corr past (aloqa bor!); scatter ko'r — 5.4; yoki Spearman); diagonal (matritsa diagonal — 1.0 (o'zi bilan; ma'nosiz; e'tibor — diagonaldan tashqari)); heatmap masshtab (vmin=-1, vmax=1, center=0 — qat'iy; avtomatik — aldash 5.13); multikollinearlik e'tiborsiz (feature-feature kuchli 0.9-bob — model muammo (regressiya koeffitsiyent); e'tibor — 15-qism); outlier (korrelyatsiya — outlier buzadi (bir nuqta corr tortadi; 6.3); tekshir); feature faqat corr (feature tanlash — faqat korrelyatsiya (chiziqli; nochiziqli/o'zaro ta'sir o'tkazib; 19-qism boshqa usullar)); tasodifiy korrelyatsiya (ko'p juftlik — ba'zi tasodifan to'q (n ustun ko'p; p-qiymat 4.12; tekshir)); kam namuna (kichik namuna — corr beqaror (ishonch oralig'i keng; katta namuna)). Sabab: korrelyatsiya chiziqli/sababiyat/multikollinearlik nozik (kategoriya, nochiziqli, sabab — noto'g'ri). Yechim: numeric_only, aloqa≠sabab, scatter (nochiziqli), Spearman, multikollinearlik e'tibor. Tuzoqlar — kategoriya, sababiyat, nochiziqli, multikollinearlik.

2.8. Korrelyatsiya tahlil — barcha aloqalar birga

Korrelyatsiya tahlil asosiy g'oyasi — barcha aloqalar birga: bivariate (8.4 — bir juftlik) dan keyin — barcha juftlik birga (korrelyatsiya matritsa — 4.9; bir qarashda qaysi bog'liq). Matritsa (df.corr(numeric_only=True) — n×n; diagonal 1.0; simmetrik), heatmap (5.8 — rangli; annot/coolwarm/vmin/vmax/center; naqsh bir qarashda), feature-target (target bilan xususiyat — kuchli muhim feature (maydon 0.85), kuchsiz mustaqil (joylashuv id 0.02); model — 19-qism), multikollinearlik (feature-feature kuchli 0.9-bob — o'zaro bog'liq; model muammo; tashla/PCA 18-qism), Pearson vs Spearman (chiziqli vs tartib/monoton — 4.9; outlier chidamli). Foydalanish: barcha aloqa (bir qarashda — matritsa/heatmap), feature tanlash (target bilan — muhim; 19-qism), multikollinearlik (o'zaro — model muammo). Tuzoqlar: kategoriya (numeric_only — crosstab), sababiyat (4.10 — aloqa), nochiziqli (Pearson past — scatter/Spearman), diagonal (1.0 — ma'nosiz), heatmap masshtab (vmin/vmax — 5.13), multikollinearlik e'tiborsiz (15-qism), outlier (buzadi — 6.3). Bu 4.9 (korrelyatsiya), 5.8 (heatmap), 8.4 (bivariate), 4.10 (sabab), 19-qism (feature), 15-qism (regressiya — multikollinearlik) bilan. Korrelyatsiya tahlil — barcha aloqalar birga (matritsa/heatmap; feature/multikollinearlik). Korrelyatsiya. Matritsa. Feature.

3. Tez ma'lumotnoma

import pandas as pd
import seaborn as sns

# KORRELYATSIYA MATRITSASI (barcha son juftlik):
korr = df.corr(numeric_only=True)      # n×n (-1..+1; 4.9)
korr["narx"].sort_values(ascending=False)   # target bilan (feature)

# HEATMAP (rangli — 5.8):
sns.heatmap(korr, annot=True, cmap="coolwarm", vmin=-1, vmax=1, center=0)
#   to'q qizil (kuchli+), to'q ko'k (-), och (kuchsiz)

# FEATURE-TARGET: kuchli (muhim feature) · kuchsiz (mustaqil — tashla)
# MULTIKOLLINEARLIK: feature-feature kuchli 0.9-bob → o'zaro (model muammo)

# TUR:
df.corr()                              # Pearson (chiziqli)
df.corr(method="spearman")             # Spearman (tartib/monoton; chidamli)

# ALOQA ≠ SABAB 4.10-bob · nochiziqli → Pearson past (scatter ko'r)
QOIDA: numeric_only · scatter (nochiziqli) · aloqa≠sabab · multikollinearlik

Korrelyatsiya tahlil xulosasi

Korrelyatsiya tahlil — barcha aloqalar birga (matritsa/heatmap)
Matritsa — df.corr() (n×n; diagonal 1.0; target ustun)
Heatmap — rangli (coolwarm, annot; naqsh bir qarashda)
Feature-target — kuchli (muhim feature) / kuchsiz (mustaqil)
Multikollinearlik — feature-feature o'zaro (0.9; model muammo)

4. Batafsil misollar

Misollar real pandas/numpy bilan (Python 3.14).

Misol 1 — Korrelyatsiya matritsasi

"""Korrelyatsiya matritsasi (real pandas/numpy)."""

import numpy as np
import pandas as pd


def main() -> None:
    np.random.seed(0)
    n = 200
    maydon = np.random.normal(80, 20, n)
    narx = maydon * 1.5 + np.random.normal(0, 15, n)   # bog'liq
    yosh = np.random.normal(15, 5, n)                  # mustaqil
    df = pd.DataFrame({"maydon": maydon, "narx": narx, "yosh": yosh})

    print("=== 1. Matritsa (df.corr) ===")
    korr = df.corr().round(2)
    print(korr.to_string())

    print("\n=== 2. Diagonal 1.0-bob ===")
    print(f"  maydon-maydon: {korr.loc['maydon', 'maydon']} (o'zi)")

    print("\n=== 3. Target bilan (narx) ===")
    target = korr["narx"].drop("narx").sort_values(ascending=False)
    print(f"  {target.to_dict()}")

    print("\n=== 4. Talqin ===")
    print(f"  maydon-narx: {korr.loc['maydon', 'narx']} (kuchli)")
    print(f"  yosh-narx: {korr.loc['yosh', 'narx']} (kuchsiz — mustaqil)")
    print("  ⭐ Matritsa — barcha juftlik")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

=== 1. Matritsa (df.corr) ===
        maydon  narx  yosh
maydon    1.00  0.91 -0.06
narx      0.91  1.00  0.02
yosh     -0.06  0.02  1.00

=== 2. Diagonal 1.0-bob ===
  maydon-maydon: 1.0 (o'zi)

=== 3. Target bilan (narx) ===
  {'maydon': 0.91, 'yosh': 0.02}

=== 4. Talqin ===
  maydon-narx: 0.91 (kuchli)
  yosh-narx: 0.02 (kuchsiz — mustaqil)
  ⭐ Matritsa — barcha juftlik

Nima ko'rsatdi: 2.1, 2.3-bo'limlar.

Misol 2 — Heatmap (rangli tekshir)

"""Heatmap korrelyatsiya (real seaborn/pandas)."""

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns


def main() -> None:
    np.random.seed(0)
    n = 200
    maydon = np.random.normal(80, 20, n)
    df = pd.DataFrame({
        "maydon": maydon,
        "narx": maydon * 1.5 + np.random.normal(0, 15, n),
        "yosh": np.random.normal(15, 5, n),
    })
    korr = df.corr()

    print("=== 1. Heatmap ===")
    fig, ax = plt.subplots()
    sns.heatmap(korr, annot=True, cmap="coolwarm", vmin=-1, vmax=1, center=0, ax=ax)
    print(f"  matritsa: {korr.shape}")

    print("\n=== 2. Rang talqini ===")
    print("  to'q qizil (+1), to'q ko'k (-1), och (0)")

    print("\n=== 3. annot (son) ===")
    print(f"  annot matnlari: {len(ax.texts) > 0}")

    print("\n=== 4. Qat'iy masshtab ===")
    print("  vmin=-1, vmax=1, center=0 (aldamas — 5.13)")
    plt.close(fig)
    print("  ⭐ Heatmap — rangli (naqsh)")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

=== 1. Heatmap ===
  matritsa: (3, 3)

=== 2. Rang talqini ===
  to'q qizil (+1), to'q ko'k (-1), och (0)

=== 3. annot (son) ===
  annot matnlari: True

=== 4. Qat'iy masshtab ===
  vmin=-1, vmax=1, center=0 (aldamas — 5.13)
  ⭐ Heatmap — rangli (naqsh)

Nima ko'rsatdi: 2.2-bo'lim.

Misol 3 — Multikollinearlik

"""Multikollinearlik (real pandas/numpy)."""

import numpy as np
import pandas as pd


def main() -> None:
    np.random.seed(0)
    n = 200
    maydon = np.random.normal(80, 20, n)
    xonalar = maydon / 20 + np.random.normal(0, 0.3, n)   # maydon bilan bog'liq
    narx = maydon * 1.5 + np.random.normal(0, 15, n)
    df = pd.DataFrame({"maydon": maydon, "xonalar": xonalar, "narx": narx})

    print("=== 1. Feature-feature (o'zaro) ===")
    r = df["maydon"].corr(df["xonalar"]).round(2)
    print(f"  maydon-xonalar: {r} (kuchli — o'zaro bog'liq)")

    print("\n=== 2. Multikollinearlik ===")
    print("  maydon va xonalar bir narsani o'lchaydi (ortiqcha)")

    print("\n=== 3. Ikkalasi target bilan ===")
    print(f"  maydon-narx: {df['maydon'].corr(df['narx']).round(2)}")
    print(f"  xonalar-narx: {df['xonalar'].corr(df['narx']).round(2)}")

    print("\n=== 4. Hal ===")
    print("  bittasini tashla (yoki PCA — 18-qism); model muammo (15)")
    print("  ⭐ Multikollinearlik — o'zaro bog'liq")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

=== 1. Feature-feature (o'zaro) ===
  maydon-xonalar: 0.97 (kuchli — o'zaro bog'liq)

=== 2. Multikollinearlik ===
  maydon va xonalar bir narsani o'lchaydi (ortiqcha)

=== 3. Ikkalasi target bilan ===
  maydon-narx: 0.89
  xonalar-narx: 0.88

=== 4. Hal ===
  bittasini tashla (yoki PCA — 18-qism); model muammo (15)
  ⭐ Multikollinearlik — o'zaro bog'liq

Nima ko'rsatdi: 2.4-bo'lim.

Misol 4 — Pearson vs Spearman (nochiziqli)

"""Pearson vs Spearman (real pandas/numpy)."""

import numpy as np
import pandas as pd


def main() -> None:
    np.random.seed(0)
    x = np.arange(1, 101)
    # monoton nochiziqli (o'suvchi, lekin egri)
    y = x ** 2 + np.random.normal(0, 100, 100)
    df = pd.DataFrame({"x": x, "y": y})

    print("=== 1. Pearson (chiziqli) ===")
    p = df["x"].corr(df["y"], method="pearson").round(2)
    print(f"  Pearson: {p}")

    print("\n=== 2. Spearman (tartib/monoton) ===")
    s = df["x"].corr(df["y"], method="spearman").round(2)
    print(f"  Spearman: {s} (yuqoriroq — monoton)")

    print("\n=== 3. Farq (nochiziqli) ===")
    print("  monoton egri: Pearson past, Spearman yuqori")

    print("\n=== 4. Qachon qaysi ===")
    print("  Pearson — chiziqli; Spearman — monoton/chidamli")
    print("  ⭐ Pearson vs Spearman (chiziqli/tartib)")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

=== 1. Pearson (chiziqli) ===
  Pearson: 0.96

=== 2. Spearman (tartib/monoton) ===
  Spearman: 1.0 (yuqoriroq — monoton)

=== 3. Farq (nochiziqli) ===
  monoton egri: Pearson past, Spearman yuqori

=== 4. Qachon qaysi ===
  Pearson — chiziqli; Spearman — monoton/chidamli
  ⭐ Pearson vs Spearman (chiziqli/tartib)

Nima ko'rsatdi: 2.5-bo'lim.

5. To'g'ri va noto'g'ri tushunishlar

6. Keng tarqalgan xatolar va yechimlari

1. Kategoriya

df.corr()   # kategoriya kod (ma'nosiz)                           # ⚠️
df.corr(numeric_only=True)   # son                               # ✅

2. Sababiyat

# to'q qizil 0.85-bob → sabab (xato)                                # ⚠️
# aloqa (sabab emas — 4.10)                                       # ✅

3. Nochiziqli

df.corr()   # nochiziqli → past (aloqa bor)                       # ⚠️
df.corr(method="spearman")   # yoki scatter 5.4-bob                # ✅

4. Multikollinearlik e'tiborsiz

# maydon+xonalar 0.9-bob modelga (koeffitsiyent buzuq)             # ⚠️
# bittasini tashla / PCA (18)                                    # ✅

5. Heatmap masshtab

sns.heatmap(korr)   # avtomatik masshtab (aldash)                # ⚠️
sns.heatmap(korr, vmin=-1, vmax=1, center=0)   # qat'iy          # ✅

6. Feature faqat corr

# feature tanlash faqat korrelyatsiya (chiziqli)                  # ⚠️
# boshqa usullar (nochiziqli, o'zaro — 19)                       # ✅

7. Diagonal e'tibor

# diagonal 1.0 → kuchli (o'zi — ma'nosiz)                         # ⚠️
# diagonaldan tashqari (haqiqiy aloqa)                           # ✅

7. Integratsiya — bu bilim qayerda kerak bo'ladi

• 4.9-dars (o'tilgan): Korrelyatsiya
• 5.8-dars (o'tilgan): Heatmap
• 8.4-dars (o'tilgan): Bivariate (juftlik)
• 19-qism (reja): Feature engineering (tanlash)
• 15-qism (reja): Regressiya (multikollinearlik)

8. Eng yaxshi amaliyotlar

1. Matritsa — df.corr(numeric_only=True).

2. Heatmap — coolwarm, vmin/vmax.

3. Feature-target — kuchli (muhim).

4. Multikollinearlik — o'zaro (model).

5. Nochiziqli — Spearman/scatter.

6. Aloqa ≠ sabab — 4.10.

7. Diagonal — e'tibor bermang 1.0-bob.

8. Kuchli juftlik — scatter (shakl).

9. Amaliy topshiriq

Vazifa 1: Bashorat qiling

1.  # korrelyatsiya matritsasi?
2.  # diagonal nima?
3.  # heatmap nima?
4.  # feature-target?
5.  # multikollinearlik?
6.  # Pearson?
7.  # Spearman?
8.  # kategoriya korrelyatsiya?
9.  # to'q qizil = sabab?
10. # nochiziqli corr?
11. # heatmap masshtab?
12. # nega barcha birga?

Vazifa 2: Xatolarni tuzating

1.  df.corr()   # kategoriya

2.  # to'q qizil → sabab

3.  df.corr()   # nochiziqli

4.  # maydon+xonalar 0.9-bob modelga

5.  sns.heatmap(korr)   # masshtab

1.  numeric_only=True

2.  aloqa (4.10)

3.  method="spearman" / scatter

4.  bittasini tashla (multikoll.)

5.  vmin=-1, vmax=1, center=0

Vazifa 3: Matritsa va heatmap

Modellang:

1. df.corr()
2. Diagonal
3. Heatmap
4. Rang

Vazifa 4: Feature va multikollinearlik

Modellang:

1. Feature-target
2. Kuchli/kuchsiz
3. Multikollinearlik
4. O'zaro

Vazifa 5: Tur

Modellang:

1. Pearson
2. Spearman
3. Chiziqli
4. Monoton

Vazifa 6: Integratsiya

Modellang:

1. Korrelyatsiya (4.9)
2. Heatmap (5.8)
3. Feature (19)
4. Regressiya (15)

Vazifa 7: O'ylash

Korrelyatsiya tahlil ikki savolga javob beradi: (1) qaysi xususiyat target bilan bog'liq (muhim feature), (2) qaysi xususiyatlar o'zaro bog'liq (multikollinearlik). Lekin korrelyatsiya faqat chiziqli aloqani o'lchaydi va sababni isbotlamaydi. Nima uchun korrelyatsiya "kuchli, lekin cheklangan vosita", va nima uchun uni faqat gipoteza generatori (dalil emas) sifatida ishlatish kerak?

Nimani mustahkamlaydi: 2.3, 2.7-bo'limlar.

Xulosa

Bu darsda korrelyatsiya tahlilni o'rgandik.

Eng muhim uch fikr:

1. Matritsa va heatmap. df.corr(numeric_only=True) — barcha son ustun juftligi korrelyatsiyasi (n×n; -1..+1; diagonal 1.0; simmetrik; 4.9). Heatmap — sns.heatmap(korr, annot=True, cmap="coolwarm", vmin=-1, vmax=1, center=0) (rangli — 5.8; to'q qizil kuchli musbat, to'q ko'k manfiy, och kuchsiz; naqsh bir qarashda).

2. Feature va multikollinearlik. Feature-target — target bilan xususiyat korrelyatsiyasi (korr["narx"].sort_values; kuchli — muhim feature maydon 0.85; kuchsiz — mustaqil id 0.02; model 19-qism). Multikollinearlik — feature-feature kuchli (maydon-xonalar 0.9 — o'zaro bog'liq; model muammo — regressiya koeffitsiyent 15-qism; tashla/PCA 18-qism).

3. Barcha aloqa. Pearson vs Spearman — Pearson (chiziqli), Spearman (tartib/monoton — nochiziqli egri; outlier chidamliroq). Korrelyatsiya — kuchli, lekin cheklangan vosita (chiziqli; sabab emas — 4.10); faqat gipoteza generatori (kashfiyot — dalil emas; eksperiment/scatter tekshir). Tuzoqlar: kategoriya (numeric_only), sababiyat 4.10-bob, nochiziqli (Pearson past — scatter/Spearman), diagonal (1.0 — ma'nosiz), heatmap masshtab (vmin/vmax — 5.13), multikollinearlik e'tiborsiz (15-qism), outlier (buzadi — 6.3).

Keyingi darsda anomaliya topishni o'rganamiz: g'ayrioddiy qiymatlar va naqshlar — outlier (IQR/z-score — 6.3), g'alati kombinatsiya 6.10-bob, naqshdan chetlanish; xato yoki muhim kashfiyot (firibgarlik, buzilish).