IlmHamroh
Data Science va sun'iy intellekt/EDA3/10-dars17 daqiqa
Mundarija (22)

8.3-dars: Bir o'zgaruvchi tahlil (univariate)

8-QISM — EDA (RAZVEDKA TAHLILI) · 3-dars

1. Kirish va motivatsiya

EDA'ning birinchi bosqichi — har ustunni alohida o'rganish. Bu univariate (bir o'zgaruvchi) tahlil: bitta ustun haqida hamma narsani bilish. Son ustun uchun: qanday taqsimlangan (shakl), markazi qayerda (o'rtacha/median), qanchalik tarqoq (std/IQR), outlier bormi. Kategoriya ustun uchun: qaysi qiymatlar, qaysi eng ko'p (value_counts), muvozanatli mi. Bu "har ustun haqida hikoya" — ustunni chuqur tushunish, keyingi (bivariate) tahlilga asos. Bu darsda son va kategoriya ustunlarni univariate tahlil qilish: statistika (describe) + grafik (histogram, box, bar). Nega muhim? (1) Har ustun — alohida tushunish (asos); (2) Shakl — taqsimot (normal? cho'zilgan?); (3) Muammo — outlier, muvozanatsizlik. Bu dars univariate tahlilni o'rgatadi — har ustun haqida.

Bir o'zgaruvchi tahlil (univariate) — har ustun alohida: son ustun (taqsimot shakli — histogram/KDE 5.5; markaz — o'rtacha/median 4.2; tarqoqlik — std/IQR 4.3; outlier — box 5.6), kategoriya ustun (value_counts — qaysi qiymat, eng ko'p; bar grafik; muvozanat), shakl (normal/o'ngga cho'zilgan/bimodal — 5.9), describe (son statistika), hikoya (har ustun haqida). Foydalanish: har ustunni tushunish, taqsimot/muammo. Bu 4.2/4.3 (markaz/tarqoqlik), 5.5/5.6 (histogram/box), 8.4 (bivariate) bilan bog'liq. Bir o'zgaruvchi — har ustun (univariate). Univariate. Ustun.

Real vaziyat. Data Scientist uy narxi ma'lumotini EDA qilmoqchi — har ustundan boshlaydi (univariate): (1) narx (son) — histogram (o'ngga cho'zilgan — ko'p arzon, kam qimmat; median 120, o'rtacha 135 — outlier tortdi; box — bir necha outlier), (2) maydon (son) — normal taqsimot (o'rtacha 80 m2), (3) shahar (kategoriya) — value_counts (Toshkent 40%, Samarqand 30% — muvozanatli), (4) holat (kategoriya) — value_counts (yaxshi 60%, o'rtacha 30% — nomuvozanat?). Data Scientist har ustun haqida bildi (narx cho'zilgan, shahar muvozanatli), keyingi tahlilga tayyorlandi. Bir o'zgaruvchi — har ustun haqida.

Bu darsda univariate tahlilni o'rganamiz.

Bu darsda:

  • Son ustun (taqsimot, markaz, tarqoqlik)
  • Kategoriya ustun (value_counts, bar)
  • Taqsimot shakli (normal/cho'zilgan)
  • Outlier va muammo
  • Muvozanat (kategoriya)
  • Univariate amaliyoti
  • Univariate tuzoqlari
  • Amaliy: univariate tahlil

ℹ Misollar real pandas/numpy bilan (Python 3.14).

2. Nazariya — chuqur tushuntirish

2.1. Son ustun (taqsimot, markaz, tarqoqlik)

Son ustunni tahlil:

python
import pandas as pd

# SON USTUN — univariate:
df["narx"].describe()    # markaz (mean/50%), tarqoqlik (std, min/max)
df["narx"].median()      # median (outlierga chidamli — 4.2)
df["narx"].skew()        # egrilik (o'ngga/chapga cho'zilgan)

# grafik: histogram (taqsimot), box (outlier)
df["narx"].hist(bins=30)
df["narx"].plot.box()

Son ustun (taqsimot, markaz, tarqoqlik) — univariate: son ustuni alohida — markaz (describe mean, median 4.2 — outlierga chidamli), tarqoqlik (std, IQR — 4.3; min/max), shakl (skew egrilik 5.5; hist histogram — taqsimot), outlier (box — 5.6; IQR). Sabab: son ustun taqsimot (qanday tarqalgan — shakl; markaz qayerda; qanchalik tarqoq; outlier); "ustun haqida hikoya" (markaz + tarqoqlik + shakl + outlier). describe (markaz/tarqoqlik), median (chidamli), hist (shakl), box (outlier). Son ustun — taqsimot/markaz/tarqoqlik. Son. Taqsimot.

2.2. Kategoriya ustun (value_counts, bar)

Kategoriya ustunni tahlil:

python
import pandas as pd

# KATEGORIYA USTUN — univariate:
df["shahar"].value_counts()             # har qiymat soni
df["shahar"].value_counts(normalize=True)  # foiz (nisbat)
df["shahar"].nunique()                  # noyob qiymat soni

# grafik: bar (kategoriya taqsimoti)
df["shahar"].value_counts().plot.bar()

Kategoriya ustun (value_counts, bar) — univariate: kategoriya ustuni alohida — value_counts() (har qiymat soni; normalize=True — foiz), nunique() (noyob qiymat soni), bar grafik (kategoriya taqsimoti — 5.3), muvozanat (guruhlar teng?). Sabab: kategoriya ustun taqsimot (qaysi qiymat, qaysi eng ko'p, muvozanatlimi); son emas — value_counts (sanoq/foiz; describe emas). value_counts (sanoq), normalize (foiz), nunique (noyob), bar (grafik). Kategoriya ustun — value_counts/bar (sanoq). Kategoriya. value_counts.

2.3. Taqsimot shakli (normal/cho'zilgan)

Taqsimot shakli (normal/cho'zilgan) — muhim: son ustun shakli — normal (simmetrik qo'ng'iroq — 4.6; o'rtacha=median), o'ngga cho'zilgan (right-skewed — quyruq o'ngda; daromad, narx — ko'p arzon, kam qimmat; o'rtacha > median; 5.5), chapga cho'zilgan (left-skewed — quyruq chapda), bimodal (ikki cho'qqi — ikki guruh; 5.9), bir tekis (uniform). Sabab: shakl — tahlil/model uchun muhim (normal — o'rtacha ma'noli; cho'zilgan — median; log 6.5; bimodal — ikki guruh ajrat); skew (egrilik — musbat o'ngga, manfiy chapga). Normal (simmetrik), cho'zilgan (skew — median), bimodal (ikki guruh). Taqsimot shakli — normal/cho'zilgan (skew). Shakl. Skew.

2.4. Outlier va muammo

Outlier va muammo (univariate) — topish: univariate — muammolar topiladi: outlier (box/IQR — 5.6/6.3; chetki qiymat — xato yoki haqiqiy), NaN (isna — 6.1; yetishmayotgan), noto'g'ri (imkonsiz — yosh 250; 6.10), g'alati taqsimot (barcha bir xil — o'zgarmas; yoki bimodal — ikki guruh). Sabab: univariate — muammoni ko'rsatadi (har ustun — outlier/NaN/xato; tozalash 6-qism yoki e'tibor); EDA'da muammo topish (modeldan oldin). Outlier (box/IQR), NaN (isna), noto'g'ri (domen), g'alati (o'zgarmas/bimodal). Outlier va muammo — topish (univariate). Outlier. Muammo.

2.5. Muvozanat (kategoriya)

Muvozanat (kategoriya) — guruhlar teng?: kategoriya ustun muvozanat — guruhlar tengmi (value_counts — Toshkent 40%, Samarqand 30% — muvozanatli; yoki 95% A, 5% B — nomuvozanat); nomuvozanat (imbalance) — muammo (model — kam guruh o'rganmaydi; klassifikatsiya 16-qism — nomuvozanatli sinf). Sabab: muvozanat — model/tahlil uchun (nomuvozanat — kam guruh e'tiborsiz; masalan firibgarlik 1% — model "hammasi toza" deb 99% aniq, lekin foydasiz); EDA'da muvozanat ko'r (kam guruh — ehtiyot). Muvozanatli (teng), nomuvozanat (95/5 — muammo), kam guruh (model e'tiborsiz). Muvozanat — kategoriya (teng?; nomuvozanat). Muvozanat. Nomuvozanat.

2.6. Univariate amaliyoti

Univariate amaliyoti: har ustun (birma-bir — son va kategoriya); son (describe markaz/tarqoqlik; median/skew; hist shakl; box outlier); kategoriya (value_counts(normalize) sanoq/foiz; nunique; bar; muvozanat); muammo (outlier/NaN/noto'g'ri — 6-qism); shakl (normal/cho'zilgan/bimodal — 5.9); hikoya (har ustun — xulosa); keyingi (bivariate — 8.4). Tuzoqlar: grafiksiz (raqam — Anscombe), o'rtacha cho'zilganda (median), muvozanat e'tiborsiz (nomuvozanat), bimodal o'tkazish (ikki guruh), NaN e'tiborsiz (taqsimot buziladi). Amaliyot — har ustun, son, kategoriya, shakl. Univariate. Ustun.

2.7. Univariate tuzoqlari

Univariate asosiy tuzoqlari: grafiksiz (faqat describe (son) — shakl yo'q (o'ngga cho'zilgan? bimodal? — Anscombe/Datasaurus 5.4; histogram kerak)); o'rtacha cho'zilganda (o'ngga cho'zilgan (narx) — o'rtacha outlier/quyruq tortadi (o'rtacha > median; noto'g'ri markaz); median 4.2); muvozanat e'tiborsiz (kategoriya — nomuvozanat (95/5) e'tiborsiz → model kam guruh o'rganmaydi (klassifikatsiya 16-qism); value_counts ko'r); bimodal o'tkazish (ikki cho'qqi (ikki guruh — 5.9) — o'rtacha ma'nosiz (guruhlar orasida); histogram/KDE ko'r, ajrat); NaN e'tiborsiz (NaN — describe o'tkazib yuboradi (jimda; taqsimot noto'g'ri ko'rinadi); isna tekshir — 6.1); kategoriya son deb (kod 1,2,3 (kategoriya) — describe (o'rtacha 2 — ma'nosiz); value_counts — kategoriya); bin soni (histogram — bins (kam — silliq; ko'p — shovqin; 5.5); mos tanla); noyob ko'p (kategoriya — 1000 noyob (id) — value_counts foydasiz; noyob — id/matn, kategoriya emas); taqsimot faraz (normal deb faraz — tekshir (skew/hist); ko'p real — cho'zilgan). Sabab: univariate shakl/markaz/muvozanat nozik (grafiksiz, o'rtacha, muvozanat — noto'g'ri). Yechim: histogram, median (cho'zilgan), muvozanat ko'r, bimodal ajrat, NaN tekshir. Tuzoqlar — grafiksiz, o'rtacha, muvozanat, bimodal.

2.8. Bir o'zgaruvchi tahlil — har ustun haqida hikoya

Univariate asosiy g'oyasi — har ustun haqida hikoya: EDA'ning birinchi bosqichi — har ustunni alohida o'rganish (univariate — bir o'zgaruvchi; ustun haqida hamma narsa); keyingi (bivariate — 8.4) uchun asos. Son ustun (taqsimot — hist/skew shakl 5.5; markaz — describe/median 4.2; tarqoqlik — std/IQR 4.3; outlier — box 5.6), kategoriya ustun (value_counts(normalize) sanoq/foiz; nunique; bar 5.3; muvozanat), taqsimot shakli (normal/o'ngga cho'zilgan/bimodal — 5.9; skew), outlier va muammo (outlier/NaN/noto'g'ri — 6-qism topish), muvozanat (kategoriya — teng? nomuvozanat — model; 16-qism). Foydalanish: har ustunni tushunish (taqsimot, markaz, muammo), muammo topish (outlier/NaN/nomuvozanat — modeldan oldin). "Har ustun haqida hikoya" (markaz + tarqoqlik + shakl + outlier/muammo). Tuzoqlar: grafiksiz (Anscombe — histogram), o'rtacha cho'zilganda (median), muvozanat e'tiborsiz (nomuvozanat), bimodal o'tkazish (ikki guruh), NaN e'tiborsiz (isna), kategoriya son deb (value_counts). Bu 4.2/4.3 (markaz/tarqoqlik), 5.5/5.6/5.9 (histogram/box/taqsimot), 6-qism (muammo), 8.4 (bivariate) bilan. Bir o'zgaruvchi — har ustun haqida hikoya (univariate; taqsimot/markaz/muammo). Univariate. Ustun. Hikoya.

3. Tez ma'lumotnoma

python
import pandas as pd

# SON USTUN (univariate):
df["narx"].describe()           # markaz (mean/50%), tarqoqlik (std/min/max)
df["narx"].median()             # median (outlierga chidamli — 4.2)
df["narx"].skew()               # egrilik (>0 o'ngga, <0 chapga cho'zilgan)
df["narx"].hist(bins=30)        # taqsimot shakli (5.5)
df["narx"].plot.box()           # outlier (5.6)

# KATEGORIYA USTUN (univariate):
df["shahar"].value_counts()               # har qiymat soni
df["shahar"].value_counts(normalize=True) # foiz (muvozanat)
df["shahar"].nunique()                    # noyob soni
df["shahar"].value_counts().plot.bar()    # bar (5.3)

# MUAMMO (univariate topadi):
df["x"].isna().sum()            # NaN (6.1)
# outlier (box/IQR — 6.3) · noto'g'ri (domen — 6.10)

# SHAKL: normal · o'ngga cho'zilgan (median) · bimodal (ikki guruh)
QOIDA: histogram (Anscombe) · skewda median · muvozanat · NaN tekshir

Univariate xulosasi

Bir o'zgaruvchi tahlil — har ustun haqida hikoya (univariate)
Son ustun — taqsimot (hist/skew), markaz (median), tarqoqlik, outlier (box)
Kategoriya ustun — value_counts (sanoq/foiz), bar, muvozanat
Taqsimot shakli — normal / o'ngga cho'zilgan / bimodal (5.9)
Muammo — outlier/NaN/noto'g'ri (topish — modeldan oldin)

4. Batafsil misollar

Misollar real pandas/numpy bilan (Python 3.14).

Misol 1 — Son ustun (taqsimot, markaz)

python
"""Son ustun univariate (real pandas/numpy)."""

import numpy as np
import pandas as pd


def main() -> None:
    np.random.seed(0)
    # o'ngga cho'zilgan (narx — ko'p arzon, kam qimmat)
    narx = np.concatenate([np.random.normal(100, 20, 900),
                           np.random.normal(300, 50, 100)])
    s = pd.Series(narx)

    print("=== 1. describe (markaz/tarqoqlik) ===")
    print(f"  o'rtacha: {s.mean().round(1)}, median: {s.median().round(1)}")
    print(f"  std: {s.std().round(1)}, min-max: {s.min().round(0)}-{s.max().round(0)}")

    print("\n=== 2. O'rtacha vs median (cho'zilgan) ===")
    print(f"  mean ({s.mean().round(0)}) > median ({s.median().round(0)})")
    print("  → o'ngga cho'zilgan (outlier tortdi)")

    print("\n=== 3. skew (egrilik) ===")
    print(f"  skew: {s.skew().round(2)} (>0 — o'ngga cho'zilgan)")

    print("\n=== 4. Hikoya ===")
    print("  narx o'ngga cho'zilgan (median afzal); yuqori quyruq")
    print("  ⭐ Son ustun — taqsimot, markaz")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. describe (markaz/tarqoqlik) ===
  o'rtacha: 119.6, median: 101.0
  std: 67.3, min-max: 39.0-438.0

=== 2. O'rtacha vs median (cho'zilgan) ===
  mean 120.0-bob > median 101.0-bob
  → o'ngga cho'zilgan (outlier tortdi)

=== 3. skew (egrilik) ===
  skew: 2.53 (>0 — o'ngga cho'zilgan)

=== 4. Hikoya ===
  narx o'ngga cho'zilgan (median afzal); yuqori quyruq
  ⭐ Son ustun — taqsimot, markaz

Nima ko'rsatdi: 2.1, 2.3-bo'limlar.

Misol 2 — Kategoriya ustun (value_counts)

python
"""Kategoriya ustun univariate (real pandas/numpy)."""

import numpy as np
import pandas as pd


def main() -> None:
    np.random.seed(0)
    shahar = np.random.choice(["Toshkent", "Samarqand", "Buxoro", "Xiva"],
                              1000, p=[0.4, 0.3, 0.2, 0.1])
    s = pd.Series(shahar)

    print("=== 1. value_counts (sanoq) ===")
    print(s.value_counts().to_string())

    print("\n=== 2. Foiz (normalize) ===")
    foiz = (s.value_counts(normalize=True) * 100).round(1)
    print(foiz.to_string())

    print("\n=== 3. nunique (noyob) ===")
    print(f"  noyob shaharlar: {s.nunique()}")

    print("\n=== 4. Muvozanat ===")
    print("  Toshkent 40%, Xiva 10% — nisbatan muvozanatli")
    print("  ⭐ Kategoriya — value_counts (sanoq/foiz)")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. value_counts (sanoq) ===
Toshkent     415
Samarqand    294
Buxoro       183
Xiva         108

=== 2. Foiz (normalize) ===
Toshkent     41.5
Samarqand    29.4
Buxoro       18.3
Xiva         10.8

=== 3. nunique (noyob) ===
  noyob shaharlar: 4

=== 4. Muvozanat ===
  Toshkent 40%, Xiva 10% — nisbatan muvozanatli
  ⭐ Kategoriya — value_counts (sanoq/foiz)

Nima ko'rsatdi: 2.2-bo'lim.

Misol 3 — Muvozanat (nomuvozanat)

python
"""Muvozanat: nomuvozanat (real pandas/numpy)."""

import numpy as np
import pandas as pd


def main() -> None:
    np.random.seed(0)
    # nomuvozanat (firibgarlik — kam)
    firibgarlik = np.random.choice(["toza", "firibgar"], 1000, p=[0.97, 0.03])
    s = pd.Series(firibgarlik)

    print("=== 1. value_counts (nomuvozanat) ===")
    print(s.value_counts().to_string())

    print("\n=== 2. Foiz ===")
    foiz = (s.value_counts(normalize=True) * 100).round(1)
    print(f"  {foiz.to_dict()}")

    print("\n=== 3. Muammo (nomuvozanat) ===")
    print("  firibgar 3% — model 'hammasi toza' → 97% aniq (foydasiz)")

    print("\n=== 4. E'tibor (16-qism) ===")
    print("  nomuvozanat — kam guruhga e'tibor (klassifikatsiya)")
    print("  ⭐ Muvozanat — nomuvozanat (muammo)")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. value_counts (nomuvozanat) ===
toza        969
firibgar     31

=== 2. Foiz ===
  {'toza': 96.9, 'firibgar': 3.1}

=== 3. Muammo (nomuvozanat) ===
  firibgar 3% — model 'hammasi toza' → 97% aniq (foydasiz)

=== 4. E'tibor (16-qism) ===
  nomuvozanat — kam guruhga e'tibor (klassifikatsiya)
  ⭐ Muvozanat — nomuvozanat (muammo)

Nima ko'rsatdi: 2.5-bo'lim.

Misol 4 — Muammo topish (NaN, kategoriya son)

python
"""Muammo topish (real pandas/numpy)."""

import numpy as np
import pandas as pd


def main() -> None:
    df = pd.DataFrame({
        "narx": [100, 200, np.nan, 300, 150],
        "mahalla_kod": [1, 2, 1, 3, 2],   # kategoriya (kod)
    })

    print("=== 1. NaN (describe o'tkazadi) ===")
    print(f"  describe count: {int(df['narx'].describe()['count'])} (5 dan 4)")
    print(f"  NaN: {int(df['narx'].isna().sum())} (isna tekshir)")

    print("\n=== 2. Kategoriya son deb (xato) ===")
    print(f"  mahalla_kod o'rtacha: {df['mahalla_kod'].mean()} (ma'nosiz!)")

    print("\n=== 3. To'g'ri (value_counts) ===")
    print(f"  mahalla_kod sanoq: {df['mahalla_kod'].value_counts().to_dict()}")

    print("\n=== 4. Muammo topildi ===")
    print("  NaN (isna) + kategoriya son (value_counts)")
    print("  ⭐ Univariate — muammo topish")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

text
=== 1. NaN (describe o'tkazadi) ===
  describe count: 4 (5 dan 4)
  NaN: 1 (isna tekshir)

=== 2. Kategoriya son deb (xato) ===
  mahalla_kod o'rtacha: 1.8 (ma'nosiz!)

=== 3. To'g'ri (value_counts) ===
  mahalla_kod sanoq: {1: 2, 2: 2, 3: 1}

=== 4. Muammo topildi ===
  NaN (isna) + kategoriya son (value_counts)
  ⭐ Univariate — muammo topish

Nima ko'rsatdi: 2.4, 2.7-bo'limlar.

5. To'g'ri va noto'g'ri tushunishlar

Noto'g'ri fikr To'g'risi
"describe yetarli" Grafik (histogram — shakl)
"o'rtacha markaz" Cho'zilganda median
"kategoriya son" value_counts (mean emas)
"muvozanat muhim emas" Nomuvozanat (model)
"bimodal o'rtacha OK" Ikki guruh (ajrat)
"NaN describe'da" isna (o'tkazadi)
"univariate yetarli" Bivariate (aloqa — 8.4)
"har ustun bir xil" Har ustun hikoya

6. Keng tarqalgan xatolar va yechimlari

1. Grafiksiz

python
df["narx"].describe()   # shakl yo'q (Anscombe)                   # ⚠️
df["narx"].hist(bins=30)   # taqsimot shakli                     # ✅

2. O'rtacha cho'zilganda

python
df["narx"].mean()   # outlier tortadi (cho'zilgan)                # ⚠️
df["narx"].median()   # chidamli 4.2-bob                           # ✅

3. Kategoriya son deb

python
df["mahalla_kod"].mean()   # kod o'rtacha (ma'nosiz)              # ⚠️
df["mahalla_kod"].value_counts()   # kategoriya                  # ✅

4. Muvozanat e'tiborsiz

python
# nomuvozanat (95/5) e'tiborsiz (model xato)                      # ⚠️
df["y"].value_counts(normalize=True)   # muvozanat ko'r          # ✅

5. Bimodal o'tkazish

python
df["boy"].mean()   # bimodal (ikki guruh — o'rtacha ma'nosiz)     # ⚠️
df["boy"].hist()   # ikki cho'qqi ko'r (ajrat — 5.9)             # ✅

6. NaN e'tiborsiz

python
df["narx"].describe()   # NaN o'tkazadi (taqsimot noto'g'ri)      # ⚠️
df["narx"].isna().sum()   # NaN tekshir 6.1-bob                    # ✅

7. Noyob ko'p (kategoriya)

python
df["id"].value_counts()   # 1000 noyob (foydasiz)                 # ⚠️
# id — kategoriya emas (noyob identifikator)                     # ✅

7. Integratsiya — bu bilim qayerda kerak bo'ladi

  • 4.2/4.3-dars (o'tilgan): Markaz/tarqoqlik
  • 5.5/5.6/5.9-dars (o'tilgan): Histogram/box/taqsimot
  • 6-qism (o'tilgan): Tozalash (muammo)
  • 8.4-dars: Ikki o'zgaruvchi (bivariate)
  • 16-qism (reja): Klassifikatsiya (nomuvozanat)

8. Eng yaxshi amaliyotlar

  1. Har ustun — alohida (univariate).

  2. Son — describe + histogram (shakl).

  3. Cho'zilgan — median (o'rtacha emas).

  4. Kategoriya — value_counts (bar).

  5. Muvozanat — ko'r (nomuvozanat).

  6. Muammo — outlier/NaN/noto'g'ri.

  7. Bimodal — ajrat (ikki guruh).

  8. Hikoya — har ustun (xulosa).

9. Amaliy topshiriq

Vazifa 1: Bashorat qiling

python
1.  # univariate nima?
2.  # son ustun tahlili?
3.  # kategoriya ustun?
4.  # value_counts?
5.  # taqsimot shakli?
6.  # o'ngga cho'zilgan?
7.  # skew nima?
8.  # muvozanat?
9.  # nomuvozanat muammosi?
10. # NaN va describe?
11. # kategoriya son?
12. # nega har ustun?
Javoblar
  1. Har ustun alohida (bir o'zgaruvchi)
  2. describe, median, hist, box
  3. value_counts, bar
  4. Har qiymat soni/foiz
  5. Normal/cho'zilgan/bimodal
  6. Quyruq o'ngda (mean > median)
  7. Egrilik (>0 o'ngga)
  8. Guruhlar teng?
  9. Kam guruh (model o'rganmaydi)
  10. describe o'tkazadi (isna)
  11. value_counts (mean emas)
  12. Alohida tushunish (asos)

Vazifa 2: Xatolarni tuzating

python
1.  df["narx"].describe()   # shakl

2.  df["narx"].mean()   # cho'zilgan

3.  df["mahalla_kod"].mean()

4.  # nomuvozanat e'tiborsiz

5.  df["narx"].describe()   # NaN
Javoblar
python
1.  df["narx"].hist() (grafik)

2.  df["narx"].median()

3.  value_counts() (kategoriya)

4.  value_counts(normalize=True)

5.  isna().sum() (NaN)

Vazifa 3: Son ustun

Modellang:

  1. describe
  2. median
  3. Histogram
  4. Box (outlier)

Vazifa 4: Kategoriya ustun

Modellang:

  1. value_counts
  2. Foiz
  3. Bar
  4. Muvozanat

Vazifa 5: Shakl va muammo

Modellang:

  1. Normal/cho'zilgan
  2. Skew
  3. Outlier
  4. NaN

Vazifa 6: Integratsiya

Modellang:

  1. Markaz (4.2)
  2. Histogram (5.5)
  3. Tozalash (6)
  4. Bivariate (8.4)

Vazifa 7: O'ylash

Univariate tahlil "har ustun haqida hikoya" — lekin ustunni faqat o'rtacha bilan tavsiflash yetarli emas (o'ngga cho'zilgan taqsimotda o'rtacha aldaydi, bimodalda ma'nosiz). Nima uchun taqsimot shaklini ko'rish (bir son emas) univariate tahlilning mohiyati, va nima uchun turli ustunlar turli tavsif talab qiladi?

Javob

Qisqa javob: Univariate "har ustun hikoya", lekin faqat o'rtacha yetmaydi (cho'zilgan — aldaydi; bimodal — ma'nosiz); taqsimot shaklini ko'rish (bir son emas) univariate mohiyati; turli ustun turli tavsif, chunki: (1) bir son yashiradi — o'rtacha (bir son) — taqsimot shaklini yo'qotadi (cho'zilgan — mean quyruq tortadi; bimodal — mean guruhlar orasida; 5.9); "shakl — hikoya" (bir son emas); (2) shakl — ma'no (normal — o'rtacha ma'noli; cho'zilgan — median; bimodal — ikki guruh; shakl tavsif usulini belgilaydi); (3) turli ustun — son (taqsimot, markaz, tarqoqlik — describe/hist), kategoriya (sanoq, muvozanat — value_counts); tur tavsifni belgilaydi (son ≠ kategoriya); (4) grafik (Anscombe/Datasaurus — bir statistika, turli shakl; grafik shart — 5.4). "Nega shakl mohiyat": (a) taqsimot hikoya (shakl — ustun haqida; markaz + tarqoqlik + shakl + outlier); (b) bir son aldaydi (cho'zilgan — mean; bimodal — o'rtacha; shakl ko'rmasa — noto'g'ri); (c) usul (shakl — markaz usuli (mean/median), model (normal faraz?)); (d) muammo (shakl — outlier, bimodal, o'zgarmas — ko'rinadi). "Nega turli ustun turli tavsif": (1) tur (son — describe/hist; kategoriya — value_counts/bar; tur mos); (2) shakl (normal — mean/std; cho'zilgan — median/IQR; shakl mos); (3) ma'no (kategoriya son emas — sanoq; son — markaz); (4) savol (son — "qanday tarqalgan?"; kategoriya — "qaysi ko'p?"). "Data Scientist qanday": (1) tur (son/kategoriya — usul); (2) shakl (histogram — normal/cho'zilgan/bimodal); (3) markaz (shaklga — mean/median); (4) muammo (outlier/NaN/nomuvozanat); (5) hikoya (har ustun — to'liq tavsif). Saboqlar: bir son yashiradi (shakl — hikoya); shakl mohiyat (bir son emas — grafik); turli ustun turli tavsif (son/kategoriya; normal/cho'zilgan); tur/shakl usulni. To'g'ri: taqsimot shakli (bir son emas — mohiyat); turli ustun (tur/shakl — tavsif); grafik (Anscombe). Muvozanat: son (qisqa — mean) + shakl (to'liq — histogram; hikoya). Bu Data Science EDA asosiy (shakl — hikoya; bir son aldaydi; turli ustun turli tavsif). Univariate — har ustun hikoya (shakl mohiyat; turli ustun turli tavsif).

1. Nega shakl mohiyat

  • Bir son yashiradi (mean — cho'zilgan/bimodal aldaydi)
  • Taqsimot hikoya (markaz+tarqoqlik+shakl+outlier)
  • Usul (shakl — markaz/model belgilaydi)
  • Grafik (Anscombe — bir statistika, turli shakl)

2. Nega turli ustun turli tavsif

  • Tur (son — describe; kategoriya — value_counts)
  • Shakl (normal — mean; cho'zilgan — median)
  • Ma'no (kategoriya son emas — sanoq)
  • Savol (son "tarqalgan?"; kategoriya "ko'p?")

3. Bir son vs shakl

Bir son (mean) Shakl (histogram)
Yashiradi (cho'zilgan) Ko'rsatadi (hikoya)
Aldaydi (bimodal) Ikki guruh
Qisqa To'liq tavsif

4. Data Scientist qanday

  1. Tur (son/kategoriya — usul)
  2. Shakl (histogram — normal/cho'zilgan)
  3. Markaz (shaklga — mean/median)
  4. Muammo (outlier/NaN/nomuvozanat)

5. Xulosa

  1. Bir son yashiradi (mean — cho'zilgan/bimodal aldaydi)
  2. Shakl mohiyat (bir son emas — histogram; hikoya)
  3. Turli ustun turli tavsif (son/kategoriya; normal/cho'zilgan)
  4. Tur/shakl usulni belgilaydi (grafik — Anscombe)

Nimani mustahkamlaydi: 2.3, 2.7-bo'limlar.

Xulosa

Bu darsda univariate tahlilni o'rgandik.

Eng muhim uch fikr:

  1. Son va kategoriya ustun. Son ustun — taqsimot (hist/skew shakl — 5.5), markaz (describe/median — 4.2; cho'zilganda median), tarqoqlik (std/IQR — 4.3), outlier (box — 5.6). Kategoriya ustun — value_counts(normalize=True) (sanoq/foiz), nunique (noyob), bar 5.3-bob, muvozanat.

  2. Shakl va muammo. Taqsimot shakli — normal (simmetrik — mean=median), o'ngga cho'zilgan (quyruq o'ngda — mean>median; median afzal), bimodal (ikki cho'qqi — ikki guruh; 5.9); skew (egrilik). Outlier va muammo — univariate topadi (outlier box/IQR — 6.3; NaN isna — 6.1; noto'g'ri — 6.10; nomuvozanat). Muvozanat — kategoriya (teng? nomuvozanat 95/5 — model kam guruh; 16-qism).

  3. Har ustun hikoya. Univariate — har ustun haqida hikoya (markaz + tarqoqlik + shakl + outlier/muammo; keyingi bivariate 8.4 uchun asos). Taqsimot shaklini ko'rish (bir son emas — mohiyat; mean aldaydi cho'zilgan/bimodalda; grafik Anscombe); turli ustun turli tavsif (son/kategoriya; tur/shakl usulni). Tuzoqlar: grafiksiz (histogram), o'rtacha cho'zilganda (median), kategoriya son deb (value_counts), muvozanat e'tiborsiz (nomuvozanat), bimodal o'tkazish (ajrat), NaN e'tiborsiz (isna), noyob ko'p (id — kategoriya emas).

Keyingi darsda ikki o'zgaruvchi tahlil (bivariate)ni o'rganamiz: ikki ustun orasidagi aloqa — son-son (scatter — 5.4), kategoriya-son (guruh box — 5.6), kategoriya-kategoriya (crosstab); "ustunlar qanday bog'liq".

Ulashish:Telegram'da

Izohlar (0)

Izoh yozish uchun kiring.

  • Hozircha izoh yo'q. Birinchi bo'ling!
8.3-dars: Bir o'zgaruvchi tahlil (univariate) — IlmHamroh