3.14-dars: Pandas amaliy loyiha

3-QISM — PANDAS · 14-dars (yakuniy)

1. Kirish va motivatsiya

3-qism davomida Pandas'ning barcha qismlarini o'rgandik: Series, DataFrame, o'qish/yozish, tanlash, filtrlash, ustunlar, NaN, guruhlash, birlashtirish, saralash, apply, vaqt qatorlari, pivot. Endi bularni birlashtirib, real bir tahlil loyihasini boshidan oxirigacha bajaramiz. Loyiha: do'kon savdo tahlili — CSV o'qish, tozalash (NaN, tur), boyitish (yangi ustunlar), filtrlash, guruhlash (shahar/kategoriya bo'yicha), birlashtirish (mijoz ma'lumoti), va xulosa. Bu — Data Science ish oqimining (CRISP-DM) markaziy qismi: xom ma'lumot → toza ma'lumot → tahlil → xulosa. Nega muhim? (1) Sintez — alohida ko'nikmalar birga; (2) Real oqim — Data Science ishining Pandas qismi; (3) Mustahkamlash — hamma narsani amalda. Bu dars 3-qismni yakunlaydi va statistika/vizualizatsiyaga tayyorlaydi.

Pandas amaliy loyiha — butun Pandas bilimini birlashtirish: o'qish (read_csv — 3.3), tozalash (NaN — 3.7, tur — 3.6), boyitish (yangi ustun — 3.6), filtrlash 3.5-bob, guruhlash (groupby — 3.8), birlashtirish (merge — 3.9), saralash (sort_values — 3.10), xulosa (hisobot). Foydalanish: real tahlil, sintez, mustahkamlash. Bu butun 3-qism (3.1-3.13) yig'indisi va 4-qism (statistika), 5-qism (vizualizatsiya) ga zamin. Loyiha — sintez. Real oqim. Yakun.

Real vaziyat. Data Scientist do'kon savdo tahlilini so'radi. Butun Pandas bilimi ishga tushdi: o'qish (read_csv — savdo ma'lumoti); tozalash (NaN to'ldirish — 3.7, tur tuzatish — 3.6); boyitish (df["summa"] = df["narx"] * df["miqdor"] — 3.6); filtr (2024 yil — 3.5); guruh (shahar bo'yicha jami — groupby 3.8); birlashtirish (mijoz shahri — merge 3.9); saralash (top mahsulot — nlargest 3.10); xulosa (hisobot — jami savdo, eng ko'p shahar/mahsulot). Alohida ko'nikmalar birga real tahlil berdi (savdo hisoboti). Loyiha — Pandas sintezi (butun bilim bir tahlilda). 3-qism yakuni.

Bu darsda butun Pandas bilimini loyihada qo'llaymiz.

Bu darsda:

• Loyiha rejasi
• O'qish va tozalash
• Boyitish va filtrlash
• Guruhlash va birlashtirish
• Tahlil va xulosa
• Loyiha amaliyoti
• Loyiha tuzoqlari
• Amaliy: to'liq tahlil

ℹ Misollar real pandas bilan (deterministik) ishlaydi.

2. Nazariya — chuqur tushuntirish

2.1. Loyiha rejasi

Loyiha rejasi — do'kon savdo tahlili (bosqichma-bosqich):

LOYIHA: Do'kon savdo tahlili

1. O'QISH (read_csv) — 3.3
2. TOZALASH (NaN, tur) — 3.6, 3.7
3. BOYITISH (yangi ustun — summa) — 3.6
4. FILTRLASH (kerakli qism) — 3.5
5. GURUHLASH (shahar/kategoriya) — 3.8
6. BIRLASHTIRISH (mijoz ma'lumoti) — 3.9
7. SARALASH (reyting) — 3.10
8. XULOSA (hisobot)

   → xom ma'lumot → toza → tahlil → xulosa (CRISP-DM)

Sabab: real loyiha bosqichlarga (o'qish → tozalash → boyitish → tahlil → xulosa); har bosqich Pandas ko'nikmasi; reja tartib (nima qilish — chalkashmaslik). Bu Data Science ish oqimi (CRISP-DM — 1.4) Pandas qismi (ma'lumot tayyorlash — vaqtning 80%). Loyiha rejasi — bosqichma-bosqich (o'qish → xulosa). Tartib. Sintez.

2.2. O'qish va tozalash

O'qish va tozalash — birinchi bosqichlar:

import pandas as pd

# 1. O'QISH
df = pd.read_csv("savdo.csv")
df.head(); df.info()          # tekshir (3.3)

# 2. TOZALASH
df.isna().sum()               # NaN topish (3.7)
df["narx"] = df["narx"].fillna(df["narx"].median())   # NaN to'ldir
df["narx"] = df["narx"].astype(float)                 # tur (3.6)
df.columns = df.columns.str.strip()                   # nom (3.6)

O'qish va tozalash — xom ma'lumotni tayyorlash: o'qish (read_csv — 3.3; head/info tekshir), tozalash (NaN topish isna().sum() — 3.7, to'ldirish fillna(median); tur astype/to_numeric — 3.6; nom .str.strip()). Sabab: real ma'lumot iflos (NaN, noto'g'ri tur, chalkash nom); tahlildan oldin tozalash shart (jim xato oldini); Data Science vaqtining 80% — tozalash (6-qism). Tekshir (head/info — o'qiqach; isna — NaN). O'qish + tozalash — read_csv, isna/fillna, astype. Iflos → toza. 80%.

2.3. Boyitish va filtrlash

Boyitish va filtrlash — ma'lumotni tayyorlash:

# 3. BOYITISH (yangi ustun — 3.6)
df["summa"] = df["narx"] * df["miqdor"]
df["sana"] = pd.to_datetime(df["sana"])       # 3.12
df["oy"] = df["sana"].dt.month

# 4. FILTRLASH (3.5)
df_2024 = df[df["sana"].dt.year == 2024]
katta = df[df["summa"] > 1000]

Boyitish va filtrlash — yangi xususiyat va kerakli qism: boyitish (yangi ustun df["summa"] = narx * miqdor — 3.6; sana qismlari .dt — 3.12), filtrlash (kerakli qism — df[df["sana"].dt.year == 2024]; shart — 3.5). Sabab: boyitish — mavjud ustunlardan yangi ma'no (summa — tahlil uchun; oy — guruh); filtrlash — kerakli qism (2024 yil, katta savdo — tahlil fokusi). Vektorlashtirilgan (ustun amallari — tez). Boyitish (yangi ustun), filtr (kerakli qism). Boyitish + filtr — summa, filtr. Yangi ma'no. Fokus.

2.4. Guruhlash va birlashtirish

Guruhlash va birlashtirish — tahlil va bog'lash:

# 5. GURUHLASH (3.8)
shahar_savdo = df.groupby("shahar")["summa"].sum()
oy_savdo = df.groupby("oy")["summa"].sum()

# 6. BIRLASHTIRISH (3.9)
df = df.merge(mijoz, on="mijoz_id", how="left")   # mijoz ma'lumoti
kategoriya_savdo = df.groupby("kategoriya")["summa"].agg(["sum", "mean"])

Guruhlash va birlashtirish — tahlil yuragi: guruhlash (groupby("shahar")["summa"].sum() — shahar bo'yicha jami; oy bo'yicha trend — 3.8), birlashtirish (merge(mijoz, on="mijoz_id") — mijoz ma'lumoti qo'shish; turli jadval — 3.9). Sabab: guruhlash — guruh darajasidagi xulosa (shahar/kategoriya bo'yicha — solishtirish, naqsh); birlashtirish — turli jadval (savdo + mijoz — to'liq tahlil). groupby (guruh statistika), merge (bog'lash). Guruh + merge (tahlil asosi). Guruhlash + birlashtirish — groupby, merge. Xulosa. Bog'lash.

2.5. Tahlil va xulosa

Tahlil va xulosa — natija chiqarish:

# 7. SARALASH (reyting — 3.10)
top_mahsulot = df.groupby("mahsulot")["summa"].sum().nlargest(5)
top_shahar = shahar_savdo.sort_values(ascending=False)

# 8. XULOSA (hisobot)
print(f"Jami savdo: {df['summa'].sum()}")
print(f"Eng ko'p shahar: {top_shahar.index[0]}")
print(f"Eng ko'p mahsulot: {top_mahsulot.index[0]}")

Tahlil va xulosa — savollarga javob: saralash (reyting — nlargest(5) top mahsulot; sort_values — shahar reytingi — 3.10), xulosa (natijalarni jamlash — jami savdo, eng ko'p shahar/mahsulot, trend — hisobot). Sabab: tahlil savollarga javob (kim/nima eng ko'p — reyting; jami — summa; trend — oy); xulosa natijani tushunarli (raqamlar → ma'no; hisobot — qaror). nlargest (reyting), xulosa (jamlash). Bu loyiha maqsadi (tahlil — javob, qaror). Tahlil + xulosa — nlargest, hisobot. Javob. Qaror.

2.6. Loyiha amaliyoti

Loyiha amaliyoti: reja (bosqichlar — o'qish → xulosa); o'qish (read_csv + tekshir); tozalash (NaN, tur, nom); boyitish (yangi ustun); filtr (kerakli qism); guruh (groupby); merge (bog'lash); saralash (reyting); xulosa (hisobot); vektorlashtir (ustun amallari — tez). Tuzoqlar: tozalamasdan tahlil (jim xato — NaN/tur), SettingWithCopy (.loc — 3.4), merge how (yo'qotish — 3.9), tur (matn son — 3.6), tekshirmaslik (shape/head). Amaliyot — o'qish, tozalash, guruh, merge, xulosa. Sintez. Oqim.

2.7. Loyiha tuzoqlari

Loyiha asosiy tuzoqlari (butun Pandas tuzoqlari birga): tozalamasdan tahlil (NaN/noto'g'ri tur — jim xato; avval tozala — isna, astype, tekshir); SettingWithCopy (df[shart]["ustun"] = ... — ogohlantirish; .loc[shart, ustun] — 3.4); merge how (inner — qator yo'qoladi; how="left" saqlash — 3.9); tur (CSV — matn; son astype/to_numeric — 3.6); NaN tarqalish (amal — NaN; statistika o'tkazadi — jim; 3.7); apply sekin (vektorlashtir — 3.6/2.12); indeks tarqoq (filtr/saralashdan keyin — reset_index — 3.4); tekshirmaslik (har bosqichda shape/head — kutilganchami). Sabab: loyiha barcha ko'nikmalarni birlashtiradi (har tuzoq mumkin — tozalash, merge, tur, SettingWithCopy). Yechim: avval tozala, .loc, how to'g'ri, tekshir. Tuzoqlar — tozalash, SettingWithCopy, merge, tur (butun 3-qism).

2.8. Loyiha — Pandas sintezi

Loyiha asosiy g'oyasi — Pandas sintezi: 3-qism ko'nikmalar (Series, DataFrame, o'qish, filtr, tozalash, guruh, merge, saralash) real loyihada birga ishlaydi; sintez — bo'laklarni butun qilish (real tahlil — xom ma'lumot → toza → tahlil → xulosa). Do'kon savdo tahlili: o'qish (read_csv — 3.3), tozalash (NaN/tur — 3.6/3.7), boyitish (yangi ustun — 3.6), filtr 3.5-bob, guruh (groupby — 3.8), merge 3.9-bob, saralash (nlargest — 3.10), xulosa (hisobot). Bu Data Science ish oqimi (CRISP-DM — 1.4) Pandas qismi (ma'lumot tayyorlash — vaqtning 80%; tozalash 6-qism). Real loyihada ko'nikmalar yakka emas, birga (biri boshqasiga tayanadi — o'qish → tozalash → tahlil). Bu butun 3-qism yig'indisi va 4-qism (statistika — toza ma'lumot), 5-qism (vizualizatsiya — natija) ga zamin. Loyiha — Pandas sintezi (ko'nikmalar birga, real tahlil). Butun. Yakun. Oqim.

3. Tez ma'lumotnoma

import pandas as pd

# TO'LIQ LOYIHA (do'kon savdo tahlili):

# 1. O'QISH (3.3)
df = pd.read_csv("savdo.csv")
df.head(); df.info()

# 2. TOZALASH (3.6, 3.7)
df["narx"] = df["narx"].fillna(df["narx"].median())
df["narx"] = df["narx"].astype(float)
df.columns = df.columns.str.strip()

# 3. BOYITISH (3.6, 3.12)
df["summa"] = df["narx"] * df["miqdor"]
df["sana"] = pd.to_datetime(df["sana"])

# 4. FILTR (3.5)
df = df[df["summa"] > 0]

# 5. GURUH (3.8)
shahar_savdo = df.groupby("shahar")["summa"].sum()

# 6. MERGE (3.9)
df = df.merge(mijoz, on="mijoz_id", how="left")

# 7. SARALASH (3.10)
top = df.groupby("mahsulot")["summa"].sum().nlargest(5)

# 8. XULOSA
print(f"Jami: {df['summa'].sum()}, eng ko'p: {top.index[0]}")

QOIDA: avval tozalash · .loc o'zgartir · merge how · tekshir (shape/head)

Loyiha xulosasi

Loyiha — Pandas sintezi (ko'nikmalar birga, real tahlil)
Oqim — xom ma'lumot → toza → tahlil → xulosa (CRISP-DM)
O'qish (read_csv) → tozalash (NaN/tur) → boyitish (yangi ustun)
Filtr (3.5) → guruh (groupby) → merge (birlashtir) → saralash (reyting)
Tozalash — vaqtning 80% · avval tozalash (jim xato oldini)

4. Batafsil misollar

Misollar real pandas bilan (deterministik) ishlaydi.

Misol 1 — O'qish va tozalash

"""Loyiha 1: o'qish va tozalash (real pandas)."""

import numpy as np
import pandas as pd


def main() -> None:
    # ma'lumot (xom — NaN, matn narx)
    df = pd.DataFrame({
        "mahsulot": ["olma", "non", "sut", "choy"],
        "narx": ["100", "50", np.nan, "120"],
        "miqdor": [10, 20, 15, 5],
    })

    print("=== 1. Xom holat ===")
    print(f"  narx turi: {df['narx'].dtype}, NaN: {df['narx'].isna().sum()}")

    print("\n=== 2. Tur tuzatish ===")
    df["narx"] = pd.to_numeric(df["narx"], errors="coerce")
    print(f"  yangi tur: {df['narx'].dtype}")

    print("\n=== 3. NaN to'ldirish ===")
    df["narx"] = df["narx"].fillna(df["narx"].median())
    print(f"  NaN: {df['narx'].isna().sum()}")

    print("\n=== 4. Toza ma'lumot ===")
    print(f"  narxlar: {list(df['narx'])}")
    print("  ⭐ O'qish + tozalash (iflos → toza)")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

=== 1. Xom holat ===
  narx turi: str, NaN: 1

=== 2. Tur tuzatish ===
  yangi tur: float64

=== 3. NaN to'ldirish ===
  NaN: 0

=== 4. Toza ma'lumot ===
  narxlar: [100.0, 50.0, 100.0, 120.0]
  ⭐ O'qish + tozalash (iflos → toza)

Nima ko'rsatdi: 2.2-bo'lim.

Misol 2 — Boyitish va filtrlash

"""Loyiha 2: boyitish va filtrlash (real pandas)."""

import pandas as pd


def main() -> None:
    df = pd.DataFrame({
        "mahsulot": ["olma", "non", "sut", "choy"],
        "narx": [100.0, 50.0, 80.0, 120.0],
        "miqdor": [10, 20, 15, 5],
    })

    print("=== 1. Boyitish (summa) ===")
    df["summa"] = df["narx"] * df["miqdor"]
    print(f"  summalar: {list(df['summa'])}")

    print("\n=== 2. Kategoriya (apply) ===")
    df["kat"] = df["summa"].apply(lambda x: "katta" if x > 800 else "kichik")
    print(f"  kategoriyalar: {list(df['kat'])}")

    print("\n=== 3. Filtr (katta) ===")
    katta = df[df["summa"] > 800]
    print(f"  katta savdo: {list(katta['mahsulot'])}")

    print("\n=== 4. Jami ===")
    print(f"  jami savdo: {df['summa'].sum()}")
    print("  ⭐ Boyitish (yangi ustun) + filtr (kerakli qism)")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

=== 1. Boyitish (summa) ===
  summalar: [1000.0, 1000.0, 1200.0, 600.0]

=== 2. Kategoriya (apply) ===
  kategoriyalar: ['katta', 'katta', 'katta', 'kichik']

=== 3. Filtr (katta) ===
  katta savdo: ['olma', 'non', 'sut']

=== 4. Jami ===
  jami savdo: 3800.0
  ⭐ Boyitish (yangi ustun) + filtr (kerakli qism)

Nima ko'rsatdi: 2.3-bo'lim.

Misol 3 — Guruhlash va birlashtirish

"""Loyiha 3: guruhlash va birlashtirish (real pandas)."""

import pandas as pd


def main() -> None:
    savdo = pd.DataFrame({
        "mijoz_id": [1, 2, 1, 3, 2],
        "summa": [100, 200, 150, 300, 250],
    })
    mijoz = pd.DataFrame({
        "mijoz_id": [1, 2, 3],
        "shahar": ["Toshkent", "Samarqand", "Toshkent"],
    })

    print("=== 1. Birlashtirish (merge) ===")
    df = savdo.merge(mijoz, on="mijoz_id", how="left")
    print(f"  qatorlar: {len(df)}, ustunlar: {list(df.columns)}")

    print("\n=== 2. Shahar bo'yicha guruh ===")
    shahar_savdo = df.groupby("shahar")["summa"].sum()
    print(f"  {shahar_savdo.to_dict()}")

    print("\n=== 3. Mijoz bo'yicha ===")
    mijoz_savdo = df.groupby("mijoz_id")["summa"].sum()
    print(f"  eng faol mijoz: {mijoz_savdo.idxmax()}")

    print("\n=== 4. Eng ko'p shahar ===")
    print(f"  eng ko'p savdo: {shahar_savdo.idxmax()}")
    print("  ⭐ merge (bog'lash) + groupby (guruh tahlil)")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

=== 1. Birlashtirish (merge) ===
  qatorlar: 5, ustunlar: ['mijoz_id', 'summa', 'shahar']

=== 2. Shahar bo'yicha guruh ===
  {'Samarqand': 450, 'Toshkent': 550}

=== 3. Mijoz bo'yicha ===
  eng faol mijoz: 2

=== 4. Eng ko'p shahar ===
  eng ko'p savdo: Toshkent
  ⭐ merge (bog'lash) + groupby (guruh tahlil)

Nima ko'rsatdi: 2.4-bo'lim.

Misol 4 — To'liq tahlil hisoboti

"""Loyiha 4: to'liq tahlil hisoboti (real pandas)."""

import pandas as pd


def main() -> None:
    df = pd.DataFrame({
        "mahsulot": ["olma", "non", "sut", "olma", "non", "choy"],
        "shahar": ["T", "S", "T", "S", "T", "S"],
        "narx": [100, 50, 80, 100, 50, 120],
        "miqdor": [10, 20, 15, 8, 25, 5],
    })
    df["summa"] = df["narx"] * df["miqdor"]

    print("=== SAVDO HISOBOTI ===")
    print(f"  Yozuvlar: {len(df)}, jami savdo: {df['summa'].sum()}")

    print("\n=== SHAHAR BO'YICHA ===")
    shahar = df.groupby("shahar")["summa"].sum().sort_values(ascending=False)
    print(f"  {shahar.to_dict()}")

    print("\n=== TOP MAHSULOT ===")
    top = df.groupby("mahsulot")["summa"].sum().nlargest(2)
    print(f"  eng ko'p: {list(top.index)}")

    print("\n=== O'RTACHA CHEK ===")
    print(f"  o'rtacha summa: {round(df['summa'].mean(), 1)}")

    print("\n=== XULOSA ===")
    print("  butun Pandas: boyitish, groupby, sort, nlargest")
    print("  ⭐ Loyiha — Pandas sintezi (3-qism yakuni)")


if __name__ == "__main__":
    main()

Natijaning muhim qismi:

=== SAVDO HISOBOTI ===
  Yozuvlar: 6, jami savdo: 5850

=== SHAHAR BO'YICHA ===
  {'T': 3450, 'S': 2400}

=== TOP MAHSULOT ===
  eng ko'p: ['non', 'olma']

=== O'RTACHA CHEK ===
  o'rtacha summa: 975.0

=== XULOSA ===
  butun Pandas: boyitish, groupby, sort, nlargest
  ⭐ Loyiha — Pandas sintezi (3-qism yakuni)

Nima ko'rsatdi: 2.8-bo'lim.

5. To'g'ri va noto'g'ri tushunishlar

6. Keng tarqalgan xatolar va yechimlari

1. Tozalamasdan tahlil

df.groupby("shahar")["summa"].sum()   # NaN/tur xato          # ⚠️
# avval tozalash (isna, astype), keyin tahlil                  # ✅

2. SettingWithCopy

df[df.summa > 0]["kat"] = "X"   # ogohlantirish               # ⚠️
df.loc[df.summa > 0, "kat"] = "X"   # .loc                     # ✅

3. merge how

savdo.merge(mijoz, on="id")   # inner (savdo yo'qolishi)      # ⚠️
savdo.merge(mijoz, on="id", how="left")   # savdo saqlanadi    # ✅

4. Tur

df["narx"] * df["miqdor"]   # narx matn bo'lsa xato           # ⚠️
df["narx"] = pd.to_numeric(df["narx"], errors="coerce")        # ✅

5. Tekshirmaslik

df = df.merge(...)   # qator soni noma'lum                    # ⚠️
print(df.shape); df.head()   # har bosqichda tekshir           # ✅

6. NaN

df["summa"].sum()   # NaN jim o'tkaziladi                     # ⚠️
df.isna().sum()   # avval NaN tekshir                          # ✅

7. Indeks tarqoq

top = df.groupby(...).sum().nlargest(5)   # ishlaydi          # ⚠️
top.reset_index()   # DataFrame (CSV/grafik)                   # ✅

7. Integratsiya — bu bilim qayerda kerak bo'ladi

• 3.1-3.13 (o'tilgan): Butun Pandas (bu loyihada birga)
• 4-qism: Statistika (toza ma'lumot tahlili)
• 5-qism: Vizualizatsiya (natijalarni ko'rsatish)
• 6-qism: Tozalash (chuqurroq)
• 20-qism: ML (ma'lumot tayyorlash)

8. Eng yaxshi amaliyotlar

1. Reja — bosqichma-bosqich (o'qish → xulosa).

2. Avval tozalash (NaN, tur — jim xato oldini).

3. .loc bilan o'zgartir (SettingWithCopy).

4. merge how (inner yo'qotadi, left saqlaydi).

5. Vektorlashtir (ustun amallari — tez).

6. Har bosqichda tekshir (shape/head).

7. Xulosa — jamlash (hisobot).

8. Loyiha — Pandas sintezi.

9. Amaliy topshiriq

Vazifa 1: Bashorat qiling

1.  # loyiha nima?
2.  # reja nima uchun?
3.  # oqim qanday?
4.  # nima uchun avval tozalash?
5.  # tozalash necha % vaqt?
6.  # boyitish nima?
7.  # guruh nima uchun?
8.  # merge nima uchun?
9.  # saralash nima uchun?
10. # SettingWithCopy?
11. # merge how?
12. # loyiha nima o'rgatadi?

Vazifa 2: Xatolarni tuzating

1.  df.groupby(...)   # tozalamasdan

2.  df[shart]["k"] = "X"   # o'zgartirish

3.  savdo.merge(mijoz, on="id")   # savdo saqlansin

4.  df["narx"] * df["miqdor"]   # narx matn

5.  df = df.merge(...)   # tekshirmasdan

1.  avval tozalash (isna, astype)

2.  df.loc[shart, "k"] = "X"

3.  how="left"

4.  pd.to_numeric(df["narx"], errors="coerce")

5.  print(df.shape); df.head()

Vazifa 3: Reja

Modellang:

1. O'qish
2. Tozalash
3. Tahlil
4. Xulosa

Vazifa 4: Tozalash

Modellang:

1. NaN
2. Tur
3. Nom
4. 80%

Vazifa 5: Tahlil

Modellang:

1. Guruh
2. Merge
3. Saralash
4. Xulosa

Vazifa 6: Oqim

Modellang:

1. Xom
2. Toza
3. Tahlil
4. Xulosa

Vazifa 7: O'ylash

Data Science'da aytiladi: "ma'lumotni tozalash va tayyorlash — ishning 80%, model qurish — 20%". Bu loyiha ham asosan tozalash va tayyorlashga (o'qish, NaN, tur, boyitish) bag'ishlandi. Nima uchun Pandas ko'nikmalari (tozalash, tayyorlash) Data Science'da shunchalik markaziy, va nega "toza ma'lumot" murakkab modeldan ko'ra muhimroq bo'lishi mumkin?

Nimani mustahkamlaydi: 2.2, 2.8-bo'limlar.

Xulosa

Bu darsda butun Pandas bilimini loyihada qo'lladik — 3-qism yakuni.

Eng muhim uch fikr:

1. Loyiha va oqim. Loyiha — Pandas ko'nikmalar sintezi (alohida amal emas — birga, real tahlil). Oqim (CRISP-DM): xom ma'lumot → toza → tahlil → xulosa; bosqichlar — o'qish (read_csv — 3.3) → tozalash (NaN/tur — 3.6/3.7) → boyitish (yangi ustun — 3.6) → filtr 3.5-bob → guruh (groupby — 3.8) → merge 3.9-bob → saralash (nlargest — 3.10) → xulosa; reja bilan.

2. Tozalash va tahlil. O'qish + tozalash — xom ma'lumotni tayyorlash (real ma'lumot iflos — NaN, noto'g'ri tur; isna/fillna/astype; vaqtning 80%). Boyitish + filtr — yangi ustun (summa = narx * miqdor), kerakli qism. Guruh + merge — guruh xulosa (groupby("shahar")["summa"].sum()), turli jadval bog'lash (merge(mijoz)). Tahlil + xulosa — reyting (nlargest), hisobot (jami, eng ko'p).

3. Pandas sintezi. Loyiha — Pandas sintezi (3-qism ko'nikmalar birga — real oqim: xom → toza → tahlil → xulosa; CRISP-DM Pandas qismi — ma'lumot tayyorlash 80%). "Axlat kirdi — axlat chiqdi" (iflos ma'lumot → noto'g'ri model; toza ma'lumot murakkab modeldan muhimroq — poydevor; Pandas — poydevor vositasi). Sintez — o'rganish eng muhim (alohida ko'nikma yetarli emas — birlashtira olish). Tuzoqlar (butun 3-qism): tozalamasdan tahlil, SettingWithCopy (.loc), merge how, tur (to_numeric), tekshirmaslik (shape/head).

3-QISM YAKUNLANDI. Pandas'ni to'liq o'rgandik: Series 3.1-bob, DataFrame 3.2-bob, o'qish/yozish 3.3-bob, tanlash 3.4-bob, filtr 3.5-bob, ustunlar 3.6-bob, NaN 3.7-bob, guruhlash 3.8-bob, merge/concat 3.9-bob, saralash 3.10-bob, apply/transform 3.11-bob, vaqt qatorlari 3.12-bob, pivot 3.13-bob, loyiha 3.14-bob. Pandas — Data Science'ning eng ko'p ishlatiladigan vositasi (jadval ma'lumot — tozalash, tahlil, tayyorlash).

Keyingi qismda (4-qism — Statistika) ma'lumotni statistik tahlil qilishni o'rganamiz: markazli tendensiya (o'rtacha, median, mod), tarqoqlik (std, dispersiya), taqsimotlar, korrelyatsiya, gipoteza tekshirish — ma'lumotdan ishonchli xulosa chiqarish.