Mundarija (22)
- 1. Kirish va motivatsiya
- 2. Nazariya — chuqur tushuntirish
- 2.1. apply (Series va DataFrame)
- 2.2. groupby.transform (guruh statistikasi)
- 2.3. map va almashtirish
- 2.4. lambda funksiyalar
- 2.5. Vektorlashtirish afzal
- 2.6. Apply amaliyoti
- 2.7. Apply tuzoqlari
- 2.8. Apply — ma'lumotni kuchli o'zgartirish
- 3. Tez ma'lumotnoma
- 4. Batafsil misollar
- Misol 1 — apply
- Misol 2 — groupby.transform
- Misol 3 — map va replace
- Misol 4 — Vektorlashtirish vs apply
- 5. To'g'ri va noto'g'ri tushunishlar
- 6. Keng tarqalgan xatolar va yechimlari
- 7. Integratsiya — bu bilim qayerda kerak bo'ladi
- 8. Eng yaxshi amaliyotlar
- 9. Amaliy topshiriq
- Xulosa
3.11-dars: Apply va transformatsiya
3-QISM — PANDAS · 11-dars
1. Kirish va motivatsiya
3.6-darsda applyni qisqacha ko'rdik. Endi uni va boshqa transformatsiya usullarini chuqurroq o'rganamiz: apply (Series va DataFrame'da, qator/ustun bo'yicha), map (qiymat almashtirish), va eng muhimi — groupby bilan transform (guruh statistikasini har qatorga qaytarish). Bu usullar ma'lumotni kuchli o'zgartirish imkonini beradi: har qatorga o'z guruhining o'rtachasidan farqini hisoblash, murakkab shartli hisob, standartlashtirish. Nega muhim? (1) Murakkab o'zgartirish — vektorlashtirib bo'lmaydigan mantiq; (2) Guruh transform — har qatorga guruh statistikasi (feature engineering); (3) Moslashuvchanlik — istalgan funksiya. Lekin doim eslash kerak: iloji bo'lsa vektorlashtir (apply sekin). Bu dars apply va transformatsiyani chuqur o'rgatadi — ma'lumotni kuchli o'zgartirish.
Apply va transformatsiya — funksiya bilan o'zgartirish: apply (Series — element, DataFrame — qator/ustun axis), map (Series — qiymat almashtirish), transform (guruh — har qatorga guruh statistikasi), groupby.transform (guruh o'rtachasi har qatorga — feature engineering), lambda (nomsiz funksiya), vektorlashtirish (afzal — apply sekin). Foydalanish: murakkab o'zgartirish, guruh transform, feature engineering. Bu 3.6 (ustunlar), 3.8 (groupby) bilan bog'liq. apply — funksiya. transform — guruh. O'zgartirish.
Real vaziyat. Data Scientist talaba baholari bilan ishlar edi (sinf, ball). Guruh transform kerak edi: har talaba balli o'z sinfi o'rtachasidan qancha farq qiladi — df["farq"] = df["ball"] - df.groupby("sinf")["ball"].transform("mean") (har qatorga o'z sinfi o'rtachasi). apply (murakkab baho) — df["baho"] = df["ball"].apply(lambda x: "a'lo" if x >= 90 else "yaxshi" if x >= 70 else "qoniqarli") (uch daraja). map (kod) — df["sinf_kod"] = df["sinf"].map({"A": 1, "B": 2}). groupby.transform har qatorga guruh statistikasi qaytardi (feature engineering — sinf o'rtachasiga nisbatan). Apply/transform ma'lumotni kuchli o'zgartirdi (guruh, murakkab mantiq). transform — Data Science feature engineering asosi.
Bu darsda apply va transformatsiyani o'rganamiz.
Bu darsda:
- apply (Series va DataFrame)
- groupby.transform (guruh statistikasi)
- map va almashtirish
- lambda funksiyalar
- Vektorlashtirish afzal
- Apply amaliyoti
- Apply tuzoqlari
- Amaliy: transform modeli
ℹ Misollar real pandas bilan (deterministik) ishlaydi.
2. Nazariya — chuqur tushuntirish
2.1. apply (Series va DataFrame)
Funksiya qo'llash:
import pandas as pd
df = pd.DataFrame({"a": [1, 2, 3], "b": [10, 20, 30]})
# Series apply (har element)
df["a"].apply(lambda x: x ** 2) # [1, 4, 9]
# DataFrame apply (ustun bo'yicha — axis=0)
df.apply(lambda ustun: ustun.max()) # har ustun max
# DataFrame apply (qator bo'yicha — axis=1)
df.apply(lambda qator: qator["a"] + qator["b"], axis=1) # a+b apply (Series va DataFrame) — funksiya qo'llash: Series (df["a"].apply(func) — har element), DataFrame (df.apply(func) — ustun bo'yicha axis=0; axis=1 — qator bo'yicha). Sabab: murakkab o'zgartirish (vektorlashtirib bo'lmaydigan — shart, o'z funksiya); apply moslashuvchan (istalgan funksiya). axis muhim (axis=0 — har ustun; axis=1 — har qator, bir nechta ustundan hisob). Series (element), DataFrame (qator/ustun — axis). apply — funksiya (Series element, DataFrame axis). Moslashuvchan. axis.
2.2. groupby.transform (guruh statistikasi)
Guruh statistikasi har qatorga:
df = pd.DataFrame({
"sinf": ["A", "A", "B", "B"],
"ball": [80, 90, 60, 70],
})
# transform — guruh statistikasi HAR QATORGA
df["sinf_ort"] = df.groupby("sinf")["ball"].transform("mean")
# A: 85, A: 85, B: 65, B: 65 (har qatorga o'z sinf o'rtachasi)
# farq (o'z o'rtachasidan)
df["farq"] = df["ball"] - df["sinf_ort"] groupby.transform (guruh statistikasi) — guruh statistikasini har qatorga qaytaradi: df.groupby("sinf")["ball"].transform("mean") (har qator uchun o'z sinfi o'rtachasi — A qatorlar 85, B qatorlar 65). Sabab: ba'zan har qatorga guruh statistikasi kerak (o'z sinf o'rtachasidan farq — standartlashtirish; guruh bo'yicha normallashtirish); groupby.agg (guruh — kichik natija, 3.8) vs groupby.transform (har qatorga — asl o'lcham). Farq: agg (guruh soni — 2 sinf); transform (qator soni — 4 talaba, har biriga guruh qiymati). Feature engineering (guruh xususiyat). transform — guruh statistikasi har qatorga (asl o'lcham). agg emas. Feature.
2.3. map va almashtirish
Qiymat almashtirish:
df = pd.DataFrame({"sinf": ["A", "B", "A", "C"]})
# map — dict bilan almashtirish
df["kod"] = df["sinf"].map({"A": 1, "B": 2, "C": 3}) # [1,2,1,3]
# map — funksiya bilan
df["katta"] = df["sinf"].map(lambda x: x.upper())
# replace — qiymatlarni almashtirish (DataFrame)
df.replace({"A": "Alfa", "B": "Beta"}) map va almashtirish — qiymat almashtirish: map (Series — dict {"A": 1} yoki funksiya; qiymat → yangi qiymat), replace (DataFrame — qiymatlarni almashtirish; {"eski": "yangi"}). Sabab: kategoriya → kod (matn → son — ML; map dict), qiymat tuzatish (xato → to'g'ri; replace). map (Series — bir ustun), replace (DataFrame — hamma). map (dict — tez, aniq; yo'q kalit → NaN — 3.6). map/replace — qiymat almashtirish (dict). Kategoriya→kod. Tuzatish.
2.4. lambda funksiyalar
Nomsiz funksiya:
# lambda — qisqa nomsiz funksiya
kvadrat = lambda x: x ** 2 # oddiy funksiya kabi
df["a"].apply(lambda x: x ** 2) # apply ichida
# shartli lambda
df["baho"] = df["ball"].apply(
lambda x: "a'lo" if x >= 90 else "yaxshi" if x >= 70 else "past"
)
# murakkab — oddiy funksiya afzal (o'qilishli)
def bahole(x):
if x >= 90: return "a'lo"
return "past" lambda funksiyalar — nomsiz funksiya: lambda x: x ** 2 (qisqa — bir ifoda; apply ichida ko'p ishlatiladi). Sabab: apply/map funksiya talab qiladi (har elementga); oddiy amal uchun lambda (qisqa — alohida funksiya yozmasdan); shartli (lambda x: "a" if x > 0 else "b"). Murakkab bo'lsa — oddiy funksiya afzal (def — o'qilishli; lambda bir qatorda — chalkash). lambda (oddiy — bir ifoda), def (murakkab — o'qilishli). lambda — nomsiz funksiya (qisqa, apply ichida). Oddiy. Murakkabda def.
2.5. Vektorlashtirish afzal
Vektorlashtirish afzal — apply oxirgi chora: iloji bo'lsa vektorlashtir (df["a"] * 2 — tez, C; apply(lambda x: x*2) — sekin, Python sikl — 2.12/3.6); apply faqat vektorlashtirib bo'lmaganda (murakkab mantiq, matn regex, tashqi funksiya). Vektorlashtirilgan mumkin amallar: arifmetik (* + - /), taqqoslash (> ==), np.where (shartli — 2.9), matn (.str — 3.5), map (dict almashtirish). apply — oxirgi chora (moslashuvchan lekin sekin). Sabab: apply Python sikl (har element — sekin; million qator — farq katta 2.12); vektorlashtirilgan C (tez). To'g'ri tartib: avval vektorlashtir (* , np.where, .str, map), keyin apply (iloji bo'lmasa). Vektorlashtirish afzal — apply oxirgi chora (sekin). Tez. Iloji bo'lsa.
2.6. Apply amaliyoti
Apply amaliyoti: apply (Series element, DataFrame axis — murakkab mantiq); groupby.transform (guruh statistikasi har qatorga — feature engineering); map (dict almashtirish — kategoriya→kod); replace (qiymat tuzatish); lambda (oddiy — apply ichida; murakkab — def); vektorlashtir (avval — *, np.where, .str, map); axis (apply — 0 ustun, 1 qator). Tuzoqlar: apply sekin (vektorlashtir), axis (0 vs 1), transform vs agg (har qator vs guruh), map yo'q kalit (NaN), lambda murakkab (def afzal). Amaliyot — apply, transform, map, lambda. Feature. Vektorlashtir.
2.7. Apply tuzoqlari
Apply asosiy tuzoqlari: apply sekin (Python sikl — vektorlashtir afzal; 2.12); axis chalkash (apply(func) — ustun bo'yicha axis=0 (standart); qator uchun axis=1; noto'g'ri axis — xato natija); transform vs agg (transform — har qatorga (asl o'lcham); agg — guruh (kichik); chalkash — transform feature uchun, agg xulosa uchun); map yo'q kalit (map({"A": 1}) — "B" bo'lsa → NaN; to'liq dict yoki fillna); lambda murakkab (uzun lambda — o'qib bo'lmaydi; def afzal); transform funksiya (transform("mean") — string yoki funksiya; asl o'lcham qaytarishi kerak — sum/mean/std); apply natija turi (Series/skalyar — turli natija shakl); inplace yo'q (apply/map — yangi, df["a"] = ... saqlash). Sabab: apply axis/tur/tezlik nozik (axis, transform/agg, sekin — jim xato yoki sekin). Yechim: vektorlashtir, axis to'g'ri, transform (har qator), def (murakkab). Tuzoqlar — sekin, axis, transform/agg, map.
2.8. Apply — ma'lumotni kuchli o'zgartirish
Apply asosiy g'oyasi — ma'lumotni kuchli o'zgartirish: vektorlashtirib bo'lmaganda (murakkab mantiq, guruh statistikasi) apply/transform moslashuvchan o'zgartirish beradi. apply (Series element, DataFrame axis — murakkab funksiya), groupby.transform (guruh statistikasi har qatorga — feature engineering, standartlashtirish; agg dan farq — asl o'lcham), map (qiymat almashtirish — kategoriya→kod), lambda (oddiy funksiya). Eng muhim: transform (guruh xususiyat — har qatorga o'z guruh statistikasi; ML feature). Lekin vektorlashtirish afzal (apply sekin — Python sikl; avval */np.where/.str/map, keyin apply). Data Science'da (murakkab o'zgartirish — apply; guruh feature — transform; ML tayyorgarlik — 20-qism). Bu 3.6 (ustunlar — apply) va 3.8 (groupby — transform) davomi, feature engineering uchun. Apply — ma'lumotni kuchli o'zgartirish (apply, transform, map). Feature. Moslashuvchan.
3. Tez ma'lumotnoma
import pandas as pd
# apply (Series — element):
df["a"].apply(lambda x: x ** 2)
# apply (DataFrame — axis):
df.apply(lambda ustun: ustun.max()) # ustun (axis=0)
df.apply(lambda qator: qator["a"]+qator["b"], axis=1) # qator
# groupby.transform (guruh statistikasi HAR QATORGA):
df["sinf_ort"] = df.groupby("sinf")["ball"].transform("mean")
df["farq"] = df["ball"] - df["sinf_ort"] # o'z guruhidan farq
# agg — guruh (kichik) · transform — har qator (asl o'lcham)
# map (qiymat almashtirish):
df["kod"] = df["sinf"].map({"A": 1, "B": 2}) # dict
df.replace({"A": "Alfa"}) # DataFrame
# lambda (oddiy) / def (murakkab):
df["a"].apply(lambda x: x*2) # oddiy
# VEKTORLASHTIR AFZAL (apply sekin):
df["a"] * 2 # tez (apply emas)
QOIDA: vektorlashtir avval · transform har qator (agg guruh) · axis · def murakkabApply xulosasi
Apply — ma'lumotni kuchli o'zgartirish (murakkab, guruh)
apply — Series (element), DataFrame (axis=0 ustun, axis=1 qator)
groupby.transform — guruh statistikasi HAR QATORGA (feature)
agg (guruh, kichik) vs transform (har qator, asl o'lcham)
map — qiymat almashtirish (dict) · vektorlashtir afzal (apply sekin)4. Batafsil misollar
Misollar real pandas bilan (deterministik) ishlaydi.
Misol 1 — apply
"""apply: Series va DataFrame (real pandas)."""
import pandas as pd
def main() -> None:
df = pd.DataFrame({"a": [1, 2, 3], "b": [10, 20, 30]})
print("=== 1. Series apply (element) ===")
print(f" kvadrat: {list(df['a'].apply(lambda x: x ** 2))}")
print("\n=== 2. DataFrame apply (ustun, axis=0) ===")
print(f" har ustun max: {df.apply(lambda u: u.max()).to_dict()}")
print("\n=== 3. DataFrame apply (qator, axis=1) ===")
yigindi = df.apply(lambda q: q["a"] + q["b"], axis=1)
print(f" a+b: {list(yigindi)}")
print("\n=== 4. Shartli apply ===")
df["kat"] = df["a"].apply(lambda x: "katta" if x > 1 else "kichik")
print(f" kategoriya: {list(df['kat'])}")
print(" ⭐ apply — funksiya (Series element, DataFrame axis)")
if __name__ == "__main__":
main()Natijaning muhim qismi:
=== 1. Series apply (element) ===
kvadrat: [1, 4, 9]
=== 2. DataFrame apply (ustun, axis=0) ===
har ustun max: {'a': 3, 'b': 30}
=== 3. DataFrame apply (qator, axis=1) ===
a+b: [11, 22, 33]
=== 4. Shartli apply ===
kategoriya: ['kichik', 'katta', 'katta']
⭐ apply — funksiya (Series element, DataFrame axis)Nima ko'rsatdi: 2.1-bo'lim.
Misol 2 — groupby.transform
"""groupby.transform: guruh statistikasi (real pandas)."""
import pandas as pd
def main() -> None:
df = pd.DataFrame({
"sinf": ["A", "A", "B", "B", "A"],
"ball": [80, 90, 60, 70, 100],
})
print("=== 1. transform (har qatorga) ===")
df["sinf_ort"] = df.groupby("sinf")["ball"].transform("mean")
print(f" sinf o'rtachalari: {list(df['sinf_ort'])}")
print("\n=== 2. Farq (o'z guruhidan) ===")
df["farq"] = df["ball"] - df["sinf_ort"]
print(f" farqlar: {list(df['farq'])}")
print("\n=== 3. agg vs transform ===")
agg = df.groupby("sinf")["ball"].mean()
print(f" agg (guruh): {len(agg)} qator · transform (har qator): {len(df)}")
print("\n=== 4. Guruh max ===")
df["sinf_max"] = df.groupby("sinf")["ball"].transform("max")
print(f" sinf max: {list(df['sinf_max'])}")
print(" ⭐ transform — guruh statistikasi har qatorga (feature)")
if __name__ == "__main__":
main()Natijaning muhim qismi:
=== 1. transform (har qatorga) ===
sinf o'rtachalari: [90.0, 90.0, 65.0, 65.0, 90.0]
=== 2. Farq (o'z guruhidan) ===
farqlar: [-10.0, 0.0, -5.0, 5.0, 10.0]
=== 3. agg vs transform ===
agg (guruh): 2 qator · transform (har qator): 5
=== 4. Guruh max ===
sinf max: [100, 100, 70, 70, 100]
⭐ transform — guruh statistikasi har qatorga (feature)Nima ko'rsatdi: 2.2-bo'lim.
Misol 3 — map va replace
"""map va replace (real pandas)."""
import pandas as pd
def main() -> None:
df = pd.DataFrame({"sinf": ["A", "B", "A", "C"]})
print("=== 1. map (dict) ===")
df["kod"] = df["sinf"].map({"A": 1, "B": 2, "C": 3})
print(f" kodlar: {list(df['kod'])}")
print("\n=== 2. map (funksiya) ===")
df["kichik"] = df["sinf"].map(lambda x: x.lower())
print(f" kichik: {list(df['kichik'])}")
print("\n=== 3. replace ===")
d2 = df.replace({"A": "Alfa", "B": "Beta"})
print(f" almashtirilgan: {list(d2['sinf'])}")
print("\n=== 4. map yo'q kalit → NaN ===")
natija = df["sinf"].map({"A": 1})
print(f" NaN soni: {natija.isna().sum()} (B, C yo'q)")
print(" ⭐ map — qiymat almashtirish (dict)")
if __name__ == "__main__":
main()Natijaning muhim qismi:
=== 1. map (dict) ===
kodlar: [1, 2, 1, 3]
=== 2. map (funksiya) ===
kichik: ['a', 'b', 'a', 'c']
=== 3. replace ===
almashtirilgan: ['Alfa', 'Beta', 'Alfa', 'C']
=== 4. map yo'q kalit → NaN ===
NaN soni: 2 (B, C yo'q)
⭐ map — qiymat almashtirish (dict)Nima ko'rsatdi: 2.3-bo'lim.
Misol 4 — Vektorlashtirish vs apply
"""Vektorlashtirish vs apply (real pandas)."""
import numpy as np
import pandas as pd
def main() -> None:
df = pd.DataFrame({"narx": [100, 200, 150, 300]})
print("=== 1. Vektorlashtirilgan (afzal) ===")
df["chegirma"] = df["narx"] * 0.9
print(f" chegirma: {list(df['chegirma'])}")
print("\n=== 2. np.where (shartli — vektor) ===")
df["kat"] = np.where(df["narx"] > 150, "qimmat", "arzon")
print(f" kategoriya: {list(df['kat'])}")
print("\n=== 3. apply (murakkab mantiq) ===")
df["daraja"] = df["narx"].apply(
lambda x: "yuqori" if x > 250 else "o'rta" if x > 120 else "past"
)
print(f" daraja: {list(df['daraja'])}")
print("\n=== 4. Xulosa ===")
print(" oddiy → vektorlashtir (tez) · murakkab → apply")
print(" ⭐ Vektorlashtirish afzal (apply sekin — oxirgi chora)")
if __name__ == "__main__":
main()Natijaning muhim qismi:
=== 1. Vektorlashtirilgan (afzal) ===
chegirma: [90.0, 180.0, 135.0, 270.0]
=== 2. np.where (shartli — vektor) ===
kategoriya: ['arzon', 'qimmat', 'arzon', 'qimmat']
=== 3. apply (murakkab mantiq) ===
daraja: ['past', "o'rta", "o'rta", 'yuqori']
=== 4. Xulosa ===
oddiy → vektorlashtir (tez) · murakkab → apply
⭐ Vektorlashtirish afzal (apply sekin — oxirgi chora)Nima ko'rsatdi: 2.5-bo'lim.
5. To'g'ri va noto'g'ri tushunishlar
| Noto'g'ri fikr | To'g'risi |
|---|---|
| "apply doim kerak" | Vektorlashtir afzal (sekin) |
| "apply axis muhim emas" | axis=0 ustun, axis=1 qator |
| "transform = agg" | transform har qator, agg guruh |
| "map hamma kalitni" | Yo'q kalit → NaN |
| "lambda murakkab uchun" | def afzal (o'qilishli) |
| "transform xulosa" | Feature (har qator) |
| "apply tez" | Sekin (Python sikl) |
| "map = replace" | map Series, replace DataFrame |
6. Keng tarqalgan xatolar va yechimlari
1. apply sekin
df["a"].apply(lambda x: x * 2) # sekin # ⚠️
df["a"] * 2 # vektorlashtir (tez) # ✅2. axis
df.apply(func) # ustun bo'yicha (axis=0) # ⚠️
df.apply(func, axis=1) # qator bo'yicha # ✅3. transform vs agg
df.groupby("g")["b"].mean() # guruh (kichik) # ⚠️
df.groupby("g")["b"].transform("mean") # har qator # ✅4. map yo'q kalit
df["s"].map({"A": 1}) # "B" → NaN # ⚠️
df["s"].map({"A": 1, "B": 2}) # to'liq dict # ✅5. lambda murakkab
df["a"].apply(lambda x: ... uzun mantiq ...) # o'qib bo'lmaydi # ⚠️
def func(x): ...; df["a"].apply(func) # def (o'qilishli) # ✅6. Natijani saqlamaslik
df["a"].apply(func) # asl o'zgarmaydi # ⚠️
df["a"] = df["a"].apply(func) # saqla # ✅7. transform funksiya
df.groupby("g")["b"].transform(lambda x: x.head()) # o'lcham xato # ⚠️
df.groupby("g")["b"].transform("mean") # asl o'lcham # ✅7. Integratsiya — bu bilim qayerda kerak bo'ladi
- 3.6-dars (o'tilgan): Ustunlar (apply, map)
- 3.8-dars (o'tilgan): groupby (transform)
- 2.9-dars (o'tilgan): np.where (vektorlashtirilgan shart)
- 6-qism: Tozalash (murakkab o'zgartirish)
- 20-qism: ML (feature engineering — transform)
8. Eng yaxshi amaliyotlar
Vektorlashtir avval (
*,np.where,.str,map).apply— faqat murakkab (sekin).axis— 0 ustun, 1 qator (apply).transform— guruh feature (har qator).agg— guruh xulosa (kichik).map— to'liq dict (yo'q → NaN).def— murakkab lambda (o'qilishli).Apply — ma'lumotni kuchli o'zgartirish.
9. Amaliy topshiriq
Vazifa 1: Bashorat qiling
1. # apply nima?
2. # Series vs DataFrame apply?
3. # axis nima?
4. # transform nima?
5. # transform vs agg?
6. # map nima?
7. # map yo'q kalit?
8. # lambda nima?
9. # lambda vs def?
10. # apply tez?
11. # vektorlashtir yoki apply?
12. # nega transform muhim?Javoblar
- Funksiya qo'llash
- Series element, DataFrame axis
- axis=0 ustun, axis=1 qator
- Guruh statistikasi har qatorga
- transform har qator, agg guruh
- Qiymat almashtirish (dict)
- NaN
- Nomsiz funksiya
- lambda oddiy, def murakkab
- Sekin (Python sikl)
- Vektorlashtir (apply oxirgi)
- Feature engineering (guruh xususiyat)
Vazifa 2: Xatolarni tuzating
1. df["a"].apply(lambda x: x*2) # tez
2. df.apply(func) # qator bo'yicha
3. df.groupby("g")["b"].mean() # har qatorga
4. df["s"].map({"A": 1}) # "B" bor
5. df["a"].apply(func) # saqlashJavoblar
1. df["a"] * 2
2. df.apply(func, axis=1)
3. ....transform("mean")
4. {"A": 1, "B": 2}
5. df["a"] = df["a"].apply(func)Vazifa 3: apply
Modellang:
- Series
- DataFrame
- axis
- Murakkab
Vazifa 4: transform
Modellang:
- Guruh
- Har qator
- agg farqi
- Feature
Vazifa 5: map
Modellang:
- Dict
- Almashtirish
- Yo'q kalit
- Kod
Vazifa 6: Vektorlashtirish
Modellang:
- Afzal
- Tez
- apply sekin
- Oxirgi chora
Vazifa 7: O'ylash
groupby.transform guruh statistikasini (masalan o'rtacha) har qatorga qaytaradi, groupby.agg esa har guruh uchun bitta qiymat. Nima uchun bu farq feature engineering (ML uchun xususiyat yaratish) da shunchalik muhim, va nega "har qatorga o'z guruhining statistikasi" kuchli xususiyat bo'lishi mumkin?
Javob
Qisqa javob: transform guruh statistikasini har qatorga (asl o'lcham), agg guruh (kichik); farq feature engineering'da muhim, chunki: (1) feature — har qatorga — ML har namuna (qator) uchun xususiyat talab qiladi (model qatorlar bilan ishlaydi); transform har qatorga qiymat beradi (yangi ustun — model uchun); agg kichik (guruh soni — qatorga mos kelmaydi, model uchun to'g'ridan ishlamaydi); (2) kontekst xususiyat — "har qatorga o'z guruhining statistikasi" — kuchli (talaba balli o'z sinfiga nisbatan — 90 ball A sinfda o'rtacha, B sinfda a'lo; kontekst muhim); transform bu kontekstni qo'shadi (guruh o'rtachasi — har qator); (3) nisbiy xususiyat — mutloq qiymat (ball 90) vs nisbiy (o'z guruhidan farq — +5); nisbiy ko'pincha kuchliroq (kontekst — model yaxshi o'rganadi). "Nega kuchli xususiyat": (a) kontekst — qiymat o'z guruhida ma'no (90 ball — qaysi sinfda?; o'z sinf o'rtachasidan farq — aniq); model kontekstni ko'radi (guruh bilan solishtirish); (b) normallashtirish — guruh bo'yicha (har sinf o'z o'lchovida — z-score guruhda; 2.6); (c) naqsh — guruh ichidagi holat (o'rtachadan yuqori/past — model uchun signal); (d) feature engineering — yangi ma'noli ustun (o'z guruhidan farq — model kirishi). Misol: mijoz xaridi (1000) vs shahri o'rtachasi (transform — shahar o'rtachasi); farq (mijoz o'z shahrida ko'p/kam xarid — kontekst; VIP aniqlash). "Nega transform (agg emas)": agg kichik (guruh — 5 shahar; qatorga (10000 mijoz) mos kelmaydi; qo'lda merge kerak); transform har qatorga (10000 — to'g'ridan ustun; oson). transform avtomatik (guruh → har qator — merge kerak emas). Saboqlar: transform har qatorga (feature — model uchun); agg guruh (xulosa — kichik); kontekst xususiyat (o'z guruhidan — kuchli); nisbiy (mutloqdan yaxshi — ko'pincha). To'g'ri: feature — transform (har qator); xulosa — agg (guruh). Muvozanat: xulosa (agg — hisobot) + feature (transform — ML) — turli maqsad. Bu feature engineering asosi (guruh xususiyat — transform; ML — 20-qism); "o'z guruhidan farq" — kuchli signal (kontekst). transform — ML feature yaratish vositasi (guruh statistikasi har qatorga).
1. Nega transform har qatorga
- ML har namuna (qator) — xususiyat
- transform har qatorga (yangi ustun — model)
- agg kichik (guruh — qatorga mos emas)
2. Nega kuchli xususiyat
- Kontekst (o'z guruhida ma'no)
- Normallashtirish (guruh bo'yicha)
- Naqsh (guruh ichidagi holat — signal)
- Nisbiy (mutloqdan yaxshi)
3. transform vs agg (feature)
| transform | agg |
|---|---|
| Har qator (asl o'lcham) | Guruh (kichik) |
| Feature (model) | Xulosa (hisobot) |
| Merge kerak emas | Merge kerak |
4. Misol
- Mijoz xarid (1000) vs shahar o'rtacha (transform)
- Farq (o'z shahrida ko'p/kam — VIP signal)
5. Saboqlar
- transform har qatorga (feature)
- agg guruh (xulosa)
- Kontekst xususiyat (kuchli)
- Nisbiy (mutloqdan yaxshi)
6. Xulosa
- transform har qatorga (feature — ML)
- agg guruh (xulosa — kichik)
- "O'z guruhidan farq" (kontekst — kuchli)
- transform — feature engineering vositasi
Nimani mustahkamlaydi: 2.2, 2.8-bo'limlar.
Xulosa
Bu darsda apply va transformatsiyani o'rgandik.
Eng muhim uch fikr:
apply va transform.
apply— funksiya qo'llash: Series (df["a"].apply(func)— har element), DataFrame (df.apply(func)— ustunaxis=0;axis=1— qator).groupby.transform— guruh statistikasini har qatorga qaytaradi (df.groupby("sinf")["ball"].transform("mean")— har qator o'z sinfi o'rtachasi); agg dan farq (agg — guruh, kichik; transform — har qator, asl o'lcham; feature engineering).map va lambda.
map— qiymat almashtirish (Series — dict{"A": 1}yoki funksiya; kategoriya→kod; yo'q kalit → NaN),replace(DataFrame — qiymat tuzatish). lambda — nomsiz funksiya (lambda x: x**2— qisqa, apply ichida; murakkab bo'lsadefafzal — o'qilishli).Vektorlashtirish afzal. Vektorlashtirish afzal —
applyoxirgi chora (Python sikl — sekin 2.12; avval*/np.where/.str/map— tez);applyfaqat murakkab mantiq/tashqi funksiya uchun. Apply — ma'lumotni kuchli o'zgartirish (murakkab, guruh);transform— feature engineering (o'z guruhidan farq — kontekst xususiyat, ML kuchli signal; 20-qism). Data Science'da (murakkab o'zgartirish — apply; guruh feature — transform). Tuzoqlar: apply sekin (vektorlashtir), axis (0/1), transform vs agg (har qator vs guruh), map yo'q kalit (NaN), lambda murakkab (def).
Keyingi darsda vaqt qatorlari (datetime)ni o'rganamiz: sana/vaqt bilan ishlash, vaqt bo'yicha guruhlash (resample), trend — vaqtga bog'liq ma'lumot tahlili.
Izohlar (0)
Izoh yozish uchun kiring.
- Hozircha izoh yo'q. Birinchi bo'ling!