Cannot Use Fillna When A Condition Is Introduced

I am very new to python. Trying to do some imputation in my data. however, I could not manage. Here is the simple code: df['a'] = '' df.loc[(df['c'] >= 0) & (df['c'] <= 4

Solution 1:

Use loc:

df = pd.DataFrame({'c':[10,50,100,200] * 3,
                   'b':[1,3,8,np.nan,5,8,np.nan,7, np.nan, 4,1,0]})
#print (df)
m1 = (df['c'] >= 0) & (df['c'] <= 43)
m2 = (df['c'] >= 44) & (df['c'] <= 96)
m3 = (df['c'] >= 97) & (df['c'] <= 151)
m4 = (df['c'] >= 152) & (df['c'] <= 273)

df.loc[m1,'b'] = df.loc[m1,'b'].fillna(df.loc[m1,'b'].median())
df.loc[m2,'b'] = df.loc[m2,'b'].fillna(df.loc[m2,'b'].median())
df.loc[m3,'b'] = df.loc[m3,'b'].fillna(df.loc[m3,'b'].median())
df.loc[m4,'b'] = df.loc[m4,'b'].fillna(df.loc[m4,'b'].median())

print (df)
      b    c
0   1.0   10
1   3.0   50
2   8.0  100
3   3.5  200
4   5.0   10
5   8.0   50
6   4.5  100
7   7.0  200
8   3.0   10
9   4.0   50
10  1.0  100
11  0.0  200

But better is use cut for category column and then groupby with custom function with fillna and median:

bins = [0,43,96,151,273]
labels=[1,2, 3, 4]
df['a'] = pd.cut(df['c'], bins=bins, labels=labels, include_lowest=True)
df['b'] = df.groupby('a')['b'].apply(lambda x: x.fillna(x.median()))
print (df)
      b    c  a
0   1.0   10  1
1   3.0   50  2
2   8.0  100  3
3   3.5  200  4
4   5.0   10  1
5   8.0   50  2
6   4.5  100  3
7   7.0  200  4
8   3.0   10  1
9   4.0   50  2
10  1.0  100  3
11  0.0  200  4

Solution 2:

#Use.loc when you try to change df values.
df.loc[df.a==1,'b'] = df.loc[df.a==1,'b'].fillna(df[df['a'] == 1]['b'].median())