I have a column with some NaNs in it and I want to replace those NaNs with the average/median/mode.
Technically, the validation/ test data has never been seen before - so how could I include it in the average? That would bias the values.
Do I "fit" the average to my training data only, just like scaling? Or do I take the average using the entire dataset?
Imputing all missing values before the split would mean data leakage - you would use test set information to influence the training set.
I had a similar problem recently, I ended up using complete cases of the source data for the test set and then imputing the training data using medians calculated by class.
Otherwise, you might split the data and then impute the test set using the training set average/median/mode.
