I know for a fact that complex features projects the data into higher dimensions which makes the previously non-separable data linearly separable.
But, Is this not true that these complex features will be highly correlated with the features from which they are derived.
My question is, Should i drop the simpler features from training set?
It will depend on the underlying relationship e.g.,
$Y = \alpha X_1 + \beta X_2 + \gamma X_1X_2$
In the above case, the Target is dependent on individual features as well as the Interaction features($X_1X_2$). So, you can't skip the individual.
Same will be True for polynomial too,
$Y = \alpha X + \beta X^2$
You may remove the original features for a scenario similar to this,
$Y = \alpha X_1X_2$ Or,
$Y = \alpha X^2$
Another intuition is that we never see a zero correlation to the original features nor we get a 100% correlation to the Interaction features.
It depends on the machine learning algorithm that the features go into. If you select an algorithm that automatically does feature selection then you do not have manually drop features. Examples of machine learning algorithms that do automatic feature selection are tree-based model and neural networks.
