In training session, model fitting happens to reduce error. But does KNN do this?
Reducing error only happens due to changing K value and number of features, isn't it?
So training set and test set is only for do things below, right?
Short version: this is just terminology, but arguably K-NN doesn't actually fit a model.
Conceptually, K-NN is a lazy learning model. This means that, in a sense, there's no fitting until a new instance arrives to be classified or a value has to be predicted (depending if you're using K-NN for classifying or regression, both are possible). Even then, using the term "fitting the model" is a bit out-of-place, in my opinion.
I think your confusion is in the sentence "model fitting happens to reduce error". No, it does not. Model fitting is simply getting a model (for example, a family of data distributions) and fitting the model by finding the parameters that better describe the data (thus choosing a member from that family).
Maybe you're thinking of neural network's epochs as "fitting"? What happens there is that there's a family of possible models (all the possible values for each weight in the network), and fitting the model is just finding the best possible values. The fact that neural networks do so in an iterative manner does not mean that fitting is an iterative process, fitting is the end result.
So training set and test set is only for do things below, right?
Well, that's true for every model, yes. Training and test set separation serves the only purpose of evaluating the model on the chosen metrics. At the end of the day, the final model you choose to deploy, be it lazy or eager, will use all the available data (or a portion/transformation of it, in some cases), not just the training set.
