I've been studying pattern recognition/machine learning and the theory behind it for some time now and I notice that I find myself seeing the same things over and over again, yet without fully understanding the reason for it...I'm talking about eigenvectors and eigenvalues. These guys just keep coming back again and again and they've been giving me such a headache and I haven't yet been able to discover an intuitive explanation for them :/
I've been studying lately about multivariable Gaussian functions and even there again I bump into eigenvalues/eigenvectors. I know the very basics behind eigenvectors and eigenvalues, that is I can calculate them and apply the formulas. I know that eigenvector is a vector $v$ such that for a linear transformation $A$, there is a scalar $\lambda$, such that
$$Av=\lambda v.$$ Okay, I've seen this definition countless many times, but I haven't yet discovered: "Why should I care?" That's a cool property I give it that, but why would I want to calculate the eigenvectors/eigenvalues? I know that: "eigenvector is a vector such that when linear transformation is applied into it its directions doesn't change, only scales by a factor of $\lambda$". Alright about that...so? But why do I care? Why do I want to calculate it?
For example Fourier transformation I can understand (not an expert on the subject, but I think I got the idea and motivation). If I have a signal of some sort I can use Fourier transformation to disect the signal and see the basic ingredients it is made of. Then I can apply low-pass/high-pass filters, clear the noise from the signal etc. This is something I can understand.
What about eigenvectors and eigenvalues? Can somebody give a concerete example, that would show the motivation and usefulness of eigenvectors and eigenvalues like Fourier transform is in the case of signal analysis. The example doesn't have to be long, short and simple is fine, but one that would sell the idea well.
Are they used because they make calculations simpler? Are they used because they describe the effect of the linear transformation? If so, what can I do with this information? etc.
Thank you for any help! =)
