1

Based on this quora, the first eigenvector of covariance matrix is the direction in which the data varies the most, the second eigenvector is the direction of greatest variance among those that are orthogonal (perpendicular) to the first eigenvector, the third eigenvector is the direction of greatest variance among those orthogonal to the first two, and so on.

Now I want to know whether this statement is true for any square matrix or it is only true for the covariance matrix? or what kind of condition is necessary for the statement to be true?

Optimized Life
  • 312

1 Answers1

0

A covariance matrix $\Sigma$ is associated with data $X_1,\cdots,X_n$ as follows: $$\Sigma_{i,j}=\text{cov}(X_i,X_j)$$ Covariance matrices are always symmetric and positive semi-definite, and indeed all symmetric positive semi-definite matrices are covariance matrices.

General square matrices, that are not symmetric positive semi-definite, are not covariance matrices, they are not associated with any data in the above sense, so we cannot say anything about directions in which the data varies more or less - since there is no data. Also, the eigenvalues are in general complex (those of a covariance matrix are guaranteed to be real), so they do not have an ordering: there is no "first" eigenvector.

Wouter
  • 7,673