1

There are lots of questions on here about the relationship between SVD and EVD. As I understand the singular vectors of SVD will always constitute an orthonormal basis while eigenvectors from EVD are not necessarily orthogonal (for example, [1]).

On the other hand, various sources on SE & elsewhere seem to state that both methods are easily related or even in some sense equivalent. [2]

It's my understanding that in some way both the singular vectors and the eigenvectors are supposed to represent the same thing.

My question is: is there a simple relationship between the orthonormal singular vectors of SVD and the eigenvectors from EVD (e.g. can we get from the latter to the former by Gram-Schmidt orthogonalization or some other method)?

Forgive me if I misunderstand something.

corvus
  • 111
  • A nit - better to call the vectors associated with the SVD "singular vectors". – John Polcari Jul 15 '18 at 14:56
  • Got it — I've rephrased my question slightly to address this. So are the singular vectors inherently different from eigenvectors, then? And, is there a well-defined relationship between them? – corvus Jul 15 '18 at 15:06
  • Also, "singular values" is better than SVD eigenvalues. Formally the two sets are different, because they are defined differently. There is always a well-defined relationship between the two sets - sometimes (but not always) it can be simple. For example, if the matrix is positive definite, the singular values and eigenvalues are the same, and the singular vectors and eigenvectors are generally within a sign/phase choice of each other, although repeated values complicate this. – John Polcari Jul 15 '18 at 15:16
  • Part of the issue is that it takes a series of conventions to make the SVD fully unique. See this question. – John Polcari Jul 15 '18 at 15:19
  • And although my eigen-theory is somewhat rusty, I expect there are probably a different set of conventions needed to make the set of eigenvalues and eigenvectors unique. – John Polcari Jul 15 '18 at 15:29
  • 2
    The SVD of $A$ is related to the EVD of $\pmatrix{0&A\A^T&0}$ which is the basis of the modern fast SVD algorithm. – Lutz Lehmann Jul 15 '18 at 17:45

0 Answers0