2

Problem Definition

I would like to code an algorithm for decomposing a covariance matrix into its eigensolution (set of eigenvalues and corresponding eigenvectors. In my specific case I want to deal only with $3\times 3$ covariance matrices.

It is my understanding that symmetric and positive semi-definite matrices decompose into positive real roots. This means I'm trying to solve a cubic polynomial and skip the cases of complex roots. This should also mean my resulting eigenvectors are orthogonal.

I am trying to understand if I need to consider the cases where I do not end up with $3$ unique real roots. I plan to use this algorithm on a data set containing points in $3D$ Euclidean space and am unsure if there exist further simplifications in solving for the eigenvalues.

Main Question

What exactly would it mean (geometrically) if I have one, two or three distinct eigenvalues in terms of a $3D$ pointset in Euclidean space? This is where my understanding fails.

Source of my Confusion

My main reference was in the book Graphics Gems IV, page $193$ by Cromwell. In this section of the book Cromwell describes an optimized method for computing the roots of this particular cubic polynomial, but the text confuses me as it doesn't seem to care about the cases without $3$ distinct roots. The problem definition in the Cromwell section is the exact same as mine: decomposing a covariance matrix from a $3D$ pointset. Other online sources seem to be more general (not specific to $3\times 3$ covariance matrices) and consider all positive-real cases (see Eberly).

Roby5
  • 4,287
RandyGaul
  • 135

1 Answers1

1

It's true that a symmetric positive semi-definite real $3 \times 3$ matrix will always have 3 positive real eigenvalues.

And the eigenvectors corresponding to distinct eigenvalues will be orthogonal.

But, in your problem, I don't think you can be guaranteed that the three eigenvalues will be distinct. Certainly in the (related) problem of computing moments of inertia, symmetries in the object will lead to repetition of eigenvalues.

bubba
  • 43,483
  • 3
  • 61
  • 122
  • Oh this certainly makes sense, thank you for the answer! I am curious as to why two non-distinct eigenvalues cannot have orthogonal eigenvectors? I would imagine that two orthogonal vectors can have the same scalar eigenvalue if the given data input had similar variance along those two axes. – RandyGaul Feb 09 '14 at 04:14
  • A standard result in linear algebra says that if the eigenvalues are distinct, then the corresponding eigenvalues will be orthogonal. See this question: http://math.stackexchange.com/questions/82467/eigenvectors-of-real-symmetric-matrices-are-orthogonal – bubba Feb 09 '14 at 04:27
  • Okay, and to clarify: eigenvalues that aren't distinct will still correspond to orthogonal eigenvectors for this particular problem? If this is true I think I have a good enough understanding. – RandyGaul Feb 09 '14 at 04:47
  • Quoting Arturo Magidin's comment in the question I cited above: "Eigenvectors corresponding to the same eigenvalue need not be orthogonal to each other. However, since every subspace has an orthonormal basis, you can find orthonormal bases for each eigenspace, so you can find an orthonormal basis of eigenvectors". You can construct orthonormal bases using the Gram-Schmidt process (for example). – bubba Feb 09 '14 at 05:19