7

If I have only the upper triangular part of a symmetric matrix $A$, how could I compute the SVD?

$$\begin{pmatrix} 1 & 22 & 13 & 14 \\ & 1 & 45 & 24 \\ & & 1 & 34 \\ & & & 1\end{pmatrix}$$

Does having this upper triangular make the computing easier?

Rodrigo de Azevedo
  • 1
edgarmtze
  • 437

2 Answers2

32

The SVD for a symmetric matrix $A = U \Sigma V^T$, where $U$ and $V$ are unitary matrices with $U = \left[u_1 | u_2 | \ldots | u_n \right]$, $V = \left[v_1 | v_2 | \ldots | v_n \right]$ and $\Sigma$ is a diagonal matrix with non-negative diagonal entries and $v_i = \pm u_i$

For a symmetric matrix the following decompositions are equivalent to SVD. (Well, almost equivalent if you do not worry about the signs of the vectors).

  1. Eigen-value decomposition i.e. $A = X \Lambda X^{-1}$. When $A$ is symmetric, the eigen values are real and the eigenvectors can be chosen to be orthonormal and hence $X^TX = XX^T = I$ i.e. $X^{-1} = X^T$. The only difference is that the singular values are the magnitudes of the eigen values and hence the column of $X$ needs to be multiplied by a negative sign if the eigen value turns out to be negative to get the singular value decomposition. Hence, $U = X$ and $\sigma_i = |\lambda_i|$.

  2. Orthogonal decomposition i.e. $A = PDP^T$, where $P$ is a unitary matrix and $D$ is a diagonal matrix. This exists only when matrix $A$ is symmetric and is the same as eigen value decomposition.

  3. Schur decomposition i.e. $A = Q S Q^T$, where $Q$ is a unitary matrix and $S$ is an upper triangular matrix. This can be done for any matrix. When $A$ is symmetric, then $S$ is a diagonal matrix and again is the same as the eigen value decomposition and orthogonal decomposition.

I do not remember the cost for each of these operations i.e. I don't remember the coefficients before the leading order $n^3$ term. If my memory is right, the typical algorithm for orthogonal decomposition is slightly cheaper than the other two, though I cannot guarantee.

  • Do you know any library to test this great idea?, LAPACK, BLAS? – edgarmtze Feb 20 '11 at 00:06
  • @darkcminor: I think yes. Check this out. http://www.netlib.org/lapack/double/ –  Feb 20 '11 at 00:40
  • One more question @Sivaram Ambikasaran, you say if we don´t worry about the signs, could this be a dangerous problem??, How would look like the output of any of this decompositions of the symmetric matrix?? – edgarmtze Feb 20 '11 at 01:12
  • @darkcminor: By not worrying about the signs, I mean, that it is easy to keep track of the signs and change it at the end. Further depending on the problem, you might not need an SVD if you are satisfied with eigenvalue decomposition or orthogonal decomposition or Schur decomposition for the problem at hand. –  Feb 20 '11 at 01:26
  • If I compute Orthogonal decomposition and get PDP^T, an then U \Sigma V^T. the results are similar or the same?? – edgarmtze Feb 20 '11 at 01:26
  • @darkcminor: No. They are not the same. They are similar. It is easy to go from PDP^T to SVD. –  Feb 20 '11 at 01:27
  • And what is that way from PDP^T to SVD?. – edgarmtze Feb 20 '11 at 01:31
  • The singular values are same as the magnitude of diagonal entries of $D$ and get the negative sign to the respective columns of $P^T$ to get $V^T$. –  Feb 20 '11 at 01:34
  • In the case of S(upper triangular matrix)??? or U, sorry for trouble.. – edgarmtze Feb 20 '11 at 01:36
  • $S$ is diagonal when the matrix $A$ is symmetric. Have you taken linear algebra before? –  Feb 20 '11 at 01:37
  • yes, but it confuse me when you say the way to get SVD – edgarmtze Feb 20 '11 at 01:38
  • The idea is the same as getting from PDP^T to SVD. –  Feb 20 '11 at 01:39
  • Alright, I guess I´m understanding now, thank you @ Sivaram Ambikasaran – edgarmtze Feb 20 '11 at 01:40
  • Write it down for a small sized matrix and you will see what happens and how to change signs accordingly. Try a 3 by 3 symmetric matrix. Find the eigen value decomposition and SVD and find their similarities. –  Feb 20 '11 at 01:40
  • Thats a very good advice, Apreciate your help :) – edgarmtze Feb 20 '11 at 01:41
0

A symmetric matrix always has an eigenvalue decomposition (for which there a specialized routines). I'm not really sure for what application one would need the SVD.

Fabian
  • 23,360