1

I'm trying to do a singular value decomposition ($A = USV^T$) of the following matrix:

$A = \begin{bmatrix}1&1\\1&0\end{bmatrix}$

First of all I calculate $A^TA$ and $AA^T$ which in this case is the same: $\begin{bmatrix}2&1\\1&1\end{bmatrix}$

Now I get the eigenvalues/eigenvectors of the previous matrix, which are $\frac{3 + \sqrt{5}}{2}$ and $\frac{3 - \sqrt{5}}{2}$. The eigenvectors are [$\frac{1 + \sqrt{5}}{2}$, 1] and [$\frac{1 - \sqrt{5}}{2}$, 1].

Let's define $\phi = \frac{1 + \sqrt{5}}{2}$

I can now construct both U and V with the normalized eigenvectors, which are orthogonal matrices: $U = V = \begin{bmatrix}\frac{\phi}{\sqrt{\phi^2 + 1}}&\frac{1 - \phi}{\sqrt{(1 - \phi)^2 + 1}}\\\frac{1}{\sqrt{\phi^2 + 1}}&\frac{1/}{\sqrt{(1 - \phi)^2 + 1}}\end{bmatrix}$

Finally, I can calculate S which is the diagonal matrix with square roots of the eigenvalues of $AA^T$: $S = \begin{bmatrix}\phi&0\\0&1-\phi\end{bmatrix}$

If I now calculate $USV^T$ I'm not getting A! I however realized that if I multiply the second column of V by -1 I then get A when doing $USV^T$. What am I doing wrong?

Ivan
  • 113

2 Answers2

1

There is no advantage in computing the SVD of a real symmetric matrix instead of the symmetric eigenvalue decomposition. The natural decomposition for a symmetric matrix is the symmetric eigenvalue decomposition (SED) and not the SVD. Everything which can be done using the SVD of a symmetric matrix can be done using the SED provided the eigenvalues are ordered such that the absolute values of the eigenvalues are in the non-increasing order.

The main aim of the SVD is NOT to compute products such as $A^tA$ and $AA^t$. The computation of such products is considered as the worst "sin" one can do by numerical analysts.

However, if you insists that you would like to compute the SVD of a symmetric matrix then you may follow the following steps. I will be using numbers not symbolic variables. In any case, symbolic variables are difficult to handle with large matrices.

(a) Compute the SEP of the matrix $A$. This is given by $$ A = U \Lambda U^t $$ where $$ U = \left[ \begin{array}{rr} 0.5257 &-0.8507 \\ -0.8507 & -0.5257 \end{array} \right] ,\;\;\; \Lambda = \left[ \begin{array}{rr} -0.6180 & 0 \\ 0 & 1.6180 \end{array} \right] . $$

(b) Order the eigenvalues such that the absolute values are non-decreasing. $$ \Lambda = \left[ \begin{array}{rr} 1.6180 & 0 \\ 0 & -0.6180 \end{array} \right] . $$ Because of the permutation, we have to apply the same to the eigenvector matrix. In that case, the new matrix $U$ is given by $$ U = \left[ \begin{array}{rr} -0.8507 & 0.5257 \\ -0.5257 & -0.8507 \end{array} \right] . $$ Note that, $A = U\Lambda U^t$ even after the reordering of the eigenvalues. That is, $$ A = \left[ \begin{array}{rr} -0.8507 & 0.5257 \\ -0.5257 & -0.8507 \end{array} \right] \left[ \begin{array}{rr} 1.6180 & 0 \\ 0 & -0.6180 \end{array} \right] \left[ \begin{array}{rr} -0.8507 & -0.5257 \\ 0.5257 & -0.8507 \end{array} \right]. $$

(c) Singular values are always non-negative. We negate the second eigenvalue and to compensate that negate the second row of $U^t$. Thus,

$$ A = \left[ \begin{array}{rr} -0.8507 & 0.5257 \\ -0.5257 & -0.8507 \end{array} \right] \left[ \begin{array}{rr} 1.6180 & 0 \\ 0 & 0.6180 \end{array} \right] \left[ \begin{array}{rr} -0.8507 & -0.5257 \\ -0.5257 & 0.8507 \end{array} \right] $$ Thus, the SVD is of the symmetric matrix is given by $$ A = U \Sigma V^t $$ where $$ U = \left[ \begin{array}{rr} -0.8507 & 0.5257 \\ -0.5257 & -0.8507 \end{array} \right] , \;\;\; \Sigma = \left[ \begin{array}{rr} 1.6180 & 0 \\ 0 & 0.6180 \end{array} \right] ,\;\;\; V = \left[ \begin{array}{rr} -0.8507 & -0.5257 \\ -0.5257 & 0.8507 \end{array} \right] $$

We have demonstrated how to get the SVD of a symmetric matrix from the SED. This can be done for any symmetric matrix, small or large. Note that the columns of $U$ and $V$ are identical except perhaps for a signature change.

Vini
  • 526
  • 3
  • 9
0

@Vini provides a good solution. Another presentation1 follows.

Definition of the singular value decomposition: $$ \mathbf{A} = \mathbf{U} \, \Sigma \, \mathbf{V}^{*} $$ $$ % \mathbf{A} \in \mathbb{C}^{m\times n}_{\rho}, \quad % \mathbf{U} \in \mathbb{C}^{m\times m}, \quad % \mathbf{V} \in \mathbb{C}^{n\times n}, \quad % \Sigma \in \mathbb{R}^{m\times n}, \quad % \mathbf{S} \in \mathbb{R}^{\rho\times\rho} % $$ The domain matrices are unitary: $$ \begin{align} \mathbf{U}^{*} \mathbf{U} &= \mathbf{U} \, \mathbf{U}^{*} = \mathbf{I}_{m} \\ \mathbf{V}^{*} \mathbf{V} &= \mathbf{V} \, \mathbf{V}^{*} = \mathbf{I}_{n} \end{align} $$ For this problem $m=n=\rho=2$ and both nullspaces are trivial.

1 Compute product matrix

$$ \mathbf{A}^{*} \mathbf{A} = % \left[ \begin{array}{cc} 1 & 1 \\ 1 & 0 \\ \end{array} \right] % \left[ \begin{array}{cc} 1 & 1 \\ 1 & 0 \\ \end{array} \right] % = % \left[ \begin{array}{cc} 2 & 1 \\ 1 & 1 \\ \end{array} \right] $$

2 Resolve eigensystem

$$ \lambda \left( \mathbf{A}^{*} \mathbf{A} \right) = \frac{1}{2} \left( 1 \pm \sqrt{5} \right) $$ The choice of eigenvectors involves a choice of sign. This is the origin of the sign ambiguities of the SVD. The signs are chosen here with the domain matrix $\mathbf{V}$ and they will imprint in the codomain matrix $\mathbf{U}$. $$ v_{1} = \left[ \begin{array}{c} \frac{1}{2} \left(1+\sqrt{5}\right) \\ 1 \end{array} \right], \quad % v_{2} = \left[ \begin{array}{c} \frac{1}{2} \left(1 - \sqrt{5}\right) \\ 1 \end{array} \right] $$

3 Assemble $\Sigma$

The singular values of $\mathbf{A}$ are the square roots of the nonzero eigenvalues of the product matrix: $$ \sigma = \sqrt{\lambda \left( \mathbf{A}^{*} \mathbf{A} \right)} = \sqrt{\frac{1}{2} \left( 1 \pm \sqrt{5} \right)}. $$ The eigenvalues are ordered: $$ \sigma_{1} \ge \sigma_{2} \gt 0 $$ The diagonal matrix of singular values is $$ \mathbf{S} = \left[ \begin{array}{cc} \sigma_{1} & 0 \\ 0 & \sigma_{2} \\ \end{array} \right] % = % \frac{1}{2} \left[ \begin{array}{cc} 1 + \sqrt{5} & 0 \\ 0 & 1 - \sqrt{5} \\ \end{array} \right]. $$ in the general case this matrix is embedded in a sabot matrix which guarantees conformability: $$ \Sigma = \left[ \begin{array}{cc} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \\ \end{array} \right] \in \mathbb{R}^{m\times n}. $$ Because both nullspace are trivial in this example, $\Sigma = \mathbf{S}.$

4 Assemble domain matrix $\mathbf{V}$

$$ \begin{align} \mathbf{V} = % \left[ \begin{array}{cc} \frac{v_{1}} {\lVert v_{1} \rVert} & \frac{v_{2}} {\lVert v_{2} \rVert} \end{array} \right] % &= % \left[ \begin{array}{cc} \frac{1}{\sqrt{ \frac{1}{2} \left(5+\sqrt{5}\right)}} \left[ \begin{array}{c} \frac{1}{2} \left(1+\sqrt{5}\right) \\ 1 \end{array} \right] & \frac{1}{\sqrt{ \frac{1}{2} \left(5-\sqrt{5}\right)}} \left[ \begin{array}{c} \frac{1}{2} \left(1-\sqrt{5}\right) \\ 1 \end{array} \right] \end{array} \right] \\[5pt] % &= \left[ \begin{array}{cc} \sqrt{\frac{1}{10} \left(5+\sqrt{5}\right)} & -\sqrt{\frac{1}{10} \left(5-\sqrt{5}\right)} \\ \sqrt{\frac{1}{10} \left(5-\sqrt{5}\right)} & \sqrt{\frac{1}{10} \left(5+\sqrt{5}\right)} \\ \end{array} \right] % \end{align} % $$

5 Construct codomain matrix $\mathbf{U}$

From the definition $$ \mathbf{A} = \mathbf{U} \, \mathbf{S} \, \mathbf{V}^{*} \qquad \Rightarrow \qquad \mathbf{A} \, \mathbf{V} = \mathbf{S} \,\mathbf{U} $$ Therefore $$ \mathbf{U}_{k} = \sigma_{k}^{-1} \left[ \mathbf{A} \, \mathbf{V} \right]_{k} $$ After some algebra, $$ \mathbf{U} = \left[ \begin{array}{rr} \sqrt{\frac{1}{10} \left(5+\sqrt{5}\right)} & \sqrt{\frac{1}{10} \left(5-\sqrt{5}\right)} \\ \sqrt{\frac{1}{10} \left(5-\sqrt{5}\right)} & -\sqrt{\frac{1}{10} \left(5+\sqrt{5}\right)} \\ \end{array} \right] $$

Verify decomposition $$ \begin{align} % \mathbf{A} &= \mathbf{U} \, \mathbf{S} \, \mathbf{V}^{*} \\ % A \left[ \begin{array}{cc} 1 & 1 \\ 1 & 0 \\ \end{array} \right] % &= % U \left[ \begin{array}{cc} \sqrt{\frac{1}{10} \left(5+\sqrt{5}\right)} & \sqrt{\frac{1}{10} \left(5-\sqrt{5}\right)} \\ \sqrt{\frac{1}{10} \left(5-\sqrt{5}\right)} & -\sqrt{\frac{1}{10} \left(5+\sqrt{5}\right)} \\ \end{array} \right] % S \frac{1}{2} \left[ \begin{array}{cc} 1 + \sqrt{5} & 0 \\ 0 & 1 - \sqrt{5} \\ \end{array} \right] % V* \left[ \begin{array}{rr} \sqrt{\frac{1}{10} \left(5+\sqrt{5}\right)} & \sqrt{\frac{1}{10} \left(5-\sqrt{5}\right)} \\ -\sqrt{\frac{1}{10} \left(5-\sqrt{5}\right)} & \sqrt{\frac{1}{10} \left(5+\sqrt{5}\right)} \end{array} \right] % \end{align} $$

Compare to diagonalization

$$ \mathbf{A} = \mathbf{P} \, \mathbf{D} \, \mathbf{P}^{-1} $$ The diagonal matrix $\mathbf{D}$ uses the eigenvalues of $\mathbf{A}$, diagonalizer $\mathbf{P}$ is constructed from eigenvectors of $\mathbf{A}$. $$ % \mathbf{P} = \frac{1}{2} \left[ \begin{array}{cc} \left(1+\sqrt{5}\right) & 2 \\ \left(1-\sqrt{5}\right) & 2 \\ \end{array} \right], \quad % \mathbf{D} = \frac{1}{2} \left[ \begin{array}{cc} 1+\sqrt{5} & 0 \\ 0 & 1-\sqrt{5} \\ \end{array} \right], \quad % \mathbf{P}^{-1} = \frac{1}{2\sqrt{5}} \left[ \begin{array}{cc} 2 & -2 \\ -1+\sqrt{5} & 1+\sqrt{5} \\ \end{array} \right] % $$

dantopa
  • 10,342