4

I understand that the Singular Value Decomposition is defined as SVD = $U\Sigma V^T$ , but I am slightly confused about the calculations when the matrix is not square. For example, I have the matrix: $$ \begin{bmatrix} 1 & -1 \\ -2 & 2 \\ 2 & -2 \end{bmatrix} $$ When I am solving for $V$, however, I am missing the last component. Have I done something wrong when calculating for matrices that are not square matrices?

$$\det(A^T A - \lambda I) = \begin{bmatrix} 2 - \lambda & -4 & 4 \\ -4 & 8 - \lambda & -8\\ 4 & -8 & 8 - \lambda \end{bmatrix} $$ $\lambda = 0, 2, 16$ Eigenvectors respectively are: \begin{bmatrix} 1 \\ 1/2 \\ 0 \end{bmatrix}
\begin{bmatrix} 1 \\ 2/7 \\ -2/7 \end{bmatrix}
\begin{bmatrix} 0 \\ 1 \\ -1 \end{bmatrix}
Therefore $$\Sigma = \begin{bmatrix} \sqrt 2 & 0 & 0 \\ 0 & \sqrt 16 & 0 \\ 0 & 0 & 0 \end{bmatrix}$$ Also, $$V = \begin{bmatrix} 7/\sqrt 57 & 0 & 2/\sqrt 5 \\ 2/\sqrt 57 & 1/\sqrt 2 & 1/\sqrt 5 \\ 2/\sqrt 57 & -1/\sqrt 2 & 0 \end{bmatrix}$$

This is the portion I am confused about. Is $U = AV / \sqrt\lambda $ ? What if I have am missing a vector so that I can only get the first two columns of $U$?

Git Gud
  • 31,356
Shanty
  • 279
  • To answer the question you should describe your process of obtaining an SVD, perhaps even exemplifying with a working example. – Git Gud Mar 03 '15 at 01:21
  • 1
    Will do! I may need a few minutes to type everything out. – Shanty Mar 03 '15 at 01:27
  • Your matrix has 3 rows and 2 columns, so just to make the dimensions match you must have $U$ with 3 rows and $V^T$ with 2 columns. Is that what you mean by the "missing last component"? –  Mar 03 '15 at 01:35
  • I believe so. I'm having issues with the calculations. – Shanty Mar 03 '15 at 01:47
  • @Shanty I don't have time to go into detail. But $\Sigma$ must be the same size as $A$, i.e., $3\times 2$. And $U$ is going to be $3\times 3$ while $V$ is going to be $2\times 2$. You didn't actually find $\det\left(A^A-\lambda I\right)$, you found $\det\left(AA^-\lambda I\right)$. If there were no computational mistakes, you got your $U$. – Git Gud Mar 03 '15 at 01:56
  • I'm glad I'm on the right track then! Thank you! – Shanty Mar 03 '15 at 02:31
  • @Shanty By the way, I didn't realise that the $$ in your question was just multiplication, I thought it was transconjugation, so in my comment above replace $^$ with $^T$. – Git Gud Mar 03 '15 at 10:18

1 Answers1

2

First, correct a simple error $$ \mathbf{A}^{*}\mathbf{A} = \left[ \begin{array}{rr} 9 & -9 \\ -9 & 9 \\ \end{array} \right]. $$

This system matrix $\mathbf{A}$ is a rank one matrix (column 2 = $-$column 1) with singular value decomposition $$ \begin{align} \mathbf{A} &= \mathbf{U}\, \Sigma \, \mathbf{V}^{*} \\ % \left[ \begin{array}{rr} 1 & -1 \\ -2 & 2 \\ 2 & -2 \\ \end{array} \right] &= % U \left[ \begin{array}{ccc} \frac{1}{3} \color{blue}{\left[ \begin{array}{r} -1 \\ 2 \\ -2 \end{array} \right]} & \frac{1}{\sqrt{5}} \color{red}{\left[ \begin{array}{r} -2 \\ 0 \\ 1 \end{array} \right]} & \frac{1}{3\sqrt{5}} \color{red}{\left[ \begin{array}{r} 2 \\ 5 \\ 4 \end{array} \right]} % \end{array} \right] % sigma \left[ \begin{array}{cc} 3 \sqrt{2} & 0 \\ 0 & 0 \\ 0 & 0 \\ \end{array} \right] % V \frac{1}{\sqrt{2}} \left[ \begin{array}{rc} \color{blue}{-1} & \color{blue}{1} \\ \color{red}{1} & \color{red}{1} \\ \end{array} \right]. \end{align} $$ Blue vectors are in range spaces, red vectors in null spaces.

The thin SVD uses the range space components only: $$ \mathbf{A} = % U \frac{1}{3} \color{blue}{\left[ \begin{array}{r} -1 \\ 2 \\ -2 \end{array} \right]} % S \left( 3\sqrt{2} \right) % V \frac{1}{\sqrt{2}} \left[ \begin{array}{rc} \color{blue}{-1} & \color{blue}{1} \end{array} \right]. $$

You may benefit from this example: SVD and the columns — I did this wrong but it seems that it still works, why?
dantopa
  • 10,342