8

Is it possible to draw any parallels between the SVD and QR decomposition of a matrix?

Moreover, for a given matrix $\mathbf{A}\in\mathbb{R}^{n\times m}$, under what conditions, the $\mathbf{U}$ matrix coming from singular value decomposition of $\mathbf{A}$ is equal to $\mathbf{Q}$ matrix obtained via QR-decomposition of $\mathbf{A}$

NAASI
  • 997

2 Answers2

9

I don't know if there really is a link between them... Notice that those decompositions are not unique, so the question is a bit unclear. But here is some thoughts on your second question. Maybe it can help you ...

Consider $A \in \Bbb C^{m \times n}$, $m \geq n$. (In the other case, just take the transpose...)

We first fix some notations :

1) A $QR$ decomposition of $A$ is any decomposition $A = QR = Q_1R_1$, where $Q = \begin{pmatrix} Q_1 & Q_2 \end{pmatrix} \in \Bbb C^{m \times m}$ is a unitary matrix, $Q_1 \in \Bbb C^{m \times n}$, and $R = \begin{pmatrix} R_1 \\ 0\end{pmatrix} \in \Bbb C^{m \times n}$, $R_1 \in \Bbb C^{n \times n}$ is upper triangular. We say such a $R$ is upper triangular as well (even if it's not a square matrix).

2) A $SVD$ of $A$ is any decomposition $A = U \Sigma V^*$, where $\Sigma = \begin{pmatrix} \Sigma_1 \\ 0\end{pmatrix} \in \Bbb C^{m \times n}$, $\Sigma_1 \in \Bbb C^{n \times n}$ is diagonal with non negative entries (we say such a $\Sigma$ is diagonal with non negative diagonal entries) and $U \in \Bbb C^{m \times m}$, $V \in \Bbb C^{n \times n}$ are unitary.

Suppose that $A$ has full column rank and let $A = QR$ be a $QR$ decomposition for $A$. There exists a $SVD$ of $A$ such that $Q = U$ if and only if $A^*A$ is diagonal.

Suppose $A^*A$ is diagonal. Then, $A^*A = R^*Q^*QR = R^*R = R_1^*R_1$ so $R_1$ must be diagonal (using the fact that $A$ has full column rank and $R_1$ is upper triangular). Then $A = QR = QRI$ and we can multiply some columns of $R$ and the corresponding rows of $I$ by $-1$ so that we get $A = QR'J = U \Sigma V$, where $R'$ is diagonal with non negative diagonal entries, $Q$ and $J$ are unitary. This is a $SVD$ decomposition for $A$ with $Q = U$.

Now, suppose $Q = U$ for some $SVD$ of $A$. Then we have $\Sigma_1$ is diagonal with strictly positive entries (since $A$ has full column rank) and $\Sigma V^* = R \Rightarrow \Sigma_1 V^* = R_1$ so $V^*$ is upper triangular. Since $V^*$ is unitary and upper triangular, it must be diagonal. But the columns of $V$ form a basis of eigenvectors of $A^*A$. This means $e_i \in \Bbb R^n$ is an eigenvector of $A^*A$ for all $i = 1, ..., n$. Therefore, $A^*A$ is diagonal.

Desura
  • 2,011
  • This is a very nice result, @Desura. Does it exist in a paper or book anywhere that you know of? Or is this your own work? – kdbanman Dec 15 '19 at 06:58
  • 1
    @kdbanman I'm sorry, I deduced it by myself so I cannot think of any book or paper. – Desura Dec 16 '19 at 07:58
0

Singular value decomposition factors matrix A into three matrices, $A = U S V^T$

The QR decomposition factors matrix A into two matrices, $A = Q R$

The question you should have asked is for the relationship between the SVD and the complete orthogonal decomposition. https://en.wikipedia.org/wiki/Complete_orthogonal_decomposition, $A = U T V^T$, where

$A$ is $m$ by $n$
$U$ is $m$ by $r$
$T$ is $r$ by $r$
$V$ is $n$ by $r$
and $r$ is the rank of $A$

The similarities between SVD and COD: $U$ and $V$ have orthogonal columns. Both SVD and COD are rank-revealing decompositions. Both SVD and SVD can give the least-squares solution to an underdetermined or overdetermined system of linear equations $Ax = b$

The differences: SVD is near unique. COD depends on the order of columns and rows in $A$. $S$ is a diagonal matrix. $T$ is a triangular matrix. The COD can be computed in a fixed number of steps, by performing performing the QR factorization twice, first $A = UR$, second $R^T = V T^T$ To compute the SVD requires iteration.

A bad way to compute the SVD is to repeat the QR factorization on A:

Method PDQ:
Factor $A = U R$
Factor $R^T = D^T V^T$
Do:
Multiply $P := U D$
Factor $U D := P$
Multiply $Q^T := V D^T$
Factor $V D^T := Q^T$
:Loop until D is diagonal

SVD is implemented explicitly in LAPACK. COD is implemented to solve least-squares problems. LAPACK certainly has much better algorithms for computing the SVD than given here.

andy_a
  • 86