1

Suppose $A \in \mathbb{C}^{m \times n}$. I know that if $m \ge n$ and $A$ has full rank, then $A^*A$ is invertible so $A^*A x = A^*b$ has a solution. However, looking at this question, I believe that this system has a solution even if $A$ is not of full rank, and even if $m \le n$.

I believe I have a solution using the SVD, but it is rather long and I'm sure that there is a better way. My question is: can you please give me a hint on how to prove this in an easier way?

Solution using SVD

Let $A^* = U \Sigma V^*$ be an SVD of $A^*$. That is $U, V$ are unitary and $$\Sigma = \begin{bmatrix} \sigma & 0\\ 0^T & 0 \end{bmatrix}$$ where $\sigma = diag(\sigma_1, ..., \sigma_r)$ where $r$ is the rank of $A^*$.

The equation $A^*A x = A^*b$ becomes $$U \Sigma \Sigma^TV^* x= U \Sigma V^* b$$ $$\Sigma(\Sigma^TV^* x- V^*b) = 0$$ $$\Sigma(\Sigma^Ty- V^*b) = 0$$ where $y = V^*x$. Since $V$ is invertible, we are free to select any $y$ that we want. Now, $$\Sigma^Ty = [\sigma_1 y_1 \cdots \sigma_r y_r \ \ 0 \cdots 0]^T$$ Select $y_1, ..., y_r$ so that the first $r$ components of $\Sigma^Ty$ match the first $r$ components of $V^*b$. Then the first $r$ components of $\Sigma^Ty- V^*b$ are $0$, and as a result, $$\Sigma(\Sigma^Ty- V^*b) = 0.$$

user56202
  • 930

1 Answers1

1

Essentially, what you are trying to prove is that the column space of $A^*A$ (the set of vectors $y$ for which $A^*Ax = y$ has a solution) is equal to the column space of $A^*$ (the set of possible vectors $y = A^*b$). Because the column space of a matrix $M$ is equal to the orthogonal complement of the kernel of $M^*$ (the "left kernel" of $M$), it is equivalent to show that the kernel of $A^{**} = A$ is equal to the kernel of $(A^*A)^* = A^*A$.

It is easy to see that $\ker(A) \subseteq \ker(A^*A)$. Indeed, if $Ax = 0$, then $$ A^*Ax = A^*(Ax) = A^*(0) = 0. $$ On the other hand, we can see that $\ker(A) \supseteq \ker(A^*A)$ as follows. If $x$ is such that $A^*Ax = 0$, then it holds that $$ 0 = x^*0 = x^*A^*Ax = (Ax)^*Ax = \|Ax\|^2. $$ That is, we have $\|Ax\| = 0$ and hence $Ax = 0$.

Ben Grossmann
  • 225,327
  • Thank you very much for this beautiful solution! I think what I can remember from here is that one should always try to work with null spaces instead of column spaces if possible, because equations of the form $My = 0$ are easier than those of the form $My = v$. This is a potential exam question for me and I'm not sure what theorems I can use, so I will try to transform this proof into one which does not appeal to null spaces. – user56202 Mar 13 '21 at 15:20