2

Suppose we have a $n \times m$ rectangular matrix $A$ with row space $\text{row}(A)$ and column space $\text{col}(A)$.

What are the row spaces and the column spaces of $A^T A$ and $A A^T$?

Eugene Zhang
  • 16,805
Alexey Grigorev
  • 305

2 Answers2

4

Since $A^TA$ and $AA^T$ are symmetric, we have $$ \operatorname{row}(A^TA) = \operatorname{col}(A^TA) \\ \operatorname{row}(AA^T) = \operatorname{col}(AA^T) $$

Note that each column of $A^TA$ is in $\operatorname{row}(A)$, thus $\operatorname{row}(A^TA) = \operatorname{col}(A^TA) \subseteq \operatorname{row}(A)$. Since $\operatorname{rank}(A^TA) = \operatorname{rank}(A)$, thus in fact $$ \operatorname{row}(A^TA) = \operatorname{col}(A^TA) = \operatorname{row}(A) $$

A similar derivation shows that $$ \operatorname{row}(AA^T) = \operatorname{col}(AA^T) = \operatorname{col}(A) $$

Alexey Grigorev
  • 305
PSPACEhard
  • 10,283
  • Why from $\operatorname{rank}(A^T A) = \operatorname{rank}(A)$ it follows that $\operatorname{row}(A^T A) = \operatorname{row}(A)$? – Alexey Grigorev May 08 '15 at 08:32
  • @AlexeyGrigorev $\operatorname{col}(A^TA)$ is a subspace of $\operatorname{row}(A)$, as shown in the answer, thus $\operatorname{rank}(A^TA) \leq \operatorname{rank}(A)$. But it is a fact in linear algebra that $\operatorname{rank}(A^TA) = \operatorname{rank}(A)$ for any $A$, this implies $\operatorname{col}(A^TA)$ is in fact the space $\operatorname{row}(A)$. Also note that $\operatorname{col}(A^TA) = \operatorname{row}(A^TA)$. – PSPACEhard May 08 '15 at 08:40
  • Why are each of the columns of $A^TA$ in row(A)? – FafaDog Oct 02 '20 at 21:50
  • I think he means each column is in the column space of ,,, – Steven Alexis Gregory Jul 24 '21 at 04:20
3

If $Ax=0$, then $A^TAx=0$, which means $N(A)\subset N(A^TA)$, $N(A)$ is the null space of $A$.

On the other hand, if $A^TAx=0$, then $$ x^TA^TAx=0, \hspace{3 mm}\text{or} \hspace{3 mm} \|Ax\|=0 $$ which means $Ax=0$, and thus $$N(A^TA)\subset N(A) \hspace{3 mm}\text{and} \hspace{3 mm} N(A^TA)=N(A) $$ Since $\operatorname{rank}(A) = n-N(A)$, there is $$ \operatorname{rank}(A) = \operatorname{rank}(A^TA) $$

Suppose $A=[\alpha_1,\cdots,\alpha_n] \hspace{2 mm} (\alpha_i $ is the column vector of $A$), then $$ A^TA=A^T[\alpha_1,\cdots,\alpha_n]=[A^T\alpha_1,\cdots,A^T\alpha_n] $$

For each column of $A^TA$

\begin{align} A^T\alpha_i&=[\beta_1 \cdots \beta_n]\alpha_i \hspace{20 mm} (\beta_i \text{ is the column of } A^T \text{ and row of }A) \\ &=[\beta_1 \cdots \beta_n]\left[ \begin{array}{} a_{i1} \\ \vdots \\ a_{in} \\ \end{array} \right] \\ &=\sum \limits_{j=1}^{n}a_{ij}\beta_j \end{align}

So column of $A^TA$ is the linear combination of rows of $A$, or $$ \operatorname{col}(A^TA) =\operatorname{row}(A) $$

Obviously $\operatorname{rank}(A^T)=\operatorname{rank}(A)$, so $$ \operatorname{row}(A^TA) =\operatorname{col}(A^TA) =\operatorname{row}(A) $$

Similarly we have $$ \operatorname{row}(AA^T) =\operatorname{col}(AA^T) =\operatorname{row}(A^T)=\operatorname{col}(A) $$

Eugene Zhang
  • 16,805