Transpose of matrix inverse: $(AA^T)^{-1}A^Tb \stackrel{?}{=} (A^TA)^{-1}A^Tb$

Question

Given the matrix equation:

$$ x^TA^TA = b^TA $$

I'm trying to find the least squares solution (i.e.; trying to minimize $r=||Ax-b||$). The matrix $A$ is not necessarily symmetric.

When I solve it in following way I find $x$ as:

$$ \begin{array}{rcl} x^TA^TA &=& b^TA \\ x^T &=& b^TA(A^TA)^{-1} \\ x &=& (AA^T)^{-1}A^Tb \end{array} $$

But I find a different solution for $x$ when I solve it this way:

$$ \begin{array}{rcl} x^TA^TA &=& b^TA \\ A^TAx &=& A^Tb \\ x &=& (A^TA)^{-1}A^Tb \end{array} $$

Apparently the latter one is the correct solution. But what is wrong with the first one? What am I doing wrong?

score 3 · Accepted Answer · answered Apr 25 '13 at 08:40

3

The last step of the first approach should be, $x = ((A^TA)^{-1})^TA^Tb$

As $A^TA$ is symmetric, its inverse should also be symmetric. (if exist)

Thus $x = (A^TA)^{-1}A^Tb$

answered Apr 25 '13 at 08:40

Rein

score 2 · Answer 2 · answered Apr 25 '13 at 08:46

2

In the first derivation the transpose was done wrong on the RHS of the last step, ${(A^TA)^{-1}}^T$ = ${(A^TA)^{T}}^{-1}$ = ${(A^TA)^{-1}}$

answered Apr 25 '13 at 08:46

peterm

But why? Any references? – hkBattousai Apr 25 '13 at 09:15
@hkBattousai: The transpose of a product is always the reverse of the transposes: $(AB)^T = B^T A^T$. So $(A^T A)^T = A^T (A^T)^T = A^T A$. And inverse and transpose commute. – Nick Matteo Mar 26 '14 at 17:50

2 Answers2