How can I show that $(A^TA)^{-1} A^T$ minimizes $\mathbf{Tr}(X^TX)$ over all matrices $X$ such that $XA = I$?
[Note that $A$ is an $m \times n$ matrix.]
I've tried rearranging the trace, and expanding using eigenvalues, and haven't had any luck yet!
Asked
Active
Viewed 216 times
1 Answers
6
We have the following least-norm problem
$$\begin{array}{ll} \text{minimize} & \mathrm \| \mathrm X \|_{\text F}^2\\ \text{subject to} & \mathrm X \mathrm A = \mathrm I\end{array}$$
where $\mathrm A \in \mathbb R^{m \times n}$ is given. Let the Lagrangian be
$$\mathcal L (\mathrm X, \Lambda) := \frac 12 \mathrm \| \mathrm X \|_{\text F}^2 + \langle \Lambda, \mathrm X \mathrm A - \mathrm I \rangle$$
Taking the derivatives of the Lagrangian with respect to $\rm X$ and $\Lambda$ and finding where they vanish, we obtain a system of coupled matrix equations
$$\begin{aligned} \mathrm X + \Lambda \mathrm A^\top &= \mathrm O\\ \mathrm X \mathrm A &= \,\mathrm I\end{aligned}$$
Right-multiplying the first matrix equation by $\rm A$, we obtain $\mathrm I + \Lambda \mathrm A^\top \mathrm A = \mathrm O$. Assuming that $\rm A$ has full column rank, then $\mathrm A^\top \mathrm A$ is invertible and, thus, $\Lambda = - ( \mathrm A^\top \mathrm A )^{-1}$ and $\mathrm X_{\text{LN}} := \color{blue}{( \mathrm A^\top \mathrm A )^{-1} \mathrm A^\top}$.