0

I need to find the least squares solution using the Pseudo=oinverse, min$_\mathbf x $$_\in$$_\Bbb R$3||A$\mathbf x - \mathbf b$||$_2$, where

A= $\begin{bmatrix} 1 & 3 & 5\\ 1 & 1 & 0\\ 1 & 1 & 2\\ 1 & 3 & 3\\ \end{bmatrix}$ and $\mathbf b$= $\begin{bmatrix} 3\\ 5\\ 7\\ -3\\ \end{bmatrix}$.

And the pseudo inverse $A^\dagger$ = ($A^T$$A$)$^{-1}$$A^T$.

How do we apply $A^\dagger$ to solve the problem?

user409800
  • 117
  • 1
  • 8

1 Answers1

1

$A^\dagger b$ provides the vector $x$ that minimizes $\|Ax - b\|_2$ (in the case that $A^TA$ is invertible, this minimum is unique). Thus, the solution to your problem will be $$ \|A(A^\dagger b) - b\|_2 = \|(AA^\dagger - I)b\| $$

Ben Grossmann
  • 225,327
  • what is the proof that the pseudo-inverse minimizes the least square problem? – Charlie Parker Oct 22 '17 at 05:40
  • @Pinocchio a few proofs are given here – Ben Grossmann Oct 22 '17 at 15:01
  • thanks I will check it out! But before I do, super quick question (to make sure my point is addressed explicitly somewhere). Does pseudo-inverse minimize least squares even if the matrix is not full column nor full row rank? i.e. $r<n,r<m$ – Charlie Parker Oct 22 '17 at 16:57
  • @Pinocchio here's the way to think about it: among the least-squares solutions to $Ax = b$, $x = A^\dagger b$ is the one with minimal $|x|$. If $A$ has full column-rank, then there will be a unique least squares solution. If $A$ has full row-rank, then every least squares solution will be an exact solution. – Ben Grossmann Oct 23 '17 at 00:30