Computing least square solution when eigenvalue and eigenvectors are known.

Question

Suppose a matrix $A$ has eigenvalues 0, 3, 7 and eigenvectors $\mathbf{u, v, w,}$ respectively. Find the least square minimum length solution for $A\mathbf{x} = \mathbf{u+v+w}$.

This was on our engineering math final exam last year and we've tried some techniques about Moore-Penrose pseudoinverse, which didn't seem to work. Can someone help?

Answer 1

Consider the basis made up of the eigenvectors, say $B$. Then use the fact that $A$ is diagonisable. Then you can appky least squares formula to $D[x]_B=[Ax]_B=[1 \ 1 \ 1]^T$.

Answer 2

$Ax$ owns to the set $E$ generated by $v$ and $w$.

Let us call $u_0$ the projection of $u$ on $E$.

Then $u =u_0+u_1$ with $u_1$ orthogonal to $E$.

The best you can obtain is $Ax = u_0 + v + w$, projection theorem.
As $u_0$ owns to $E$, it can be represented as $u_0 =\alpha v + \beta w$, and one solution is $$ x = \frac{1+\alpha}{3}v + \frac{1+\beta}{7}w $$

This simple solution minimizes the error on the output.

Answer 3

The problem here is that there is no such vector $x$ that satisfies $$ Ax = u+v+w.$$ We can see this easily from the fact that $$\operatorname{ran} A =\operatorname{span}\{v,w\}.$$

$\textbf{EDIT:}$ Our problem can be written as follows. First, find $y \in \operatorname{ran}A$ (that is, $y=Ax$ for some $x$) such that $$ \lVert y-(u+v+w)\rVert \leq \lVert z-(u+v+w)\rVert,\quad\forall z\in\operatorname{ran}A. $$ Then find $x\in\mathbb{R}^3$ such that $Ax=y$ and $$ \lVert x\rVert\leq \lVert x'\rVert, \quad\forall x':Ax'=y. $$

Firstly, the solution $y$ is given by the orthogonal projection $P(u+v+w)$ of $u+v+w$ onto $\operatorname{ran} A$. What we need to do is to compute $Pu$ since $P(v+w) = v+w$. To do this, let $$ Pu = \alpha v +\beta w.$$ It should be that $$ u-Pu = u-(\alpha v +\beta w) \perp v,w. $$ This gives a system of equations about $\alpha,\beta$: $$ \langle u,v\rangle=\alpha \langle v,v \rangle +\beta \langle w,v \rangle, $$ and $$ \langle u,w\rangle= \alpha\langle v,w \rangle+ \langle w,w \rangle. $$ (If $A$ is normal, then we have $\langle u,v\rangle=\langle v,w\rangle=\langle w,v\rangle=0$. But this is not true in general.)

Solving this equation gives $y =(\alpha+1)v + (\beta+1)w$. Finally, note that $$ A^{-1}(\{y\}) = \{\frac{(\alpha+1)}{3}v+\frac{(\beta+1)}{7}w+\gamma u\;|\;\gamma\in\mathbb{R}\}.$$ To minimize $\lVert x \rVert$ over $x\in A^{-1}(\{y\})$, it must be that $$ x =\frac{(\alpha+1)}{3}v+\frac{(\beta+1)}{7}w+\gamma u \perp u. $$ This gives us $$ x = \frac{(\alpha+1)}{3}\left(v-\frac{\langle v,u\rangle}{\lVert u \rVert^2}u\right) + \frac{(\beta+1)}{7}\left(w-\frac{\langle w,u\rangle}{\lVert u \rVert^2}u\right). $$