3

Consider the following Least Square Minimization problem:$min_{x \in \mathbb C^3} |x_1 +x_3-3|^2+|x_2 -x_3|^2+|x_1+x_3-4|^2$ Find the pseudo-inverse of the matrix $A=\begin{bmatrix} 1 & 0 & 1 \\ 0 & 1 & -1 \\ 1 & 0 & 1 \end{bmatrix}$ without computing singular values of $A$.

It seems we cannot use normal equations to denote $A^{+}=(A^{T}A)^{-1}A^{T}$ since $A^{T}A^{-1}$ is not invertible. Any hints would be appreciated.

Khoa Le
  • 107
  • 1
    What exactly is your definition of a pseudoinverse? – sranthrop Dec 29 '16 at 17:08
  • @sranthrop presumably Moore-Penrose – Ben Grossmann Dec 29 '16 at 17:14
  • @Omnomnomnom Yes. – Khoa Le Dec 29 '16 at 17:16
  • Why not just sovlve the system $AA^+A=A,A^+AA^+=A^+,AA^+=(AA^+)^T,A^+A=(A^+A)^T$? Do you have to do your calculations by hand? – sranthrop Dec 31 '16 at 18:35
  • @sranthrop How to solve? – Khoa Le Dec 31 '16 at 21:02
  • Well, just set $A^+=\begin{pmatrix}a & b & c \ d & e & f \ g & h & i\end{pmatrix}$ and write down the four equations above. You'll get a linear system with 9 variables, that has a unique solution. – sranthrop Dec 31 '16 at 22:33
  • @sranthrop Thanks! I happened to know someone wrote the answer but I don't know why. It's like this. So, it's written like if we consider the minimum norm solution of the least square problem $$\underset{{x \in \mathbb C^3}} \min |x_1 +x_3 -b_1|^2 + |x_2 -x_3 -b_2|^2 +|x_1 +x_3 -b_3|^2$$ and the solution has the form $x_1 +x_3 =(b_1 + b_3)/2$ and $x_2 -x_3 -b_2=0$ and $-x_1 +x_2 +x_3=0$ – Khoa Le Dec 31 '16 at 22:53

2 Answers2

1

There are several methods to calculate $A^+$ without calculating singular values. Here are two of them.

Method 1.

Use the formula $A^+=\lim_{t\to0}A^T(AA^T+tI)^{-1}$. In your case, you should get something like \begin{align*} A^+&=\lim_{t\to0}A^T(AA^T+tI)^{-1}\\ &=\lim_{t\to0}\pmatrix{1&0&1\\ 0&1&0\\ 1&-1&1}\pmatrix{t+2&-1&2\\ -1&t+2&-1\\ 2&-1&t+2}^{-1}\\ &=\lim_{t\to0}\frac1{t^2+6t+6} \pmatrix{t+2&2&t+2\\ 1&t+4&1\\ t+1&-t-2&t+1}\\ &=\frac16\pmatrix{2&2&2\\ 1&4&1\\ 1&-2&1}. \end{align*}

Method 2.

Use the property that if $X$ has orthonormal columns or $Y$ has orthonormal rows, then $(XY)^+=Y^+X^+$. Decompose $A$ as the product $\pmatrix{1&0\\ 0&1\\ 1&0}\pmatrix{1&0&1\\ 0&1&-1}$. Gram-Schmidt orthogonalisation gives \begin{align*} A &=\left[ \pmatrix{\frac1{\sqrt{2}}&0\\ 0&1\\ \frac1{\sqrt{2}}&0} \pmatrix{\sqrt{2}\\ &1} \right] \left[ \pmatrix{\sqrt{2}&0\\ -\frac1{\sqrt{2}}&\sqrt{\frac32}} \pmatrix{\frac1{\sqrt{2}}&0&\frac1{\sqrt{2}}\\ \frac1{\sqrt{6}}&\sqrt{\frac23}&-\frac1{\sqrt{6}}} \right]\\ &=\underbrace{\pmatrix{\frac1{\sqrt{2}}&0\\ 0&1\\ \frac1{\sqrt{2}}&0}}_{X} \underbrace{\pmatrix{2&0\\ -\frac1{\sqrt{2}}&\sqrt{\frac32}}}_{Y} \underbrace{\pmatrix{\frac1{\sqrt{2}}&0&\frac1{\sqrt{2}}\\ \frac1{\sqrt{6}}&\sqrt{\frac23}&-\frac1{\sqrt{6}}}}_{Z}. \end{align*} Since $X$ has orthonormal columns, $Y$ is invertible and $Z$ has orthonormal rows, we have $$ A^+=(XYZ)^+=Z^+Y^+X^+=Z^TY^{-1}X^T. $$

user1551
  • 139,064
0

$$ \mathbf{A}^{\dagger} = \frac{1}{6} \left[ \begin{array}{rrr} 2 & -2 & 2 \\ 1 & -4 & 1 \\ 1 & 2 & 1 \\ \end{array} \right] $$

dantopa
  • 10,342