0

I went through the answer to the question Find the Matrix Projection of a Symmetric Matrix onto the set of Symmetric Positive Semi Definite (PSD) Matrices. I have a doubt about the answer given. $$ \|\Lambda_X -V^{\top}A V\|^2_F = \sum_{i \neq j}b_{ij}^2+\sum_i (\Lambda_{X_{ii}}-b_{ii})^2 $$

Minimum occurs when $b_{ij}=0$, and $\Lambda_{X_{ii}}=b_{ii}$. Note that $\Lambda_{X_{ii}} \geq 0$, so $\Lambda_{X_{ii}}=\max\{0,b_{ii}\}$. B is a diagonal matrix. So, V=U.

Our optimization variables are $\Lambda_X, V$. Hence in the final equation we have freedom over $b_{ij}, b_{ii}$ plus some constraints as $ B = V^{\top}A V\ $. I don't understand why is $b_{ij}=0$ and V=U the optimal solution, since $b_{ij}, b_{ii}$ are dependent on each other and $\Lambda_{X_{ii}} \geq 0$. Can we not find a better solution like the square of negative $b_{ii}$ is smaller and in turn, $b_{ij}^2$ is not zero but overall cost is minimised?

Royi
  • 8,711
Shiv Tavker
  • 1,612
  • 7
  • 19

1 Answers1

1

It might be easier to consider the problem in the following way. Let $W$ be the sum of eigenspaces of $A$ corresponding to positive eigenvalues, $P_1$ be the orthogonal projector on to $W$, and $P_2=I-P_1$ be the orthogonal projector on to $W^\perp$. Then every symmetric matrix $M$ can be written as an orthogonal sum $M_1+M_2+M_3$ where $M_1=P_1MP_1,\,M_2=P_2MP_2$ and $M_3=M-M_1-M_2$.

If this sounds too abstract to you, assume that $A=\operatorname{diag}(a_1,a_2,\ldots,a_n)$ where $a_1,a_2,\ldots,a_k>0\ge a_{k+1},a_{k+2},\ldots,a_n$. Then $P_1=I_k\oplus0,\ P_2=0\oplus I_{n-k}$ and $M_1,M_2,M_3$ are respectively symmetric matrices of the forms $$ \pmatrix{\ast&0\\ 0&0_{(n-k)\times(n-k)}},\ \pmatrix{0_{k\times k}&\ast\\ \ast&0_{(n-k)\times(n-k)}}\text{ and } \pmatrix{0_{k\times k}&0\\ 0&\ast}. $$

Now, decompose $A$ and $X$ accordingly as $A_1+A_2+A_3$ and $X_1+X_2+X_3$. By the definition of $W$, we have $A_1\succ0,\,A_2\preceq0$ and $A_3=0$. Hence $$ \|X-A\|_F^2=\|X_1-A_1\|_F^2+\|X_2-A_2\|_F^2+\|X_3\|_F^2 $$ and we may consider the summands on the RHS one by one:

  • By Sylvester's criterion, if $X$ is positive semidefinite, so are $X_1$ and $X_2$. Therefore $\|X-A\|_F^2$ is minimised only if $X_3=0$.
  • As both $X_2$ and $-A_2$ are positive semidefinite, Weyl's inequality implies that $\|X_2-A_2\|_F^2\ge\|-A_2\|_F^2$ and equality holds only when $X_2=0$. Therefore $\|X_2-A_2\|_F^2$ is minimised only when $X_2=0$.
  • Clearly, $\|X_1-A_1\|_F^2$ is minimised only if $X_1=A_1$.

Consequently, the global minimiser of $\|X-A\|_F^2$ is given by $X=A_1$. When $A$ is a diagonal matrix, this just means $X$ is the entrywise positive part of $A$.

user1551
  • 139,064