0

Let $0<a<b$, $\mathbb{S}^n$ be the set of all $n\times n$ real symmetric matrices, and $\mathcal{V} \subset\mathbb{S}^n $ be the set of symmetric matrices such that $aI\le X \le b I$, where $\le $ means the Loewner order. Consider the orthogonal projection $\Pi:\mathbb{S}^n \to \mathcal{V}$ such that for all $A\in \mathbb{S}^n$, $$ \Pi(A)=\arg\min_{X\in \mathcal{V}}\|X-A\|^2. $$ where $\|\cdot\|$ indicates the Frobenius norm or spectral norm.

I was wondering whether there is a simple procedure to evaluate $\Pi$ (for one of the above matrix norms). Can the projection operator $\Pi$ be expressed in terms of projecting the corresponding eigenvalues of $A$?

As shown here, the claim holds if $a=0$ and $b=\infty$. The argument relies on any positive semidefinite matrices have nonnegative diagonals, which can not be easily extended to general $a,b$.

John
  • 13,204

1 Answers1

2

You can still reduce the problem to diagonals. For the Frobenius norm, von-Neumann's trace inequality gives

$$ \|X - A\|_F^2 = \|X\|_F^2 + \|A\|_F^2 - 2 \langle X, A\rangle \geq \|X\|_F^2 + \|A\|_F^2 - 2 \sum_{i = 1}^n \sigma_i(X)\sigma_i(A) \\ = \| \Sigma_X \|_F^2 + \| \Sigma_A \|_F^2 - 2 \langle \Sigma_A, \Sigma_X \rangle = \| \Sigma_X - \Sigma_A \|_F^2 $$

where $\sigma_i(\cdot)$ denotes the $i$-th singular value and $\Sigma_A, \Sigma_X$ are diagonal matrices holding the singular values of $A$ and $X$.

Note that the inequality is actually an equality when $X$ and $A$ commute; i.e., $X$ and $A$ have the same set of eigenvectors. Therefore, the optimal solution will be

$$ X = U \mathbf{diag}\left(\mathrm{proj}_{[a, b]} \sigma_i(A)\right) U^*, $$

where $U$ is the matrix of eigenvectors of $A$.

VHarisop
  • 3,840
  • Thanks for your answer. May you kindly provide a reference for the equality condition of von-Neumann's trace inequality? – John Aug 05 '22 at 23:18
  • I don't have an easily accessible reference at hand, but one way to verify that the condition is sufficient for equality is to simply set $X$ equal to $U D U^*$ for any diagonal matrix $D$ and $U$ being the eigenvectors of $A$ and notice that it is equal to the lower bound. – VHarisop Aug 06 '22 at 17:54
  • To verify the claim, have you assumed that $A=U\Sigma_A U^$? As $\Sigma_A$ contains the singular values of $A$, and $A$ is not necessarily positive semidefinite, it seems that $U$ is not the eigenvectors of $A$. In fact, I feel the correct formula is $\Pi(A)= U \mathbf{diag}\left(\mathrm{proj}_{[a, b]} \sigma_i(A)\right) V^$, where $A=U\Sigma_A V^*$ is the singular value decomposition of $A$ (here a related post). Please kindly let me know if I overlooked anything. – John Aug 07 '22 at 23:27
  • May you kindly clarify whether you have assumed that $A=U \Sigma_A U^*$ for the SVD? I don't think this expression is true for general symmetric $A$, as it implies that $A$ is positive semidefinite. In particular, your expression of $X$ does not recover the solution in here if $A$ is negative definite. – John Aug 11 '22 at 20:57
  • Sorry, I missed your previous comment! Yes, I did overlook that possibility. However, in that case you can just replace the von-Neumann inequality with Fan's trace inequality, which is expressed in terms of eigenvalues instead of singular values. I can clarify in my original answer if you need more context. – VHarisop Aug 11 '22 at 21:19
  • Many thanks for your response. I am not familiar with these trace inequalities, and I was not able to find a suitable version of Fan's trace inequality online. It seems that it is less known than the von-Neumann inequality. May you kindly clarify the details and give a reference of the inequality in your answer? I guess the final answer should replace the singular value by eigenvalues? – John Aug 11 '22 at 21:33
  • In particular, are you suggesting that one can have $\langle X, A\rangle \le \langle \Lambda_X, \Lambda_A\rangle $ for any positive semidefinite $X$ and any symmetric $A$, where $\Lambda_X$, $\Lambda_A$ are diagonal matrices of eigenvalues? – John Aug 11 '22 at 21:52
  • Yes, as long as the diagonals of $\Lambda_X$ and $\Lambda_A$ are ordered in descending order. – VHarisop Aug 11 '22 at 22:22
  • May you give a reference for the claim? – John Aug 11 '22 at 22:26
  • It's Theorem 1.2.1 in "Convex analysis and nonlinear optimization" by Borwein and Lewis. – VHarisop Aug 11 '22 at 22:28
  • Fantastic. Thank you very much! – John Aug 11 '22 at 22:35