Is Frobenius-norm projection Lipschitz continuous under operator norm?

Question

Definition

Let $d \in \mathbb{N}$. Let $A \in \mathbb{R}^{d\times d}$ be a PSD matrix. Define the following operator: $T:\mathbb{R}_{+} \times \mathbb{R}^{d\times d} \to \mathbb{R}^{d\times d} $: Let $\{u_1,\dots,u_d\} \subset \mathbb{R}^d$ and $\{\lambda_1,\dots,\lambda_d\}\subset \mathbb{R}$ be the eigenvectors and eigenvalues of $A$. Then, \begin{equation} T_{\lambda_0}(A) = \sum_{i=1}^{d} \max\{\lambda_0,\lambda_i\} u_i u_i^T. \end{equation}

It is not difficult to see that:

\begin{equation} T_{\lambda_0}(A) = \arg\min_{\hat{A}\in \mathcal{S}_{+}^{d} : ~\forall x \in \mathbb{R}^d ~ x^T \hat{A} x \ge \lambda_0 \|x\|_2^2}\|\hat{A}-A\|_F. \end{equation} where $\|\cdot\|_F$ is Frobenius norm.

In other words, $T_{\lambda_0}(A)$ is the Frobenius-norm projection of $A$ onto the the set of PSD matrices with minimum eigenvalue of $\lambda_0$.

Conjecture

For every $\lambda_0>0$, $A \in \mathbb{R}^{d \times d}$, and $B \in \mathbb{R}^{d \times d}$ such that $A$ and $B$ are PSD, we have \begin{equation} \| T_{\lambda_0}(A+B) - T_{\lambda_0}(A) \| \leq \| B \|. \end{equation} where $\|\cdot\|$ denotes the operator norm $\ell_2/ \ell_2$. (Notice that we don't use Frobenius norm here.)

I numerically verified this conjecture over the random draws of $A$ and $B$. However, I don't have any proof. Please let me know if you have any hints!

Answer 1

This is not an answer. This proves non-expansiveness with respect to the Frobenius norm, which is not the norm from the question. I'm leaving this up so that (a) people can take ideas from it (which, I suspect, won't be too helpful), and more importantly, (b) so that people don't make the same mistake.

The Frobenius norm on the space of $n \times n$ real square matrices is derived from the inner product $$\langle A, B \rangle = \operatorname{Tr}(A^\top B).$$ We know that projections onto closed, convex sets in Hilbert spaces are (firmly) non-expansive, so we just need to verify that the set of PSD matrices with minimum eigenvalue $\lambda_0$ is a closed and convex set.

Consider the map $\phi_{\max}$ from the symmetric $n \times n$ matrices to $\Bbb{R}$ defined by $$\phi_{\max}(A) = \sup_{\substack{x \in \Bbb{R}^n \\ \|x\| = 1}} x^\top A x.$$ The value of $\phi_{\max}(A)$ is the greatest eigenvalue of $A$. Written as above, it is also the supremum of (convex) linear functionals, which makes it convex.

We can also see that $\phi_{\max}$ has full domain, since $|x^\top Ax| \le \|x\|_2 \|Ax\|_2$ by Cauchy Schwarz (where $\| \cdot \|_2$ is the Euclidean norm), and $\|x\|_2\|Ax\|_2 \le \|x\|_2\|A\|_2\|x\|_2$ (where $\|A\|_2$ is the operator norm, derived from the Euclidean norm, on $A$). In other words, this supremum is finite, so $\phi_{\max}$ has full domain. This implies $\phi_{\max}$ is continuous everywhere.

We can also define $\phi_{\min}(A) = -\phi_{\max}(-A)$, which is therefore concave and continuous everywhere.

The set we're interested in is precisely $\phi_{\min}^{-1}[\lambda_0, \infty) = -\phi_{\max}^{-1}(-\infty, -\lambda_0]$. The set $\phi_{\max}^{-1}(-\infty, -\lambda_0]$ is the closed lower level set of a continuous convex function, which makes the set closed and convex. The set we want is a reflection of this set about the origin, which still makes the set closed and convex (as it is the image of the set under the continuous, linear map $x \mapsto -x$).

So, yes, the set of symmetric (and necessarily PSD) matrices with minimum eigenvalue $\lambda_0 \ge 0$ is a closed, convex set. Therefore, the projection onto said set is non-expansive.

Answer 2

Your conjecture is false. Here is a random counterexample: $$ A=\pmatrix{15&2&5\\ 2&4&6\\ 5&6&12}, \ B=\pmatrix{7\\ &6\\ &&1},\ \lambda_0=8. $$ The LHS of your inequality is $7.0141$ (four decimal places) but the RHS is $\|B\|=7$.

However, the map $T_{\lambda_0}$ is indeed Lipschitz-continuous, although the modulus of continuity is not equal to $1$ but a constant that depends on the size of the matrices. Denote by $|X|$ the matrix absolute value $(X^\ast X)^{1/2}$ of $X$, then $T_{\lambda_0}(A)=\frac12\left(|A-\lambda_0I|+A+\lambda_0I\right)$ and $$ \|T_{\lambda_0}(A+B)-T_{\lambda_0}(A)\| =\frac12\|\,|A+B-\lambda_0I|-|A-\lambda_0I|+B\,\|. $$ By Araki and Yamagami, An Inequality for Hilbert-Schmidt Norm, Commun. Math. Phys. 81: 89-96 (1981), the following inequality holds for any two matrices $X,Y\in M_d(\mathbb C)$: $$ \|\,|X|-|Y|\,\|_F\le\sqrt{2}\|X-Y\|_F. $$ (The authors mentioned that the Lipschitz constant $\sqrt{2}$ can be improved to $1$ when $X$ and $Y$ are self-adjoint, but the proof was given in a paper that I have no access to.) It follows that $$ \|\,|X|-|Y|\,\|\le\|\,|X|-|Y|\,\|_F\le\sqrt{2}\|X-Y\|_F\le\sqrt{2d}\,\|X-Y\|. $$ So, in your case, $$ \begin{align} &\|T_{\lambda_0}(A+B)-T_{\lambda_0}(A)\|\\ &=\frac12\|\,|A+B-\lambda_0I|-|A-\lambda_0I|+B\,\|\\ &\le\frac12\|\,|A+B-\lambda_0I|-|A-\lambda_0I|\,\|+\frac12\|B\|\tag{$\#$}\\ &\le\frac12(\sqrt{2d}+1)\|B\|.\\ \end{align} $$ For more information about the Lipschitz constant for $X\mapsto|X|$, see also R. Bhatia, Modulus of continuity of the matrix absolute value, Indian Journal of Pure and Applied Mathematics volume 41, pages 99–111 (2010).

It is also possible to remove the dependence on $d$ if you drop Lipschitz continuity for continuity. According to Tosio Kato, Continuity of the Map $S\to|S|$ for Linear Operators, Proc. Japan Acad., 49: 157-160 (1973), the map $S\mapsto|S|$ is almost Lipschitz-continuous on the space of all real symmetric matrices, in the sense that $$ \|\,|S|-|T|\,\|\le\frac{2}{\pi}\|S-T\|\left(2+\log\frac{\|S\|+\|T\|}{\|S-T\|}\right). $$ In your case, if we continue from $(\#)$, this translates to $$ \begin{aligned} &\|T_{\lambda_0}(A+B)-T_{\lambda_0}(A)\|\\ &\le\|B\|\left(\frac{2}{\pi}+\frac{1}{2}+\frac{1}{\pi}\log\frac{\|A+B-\lambda_0I\|+\|A-\lambda_0I\|}{\|B\|}\right). \end{aligned} $$