Prove the Lemma for Poincaré's inequalities

Question

There's a problem in my textbook:

Prove the following Lemma (Poincare's inequalities). For any $p\times p$ orthogonal projection matrix $P$,

$$\lambda_{p-q+1}(A)+...+\lambda(A)\leq Tr(AP) \leq \lambda_1(A)+...+\lambda_q(A),$$

for all $p\times p$ symmetric matrices $A$, where $q=Tr(P)$ and $\lambda_i(A)$ is the $i$-th largest eigenvalue of $A$.

Here is my approach:

By the Spectral Decomposition Theorem:

$$A=\lambda_1e_1e_1^T+...+\lambda_pe_pe_p^T=E\Lambda E^T$$

With $e_1,...,e_p$ orthonormal. By the following proposition in my textbook:

Let $P$ be a $p\times p$ orthogonal projector. Then 1) The eigenvalues of $P$ are either $0$ or $1$, 2) $q=Tr(P)\in\{0,1,...,p\}$, 3) $P = BB^T$ for some $p\times q$ matrix $B$ satisfying $B^TB=I_q$

We may write $P=BB^T$. Then $$Tr(AP)=\sum^p_{j=1}\lambda_i Tr(e_je_j^TBB^T)$$

Three facts:

1. $\sum^p_{j=1}Tr(e_je_j^TBB^t)= Tr((\sum^p_{j=1}e_je_j^T)BB^T)=Tr(EE^TBB^T)=Tr(BB^T)=Tr(B^TB)=Tr(I_q)=q$

2. $Tr(e_je_j^TBB^T)=Tr(e_j^TBB^Te_j)=e_j^TBB^Te_j=||B^Te_j||^2\geq 0$

3. $Tr(e_je_j^TBB^T)=e_j^TBB^Te_j=e_j^TPe_j=e_j^TP^TPe_j=||Pe_j||^2_2\leq 1$

So, if we write $w_j=Tr(e_je_j^TBB^T)$ we have $w_j\in[0,1]$, $\sum^p_{j=1}w_j=q$ and $Tr(AP)=\sum^p_{j=1}\lambda_jw_j$

So for the upper bound:

$$\sum^p_{j=1}\lambda_j w_j=\lambda_1+...+\lambda_q+\sum^q_{j=1}(w_j-1)\lambda_j+sum^p_{j=q+1}w_j\lambda_j leq \lambda_1+...\lambda_q+\sum^q_{j=1}(w_j-1)\lambda_q$$

And since $w_j\leq 1$ and $\lambda_1\geq \lambda_2\geq ...\geq \lambda_q$:

$$\leq \lambda_1+...\lambda_q+sum^q_{j=1}(w_j-1)\lambda_q+\sum^p_{j=q+1}w_j\lambda_q$$

And since $w_j\geq 0$ and $\lambda_q\geq \lambda_{q+1}\geq ...\geq \lambda_p$

$$=\lambda_1+...\lambda_q+\lambda_q(sum^p_{j=1}w_j-q)$$

And by fact (1)

$$=\lambda_1+...+\lambda_q$$

Lower Bound:

$$Tr(AP)=\sum^p_{j=1}\lambda_jw_j=\lambda_{p-q+1}+...+\lambda_p+\sum^p_{j=q+1}(w_j-1)\lambda_j+\sum^q_{j=1}\lambda_j\geq \lambda_{p-q+1}+...+\lambda_p+\sum^p_{j=q+1}(w_j-1)\lambda_q+\sum^q_{j=1}w_j\lambda_j\geq \lambda_{p-q+1}+...+\lambda_p+\sum^p_{j=q+1}(w_j-1)\lambda_q+\sum^q_{j=1}w_j\lambda_q=\lambda_{p-q+1}+...+\lambda_p$$

Answer 1

Note that for any such $P$, there exist a matrix $U$ with $q$ orthonormal columns such that $$ P = UU^T, $$ so that $$ \operatorname{trace}(AP) = \operatorname{trace}(AUU^T) = \operatorname{trace}(U^TAU). $$ From there, the inequality you want can be obtained using any of the approaches outlined here or here.