2

For a symmetric positive semi-definite matrix $A = Q\Lambda Q^T \in \mathbb{R}^{n\times n}$ and $\lambda_1 \geq \lambda_2 ....\geq \lambda_n$ find $$ \max \ (x^TAx+y^TAy) \ \ \text{s.t} \ x^Tx = y^Ty = 1, x^Ty = 0 $$ and corresponding $x, y$.

My approach was to define $a = Q^Tx$ and $ b=Q^Ty$. $$ \max \ (x^TAx+y^TAy) =\max \ (a^T\Lambda a+b^T\Lambda b)\\ =\max (\sum_{i=1}^{n}\lambda_ia_i^2+\sum_{i=1}^{n}\lambda_ib_i^2) $$ Clearly, $\sum_{i=1}^{n}\lambda_ia_i^2$ is maximized when $a=e_1$. Since $a^Tb=x^Ty=0$ the same maximization $b=e_1$ is impossible as $b^Ta=b^Te_1 = e_1^Te_1\neq 0$. The next possible maximum for $\sum_{i=1}^{n}\lambda_ib_i^2$ is $b=e_2$.

Therefore, $$ x = Qe_1 = q_1 \\ y = Qe_2 = q_2 \\ \max \ (x^TAx+y^TAy) = \lambda_1+\lambda_2 $$

This is my intuition for what should be the solution, but I can't think of a way to definitely show that $x=q_1$, $y=q_2$ maximizes $x^TAx+y^TAy$. How can I show that $x=q_1$, $y=q_2$ is the optimal solution?

Ernie060
  • 6,073
James
  • 21
  • Haven't you shown it already? $Q$ is invertible so the change of variables you did from $x,y$ to $a,b$ is reversible; you solved the problem by finding the optimal $a$ and $b$, and just transformed back to $x$ and $y$. – angryavian Apr 24 '20 at 00:25
  • I don't think so, because I first solved $max(x^TAx)$ such that $x^Tx = 1$, then $max(y^TAy)$ such that $y^Tx=0$ and $y^Ty=1$. I guess what I'm asking is if $max(f(x)+f(y)) | condition (1,2)$ = $(max(f(x)) | cond(1))+(max(f(y)) | cond(1,2 ))$ – James Apr 24 '20 at 00:31
  • In general that is not true, but your solution to the auxiliary problem (minimizing $a^\top \Lambda a + b^\top \Lambda b$) is correct – angryavian Apr 24 '20 at 00:36
  • Hmm.. I'm sure that you are right, but I'm also convinced I didn't show that $x=q_1, y=q_2$ maximizes $x^TAx + y^TAy$ because my underlying assumption is that I choose $x=q_1$ to maximize $x^TAx$ without taking $y$ into consideration – James Apr 24 '20 at 00:43
  • It seems $a=(1/\sqrt{2}, 1/\sqrt{2})$ and $b=(1/\sqrt{2}, -1/\sqrt{2})$ is also another solution – angryavian Apr 24 '20 at 00:58
  • Oh I figured it out using Courant Minimax principle! According to this principle, I don't think your alternative solution is correct. https://en.wikipedia.org/wiki/Courant_minimax_principle – James Apr 24 '20 at 01:10
  • If you plug in $a=\frac{1}{\sqrt{2}}(e_1 + e_2)$ and $b=\frac{1}{\sqrt{2}}(e_1 - e_2)$ into $a^\top \Lambda a + b^\top \Lambda b$ you get $\lambda_1 + \lambda_2$ as well. – angryavian Apr 24 '20 at 01:12
  • Sorry, you are correct. Courant Minimax Principle doesn't show uniqueness – James Apr 24 '20 at 01:17
  • That is known as Ky-Fan Maximum principle. – Toni Mhax Apr 24 '20 at 09:16

0 Answers0