2

Suppose $\mathbf{Q} \in \mathbb{R}^{m \times k}$ where $k < m$ and $\mathbf{Q}^\top\mathbf{Q} = \mathbf{I}$.

How can I find $\mathbf{Q}$ such that $$ \operatorname{trace}(\mathbf{Q}^\top \mathbf{AQ}) $$ is minimized where $\mathbf{A}$ is positive semi-definite?

I tried the following simplification: $$ \mathbf{Q}^\top \mathbf{AQ} = \mathbf{Q}^\top \mathbf{UD}\mathbf{U}^\top \mathbf{Q} = \mathbf{Q}^{*\top} \mathbf{DQ}^* $$ where $\mathbf{Q}^* = \mathbf{U}^\top\mathbf{Q}$, $\mathbf{A} = \mathbf{UDU}^\top$ and $\mathbf{Q}^{*\top}\mathbf{Q}^* = \mathbf{I}$.

If I minimize this quantity, does that mean the columns of $\mathbf{Q}^*$ is given by the eigenvectors corresponding to the smallest eigenvalues of $\mathbf{A}$?

Gary
  • 31,845
shani
  • 779
  • 1
  • 5
  • 12

2 Answers2

1

Using your notation but replacing $Q^*$ by $V$, the problem is $$\min_{V\in \mathbb{R}^{m \times k}\\ V^{\mathrm{T}}V = I}\ trace(V^{\mathrm{T}}DV).$$

Suppose $V = (v_{ij})_{m \times k}$ and $D = diag\{\lambda_1,\cdots,\lambda_m\}$ where $0 \leq \lambda_1 \leq \cdots \leq \lambda_m$, then it is easy to coumpute that $$(V^{\mathrm{T}}DV)_{jj} = \sum_{i=1}^m \lambda_i v_{ij}^2.$$ So $$trace(V^{\mathrm{T}}DV) = \sum_{j=1}^k \sum_{i=1}^m \lambda_i v_{ij}^2 \geq \sum_{j=1}^k\lambda_j$$ And when taking $v_{jj} = 1,\ 1 \leq j \leq k$ and $v_{ij} = 0,\ i \neq j$ we get the equality.

Jingyu Liu
  • 196
1

Your analysis is basically correct. But more precisely, the answer should be like that the column vectors of the matrix $Q$ should be equal to the k unit eigenvectors corresponding to the k smallest eigenvalues of the matrix $A$.

Ken T
  • 337