Is the function $\mbox{tr}(XAX')$ convex?

Question

Let matrix $A$ be symmetric and positive semidefinite (PSD). Is the function $X \mapsto \mbox{tr}(XAX')$ convex? I know that, for a general $A$, the above trace function is not convex. But for a PSD $A$, is the function convex?

Answer 1

Look at the table for scalar-by-matrix derivatives. In particular, one has

$$\nabla_X \text{tr}(XAX^T) = X^T(A+A^T).$$

Thus if $A$ is psd, then so is the Hessian $A + A^T = 2A$, from which convexity of $X \mapsto \text{tr}(XAX^T)$ follows.

Answer 2

If $\bf A$ is symmetric and positive semidefinite, then there is a matrix $\bf M$ such that $\bf A = M M^{\top}$. Hence,

$$\mbox{tr} \left( \mathbf X \mathbf A \mathbf X^{\top} \right) = \mbox{tr} \left( \mathbf X \mathbf M \mathbf M^{\top} \mathbf X^{\top} \right) = \| \mathbf X \mathbf M \|_{\text{F}}^2$$

is indeed convex.

Answer 3

Let $\mathrm X \in \mathbb R^{m \times n}$. Hence,

$$\mbox{tr} \left( \mathrm X \mathrm A \mathrm X^{\top} \right) = (\mbox{vec} (\mathrm X))^{\top} \left( \mathrm A \otimes \mathrm I_m \right) \mbox{vec} (\mathrm X)$$

If $\rm A$ is symmetric and positive semidefinite, then the Kronecker product $\mathrm A \otimes \mathrm I_m$ is also symmetric and positive semidefinite and, thus, $\mbox{tr} \left( \mathrm X \mathrm A \mathrm X^{\top} \right)$ is convex.

Answer 4

You can easily see that the function $\varphi: \mathbb R^{n^2}\to \mathbb R$ such that $\varphi(X)=\text{tr}(XAX^T)$ is the trace of the matrix whose entries are the scalar products of a set $ (x_i)_{i=1,\ldots,n}$ of vectors in $\mathbb R^n$. Call their components $(x_i^j)_{j=1,\ldots,n}$ for every $i$.

The elements on the diagonal of your matrix are $$(XAX^T)_{ii}= (X^TAX)_{ii}=x_i^TAx_i=\sum_{j,k=1}^n x_i^ja_{jk}x_i^k$$ so the trace is equal to $$\sum_{i,j,k=1}^n x_i^ja_{jk}x_i^k=\sum_{i=1}^n(x_i,x_i)_A=\sum_{i=1}^n x_i^T A x_i$$

This makes $$\frac{\partial^2}{\partial (x^a_b)^2}\varphi=2a_{aa}$$ $$\frac{\partial^2}{\partial (x^a_b)\partial (x^c_d)}\varphi=a_{ac}+a_{ca}$$ so your Hessian is $$H_\varphi=A+A^T$$ Since $A$ is psd, if $y\neq 0$ $$y^T(A+A^T)y=y^T A y+ (y^TAy)^T=2y^TAy\ge 0$$ and $\varphi$ is convex, because $H_\varphi$ is positive semidefinite.

Answer 5

The function $tr : M_n(R) \rightarrow R$ is linear (so convex) and increasing w.r.t Positive matrix cones. And the function $Q:M_n(R) \rightarrow M_n(R) $ with $Q(X)=X A X^t$ is convex w.r.t positive cones (i.e., each row of $XAX^t$ is convex function), since $A$ is PSD (look at it Hessian of each rows ), Therefore, $F=tr oQ :M_n(R) \rightarrow R $ with $F= tr(XAX^t)$ is convex.