Is the function $X \mapsto \mbox{trace} \left( X X^T \right)$ convex?

Question

Is function $X \mapsto \mbox{trace} \left( X X^T \right)$ convex?

A linear function of a quadratic should be convex, but I could not prove by definition.

More importantly, What about trace($X^TAXB)$ ? Is it a convex function of $X$? Suppose $A, B$ are constant matrices. — user25004, Aug 29 '12 at 01:44

score 4 · Accepted Answer · edited Apr 22 '13 at 18:51

4

This might be closer to what you mean by "by definition". First observe that $\operatorname{Tr}(XX^T)=\sum_{i,j} X_{ij}^2$. Then for any $\lambda \in [0,1]$ and $Y$ of the same size as $X$, $$\operatorname{Tr}[ (\lambda X + (1-\lambda)Y)(\lambda X + (1-\lambda)Y)^T]$$ $$= \sum_{i,j} (\lambda X_{ij} + (1-\lambda)Y_{i,j})^2$$ $$\leq \lambda \sum_{i,j} X_{ij}^2 + (1-\lambda) \sum_{i,j} Y_{i,j}^2$$ $$=\lambda \operatorname{Tr}(XX^T) + (1-\lambda)\operatorname{Tr}(YY^T).$$

edited Apr 22 '13 at 18:51

jinawee

2,585

answered Aug 29 '12 at 02:37

shoda

1,468

How did you arrive at step 3 from step 2? Sorry but I tried a lot but couldn't figure out. – stillanoob Aug 25 '16 at 16:38
Jensen Inequality – LinTIna May 11 '21 at 14:55
@LinTIna Wouldn't using that inequality need the assumption of convexity? – ali Dec 17 '22 at 01:37
(Edit: I see, trace is convex because it's linear) – ali Dec 17 '22 at 01:44

score 2 · Answer 2 · answered Aug 29 '12 at 01:24

2

The function $X\mapsto (trace(X^TX))^{1/2}$ is known as a Frobenius norm , it is indeed a norm. And as every norm it is a convex function. Thus $X\mapsto trace(X^TX)$ is also a convex function. Since matrices $A$ and $A^T$ has the same eigenvalues your function is just a second power of a Frobenius norm and thus it is a convex function.

answered Aug 29 '12 at 01:24

Godot

2,082

What about trace($X^TAXB)$ ? – user25004 Aug 29 '12 at 01:49
@user25004 Trace$(X^\top A X B)$ is actually the quadratic assignment problem. No closed solution. And if you take the gradient of it wrt to X you will get $AXB + A^\top X B^\top$. – Kimmi May 03 '21 at 17:51

score 1 · Answer 3 · edited Apr 25 '22 at 18:18

1

$\text{trace}\left(X Y^T \right)$ is an inner product over the space of matrices. $\text{trace}(XX^T) = \|X\|_F^2$, where $\| \cdot \|_F$ denotes the Frobenius norm of the matrix $X$. The Frobenius norm squared is nothing but the sum of squares of entries of the matrix and is, hence, strictly convex.

edited Apr 25 '22 at 18:18

Rodrigo de Azevedo

1

answered Aug 29 '12 at 01:25

Kartik Audhkhasi

1,496
9
12

What about trace$(X^TAXB)$? Does it require $A$ and $B$ to be psd to be convex? – user25004 Aug 29 '12 at 02:04
I think we do need that constraint. Take $B = I$ for a simple case. We have $\text{trace}(X^TAX)$, which is a quadratic term in the inner product space of matrices. This is analogous to $\mathbf{x}^T A \mathbf{x}$ in the inner product space of finite-dimensional real vectors. We know that $A$ must be positive semi-definite for this quadratic term to be convex. I believe that a similar argument can be made for the trace inner product. – Kartik Audhkhasi Aug 29 '12 at 05:54
@KartikAudhkhasi, It is strange. The term is Convex yet when one derives the PCS as in here: https://www.quora.com/How-does-PCA-maximize-projected-variance-through-the-covariance-matrix the one tries to maximize the objective. – Royi Dec 24 '16 at 09:32

Is the function $X \mapsto \mbox{trace} \left( X X^T \right)$ convex?

3 Answers3

Linked