4

Given $\mathrm A \in \mathbb R^{m \times n}$,

$$\begin{array}{ll} \text{maximize} & \langle \mathrm A , \mathrm X \rangle\\ \text{subject to} & \| \mathrm X \|_2 \leq 1\end{array}$$

Since $\| \mathrm X \|_2 \leq 1$ is equivalent to $\sigma_{\max} (\mathrm X ) \leq 1$, which is equivalent to $\lambda_{\max} (\mathrm X^\top \mathrm X) \leq 1$, we have

$$1 - \lambda_{\max} (\mathrm X^\top \mathrm X) = \lambda_{\min} (\mathrm I_n - \mathrm X^\top \mathrm X)\geq 0$$

and, thus,

$$\mathrm I_n - \mathrm X^\top \mathrm X \succeq \mathrm O_n$$

Using the Schur complement test for positive semidefiniteness, we obtain the following linear matrix inequality (LMI)

$$\begin{bmatrix} \mathrm I_m & \mathrm X\\ \mathrm X^\top & \mathrm I_n\end{bmatrix} \succeq \mathrm O_{m+n}$$

Hence, we have the following semidefinite program (SDP)

$$\begin{array}{ll} \text{maximize} & \langle \mathrm A , \mathrm X \rangle\\ \text{subject to} & \begin{bmatrix} \mathrm I_m & \mathrm X\\ \mathrm X^\top & \mathrm I_n\end{bmatrix} \succeq \mathrm O_{m+n}\end{array}$$

Is this correct? Any feedback would be highly appreciated. Thank you.

Community
  • 1
Rodrigo de Azevedo
  • 1

2 Answers2

7

This can be computed in closed-form via SVD. Indeed, given a matrix norm $\|.\|$ on $\mathbb C^{n,m}$, there is always a corresponding dual norm (i.e on the dual space $\mathbb C^{m,n}$) given by

$$\|A\|^{D} := \max_{\|X\| \le 1}\langle A, X\rangle_{\text{F}}, $$ where $\langle A, X\rangle_{\text{F}} := \text{trace}(AX^T)$ is the Frobenius / Hilbert-Schmidt inner product. In particular, it's not hard to show that $\|A\|_2^D = \|A\|_* := \sum_{k}\sigma_k(A)$, the nuclear norm of $A$.

dohmatob
  • 9,535
  • 2
    Nice answer. Here's a question that asks to show the dual of the spectral norm is the nuclear norm: http://math.stackexchange.com/questions/1158798/show-that-the-dual-norm-of-spectral-norm-is-nuclear-norm – littleO Oct 12 '16 at 08:25
  • Nice pointer. This bails me out of the debt of having to write that part of the solution. Also, the algebra therein contained is really instructive. – dohmatob Oct 12 '16 at 08:50
1

Let $A=\sum_i \lambda_i u_i v_i^T$ be the SVD of $A$. The nuclear norm of $A$ is by definition the sum of the singular values, $\sum_i \lambda_i$.

  • The fact that the maximum of your optimization problem is at most the nuclear norm of $A$ follows from $$\langle A, X\rangle = trace(A^TX) =\sum_i \lambda_i v_i^T X u_i \le \sum_i \lambda_i $$ by the triangle inequality because $X$ has operator norm at most one, so $|v_i^T X u_i|\le 1$.

  • The fact that $\sum_i \lambda_i$ can be achieved for some matrix $X$ with operator norm at most one follows by taking $X= \sum_i u_iv_i^T$.

jlewk
  • 1,867
  • Aren't $u$ and $v$ swapped in your first bullet point? I.e. $\mathrm{trace}(A^TX) = \sum_i \lambda_i \mathrm{trace}(v_i u_i^T X) = \sum_i \lambda_i u_i^T X , v_i$. – Luca Citi Dec 20 '21 at 01:31