Reference for trace/norm inequality

Question

I'm looking for a reference for a matrix-norm inequality that I used in this answer, which has a few equivalent forms. I will use notation that applies to complex vector spaces with a sesquilinear inner product, but of course the same applies over real matrices.

The statement is as follows:

Take $A,B \in \Bbb F^{n \times n}$. Then $$\vert\operatorname{tr}(A^*B)\vert \leq \sigma_1(A)\sum_{i=1}^n \sigma_i(B) = \|A\| \operatorname{tr}|B|$$ where $\sigma_i$ denotes the $i$th singular value, $|B| = (B^*B)^{1/2}$, and $\|\cdot\|$ denotes the spectral norm (induced Euclidean norm).

I did manage to find some references, but they're overkill, and the texts themselves are not readily accessible to the faint of heart (Bhatia's text is dense and Pedersen's is not about matrices in particular).

A suitable reference would be greatly appreciated.

Horn and Johnson's Matrix Analysis would be a decent place to go. And the $B=(B^*B)^{1/2}$ is confusing. — Batman, Apr 19 '17 at 14:37
@Batman typo! Good thinking. I don't have it on hand. If someone can point to the page/theorem from Horn and Johnson, I'd accept that. — Ben Grossmann, Apr 19 '17 at 14:39
I wonder how does this inequality is relate to Cauchy-Schwartz? — Boby, Apr 19 '17 at 14:43
@Boby This inequality is more closely analogous to Hölder's inequality with $p = \infty, q = 1$. The inequality I use in the first part of my answer is exactly the Cauchy-Schwarz inequality. — Ben Grossmann, Apr 19 '17 at 14:46
The proof is not so complicated. Bhatia is not cryptic when you know how to read it — Exodd, Apr 19 '17 at 15:19

score 7 · Accepted Answer · edited Apr 03 '20 at 18:13

A proof in linear algebra. I hope you're familiar with SVD.

Lemma 1 For any matrix $A$, $|tr(A)|\le \sum_i \sigma_i(A)$

Proof: By SVD decomposition, and properties of the trace function $$tr(A) = tr(U\Sigma V) = tr(\Sigma VU) $$ If $Z=VU$ then it is still an unitary matrix, and $$|tr(\Sigma Z)| = |\sum_i \sigma_i(A)z_{ii}|\le \sum_i |\sigma_i(A)z_{ii}|\le \sum_i \sigma_i(A) $$ since $|z_{ii}|\le 1$.

Lemma 2 For any matrix $A,B$, $\sigma_i(A^*B)\le \sigma_i(A)\sigma_1(B)$

Proof: Using Fischer minmax theorem, we know $$ \sigma_i(A^*B) = \max_{\dim V=i}\min_{x\in V,\,\|x\|=1}\|A^*Bx\| $$ but $$ \min_{x\in V,\,\|x\|=1} \|A^*Bx\| \le \max_{x\in V,\,\|x\|=1}\|Bx\| \min_{y\in BV,\,\|y\|=1}\|A^*y\| $$ so $$ \sigma_i(A^*B) \le \max_{\dim V=i}(\max_{x\in V,\,\|x\|=1}\|Bx\| \min_{y\in BV,\,\|y\|=1}\|A^*y\|) $$ $$ \le \max_{\dim V=i}\max_{x\in V,\,\|x\|=1}\|Bx\| \max_{\dim V=i}\min_{y\in BV,\,\|y\|=1}\|A^*y\| \le \sigma_1(B)\sigma_i(A^*) $$

That is a nifty proof! I think it's fair to say that any numerical linear algebra person is fine using SVD. Thanks. — Ben Grossmann, Apr 19 '17 at 18:45
@Omnomnomnom Don't take it for granted. I've met people in numerical l.a. who weren't aware of its existence — Exodd, Apr 19 '17 at 18:55

Hanno · Answer 2 · 2017-04-19T15:34:49.640

This holds true more generally when $\Bbb F^n$ is replaced by a(-ny) Hilbert space $H$.

Let $\mathcal K(H)\,$, $\ell^1(H)$, and $\mathcal L(H)$ denote the compact, the trace class, and all bounded linear operators on $H$, respectively. They form a chain of dual Banach spaces, i.e., each is followed by its (topological) dual.
The duality map is given by the trace-based pairing, which is central to the OP, in detail:
Every continuous linear functional $\varphi$ on $\mathcal K(H)$ has the form $$\varphi(k)=\operatorname{tr}(kt)$$ for some fixed $t\in\ell^1(H)$, and one gets $\big(\mathcal K(H)\big)'=\ell^1(H)$.
Furthermore, every continuous linear functional $\phi$ on $\mathcal\ell^1(H)$ has the form $$\phi(t)=\operatorname{tr}(tx)$$ for some $x\in\mathcal L(H)$, hence $\big(\ell^1(H)\big)'=\mathcal L(H)$.

Note that $\sum_{i=1}^\infty \sigma_i(t) = \operatorname{tr}(\,|t|\,) = \|t\|_{\ell^1}$ which is the trace-class norm; and the norm inequality expresses the continuity in each case.
In the finite-dimensional case one has $\mathcal K(H)=\ell^1(H)=\mathcal L(H)$, but what "remains" is the conceptual & worthwhile view, that the trace implements the duality.

Hope all this is helpful and not an overkill.

Ref's out of my mind are

Barry Simon: Trace ideals and their applications
Gohberg & Krein: Introduction to the Theory of Linear nonselfadjoint operators
Reed & Simon: Methods of Modern mathematical physics, Volume 1

Again, I was looking for a reference that focuses on the finite dimensional case. I appreciate it, though! — Ben Grossmann, Apr 19 '17 at 18:42
@Omnomnomnom Welcome! (Appears as if I feel less comfortable if there are only finitely many dim's around :-) Just spotted in your post the absence of the absolute value bars to enclose the LHS $,\operatorname{tr}(A^*B)$. — Hanno, Apr 20 '17 at 06:25

Reference for trace/norm inequality

2 Answers2

Linked

Related