Proving that $(x,Q) \mapsto \langle x,Q^{-1}x\rangle$ is convex

Question

I'm studying

$$ F( (x,Q) ) = \langle x,Q^{-1}x\rangle $$

considered over $\mathbb{R}^n\times\mathbb{R}^{n\times n}_+$, where $\mathbb{R}^{n\times n}_+$ is the set of all positive definite $n\times n$ matrices. Here, $\langle x,y\rangle$ stands for the dot product.

I'm trying to prove that $F$ will be convex when considered over this set. That is, it's obvous that $F$ is convex in $x$ with fixed $Q$ (and in $Q$ with fixed $x$), but the trick is to prove that it will be convex with respect to the pair $(x,Q)$. It can be straightforwardly checked for $n=1$ (considering the Hessian), but how can it be done for an arbitrary $n$?

Answer 1

This seems to be a little trickier than I thought first!

I'll be using the following lemma:

A function $f$ is convex if and only if its epigraph $$\text{epi}(f) = \{(p, t) | \; p \; \text{in domain of} \;f \; \text{and} \; f(p) \leq t\}$$

is a convex set.

Let $f(x, Q) = x^T Q^{-1}x$ with domain $\mathbb{R}^n \times S^n_{++}$ where $S^n_{++}$ is the set of $n$ by $n$ symmetric positive definite matrices over $\mathbb{R}.$ Now, its epigraph is

$$\{((x, Q), t) | \; Q \succ 0, \; x^TQ^{-1}x \leq t\}\stackrel{\text{Schur Complement}}{=}\{((x, Q), t) | \; Q \succ 0, \; \begin{pmatrix} Q & x \\ x^T &t \end{pmatrix} \succeq 0\}.$$ Last condition is an LMI in $(x,Q, t),$ so is convex.

Answer 2

It can be proved using derivatives and elementary calculus rules for matrices.

Let $x,y$ be two vectors, $P$ and $Q$ two positive definite matrices. The segment between $(x,Q)$ and $(y,P)$ is parametrized as $(x(t),Q(t))$ with $x(t)=x+tv$ and $Q(t)=Q+tV$ for $v=y-x$ and $V=P-Q$. The desired convexity in follows if we can prove convexity in every segment of the form $\{x(t),Q(t), t\in[0,1]\}$. Denote by $\dot f(t)$ the derivative of any function of $t$, and $\ddot f(t)$ for the second derivative. Here, the functions $x(t), Q(t)$ and $M(t)=Q(t)^{-1}$ are twice continuous differentiable function of $t$, with $\dot x(t)=v$, $\dot Q(t)=V$ and $\dot M(t)= - M(t)\dot Q(t) M(t) = -M(t) V M(t)$.

Now $F(t)=x(t)^T Q(t)^{-1} x(t)=x(t)^T M(t)x(t)$ has derivative (by the product rule) \begin{align*} \dot F(t) &= 2\dot x(t) M(t) x(t) - x(t)^TM(t)\dot Q(t) M(t)x(t) \\&= 2v^TM(t) x(t) - x(t)^TM(t)VM(t)x(t) \end{align*} Now $F(t)$ has second derivative: \begin{align*} \ddot F(t) &= 2 v^TM(t)v \\& \quad - 2 v^T M(t) V M(t)x(t) \\&\quad + 2 x(t)^T M(t) V M(t) V M(t) x(t). \end{align*} The first term on the first line can be written $2\|\sqrt{M(t)}v\|_2^2$. The last term on the third line can be written $\|\sqrt{M(t)}V M(t)x(t)\|_2^2$. The only non-positive term is the second line. Using Cauchy-Schwarz and $2ab\le a^2+b^2$, the second line is bounded from below by \begin{align*} &-2 \|\sqrt{M(t)}v\|_2 \|\sqrt{M(t)}VM(t)x(t)\|_2 \\&\ge - \|\sqrt{M(t)}v\|_2^2 - \|\sqrt{M(t)}VM(t)x(t)\|_2^2 \end{align*} so $\ddot F(t)\ge 0$ and the proof is complete.