1

In example 3.4 of Boyd & Vandenberghe's Convex Optimization, function $f : \mathbb{R}^n \times \mathbb{S}^n \to \mathbb{R}$, defined as $$f(x, Y) := x^T Y^{-1}x$$ is convex on $\mathrm{dom} f = \mathbb{R}^n \times \mathbb{S}_{++}^n$, where $\mathbb{S}$ denotes the space of symmetric matrices and $\mathbb{S_{++}}$ denotes the space of positive definite matrices. It then shows $f$'s convexity via epigraph.

$$\text{epi} f = \left\{(x, Y, t) \mid Y \succ 0 , x^T Y^{-1} x \leq t \right\} = \left\{(x, Y, t) \mid \begin{pmatrix} Y & x \\ x^T & t \end{pmatrix} \succeq 0, Y \succ 0 \right\}$$

and says "the last condition is a linear matrix inequality in $(x, Y, t)$, and therefore $\text{epi} f$ is convex".

I'm confused about the quoted line.

  1. Why is it an LMI? I would think $x^T a + b^T Y b + c \cdot d < e$ is a LMI, but here there are multiplications between $x$ and $Y$, why it's still a LMI in $(x, Y, t)$?

  2. Why does being an LMI leads to convexity? In example 2.10 of Boyd & Vandenberghe's book, it's the solution set of a linear matrix inequality is convex, but here it's not the solution set.

user21
  • 265

2 Answers2

0

We can think of the condition $$\begin{bmatrix}Y & x \\ x^{\mathsf T} & t\end{bmatrix} \succeq 0$$ as the intersection of infinitely many inequalities in $Y, x, t$: the inequalities $$u^{\mathsf T}\begin{bmatrix}Y & x \\ x^{\mathsf T} & t\end{bmatrix} u \ge 0$$ for all $u \in \mathbb R^{n+1}$.

These are, individually, linear inequalities in $Y,x,t$, and therefore each one defines a closed half-space; the intersection of arbitrarily many half-spaces is convex.

It is for the same reason that the condition $Y \succ 0$ defines a convex set $\mathbb{S}^n_{++}$.

Misha Lavrov
  • 142,276
  • Could you give a more detailed explanation? why the definition of positive semidefinite is LMI about the matirx element $Y,x,t$? because it is linear in inequalities after expanding the matrix vector multiplication? In this meaning the matrix in LMI is $u_i u_j$ and the element i.e. the real number in matrix is the solution set. – eason Feb 22 '22 at 18:31
  • The inequality $u^{\mathsf T}!A u \ge 0$ is linear in the matrix $A$ because the expression $u^{\mathsf T}!A u$ is a linear function of $A$: we have $u^{\mathsf T}(cA)u = c (u^{\mathsf T}!A u )$ for all $A, c$, and we have $u^{\mathsf T}!A u + u^{\mathsf T}B u = u^{\mathsf T}(A+B)u$ for all $A,B$. You can also check this by expanding the multiplication, but you don't have to. – Misha Lavrov Feb 22 '22 at 19:12
  • In fact,the inequality $u^TYu≥0$ is linear in the elements of matrix $Y$, does not it matter between the matrix form and the its element form? convex optimization example 2.10 says $A(x) = x_1A_1 + \cdots + x_nA_n \preccurlyeq B \text{ where } B, A_i \text{ in } S_m$ is called a linear matrix inequality (LMI) in x.The solution set of a linear matrix inequality,$ {x | A(x) \preccurlyeq B}$, is convex. If I collect $x_1 , \cdots ,x_n$ as a matrix $Z$ , then $Z$ is a convex set? what should be $Z$ like? – eason Feb 23 '22 at 06:43
  • I'm confused by what you're asking. What's the difference between your assertion that ${x | A(x) \preccurlyeq B}$ is convex and your later question about whether the set of the $x_1, \dots, x_n$ is convex? – Misha Lavrov Feb 23 '22 at 06:53
  • how could $u^TYu≥0$ be written in the form $A(x) = x_1A_1 + \cdots + x_nA_n \preccurlyeq B \text{ where } B, A_i \text{ in } S_m$ which is the definition of LMI. What is the $x_i$ , $A_i$ corresponding to $u^TYu≥0$?, it seems that $x_i$ is collected as the matrix $Y$, thanks for you patience. – eason Feb 23 '22 at 07:45
  • If that's what you want, then $\begin{bmatrix}Y & x \ x^{\mathsf T} & t\end{bmatrix} \succeq 0$ is already in that form and you don't even have to think about $u$. – Misha Lavrov Feb 23 '22 at 15:44
0

Note that if $Y >0$ then $t-x^TY^{-1}x \ge 0$ iff $t+2 x^Tu + uY^Tu \ge 0$ for all $u$.

In particular $\operatorname{epi} f = \{ (x,Y,t) | u^TYu > 0 \text{ and } t+2 x^Tu + uY^Tu \ge 0 \text{ forall } u \neq 0 \}$.

For each fixed $u$ the function $l_u(x,Y,t) = t+2 x^Tu + uY^Tu$ is linear.

copper.hat
  • 172,524