1

In example 3.4 of Boyd & Vandenberghe's Convex Optimization, we have following inequality $$t-\mathbf{x^TY^{-1}x}\geq 0$$ which can be equivalently written in matrix form as follows $$\left[ {\begin{array}{cc} \mathbf{x^T} & 1 \\ \end{array} } \right]\left[ {\begin{array}{cc} \mathbf{-Y^{-1} ~~~~0\\ ~~~~0~~~~~~~~~~ t} \end{array} } \right]\left[ {\begin{array}{c} \mathbf{x} \\ 1 \\ \end{array} } \right] \geq 0$$ I want to know how this condition transforms into $$\left[ {\begin{array}{cc} \mathbf{Y} & \mathbf{x} \\ \mathbf{x^T} & t \\ \end{array} } \right]\succeq 0.$$Any help in this regard will be much appreciated. Thanks in advance.

Rodrigo de Azevedo
  • 1
Frank Moses
  • 2,718
  • 2
    No offense, but right under the example it says you need the Schur complement explained in §A.5.5. – LinAlg Jun 22 '18 at 11:19
  • @linalg yes but i do not understand how that applies here – Frank Moses Jun 22 '18 at 15:01
  • 2
    $(A,B,C)$ in the appendix is $(Y,x,t)$ in your question. The second bullet point on page 651 answers your question. – LinAlg Jun 22 '18 at 16:05
  • @linalg thank you for your comment. But as you can see I have written my first qaudratic inequality in matrix form in my second inequality. Now if you compare the left side of this inequality with the A.14 on page 650 of the book then you can see that $A=-Y^{-1} $, $B=0$ ( a column vector of zeros actually) and $C=t $. Where is my understanding wrong? – Frank Moses Jun 22 '18 at 16:19
  • 1
    There's really no substance in writing the first inequality in terms of the second. They're basically exactly the same, and there's nothing useful added. It is precisely because the Schur complement allows you to get to the third form that it is useful—because now there are no nonlinearities of any kind. – Michael Grant Jun 22 '18 at 16:42
  • 2
    It is incorrect to call the first or second form a "quadratic" inequality, BTW. $Y$ is not constant. – Michael Grant Jun 22 '18 at 16:42
  • @Michael grant thank you for your comment. I still cannot understand it how Schur complement works here. I also read a document by Jean Gallier but still am unable to connect the dots. There is another post that also proves the convexity of matrix fractional function but that post uses the first definition of convexity and also does not give detailed reasoning for use of the schur complement. If you have a detailed readoning then please share it with me. Thank you. – Frank Moses Jun 22 '18 at 16:54
  • In your first inequality $t$, $Y$ and $x$ are fixed. Let me give an example: for $(Y,x,t) = (2,2,5)$, $5-2\cdot 0.5 \cdot 2 \geq 0$ and indeed [2 2; 2 5] is positive semidefinite. – LinAlg Jun 22 '18 at 18:33
  • Frank, you're the star of The List. – Rodrigo de Azevedo Apr 06 '21 at 18:42
  • @RodrigodeAzevedo thanks for pointing out. Last three years I tried to understand all the exercises of the book but I could only finish first 5 chapters. That is why I posted questions on this site. Perhaps too many and too basic. – Frank Moses Apr 08 '21 at 14:48
  • @FrankMoses The material on B&V is not quite basic. In any case, you catalyzed the creation of Math SE's "solution manual" for B&V. Many people interested in, say, machine learning may need to consult B&V and a "solution manual" might be very helpful. – Rodrigo de Azevedo Apr 08 '21 at 19:40

1 Answers1

6

I guess what you're looking for is a derivation of the Schur complement, or at least a proof that doesn't directly invoke it. So here we go. Let's assume that $Y$ is nonsingular. Then it is the case that $$\begin{bmatrix} Y & x \\ x^T & t \end{bmatrix} = \begin{bmatrix} I & 0 \\ x^T Y^{-1} & 1 \end{bmatrix} \begin{bmatrix} Y & 0 \\ 0 & t - x^TY^{-1} x \end{bmatrix} \begin{bmatrix} I & Y^{-1} x \\ 0 & 1 \end{bmatrix} $$ If you don't see that this is true, just multiply it out.

Now, for any symmetric $Q$ and any nonsingular $P$ of the same size, $Q\succeq 0$ if and only if $PQP^T\succeq 0$. Let's choose $$Q=\begin{bmatrix} Y & x \\ x^T & t \end{bmatrix} \quad P=\begin{bmatrix} I & 0 \\ x^TY^{-1} & 1 \end{bmatrix}^{-1}$$ Therefore, $$\begin{bmatrix} Y & x \\ x^T & t \end{bmatrix} \succeq 0 \quad\Longleftrightarrow\quad \begin{bmatrix} Y & 0 \\ 0 & t - x^T Y^{-1} x \end{bmatrix} \succeq 0 \quad\Longleftrightarrow\quad Y\succ 0, ~ t - x^TY^{-1}x \geq 0.$$ Note that the $(1,1)$ block of the second inequality implies only that $Y\succeq 0$. But combined with the assumption that $Y$ is nonsingular, implied by the presence of the inverse in the $(2,2)$ term, you get that $Y\succ 0$.

Michael Grant
  • 19,450