14

In Boyd's book on convex optimization he proves convexity of log det X by proving it to be concave along a line i.e. he proves that the Hessian of the function $g(t) = f(Z+tV)$ is negative therefore this function is concave. He assumes that $Z \in S_{++}^n$ and $V \in S^n$ to ensure $Z+tV>0$. I understand that to ensure that the domain of this function is positive he assumes $Z \in S_{++}^n$ but why does he assume that $V \in S^n$. In the videos he says that V is a direction so it can either be positive or negative hence only the condition of symmetric is needed for V. I am not sure what that means.Can someone please explain?

I even tried an example of $Z = \bigl(\begin{smallmatrix} 1 & 1\\ 1 & 2 \end{smallmatrix}\bigr)$ and $V = \bigl(\begin{smallmatrix} 0 & 1\\ 1 & -1 \end{smallmatrix}\bigr)$ where $Z \in S_{++}^n$ and $V \in S^n$. Assume t = 1, we get $Z+tV = \bigl(\begin{smallmatrix} 1 & 2\\ 2 & 1 \end{smallmatrix}\bigr)$ which is not positive definite.

Shirin
  • 465
  • Nitpicking: what you call Hessian here is simply the second derivative of $g:\mathbb{R}\longrightarrow\mathbb{R}$. Although it makes sense, I believe we start talking about Hessians for $g:\mathbb{R}^n\longrightarrow\mathbb{R}$, $n\geq 2$. – Julien Mar 14 '13 at 01:19
  • So $\log\det(X)$ is concave, not convex. Sorry, I just got confused by the opening sentence and want to clarify that for future reference. – a06e Jan 21 '21 at 15:05

1 Answers1

8

Basically Boyd is taking the approach of showing what happens with $\log \det$ in a neighborhood of $t=0$. You're quite correct in that you can make $Z+tV$ into something that's not positive definite, but that isn't really the point here. In fact if you let $t=0.1$, then the sum is still p.d.

What this approach is doing is to show that if you have a line through any $Z$, then you can show that it's concave along that line. That doesn't mean that you have to follow that line out of the domain under scrutiny. It just gives you a way to show convexity locally.

As for $V$ being a "direction": this might be a little confusing until you realize that matrices can be vectors, and that $S^n$ is just a vector space. $Z+tV$ is just the expression of the fact that $Z$ is a vector, and by adding $tV$ (also a vector), you're moving in a line through $Z$. $V$ can be any symmetric matrix, because the important thing is that $Z$ is a member of the domain you're looking at ($S^n_{++}$, the domain of $\log \det$).

So assuming $t$ is small enough, you can answer the question of whether $g$ is concave on a line through $Z$. The fact that $S^n_{++}$ is an open subset of $S^n$ lets us do that.

Josephine Moeller
  • 2,601
  • Thanks a lot.....What does it mean when you say that "The fact that $S_{++}^n$ is an open subset of $S^n$ lets you do that"? It lets me do what? – Shirin Mar 14 '13 at 04:37
  • 1
    It lets you travel a small distance around $Z$ in any direction, because being part of an open subset means that there's a ball around it contained completely inside that subset (this is basic topology). So as long as $t$ is small enough, $Z+tV$ is positive definite and you can reason about whether $g$ is concave. – Josephine Moeller Mar 14 '13 at 04:44
  • "$Z+tV$ is just the expression of the fact that $Z$ is a vector, and by adding $tV$ (also a vector), you're moving in a line through $Z$". Can u plz elaborate on that? @John Moeller – MAS Apr 02 '17 at 08:14
  • @MAS I'm not sure how you want me to elaborate. What part is confusing? – Josephine Moeller Apr 02 '17 at 12:13
  • I don't understand how when we add the tV component, I am moving in a line through Z @John Moeller – MAS Apr 02 '17 at 14:46
  • 1
    @MAS This is one way that you can describe a line in a vector space. This works in $\mathbb{R}^n$ as well. You can describe a line that goes through a point $\mathbf{z}$ in direction $\mathbf{v}$ by $f(t) = \mathbf{z}+t\mathbf{v}$. When $t=0$, $f(t) = \mathbf{z}$. If you take the derivative with respect to $t$, you get $f'(t) = \mathbf{v}$, which is constant. This way you know that when you change t, you're moving on a line. – Josephine Moeller Apr 02 '17 at 14:53