0

This question is a continuation from the question: Show there exists a global minimum

In the accepted answer, the solution continued to:

$g(y):=\frac{1}{2}\sum_{i=1}^n\left(\left(\sqrt{\lambda_i}y_i -c_i\right)^2-c_i^2\right) $

I understand how to go from $g(y)$ convex $\implies g(y) $ has exactly one global minimum.

But how can I show that this function is convex? Do I apply the definition of convexity or should I take the second derivative of this function?

Rico
  • 193
  • The Hessian is a diagonal matrix and it has all positive values on the diagonal which it implies that the function is strictly convex. So, take second derivatives. (I am assuming $\lambda_i>0$.) – irchans Oct 09 '19 at 05:03
  • @irchans $\lambda_i>0$ is true. Moreover, the hessian will be a $n x n$ matrix correct? as we have n different "direction" – Rico Oct 09 '19 at 05:05

2 Answers2

1

You can show the (strict) convexity of $g(y)$ without any Hessian by noting that $g(y)$ is (up to the positive factor $\frac{1}{2}$) the sum of functions of the form $h(t) = (\sqrt{\lambda}t-c)^2- c^2 = \lambda t^2 - 2ct$ with $\lambda > 0$ for each coordinate $y_i, \;(i=1, \ldots , n)$.

So, if you can show the (strict) convexity of $h$ you are basically done.

Since $-2ct$ is linear and $\lambda > 0$, the only thing to show is that $t^2$ is (strictly) convex. At this point one would usually just point out that $h''(t) = 2\lambda > 0$ (the 1-dimensional "Hessian").

But it is also very easy to show (strict) convexity of $t^2$ directly by noting that if $p \in [0,1]$ and $u,v \in \mathbb{R}$, you get by rearranging $$(pu+(1-p)v)^2 \leq pu^2+(1-p)v^2\Leftrightarrow p(1-p)(u-v)^2\geq 0$$

The inequality on the RHS is always true and equality holds if and only if $p=0$ or $p=1$ or $u=v$. This is exactly strict convexity.

trancelocation
  • 32,243
1

Rico, I wrote the long comment below, but it had too many characters, so I am posting it as an "answer" instead.

If you compute the Hessian of $g$, then you will notice that the diagonal entries are always positive and the off diagonal entries are zero. This makes the eigenvalues all positive. If a symmetric matrix has all positive eigenvalues, it is called positive definite. If the Hessian is positive definite everywhere, then the function is strictly convex.

In fact, if the minimum of $g$ is attained at $y^*\in \mathbb R^n$, then $$g(y)\geq \lambda \frac{\lVert y-y^*\rVert^2}2$$ for all $y$ where $\lambda$ is the infimum of the minimum eigenvalues of all of the Hessians. There are several useful bounds that you can place on a smooth real-valued functions using Hessians.

irchans
  • 1,915