Proof for strongly convex function is strictly convex

Question

The following is my proof for the title:

We have to show that a strongly convex function $f$ meets the following $$f(\lambda x + (1-\lambda)y)<\lambda f(x) + (1-\lambda)f(y). $$

By the definition of the strongly convex function $f$:
\begin{align*} f(y)\geq f(x) + \langle \nabla f(x),y-x\rangle + \frac{m}{2}\|y-x\|_2^2, \end{align*}where $m>0$. Consider both sides of the definition of strictly convex $f$:

The right-hand side: \begin{align*} &\lambda f(x)+(1-\lambda)f(y) \\ &\geq \lambda f(x) + (1-\lambda) \bigg[f(x) + \langle \nabla f(x),y-x\rangle + \frac{m}{2}\|y-x\|_2^2 \bigg] \\ & = f(x) + (1-\lambda)\langle \nabla f(x),y-x\rangle + \frac{m}{2}(1-\lambda) \|y-x\|_2^2 \end{align*}
The left-hand side: \begin{align*} &f(\lambda x + (1-\lambda)y) \\ &\geq f(x) + \langle \nabla f(x),(\lambda x + (1-\lambda)y)-x\rangle + \frac{m}{2}\|(\lambda x + (1-\lambda)y)-x\|_2^2 \\ &= f(x) + (1-\lambda)\langle \nabla f(x),y-x\rangle + \frac{m}{2}(1-\lambda)^2\|y-x\|_2^2 \end{align*}

Since $\lambda \in (0,1)$, $(1-\lambda)>(1-\lambda)^2$; therefore, we prove that $\lambda f(x)+(1-\lambda)f(y) > f(\lambda x + (1-\lambda)y)$

My question: There is no evidence show that this is a tight lower bound.
EX: $10>9$ and $20>1$, we cannot say that $(20-10) > (1 - 9)$. How to fix this problem?

Why you taking derivatives ? Write down the definition of strong-convexity without using assuming / gradients. Let the modulus of convexity tend to zero; you obtain the definition of strict convexity. — dohmatob, Jan 10 '17 at 00:19
Strongly convex functions need not be differentiable; for instance, $f(x)=|x|_1+|x|_2^2$. Thus the definition you have chosen involving a derivative is not correct. — Michael Grant, Jan 10 '17 at 03:31
@MichaelGrant I guess maybe this definiteion: $f(x)-\frac{m}{2}|x|^2$ being convex is better. — sleeve chen, Jan 10 '17 at 03:40
https://en.wikipedia.org/wiki/Convex_function#Strongly_convex_functions — Michael Grant, Jan 10 '17 at 04:51
From that Wikipedia page I would think this definition is easiest to use in this context: $f(tx+(1-t)y) \le t f(x)+(1-t)f(y) - \frac{1}{2} m t(1-t) |x-y|_2^2$ — Michael Grant, Jan 10 '17 at 15:50
@MichaelGrant How can we show that proof applies for every $x,y$ in $[0,1]$? — Ilan Aizelman WS, May 04 '19 at 15:07

score 6 · Accepted Answer · answered Jul 05 '18 at 12:49

To prove that a strongly convex function is convex, take the definition of strongly convex:

$f(y) \geq f(x) + \nabla f(x)^T(y-x) + \frac{m}{2}||x-y||^2$

clearly:

$f(y) > f(x) + \nabla f(x)^T(y-x)$, as $\frac{m}{2}||x-y||^2$ is positive by definition.

Now take $z = \lambda x + (1-\lambda) y$, by the strong convexity of $f$, we get:

$f(x) > f(z) + \nabla f(z)^T(x-z) = f(\lambda x + (1-\lambda) y) + \nabla f(z)^T(1-\lambda)(x-y)$

and

$f(y) > f(z) + \nabla f(z)^T(y-z) = f(\lambda x + (1-\lambda) y) + \nabla f(z)^T\lambda(y-x)$

Then add those equations with the coefficient $\lambda, (1-\lambda)$ respectively. The gradients terms will cancel and you will be left with:

$\lambda f(x) + (1-\lambda) f(y) > f(\lambda x + (1-\lambda) y)$, which is the definition of strict convexity

How can we show that proof applies for every $x,y$ in $[0,1]$? — Ilan Aizelman WS, May 04 '19 at 15:07
I think we need to add the following two arguments to make the proof more concrete.

$x \neq y$ that is the reason we can write the strict inequality at step 2.

$\lambda \in (0,1)$ to preserve the strict inequalities.

These are also the requirements for the definition of the strictly convex function. — Rajat, Mar 13 '20 at 04:16

Proof for strongly convex function is strictly convex

1 Answers1

Linked