Details on equivalent strong convexity

Question

Show $f(x)$ ($C^1$ smooth) is strongly convex iff $g(x)=f(x)-\frac{\alpha}{2}\|x\|^2$ is convex.

I'll use the definition of convexity, $f(y)\ge f(x)+\langle\nabla f(x), y-x\rangle$.

$f(y)-\frac{\alpha}{2}\|y\|^2\ge f(x)-\frac{\alpha}{2}\|x\|^2+\langle\nabla(f(x)-\frac{\alpha}{2}\|x\|^2),y-x\rangle$

I can distribute the $\nabla$ and break apart the inner product.

$f(y)-\frac{\alpha}{2}\|y\|^2\ge f(x)-\frac{\alpha}{2}\|x\|^2+\langle\nabla f(x),y-x\rangle-\langle\nabla\frac{\alpha}{2}\|x\|^2,y-x\rangle$

I can move the $\frac{\alpha}{2}\|y\|^2$ to the other side.

$f(y)\ge f(x)+\langle\nabla f(x),y-x\rangle+\frac{\alpha}{2}(\|y\|^2-\|x\|^2)-\langle\nabla\frac{\alpha}{2}\|x\|^2,y-x\rangle$

I'm not sure what to do with the last two terms to move toward the definition of strong convexity, namely $f(y)\ge f(x)+\langle\nabla f(x), y-x\rangle+\frac{\alpha}{2}\|y-x\|^2$.

Answer 1

I think the "trick" you're missing is the identity $$ \|x-y\|_2^2=\|x\|_2^2+\|y\|_2^2-2\langle{y,x\rangle}. $$

($\Rightarrow$) Suppose $f$ is a strongly convex function with constant $\alpha>0$. Then \begin{align*} g(x)-g(y)&=f(x)-f(y)-\tfrac{1}{2}\alpha\big(\|x\|^2-\|y\|^2\big)\\ &\geq\langle\nabla{f}(y),x-y\rangle+\tfrac{1}{2}\alpha\|x-y\|^2-\tfrac{1}{2}\alpha\big(\|x\|^2-\|y\|^2\big) &&\text{b/c $f$ strongly convex}\\ &=\langle\nabla{f}(y),x-y\rangle+\tfrac{1}{2}\alpha\|x\|^2+\tfrac{1}{2}\alpha\|y\|^2-\alpha\langle{x,y}\rangle-\tfrac{1}{2}\alpha\big(\|x\|^2-\|y\|^2\big) &&\text{ suggested identity}\\ &=\langle\nabla{f}(y),x-y\rangle-\alpha\langle{y,x}\rangle+\alpha\|y\|^2\\ &=\langle\nabla{f}(y)-\alpha{y},x-y\rangle\\ &=\langle\nabla{g}(y),x-y\rangle \end{align*} and thus $g$ is convex.

($\Leftarrow$) Now suppose there exists $\alpha>$ such that $g$ is convex. Then \begin{align*} f(x)-f(y)&=g(x)-g(y)+\tfrac{1}{2}\alpha\big(\|x\|^2-\|y\|^2\big)\\ &\geq\langle\nabla{g}(y),x-y\rangle+\tfrac{1}{2}\alpha\big(\|x\|^2-\|y\|^2\big)&&\text{b/c/ $g$ convex} \end{align*} Using the suggested trick, you can show that this last line is equal to $$ \langle\nabla{f}(y),x-y\rangle+\tfrac{1}{2}\alpha\|x-y\|^2, $$ and thus $f$ is strongly convex. I'll leave the details to you.