Smallest Lipschitz constant of gradient of a function

Question

I have to show that $2\Vert{A}\Vert$ is the smallest Lipschitz constant of $\nabla f$, where $f(x)=x^{T}Ax+2b^{T}x+c$.

$\forall x,y\in \mathbb{R}^n, \Vert\nabla f(x)-\nabla f(y)\Vert = \Vert2(Ax+b)-2(Ay+b)\Vert = \Vert2A(x-y)\Vert \le \Vert2A\Vert \Vert x-y\Vert$

but I don't know how to show that it's the smallest.

mathcounterexamples.net · Answer 1 · 2021-05-07T16:42:01.013

2

With the inequality you mentioned in the question, you proved that $2\Vert A\Vert$ is larger than the smallest Lipschitz constant.

Now I imagine that the matrix norm you’re using is the sup norm. That is

$$\Vert A \Vert =\sup\limits_{\Vert x \Vert =1} \Vert Ax\Vert$$ You can easily conclude using this definition as for any $c \lt \Vert A\Vert $ it exists $z$ with $\Vert z \Vert =1$ and $$\Vert Az\Vert \ge c.$$ Then use the linearity of $A$.

edited May 07 '21 at 16:42

answered May 07 '21 at 15:27

mathcounterexamples.net

70,018

I don't get it. How does it show that it's the smallest? – Mina May 07 '21 at 15:53
I updated my answer with additional elements. – mathcounterexamples.net May 07 '21 at 15:58

score 2 · Accepted Answer · answered May 07 '21 at 16:18

2

You can take $y = 0$ and then maximize over all $x$ with $||x|| = 1$ to get that $2||A||$ is the best constant.

answered May 07 '21 at 16:18

Zarrax

44,950

How does it prove that it's the best? I can't get it! – Mina May 07 '21 at 16:36
1

For that value of x, and $y = 0$, you'll have $||f(x) - f(y)|| = 2||A|| || x - y||$. – Zarrax May 07 '21 at 16:39
Don't we have $\Vert Ax \Vert \le \Vert A \Vert \Vert x \Vert$?@Zarrax – Mina May 08 '21 at 11:06

Smallest Lipschitz constant of gradient of a function

2 Answers2