0

A differentiable function $f: \mathbb{R}^n \rightarrow \mathbb{R}$ is said to have Lipchitz continuous gradient or is L-LG if the following holds for some $L>0$:

$$ \|\nabla f(\mathbf{x}) - \nabla f(\mathbf{y})\|\leq L\|\mathbf{x}-\mathbf{y}\|,\, \forall \mathbf{x},\mathbf{y} \in \mathbb{R}^n \label{1}\tag{1}. $$

Now, consider a sigmoid function $\sigma : \mathbb{R}\to [0,1]$ where $\sigma(x)=\frac{e^x}{1+e^x}$ whose derivative is the following:

$$ \nabla_x \sigma(x)=\sigma'(x)= \sigma(x) -\sigma^2(x) $$ where $\sigma^2(x)=(\sigma(x))^2$.

Is sigmoid function L-LG? In other words, is there any $L>0$ such that the following holds for all $x, y \in \mathbb{R}^n$?

$$ |\sigma'(y) - \sigma'(x)|\leq L |y-x| $$

Saeed
  • 175

1 Answers1

1

Yes, $\sigma'$ is Lipschitz because any function with bounded derivative is Lipschitz, and $\sigma''$ is bounded. It looks like you're defining "$\sigma$ is L-LG" to mean the same as "$\sigma'$ is Lipschitz".

Edit: There's a relevant MSE answer here.

Dennis
  • 1,358