4

Why doesn't the $L_2$ norm differentiable at $x=0$?

Let's define $N(x)$ as the norm function. I know that for every $x\ne 0$:

$$\frac{\partial N}{\partial x_i}(x) = \frac{x_i}{\|x\|}$$

What happens at the origin? I'd be glad to get an explanation involving minimal linear algebra :)

Thanks in advance.

Royi
  • 8,711
Elimination
  • 3,150
  • 19
  • 42
  • 1
    You are correct that $|x|_2$ is not differentiable at $x=0$. We usually use the square of the normal $|x|_2^2$, out of convenience since it is differentiable at $x=0$. See this question: http://math.stackexchange.com/questions/883016/gradient-of-l2-norm-squared – eigenchris Mar 04 '15 at 17:41

2 Answers2

3

As another commmenter noted, you can get different limits by choosing different paths to the origin. For simplicity, consider the two dimensional case. Let $f : \mathbb{R}^2 \rightarrow \mathbb{R}$ be defined by $$f(x) = \sqrt{x_1^2 + x_2^2} = \|x\|_2.$$ If $f$ were differentiable at $(0,0)$, there would exist a linear map $A: \mathbb{R}^2 \rightarrow \mathbb{R}$ of the form $(a_1,a_2)$ such that for sufficiently small $(h_1,h_2) \in \mathbb{R}^2$ near the origin, $$\lim_{\|h\|_2 \rightarrow 0} \frac{f(h_1,h_2) - f(0,0) - a_1h_1 - a_2 h_2}{\|h\|_2} = 0.$$ If we simplify this, we get $$\lim_{\|h\|_2 \rightarrow 0} \frac{\sqrt{h_1^2 + h_2^2} - a_1h_1 - a_2h_2}{\sqrt{h_1^2 + h_2^2}} = \lim_{\|h\|_2 \rightarrow 0} 1 - \frac{a_1h_1 + a_2h_2}{\sqrt{h_1^2 + h_2^2}} = 0$$ or equivalently $$\lim_{\|h\|_2 \rightarrow 0} \frac{a_1h_1 + a_2 h_2}{\sqrt{h_1^2 + h_2^2}} = 1.$$ However, if you choose different sequences for $(h_1,h_2)$ that go to the origin, you can get different limits. Try for example taking $(h_1,h_2) = (1/n,0)$ and taking the limit as $n \rightarrow \infty$. Then modify this example sequence to get a different limit.

xk3
  • 594
1

Just like for the 1D case of the $ \left| \cdot \right| $ function. You can get different derivatives by choosing different paths which goes to the origin.

Hence it is not differntiable.

Royi
  • 8,711