Differentiation of norm

Question

How do I differentiate the "norm" of $(x-μ)$, with respect to $μ$, where both $x$ and $μ$ are vectors ?

How will I start and proceed ?

Thank you in advance.

Answer 1

I'll assume 2D vectors to show the idea:

$$x = (x_1, x_2)^T$$ $$\mu = (\mu_1, \mu_2)^T$$

$$\|x-\mu\| = \sqrt{(x_1-\mu_1)^2 + (x_2-\mu_2)^2}$$

\begin{align} \frac{d}{d\mu} \|x-\mu\| &= \frac{-2(x_1-\mu_1)\frac{d\mu_1}{d\mu} - 2(x_2-\mu_2)\frac{d\mu_2}{d\mu}}{2\sqrt{(x_1-\mu_1)^2 + (x_2-\mu_2)^2}} \\ &= \frac{\mu_1-x_1}{\|x-\mu\|} \frac{d\mu_1}{d\mu} - \frac{\mu_2-x_2}{\|x-\mu\|}\frac{d\mu_2}{d\mu} \\ &= \frac{\mu_1-x_1}{\|\mu-x\|} (1,0)^T + \frac{\mu_2-x_2}{\|\mu-x\|} (0,1)^T \\ &= \frac{(\mu_1-x_1,\mu_2-x_2)^T}{\|\mu-x\|} \\ &= \frac{\mu-x}{\|\mu-x\|} \end{align}

The result is $\mu-x$ normalized, i.e. a unit vector in the same direction.

Answer 2

Assuming the Euclidean norm in finite dimensional $\mathbb{R}^n$, the function you want to differentiate is this:

$$f(x_1,\dots, x_n)=\left(\sum_{i=1}^n(x_i-\mu_i)^2\right)^{1/2}$$

The derivative of this function is it's gradient, namely, the vector of it's partial derivatives. Now, for each $i$, you have

$${\partial f\over\partial x_i}(x_1,\dots,x_n)={(x_i-\mu_i)\over f(x_1,\dots,x_n)}$$

(Prove this!), and so you have the gradient.

In vector notation it looks much nicer:

$$D(||x-\mu||)={x-\mu\over||x-\mu||}$$

Answer 3

Let $\;\phi=\|x-\mu\|\;\;{\rm and}\;\;w = (\mu-x)$
Use a dot product to rewrite the norm to obtain $$\eqalign{ \phi^2 &= \|w\|^2 \;=\; w\cdot w \\ 2\phi\,d\phi &= 2\,w\cdot dw \;=\; 2\,w\cdot d\mu \\ \frac{d\phi}{d\mu} &= \frac{w}{\phi} \;=\; \frac{\mu-x}{\|x-\mu\|} \\ }$$