Derivative of the $2$-norm of a multivariate function

Question

I've got a function $$g(x,y) = \| f(x,y) \|_2$$ and I want to calculate its derivatives with respect to $x$ and $y$.

Using Mathematica, differentiating w.r.t. $x$ gives me $ f'_x(x,y) \text{Norm}'( f(x,y))$, where Norm is $\| \cdot \|$.

I read here that

$$d\|{\bf x}\| = \frac{ {\bf x}^Td{\bf x}}{\|{\bf x}\|}$$

at least for the $2$-norm. Point is, as inside the norm I have a multivariate function, I'm still confused on how to calculate $ f'_x(x,y) \text{Norm}'( f(x,y))$

I think it should be $f'_x(x,y) \frac{f(x,y)}{||f(x,y)||}$, but some verification would be great :)

Answer 1

Suppose $f:\mathbb R^m \to \mathbb R^n$. Decompose into $f = (f_1, \ldots, f_n)$. Each $f_i$ is a real-valued function, i.e., $f_i: \mathbb R^m \to \mathbb R$. Then $$ g(X) = \|f(X)\|_2 = \sqrt{\sum_{i=1}^n f_i(X)^2}. $$ Therefore, $$\nabla g(X) = \frac 12\left(\sum_{i=1}^n f_i(X)^2\right)^{-\frac 12}\left(\sum_{i=1}^n 2f_i(X)\nabla f_i(X)\right) = \frac{\sum_{i=1}^n f_i(X)\nabla f_i(X)}{\|f(X)\|_2}. $$ This matches your answer.

If you want to write in terms of the Jacobian matrix of $f$ instead of components $f_i$, you can: $$ \nabla g(X) = \frac{J_f(X)^T f(X)}{\|f(X)\|_2}. $$

Answer 2

To calculate the derivative of the function $ g(x, y) = \|f(x, y)\|^2 $ with respect to $x$, you can use the chain rule and the derivative of the Euclidean norm $\|x\|$ as you mentioned. Here's the correct calculation:

Given $$g(x, y) = \|f(x, y)\|^2,$$ let's find $\dfrac{{\partial g}}{{\partial x}}$. We have

$$ \begin{align*} g(x, y) &= \|f(x, y)\|^2 \\ &= [f(x, y) \cdot f(x, y)] \\ &= [f(x, y)]^T \cdot f(x, y) \quad \text{ (assuming \(f\) is a column vector)} \\ \end{align*} $$ Now, we can differentiate both sides with respect to $x$: $$ \begin{align*} \frac{{\partial g}}{{\partial x}} &= \frac{{\partial}}{{\partial x}} \left([f(x, y)]^T \cdot f(x, y)\right) \\ &= \left(\frac{{\partial}}{{\partial x}}[f(x, y)]^T\right) \cdot f(x, y) + [f(x, y)]^T \cdot \left(\frac{{\partial}}{{\partial x}} f(x, y)\right) \\ \end{align*} $$

Now, the first term is the derivative of the transpose of $f$ which is simply the transpose of the derivative of $f$ with respect to $x$. So, it becomes $f_x^\prime(x, y)^T$.

The second term $$\frac{{\partial}}{{\partial x}} f(x, y)$$ is the derivative of $f$ with respect to $x$, therefore the final result for $\dfrac{{\partial g}}{{\partial x}}$ is:

$$\frac{{\partial g}}{{\partial x}} = f_x'(x, y)^T \cdot f(x, y) + [f(x, y)]^T \cdot f_x(x, y)$$

This expression accounts for the derivative of the norm as well as the derivative of the function $f(x, y)$ with respect to $x$.

The approach to calculate $\dfrac{{\partial g}}{{\partial y}}$ is similar.