0

I would like to differentiate the induced norm $\|I - \gamma P\|_2$ with respect to $\gamma$, where $I$ and $P$ are $n\times n$ matrices and $\gamma$ is a scalar. How do I proceed?

The Matrix Cookbook doesn't seem to address this. This question seems related, but I think it's unlikely that the result generalizes to matrices.

Thanks in advance.

Rodrigo de Azevedo
  • 1
  • Please clarify what kind of a matrix norm are you talking about. Vector based, Schatten, etc. ? – ConvXET Oct 15 '20 at 14:21
  • @ConvXET Vector-based, i.e. induced. I have edited my question to reflect this. –  Oct 15 '20 at 14:25

1 Answers1

1

$\def\p#1#2{\frac{\partial #1}{\partial #2}}\def\pp#1#2{\frac{d#1}{d#2}}$ The induced ${\tt2}$-norm is identical to the Schatten ${\infty}$-norm (also known as the spectral norm).

The spectral norm of $A$ can be written in terms of its SVD $$\eqalign{ A &= USV^T = \sum_{k=1}^{rank(A)} \sigma_k u_k v_k^T\\ \|A\| &= \sigma_1 \qquad\big(\sigma_1\ge\sigma_2\ge\ldots\ge\sigma_n\ge 0\big) \\ }$$ and its differential and gradient can be written using the first columns of $U$ and $V$ $$\eqalign{ d\,\|A\| &= u_1^T\,dA\;v_1 \\ \p{\|A\|}{A} &= u_1v_1^T \\ }$$ Setting $\,A=\gamma P-I\;$ yields the desired derivative $$\eqalign{ d\,\|A\| &= u_1^T\big(P\,d\gamma\big)\,v_1 \\ \pp{\|A\|}{\gamma} &= u_1^TPv_1 \\ }$$

greg
  • 35,825