Taylor approximation of matrix to matrix function

Question

Consider a function $f:M_1\to M_2$ where $M_1$ are $n \times n$ matrices and $M_2$ are $m \times m$ matrices. One could give an ordering of the entries of the matrices and thereby instead consider instead $f:\mathbb{R}^{n^2}\to\mathbb{R}^{m^2}$. We then have the taylor approximation $f(\mathbf{x+h}) = f(\mathbf{x}) + A\mathbf{x} + \ldots$ where $A$ is the jacobian matrix. Is there any exposition on how to express the taylor approximation where we stay with $f:M_1\to M_2$.

Answer 1

Yes, it is possible to express the Taylor approximation for a function $f:M1 \to M2$ in a way that stays within the space of matrices. The key is to use the notion of the matrix derivative, which is a generalization of the ordinary derivative to matrices.

To start, let's suppose that $f:M1 \to M2$ is a differentiable function, which means that it has a matrix derivative at each point in its domain. The matrix derivative, denoted by $()$, is a matrix that encodes the rate of change of $$ at the point $$.

Now, we can use the matrix derivative to express the Taylor approximation of $$ at a point $$. Just as in the ordinary case, the Taylor approximation of $$ at $$ is given by the following expression:

$(+) ≈ () + () + ⋯$

Here, the matrix $()$ represents the linear approximation of $$ at the point $$, and the higher-order terms (represented by the dots) capture the deviation of $$ from its linear approximation.

In general, computing the matrix derivative of a function $f:M1 \to M2$ can be challenging, as it involves taking partial derivatives of the entries of the matrices $()$ and $$. However, in some special cases, the matrix derivative has a particularly simple form. For example, if is a polynomial function of the entries of $$, then $()$ is simply the matrix of the partial derivatives of with respect to the entries of $$.