Is the derivative of the $ij$th entry of a matrix equal to the $ij$th entry of the derivative?

Question

An alternative title to my question could perhaps be:What is the derivative of the projection mapping?

In any case, let $X \in \mathbb{C}^{n\times n}$ be any matrix and consider the function $f(t) = \exp(tX), \forall t \in \mathbb{R}$. While working with matrix exponentials, I've come across the situation in which I'd need to evaluate $\frac{d}{dt}(f(t))_{ij} = \frac{d}{dt}(\exp(tX))_{ij}$ at some $t = t_0$, i.e. the derivative of the $ij$th entry of the matrix, but none of my reading materials address whether $\frac{d}{dt}(\exp(tX))_{ij} = \left(\frac{d}{dt}\exp(tX)\right)_{ij}$ as the presented proofs have not required this property.

I also couldn't find material addressing this online, so one idea I had to see whether the equality holds is to first view an $n\times m$ matrix as a $nm$-dimensional vector, so that $(\exp(tX))_{ij}$ is nothing but the projection onto the $ij$th coordinate: $(\exp(tX))_{ij} = \mathrm{pr}_{ij}(\exp(tX))$. Then, as $\mathrm{pr}_{ii}:\mathbb{C}^{n^2}\to \mathbb{C}, f:\mathbb{C}\to\mathbb{C}^{n^2}$ I thought that I could apply the chain rule of multivariate functions to $\mathrm{pr}_{ii}\circ f:\mathbb{C}\to\mathbb{C}$. But the issue with this approach is that at $t_0$ let $N = f(t_0)$, then by the chain rule of multivariate functions, $D_1(\mathrm{pr}_{ii}\circ f)_1(t_0) = \sum_{k=1}^{n^2}D_k\mathrm{pr}_{ii}(M)|_{N = M}D_1f_k(t)|_{t = t_0} = \sum_{k=1}^{n^2}\frac{\partial}{\partial M_{ij}}\mathrm{pr}_{ii}(M)|_{M = N}\frac{d}{dt}(\exp(t))_k|_{t = t_0}$

In other words, I'm still stuck with the $\frac{d}{dt}$ w.r.t. the $ij := k$th entry of the $\exp(tX)$ matrix. So what should I do?

Answer 1

Yes, the derivative of matrices are the matrix of derivatives. Also, the (Frechet) derivative of the projection mapping at any point is itself, because it is a linear transformation (i.e for any $A\in M_{n\times n}(\Bbb{R})$, we have $D(\pi_{ij})_A=\pi_{ij}$ as an equality of linear transformations). In this regard, I would highly suggest you read Loomis and Sternberg's Advanced Calculus. It is an excellent text, and covers differential calculus in Banach spaces in detail (particularly sections 3.6 and 3.7 for the basics, and 3.8,3.9 to see how the general theory works out in $\Bbb{R}^n$, and also 4.8, which talks about Banach algebras, and exponentials).

So, in general, if you have a mapping $f:\Bbb{R}\to M_{n\times n}(\Bbb{R})$ (if you want you can replace $\Bbb{R}$ with $\Bbb{C}$, it doesn't change anything), then its derivative at a point $t$ is an element $f'(t)\in M_{n\times n}(\Bbb{R})$. And just so you recall, this element $f'(t)$ is related to the Frechet derivative $Df_t:\Bbb{R}\to M_{n\times n}(\Bbb{R})$ as $f'(t)=Df_t(1)$ (this is essentially the isomorphism $\text{Hom}(\Bbb{R},W)\cong W$, $T\mapsto T(1)$, for any vector space $W$). So, \begin{align} \pi_{ij}(f'(t))&= D(\pi_{ij})_{f(t)}[f'(t)]\\ &=D(\pi_{ij})_{f(t)}[Df_t(1)]\\ &=D(\pi_{ij}\circ f)_t(1)\tag{chain rule}\\ &=(\pi_{ij}\circ f)'(t) \end{align} This equation is saying precisely that the derivative of a matrix-valued function is the matrix whose entries are the derivatives of those entries. In your case, you're just looking at the special case $f(t)=\exp(tX)$ for some fixed $X$.

And even more generally, if $V$ is any Banach space and you have a mapping $f:\Bbb{R}\to V$ which is differentiable at a point $t\in\Bbb{R}$, then for any $\lambda\in V^*$ (the continuous dual space), we have that $\lambda(f'(t))=(\lambda\circ f)'(t)$. THe proof is identical to what I wrote above; just replace $\pi_{ij}$ with $\lambda$ everywhere. This equation tells us that if we look at "a component" of the vector $f'(t)$, then that is the same as the derivative of "the component" at $t$, i.e $(\lambda\circ f)'(t)$.