Differential and derivative of the trace of a matrix

Question

If $X$ is a square matrix, obtain the differential and the derivative of the functions:

1. $f(X) = \operatorname{tr}(X)$,
2. $f(X) = \operatorname{tr}(X^2)$,
3. $f(X) = \operatorname{tr}(X^p)$ ($p$ is a natural number).

---

To find the differential I thought I could just find the differential of the compostion function first and then take the trace of that differential. Am I right in saying so? As for the derivative, I have no idea how I should do it for traces. Could anyone please help me out?

wj32's answer makes sense to me, however, I wonder if it is also possible to solve this question by using the ordinary way of finding differentials and derivatives, namely f(x+dx)-f(x). Is there someone who could maybe show me how this would be done (if possible)?

Answer 1

1) The trace is linear and bounded (which is automatic in finite dimension) so its derivative is equal to itself everywhere $$ df_X(H)=\mbox{tr}(H). $$

2) This is the composition of the trace with the bounded bilinear map $g:(X,Y)\longmapsto XY$ whose derivative is $$ dg_{(X,Y)}(H,K)=g(X,K)+g(H,Y)=XK+HY. $$ and the bounded linear map $h:X\longmapsto (X,X)$ whose derivative is itself at every point.

So by the chain rule $$ df_X(H)=d\mbox{tr}_{X^2}\circ dg_{(X,X)}\circ dh_{X}(H)=\mbox{tr}(XH+HX). $$ And by commutativity of the trace, this yields $$ 2\mbox{tr}(XH). $$

3) This is a composition again, of the trace with the $p$-linear map $$ k:(X_1,\ldots,X_p)\longmapsto X_1\cdots X_p $$ and the linear map $l:X\longmapsto (X,\ldots,X)$.

The derivative of $k$ is $$ dk_{(X_1,\ldots,X_p)}(H_1,\ldots,H_p)=H_1X_2\cdots X_p+X_1H_2X_3\cdots X_p+\ldots+X_1\cdots X_{p-1}H_p. $$

And the derivative of $l$ is itself. So by the chain rule $$ df_X(H)=d\mbox{tr}_{X^p}\circ dk_{(X,\ldots,X)}\circ dl_X(H). $$ Using the commutativity of the trace, we find $$ p\mbox{tr} (X^{p-1}H). $$

Answer 2

$\newcommand{\tr}{\operatorname{tr}}$I'm not familiar with matrix-ey notation, so I'll just write down what I know.

Let $V$ be a finite-dimensional real vector space (or more generally a Banach space) and let $L(V)$ be the space of continuous linear operators on $V$. The trace is a linear map $\tr:L(V)\to\mathbb{R}$, so $D\tr(x)=\tr$ for every $x\in L(V)$. That answers your first question.

For the third question, we just use the chain rule. Define $p_n:L(V)\to L(V)$ by $p_n(x)=x^n$. You want \begin{align} D(\tr\circ p_n)(x)u &= (D\tr(x^n)\circ Dp_n(x))u \\ &= \tr\left(\sum_{k=0}^{n-1} x^kux^{n-k-1}\right) \\ &= \sum_{k=0}^{n-1}\tr\left(x^kux^{n-k-1}\right) \\ &= n\tr(x^{n-1}u). \end{align}

---

Lemma (Power rule). Let $E$ be a Banach algebra and let $p_n:E \to E$ be the map defined by $p_n(x)=x^n$. Then $$Dp_n(x)u=\sum_{k=0}^{n-1} x^kux^{n-k-1}.$$ In particular, if $E$ is commutative then $$Dp_n(x)u=nx^{n-1}u,$$ which is just the plain old power rule.

Proof. We use induction on $n$. The case $n=0$ is clear, so suppose the result holds for $n-1$. Since $p_n(x)=xp_{n-1}(x)$, the product rule shows that \begin{align} Dp_n(x)u &= up_{n-1}(x)+xDp_{n-1}(x)u \\ &= ux^{n-1}+x\sum_{k=0}^{n-2}x^kux^{n-k-2} \\ &= \sum_{k=0}^{n-1}x^kux^{n-k-1}. \end{align}

Answer 3

I found a really good and well understandable explanation here:

Matrix Calculus - Notes on the Derivative of a Trace, Johannes Traa

It writes matrix calculations as sums, where you can find the derivative with the known rules for scalars. In the end, you can convert the result back to matrix notation.