9

Given a real rectangular matrix $X$, and two scalar-valued matrix functions, $f(X)$ and $g(X)$, does the product rule for differentiation of a product of scalar valued functions, hold when differentiating the product, $f(X)g(X)$ w.r.t $X$ ? If not, what would be the corresponding product rule? Let's assume that the product $f(X)g(X)$ gives a real valued scalar, and is well-defined in terms of the dimensions.

Note: $f(.)$ and $g(.)$ can be the matrix trace function for example.

halms
  • 643
  • 1
  • 6
  • 19

2 Answers2

25

The product rule holds in very great generality. Let $X,Y,Z,W$ be Banach spaces with open subset $U \subset X$, and suppose $f: U \rightarrow Y$ and $g: U \rightarrow Z$ are Frechet differentiable. If $B(\cdot, \cdot): Y \times Z \rightarrow W$ is a continuous bilinear map, then for any $\xi \in X$,

$$ \frac{d}{dx}[ B(f(x), g(x))](\xi) = B(f'(x)\xi, g(x)) + B(f(x), g'(x)\xi)$$

where all the derivatives in question are Frechet derivatives. To apply to your case, we take $U = X = \mathbb{R}^{n \times n}$, $Y =Z = W = \mathbb{R}$, and $B(y,z) = yz$.

Christopher A. Wong
  • 22,445
  • 3
  • 51
  • 82
  • This notation is a little imprecise, since we write $B(f'(x), g(x))$ where $f'(x)$ is not an element of $Y$ but rather of $L(X,Y)$. So I guess we are meant to understand $B(f'(x), g(x))$ as the linear operator $\xi \mapsto B(f'(x) \xi, g(x))$ in $L(X,W)$. – Nate Eldredge Apr 25 '20 at 15:06
  • @NateEldredge Thank you, you are absolutely correct. I'll make a correction! – Christopher A. Wong Apr 30 '20 at 02:27
17

Yes, the standard product rule applies. The gradient of the product is $$f(X)\nabla_X g(X)+g(X)\nabla_X f(X).$$ The dimensions of the gradients, of course, are the same as those of $X$ itself.

You might find The Matrix Cookbook useful here.

Michael Grant
  • 19,450
  • 2
    Isn't the order of multiplication in the second term reversed? The derivative of $f$ should go on the left, right? As in line (33) in the Matrix Cookbook. – Frank Seidl Jan 28 '22 at 23:05