1

I am trying to find $\frac{\partial (x'Ax)}{\partial x}$ where x is a vector (2 x 1 vector) and A is a matrix (say 2x2 dimensions).

When I looked up in http://www.matrixcalculus.org/ I found the answer to be $(A.x)' + x'.A$ where $'$ stands for transpose.

I tried to solve this using $\frac{\partial (CB)}{\partial x} = \frac{\partial{C}}{\partial x}B + C\frac{\partial B}{\partial x}$ where $C = x'A$ and $B = x$. With this in perspective, I am getting the derivative as $A'x + x'A$.

Clearly with $x$ being a column vector and $A$ being a square matrix, my answer is wrong since individual terms ($A'x$ and $x'A$)have different shapes.

Where am I getting the calculations wrong. Any help ?

Upendra01
  • 133

1 Answers1

2

The rule $\frac{\partial \mathbf C\mathbf B}{\partial x} = \frac{\partial{\mathbf C}}{\partial x}\mathbf B + \mathbf C\frac{\partial \mathbf B}{\partial x}$ is for differentiation by a scalar $x$ where $\mathbf B$ and $\mathbf C$ are matrices. It is not a rule for differentiation by a vector.

With $\mathbf C = \mathbf x^\top \mathbf A$ and $\mathbf B = \mathbf x,$ you have a row vector $\mathbf C$ and a column vector $\mathbf B,$ so you can apply the rule $$ \frac{\partial \mathbf u^\top\mathbf v}{\partial \mathbf x} = \mathbf u^\top \frac{\partial{\mathbf v}}{\partial \mathbf x} + \mathbf v^\top \frac{\partial{\mathbf u}}{\partial \mathbf x} $$ with $\mathbf u^\top = \mathbf C$ and $\mathbf v = \mathbf B,$ so \begin{align} \frac{\partial \mathbf C\mathbf B}{\partial \mathbf x} &= \mathbf C \frac{\partial{\mathbf B}}{\partial \mathbf x} + \mathbf B^\top \frac{\partial{\mathbf C^\top}}{\partial \mathbf x}\\ &= \mathbf x^\top \mathbf A \frac{\partial{\mathbf x}}{\partial \mathbf x} + \mathbf x^\top \frac{\partial{\mathbf A^\top\mathbf x}}{\partial \mathbf x}\\ &= \mathbf x^\top \mathbf A + \mathbf x^\top \mathbf A^\top\\ &= \mathbf x^\top \mathbf A + (\mathbf A\mathbf x)^\top. \end{align}

Done!

For differentiation of matrices by vectors, refer to Derivative of a Matrix with respect to a vector. But that seems much more than you need or want.

David K
  • 98,388