Derivative of a quadratic form — how to derive it?

Question

I want to know how $$\frac{\delta(x^TAx)}{\delta(x)}=2Ax$$

I think here's what happens (please correct me where wrong):

(by: the rule for matrix derivative) $$ \frac{\delta(x^TAx)}{\delta(w)}=x^T(A^T+A) $$

(by: I assume we can transpose matrices whenever we need to?) $$ x^T(A^T+A) = x^T(A^T+A^T) = x^T(2A^T) $$

(by: transpose property) $$ x^T(2A^T) = (x^T2A^T)^T = 2Ax $$

Is that how this happens?

score 4 · Accepted Answer · answered May 11 '19 at 17:23

4

I assume by $\delta$ you mean $\partial$, viz. \partial. I'll use $\delta$ in the Kronecker delta $\delta_{ij}$, which is the $_{ij}$ entry of the identity matrix. With implicit summation over repeated indices,$$\frac{\partial x^TAx}{\partial x_i}=\frac{\partial(A_{jk}x_jx_k)}{\partial x_i}=A_{jk}\delta_{ij}x_k+A_{jk}x_j\delta_{ik}=A_{ik}x_k+A_{ji}x_j=[(A+A^T)x]_i.$$So actually, the result is $\frac{\partial x^TAx}{\partial x}=(A+A^T)x$, although this reduces to $2Ax$ for symmetric $A$.

answered May 11 '19 at 17:23

J.G.

115,835

awesome... thank you! is this a general pattern for derivative proofs for matrices? (turn into the sum form, simplify, rewrite back into matrix form)? – William Falcon May 11 '19 at 17:38
q2: is there a cheatsheet of known derivatives for matrices? (ie: what can I google)? – William Falcon May 11 '19 at 17:39
@WilliamFalcon To Q1, yes; to Q2, you want matrix calculus identities. – J.G. May 11 '19 at 17:56

Derivative of a quadratic form — how to derive it?

1 Answers1