Why is $\triangledown_x x^TA^TAx = 2A^TAx$ not $2x^TA^TA$?

Question

A proof I'm looking at shows $\triangledown_x x^TA^TAx = 2A^TAx$.

I did Matlab symbolic calculation to verify this, but I found the converse. It should be $2x^TA^TA$.

$\frac{d}{dx}x^TA^TAx = \\ [ 2x_1A_{1,1}^2 + 2A_{1,2}x_2A_{1,1} + 2x_1A_{2,1}^2 + 2A_{2,2}x_2A_{2,1},\\ 2x_2A_{1,2}^2 + 2A_{1,1}x_1A_{1,2} + 2x_2A_{2,2}^2 + 2A_{2,1}x_1A_{2,2}]$

I ran jacobian(x'*A'Ax, x) for this one.

$2x^TA^TA=\\ [ 2x_1A_{1,1}^2 + 2A_{1,2}x_2A_{1,1} + 2x_1A_{2,1}^2 + 2A_{2,2}x_2A_{2,1}, \\2x_2A_{1,2}^2 + 2A_{1,1}x_1A_{1,2} + 2x_2A_{2,2}^2 + 2A_{2,1}x_1A_{2,2}]$

$2A^TAx = \\ [ 2x_1A_{1,1}^2 + 2A_{1,2}x_2A_{1,1} + 2x_1A_{2,1}^2 + 2A_{2,2}x_2A_{2,1};\\ 2x_2A_{1,2}^2 + 2A_{1,1}x_1A_{1,2} + 2x_2A_{2,2}^2 + 2A_{2,1}x_1A_{2,2}$

Am I missing something here?

Note that $x^\top A^\top Ax=|Ax|^2_2$ and https://math.stackexchange.com/q/883016/154545 — Surb, Oct 16 '18 at 00:20

score 4 · Answer 1 · answered Oct 16 '18 at 00:08

4

The only difference between the two answers is whether you want your gradient to be a row vector or a column vector.

answered Oct 16 '18 at 00:08

angryavian

89,882

When performing matrix calculus, which one is the right one though? – drerD Oct 16 '18 at 00:30

score 1 · Accepted Answer · answered Oct 17 '18 at 00:11

1

If I do graident(x'*A'Ax, x), then I get $2A^TAx$, so the original proof is correct.

answered Oct 17 '18 at 00:11

drerD

589

Why is $\triangledown_x x^TA^TAx = 2A^TAx$ not $2x^TA^TA$?

2 Answers2