2

This question is similar to How to take the gradient of the quadratic form?

But I do not like the answer to that question.

The reason is because the answerer is mixing $\nabla$ with $\dfrac{\partial}{\partial x}$. What I have learned is that $\nabla f$ is a column vector, it is the gradient of $f$, and$\dfrac{\partial}{\partial x} f$ is the Jacobian of $f$, which evaluates to a row vector when $f$ is a scalar.

Why is it even appropriate to mix these notations...?

... In any case, is there a straight forward way of using properties of $\nabla$ to show $$\nabla x^TAx = (A + A^T)x, A \in \mathbb{R}^{n \times n}, x \in \mathbb{R}^n$$

I like the approach used in the linked question. Let $y(x) = Ax$, then

$$\nabla x^TAx = \nabla x^Ty(x)$$

Is there a way to use a chain rule for $\nabla$ at this stage?

I'm thinking of something like

$$\nabla x^TAx = \nabla x^Ty(x) = y(x)^T \nabla x + x^T\nabla y(x)$$

But I have no idea if the above holds.

Rodrigo de Azevedo
  • 1
Norman
  • 1,136

1 Answers1

5

Here is a proof from first principles:

Let $$f(\mathbf{x})= \mathbf{x}^T A \mathbf{x},$$ and observe that $$f(\mathbf{x}+\mathbf{h})-f(\mathbf{x})=(\mathbf{x}+\mathbf{h})^T A( \mathbf{x}+\mathbf{h})-\mathbf{x}^T A \mathbf{x}=\mathbf{x}^T A \mathbf{h}+\mathbf{h}^T A \mathbf{x}+\mathbf{h}^T A \mathbf{h}. $$ As the last term is of order $\mathcal{O} \left(|\mathbf{h}|^2\right)$, the differential of $f$ is the linear map $$df_\mathbf{x} (\mathbf{h})=\mathbf{x}^T A \mathbf{h}+\mathbf{h}^T A \mathbf{x}=\mathbf{x}^T A \mathbf{h}+\mathbf{x}^T A^T \mathbf{h}=\left( \mathbf{x}^TA+\mathbf{x}^T A^T \right) \mathbf{h} = \left( A^T \mathbf{x}+A \mathbf{x} \right) \mathbf{h}. $$

Albt.Kant
  • 3
  • 1
user1337
  • 24,381
  • sorry if this is a dense question, but why is your answer with a $x^T$ instead of a $x$, i.e. usually the gradient of it is $(A+A^T) x$ I thought (compared to for e.g. https://math.stackexchange.com/questions/222894/how-to-take-the-gradient-of-the-quadratic-form?noredirect=1&lq=1 ). – Charlie Parker Oct 21 '17 at 23:48
  • 1
    @CharlieParker Perhaps this will help. – user1337 Oct 22 '17 at 00:33