How to prove differentiability of $g(x)=x^TAx$?

Question

What I've started with is the definition of differentiability:

Let $G \subset \mathbb{R}^n$ be open. $g:G \rightarrow \mathbb{R}^n$ is differentiable at $x \in G$ if exists a linear transformation $L: \mathbb{R}^n \rightarrow \mathbb{R}^m$ s.t.

$$\lim_{h \rightarrow 0} \frac{g(x+h)-g(x)-Lh}{||h||}=0$$

Now since $x^TAx$ is not a linear form, then I don't understand how can do anything with the above definition.

Related: http://math.stackexchange.com/questions/312077/differentiate-fx-xtax?rq=1 — Arthur, Nov 17 '16 at 09:46
@Arthur Yes, but I'm confused about what to do with $$\lim_{h \rightarrow 0} \frac{2x^TA \cdot h + h^TAh - Lh}{||h||}$$ that results from that first answer. Is $\frac{h}{||h||}$ perhaps something special? — mavavilj, Nov 17 '16 at 09:52

score 2 · Accepted Answer · answered Nov 17 '16 at 10:30

Here, $g(x+h) = (x+h)^T A (x+h) = x^TAx + x^TAh + h^TAx + h^TAh$.

So that $$\frac{g(x+h)-g(x)}{||h||} = \frac{x^TAh + h^TAx + h^TAh}{||h||}$$.

Let $L(h) = x^TAh + h^TAx$. You can check that this is linear (in $h$).

Then, $$\frac{g(x+h)-g(x)-L(h)}{||h||} = \frac{h^TAh}{||h||}$$

Finally, note that $\lim_{h \to 0}\frac{h^TAh}{||h||} = 0$. Hence, the linear map $L(h) = x^TAh + h^TAx$ is the derivative of the map.

The way of picking $L$ was simple: I just took out all the linear terms of $h$ from the numerator using $L$. What remained were higher order terms, and these dominate $||h||$ as the quotient goes to zero. Hence, $L$ is very logically picked.

There are many ways of doing this problem. One is via what Denis suggested above. Another is via chain rule. But this is the elementary method.

How to prove differentiability of $g(x)=x^TAx$?

1 Answers1