Compute hessian using gradient operator

Question

I want to compute the hessian of following function $$f(x) = (a^T x)^2$$

where $a,x \in \mathbb{R^n}$

Therefore H is given by: $$H = \nabla_a \nabla_a^T f = \nabla_a \nabla_a^T (a^T x)^T (a^T x ) $$.

How do I apply $\nabla_a^T$. Is this to be understood element wise, so I should end up with a row vector, in this case the jacobian and in the second step then with a column vector: $\nabla_a (2a^Txx^T) = 2xx^T $

So my heurstic way is to apply $\nabla^T$ on the column vector $a$ and in the second step apply $\nabla$ on the row vector $a^T$ ?

Why don't you work on the "coordinate expression" for $f$, for exemple in dimension 2 : $f(x_1,x_2)=(a_1x_1+a_2x_2)^2$ ? — Jean Marie, Feb 14 '23 at 08:20
I see, but I do not know how to stack the vector after that as row or column? — Sarah, Feb 14 '23 at 08:22
Yes the hessian is a matrix, but I am refering to first step when I apply $\nabla^T$. What should be the shape of the result? — Sarah, Feb 14 '23 at 08:32

score 0 · Answer 1 · answered Feb 14 '23 at 08:27

0

For the first gradient, apply chain rule. Remember that gradients are column vectors. $$ \nabla f(x) = 2(a^\top x) a = 2aa^\top x $$ This is a linear mapping, so $$ Hf(x) = 2aa^\top. $$

answered Feb 14 '23 at 08:27

Hyperbolic PDE friend

6,263
8
22

So you apply $\nabla$ at first and not $\nabla^T$? – Sarah Feb 14 '23 at 08:30
Well, $Hf(x) = D(\nabla f(x)) = \nabla^\top (\nabla f(x))$ – Hyperbolic PDE friend Feb 14 '23 at 08:31
How does this make sense. Shouldn't it be the other way round, such that the gradient vectors are multiplied as outer product to build a matrix, where each entry is applied to the function? – Sarah Feb 14 '23 at 08:35
See this answer advising a row vector interpretation. – Jean Marie Feb 14 '23 at 08:45
If we assume to have computed $\nabla f$. Then$ \nabla^T$ is applied to each component of the resulting row vector, so that I end up with row vector in each component of the beforehand computed column vector, yielding a matrix? – Sarah Feb 14 '23 at 08:52

Compute hessian using gradient operator

1 Answers1