0

Could a helpful soul tell me if this approach is correct?

I need to compute the gradient $\nabla f$ and find the result and deriviation of $f(\vec{x}) = \vec{x}^T\vec{x}+c$

To solve this, I'm using a table in my textbook (First course in Machine Learning, 2nd edition, p. 23)): enter image description here

If above table can be used as I imagine, I get:

$f(\vec{x}) = \vec{x}^T\vec{x}+c \leftrightarrow \frac{\partial f}{\partial \vec{x}}(2\vec{x}+c)$.

Calculating $ \frac{\partial f}{\partial \vec{x}}(2\vec{x}+c) = 2$

The result is 2 and the derivative is $ \frac{\partial f}{\partial \vec{x}}(2\vec{x}+c)$

Rodrigo de Azevedo
  • 1
user10829235
  • 23
  • The derivative of constant $c$ is zero, not $c$ – Andrei Nov 18 '19 at 16:55
  • I thought the notation $\frac{\partial f}{\partial \vec{x}} = 2\vec{x}+c$ = 2 was correct? I excluded some steps – user10829235 Nov 18 '19 at 16:58
  • The derivative is linear so you can compute the derivative if $x^Tx$ and $c$ separately. The derivative of a constant should be very straightforward to compute. – copper.hat Nov 18 '19 at 17:07
  • Thank you for your time! It would just think the answer was 2 and not 2x (I thought I also had to take the derivative of 2x, (2x+c)' = 2, so $\nabla f$ = 2). – user10829235 Nov 18 '19 at 17:11
  • The derivative (or the dual gradient in this case) is the 'best' linear (in $h$) approximation to $f(x+h)-f(x)$. Just as the derivative of the scalar $t^2$ is $2t$ you should expect to see $x$ in the derivative of $x^Tx$. – copper.hat Nov 18 '19 at 17:20
  • 1
    Thank you, copper.hat! – user10829235 Nov 18 '19 at 17:24

1 Answers1

0

$f(x+h) = f(x)+ 2 x^T h + h^Th$ from which we can read off the derivative as $Df(x)h = 2 x^T h$, or $\nabla f(x) = 2 x$.

copper.hat
  • 172,524