0

I'm trying to derive the derivative of the chi-squared function w.r.t. the parameter vector $p$.

With $\chi^2$ being:

$\chi^2(p) = \sum \left[ \frac{y(t_i) - \hat{y}(ti_i; p)}{\sigma_{y_i}} \right]^2$

with $y$ being a 1xn vector, $\hat{y}$ being 1xn and $\sigma$ being a nxn symmetric matrix. In vector notation:

$$ (y - \hat{y}(p))^TW(y-\hat{y}(p)) \\ = y^TWy - 2y^TW\hat{y} + \hat{y}^TW\hat{y} $$ with $W = 1/\sigma^2_{yi}$. I dropped the dependence of $\hat{y}$ on $p$ on the second line for simplicity.

What i get is this:

$$ \frac{\partial}{\partial{p}}\chi^2 = \frac{\partial}{\partial{p}} \left( y^TWy - 2y^TW\hat{y} + \hat{y}^TW\hat{y} \right) \\ = -2y^TW + \hat{y}^T(W + W^T) \\ = -2(y-\hat{y})^TW $$

However according to this the solution should be:

$$ \frac{\partial}{\partial{p}}\chi^2 = 2(y-\hat{y})^TW \frac{\partial}{\partial{p}} (y-\hat{y}) \\ = -2(y-\hat{y})^TW \left[ \frac{\partial{\hat{y}}}{\partial{p}}\right] \\ = -2(y-\hat{y})^TWJ $$

where $J$ is the Jacobian. So somehow i seem to be missing the jacobian $J$ in my derivation. Can someone explain why?

Tadzio
  • 85
  • There seem to be a few problems, most of which seem to be because you are forgetting to use the chain rule. Y hat depends on p! And also y is constant (with respect to p), so its derivative is zero. – Elle Najt Jun 02 '18 at 18:50
  • Try writing it out in coordinates – Elle Najt Jun 02 '18 at 18:51
  • @AreaMan , how so?

    $$ \frac{\partial}{\partial{p}}\chi^2 = \frac{\partial}{\partial{p}} (y^TWy) - \frac{\partial}{\partial{p}} (2y^TW\hat{y}) + \frac{\partial}{\partial{p}} (\hat{y}^TW\hat{y}) \ = -2y^TW + \hat{y}^T(W + W^T) \ = -2(y-\hat{y})^TW $$

    the first term drops since it's a constant, $2y^TW$ is a constant so $\frac{\partial}{\partial{p}}(2y^TW\hat{y}) = 2y^TW $ no? and the derivative of the last term $\hat{y}^TW\hat{y}$ is $\hat{y}^T(W + W^T)$; just like the derivative of the quadratic form

     – Tadzio Jun 03 '18 at 10:01
  • 1
    What's the derivative of $ 2 y f(p)$? With respect to p. (The one dimensional case of the quadratic form. Here f is y hat.) You need the chain rule, or similar. – Elle Najt Jun 03 '18 at 12:52
  • 1
    I see.. Thanks! – Tadzio Jun 03 '18 at 13:53

0 Answers0