0

I am trying to fully understand how I can use Levenberg-Marquardt to minimise a 4 parameter equation. There are lots of fancy programs to do this but the documentation about the mathematics is confusing to me. I would really like to understand what is going on to obtain an answer.

My equation is of the form: $$ x = p_2 * [\frac{2p_1 + p_4 - y}{y - p_1}]^{(1/p_3)} $$ where $p_1$ to $p_4$ are the parameters to be minimised (initial guesses used for them). $x$ is known and $y$ is measured.

From what I have read (in particular Numerical Optimization by Nocedal and Wright) I understand the Jacobian used in the algorithm to be the derivative of residuals. The residuals being: known $x$ - $x$ calculated, using $p$ values and $y$ values.

I want to ensure that my Jacobian format is correct. At the moment say I have 10 $x$ and $y$ values. I believe my jacobian to be a 10 row * 4 column matrix, where each column represents the derivative of the function with respect to one of the variables as below, where $x$ represents the function above:

$$\begin{bmatrix} \frac{\partial x_1}{\partial p_1} \frac{\partial x_1}{\partial p_2} \frac{\partial x_1}{\partial p_3} \frac{\partial x_1}{\partial p_4} \\\ \frac{\partial x_2}{\partial p_1} \frac{\partial x_2}{\partial p_2} \frac{\partial x_2}{\partial p_3} \frac{\partial x_2}{\partial p_4} \\\ .................. \\\ .................. \\\ \frac{\partial x_{10}}{\partial p_1} \frac{\partial x_{10}}{\partial p_2} \frac{\partial x_{10}}{\partial p_3} \frac{\partial x_{10}}{\partial p_4} \end{bmatrix}$$

Have I misinterpreted Find the Jacobian article or is this format correct?

Also as it is the Jacobian of residuals to be used is the equation below correct for the entry on the first row & second column?

$$ \frac{\partial}{\partial p_2} = x_1 - [\frac{2p_1 + p_4 - y}{y_1 - p_1}]^{(1/p_3)}$$

Any help and guidance appreciated as I am worried I am on the wrong track entirely!

Community
  • 1
Ahrz
  • 86

1 Answers1

0

I realise now that I have made a basic mathematical error with this. I now have a working Levenberg-Marquardt algorithm with the following Jacobian format:

$$\begin{bmatrix} \frac{\partial x_1}{\partial p_1} \frac{\partial x_1}{\partial p_2} \frac{\partial x_1}{\partial p_3} \frac{\partial x_1}{\partial p_4} \\\ \frac{\partial x_2}{\partial p_1} \frac{\partial x_2}{\partial p_2} \frac{\partial x_2}{\partial p_3} \frac{\partial x_2}{\partial p_4} \\\ .................. \\\ .................. \\\ \frac{\partial x_{10}}{\partial p_1} \frac{\partial x_{10}}{\partial p_2} \frac{\partial x_{10}}{\partial p_3} \frac{\partial x_{10}}{\partial p_4} \end{bmatrix}$$

As this algorithm uses the Jacobian of residuals, and the residuals would take the form:

$$ x - p_2 * [\frac{2p_1 + p_4 - y}{y - p_1}]^{(1/p_3)} $$

Differentiating would cause the $x$ to "disappear" as it is a constant and so disappears when differentiation is performed.

So the entry on the first row and second column would be:

$$ \frac{\partial}{\partial p_2} = - [\frac{2p_1 + p_4 - y}{y_1 - p_1}]^{(1/p_3)}$$

My confusion stemmed from the fact that the Jacobian is referred to using many names in texts such as the negative Gradient, change in residuals and determinant of the negative Gradient.

Ahrz
  • 86