2

Prove that the formula for the secant method can be written in the form $$x_{n+1}=\dfrac{f(x_n)x_{n-1}-x_nf(x_{n-1})}{f(x_n)-f(x_{n-1})}$$ Explain why this formula is inferior to $$x_{n+1}=x_n-f(x_n)\left[\dfrac{x_n-x_{n-1}}{f(x_n)-f(x_{n-1})}\right] \hspace{0.5cm} (n\geq 1)$$ in practice.

Proof: We would like to rewrite the secant method as $$x_{n+1}=\dfrac{f(x_n)x_{n-1}-x_nf(x_{n-1})}{f(x_n)-f(x_{n-1})}.$$ For $n\geq 1$, \begin{equation*} \begin{aligned} x_{n+1}& =x_n-f(x_n)\left[\frac{x_n-x_{n-1}}{f(x_n)-f(x_{n-1})}\right] \\[8pt] & =x_n-\dfrac{x_nf(x_n)-x_{n-1}f(x_n)}{f(x_n)-f(x_{n-1})} \\[8pt] & =\frac{x_nf(x_n)-x_nf(x_{n-1})-x_nf(x_n)+x_{n-1}f(x_n)}{f(x_n)-f(x_{n-1})} \\[8pt] & =\frac{x_{n-1}f(x_n)-x_nf(x_{n-1})}{f(x_n)-f(x_{n-1})} \end{aligned} \end{equation*}

The question states that the first equation is inferior to the second equation but I don't know why this is. Can anyone help me understand why?

Michael Hardy
  • 1
Username Unknown
  • 4,633
  • 1
    I think that it is because of the iteration. In the first case you can make 1)a step that take care completely of n-1 terms and 2) find the n+1 using just the terms n and the last term calculated independently. In the formula that you demonstrated you can't do that because the terms are mixed. – R.W Sep 22 '17 at 16:34
  • The simplified version doesn't save any arithmetic, and I would guess that it is more susceptible to floating point error (because the term that involves subtraction of nearly equal numbers has no small coefficient out in front of it). I'm having some trouble making up an example though. – Ian Sep 22 '17 at 16:48

2 Answers2

4

The second expression is not inferior to the first expression because of the division of two small numbers. The second expression is inferior to the first expression because the subtraction $$d = x - y$$ of two real numbers is ill conditioned when $$x \approx y.$$ When the secant method convergences, and $$x_{n-1} \approx x_*, \quad x_n \approx x_*,$$ both the nominator and the denominator of the second expression are not necessarily computed with a small relative error. Therefore, there is no guarantee that the value of $x_{n+1}$ is computed with a small relative error and in general, the relative error will not be small.

In contrast, the first expression, i.e., $$ x_{n+1} = x_n - f(x_n) \frac{x_n - x_{n-1}}{f(x_n - x_{n-1}},$$ is of the preferred form $$ \text{improved approximation} = \text{good approximation $+$ small correction}.$$ It does not matter that the correction experiences subtractive cancellation as long as the correction is orders of magnitude smaller than the current approximation. In popular terms, you do not need 16 or even 10 correct decimal digits to correct the last 5 digits of an approximation which has 11 correct digits.

This principle is ruthlessly exploited when doing iterative refinement using mixed precision arithmetic or when computing Lagrange multipliers when integrating a constrained system of ordinary differential equations.

Carl Christian
  • 12,583
  • 1
  • 14
  • 37
2

As the secant method converges, the $x_n$ is expected to converge to the value $x^*$ where $f(x^*)=0$.

The problem with the alternative formula is that the result of dividing numerator and denominator is supposed to converge to $x^*$, yet the numerator and denominator are both getting closer to zero. Dividing these almost-zero numbers will lead to more and more numerical instability as the iterations increase.

OTOH in the standard formula the next value of $x_n$ is obtained from the previous by subtracting a delta which tends to zero in a more stable manner. It's true that the ratio $$\frac{x_n - x_{n-1}}{f(x_n)-f(x_{n-1})}$$ (which is tending to $1/f'(x^*)$) is also approaching $0/0$, but this is moderated by the fact the ratio is being multiplied by $f(x_n)$, which is tending to $f(x^*)=0$. It should take longer for the numerical instability to emerge, typically long after you've halted the iteration.

grand_chat
  • 38,951