1

What exactly does it mean when the form of a formula for computing something is "computationally inferior"? I get that some forms of a formula could be slower than other (equivalent) forms, but how exactly is this determined?

For example, let's say I have the equation: (side note: This is the formula for the secant method)

$x_{k+1}$ = $x_k$ - $f(x_k)$[$\frac{x_k-x_{k-1}}{f(x_k)-f(x_{k-1})}$]

How is the above form computationally superior than

$x_{k+1}$ = $\frac{f(x_k)x_{k-1}-x_kf(x_{k-1})}{f(x_k)-f(x_{k-1})}$ (which is equivalent to the formula above - written differently)

I'm a bit confused and I'm not sure how to compare the two! Any help?

Lol
  • 13
  • Skimmed over the wikipedia description of secant method. It seems both the numerator and denominator in your fraction are supposed to converge to $0$. Problem is, if you use a non-exact representation of real numbers, you are prone to small rounding errors. When computing a ratio, small quantities of the values can lead to "large" variation of the ratio. In that case, when the ratio is performed last, you get the full brunt of the instability in your value $x_{k+1}$. In the other form, the instability is somewhat compensated by the multiplication by $f(x_k)$, since that also tends to $0$. – N.Bach Nov 20 '17 at 18:39
  • 1
    Funnily enough, I just noticed the side panel proposed this related question: https://math.stackexchange.com/questions/2440646/explain-why-this-formula-is-inferior-to-x-n1-x-n-fx-n-left-dfracx-n-x-n It nearly is the same question, with a much better explanation than mine. – N.Bach Nov 20 '17 at 18:41
  • 1

1 Answers1

3

Your specific question related to secant method has already been answered here, so I will instead address you more general question, i.e., why are some formula inferior to others.

While computational speed is important, it is completely secondary to meeting the accuracy goal for all possible input. Two different formula can be equivalent, yet differ wildly when evaluated using finite precision arithmetic.

A good example are the functions $f$ and $g$ given by $$f(x) = \sqrt{4 + x^2} - 2, \quad g(x) = \frac{x^2}{2 + \sqrt{4 + x^2}}.$$ It is trivial to verify that $f(x) = g(x)$ for all $x$, yet when evaluated using, say, IEEE double precision floating point arithmetic, profound difference immediately emerge. For small values of $x$, the graph of $f$ is shaped like a staircase, while the graph of $g$ has the correct parabolic shape.

enter image description here

Moreover, for $|x| < 2^{-26}$, the computed value of $f$ is $0$, despite the fact that $f(x) > 0$ for all $x \not = 0$. In on the set $0 < |x| < 2^{-26}$ the relative error is 100%. The function $g$ does not suffer from this problem.

As in the case of the secant method, the departure from the mathematical reality is due to an ill conditioned step in the computation of $f$. In general, the subtraction $d = a - b$ of two real numbers is an ill conditioned problem, when $a$ and $b$ are close to each other. This is a phenomenon which is explained in some detail here.

In contrast, the addition of positive real numbers, multiplication and division of real numbers as well as the computation of square roots are all well conditioned problems. Therefore, the formula for $g$ can be evaluated accurately. However, since $\sqrt{4 + x^2} \approx 2$ for $x \approx 0$, the subtraction dramatically magnifies the relative importance of the rounding errors committed when computing $\sqrt{4 + x^2}$.

In general, we have to rewrite expressions to avoid steps which are ill conditioned.

Lutz Lehmann
  • 126,666
Carl Christian
  • 12,583
  • 1
  • 14
  • 37