Subtractive cancellation (same sign)

Question

I know that subtractive cancellation occurs when subtracting two numbers close to each other. But if you have two numbers close to each other, one being negative and the other postive. You subtract the postive from the negative. Will there be subtractive cancellation here?

Answer 1

Summary: If we consider the relative error then the subtraction of two real numbers is ill-conditioned if the relative error is small. If we consider the absolute error, then the subtraction of two real number is well-conditioned.

Analysis: By design, condition numbers measure the sensitivity of the output to small changes in the input. It is critical that we carefully consider how we measure size. Let $f : \mathbb{R}^2 \rightarrow \mathbb{R}$ be given by $$f(x) = x_1-x_2.$$ and consider the auxiliary function $$\kappa_f^{\text{abs}}(x,\delta) = \sup \left\{ \frac{|f(x)-f(y)|}{\|x-y\|_\infty} \: : \: 0 < \|x-y\|_\infty \leq \delta \right\}.$$ It is clear that $\delta \rightarrow \kappa_f^{\text{abs}}(x,\delta)$ is a monotone increasing and nonnegative function. This shows that the limit $$ \underset{\delta \rightarrow 0_+}{\lim}\kappa_f^{\text{abs}}(x,\delta)$$ exists. By definition, $$ \kappa_f^{\text{abs}}(x) = \underset{\delta \rightarrow 0_+}{\lim}\kappa_f^{\text{abs}}(x,\delta) $$ is the normwise absolute condition number of $f$ at the point $x$. It is easy to see that $\kappa_f^{\text{abs}}(x) = 2$. Simply choose $y_1 = x_1 - \delta$ and $y_2 = x_2 + \delta$, so that $$f(x)-f(y) = 2\delta, \quad \|x-y\|_\infty = \delta.$$ This shows that $$\kappa_f^{\text{abs}}(x,\delta) = 2.$$ It follows that subtraction is well-conditioned with respect to absolute errors.

We now consider the relative error and define a new auxiliary function $$\kappa_f^{\text{rel}}(x,\delta) = \sup \left\{ \frac{|f(x)-f(y)|}{|f(x)|} \big{/} \frac{\|x-y\|_\infty}{\|x\|_\infty} \: : \: 0 < \|x-y\|_\infty \leq \delta \|x\|_\infty \right\}.$$ It is clear that $\delta \rightarrow \kappa_f^{\text{rel}}(x,\delta)$ is a monotone increasing nonnegative function. By definition, $$\kappa_f(x) = \underset{\delta \rightarrow 0_+}{\lim}\kappa_f^{\text{rel}}(x,\delta)$$ is the normwise relative condition number of $f$ at the point $x$. However, this number is not necessarily small. The choice of $$y_1 = x - \delta |x_1|, \quad y_2 = x_2 + \delta |x_2|$$ ensures that $$\frac{f(x)- f(y)}{f(x)} = \delta \frac{|x_1|+|x_2|}{|x_1-x_2|}, \quad \|x-y\|_\infty = \delta \|x\|_\infty.$$ This implies that $$\kappa_f^{\text{rel}}(x,\delta) \geq \frac{|x_1|+|x_2|}{|x_1-x_2|}$$ and equality is not hard to verify. We conclude that $$\kappa_f^{\text{rel}}(x) = \frac{|x_1|+|x_2|}{|x_1-x_2|}$$ It follows that subtraction of two real numbers is ill-conditioned in the relative sense if $$|x_1-x_2| \ll |x_1|+|x_2|.$$ In particular, this happens when then relative difference between $x_1$ and $x_2$ is tiny.

Emphasis: Subtractive cancellation cannot happen if $x_1$ and $x_2$ have different sign, because in that case $$|x_1-x_2| = |x_1| + |x_2|.$$ Subtractive cancellation is unrelated to the absolute size of the numbers.

Intuition. Suppose that multi-billion dollar company is almost breaking even. A small relative change in the earning can easily cause a net profit. The switch from a negative to positive result is relative change which is larger than unity. This illustrates why the subtraction of near equals is ill-conditioned in the relative sense.

---

When I initially commented on your question I had not identified what could be the root cause of the confusion. It is the distinction between absolute and relative errors that must be emphasized.