3

I found this answer that stated the following theorem -

Theorem: Let be a real valued function defined in a neighbourhood of point and continuous at and lets assume that it is approximated by a linear function given by ()=+ in the neighbourhood of . Then we say that is best linear approximation of in the neighbourhood of if the following equation holds :

$$ \lim_{x\to a} \frac{f(x)-g(x)}{x-a}=0$$

Such a linear approximation exists if and only if ′() exists and moreover in that case we have ()=()+′()(−).

This answer also uses this theorem to prove that the derivative is truly the best linear approximation. More like this is the 'sense' in which it is the best approximation.

After researching online I found that the idea seems to be that the derivative is the only linear approximation for which the approximation error tends to $0$ faster than $-$ as $→$, and based on this we call it the best approximation.

My question is, how does this actually prove that the derivative will beat any other linear approximation? How does it formally (if possible intuitively also) prove that the derivative is better than all the other approximations.

Community
  • 1
chetan
  • 133

2 Answers2

5

Let $f$ your function, $f_1 : x \rightarrow a(x-x_0) + b$ the approximation given by the derivative, and $f_2 : x \rightarrow a'(x-x_0) + b'$ some other approximation. The approximations are done in a neighborhood of $x_0$.

Of course, if $b' \neq b$, $f_2$ is a bad approximation since $f(x_0) \neq f_2(x_0)$.

Let us suppose $a \neq a'$.

You have $$\lim_{x \rightarrow x_0} \frac{f(x) - f_1(x)}{x - x_0} = 0$$. And : $$\frac{f(x) - f_2(x)}{x-x_0} = \frac{f(x) - f_1(x)}{x-x_0} + \frac{f_1(x) - f_2(x)}{x-x_0} = \frac{f(x) - f_1(x)}{x-x_0} + a - a'$$

In such a way that :

$$\lim_{x \rightarrow x_0} \frac{f(x) - f_2(x)}{x - x_0} = a - a'$$

This enables to show that (take the quotient of the two preceding limits) :

$$ \lim_{x \rightarrow x_0} \frac{f(x) - f_1(x)}{f(x) - f_2(x)} = 0$$

This last equality encapsulates the intuition that the (local at $x_0$) approximation $f_1$ is far better than $f_2$. If you write it with the epsilons : for all $\epsilon > 0$ there exists $\delta > 0$ such that for $x \in ]x_0 - \delta, x_0 + \delta[$, $|f(x) - f_1(x)| \leq \epsilon | f(x) - f_2(x) |$

EDIT : The last equality is also true if $b' \neq b$ (to be more precise, it is "more true", since in this case, $\lim_{x \rightarrow x_0} \frac{f(x) - f_2(x)}{x- x_0} = +\infty$). It justifies the nomination "best approximation" for the approximation given by the derivative : it is far better than any other.

Dlem
  • 888
  • 4
  • 9
  • Thanks, this is precisely what I was looking for. – chetan Feb 19 '20 at 14:43
  • why don't we just divide $\lim_{x \rightarrow x_0} \frac{f(x) - f_1(x)}{x - x_0} = 0$ with $\lim_{x \rightarrow x_0} \frac{f(x) - f_2(x)}{x - x_0} ≠ 0$ to get that last part of the answer? – chetan Feb 27 '20 at 21:18
  • 1
    This is exactly what I have done ^^ I just needed to calculate (and prove the existence of) $\lim_{x_\rightarrow x_0} \frac{f(x) - f_2(x)}{x-x_0}$. I was also assuming you did not know Taylor expansion ; with such an expansion, the reasonning is a little faster and maybe more intuitive. But it might hide some difficulties. – Dlem Mar 02 '20 at 09:24
1

Suppose that $h$ is another linear map such that$$\lim_{x\to a}\frac{f(x)-h(x)}{x-a}=0.$$Then $\lim_{x\to a}f(x)-h(x)=0$, and therefore $h(a)=f(a)$. So, $h(x)=f(a)+k(x-a)$ for some constant $k$. And the same argument applies to $g$: $g(x)=f(a)+k^\ast(x-a)$ for some constant $k^\ast$. But then\begin{align}0&=\lim_{x\to a}\frac{f(x)-g(x)}{x-a}-\frac{f(x)-h(x)}{x-a}\\&=\lim_{x\to a}\frac{h(x)-g(x)}{x-a}\\&=\lim_{x\to a}\frac{(k^\ast-k)(x-a)}{x-a}\\&=k^\ast -k\end{align}and therefore $k=k^\ast$, which means that $g=h$.

José Carlos Santos
  • 427,504
  • 1
    Okay, I understood that this proves that there is only one unique linear map that solves the equation. But how can we show that the approximation of this linear map is better than the approximation of all the other linear maps that don't solve this equation ? – chetan Feb 18 '20 at 13:10
  • 2
    Because we define a best linear approximation of $f$ in the neighbourhood of $a$ as a linear map $g$ for which$$\lim_{x\to a}\frac{f(x)-g(x)}{x-a}=0$$holds. – José Carlos Santos Feb 18 '20 at 13:13
  • So it means that this is just how it is defined and there is no other sense in which the approximation is best. Actually I thought that there might be a different reason because I found this paper that proves without any predefined notions that the approximation of tangent line will always be better than any other line. – chetan Feb 18 '20 at 13:21
  • We have to use some definition, right?! – José Carlos Santos Feb 18 '20 at 13:22
  • @chetanraina The definition of “best” in that paper is equivalent to this one. – amd Feb 18 '20 at 20:24
  • @amd I don't think it is. The definition 1 in paper is that the error of tangent line is less that any other line K. This is the correct and intuitive definition of 'being the best' in the sense that it shows how the approximation is better than all others. I don't see how it directly equates to the other definition. The other definition makes no comparison of that approximation with approximations of other lines. – chetan Feb 19 '20 at 14:38
  • 1
    @JoséCarlosSantos The definition in that paper and answer of DLeMeur show that the approximation is 'best' because it is better than any other approximation. I think that is the definition that makes more sense and maybe the definition you stated is a consequence of this. But your answer is also important as it shows that there is a unique approximation satisfying that equation and hence it is the best. – chetan Feb 19 '20 at 14:42