Show that an expression is $o(h)$

Question

I consider a function $f$ that is differentiable on an interval $I\subset\mathbb{R}$ and $r(x,h)=\frac{f(x+h)-f(x)}{h}-f'(x)$. I would like to show that $r(x,h)=o(h)$ in the sense that $\lim_{h\to 0}\frac{r(x,h)}{h}=0$. First, is this correct as statement since I usually see that when we are using the little $o$ notation when we are looking at the limit of $x$ when it goes to the infinity. Now, I cannot see how show that statement. I know that $\lim_{h\to 0}r(x,h)=0$ by construction but I'm dividing by $h$ and I take the limit when $h$ goes to zero so I cannot see how to show that the numerator goes faster to $0$ (if it is true).

Answer 1

You are possibly confusing the assertion $f(x+h)=f(x)+f'(x)\cdot h+\psi_x(h)$ where $\psi_x$ is $o(h)$ as $h\to0$. This is true, and is a template for the most common generalisation of multivariable (total) derivatives that I've seen. You can of course say $\psi_x(h)=f(x+h)-f(x)-f'(x)\cdot h$. By the way, little-o and big-o notations are fine to use as $x\to a$ for any $a$, they aren't solely used for $x\to\infty$, and your application of this notation is correct.

However, after you divide away by $h$, you cannot still expect $\psi_x(h)/h$ to be $o(h)$ - this is equivalent to $\psi_x$ being $o(h^2)$ as $h\to0$, which demands a much stronger condition on how good the derivative approximation is. As commented, a non-example is $f(x)=x^2$, for which $\psi_x(h)=2xh+h^2-2xh=h^2$ which is not $o(h^2)$ (but it is $O(h)$).

An example of the claim's strength (admittedly I've tweaked it slightly) - suppose $f$ is differentiable, and that for all $x$ in some open interval $(a,b)$, $\frac{f(x+h)-f(x)}{h}-f'(x)$ is $o(h)$ as $h\to0$ (from $x+h\in(a,b)$) uniformly in $x$. Then $f$ is twice differentiable, with its second derivative identically zero (and thus $f$ is smooth with all higher derivatives equal to $0$, a linear function). I've had to strengthen your original assertion to make this work, but I thought it was a neat result when your post inspired me to discover it!

Proof:

$$\begin{align}\lim_{h\to0}\frac{f'(x+h)-f'(x)}{h}&=\lim_{h\to0}\frac{f'(x+h)\cdot h-(f(x+h)-f(x)-\psi_x(h))}{h^2}\\&=\lim_{h\to0}\frac{-(f'(x+h)\cdot(-h)+f(x+h)-f(x))+\psi_x(h)}{h^2}\\&=\lim_{h\to0}\frac{\psi_{x+h}(-h)+\psi_x(h)}{h^2}\end{align}$$

Where $\psi_{x+h}$ is the differential error term with $f(x)=f(x+h-h)=f(x+h)+f'(x+h)\cdot(-h)+\psi_{x+h}(-h)$.

Now examine the final limit - if $\psi_{x+h}(-h)$ and $\psi_x(h)$ are both $o(h^2)$, which comes from your assertion holding for all $(x+h)$ uniformly, with $h$ sufficiently small, this limit exists and is zero, hence $f''(x)=0$.