Assume that $f$ is $C^2$ near 0. I would like to show the following Schwartz derivative $$ \frac{f''(0)}{2} =\lim_{x\to 0} \frac{\frac{f(x) -f(0)}{x}-f'(0)}{x} $$
I am able to do this by using the Taylor expansion and L'Hopital rule. I am wondering how one can prove it without using Taylor expansion or L'Hopital rule.
Suppose $x>0$. Note \begin{eqnarray} &&\frac{\frac{f(x)-f(0)}{x}-f'(0)}{x}\\ &=&\frac{f(x)-f(0)-xf'(0)}{x^2} \\ &=&\frac{\int_0^xf'(t)dt-\int_0^xf'(0)dt}{x^2} \\ &=&\frac{\int_0^x[f'(t)-f'(0)]dt}{x^2} \\ &=&\frac{\int_0^x\bigg[\int_0^tf''(s)ds\bigg]dt}{x^2} \\ &=&\frac{\int_0^x\bigg[\int_s^xf''(s)dt\bigg]ds}{x^2} \\ &=&\frac{\int_0^x(x-s)f''(s)ds}{x^2} \end{eqnarray} and $$ \int_0^x(x-s)ds=\frac12x^2. $$ So \begin{eqnarray} &&\bigg|\frac{\frac{f(x)-f(0)}{x}-f'(0)}{x}-\frac12f''(0)\bigg|\\ &=&\bigg|\frac{\int_0^x(x-s)[f''(s)-f''(0)]ds}{x^2}\bigg|\\ &\le&\bigg|\frac{\int_0^x(x-s)|f''(s)-f''(0)|ds}{x^2}\bigg| \end{eqnarray} Since $f\in C^2$, for $\forall \varepsilon>0$, $\exists \delta>0$ such that $$ |f''(x)-f''(0)|<\varepsilon \forall x\in(0,\delta). $$ Thus for $x\in(0,\delta)$, \begin{eqnarray} &&\bigg|\frac{\frac{f(x)-f(0)}{x}-f'(0)}{x}-\frac12f''(0)\bigg|\\ &\le&\frac{\int_0^x(x-s)|f''(s)-f''(0)|ds}{x^2}\\ &\le&\bigg|\frac{\int_0^x(x-s)\varepsilon ds}{x^2}\bigg|\\ &=&\frac12\varepsilon. \end{eqnarray} So $$ \lim_{x\to0}\frac{\frac{f(x)-f(0)}{x}-f'(0)}{x}=\frac12f''(0). $$
Do you mind using the following theorem:
Theorem:If $f'$ is positive in some interval then $f$ is strictly increasing in that interval.
Typical proofs of this theorem involve Rolle's theorem or mean value theorem. There is another less well known proof which involves definition of derivatives and completeness of real numbers. Let me know if you are interested in this particular proof.
Let's assume that $f''(0)$ exists and nothing more. We will prove the result in question via definition of limit. Let $\epsilon >0$ be arbitrarily given. Consider $$F(x) = f(x) - f(0)-xf'(0),x\geq 0$$ and $$G(x) =F(x) - \frac{x^{2}}{2}\{f''(0)-\epsilon\}$$ We have $G(0)=G'(0)=0,G''(0)=\epsilon>0$. It follows that there is a positive number $\delta_{1}$ such that $G' >0$ in $(0,\delta_{1})$ so that $G$ is increasing in $[0,\delta_{1}]$ and hence $G(x) >G(0)=0$. Thus $F(x) >(x^{2}/2)(f''(0)-\epsilon)$ or $$\frac{f(x) - f(0)-xf'(0)}{x^{2}}>\frac{f''(0)}{2}-\frac{\epsilon}{2}$$ for all $x\in(0,\delta_{1})$. Similarly by considering $H(x) =F(x) - (x^{2}/2)(f''(0)+\epsilon) $ we can show that $$\frac{f(x) - f(0)-xf'(0)}{x^{2}}<\frac{f''(0)}{2}+\frac{\epsilon}{2} $$ for all $x\in(0,\delta_{2})$ for some positive $\delta_{2}$. Choosing $\delta=\min(\delta_{1},\delta_{2})$ we see that $$\left|\frac{f(x)-f(0)-xf'(0)} {x^{2}} - \frac{f''(0)}{2}\right|<\frac{\epsilon} {2}<\epsilon$$ for all $x$ with $0<x<\delta$. Thus we have $$\lim_{x\to 0^{+}}\frac{f(x)-f(0)-xf'(0)}{x^{2}}=\frac{f''(0)}{2}$$ The case $x\to 0^{-}$ can be handled similarly.
Proofs based on integration (see other answers) require more assumptions like continuity of $f' '$. It is also preferable to prove a result about derivatives using techniques of differential calculus. The above proof is not so well known and it can be generalized.
WLOG we can put $f(0)=0$. Take $g(x) = \int_0^1 f'(tx)dt-f'(0) = \frac 1x\int_0^x f'(u)du-f'(0) =\frac{f(x)}{x}-f'(0)$.
It is easy to see that $g(0)=0$. Now apply the same technique to $g$:
$$h(x) = \int_0^1 g'(tx)dt = \ldots = \frac{\frac{f(x)}{x}-f'(0)}{x}$$ We also know that $g'(x) = \int_0^1tf''(tx)dt$, so $$h(x) = \int_0^1 g'(sx)ds = \int_0^1 \int_0^1tf''(tsx)\,dt\, ds.$$
All we need to do now is to pas to the limit $x\to 0$ in the integral above to obtain $$\lim_{x\to0}\frac{\frac{f(x)}{x}-f'(0)}{x} = \lim_{x\to0}h(x) = \lim_{x\to0} \int_0^1 \int_0^1tf''(tsx)\,dt\, ds = \int_0^1 \int_0^1tf''(0)\,dt\, ds = \frac{f''(0)}{2}$$
Well, whether you want it or not, this is Taylor with the integral remainder in disguise.
Since $f$ is $C^1$ then $\displaystyle \int_0^x f'(t)dt=f(x)-f(0)$
And since $f$ is also $C^2$ we can integrate by parts to find
$\displaystyle \int_0^x f'(t)dt=\bigg[(t-x)f'(t)\bigg]_0^x-\int_0^x (t-x)f''(t)dt=xf'(0)+\int_0^x (x-t)f''(t)dt$
And we find the Taylor formula with integral remainder:
$$f(x)=f(0)+xf'(0)+\int_0^x (x-t)f''(t)dt$$
Now notice that $\displaystyle \int_0^x (x-t)f''(0)dt=f''(0)\bigg[-\dfrac{(x-t)^2}2\bigg]_0^x=x^2\times \frac 12 f''(0)$
So when we regroup all this into one formula we have:
$\displaystyle h(x)=\dfrac{\dfrac{f(x)-f(0)}x-f'(0)}x-\dfrac 12f''(0)=\dfrac 1{x^2}\int_0^x (x-t)\big(f''(t)-f''(0)\big)dt$
Eventually exploiting the fact that $f''$ is continuous in $0$, we have $\bigg|f''(t)-f''(0)\bigg|<\varepsilon$
$\displaystyle\implies |h(x)|<\dfrac 1{x^2}\int_0^x (x-t)\ \varepsilon\ dt=\dfrac 1{x^2}\bigg[-\dfrac{(x-t)^2}2\varepsilon\bigg]_0^x=\dfrac{\varepsilon}2\qquad$ thus $\ \lim\limits_{x\to 0} h(x)=0$
