In this post it is said that if $f : \mathbb R \to \mathbb R$ is differentiable at $a$ then there exists a continuous function $\varphi$ defined on an interval $[-\epsilon, \epsilon]$ such that $\varphi(0) = 0$ and $$ f(a + h) = f(a) + f'(a)h + \varphi(h)h $$ for all $h \in (-\epsilon, \epsilon)$.
Now my question: Is see that $\varphi$ in some sense resembles the error term, and that as $$ \varphi(h) = \frac{f(a + h) - f(a)}{h} - f'(a) $$ we have $\varphi(h) \to 0$ as $h \to 0$, hence it is continuous at $0$ (which implies that $f$ is continuous at $a$). But is $\varphi$ necessarily continuous on the entire interval $[-\epsilon, \epsilon]$? I cannot prove that...
EDIT: Found this post, which is somehow related.