If $F:(a,b) \to \Bbb R$ is bounded below (so that the convex envelope exists) and differentiable then its convex envelope $\hat F$ is continuously differentiable, i.e. a $C^1$ function. (The continuity of $F'$ is not needed for this conclusion.)
More generally one can show that if $F:(a,b) \to \Bbb R$ is bounded below, and differentiable at a point $c \in (a, b)$ then $\hat F$ is also differentiable at $c$.
This implies the desired conclusion: If $F$ is differentiable on $(a, b)$ then the same holds for $\hat F$. The derivative of a convex function is (weakly) increasing and cannot have jump discontinuities (because of Darboux's theorem). It follows that the derivative of $\hat F$ is continuous (see also Continuity of derivative of convex function).
In the following I'll write $G = \hat F$ for the convex envelope of $F$ (to save some keystrokes, and because $\hat{F}'$ renders ugly in MathJax).
Assume that$F$ is differentiable at $c$, but $G$ is not differentiable at $c$. Then $G_-'(c) < G_+'(c)$ where $G_-'$ and $G_+'$ denote the left and right derivative of $G$. Define
$$
h(x) = \begin{cases}
G(c) + G_-'(c)(x-c) & \text{ if } a < x \le c \, ,\\
G(c) + G_+'(c)(x-c) & \text{ if } c \le x < b \, .
\end{cases}
$$
Then
$F(x) \ge G(x) \ge h(x)$ for all $x$.
Next show that $F(c) = G(c)$. Otherwise for sufficiently small $\epsilon$ the function $h_\epsilon$ obtained by replacing $h$ with a straight line segment on $[c-\epsilon, c+\epsilon]$ still satisfies $F(x) \ge h_\epsilon(x)$ for all $x$, but $h_\epsilon(c) > G(c)$. This contradicts the maximality of $G$.
It follows that
$$
F(x) \ge \begin{cases}
F(c) + G_-'(c)(x-c) & \text{ if } a < x \le c \\
F(c) + G_+'(c)(x-c) & \text{ if } c \le x < b
\end{cases}
$$
and that implies
$$
F_-'(c) \le G_-'(c) < G_+'(c) \le F_+'(c)
$$
in contradiction to the assumption that $F$ is differentiable at $c$.