I also realised I can't use l'Hôpital rule or Taylor's theorems, because the hypothesis is weaker.
The hypothesis is in fact sufficient to use L'Hôpital once.
The numerator has limit $\lim_{h\to 0} f(a+h)+f(a-h)-2f(a) = 2f(a)-2f(a)=0$ by continuity of $f$, and the denominator obviously tends to $0$ as well. Both are differentiable in $h\,$, then by L'Hôpital's rule the limits are equal provided that the RHS one exists:
$$\lim_{h\to 0} \frac{f(a+h)+f(a-h)-2f(a)}{h^2} = \lim_{h \to 0} \frac{f'(a+h)-f'(a-h)}{2h}$$
But the RHS limit is $f''(a)$ by the definition of derivatives, and given that $f'$ is differentiable at $a$ the limit exists, so the previous step holds, and the original limit exists and is equal to $f''(a)\,$.