To be precise, let me re-phrase the question: Let $I$ be an open
interval, $a\in I$, and $f:I\rightarrow\mathbb{R}$. Suppose that
$f''(a)$ exists, then
$$
f''(a)=\lim_{h\rightarrow0}\frac{f(a+h)+f(a-h)-2f(a)}{h^{2}}.
$$
Proof: That $f''(a)$ exists implies that there exists $\delta>0$ such
that $f'$ exists on $(a-\delta,a+\delta)$ and $f'$ is continuous
at $a$. Define functions $\phi:(0,\delta)\rightarrow\mathbb{R}$
and $\psi:(0,\delta)\rightarrow\mathbb{R}$ by $\phi(t)=f(a+t)+f(a-t)-2f(a)$
and $\psi(t)=t^{2}$. Clearly $\phi$ and $\psi$ are differentiable.
Note that
$$
\lim_{t\rightarrow0+}\frac{\phi(t)}{\psi(t)}
$$
is a $0/0$-indeterminate form. By L'Hospital rule, we have
$$
\lim_{t\rightarrow0+}\frac{\phi(t)}{\psi(t)}=\lim_{t\rightarrow0+}\frac{\phi'(t)}{\psi'(t)},
$$
provided that the limit on the RHS exists. But
\begin{eqnarray*}
\lim_{t\rightarrow0+}\frac{\phi'(t)}{\psi'(t)} & = & \lim_{t\rightarrow0+}\frac{f'(a+t)-f'(a-t)}{2t}\\
& = & \lim_{t\rightarrow0+}\left[\frac{1}{2}\frac{f'(a+t)-f'(a)}{t}+\frac{1}{2}\frac{f'(a)-f'(a-t)}{t}\right]\\
& = & \frac{1}{2}f''(a)+\frac{1}{2}f''(a)\\
& = & f''(a).
\end{eqnarray*}
Therefore, we conclude that
$$
f''(a)=\lim_{h\rightarrow0+}\frac{f(a+h)+f(a-h)-2f(a)}{h^{2}}.
$$
The case that $h\rightarrow0-$ can be proved similarly.