$1.$ Suppose that $f''(a)$ exists. Show that $\lim\limits_{h\to 0} \dfrac{f(a+h)+f(a-h)-2f(a)}{h^2}=f''(a).$
$2.$ Show by example that this limit may exist even when $f''(a)$ does not.
My work:
$1.$
By the derivative definition, $$f''(a) = \lim\limits_{h\to 0}\dfrac{f'(a+h)-f'(a)}{h}\\$$ $$=\lim\limits_{h\to 0} \dfrac{f'(a)-f'(a-h)}{h}.$$ To see this, let $k=-h.$ Then $k\to0\Leftrightarrow h\to 0$ and $$\lim\limits_{h\to 0}\dfrac{f'(a)-f'(a-h)}{h} = \lim\limits_{k\to 0}\dfrac{f'(a)-f'(a+k)}{-k}\\ =\lim\limits_{k\to 0}\dfrac{f'(a+k)-f'(a)}{k}\\ =\lim\limits_{h\to 0}\dfrac{f'(a+h)-f'(a)}{h}.$$ So the limit is equivalent to $$\lim\limits_{h\to 0}\dfrac{\frac{f(a+h)-f(a)}{h}-\frac{f(a)-f(a-h)}{h}}{h}\\ =\lim\limits_{h\to 0}\dfrac{f(a+h)+f(a-h)-2f(a)}{h^2}.$$
$2.$
Consider $f(x)=\begin{cases} x^2\sin (1/x)& x\neq 0\\ 0& x=0\end{cases}.$
We have that $f'(x) = 2x\sin(1/x)-\cos(1/x),x\neq 0$ and $f''(x) = 2\sin(1/x)-\dfrac{2}{x}\cos (1/x)-\dfrac{\sin(1/x)}{x^2},x\neq 0.$ Note that $f'(0)=\lim\limits_{h\to 0}\dfrac{h^2\sin (1/h)}{h}\\ =\lim\limits_{h\to 0} h\sin (1/h).$
Also, note that $\forall h>0, -h\leq h\sin(1/h)\leq h$ and $\forall h\leq 0,h \leq h\sin (1/h)\leq -h.$ Hence by the Squeeze Theorem, $\lim\limits_{h\to 0}h\sin (1/h)=\lim\limits_{h\to 0}h = 0.$ In order for $f''(0)$ to exist, we must have that $f'(x)$ is differentiable at $x=0.$ However, we will show that $f'(x)$ is discontinuous at $x=0$ and hence not differentiable there. We will do so by showing that $\lim\limits_{x\to 0^-}f'(x)$ does not exist. Consider the sequence $(x_n)_{n=1}^\infty$ such that $x_n = -\dfrac{1}{\frac\pi2 + 2n\pi}$ and the sequence $(y_n)_{n=1}^\infty$ such that $y_n=-\dfrac{1}{\frac{3\pi}{2}+2n\pi}.$ $\lim\limits_{x\to 0^-}f'(x)$ does not exist because $x_n, y_n\to 0$ as $n\to \infty\Rightarrow \forall \epsilon>0, \exists N (n\geq N \Rightarrow x_n,y_n \in (-\epsilon,0)).$ Since $f'(x_n)<0<f'(y_n)\;\forall n,$ we have that $f''(0)$ does not exist.
However, we have that
$$\lim\limits_{h\to 0}\dfrac{f(0+h)+f(0-h)-2f(0)}{h^2}=\lim\limits_{h\to 0}\dfrac{h^2\sin (1/h)-h^2\sin(1/h)}{h^2}\\ =0.$$ Thus, the limit exists at $x=0$ but the second derivative does not.
edit for the first part (i should've used the taylor series instead).
We have that $f(a+h) = f(a) + f'(a)h+f''(a)\dfrac{h^2}{2}+\dots$ and $f(a-h)=f(a)-f'(a)h+f''(a)\dfrac{h^2}{2}+\dots.$ Hence $f(a+h)+f(a-h)-2f(a)=h^2f''(a)$ and the desired limit is $\lim\limits_{h\to 0} \dfrac{h^2f''(a)}{h^2}=f''(a),$ as desired.