Recall that the definition of the derivative is:
$$f'(x) = \lim_{\Delta x \rightarrow 0} \frac{f(x + \Delta x) - f(x)}{\Delta x}$$
And the second derivative is simply the derivative of the first.
$$f''(x) = \lim_{\Delta x \rightarrow 0} \frac{f'(x + \Delta x) - f'(x)}{\Delta x}$$
Plugging in the difference quotient for $f'$ itself, we get:
$$f''(x) = \lim_{\Delta x \rightarrow 0} \frac{\frac{f(x + 2\Delta x) - f(x + \Delta x)}{\Delta x} - \frac{f(x + \Delta x) - f(x)}{\Delta x}}{\Delta x}$$
or, simplifying a little,
$$f''(x) = \lim_{\Delta x \rightarrow 0} \frac{f(x + 2\Delta x) - 2f(x + \Delta x) + f(x)}{({\Delta x})^2}$$
The $dx^2$ in the denonimator directly corresponds to the $({\Delta x})^2$ in the denominator of this definition of $f''$: It actually is the square of something.
But the numerator is not $({\Delta y})^2$, which would be $(f(x + \Delta x) - f(x))^2 = f(x + \Delta x)^2 - 2 f(x) f(x + \Delta x) + f(x)^2$. So we write $d^2y$ instead of $dx^2$ as an indication that we're performing a difference operation twice, rather than squaring a difference like we do with the denominator.