You have some good answers already but here is a less formal and maybe more intuitive style.
As everyone says: an important point about limits is that we do not need to evaluate the expression at the limit ($h = 0$ in your case). It is not necessary that the expression is even defined at the limit point. If it is then we still don't use the value at that point.
You can think of the usual $\epsilon - \delta$ definition of limits as a game.
I claim that a function $f(x)$ has the limit $y$ at $x = a$. You can now challenge me by picking positive values of $\epsilon$ and I need to answer with a positive value of $\delta$ which gets the function within $\epsilon$ of my claimed limit. If I can always manage this then I have won the game: the function has the claimed limit. If you defeat me, you find an $\epsilon$ and I cannot answer with a suitable $\delta$ then you have won: the function does not have the claimed limit or maybe has no limit.
Your example is simple: I will pick my answer for $\delta$ as $\frac{1}{3}$ of your challenge value of $\epsilon$.
The value of $\delta$ does not need to be the best in any sense (e.g. only just good enough), it just needs to be good enough. So, I might answer $1$ anytime that your challenge value is $3$ or more. If you say $1000000$ then I do not need to bother calculating $1000000 / 3$.
In the case when the function is defined at $x = a$ you can compare the limit to $f(a)$. If the limit exists and equals $f(a)$ then we say that the function is continuous at $a$. Note that this is an optional extra step in the case that the function is defined at $a$, you can calculate the limit even if it is not.
You will also see limits which appear to involve $\infty$. These are just suggestive notations and the formal definitions do not involve $\infty$ in the same way that a limit at a (normal finite) value does not need you to use the value of the function at that point.
Limits are most often studied with the real and complex numbers but it is possible to study them elsewhere if you have a way to define closeness. You can use the same definition with the rational numbers or even the integers. With the rationals, the limit may fail to exist even it looks as if it should. With the integers, limits are even more strange. Ask additional questions if you want to know more.