The limit of the difference quotient has to exist for approaches from arbitrary directions (in $\mathbb{R}$ the limit approaching from above as well as from below). For boundary points in closed sets, this fails for at least one direction.
Take $f: [0,\infty) \rightarrow [0,\infty),\,f(x) = x$. One would immediately say that $f$ is differentiable on $[0,\infty)$ and $f'(x) = 1 \hspace{3mm} \forall x \in [0,\infty)$. But, in fact, if you extend $f$ to
$$f(x) =\left\{\begin{array}{lll} x & x > 0 \\ 0 & x=0 \\ -x & x < 0 \end{array}\right.$$
you get f(x) = |x| which is not differentiable at $x=0$.