Several introductory set theory books (Hrbacek & Jech, Enderton, Pinter) use very similar reasoning to develop the natural numbers. The argument used in Hrbacek & Jech (minus some supplementary explanatory comments) is pretty much word-for-word as follows:
Definition: The successor of a set $x$ is the set $S(x) = x \cup \{x\}$
We use the more suggestive notation $n + 1$ for $S(n)$ in what follows
Definition: A set $I$ is called inductive if
(a) $\emptyset \in I$
(b) If $n \in I$, then $(n + 1) \in I$
The Axiom of Infinity: An inductive set exists.
Definition: The set of all natural numbers is the set
$\mathbf{N} = \{x \mid x \in I$ for every inductive set $I\}$.
The issue I see with this is in the last step defining the set $\mathbf{N}$ of natural numbers (which is presumably supposed to yield an $\mathbf{N}$ equal to the set of all finite von Neumann ordinals): The Axiom of Infinity guarantees the existence of an inductive set (let's call it $I_0$). But the Axiom of Infinity does not guarantee that $I_0$ doesn't contain some other elements besides ordinals, nor does it (by itself) guarantee that any other inductive set besides $I_0$ exists. If $I_0$ is the only inductive set and it contains elements that are not ordinals, then "the set of elements belonging to every inductive set" is just $I_0$ itself, which is NOT what you want (given that $I_0$ contains non-ordinals).
In order for the above-quoted development of the natural numbers to work, it seems like you need another step in the argument to get from the existence of an inductive set to the existence of other inductive sets, so that their intersection will be guaranteed to yield just the finite ordinals. Presumably the Powerset Axiom could do this for you? Perhaps the use of the Powerset Axiom is assumed but not explicitly stated? But I would think that's not a step you should skip in an *introductory* text.
So, bottom-line question: Is this development of the natural numbers indeed incomplete, or am I just missing something here?