As a formalist, I echo the comment of Neal that says "axioms are, by definition, stipulated". The real question is whether the structure so axiomatized is interesting, or worth study.
As many mathematicians make arguments that involve a set of natural numbers, it follows that a good universe of sets should include a set of natural numbers. Whether its existence is an axiom or a theorem is not really important; that's merely matter of exposition. Exposition is important of course, but I wanted to emphasize that's all it is.
Now, I also like to observe the close analogy between logic and set theory. IMO, that's really the main reason why set theory has attained the importance it has: it's really just a systematization of the things we like to do with logic.
For example, if I decide to play at being a finitist and consider finite set theory, I can still meaningfully say the word "natural number", and consider predicates like "$x$ is a natural number". This lets me say things about the class of natural numbers, consider class functions on the natural numbers like $f(x) = x+1$, consider predicates on the class of natural numbers like "E(x) := $x$ is even", quantify predicates over the natural numbers such as $\forall x \in \mathbb{N} : E(x) \leftrightarrow \neg E(x+1)$, and so forth.
And because I can consider predicates like E(x), this means I can also talk about the class of even natural numbers.
If I can consider one variable denoting a natural number, I can consider two variables denoting natural numbers: I can talk about the class $\mathbb{N} \times \mathbb{N}$.
If one is of the sort to focus on such things, one can talk about mechanically translating all such statements into equivalent statements in an untyped language of finite set theory, and keep thinking in the back of one's mind that one is actually working with these untyped statements rather than the more suggestive ones the notation indicates.
While such a translation is possible, and it's a good thing to know, I think the way of thinking is unjustified -- the fact that translation is possible means that you should have no qualms about thinking in terms of the new and suggestive ideas rather than handicapping yourself with a restrictive thought process.
So really, when I'm playing at being a finitist, I still have access to a limited amount of set theory that includes a set of natural numbers. NBG set theory codifies this into having "sets" and "classes". NBG with anti-infinity has two sorts of objects: sets, and classes. The natural numbers would be a (proper) class. I assume that in the presence of anti-infinity, NBG is still 'equivalent' to ZFC.
If I further allow myself to consider second-order logic, I can do more. My objects now include first-order predicates. I can consider the second order predicate "$\varphi(P) := \forall x: P(x) \implies x \in \mathbb{N}$". In other words, I can now talk about subsets of the natural numbers.
If I don't stop there and go all the way up to higher order logic, then I can do this for every type. In the logic-set theory analogy, I now consider power sets. I assert that this means that all of the types I can talk about are organized into a Boolean topos with natural number object -- in other words, that the higher order logic of finite set theory is an instance of the first-order logic of bounded Zermelo set theory (with infinity). Bounded means that I'm never allowed to just say $\forall x: \cdots$ instead, $x$ must be bound to some set/type, as in $\forall x \in T: \cdots$.
Because of all of these things, I am thoroughly unpersuaded when a finitist rejects the notion of a set of natural numbers. I could understand being restrictive in what sorts of things you can do with said set, but rejecting it outright is, IMO, a silly notion which I ascribe more to simply having a contrary attitude than substantive content.