I guess the point here is that you want your notion of proof to be effective. Therefore, proofs should be finite objects. Moreover, given an arbitrary finite object, it should be effectively decidable whether it constitutes a proof or not.
The second condition rules out axiom sets which are too complicated. As an extreme example, assume one sets up a calculus and declares that all theorems of first order logic are taken as axioms. Since first-order logic is undecidable, you have no computable way of telling whether an arbitrary formula is an axiom of the system or not. Clearly, this would not be a satisfying situation.
Finite axiom sets are certainly never too complicated in this sense. But there is also nothing wrong with allowing infinite sets of axioms, assuming that the set of axioms is computationally simple. As Noah pointed out, there are well-established first-order theories with infinitely many axioms.
A minimalistic requirement for a notion of proof from infinitely many axioms is then that it is computationally easier to decide what an axioms (or more generally, a proof) is than to decide which formulas are theorems. Otherwise, of what point is a proof? You might as well just list all theorems as axioms. In the case of PA and ZFC this condition is strongly satisfied: The infinite axiom sets are decidable in linear time (or at a glance, as Doug said in his comment), whereas the resulting set of theorems is undecidable (unless it is inconsistent).