A factorization has to have finite exponents but there can be infinitely many factorizations. So $3 = 1^{489} \times 3 = 1^{490} \times 3 = 1^{491} \times 3 = \ldots$
As you already know, $(-1)^k = -1$ if $k$ is odd and 1 if $k$ is even. So we have $-3 = (-1)^{489} \times 3 = (-1)^{491} \times 3 = (-1)^{493} \times 3 = \ldots$
And what about if in $\mathbb{Z}[i]$ we regard $i$ as prime? Or if in $\mathbb{Z}[\sqrt{2}]$ we regard all numbers of the form $(1 - \sqrt{2})^k$ as primes? Obviously $(1 - \sqrt{2})^k 3$ grows larger as $k$ grows, but there's something about 3 that doesn't actually change: tell me, what is $(9 - 6\sqrt{2})(9 - 6\sqrt{2})$? What about $(21 - 15\sqrt{2})(21 + 15\sqrt{2})$? $(51 - 36\sqrt{2})(51 + 36\sqrt{2})$?
But we don't have to get into algebraic number theory to understand that 1 differs from the prime numbers in far more important ways than a couple of asterisks on the fundamental theorem of arithmetic.