The notions of "integer" and "prime" are simply too good to be restricted to the familiar context of $\mathbb Z$ - the integers with which we are familiar. It turns out that there are natural generalisations of these ideas to wider contexts. What you see here is just the beginning of a whole new world for mathematicians to explore - a world in which we can study, for example, the solutions of polynomial equations, or treat the graphs (solution sets) of polynomial equations as geometric objects.
So if we have a linear polynomial with integer coefficients $p(x)=ax+b: a\neq 0$ then the (rational) root $x=-\frac ba$ is an integer precisely when $a=\pm 1$ (or we might say that $a$ is a unit). If $a=-1$ we can change all the signs in $p(x)$ and get a "monic" polynomial with the same roots (ie leading coefficient is 1). The generalisation which works as we expand to new contexts (by adding the roots of polynomial equations like $p(x)=x^2+1$ to the numbers we already have) is to say that an algebraic integer is the root of a monic polynomial with integer coefficients (allowing algebraic integer coefficients does not affect what is counted as an integer).
Then we say that a (non-unit, non-zero) algebraic integer is prime precisely when $p|ab$ ($p, a, b$ algebraic integers) implies that either $p|a$ or $p|b$ (or both). This is different from being irreducible: an example of how extending the context distinguishes between ideas which seem naturally to belong together. In fact the notion of "prime" can usefully be extended beyond this and following Kummer we find the notion of a prime ideal - an ideal is not a single number, but a set of numbers, and with a prime ideal the whole set behaves collectively like a prime number. The concept of an ideal was developed in the search for a proof for Fermat's Last Theorem, and in particular to deal with some unexpected technicalities, like the failure of unique factorisation when numbers are extended. The Gaussian Integers do still have unique factorisation, but if we were to add $\sqrt {-5}$ to the mix, for example, unique factorisation would fail. Early attempts at FLT could easily miss the fact that unique factorisation was being assumed.
Gaussian Primes help to answer the question "which integers can be expressed as the sum of two integer squares?" and also the number of such representations, where they exist.