...and if it's important, do those ideas have any generalization to more "exotic" number systems?
The motivation for my question comes from reading some of the excellent answers posted to other questions, such as a recent one asking whether $\sqrt{1 + 24n}$ always yields primes. In particular, I've been struck by commenter Bill Dubuque's repeated use of what he terms "modular reduction", casting a problem in $\mathbb{Z}_n$ to make it much easier to solve in the general case for $\mathbb{Z}$.
What I don't quite grok is why this works; why can we do this? Is there a deep reason? At first glance, to me, there doesn't seem anything inherent in the axioms of a given $\mathbb{Z}_k$ that necessarily ties it intimately with $\mathbb{Z}$; all we care is that is has $k$ elements and it is closed under the binary operator of addition. It doesn't seem to encode information about $\mathbb{Z}$'s other elements.
Now, in saying that, I'm not sure I'm on the right foot here at all, so I'll analogize to something I know a little better: One sees, in some textbook definitions, an identification of $\mathbb{C}$ with an ordered pair $(a,b)$, $(c,d)$ with rules for addition and multiplication that go to $(a + c, b + d)$ and $(ac - bd, ad + bc)$, respectively, with no immediate hint about the importance of $\mathbb{C}$ in that is the algebraic closure of $\mathbb{R}$, which is a highly nontrivial theorem that needs to be proved through the FTA. Is there a similar relationship between $\mathbb{Z}_n$ and $\mathbb{Z}$, and is that extensible to other systems?
I'd appreciate any answers and references tailored to someone who's taken up to the middling undergraduate math level. (e.g. Linear Algebra, elementary abstract Algebra, undergraduate Complex Analysis...)