Without $\aleph_1$ you will miss the internal construction of the Borel sets. This means that you will have a much harder time to prove that there are only continuum Borel sets; and you will not be able to construct a hierarchy of Borel functions which itself has similar structure which allows us to prove theorems about it.
You will also miss the construction of an Aronszajn tree, and Suslin trees which means that you will not be able to ask whether or not a dense linear order without endpoints which is c.c.c. and order complete is necessarily $\Bbb R$ or not.
You will have troubles understanding what are uncountable Whitehead groups, or use other theorems which are seemingly not about set theory, but their proofs employ deep set theoretical instruments.
Similarly you will not be able to prove Borel determinacy (which requires $\beth_{\omega_1}$ to exist), which again brings us to natural questions on Borel sets.
Another thing you might miss the the ability to compute the global dimension of a countable product of fields, since the value seems to have intrinsic connection to the value of the continuum as an $\aleph$ number. Similarly you won't be able to talk about outer automorphisms of the Calkin algebra, since their existence may also be depending on whether or not $2^{\aleph_0}=\aleph_1$ (meaning that we make use of an object of size $\aleph_1$ in the proofs).
There are many other surprising connections that pop up once you start talking about uncountable objects, and more specifically uncountably generated objects. But it is true, that $\aleph_1$ is not something that meet every day in "usual mathematics". And that's why people would be surprised to see it popping up here and there.
The question is what exactly do you want to do when you say "mathematics". If you just want to talk about classical analysis, sure you can probably go your whole life without seeing $\aleph_1$ being used explicitly. You can probably go most of your life without hearing the term "a well-ordering of the continuum" either. But both terms can appear a lot if you are willing to advance beyond 19th century and early 20th century mathematics. Since we have started talking about arbitrarily large objects (even when we are interested in smaller ones), we have involved set theoretical assumptions about the structure of sets, and more importantly their power sets, into the discussion.
Pocket set theory is very cute, and very nice. And sure it can probably talk about basic analysis and second-order number theory. But modern mathematics is a lot more, even if you don't peak behind the curtain to see where uncountable cardinals get into the picture.
Some links to read when you're bored:
- Can we distinguish $\aleph_0$ from $\aleph_1$ in Nature?
- Proofs given in undergrad degree that need Continuum hypothesis?
- Examples of sets whose cardinalities are $\aleph_{n}$, or any large cardinal. (not assuming GCH)
- What happens after the cardinality $\mathfrak{c}$?