This is very close to a duplicate of various questions, but I can't find an exact dupe at the moment.
Ultimately we are not certain of this. Indeed, although it's very rare there have been quite excellent mathematicians who suspected that $\mathsf{ZFC}$ is inconsistent after all (such as the logician Jack Silver). And this is all leaving aside the second incompleteness theorem. Personally, while I am as certain of the consistency of first-order Peano arithmetic as I am of the fact that I have two hands, I would merely be profoundly disturbed to learn that $\mathsf{ZFC}$ is inconsistent.
Fortunately, $\mathsf{ZFC}$ is so ridiculously overpowered that its inconsistency wouldn't really spill over to the rest of mathematics too much. For the vast majority of mathematical practice, galactically weaker theories like $\mathsf{ZC}$ are enough.
Incidentally, the question of exactly how much "axiomatic overhead" is needed for various parts of mathematics is studied rigorously in Reverse Mathematics. It turns out that a huge amount of mathematics can be developed in $\Pi^1_1$-$\mathsf{CA_0}$, which is a tiny fragment of $\mathsf{Z_2}$ which is itself a tiny fragment of $\mathsf{ZC}$. The primary difficulty with RM's approach is the restricted language, which makes lots of "higher-type" mathematics (e.g. measure theory, topology, ...) hard or impossible to treat faithfully, but there has been some recent work in improving this situation (see e.g. here).