If $\rm\,n = p\,$ is prime, every $\rm\:0\neq a\in\Bbb Z_p\:$ is invertible, since, by little Fermat $\rm\:1 = a^{p-1} = a (a^{p-2}).$
If $\rm\:n = ab\:$ is composite then $\rm\:a\:$ is a zero-divisor in $\rm\,\Bbb Z_n.\,$ But zero-divisors cannot be invertible:
$$\rm\begin{eqnarray} 1&=&\rm\ \color{#0A0}{\bar a\,a},\ \ \ 0\ =\,&\rm\color{#C00}{ab},&\,\rm a,b\ne 0 \\[.2em]
\rm \Rightarrow\ \ b &=&\rm (\color{#0A0}{\bar a\,a})b = \bar a&\!\!\!\rm (\color{#C00}{ab})&\rm = 0\ \ \Rightarrow\!\Leftarrow\end{eqnarray}\qquad$$
That zero-divisors are not invertible is a ring-theoretic generalization of the well-known fact that zero cannot be invertible, i.e. we cannot divide by zero in a nontrivial ring (i.e. the quotient does not uniquely exist)