If it fails then there exists a $\rm\color{#c00}{minimal}$ counterexample $\,n = ab\,$ with $\,p\mid ab,\ p\nmid a,b.\ $
$\!\begin{align} \text{Necessarily $b< p^{\phantom{|}}\!$ else} &\text{ $\,p\mid a(b\bmod p) = a(b\!-\!jp)\,$ is a $\rm\color{#c00}{smaller}$ counterexample,}\\[.3em]
\text{therefore, we deduce}\, &\text{ $\,p\mid a(\color{#0a0}{p\bmod b}) = a(p\!-\!kb)\,$ is a $\rm\color{#c00}{smaller}$ counterexample $\color{#c00}{\Rightarrow\!\Leftarrow}$}\end{align}$
Remark $ $ We used that $\,b' = \color{#0a0}{p\bmod b}\neq 0\,$ (by $p$ prime, $\,b>1)$. If $\,p\,$ were composite then this step may fail this way by reaching such a nontrivial factor $\,b'\,$ of $p.\,$. Similarly, in the first descent step we have employed that $\ b\bmod p\neq 0,\,$ since $\,p\nmid b\:$ by hypothesis.
The integers $\:\!n\:\!$ with $\,p\mid an\,$ are closed under subtraction, and contain $\,b,p,\,$ so also $\,\gcd(b,p).\ $ The descent in the above proof corresponds to using the Euclidean algorithm to compute the gcd (it uses a special form of the algorithm for prime argument, as explained below).
Essentially the same descent was employed by Gauss in Disq. Arith when he gave the first complete and rigorous proof of the Fundamental Theorem of Arithmetic, i.e. existence and uniqueness of factorizations into irreducibles (primes). As explained in that linked post, when interpreted constructively this yields an algorithm for computing inverses $\!\bmod p.\, $ See also below, from a deleted thread.
To compute inverses modulo a prime $\,p,\,$ rather than use the full-blown extended Euclidean algorithm, it is conceptually simpler to use a special case that goes back to Gauss (Disq. Arith. Art.5, 1801). The idea is so simple that it can be taught to bright grade school students. Namely, to compute the fraction $\ 1/b\pmod{\! p}\ $ we repeatedly scale the numerator and denominator in a way that reduces the value of the denominator $\!\bmod p,\,$ i.e. scale $\, a/b\to an/bn\ $ by the least $\,n\,$ such that $\,bn > p\,.\,$ For example, modulo $29$
$$ \frac{1}8\ \to\ \frac{4}{32}\ \equiv\ \frac{4}{3}\ \to\ \frac{40}{30}\ \equiv\ \frac{11}{1}\pmod{\!29}$$
Because each step reduces the denominator, eventually we must reach a denominator of $1$ (not $0$ else we would have $\,bn = p\,$ so $\,b>1\,$ would be a proper factor of the prime $\,p\,$). The algorithm has a vivid interpretation in terms of "clock arithmetic" on a clock face of $\,p\,$ minutes. Namely the reduction step is simply: starting at $\,0,\,$ take $\,n\,$ steps of size $\,b\,$ until you cycle, passing $0\,$ and landing in the interval $[0,b-1].\,$ You can't land on $0\,$ else $ b$ would properly divide $\,p.\,$ Now do the same to this smaller multiple of $\,b.\,$ Each loop around the clock face produces a smaller multiple of $\,b,\,$ so eventually it reaches the multiple $1$. This graphical formulation goes over quite well with bright grade-school students (esp. when enhanced by visual aids, e.g. periodical curves such as Roulettes (Spirograph curves), Star Polygons - see here).
From this one quickly infers that irreducible integers $\,p\,$ are prime, i.e. $\,p\ |\ a\,b\ \Rightarrow\ p\ |\ a\ \ {\rm or}\ \ p\ |\ b\,.\,$ This is essentially the way that Gauss constructed the first known proof of the Fundamental Theorem of Arithmetic, i.e. unique factorization of integers.
Note that this iterates $\,(b,\,p)\to (p\bmod b,\,p)\,$ vs. $\,(b,\,p)\to (p\bmod b,\, b)\,$ in the Euclidean algorithm. It may fail for composite moduli.
As for uniqueness of inverses $\,a,\,a'\,$ of $\,b,\ $ hint: $\ a\ =\ a\,(b\,a') \ =\ (a\,b)\,a'\, =\ a'$
