If you expand $(a + bp)^{p-1}$ using the binomial theorem, you get
$$
a^{p-1} + \binom{p-1}{1} a^{p-2} b p + \binom{p-1}{2} a^{p-3}b^2p^2 + \dots
$$
and actually on the first two terms here matter modulo $p^2$: the others have a factor of $p^2$ in them. So
$$
(a + bp)^{p-1} \equiv a^{p-1} + a^{p-2} b p(p-1) \pmod {p^2}.
$$
Note that $p(p-1) = p^2-p \equiv -p \pmod p^2$, so we can simplify this further into
$$
a^{p-2}(a - bp) \pmod {p^2}.
$$
There is only one residue modulo $p^2$ that's the inverse of $a^{p-2}$, and $a-bp$ can take on $p$ different values modulo $p^2$, so there are definitely lots of choices of $b$ that work.
More interesting is the existence of a (unique!) $b$ such that $(a+bp)^{p-1} \equiv 1 \pmod{p^2}$.
To see this from where we left off: first note that by Euler's theorem, $a^{p^2-p} \equiv 1 \pmod{p^2}$, so we have $a^{p-2}(a - bp) \equiv 1 \pmod{p^2}$ if and only if $a - bp \equiv a^{p^2-2p+2} \pmod{p^2}$.
By Fermat's little theorem, $a^{p-1} \equiv 1 \pmod{p}$, so $a^{p^2-2p+2} \equiv a^{(p-1)^2+1} \equiv a \pmod{p}$. That means $a^{p^2-2p+2} \equiv a + cp \pmod{p^2}$ for some $c$.
Choose $b = -c$, and we'll have what we want.