Writing explicitely $\frac{p!}{k!(p-k)!}$ there is a unique multiple of $p$ at the numerator. But this is an integer, hence it is a multiple of $p$.
Your argument cannot be false, meaning there is no overlap. I will try to be more explicit:
$$\binom{p}{k}=\frac{p\cdot (p-1) \cdots (p-k+1)}{k \cdot (k-1) \cdots 1}$$
We all have already agreed that, if this is an integer, then this is a multiple of $p$. The question can be then reformulated: why $\binom{p}{k}$ has to be an integer?
Motivation 1 (combinatorial interpretation, at first it may look like a cheating answer, but this is not): $\binom{p}{k}$ is the number of ways to choose $k$ objects into $p$ identical objects.
Motivation 2: it would be enough to prove that if $q^\alpha$ divides exactly the denominator (for some prime $q \le k < p$ then $q^\alpha$ divides also the numerator. How to show it? Algebraically, the number factors $q$ which divides $n!$ is exactly $$\sum_{i \ge 1}\left\lfloor \frac{n}{q^i}\right\rfloor.$$
This is commonly known as Polignac's formula. Hence, we have to show equivalently that for each prime $q<p$ we have
$$\sum_{i \ge 1}\left\lfloor\frac{p}{q^i}\right\rfloor \ge \sum_{i \ge 1}\left\lfloor \frac{p-k}{q^i}\right\rfloor+\sum_{i \ge 1}\left\lfloor \frac{k}{q^i}\right\rfloor.$$
But this is immediate, since for each positive $x,y$ we have $\lfloor x+y\rfloor \ge \lfloor x\rfloor +\lfloor y\rfloor$. The claim follows summing over $i$.
More generally, this argument can be modified for all strongly divisible sequence, i.e. sequences $(a_n)_{n \in \mathbf{N}}$ such that $\text{gcd}(a_n,a_m)=a_{\text{gcd}(n,m)}$. [For instance, it works for Fibonacci numbers].