Well we know that $\mathcal{O}_K$ is isomorphic to $\Bbb Z[x]/(p_\alpha(x))$ with $p_\alpha(x)$ the $\Bbb Z$-minimal polynomial for $\alpha$.
Then the number of primes over $p$ is determined by the splitting behavior of $p_{\alpha}(x)\mod p$. Since we see that if we have a prime $\mathfrak{p}|(p)$, then it comes from a factorization into (not necessarily unique) irreducibles
$$\overline{p}_\alpha(x)=\prod_{i=1}^r p_i(x)$$
via $\mathfrak{p}=(p_i(\alpha),p)$, since $\Bbb Z[\alpha]$ has trivial conductor in $\mathcal{O}_K$ (this is the critical fact).
But then your original result is false. Consider $K=\Bbb Q(\zeta_n)$ for a primitive $n^{th}$ root of unity, $\zeta_n$ then $\mathcal{O}_K=\Bbb Z[\zeta_n]$ by classical theory. We know that if we pick $n=p^{k}-1$ then consider the $n^{th}$ cyclotomic polynomial $\Phi_n(x)$, we know it will factor as the product of all irreducible polynomials of degree $k$ modulo $p$ since $\Bbb F_{p^k}$ is the unique finite field of order $p^k$ and consists of $0$ and the $(p^k-1)^{st}$ roots of $1$. So then if we can demonstrate more than $p$ such irreducible factors, we have disproven the statement. But we can count the number of irreducible polynomials of degree $q$, a prime not equal to $p$, namely ${p^q-p\over q}$ (I omit the proof, since it's not the subject of this post, but there are many available online, including here on MSE) so choose $p=2, q=5$ and we get $6$ distinct irreducibles modulo $p$, hence $6$ primes above $2$ in $\Bbb Z[\zeta_{p^q-1}]$, which is more than $2$.