It is well-known that in a non-cyclic Galois extension $K/\mathbf{Q}$ no prime $p$ is inert.
Say $\zeta$ is a primitive $n$-th root of unity such that $(\mathbf{Z}/n\mathbf{Z})^\times$ is non-cyclic and set $K=\mathbf{Q}(\zeta)$. The minimal polynomial of $\zeta$ is the $n$-th cyclotomic polynomial $\Phi_n$. By the Kummer-Dedekind Theorem, the factorisation of $\Phi_n$ modulo $p$ determines the decomposition of $p$ in $\mathbf{Z}[\zeta]=\mathcal{O}_K$.
Now say we have a $(\mathbf{Z}/2\mathbf{Z})^2$-extension of $\mathbf{Q}$. These are of the form $K=\mathbf{Q}(\sqrt{n},\sqrt{m})$ with $n,m\in \mathbf{Z}$ such that $n,m,nm\not\in \mathbf{Q}^{\times 2}$. A primitive element is $\sqrt{n}+\sqrt{m}$, with minimal polynomial $f=x^4+(2(n-m)-4n)x^2+(n-m)^2$. I want to now conclude that such polynomials are always reducible modulo $p$.
For instance, when $n=-1$ and $m=2$, we get $f=X^4-2X^2+9$. The only prime factors of its discriminant are $2$ and $3$. Hence, the decomposition of $p$ in $K$ is the factorisation of $f$ mod $p$ by the Kummer-Dedekind Theorem for all $p\neq 2,3$. Hence, we know by the above that $f$ is reducible mod $p$ for all $p\neq 2,3$. We can treat the cases $2$ and $3$ separately, these are easy.
But in general, Qiaochu Yuan seems to use that the minimal polynomial of a primitive element (chosen integral, of course) of a non-cyclic Galois extension is reducible mod $p$ for all $p$ here.
In the cyclotomic case, the ring of integers is monogenic and generated by a primitive element of $K$. However, in general, this need not be the case. How can we then connect the decomposition of $p$ in $\mathcal{O}_K$ to the factorisation of the minimal polynomial $f^{\alpha}_{\mathbf{Q}}$ of a primitive element $\alpha$ of $K$? Since, a priori, the factorisation of $f^{\alpha}_{\mathbf{Q}}$ modulo $p$ only gives the decomposition of $p$ in $\mathcal{O}_K$ when $p$ does not divide the index $[\mathcal{O}_K:\mathbf{Z}[\alpha]]$.
There must be something I am not seeing or forgot. Any help is welcome. :)