An elementary argument, without any explicit reference to Galois theory:
Suppose the biquadratic extension is $K = \mathbb{Q}(\sqrt{B}, \sqrt{C})$, where $B$ and $C$ are integers (and none of $B$, $C$, $BC$ a rational square). Then any element $\alpha$ of $K$ has the form
$\alpha = a + b\sqrt{B} + c\sqrt{C} + d \sqrt{BC}$
for some $a, b, c, d \in \mathbb{Q}$. This $\alpha$ satisfies the polynomial
$$\begin{align*}f(X) =& (X - a - b\sqrt{B} - c\sqrt{C} - d\sqrt{BC})\cdot(X - a + b\sqrt{B} + c\sqrt{C} - d\sqrt{BC})\\
&\qquad{}\cdot (X - a + b\sqrt{B} - c\sqrt{C} + d\sqrt{BC})\cdot(X - a - b\sqrt{B} + c\sqrt{C} + d\sqrt{BC}).\end{align*}$$
Convince yourself that $f(X)$ is in $\mathbb{Q}[X]$. (This is where a bit of Galois theory helps.) If $\alpha$ is integral over $\mathbb Q$, then in fact $f(X) \in \mathbb{Z}[X]$. If $\alpha$ generates $K$ over $\mathbb{Q}$, then $f(X)$ is irreducible and is the minimal polynomial of $\alpha$ over $\mathbb{Q}$. We want to show that if $\alpha$ is integral then $f(X)$ will always factor in $\mathbb{F}_p[X]$.
The key observation is that at least one of $B$, $C$, and $BC$ will be a square modulo $p$. Convince yourself that this is true: the product of two nonsquares modulo $p$ is a square modulo $p$. (One way to do this is note that a nonsquare has to be an odd power of a generator of $\mathbb{F}_p^\times$.)
So let's suppose that $BC \equiv S^2 \pmod{p}$. (The argument will work essentially the same way if we assume that $B$ or $C$ is a square modulo $p$ instead; convince yourself of this at the end.)
So we have
$\begin{align*}f(X) =& \big((X - a - d\sqrt{BC}) - (b\sqrt{B} + c\sqrt{C})\big)\big((X - a - d\sqrt{BC}) + (b\sqrt{B} + c\sqrt{C})\big)\\
&\qquad {} \cdot \big((X - a + d\sqrt{BC}) + (b\sqrt{B} - c\sqrt{C})\big)\big((X - a + d\sqrt{BC}) - (b\sqrt{B} - c\sqrt{C})\big)\\ =& \big((X - a - d\sqrt{BC})^2 - (b\sqrt{B} + c\sqrt{C})^2\big)\\ &\qquad {} \cdot \big((X - a + d\sqrt{BC})^2 - (b\sqrt{B} - c\sqrt{C})^2\big)\\ =& \big((X - a)^2 + d^2BC - 2d(X-a)\sqrt{BC} - b^2B - c^2C - 2bc \sqrt{BC}\big)\\ &\qquad {} \cdot \big((X - a)^2 + d^2BC + 2d(X-a)\sqrt{BC} - b^2B - c^2C + 2bc \sqrt{BC}\big)\\ \equiv& \big((X - a)^2 + d^2BC - 2d(X-a)S - b^2B - c^2C - 2bc S\big)\\ &\qquad {} \cdot \big((X - a)^2 + d^2BC + 2d(X-a)S - b^2B - c^2C + 2bcS\big) \pmod{p},\end{align*}$
which is a product of two quadratics in $\mathbb{F}_p[X]$.
In the case where all three of $B$, $C$, $BC$ are squares modulo $p$, the quadratic factors of $f(X)$ will split further into linear factors in $\mathbb{F}_p[X]$.
As to your other question -- or maybe it was just a typo, but in any case -- no, it's not true that every irreducible monic quartic polynomial in $\mathbb{Z}[X]$ factors over every $\mathbb{F}_p$. For example, $g(X) = X^4 + X + 1$ is irreducible in $\mathbb{F}_2[X]$. To see this, first plug $X= 0$ and $X =1$ into $g(X)$ to see that there are no mod $2$ roots. To see that $g(X)$ doesn't factor into a product of two irreducible quadratics, note that $X^2 + X + 1$ is the only monic irreducible quadratic of $\mathbb{F}_2$ (write them all down and check for roots), and $(X^2 + X + 1)^2 = X^4 + 2X^3 + 3X^2 + 2X + 1 \equiv X^4 + X^2 + 1 \not\equiv g(X)$. Since $g(X)$ is not divisible by any irreducible linears or quadratics, it has to be irreducible.
In fact, you can show that every $\mathbb{F}_p[X]$ has irreducible polynomials of every degree.
