The map $\mathbb{Z}\to\mathbb{F}_p$ given by reduction modulo $p$ induces a homomorphism $\mathbb{Z}[x]\to\mathbb{F}_p[x]$ (reducing coefficients modulo $p$); this is the universal property of the polynomial ring.
Now consider the composite map
$$\mathbb{Z}[x] \to \mathbb{F}_p[x] \to \frac{\mathbb{F}_p[x]}{\langle \overline{f_{\alpha}(x)}\rangle}.$$
The map is onto, because the induced map $\mathbb{Z}[x]\to\mathbb{F}_p[x]$ is onto, and the canonical projection is onto. By the isomorphism theorems, you know that $\mathbb{F}_p[x]/\langle \overline{f_{\alpha}(x)}\rangle$ is isomorphic to the quotient of $\mathbb{Z}[x]$ modulo the kernel of the map.
It should be clear that the kernel contains both $p$ and $f_{\alpha}(x)$. So you just need to show that this is the entire kernel. If $g(x)$ lies in the kernel, then $g(x)\equiv f_{\alpha}(x)\pmod {p}$; this tells you that you can express $g(x)$ as a multiple of $f_{\alpha}(x)$ up to multiples of $p$ in the coefficients, which gives the equality you want.
For the other isomorphism, consider the map $\mathbb{Z}[x]\to\mathbb{Z}[\alpha]$ obtained by "evaluation at $\alpha$". The kernel of this map is precisely the ideal generated by the minimal polynomial of $\alpha$, so
$$\mathbb{Z}[\alpha]\cong\frac{\mathbb{Z}[x]}{\langle f_{\alpha}(x)\rangle}.$$
Under this isomorphism, $p + \langle f_{\alpha}(x)\rangle$ maps to $p$ in $\mathbb{Z}[\alpha]$, so the isomorphism theorem tells you that the ideal corresponding to $(p+\langle f_{\alpha}(x)\rangle)$ in the quotient corresponds to the ideal of $\mathbb{Z}[x]$ generated by $p$ and $f_{\alpha}(x)$; this gives the second isomorphism.