In this problem even if I relax the condition that $F$ is a field,then it still remains true, I guess, because if $p$ is reducible, then $p=fg$ where $f,g$ are both nonunit polynomials. Now $fg$ is in $(p)$ but none of $f$ and $g$ are in $(p)$ which contradicts that $(p)$ is prime.
Here the only requirement is that $F$ be an integral domain so that $f(x)$ and $p(x)$ can be associate.
