I have been spending time attempting to understand the following proof by @user:544975:
By the orbit-stabilizer theorem, the size of every orbit of cosets divides $|H|$, and hence also $|G|$. Since there are exactly $p$ cosets of $H$ and $p$ is the smallest prime dividing $|G|$, it must be that either there is a single orbit of size $p$ or there are $p$ different orbits, all of size $1$.
The first option, however, is impossible, since for every $h\in H$, $hH=H$, meaning the action fixes the coset corresponding to the identity. Hence, there exists an orbit of size $1$, so they must all be of size $1$.
This means that for every $h \in H, g \in G$ we have: $$hg^{-1}H=g^{-1}H\implies \exists h_1\in H \space s.t.\space hg^{-1}=g^{-1}h_1\implies \\ ghg^{-1}=h_1\in H$$
I am not sure however why it is necessary we need p to be the smallest prime dividing G.