It is not true that every finite group is the union of conjugates of a proper subgroup. For example, if $G$ is abelian, then $gHg^{-1}=H$ for all $g\in G$, and therefore the union will just be $H$, and never all of $G$ (since $H$ is given as a proper subgroup). For example, $G=\mathbb{Z}/4\mathbb{Z}$, $H=2\mathbb{Z}/4\mathbb{Z}$.
Even if $G$ is not abelian, the result may fail. For example, if $H$ is normal, then $gHg^{-1}=H$ for all $g\in G$, so the union is never all of $G$. For example, $G=S_3$, $H=\langle (1,2,3)\rangle$.
Even if $H$ is not normal, the result may fail. For example, take $G=S_4$, and $H=\langle (1,2,3)\rangle$. Then every conjugate of $H$ lies in $A_4$, so $\cup_{g\in G}gHg^{-1}\subseteq A_4\neq S_4=G$ (in fact, the union will be all of $A_4$).
In fact, as noted in comments, a finite group is never the union of conjugates of the same proper subgroup; though that is perhaps not immediately obvious. You can find a discussion of this and what happens when $G$ is infinite here.