I consider that the elements in $G$ only have two cases:
1) $G$ has an element $a$ with order $p^2$ then it's abelian.
2) $G$ has an element $a$ with order $p$, then we have a subgroup $<a>$ and choose another element $b$ $\notin <a>$ then we have $G = <a,b>$ but under this circumstance, how to prove that $<a,b>$ is abelian?
Edit: With the help of the comments posted by @user1729 and @JohnHughes, I may have a solution:
Consider $Z(G)$, since the order of $G$ is $p^2$ then we know $G$ has nontrivial center, also the center of $G$ is a normal subgroup of $G$, we get the followings results:
(1) $Z(G)$ is abelian with order prime $p$ (2) $G/Z(G)$ has order prime $p$ then is abelian.
Using(1) and (2) we know $G$ is abelian.
Edit: The last edit is wrong, since I do not use that $Z(G)$ is generated by a single element $a$ and $ab = ba$ then $G = <a,b>$ is abelian.