Here is an argument that only uses Cauchy's theorem (but can probably avoid it as well):
Let $a$ be an element of order $2$ and $b$ be an element of order $p$.
Then the elements of $G$ are $1,b,\dots,b^{p-1},a,ab,\dots,ab^{p-1}$.
If $ba=ab$, then $ab$ has order at most $2p$ but not $1,2,p$ and so $G$ is cyclic generated by $ab$.
If $ba\ne ab$, then $ab$ cannot be a power of $b$ because $a \notin \langle b \rangle$. Therefore, $ab$ has order $2$. Thus, $G$ has presentation
$\langle a, b \mid a^2= b^p = (ab)^2 = 1 \rangle$ and is the dihedral group $D_{2p}$.
Perhaps this is a simpler argument:
If the order of $ab$ is $2p$, then $G$ is cyclic.
If the order of $ab$ is $1$ or $p$, then $ab \in \langle b \rangle$, and so $a \in \langle b \rangle$, which is false.
Therefore, the order of $ab$ is $2$ and the rest follows as above.