We can take these equations you have derived and solve them for $p$ and $q$. Starting from scratch,
$(p+qi)^2 = p^2 + 2pqi + (qi)^2 = p^2 - q^2 + 2pqi$
Hence we have $p^2 - q^2 = 3$ and $2pq = -4$. Setting $q = \frac{-2}{p}$ and plugging this into the first equation, $p^2 - \left( \frac{-2}{p} \right)^2 = 3$, or $p^4 - 3p^2 - 4 = 0$.
This can be factored to give $(p^2 - 4)(p^2 + 1) = 0$. There are no real solutions to $p^2 = -1$, so we must have $p^2 = 4$, or $p = \pm 2$. Then $q = -2/p = \mp 1$.
Hence $p + qi \in \{ 2 - i, -2 + i \}$.