So I think I have found a complete answer for the case even when $p$ is not prime. There are infinite solutions for $p$ if every prime that divides $p$ is of the form $6n+1$. To show that this is true, set $q$ to be
$$q=\frac{1}{2} \left(\sqrt{4 a p^3-3}-1\right)$$
where $a$ is a natural number such that the above expression is integral. To show that such an $a$ must exist, first note that the expression inside the square root is odd and thus it suffices to show that there exists $a$ such that
$$4ap^3-3=s^2$$
for some natural number $s$. Note for later that this $s$ (if it exists) will always be odd. But this is a simple arithmetic progression which has a solution if and only
$$s^2\equiv -3\ (\text{mod }4p^3)$$
(in fact, it will have infinite solutions). To show that this is always the case, first note that
$$1\equiv -3\ (\text{mod }4)$$
Second, we must show that $-3$ is a quadratic residue of every prime in the prime factorization of $p$. But by our assumption these primes have the form $r=6n+1$. Then by Euler's criterion the equation
$$s^2\equiv -3\ (\text{mod }6n+1)$$
has a solution if and only if
$$\left(\dfrac{-3}{6n+1}\right)=1$$
(where $\left(\dfrac{a}{p}\right)$ is the Legendre symbol). Now, if $n=2m$ is even we have
$$6n+1=12m+1$$
and therefore
$$\left(\dfrac{-3}{12m+1}\right)=\left(\dfrac{-1}{12m+1}\right)\left(\dfrac{3}{12m+1}\right)=1\cdot 1=1$$
If $n=2m+1$ then
$$6n+1=12m+7$$
and therefore
$$\left(\dfrac{-3}{12m+7}\right)=\left(\dfrac{-1}{12m+7}\right)\left(\dfrac{3}{12m+7}\right)=(-1)\cdot (-1)=1$$
Either way, we conclude there are infinite integers $a$ such that $q$ is integral.
Next, we must show that $\text{gcd}(q,p)=1$. To do this, we will show there are integers $n,m$ (different from $n$ and $m$ above) such that
$$nq+mp-1=0$$
From above, we know that there exists odd $s$ such that
$$4ap^3-3=s^2$$
Thus, define
$$n=-\frac{s+1}{2}$$
$$m=ap^2$$
Then we have
$$nq+mp-1=-\frac{s+1}{4} \left(\sqrt{4 a p^3-3}-1\right)+ap^3-1$$
Substituting $s^2=4ap^3-3$ then gives us
$$=-\frac{s+1}{4} \left(s-1\right)+\frac{s^2-1}{4}=0$$
Having shown $p$ and $q$ are relatively prime, the last thing to do is show the original fraction in question is actually an integer. Plugging in our $p$ and $q$ and simplifying gives us a final result of $a(1+p+p^2)$ as desired.
To begin showing that these are the only solutions, note that if $p=3r$ for some $r$ then there are no solutions since
$$1+q+q^2\not\equiv 0\ (\text{mod }27)$$
for any $q\in\{0,1,...,26\}$ which implies $p^3\not|(1+q+q^2)$
Finally, if $p$ has a prime factor of the form $6n-1$, then $p^3\not |(1+q+q^2)$ (see answer to this question for reasoning).
