These are the parametric equations of an epicycloid, where the large circle has radius $R=1$, and the smaller exterior rolling circle has radius $r=\frac{1}{4}$.
A possible simplification is writing
$$4a=4(x+i y) = 5 z - z^ 5\\
4b=4(x-i y) = \frac{5}{z} - \frac{1}{z^5}$$
where $z = \cos \theta + i \sin \theta$
and get a relation between $a$, $b$, that implies a relation between $x$ and $y$, see link. Or, we can just give the result provided by WA :
$$-81 - 45 x^2 + 365 x^4 - 15 x^6 - 480 x^8 + 256 x^{10} - 45 y^2 - 2395 x^2 y^2 - 45 x^4 y^2 - 1920 x^6 y^2 + 1280 x^8 y^2 + 365 y^4 - 45 x^2 y^4 - 2880 x^4 y^4 + 2560 x^6 y^4 - 15 y^6 - 1920 x^2 y^6 + 2560 x^4 y^6 - 480 y^8 + 1280 x^2 y^8 + 256 y^{10}=0$$
a curve of degree $10$
$4$ leaves" />
We can simplify the defining polynomial by writing it as a polynomial in $x^2 + y^2$ and $x^2 y^2$.
$\bf{Added:}$ Following the solution of @ACB: we have the polar representation ($x = r \cos \phi$, $y=r \sin \phi$)
$$\cos^2 2 \phi = \frac{(9 - 4 r^2)(16 r^4+ 3 r^2 + 6)^2}{3125 r^4}= f(r)$$
The function in $r>0$ on RHS is strictly decreasing on $(0, \infty)$. Indeed, we have
$$f'(r) = -24\, \frac{(-1 + r^2)^2 (9 + 16 r^2) (6 + 3 r^2 + 16 r^4)}{3125\, r^5}$$
Now, $f(1) = 1$, and $f(\frac{3}{2})=0$. We see that our curve is situated between the circles of radius $1$, and $\frac{3}{2} = 1 + 2\cdot \frac{1}{4}$, as it should.
So now we can write the curve in polar coordinates
$$r = f^{-1}(\cos^2 2\phi)$$
where $f^{-1}\colon [0,1]\to [1, \frac{3}{2}]$ is the inverse function of $f$.