You are on the right track. You could indeed check that all possible (nonempty) products of the linear factors $p_1$, $p_2$, $p_3$ and $p_4$ do not have coefficients in $\Bbb{Q}$, except of course $f=p_1p_2p_3p_4$. But this is a lot of cumbersome work.
You could also note that if some product of the $p_i$ has coefficients in $\Bbb{Q}$, then so does the product of the remaining $p_i$. For example, if $p_1p_2$ has coefficients in $\Bbb{Q}$, then so does $p_3p_4$. This is because we can do division with remainder with polynomials; there exist unique polynomials $q,r\in\Bbb{Q}[x]$ such that $f=qp_1p_2+r$ and $\deg r<\deg p_1p_2$. We know that $f=p_1p_2p_3p_4$ and hence $q=p_3p_4$ and $r=0$. In particular we see that $p_3p_4=q\in\Bbb{Q}[x]$.
By this argument, we only need to check half of the possible products. We can choose the easier polynomials; clearly the $p_i$ do not have coefficients in $\Bbb{Q}[x]$, so neither do the products $p_1p_2p_3$, $p_1p_2p_4$, $p_1p_3p_4$ and $p_2p_3p_4$. Then it suffices to check that the three products $p_1p_2$, $p_1p_3$ and $p_1p_4$ do not have coefficients in $\Bbb{Q}[x]$; it follows that the products $p_3p_4$, $p_2p_4$ and $p_2p_3$ do not have coefficients in $\Bbb{Q}[x]$ either.
Alternatively, you could note that the numbers
$$a^0=1,\qquad a^1=\sqrt{3}-\sqrt{5},\qquad a^2=8-2\sqrt{15},$$
are linearly independent over $\Bbb{Q}$, and hence the minimal polynomial of $a$ must have degree at least $3$. Then the argument above (together with your factorization) show that its degree is $4$, and hence $f$ is the minimal polynomial. You could also continue to compute
$$a^3=18\sqrt{3}-14\sqrt{5},\qquad a^4=124-32\sqrt{15},$$
and see that there is no linear dependence between the powers of $a$ until you include $a^4$, and some linear algebra will give you the coefficients of $f$.