A.1 First we show $f(x)$ is reducible mod $p$ for all $p$.
Taking $y=x^2$ ; $f(x)$ becomes $y^2+(4a+2)y+1$. Solving the equation $y^2+(4a+2)y+1=0$ we get $$ y=-(2a+1)\pm2\sqrt{a(a+1)} $$ Now replacing $y$ by $x^2$ we have $$ x=\pm \sqrt{-(2a+1)+2\sqrt{a(a+1)}} = \pm \ (i\sqrt{a} + i\sqrt{a+1}) $$ & $$ x=\pm \sqrt{-(2a+1)-2\sqrt{a(a+1)}} = \pm \ (i\sqrt{a} - i\sqrt{a+1}) $$
Hence we have obtained all the roots of the polynomial $f(x)$ say $x_1, x_2, x_3, x_4$ where
$$ x_1= (i\sqrt{a} + i\sqrt{a+1}) \ ; \ x_3= -(i\sqrt{a} + i\sqrt{a+1}) $$ $$ x_2= (i\sqrt{a} - i\sqrt{a+1}) \ ; \ x_4= -(i\sqrt{a} - i\sqrt{a+1}) $$
A.2 The three ways of expanding this quartic are as follows:
$\{(x-x_1)(x-x_2)\}\{(x-x_3)(x-x_4)\}=(x^2+1-2\sqrt{a}ix)(x^2+1+2\sqrt{a}ix)\\ = (x^2+1)^2-(2\sqrt{a}ix)^2 \\ = (x^2+1)^2-2^2\mathbf{(-a)}x^2$
$\{(x-x_1)(x-x_4)\}\{(x-x_2)(x-x_3)\}=(x^2-1-2\sqrt{a+1} \ ix)(x^2-1+2\sqrt{a+1} \ ix)\\ = (x^2-1)^2-(2\sqrt{a+1} \ ix)^2\\ =(x^2-1)^2-2^2 \ \mathbf{(-(a+1))} \ x^2$
$\{(x-x_1)(x-x_3)\}\{(x-x_2)(x-x_4)\}\\ =(x^2+(2a+1)+2\sqrt{a(a+1)})(x^2+(2a+1)-2\sqrt{a(a+1)})\\ =(x^2+(2a+1))^2-(2\sqrt{a(a+1)})^2\\ =(x^2+(2a+1))^2-2^2\mathbf{(a(a+1))}$
A.3 Now the final piece.
$f(x)$ can be factored in any of the above 3 ways. Consider any prime $p$.
If $\mathbf{-a}$ is a square element in $\mathbb{F}_p$; there exist $b \in \mathbb{F}_p$ such that $-a=b^2$. So factoring $f(x)$ as in form A.2.1. Hence considering $f(x)$ in $\mathbb{F}_p[x]$ we have $$ f(x)=(x^2+1)^2-2^2(-a)x^2=(x^2+1)^2-(2bx)^2=(x^2+1-2bx)(x^2+1+2bx) $$ Hence $f(x)$ is reducible modulo $p$ if $-a$ is a square element.
If $\mathbf{-(a+1)}$ is square element in $\mathbb{F}_p$, there exist $c \in \mathbb{F}_p$ such that $-(a+1)=c^2$. So we factor $f(x)$ as in form A.2.2. Now considering $f(x)$ in $\mathbb{F}_p[x]$ we have $$ f(x) =(x^2-1)^2-2^2 \ \mathbf{(-(a+1))} \ x^2=(x^2-1)^2-(2cx)^2=(x^2-1+2cx)(x^2-1+2cx) $$ Hence we get $f(x)$ is reducible modulo $p$ if $2$ is a square element.
Now if both $\mathbf{-a}$ and $\mathbf{-(a+1)}$ are non square elements in $\mathbb{F}_p$ then $\mathbf{a(a+1)}$ is a square element in $\mathbb{F}_p$. Hence factoring $f(x)$ as in the form A.2.3 we get that $f(x)$ is reducible modulo $p$.
But then I want to establish irreducibility in $\mathbb{Z}[X]$. Which I am not sure who to do.
– Saikat Jul 20 '21 at 07:38