You are correct, it is always irreducible.
Your polynomial factors as
$AB$ where $A=X^n-(1+\sqrt{p})$ and $B=X^n-(1-\sqrt{p})$. It will suffice
to show that $A$ (and therefore $B$ also) is irreducible over $K={\mathbb Q}[\sqrt{p}]$.
Thanks to Karpilovsky’s theorem (many thanks to Bill Dubeque for
quoting it here), it will suffice to show the following points :
(1) $c=1+\sqrt{p}$ is not a $m$-th power in $K$, for any $m\geq 2$.
(2) $c$ is not of the form $-4z^4$ with $z\in K$.
Proof of (1) : suppose $1+\sqrt{p}=(x+y\sqrt{p})^m$ with $x,y\in{\mathbb Q}$.
Then $1-p=d^m$ where $d$ is the rational number $d=x^2-py^2$. So $d$ is a rational root of the monic polynomial $X^m-(1-p)$, so $d$ is an integer. As $1-p<0$, $d$ must be a negative integer and
$m$ is odd. Then $p=1-d^m$ is divisible by $1-d>0$, so
$1-d$ can only be $1$ (clearly impossible) or $p$. So $1-d=p,d=1-p$ and hence
$(1-p)^{m-1}=1$, which occurs only when $p=2$.
We then have $1+\sqrt{2}=(x+y\sqrt{2})^m$, $m$ odd and $x^2-2y^2=-1$. Each real
number has a unique $m$-th real root, so $x+y\sqrt{2}=(1+\sqrt{2})^{\frac{1}{m}}$
and hence $x-y\sqrt{2}=\frac{x^2-2y^2}{x+y\sqrt{2}}=(1-\sqrt{2})^{\frac{1}{m}}$. Adding
those two last equalities, one obtains
$$
x=\frac{(1+\sqrt{2})^{\frac{1}{m}}+(1-\sqrt{2})^{\frac{1}{m}}}{2}
$$
Then, $r=2x$ is both rational and a sum of two algebraic integers, so it
must be an integer. Now,
$$
r=(\sqrt{2}-1)^{\frac{1}{m}}
\Bigg(\bigg(\frac{\sqrt{2}+1}{\sqrt{2}-1}\bigg)^m-1\Bigg) >0
$$
On the other hand, $\big(\frac{3}{2}\big)^3 > \sqrt{2}+1$ yields
$(\sqrt{2}+1)^{\frac{1}{m}} \leq \frac{3}{2}$, and
$\sqrt{2}-1 > \big(\frac{1}{2}\big)^3$ yields $1+(\sqrt{2}-1)^{\frac{1}{m}} \geq
\frac{3}{2}$. Combining the two, we obtain $r<1$. Finally $r$ is an integer
strictly between $0$ and $1$, which is impossible.
Proof of (2) : $1+\sqrt{p}=-4(x+y\sqrt{p})^4$ with $x,y\in{\mathbb Q}$ is clearly impossible as a fourth power cannot be negative.