If $h(x)$ is reducible over $\mathbb{Q}$, then by Gauss's Lemma, $h(x)$ is reducible over $\mathbb{Z}$. We can find two monic $p(x), q(x) \in \mathbb{Z}[x]$, both with $\deg(p), \deg(q) < n$, such that $h(x) = p(x)q(x)$.
Notice for $1 \le k \le n$,
$$
p(a_k)q(a_k) = h(a_k) = 1 \land p(a_k), q(a_k) \in \mathbb{Z}
\quad\implies\quad p(a_k) = q(a_k).
$$
This implies $p(x)$ and $q(x)$ coincide on more points than their degree and hence they are equal to each other, i.e.,
$$\prod_{k=1}^{n} (x - a_k) = h(x) - 1 = p(x)^2 - 1 = (p(x)-1)(p(x)+1).$$
A consequence of this is $n = 2\ell$ is even. Furthermore, relabel $a_k$ is required, we can assume
$$
p(x) - 1 = \prod\limits_{k=1}^\ell (x-a_k)
\quad\text{ and }\quad
p(x) + 1 = \prod\limits_{k=1}^\ell (x-a_{k+\ell}).
$$
If $h(x)$ is reducible over $\mathbb{Q}$, so does $h(x + a)$ for any constant $a$.
Using a suitable choice of $a$, we only need to study the special case where one of the $a_k$, say $a_0 = 0$. Under this assumption, we have
$$p(0) - 1 = (-1)^\ell\prod_{k=1}^\ell a_k = 0
\quad\implies\quad
p(0) + 1 = (-1)^\ell\prod_{k=1}^\ell a_{k+\ell} = 2.
$$
Since $2$ is a prime, there aren't too much choice for $a_{k+\ell}$, they can only be $\pm 1$ or $\pm 2$. The are only $4$ possibilities
and only $3$ of them leads to sensible solution.
$$\require{cancel}
\begin{array}{|r:l|}
\hline
p(x) + 1 & p(x)-1\\
\hline
(x-2)(x-1) & x(x-3)\\
\cancel{(x-2)(x-1)(x+1)} & \cancel{x(x^2-x-1)}\\
(x+2) & x\\
(x+2)(x+1) & x(x+3)\\
\hline
\end{array}
$$
Putting the offset $a$ back, this leads to following $3$ possibilities for $h(x)$:
$$\begin{align}
(x-a)(x-a-1)(x-a-2)(x-a-3) + 1 &= ((x-a)(x-a-3) + 1)^2\\
(x-a)(x-a+2) + 1 &= (x-a+1)^2\\
\cancel{(x-a)(x-a+1)(x-a+2)(x-a+3) + 1} &= \cancel{((x-a)(x-a+3) + 1)^2}
\end{align}
$$
The $3^{rd}$ set of possibility is not a new one. It can derived from the $1^{st}$ possibility by substitution $a \mapsto a - 3$. This leaves us with two possibilities and it is easy to see they are equivalent to the two exceptions mentioned in question.