Let $Q(x) = P(x+2)$ and $y = x + 1$, we have:
$$Q(y^2) = P(y^2+2) = P(x^2+2x+3) = P(x+3)^2 = Q(y)^2$$
It is then clear $Q(y) = y^n \Leftrightarrow P(x) = (x-2)^n, n \in \mathbb{N}$
is an obvious family of solutions for the functional equation.
By brute force matching of coefficients, it is easy to check for small $n$
(I have checked up to $n = 3$), this is the only solution for $P$ with degree $n$.
Let use prove that this is indeed the case.
Let $Q_1(y), Q_2(y)$ be two polynomial solutions of degree $n > 1$ for the functional equation:
$$Q(y^2) = Q(y)^2\tag{*}$$
By comparing the leading coefficients of $Q_1(y)$ and $Q_2(y)$, we know both of them are monic. This means their difference $U(y) = Q_1(y) - Q_2(y)$ is a polynomial of degree
at most $n-1$. If $U(y)$ is not identically zero, then by comparing degrees on both sides of:
$$U(y^2) = Q_1(y^2) - Q_2(y^2) = U(y)(Q_1(y) + Q_2(y))$$
We get a contradiction that $2 \deg{U} = \deg{U} + n \Leftrightarrow \deg{U} = n$.
From this, we can conclude $Q(y) = y^n \Leftrightarrow P(x) = (x-2)^n$ is the only solution of degree $n$ for corresponding functional equations.
UPDATE Related mathematical topics
Let $R(y)$ be the polynomial $y^2$, the functional equation $(*)$ can be rewritten as:
$$Q\circ R = R\circ Q$$
i.e. the two polynomials $Q$ and $R$ commute under functional composition. There is
a classification theorem which will help us to attack this type of functional equation involving commuting pair of polynomials.
First we need to define the concept of equivalence between 2 pairs of polynomials.
Let $(f_1, g_1)$ and $(f_2, g_2)$ be any two pairs of polynomials, we will call them
equivalent if we can find a linear polynomial $l(x) = ax + b, a \ne 0$ such that:
$$f_1 = l^{-1} \circ f_2 \circ l \quad\text{ and }\quad g_1 = l^{-1} \circ g_2 \circ l$$
In 1922, Ritt proved following theorem:
Let $f$ and $g$ be commuting polynomials. Then the pair $(f,g)$ is
equivalent to one of the following pairs:
- $x^m$ and $\epsilon x^n$ where $\epsilon^{m-1} = 1$.
- $\pm T_m(x)$ and $\pm T_n(x)$, where $T_m$ and $T_n$ are Chebyshev polynomials.
- $\epsilon_1 h^{\circ k}(x)$ and $\epsilon_2 h^{\circ l}(x)$, where $\epsilon_1^q = \epsilon_2^q = 1$ and $h(x)$ is a polynomial of the
form $x H(x^q)$ and $h^{\circ 1} = h$, $h^{\circ 2} = h\circ h$,
$h^{\circ 3} = h \circ h \circ h$, and so on.
If you have two commuting polynomials $f$ and $g$, then aside from the trivial case where $f$ and $g$ are functional iterate of a single underlying polynomial $h$. then up to equivalence, $f$ and $g$ can only be simple powers $x^m$ or Chebyshev polynomials $T_m(x)$.
Apply this to our function equation $(*)$. $R$ has the form of a simple power $y^2$,
So $Q(y)$ itself have to be a simple power as we have proved above.
When $R$ is something more complicated, isn't equivalent to a simple power or Chebyshev polynomial or functional iterate of other polynomial of $h$, then the only solution for
$Q$ are functional iterates $R^{\circ k}$ of $R$.
References
- Ritt's paper Permutable rational functions
- V. Prasolov's book Polynomials