Suppose $f(x)\in R[x]$ is invertible; write it as $f(x)=a+xg(x)$ and suppose $b+xh(x)$ is the inverse. Then
$$
ab+x(ah(x)+bg(x)+xg(x)h(x))=1
$$
and, comparing alike terms, we get $ab=1$. Therefore $a$ is invertible.
Until now we have not used the fact that $R$ is a domain. Suppose it is and that $f(x)$ has positive degree. Then $f(x)k(x)$ has positive degree: indeed the leading coefficient of the product is the product of the leading coefficients, which cannot be zero. Thus an invertible polynomial must have degree $0$ and, by what we saw before, it is an invertible constant.
Note that we are identifying constant polynomials in $R[x]$ with $R$.
In order to find a counterexample, we look at the simplest nondomain, that is, $R=\mathbb{Z}/4\mathbb{Z}$. The first case we can try is $1+2x$: the constant term must be invertible, and the leading coefficient must be a zero divisor, by the arguments above.
Now
$$
(1+2x)^2=…
$$
The general result that, if $R$ is not a domain, then the set of invertible polynomials contains non constant polynomials is false. See https://math.stackexchange.com/a/30390/62967, where it is proved that a polynomial $a_0+a_1x+\dots+a_nx^n$ is invertible if and only if $a_0$ is invertible and $a_i$ is nilpotent for $i>0$.
So if $R$ is not a domain but it has no (nontrivial) nilpotent element, then the invertible polynomials are just the invertible constants. A ring with such a property is the product of two fields, for instance.