Yes, this is true and can be stated slightly more abstractly. Here's a proof that doesn't use the division algorithm.
Lemma.
Let $R$ be a unital commutative ring and $R[x]$ be the one-variable polynomial ring over $R$. For $\alpha \in R$, let
\begin{align*}
\varphi = \text{eval}_\alpha: R[x] &\to R\\
x &\mapsto \alpha
\end{align*}
be the evaluation homomorphism. Then $\ker(\varphi) = (x-\alpha)$ and the induced map $\overline{\varphi}: R[x]/(x-\alpha) \to R$ is an isomorphism.
Note that $\ker(\varphi) = \{f \in R[x] : f(\alpha) = 0\}$.
Proof. Certainly $(x - \alpha) \subseteq \ker(\varphi)$, so it remains to show the reverse inclusion. Given $f = \sum_{i=0}^n a_i x^i \in \ker(\varphi)$, then
$$
0 = f(\alpha) = \sum_i a_i \alpha^i \, .
$$
We can rewrite $f$ as a polynomial in $x-\alpha$ by writing $x = (x - \alpha) + \alpha$ and using the binomial theorem:
\begin{align*}
f &= \sum_{i=0}^n a_i x^i = \sum_{i=0}^n a_i ((x - \alpha) + \alpha)^i = \sum_{i=0}^n a_i \sum_{j = 0}^i \binom{i}{j} (x-\alpha)^j \alpha^{i-j}\\
&= \sum_{i=0}^n a_i \sum_{j = 0}^n \binom{i}{j} (x-\alpha)^j \alpha^{i-j} = \sum_{j = 0}^n \underbrace{\left(\sum_{i=0}^n a_i \binom{i}{j} \alpha^{i-j}\right)}_{b_j} (x-\alpha)^j = \sum_{j=0}^n b_j (x - \alpha)^j
\end{align*}
(Note that $\binom{i}{j} = 0$ for $j > i$, which allows us to change the upper limit of the inner sum from $i$ to $n$.) Since $f(\alpha) = 0$, then
$$
b_0 = \sum_{i=0}^n a_i \binom{i}{0} \alpha^{i-0} = \sum_{i=0}^n a_i \alpha^i = f(\alpha) = 0 \, .
$$
Since $b_0 = 0$, then every term in $f = \sum_{j=0}^n b_j (x - \alpha)^j$ has a factor of $x - \alpha$, so $f \in (x - \alpha)$.
This idea also works in multivariable polynomial rings; see here for the proof.