This answer is based upon a generating function of generalized Laguerre polynomials
\begin{align*}
L_k^{(\alpha)}(t)=\sum_{i=0}^k(-1)^k\binom{k+\alpha}{k-i}\frac{t^i}{i!}\tag{1}
\end{align*}
The Laguerre polynomials have some remarkable combinatorial properties and one of them is precisely suited to answer problems of this kind. This is nicely presented in Counting words with Laguerre series by Jair Taylor.
We encode the colors (b)lue, (r)ed, (g)reen and (y)ellow of the balls with the letters
\begin{align*}
\{b,r,g,y\}
\end{align*}
and are looking for words of length $8$ built from
\begin{align*}
b,b,r,r,g,g,y,y
\end{align*}
which have the property that they contain no consecutive equal letters. These words are called Carlitz words or Smirnov words.
We find in section 2 of the referred paper Laguerre polynomials $l_k(t)$ defined by their generating function
\begin{align*}
\sum_{k=0}^\infty l_k(t)x^k=\exp\left({\frac{tx}{1+x}}\right)
\end{align*}
The first few such polynomials are
\begin{align*}
l_0(t)&=1\\
l_1(t)&=t\\
l_2(t)&=\frac{1}{2}t^2-t\\
l_3(t)&=\frac{1}{6}t^3-t^2+t\tag{2}
\end{align*}
They are a specific form of Laguerre polynomials (1), namely $$l_k(t)=(-1)^kL_k^{(-1)}(t)$$
Theorem 2.1 in the referred paper states: Given nonnegative integers $n_1,\ldots,n_k$, the number of $k$-ary Carlitz words with the letter $i$ used exactly $n_i$ times is
\begin{align*}
\int_{0}^\infty e^{-t}\left(\prod_{i=1}^kl_{n_i}(t)\right)\,dt\tag{3}
\end{align*}
Since we have four characters $b,r,g,y$ each occurring twice, we set
\begin{align*}
&n_1=n_2=n_3=n_4=2
\end{align*}
We apply theorem 2.1. and obtain using (2) and (3) and with some help of Wolfram Alpha
\begin{align*}
\int_{0}^\infty&e^{-t}\left(\prod_{i=1}^4l_{n_i}(t)\right)\,dt\\
&=\int_{0}^\infty e^{-t}\left(l_2(t)\right)^4\,dt\\
&=\int_{0}^\infty e^{-t}\left(\frac{1}{2}t^2-t\right)^4\,dt\\
&=\int_{0}^\infty e^{-t}\left(\frac{1}{16}t^{8}-\frac{1}{2}t^{7}+\frac{3}{2}t^{6}
-2t^5+t^4\right)\,dt\\
&=\color{blue}{864}
\end{align*}
Variation 2:
Another variation is directly based upon a generating function of Smirnov words. (See example III.24 Smirnov words from Analytic Combinatorics by Philippe Flajolet and Robert Sedgewick for more information.)
A generating function for the number of Smirnov words over a four letter alphabet $V=\{b,r,g,y\}$ is given by
\begin{align*}
\left(1-\frac{4z}{1+z}\right)^{-1}
\end{align*}
The number of all Smirnov words of length $8$ over a four letter alphabet is therefore
\begin{align*}
[z^8]\left(1-\frac{4z}{1+z}\right)^{-1}
\end{align*}
Since we want to count the number of words of length $8$ with each character in $V$ occurring twice, we keep track of each character. Again with some help of Wolfram Alpha we obtain
\begin{align*}
[b^2r^2g^2y^2]\left(1-\frac{b}{1+b}-\frac{r}{1+r}-\frac{g}{1+g}-\frac{y}{1+y}\right)^{-1}=\color{blue}{864}
\end{align*}