$\underline{\textbf{Overview}}$
My original answer had a fatal error as pointed out by the comment of Mike Earnest:
I have found one fatal mistake, in equation (4). When computing $T_0$, you cannot just take the number of linear arrangements and divide by $3n$. This is because not all $3n$ rotations of an arrangement are always distinct, as some may have symmetry.
For example, with $n=2$, your expression gives
$~\displaystyle T_0 = \frac{[(3\times 2)−1]!}{(3!)^2} = \frac{20}{6},$
(which is) not even an integer. This is why you need the Polya theory OP mentioned, which is what I assume is the math you were not familiar with in my answer.
He is right.
However, my error only applied to my computation of $T_0$. Using Inclusion-Exclusion, my computations of each of $T_1, T_2, \cdots, T_N$ were all correct. This is because with each of these other computations, I arbitrarily designated a specific color, and forced the $3$ seats of that color to be placed at the head of the table. This implied that the computation of all of the corresponding seatings were automatically rotationally distinct.
I decided to delete my answer and study Polya Theory to remedy my error in computing $T_0$.
Part of the way through however, I realized that $T_0$ can be accurately computed without Polya Theory. All that is necessary is for me to demonstrate an intuitive approach for computing the rotationally distinct seatings enumerated in $T_0$.
Therefore, I decided to undelete my answer, despite its being crude and long-winded, because it represents an approach that does not require any knowledge of Polya Theory.
$\underline{\textbf{The Problem}}$
$3n$ indistinguishable balls are coloured with $n$ colours so that each colour is to be used exactly three times. In how many ways these coloured balls can be placed around a circle so that $3$ balls with same colour never appears side by side?
I am assuming that $N$ is a fixed positive integer, that there are $3N$ seats around a round table, and you can have (for example) $2$ red balls next to each other but you can not have all $3$ red balls next to each other.
I used Inclusion-Exclusion. For general reference, see this article for an
introduction to Inclusion-Exclusion.
Then, see this answer for an explanation of and justification for the Inclusion-Exclusion formula.
Stars and Bars was also used.
See this article
and this article.
My response will follow the syntax used in the second Inclusion-Exclusion link above. Therefore, I will describe my answer by providing the explicit formula for each of $T_0, T_1, \cdots, T_N.$
At the end of my answer, I will provide the final computation, which (in accordance with Inclusion-Exclusion theory) will represent
$$\sum_{r=0}^N (-1)^r T_r. \tag1 $$
At the end of my answer, I will also show that my computation is mathematically equivalent to the answer of Mike Earnest.
$\underline{\textbf{Helper function} ~: ~f(r)}$
Before continuing, I need to describe a helper function.
Let $r \in \{1,2,\cdots,N\}$.
Assume that the round table has chairs that are labeled $A_1, A_2, \cdots A_{3N}$ in that order, around the table.
Further suppose that the colors are assigned the labels $C_1, C_2, \cdots, C_N.$
Further suppose that you are attempting to enumerate the number of ways that the specific colors $C_1, C_2, \cdots, C_r$ are positioned around the table, subject to the following constraints:
For each of the $r$ colors, the three balls of that color are seated right next to each other.
For the other $N-r$ colors (if any), no constraint is made on whether the corresponding $3$ balls are or are not seated next to each other.
The seats $A_1, A_2, A_3$ are specifically occupied by the $3$ balls of color $C_1$.
It is required that when examining the seats in the order $A_4, A_5, \cdots, A_{3N}$, that the color $C_2$ is the next color to appear. Then, color $C_3$ is the next color to appear. This continues so that color $C_r$ is the last of the colors $C_1, C_2, \cdots, C_r$ to appear.
By the phrase "color to appear", I intend that the $3$ balls of that color are seated next to each other.
I need to derive the function $f(r)$ that will enumerate exactly how many ways that the colors $C_1, C_2, \cdots, C_r$ can appear around the table, in that order, with the color $C_1$ specifically occupying seats $A_1, A_2, A_3.$
Note that $f(r)$ will not include the enumeration of how many ways the remaining $[3 \times (N-r)]$ balls can be seated. That is, $f(r)$ is only concerned with enumerating the number of ways of placing the $[3 \times r]$ balls.
Clearly $f(1) = 1.$
To enumerate $f(2)$ note that there will (in effect) be $3N-6$ empty seats to be situated before and after the three balls of color $C_2$.
Consequently, in accordance with Stars and Bars theory, $f(2)$ equals the number of non-negative integer solutions to
$$x_1 + x_2 = 3N-6 ~~: ~~\binom{(3N-6) + (2-1)}{2-1} ~~\text{solutions}.$$
Therefore, $f(2) = 3N-5.$
Computation of $f(r)$ for $r \in \{3,4,\cdots,N\}$ is very similar.
There will be $(3N - 3r)$ empty seats, divided into $r$ regions. So, $f(r)$ will equal the number of non-negative integer solutions to
$$x_1 + x_2 + \cdots + x_r = (3N - 3r).$$
So,
$$f(r) = \binom{(3N - 3r) + (r-1)}{r-1} = \binom{3N - 2r - 1}{r-1}. \tag2 $$
Note that the formula in (2) above is consistent with the previous computations of $f(1)$ and $f(2)$.
$\underline{\textbf{Helper function} ~: ~g(r)}$
A second helper function will also be used.
Again, assume that $r \in \{1,2,\cdots,N\}.$
Assume, consistent with the previous section, that colors $C_1, C_2, \cdots, C_r$ each have their corresponding $3$ balls seated next to each other.
This implies that there are (in effect) $(3N - 3r)$ empty seats that need to be filled with the balls of the remaining colors. $g(r)$ will represent the enumeration of the number of ways that this can be done. So,
$$g(r) = \binom{3N - 3r}{3} \times \binom{3N - 3r - 3}{3} \times \cdots \times \binom{6}{3} \times \binom{3}{3} = \frac{(3N - 3r)!}{(3!)^{N-r}}. \tag3 $$
While the computation of $f(r)$ does result in rotationally distinct seatings, the computation of $g(r)$ does not. However, in this entire answer, $g(r)$ is only used in conjunction with $f(r)$. When $g(r)$ is used in conjunction with $f(r)$, the resulting enumeration does represent rotationally distinct seatings. This explains, for example, why $g(r)$ is not used in the computation of $T_0$.
$\underline{\textbf{Computation of} ~T_0}$
Here, $T_0$ represents the number of rotationally distinct seatings without any regard to whether any of the $3$ balls of a specific color are next to each other.
Let $L$ denote the set of linearly distinct seatings, without any regard to whether any of the $3$ balls of a specific color are next to each other. That is, $|L|$, the number of elements in $L$, represents the computation that would be correct for $T_0$, if the seats were in a row, rather than at a round table.
$$|L| = \binom{3N}{3} \times \binom{3N-3}{3} \times \cdots \times \binom{6}{3} \times \binom{3}{3} = \frac{(3N)!}{(3!)^N}.$$
The set $L$ can be partitioned into small rotationally grouped subsets. That is, each subset contains a group of seatings that are linearly distinct but rotationally the same.
For illustrative purposes, I am going to explore the specific case of $N=4$, by attacking some basic questions. In each of these questions, it is assumed that there are $(12)$ seats at a round table.
$\underline{Q_1}$
What is the number of rotationally distinct ways of placing $3$ red balls around the table, leaving $9$ seats empty.
The number of linearly distinct ways is $~\displaystyle \binom{12}{3} = 220.$
Given any corresponding linearly distinct seating, such as $(1,2,3)$, let $\overline{(1,2,3)}$ denote the set of all linearly distinct seatings that are rotationally equivalent to $(1,2,3)$. Then:
$\overline{(1,2,3)} = \{(1,2,3), (2,3,4), \cdots, (12,1,2)\} ~: ~12~$ elements.
$\overline{(1,5,9)} = \{(1,5,9), (2,6,10), (3,7,11), (4,8,12)\} ~: ~4~$ elements.
So, the two subsets above represent two rotational groups of seatings. That is, each element in the rotational group is linearly distinct but rotationally equivalent. So, in order to compute the answer to question $Q_1$, you have to enumerate the number of different rotational groups.
To do this, you have to determine under what circumstances a rotational group would have less than $(12)$ elements. With respect to addition $\pmod{12}$ (i.e. clock arithmetic), the only seatings where this is possible will be seatings of the format $(x,y,z)$, where
- $y - x = 4$,
- $z - y = 4$,
- $x - z = 4$.
With such a seating, when it is rotated $4$ seats, $(x)$ becomes $(y)$, $(y)$ becomes $(z)$, and $(z)$ becomes $(x)$. For these seatings, the corresponding rotational group has $(4)$ elements instead of $(12)$ elements. The $2$nd rotational group shown above, is the only such group, for question $Q_1$. Therefore, the number of different rotational groups is
$$\frac{\binom{12}{3} - 4}{12} + \frac{4}{4} = \frac{220 - 4}{12} + \frac{4}{4} = 18 + 1 = 19.$$
$\underline{Q_2}$
What is the number of rotationally distinct ways of placing $3$ red balls and $3$ green balls around the table, leaving $6$ seats empty.
I am going to piggyback off of the analysis in $Q_1$.
The number of linearly distinct ways is $~\displaystyle \binom{12}{3}\times \binom{9}{3} = 18480.$
Let (for example) $(1_r,2_r,3_r,4_g,5_g,6_g)$ denote the seating where the $3$ red balls are placed in seats $1,2,$ and $3$, and the $3$ green balls are placed in seats $4,5,$ and $6$.
Similar to the syntax used in the analysis of $Q_1$ above, let $\overline{(1_r,2_r,3_r,4_g,5_g,6_g)}$ denote the set of all linearly distinct seatings that are rotationally equivalent to $(1_r,2_r,3_r,4_g,5_g,6_g)$.
So, to answer $Q_2$, you have to determine which linearly distinct seatings will have $4$ elements in their rotational group, rather than $(12)$ elements.
You need each of the two colors to satisfy the $x,y,z$ protocol expressed in the analysis to question $Q_1$. That is, consider the following $(3)$ rotational groups:
- $\overline{(1_r,5_r,9_r,2_g,6_g,11_g)}$.
- $\overline{(1_r,5_r,9_r,2_g,6_g,10_g)}$.
- $\overline{(2_r,6_r,10_r,1_g,5_g,9_g)}$.
In the first rotational group, the $3$ red balls obey the $x,y,z$ protocol, but the $3$ green balls do not, because the $3$rd green ball is in seat $11$, rather than seat $10$. Therefore, the $1$st rotational group has $(12)$ elements.
Both the $2$nd and the $3$rd rotational group above have only $(4)$ elements. Notice that the sole distinction between the $2$nd and $3$rd rotational groups is that the two colors, red and green have been permuted.
In order to answer question $Q_2$, the $x,y,z$ protocol discussed in the analysis to question $Q_1$ must be restated.
What is necessary is that seats $1$ through $4$ contain a single red ball, and a single green ball, in any order. Then, seats $5$ through $8$ must have seating identical to seats $1$ through $4$. Similarly, seats $9$ through $12$ must also have identical seating.
So, to determine how many distinct rotation groups there are, you have to determine how many different rotation groups will have $4$ elements instead of $12$ elements.
There are $(4 \times 3)$ ways of choosing one of seats $(1)$ through $(4)$ for the red ball, and then choosing one of the $3$ remaining seats in seats $(1)$ through $(4)$ for the green ball.
Therefore, there will be exactly $12$ linearly distinct seatings, whose rotational groups have $4$ elements, instead of $12$ elements.
Therefore, the answer to $Q_2$ is
$$\frac{18480 - \frac{4!}{(4-2)!}}{12} + \frac{\frac{4!}{(4-2)!}}{4} = \frac{18480 - 12}{12} + \frac{12}{4} = 1539 + 3 = 1542.$$
$\underline{Q_3}$
What is the number of rotationally distinct ways of placing $3$ red balls, $3$ green balls, $3$ blue balls, and $3$ brown balls around the table, leaving no seats empty.
In effect, $Q_3$ is asking for the enumeration of $T_0$, in the specific case of $N=4$. Again, this analysis will piggyback off of the previous analysis.
The number of linearly distinct ways is
$$ \binom{12}{3}\times \binom{9}{3} \times \binom{6}{3} \times \binom{3}{3} \\
= \frac{(12)!}{(3!)^4} = 369600.$$
The re-statement of the $x,y,z$ protocol, that was expressed in the analysis to question $Q_2$ is again used. In order for a linearly distinct seating to have only $4$ elements in its rotational group, rather than $12$ elements, the following constraints must be satisfied:
Each of the $4$ colors appears exactly one time each, in the seats $1$ through $4$. These $4$ colors may appear in any order, in seats $1$ through $4$.
Seats $5$ through $8$ must be identical to seats $1$ through $4$.
Seats $9$ through $12$ must be identical to seats $1$ through $4$.
Therefore, there are exactly $(4!)$ linearly distinct seatings, each of whom is in a rotational group that has $4$ elements, rather than $(12)$ elements.
Therefore, the answer to $Q_3$ is
$$\frac{369600 - (4!)}{12} + \frac{(4!)}{4} = \frac{369600 - 24}{12} + \frac{24}{4} = 30798 + 6 = 30804.$$
$\underline{\text{General Computation For} ~T_0}$
The analysis in $Q_3$ easily generalizes. So, $T_0$ can be computed by using the variable $N$, rather than the specific value $N=4$.
As a reference point, the original $\color{red}{\text{wrong}}$ computation for $T_0$ that I gave was
$$\color{red}{T_0 = \frac{(3N - 1)!}{(3!)^N}. \tag4} $$
As indicated at the start of this section, in the general case, the number of linearly distinct seatings is
$$\frac{(3N)!}{(3!)^N}.$$
In order for a linearly distinct seating to have only $(N)$ elements in its rotational group, rather than $(3N)$ elements, the following constraints must be satisfied:
Each of the $(N)$ colors appears exactly one time each, in the seats $1$ through $N$. These $N$ colors may appear in any order, in seats $1$ through $N$.
Seats $(N+1)$ through $(2N)$ must be identical to seats $1$ through $(N)$.
Seats $(2N+1)$ through $(3N)$ must be identical to seats $1$ through $(N)$.
Therefore, there are exactly $(N!)$ linearly distinct seatings, each of whom is in a rotational group that has $N$ elements, rather than $(3N)$ elements.
Therefore,
$$T_0 = \left[\left(\frac{(3N)!}{(3!)^N} - N!\right) \times \frac{1}{3N}\right] + \left[\frac{N!}{N}\right]$$
$$= \frac{(3N)!}{(3!)^N ~(3N)} - \frac{N!}{3N} + \frac{N!}{N} $$
$$= \frac{[(3N)-1]!}{(3!)^N} + \frac{2(N!)}{3N}. \tag4 $$
$\underline{\textbf{Explanation of subsets} ~S_k}$
I am going to create subsets $S_1, S_2, \cdots, S_N,$ in a manner consistent with the syntax used in the 2nd Inclusion-Exclusion link.
It is assumed that for $k \in \{1,2,\cdots, N\}$, that $S_k$ represents the subset of seatings where the specific constraint against color $C_k$ is violated. That is, it is assumed that each seating represented by $S_k$ has all three balls of color $C_k$ next to each other. In subset $S_k$ the balls of the other colors may or may not be next to each other.
$\underline{\textbf{Computation of} ~T_1}$
First, consider $S_1$.
$f(1)$ specifically enumerates how the balls of color $C_1$ will be seated. Notice that regardless of where these $3$ balls are seated (assuming that they are seated together), it will always be possible to rotate the seat numbers so that the seats assigned to color $C_1$ are the seats $A_1, A_2, A_3.$
Then, when computing $|S_1|$, once the color $C_1$ is seated, there are $g(1)$ ways of seating the remaining colors.
Therefore, $|S_1| = f(1) \times g(1).$
$T_1~$ represents $\displaystyle ~\sum_{i=1}^N |S_i|.$
By symmetry, $T_1 = \binom{N}{1} |S_1|.$
Therefore,
$$T_1 = \binom{N}{1} \times f(1) \times g(1). \tag5 $$
$\underline{\textbf{Computation of} ~T_2}$
First, consider $S_1 \cap S_2$.
$f(2)$ specifically enumerates how the balls of color $C_1$ and $C_2$ will be seated, assuming that color $C_1$ precedes color $C_2$.
This is where thing get tricky. Similar to the analysis in the previous section, it is tempting, but wrong, to enumerate
$|S_1 \cap S_2| = (2!) \times f(2) \times g(2).$
The reason that this is wrong, is that you are (for example) counting the seating of $C_1 -> A_1, A_2, A_3$ and $C_2 -> A_4, A_5, A_6$ twice. That is, it also shows in the seating of $C_2 -> A_1, A_2, A_3$, and $C_1 -> A_{3N-2}, A_{3N-1}, A_{3N}.$
Note
The way to avoid overcounting, when considering the intersection of $2$ or more of the $S_k$ subsets is to presume that the seats assigned to the color of the lowest index, will always be the seats $A_1, A_2, A_3.$
Therefore, $|S_1 \cap S_2| = f(2) \times g(2).$
Again, by symmetry, for any $1 \leq i_1 < i_2 \leq N$, you have that
$|S_{i_1} \cap S_{i_2}| = |S_1 \cap S_2|.$
$T_2~$ represents $\displaystyle ~\sum_{1 \leq i_1 < i_2 \leq N} |S_{i_1} \cap S_{i_2}|.$
Therefore,
$$T_2 = \binom{N}{2} \times f(2) \times g(2). \tag6 $$
$\underline{\textbf{Computation of} ~T_3}$
First, consider $S_1 \cap S_2 \cap S_3$.
$f(3)$ specifically enumerates how the balls of color $C_1, C_2~$ and $~C_3$ will be seated, assuming that the colors appear in strictly ascending order.
Again, things get tricky. As discussed in the previous section, to avoid over-counting, it must be assumed that the color of the lowest index is specifically assigned seats $A_1, A_2, A_3$.
However, once that assumption is made, you must also consider that the other $(3 - 1)$ colors involved in the intersection of $(3)$ subsets can be permuted in $(3-1)!$ ways.
Therefore, $|S_1 \cap S_2 \cap S_3| = [(3-1)!] \times f(3) \times g(3).$
Again, by symmetry, for any $1 \leq i_1 < i_2 < i_3 \leq N$, you have that
$|S_{i_1} \cap S_{i_2} \cap S_{i_3}| = |S_1 \cap S_2 \cap S_3|.$
$T_3~$ represents $\displaystyle ~\sum_{1 \leq i_1 < i_2 < i_3 \leq N} |S_{i_1} \cap S_{i_2} \cap S_{i_3}|.$
Therefore,
$$T_3 = \binom{N}{3} \times [(3-1)!] \times f(3) \times g(3). \tag7 $$
$\underline{\textbf{Computation of} ~T_r ~: 4 \leq r \leq N}$
The analysis here will virtually parallel the analysis of the previous section.
Therefore,
$$T_r = \binom{N}{r} \times [(r-1)!] \times f(r) \times g(r). \tag8 $$
$\underline{\textbf{Final Computation}}$
The computations are so unwieldy, that I will assemble the computations like building blocks.
In the specifications below, unless otherwise stated, it is assumed that $r \in \{1,2,\cdots,N\}.$
$$f(r) = \binom{3N - 2r - 1}{r-1}. $$
$$g(r) = \frac{(3N - 3r)!}{(3!)^{N-r}}. $$
$$ T_0 = \frac{[(3N)-1]!}{(3!)^N} + \frac{2(N!)}{3N}. $$
$$T_1 = \binom{N}{1} \times f(1) \times g(1). $$
$$T_2 = \binom{N}{2} \times f(2) \times g(2). $$
$$T_r = \binom{N}{r} \times [(r-1)!] \times f(r) \times g(r) ~: ~r \in \{3,4,\cdots,N\}. $$
The final tally is
$$\sum_{k=0}^N (-1)^k T_k.$$
$\underline{\textbf{Equivalence With the Answer of Mike Earnest}}$
For reference, here is the final computation of Mike Earnest.
$$
\left[\sum_{i=0}^n (-1)^i\binom{n}{i}\frac{(3n-2i-1)!}{\color{red}{(3!)^{n-i}}}\right]+\color{red}{\frac{2\cdot n!}{3n}} \tag5
$$
My computation of $T_0$ matches his $(i = 0)$ term in the LHS summation in (5) above, plus his $\color{red}{\text{additional term}}$. Therefore, it is sufficient to demonstrate that for all $i \in \{1,2,\cdots,N\}$, my $T_i$ enumeration matches his
$(i)$ term in the LHS summation in (5) above.
$\underline{T_1}$
$\displaystyle \binom{n}{[1]} \times \binom{3n - 2[1] - 1}{[1] - 1} \times \frac{(3n - 3[1])!}{(3!)^{n - [1]}}$
$\displaystyle = \binom{n}{[1]} \times \frac{(3n - 2[1] - 1)!}{(3!)^{n - [1]}}.$
$\underline{T_2}$
$\displaystyle \binom{n}{[2]} \times \binom{3n - 2[2] - 1}{[2] - 1} \times \frac{(3n - 3[2])!}{(3!)^{n - [2]}}$
$\displaystyle = \binom{n}{[2]} \times (3n-5) \times \frac{(3n - 6)!}{(3!)^{n - 2}}$
$\displaystyle = \binom{n}{[2]} \times \frac{(3n - 5)!}{(3!)^{n - 2}}
= \binom{n}{[2]} \times \frac{(3n - 2[2] - 1)!}{(3!)^{n - [2]}}.$
$\underline{T_r ~: r \in \{3,4,\cdots, n\}}$
$\displaystyle \binom{n}{r} \times [(r-1)!] \times \binom{3n - 2r - 1}{r-1} \times
\frac{(3n - 3r)!}{(3!)^{n - r}}$
$\displaystyle = \binom{n}{r} \times [(r-1)!] \times
\frac{(3n - 2r - 1)!}{[(r-1)!] \times [(3n - 3r)!]}
\times \frac{(3n - 3r)!}{(3!)^{n - r}}$
$\displaystyle = \binom{n}{r} \times
\frac{(3n - 2r - 1)!}{(3n - 3r)!}
\times \frac{(3n - 3r)!}{(3!)^{n - r}}$
$\displaystyle = \binom{n}{r}
\times \frac{(3n - 2r - 1)!}{(3!)^{n - r}}.$