Let $G=\langle a,b\rangle$. Prove neither $\langle a\rangle$ nor $\langle b \rangle$ are normal in $G,$ for $a=(1234)(57), b=(24)(5678)$.

Question

Let $G=\langle a,b\rangle,$ a group formed by two permutations of $S_8$:

$$a=(1 2 3 4)(5 7)\quad\text{and}\quad b=(2 4)(5 6 7 8).$$

I have to prove that neither $\langle a\rangle$ nor $\langle b \rangle$ are normal subgroups of $G$.

---

I know that if I've got a group called $G$ and its subgroup $H$, $H$ is normal in $G$ if $g^{-1} * h * g$ is in $H$ for every $h$ from $H$ and every $g$ from $G$.
But my doubt is:

$a$ and $b$ are the generators of those subgroups. If I want to see if $\langle a\rangle$ is normal in $G$, is it enough to see if $a^{-1} * a * a$ and $b^{-1} * a * b$ are in $\langle a\rangle$? And the same with $\langle b \rangle$.

Answer 1

No, in general you cannot just check $a$ and $b$. It’s not enough to check all elements in a generating set for $G$.

In general, if $G$ is generated by a set $X$, and $H$ is a subgroup, to check that $H$ is a normal subgroup it is enough to do one of two things:

1. Check that for each $x\in X$ we have $xHx^{-1}=H$; or
2. Check that for each $x\in X$, we have both $xHx^{-1}\subseteq H$ and $x^{-1}Hx\subseteq H$.

It is not enough to check that for every $x\in X$ you have $xHx^{-1}\subseteq H$, which is what you are proposing. For example, you can see this previous answer to show that in general, the set of elements such that $xHx^{-1}\subseteq H$ need not be closed under inverses, so you could end up with the semigroup generated by $X$ rather than all of $G$ if you only check $xHx^{-1}\subseteq H$.

That said, in the finite case it will be enough to check $xHx^{-1}\subseteq H$ for every $x\in X$, because the fact that $H$ is finite guarantees you get equality, so you are in fact verifying item 1 above.

Likewise, say you have a generating set $Y$ for $H$. In that case, it is enough to check $gyg^{-1}\in \langle Y\rangle$ for all $y\in Y$ and all $g\in G$. That suffices, because you get an arbitrary power of $y$ because $(gyg^{-1})^k = gy^kg^{-1}$ for all integers $k$, so if $gyg^{-1}\in H$, then $gy^kg^{-1}\in H$.

So, if $G=\langle X\rangle$, and $H=\langle Y\rangle$, it is enough to check that:

1. For all $y\in Y$, for all $g\in G$, $gyg^{-1}\in H$; or
2. For all $y\in Y$ and all $x\in X$, both $xyx^{-1}\in H$ and $x^{-1}yx\in H$.

Answer 2

Since general answers have been given, I'm gonna give one with this specific $G$.

If $\langle a \rangle$ would be normal, $bab^{-1} = (1432)(78) \in \langle a\rangle$, but since $bab^{-1}$ moves the cipher $8$ while $a$ doesn't, $bab^{-1} \not\in\langle a\rangle$, hence $\langle a\rangle$ can't be normal.

Analogously for $\langle b\rangle$ noticing that $aba^{-1} = (13)(5876)$ with similar consideration on the cipher $3$.