The (modular equivalence classes of) natural numbers that are relatively prime to $n$ are the "multiplicative units modulo $n$". Because given a natural number $a$, there exists a natural number $b$ such that $ab\equiv 1\pmod{n}$ if and only if $\gcd(a,n)=1$.
The multiplicative units modulo $n$ form a group of order $\varphi(n)$ (from which Euler's Theorem/Fermat's Little Theorem follows).
It is known that:
- If $p$ is an odd prime, then the multiplicative units modulo $p^k$ form a cyclic group of order $p^{k-1}(p-1)$.
- The multiplicative units modulo $4$ form a cyclic group of order $2$.
- The multiplicative units modulo $2^k$ with $k\geq 3$ form a group that is isomorphic to $C_2\times C_{2^{k-2}}$, the product of a cyclic group of order $2$ and a cyclic group of order $2^{k-2}$.
- If $n=p_1^{a_1}\cdots p_r^{a_r}$ is a prime factorization of $n$, then by the Chinese Remainder Theorem the multiplicative group of units modulo $n$ is isomorphic to the product of the groups of multiplicative units modulo $p_i^{a_i}$, $i=1,\ldots,r$.
Saying that every square is congruent to $1$ is equivalent to saying that your group is of exponent $2$, or that it is a product of cyclic groups of order $2$. From the above, we see that if $n$ Is divisible by any prime greater than $3$, then this cannot happen (it will have a cyclic subgroup of order $p-1\gt 2$, hence elements whose squares are not congruent to $1$); it cannot be divisible by $3^2$ either, for the same reason; and it cannot be divisible by $2^k$ with $k\geq 4$ (because it will have a subgroup that is cyclic of order $4$, so an element whose square is not congruent to $1$ either). Thus, the only possibilities for $n$ which satisfy the property that $x^2\equiv 1\pmod{n}$ whenever $\gcd(x,n)=1$ are $n=1$, $2$, $3$, $4$, $6$, $8$, $12$, and $24$. Indeed, $24$ is the largest one.
Your list is incomplete, though, as you see: you forgot $b=1$ and $b=2$.
Your argument is not very precise. "The primorial grows way too quickly for $b$ to be able to contain[sic] $p\geq 7$" is not a mathematically precise argument. What does "way too quickly" mean? This is at best a qualitative argument, when what you need is a precise quantitative argument.
In fact it is easier to argue that any number that is divisible by a prime greater than $3$ will necessarily fail (by using the Chinese Remainder Theorem to produce an integer that is $1$ modulo every other prime dividing $n$, but whose square is not $1$ modulo that particular prime), and similarly for $b$ divisible by $9$. Then the same argument gives that it cannot be divisible by $16$. Alternatively, you need to make a precise quantitative argument about the primorial, rather than the vibes-like argument of "grows way too quickly"...