I'm asking this question in the same spirit as this other question: Conjectures that have been disproved with extremely large counterexamples?.
What's a nice conjecture relating to finite groups, that first fails for a group of order $N$, with $N$ large? By "nice" I mean a conjecture with a balance between having a simple statement, one that first fails for a large $N$, and something that is not immediately obvious that it is going to fail.
An example might be the conjecture "if $G$ is a finite simple group, then it is the unique simple group of its order", which first fails for $N=20160$.
"Large" in this context is of course undefined, perhaps we'll say $N$ is "large" if the number of groups of order at most $N$ up to isomorphism is "large" in a more generic context. Perhaps I might suggest $N \geq 32$ as large, since $N= 32$ is the smallest number such that there are at least $100$ groups of order at most $N$. The bigger $N$ you can come up with, though, the better!