The order of a group is the cardinality of the underlying set, as Robinson states. Indeed, the whole point of notation is that it is universally understandable. Therefore, something is "correct" if, when you write or say it, everyone knows what you mean. For example, the following statements are clear.
- $\mathbb{Z}$ and $\mathbb{R}$ are infinite groups.
- $\mathbb{Z}$ is a countable group.
- $\mathbb{R}$ is an uncountable group.
Let $G\cong (\mathbb{Z}, +)$ and let $H=(\mathbb{R}, +)$.
- $|G|<|H|$.
- $|G|=\aleph_0$.
Defining "order" like this means you can talk about groups of "countable" and "uncountable" order. Generally speaking, "order" means "size", so it is convenient to distinguish between the "size" of $\mathbb{Z}$ and $\mathbb{R}$. That said, with infinite groups you would often talk about a countable group rather than a group of countable order, and the concept of "order" is less important in infinite group theory than in finite group theory.
Indeed, the "order" of a group can be viewed as a way of placing a partial order on groups, but in infinite groups this partial order is not awfully useful. Steve Pride introduced a rather more meaningful ordering on finitely generated groups, called the "largeness ordering". This is based on homomorphisms, which is how we study groups anyway. Basically, $H\preceq_L G$ if $G$ has a finite index subgroup which maps onto $H$. This ordering ensures that free groups sit at the top, which is what we'd want for an ordering based on homomorphic images.
To conclude, I want to mention the following rather interesting result on the order of infinite groups. Hopefully, it will persuade you why it makes sense for the order of an infinite group to correspond to the cardinality of the underlying set:
Let $G$ an infinite group $G=\langle X\rangle$ and let $X$ be an infinite generating set for $G$. Then $|X|=|G|$.
For proofs, see here.
A related exercise is the following:
Exercise: Prove that if $G=\langle X\rangle$ is generated by a finite set then $G$ is a countable group (that is, a group of cardinality no more than $|\aleph_0|$).