Assume that $p>q$, p and q are prime number, that $p^2$ is not congruent to $1\mod q$ and that $q^2$ is not congruent to $1\mod p$. Classify all groups of order $p^2q^2$.
Can somebody give me some advice on where to start here, what I know that I can use Sylows theorem in some way here, but that is where I get stuck.
By Sylow's Third Theorem we have that there's only one Sylow $p-$ subgroup, as neither $q^2$ nor $q$ is congruent to $p$. Therefore as the only subgroup of the given order that subgroup is normal. Similarly there's only one Sylow $q-$ subgroup and it's normal too. Let them be $P$ and $Q$ respectively.
Now note that $P \cap Q = \{e\}$, as we have that $P \cap Q \le P, Q$, but the only number that divides both the orders of $P$ and $Q$ is $1$. Also note that $PQ=G$, as $PQ \le G$ and $|PQ| = \frac{|P||Q|}{|P \cap Q|} = p^2q^2$. But as $P,Q$ are normal subgroup the commutator $pqp^{-1}q^{-1} = (pqp^{-1})q^{-1} = p(qp^{-1}q^{-1}) \in P,Q$, so $pqp^{-1}q^{-1} = e \implies pq = qp; \quad\forall q \in Q, p \in P$
Now consider the map $\phi: P \times Q \to G$, defined $\phi(p,q)=pq$. Then we have that:
$$\phi((p,q)(p',q')) = \phi(pp',qq') = pp'qq' = (pq)(p'q') = \phi((p,q))\phi((p',q'))$$
So $\phi$ is a group homomorphism and if $\phi((p,q)) = e \implies pq = e$. But this means that $p,q \in P,Q$, so we must have that $p=q=e$. Hence $\ker(\phi) = \{e\}$. So $\phi$ is injective and we've proven that it's surjective already, so: $G \equiv P \times Q$.
Now as every group of order square of prime is abelian we have that $G$ is a the direct product of abelian groups, hence it's abelian. Now using the fundamental theorem of finitely generated abelian groups we have that:
$$G \equiv \mathbb{Z}_{p^2} \times \mathbb{Z}_{q^2}$$ $$G \equiv \mathbb{Z}_p \times \mathbb{Z}_{pq^2}$$ $$G \equiv \mathbb{Z}_{p^2q}\times \mathbb{Z}_{q}$$ $$G \equiv \mathbb{Z}_{pq} \times \mathbb{Z}_{pq}$$
While I can't answer the entire question, I imagine this will help. There is only 1 Sylow p-subgroup, and only 1 Sylow q-subgroup.
Using Sylow's third theorem, the number of Sylow p-subgroups of $G$, $n_p$, must divide the index of the Sylow p-subgroup in $G$ (index $=q^2$). Similarly, $n_q$ must divide $p^2$. Since $p$ and $q$ are prime, only $1$,$p$,and $p^2$ divide $p^2$, while only $1$, $q$ and $q^2$ divide $q^2$.
Using Sylow's third theorem again, $n_p$ is congruent to $1\mod(p)$, and $n_q$ is congruent to $1\mod(q)$.
If $q$ were congruent to $1\mod(p)$, then $q^2$ would be congruent to $1\mod(p)$ (which the question tells us is false). Thus we know that $q$ can't be congruent to $1 \mod(p)$. We then know that $n_p = 1$, and $n_q = 1$ also (by exactly analogous reasoning).
If the above reasoning was too terse, you need to try $1$, $p$ and $p^2$ as possible values for $n_q$, and then you will find that only $n_q = 1$ works.
Try looking at the easier case where the order of $G$ is $pq$ (with similar conditions on $p$ and $q$ as your case for $p^{2}$ and $q^{2}$), which gives unique Sylow $p$-subgroup and unique Sylow $q$-subgroup. This then can tell you what type of group $G$ must be. Now try and generalize the ideas to your case.
