I found this in textbook Silverman (2012) A Friendly Introduction to Number Theory on page 232 without explanation. They make it sound so obvious so I must have missed something along the way.
The way I know to find common factors between two numbers is prime factorization and Euclidean algorithm (but I do not know how to apply them in this situation). I mean $s-t$ and $s+t$ must be even, this I know how to easily prove.
But I can't figure out why their only common factor must be $2$. Please help!