The condition $$xy\in(xN)(yN)\tag{1}$$ is not necessary for the set of cosets to be a group. Instead, (1) is the condition for multiplication on cosets to be deducible from multiplication in your original group.
(In fact, if there are $n$ cosets, then we can always make them a group by choosing an arbitrary bijection to the cyclic group on $n$ elements. But this usually isn't a very interesting group structure, because it is entirely unrelated to the original $G$ we started out with.)
Recall that each coset is the equivalence class of elements of $g$ that differ by an element of $N$; that is, $$xN=\{nx:n\in N\}$$ This set has a representation as $xN$, but this representation is not unique: if $n\in N$, then $(xn)N=xN$. (For example, consider $D_3/C_2$. This means: take the group of symmetries of a triangle; any reflection $r$ generates a (non-normal) subgroup. That subgroup has three cosets, which I will call $C_2$, $\rho C_2$, and $\rho^2C_2$. You can verify that $rC_2=C_2$, $\rho rC_2=\rho C_2$, etc.)
We want a multiplication on cosets that arises from multiplication in $G$, so that we can deduce $(xN)(yN)$ just by computing $(xy)N$. But we have no guarantee that the coset $(xy)N$ doesn't depend on which $x$ and $y$ we choose to represent $xN$ and $yN$.
One way to fix this is to say: given two cosets $K_1$ and $K_2$, the natural multiplication is $$K_1K_2=\{k_1k_2:k_1\in K_1, k_2\in K_2\}$$ This allows us to define $(xN)(yN)$, and one can check that the result is a coset precisely when $N$ is normal. We want this multiplication to line up with multiplication in $G$: $$(xN)(yN)=xyN$$ To ensure this, we follow Arturo Magidin's comment: if $xy\in(xN)(yN)$, then $$xyN\subseteq(xN)(yN)N=(xN)(yN)$$ Since both sides are cosets, they cannot contain each other strictly. Thus $xyN=(xN)(yN)$, as desired.