The other two answer assume that $a$ has order $3$ (written $o(a)=3$) and $b$ has order $2$ (so $o(b)=2$). However, there is no reason a priori why this should be true: if $a^3=E$ then either $o(a)=3$ or $o(a)=1$. This means that the question does not describe a single group, but rather a family of groups. (Possibly the question has been altered slightly in the re-telling.)
We shall prove that the question as-stated describes three groups:
- If $o(a)=3$ and $o(b)=2$ then $G$ has order $6$, and is in fact the dihedral group of order $6$ (equivalently, $S_3$).
- If $o(a)=1$ and $o(b)=2$ then $G$ is cyclic of order two.
- If $o(a)=1$ and $o(b)=1$ then $G$ is the trivial group.
There is a fourth possibility for the orders of the generators:
- $o(a)=3$ and $o(b)=1$.
However, we will prove that this is not possible.
Lets begin by proving that orders of generators are important: note that the relator $(ab)^2=E$ allows you to write every word over $a$ and $b$ in the form $a^ib^j$ for $i\in \{0, 1, 2\}$ and $b\in\{0, 1\}$. Now, suppose some element of this form is actually trivial, so $a^ib^j=E$. Then $a^i=b^{-j}$, and so $o(a^i)=o(b^j)$, and so either $i=0=j$, or $a=E$, or $b=E$, or $a=E=b$ (why?).
This means that we can focus on orders of generators, and we have the four possibilities mentioned above. So lets consider each case and prove that they give the claimed groups, or (in the case of possibility (4)) are not possible.
Now, possibility (1) is covered by the other answers. Possibility (2) is easy to check, as $o(a)=1$ implies $a=E$, and then the relations $a^3=E^3=E$ and $b^2=E$ clearly hold, while $(ab)^2=E$ as $(ab)^2=(Eb)^2=b^2=E$. Similarly, possibility (3) is easy to check. Therefore, we need to prove that possibility (4) is not possible. To see this, suppose $o(b)=1$, so $b=E$, and note the following:
\begin{align*}
(ab)^2&=E\\
(aE)^2&=E\\
a^2&=E
\end{align*}
Combining this with $a^3=E$ gives
$$E=E\cdot E=a^{-2}a^3=a$$
and so $a=E$, so $o(a)=1$. Therefore, if $o(b)=1$ then $o(a)=1$, and so possibility (4) is impossible. QED
Here's a quick conceptual explanation of what's going on: There is a "maximal", with respect to homomorphic images, group generated by $a, b$ and where the relations $a^2=E, b^3=E, (ab)^2=E$ hold. The other answers are investigating this group, which just so happens to be the group $S_3$. There is a specific notation for describing groups in this way, called presentations, and so $S_3$ has presentation $\langle a, b\mid a^2, b^3, (ab)^2\rangle$.
Adding extra relations, like $a=E$ or $b=E$ or $a^ib^j=E$, corresponds to homomorphic images (not necessarily proper homomorphic images, like if we had considered the "new" relation $a^6=E$). So my answer may be interpreted as saying "the group $S_3$ has precisely three homomorphic images: itself, the cyclic group of order two, and the trivial group".