I have seem to forgot this important fact, and I am trying to prove it to myself by looking at $A$ as matrix of the elementary row operations.
Where I seem to get stuck is that if $A$ are elementary row operations on $B$ when multiplying $B$ on the left does not multiplying $B$ from the right will no act on $B$ as elementary row operations and therefore will not result $BA=I$?