Consider the sequence of left ideals $R=Ra^0 \supset Ra^1 \supset Ra^2 \supset Ra^3 \supset\dots$ - we are going to show that all the inclusions are proper, meaning that there is in infinite number of ideals in $R$, thus $R$ itself cannot be finite.
Suppose that at some point the inclusion is not proper, i.e. $Ra^n = Ra^{n+1}$. Since $1 \in R$, $1 \cdot a^n \in R a^{n+1}$, meaning that there is $c \in R$ such that $a^n = ca^{n+1}$. Multiply this by $b^n$: $a^nb^n=ca^{n+1}b^n \Rightarrow 1 = ca$. So, $a$ posesses a left inverse $c$, and by assumption $b$ is the right inverse of $a$. If $a$ has both-sided inverses, then these inverses are equal $c = b$, so $ba=1$, contradicting the assumption. So, all inclusions are proper.
This actually proves a stronger result: the situation in question cannot happen in Artinian rings. This proves that, e.g., for matrices $ab=1$ implies $ba=1$.