Edit: as noted in the comments, this is not necessarily true for matrices with coefficients in non-commutative rings. However, the following is a positive result in the subcase where $A, B \in M_n(\mathbb{k})$ with $\mathbb{k}$ a field. In particular, this holds in $M_n(\mathbb{C})$
Let $A, B \in M_n(\mathbb{k})$. If $A$ is non singular, it has an inverse. Now, left multiplying by $A^{-1}$, we go from
$$
AB = I
$$
to
$$
B = A^{-1}
$$
Let's weaken the hypothesis and only assume that $AB = I$, $A$ not necessarily invertible. In other words, the linear mappings induced by $A$ and $B$, defined as $f(x) = Ax$ and $g(x) = Bx$ verify $fg \equiv id$, so $g$ must be injective. Since we are in finite dimension, a linear mapping is injective iff it is bijective. Therefore $g$ is invertible, or equivalently, $B$ is invertible. By the original argument, $B = A^{-1}$.