Let $A, B \in M_n(\mathbb{R})$ such that : $AB-BA$ invertible and $A^2+B^2 = AB$ then prove that : $3 \mid n$.
I have seen answers here. But I was wondering if the other direction is also true, that is: say we have $A^2+B^2 = AB$ and if $3 \mid n$, then $AB-BA$ is invertible. If it is not true, then are there any cases divided where if A and B are nonzero, then AB-BA is either invertible or not invertible?
No. Consider e.g. $A=B=$ any nonzero nilpotent matrix whose square is zero.
For a less obvious counterexample, let $$ C=\pmatrix{0&-1\\ 1&1}, \ X=\pmatrix{0&0&1\\ 1&0&0\\ 0&1&0}, \ Y=\pmatrix{0&0&\frac12\\ -1&0&0\\ 0&2&0}. $$ It can be verified that $C^2+I=C$ and $X^2+Y^2=XY$. For every $n$ that is $\ge6$ and divisible by $3$, put \begin{cases} A= C\oplus\cdots\oplus C\ \text{ and }\ B=I&\text{ when $n$ is even},\\ A= C\oplus\cdots\oplus C\oplus X\ \text{ and }\ B=I_{n-3}\oplus Y&\text{ when $n$ is odd}.\\ \end{cases} Then $A$ and $B$ are invertible, $A^2+B^2=AB$ but $AB-BA$ is singular.
In any dimension, $A=B=0$ provides an immediate counterexample.
