For any $n\times n$ matrices $A,B$, we have $\text{Tr}(AB)=\text{Tr}(BA)$. This can be easily proved by algebraic summation. Thus $\text{Tr}([A,B])=\text{Tr}(AB-BA)=0$. Also, $\text{Tr}(kI)=kn$. Therefore $[A,B]=kI$ is impossible for all $A,B$.
But if we look at infinite dimensional Hilbert spaces (let's use $L^2(\mathbb R)$ in quantum mechanics as an example), we can actually have the conical commutation relation $[x,p]=i\hbar$ for the position operator and momentum operator $x,p$. That means the above conclusion about finite square matrix does not apply here anymore. I struggle to see why this is the case.
I try to rewrite the proof of the result about $n\times n$ for infinite dimensions. The problem is that I can no longer prove $\text{Tr}(AB)=\text{Tr}(BA)$. $$ \text{Tr}(AB)=\sum_n \langle n|AB|n\rangle\\ \text{Tr}(BA)=\sum_n \langle n|BA|n\rangle $$ I can no longer write out the entries of $A,B$ explicitly.
So why $[A,B]=kI$ is impossible in finite dimensions but possible in infinite dimensions? What exactly makes this difference?
EDIT: a comment below reminds me that the trace of an operator in infinite dimension can be undefined. This is a problem. But can anyone give more details about this?