I numerically verified the case for the $2\times2$ matrix but don't know how to prove it theoretically. Let's say, if $\Sigma$ is an $n\times n$ matrix, and we have $A \Sigma A^{\dagger}=\Sigma$, where $A$ is a unitary matrix, and $\Sigma$ is a Hermitian matrix, then does it imply $\Sigma$ also to be a diagonal matrix?
For clarification, $\Sigma$ is real in my case, but I don't know if 'real' really matters to prove $\Sigma$ is diagonal.
Here is a simple numerical verification for $2\times2$ case: Let
$$A = \begin{bmatrix} \cos\theta & -\sin\theta\\ \sin\theta & \cos\theta \end{bmatrix}, $$
which is a rotation matrix, let $\Sigma = \begin{bmatrix} a & b\\ c & d \end{bmatrix}$, from the equation $A \Sigma A^{\dagger}=\Sigma$ we get $a=d, c=-b$, now hermitian implies $c=b$, so we get $c=b=0$ and prove $\Sigma$ is a diagonal matrix.