If we have a physical qubit with energy eigenstates $|0\rangle$ and $|1\rangle$ with energy separation $\Delta E$ its Hamiltonian in the absence of any interaction is
$$H=\hbar\frac{\Delta E}{2}\sigma_z $$
the time evolution operator is $U=e^{-iHt/\hbar}$, so why doesn't a qubit left to itself not just continuously rotate in the $z$ direction? And if it does, how is this not a problem? It seems to me that after some time $t$, a $\sigma_z$ gate will be applied whether we want it or not!
It does, unless you have a way to tune your Hamiltonian such that $\Delta$ becomes zero. Since a tunable Hamiltonian is something you usually want in a quantum computer implementation, this should not be a problem.
If this term is non-switchable, it just means that the basis in which you are working is continuously rotating, and you have to keep track of that and implement your "active" gates in a correspondingly rotated basis, or exactly at the time when the evolution due to your Hamiltonian has cancelled out. (Essentially, this is like working in the interaction picture.)
(By the way, it is debatable if you should call this "in the absence of any interaction", since this level splitting can very well seen as some type of interaction, and will in many cases depend on some external parameter.)
They are always rotating in the lab reference frame, but most quantum algorithms take a rotating reference frame to simply things, so that a z rotation only happens when you want it to.
The rotating reference frame spins along with the natural spinning rate of each qubit, so in general in can spin at different rates for different qubits.
