Seems like a quantum bit with $3$ orthogonal quantum states is called a qutrit - and they have been demonstrated practically. In comparison with $n$ qubits that have ~$2^{n}$ states, these have ~$3^{n}$ states.
Is there a theoretical upper-limit on how many orthogonal quantum states a quantum bit has? How would we call that unit of quantum information?
There is, in principle, no limit to the dimension of the state space of a quantum system. This includes infinite dimension (usually countable, i.e. a separable Hilbert space) and any large but finite dimension. In the context of quantum information, systems with a state space of dimension $d\geq 2$ are usually called qudits.
It's also important to mention that physical implementations of larger dimensions are harder to bolt down, and particularly it's harder to have a system of dimension $d>2$ that doesn't talk to more states of the system. This is because qubits are realized by restricting some larger system (usually infinite-dimensional) to only two states, by suitable physical considerations, and it is generally quite challenging to really cut out all the other states. Trying to restrict to three or more states while keeping the restriction clean is, generally speaking, harder.
In principle, though, there's nothing stopping you from having any large dimension you want on your system, but then you don't call it a 'bit' anymore.
