3

Let $X_n$ be an positive recurrent, aperiodic Markov Chain and $T_0$ be the time of first return to state $0$. We learned in class that the invariant measure $\mathbf{y}$ of a Markov Chain can be written

$$y_i=\mathbf{E}\left[ \sum_{i=n}^\infty \mathbf{1}\{X_n=y_i\}\mathbf{1}\{n\leq T_0\}\right], \text{ and setting } y_0=1.$$

In other words, the entries of $\mathbf{y}$ are the number of visits to that state before returning to state $0$. This definition produces an invariant measure with entries $\in\mathbb{N}$. Since the stationary distribution $\mathbf{\pi}$ is just a normalized version of the unique invariant measure, does that mean that it's entries of $\mathbf{\pi}$ are always rational, even if the entries of the transition probability matrix aren't rational?

Since the stationary distribution of a Markov Chain is an eigenvector of the TPM, this would imply that matrices with transcendental components would always have a rational eigenvector, which seems... wrong.

Michael Hardy
  • 1
Jake Stevens-Haas
  • 189
  • WHy do you think that the sum of that series is a rational number? (See https://math.stackexchange.com/questions/116269/how-can-adding-an-infinite-number-of-rationals-yield-an-irrational-number) – Mariano Suárez-Álvarez Nov 11 '17 at 02:59
  • 1
    Each sum is an integer but the expectation need not be an integer or even rational. Recall that even a Bernoulli r.v. can have an expectation anywhere in [0,1]. – Ian Nov 11 '17 at 06:00
  • I think I was thrown off taking $T_0$ to be a fixed time and not a random variable itself. I admit I feel kinda silly reading this question in the morning. Clearly an infinite series, and thanks for clarifying. Just leaving this post up to share my gratitude, and then I'll delete it. – Jake Stevens-Haas Nov 11 '17 at 16:41

1 Answers1

4

Consider the discrete markov chain with probability transition matrix $\begin{pmatrix} 1- 1/\pi & 1/\pi \\ 1 & 0 \end{pmatrix}$. The stationary distribution is $(\frac{\pi}{\pi+1}, \frac{1}{\pi+1})$.

user365239
  • 1,988