2

I have the following problem which I can't seem to figure out. The problem is as follows.

Consider simple random walk on {0, 1, ... , k} with reflecting boundaries at 0 and k, that is, random walk on the path from 0 to k. A random walker earns \$k every time the walk reaches 0 or k, but loses \$1 at each internal vertex (from 1 to k − 1). In 10,000 steps of the walk, how much, on average, will be gained?

I attempt to solve this by introducing a function $r(x) := \begin{cases} -1, \;\; x\in \{1,...,k-1\} \\ k, \;\; x \in \{0,k\}\end{cases}$. The random walk is clearly a markov chain, which I'll denote $X_0,X_1,...$. The average gain is then simply $\frac{1}{n}\sum_{k=1}^nr(X_k)$. Here $n = 10 000$.

I have access to a solution to this problem. The person finds the same transition matrix as me for the chain, which simply is defined by letting $P(X_1 = 1 | X_0 = 0) = 1$, $P(X_1 = k-1 | X_0 = k) = 1$ and e.g. $P(X_1 = 2 | X_0 = 1) = 1/2$ and $P(X_1 = 0 | X_0 = 1) = 1/2$ and so on. So there is no problem there. In this solution, however, the person uses the law of large numbers for ergodic Markov chains, i.e. $\frac{1}{n}\sum_{k=1}^nr(X_k) \approx E[r(X)]$ with $X$ distributed with the stationary distrubtion of this Markov chain.

So the problem is that I don't understand how we can use the law of large numbers of ergodic Markov chains in this situation, since I don't see how the Markov chain in this assignment could possibly be ergodic. The chain is clearly irreducible, but when drawing the transition graph for the chain, I find the period of the chain to be 2 thus not a-periodic, and hence not ergodic.

If the chain were ergodic, then there would be no problem since we then simply can find the stationary distribution to the chain and use the law of large numbers for ergodic markov chains and calculate $E[r(X)]$.

So my question is, is the chain above ergodic or not? If it is, how can one see this? If not, maybe there is still a way to use (some version) of law of large numbers to solve the problem in, more or less, the same way (even though the chain is not ergodic)?

Thanks!

tarkovsky123
  • 526
  • If the Markov chain is irreducible and has finitely many states, then it is ergodic. Aperiodicity has nothing to do with this (it is relatede to the stronger property of mixing). – D. Thomine Dec 08 '18 at 17:14
  • Aha. Ok interesting. My text book define a (finite) chain to be ergodic if it is aperiodic and irreducible. – tarkovsky123 Dec 08 '18 at 17:24
  • @D.Thomine It is true that an irreducible Markov chain on finitely many states is positive recurrent, but ergodic is usually defined as positive recurrent and aperiodic. – Math1000 Dec 08 '18 at 22:49
  • @Math1000: by people who don't know what ergodic means (which is, alas, too frequent - even Wikipedia uses the wrong definition, I should fix it). Conterpoint: a previous answer of mine. – D. Thomine Dec 09 '18 at 09:01
  • And, in order to show that this is not just me: here are a couple of notes using the definition without aperiodicity. – D. Thomine Dec 09 '18 at 09:02
  • Ok... How is it possible to give two different definitions like this? Does the law of large numbers for ergodic markov chains holds when using both definitions? – tarkovsky123 Dec 09 '18 at 10:31

0 Answers0