7

I have the following homework problem:

Let $(X_n)_{n \geq 0}$ be a Markov chain on the state space $\lbrace0,1,...\rbrace$. Writing $p_i := p_{i,i+1}$ and $q_i := p_{i,i-1}$, the transition probabilities are $$p_0 = 1, \qquad \mathrm{and}\quad p_i + q_i =1, \quad \frac{q_i}{p_i} = \left(\frac{i}{i+1}\right)^2 \mathrm{for} \; i \geq 1$$

I am asked to show that almost surely we have $X_n \rightarrow \infty$ as $n \rightarrow \infty$.

[Incidentally, I'd like to check my definition of almost sure convergence in this case: I'd say that $X_n \rightarrow \infty$ almost surely just when for every $K\geq 0$ there is $N=N_K$ such that $\mathbb{P}(X_n \geq K \;| \;n \geq N) = 1.] $

It is easy to see that $p_i > 1/2$ for all $i$. The result is then I think intuitively clear by comparison to the asymmetric random walk on the same state space with $p_i = p > 1/2$ for all $i$; the limit in that case is a consequence of the strong law of large numbers.

Now write the hitting probability $h_i^k := \mathbb{P}(X_n = k \textrm{ for some }n \geq 0 \; | \; X_0 = i)$. I believe the result would follow if I could show that

(a) wherever $i \leq k$ we have $h_i^k = 1$, and
(b) for any fixed k, we have $\lim_{i \rightarrow \infty} h_i^k = 0$

Questions: (1) Does the result indeed follow from (a) and (b)?
(2) (b) is clear. How might one show (a)?

Iain
  • 107
  • Turns out this is exercise 1.5.2 from J. R. Norris' book Markov Chains. This is in the chapter called "Recurrence and transience". – Iain Nov 25 '11 at 16:16
  • First of all, I think that the definition of a.s. convergence you wrote is incorrect: $N_K$ can depend as well on $\omega$ since a.s. convergence means $$ \mathsf P{\omega: X_n(\omega)\to\infty} = 1.$$ Second, I'm interested why b is clear to you. – SBF Nov 26 '11 at 20:05
  • 1
    There is nothing recursive in the transition probabilities. // The definition of convergence to infinity is incorrect. // Like @Ilya, I am puzzled by the assertion that (b) is clear. // The comparison argument fails since $p_i\to1/2$. // Norris develops some techniques to show transience/recurrence. Which ones do you know, which ones did you try? – Did Nov 26 '11 at 20:28
  • Thanks for the responses.

    [Didier -- thanks for pointing out my "recursive" error; now corrected. I was thinking of a previous exercise on the same MC where one show (in the above notation) that $h_1^0 = 1 - 6/\pi^2$ by writing $u_i = h_{i-1}^0 - h_i^0$.]

    I understand now that my $N$ in the definition of a.s convergence ought to also depend on the initial state.

    I see now that the comparison argument won't work here.

    Will continue responding in a further comment.

     – Iain Nov 26 '11 at 23:19
  • The claim that (b) is clear is probably a little bold. In some sense I suppose it's sort of similar to (a), and therefore rather silly of me to claim it to be obvious! (a) says roughly "we always eventually hit anything to the right (since the overall motion is in that direction)", while (b) says roughly "go far enough out to the right and the chance of hitting some fixed thing to the left gets arbitrarily small". I've not really thought about this before -- some of the examples on the previous sheet used this "obvious" fact to get a boundary condition on a recurrence relation, for example. – Iain Nov 26 '11 at 23:36
  • Actually, re-reading my first comment I think I may be on to something now: The probability of returning to state $0$ is $1- 6/\pi^2 < 1$, so state $0$ is transient. The chain is irreducible, so every state is transient. Though the question is not very clear in this regard, I think we have $X_0 = 0$. With probability 1, we make only finitely many visits to $0$, so we at some stage forever leave the set $\lbrace 0 \rbrace$. Similarly we then a.s. forever leave $\lbrace 0,1 \rbrace$, and next forever leave $\lbrace 0,1,2 \rbrace$, and so forth. Hence $X_n \overset{a.s.}\rightarrow \infty$. – Iain Nov 26 '11 at 23:50

1 Answers1

3

In the last comment you gave all the necessary ideas for the solution, so let us summarize and formalize them.

Let us denote by $\mathscr X = \{0,1,2,\dots\}$ the state space, the set $A_k = \{0,1,2,\dots,k\}\subset \mathscr X$ and an events $$ \mathrm A_k = \{X_n \in A_k\text{ for infinitely many }n\} $$ and its complement $$ \mathrm B_k = \{\text{there is }N:X_n\in A_k^c \text{ for all }n\geq N\} $$ and note that $\mathrm B_{k+1}\subset \mathrm B_k$.

  1. $\mathsf P_i$-a.s. convergence $X_n\to \infty$ means that for any $i\in\mathscr X$ it holds that $\mathsf P_i\{\mathrm B_k\} = 1$ for all $k\geq 0$ or equivalently $$ \mathsf P_i\left\{\bigcap\limits_{k=0}^\infty\mathrm B_k\right\} = \lim\limits_{k\to\infty}\mathsf P_i\{\mathrm B_k\} =1. $$

  2. You've already noticed that $\mathscr X$ is a closed communicating transitive class. Thus $$ \mathsf P_i\{X_n = i\text{ for infinitely many }n\} = 0. $$ As a consequence $\mathsf P_i\{\mathrm A_k\} = 0$ since $A_k$ is finite, hence $\mathsf P_i\{\mathrm B_k\} = 1$ and $\lim\limits_{k\to\infty}\mathsf P_i\{\mathrm B_k\} =1$.

SBF
  • 36,041