0

Suppose $X$ is an irreducible Markov Chain on a discrete state space $E$. I would like to prove that

$$ P_x[\tau_x^1 < \infty]=1 $$

where $\tau_x^1=\inf\{n>0: X_n=x\}$.

Is it necessary to know if an invariant distribution $\pi$ exists or is it sufficient to know that $X$ is irreducible?

Thank you for the help.

Edit: my attempt

Since $X$ is irreducible, we have that $\pi(\{x\})>0$ for all $x\in E$. Thus at every step $X$ can take all the values in $E$. If I consider the number of steps to reach an $x\in E$ for the first time as a Geometric distribution of parameter $\pi(\{x\})$, since its expected value if finite, I can conclude that the $P_x[\tau_x^1< \infty]=1$. So, we need also the fact that an invariant distribution exists.

Enrico
  • 553
  • Are you also assuming that the state space is finite? For exaxmple if $X_n$ is defined on the natural numbers with $\mathbb P(X_{n+1} = i+1 \mid X_n=i)=1$ for all $i$, clearly the claim is false. – Math1000 Aug 09 '23 at 22:26
  • That condition plus others would imply an invariant distribution exists, so it is a necessary condition, but whether it is necessary to know about invariance before showing Harris recurrence probably depends on the specifics of uoir problem. – Taylor Aug 10 '23 at 02:26
  • @Math1000 That violates the assumption of irreducibility – Taylor Aug 10 '23 at 05:47
  • @Taylor I know that if $X$ is irreducible then $X$ has at most one invariant distribution (Theorem 17.50 of Probability Theory by A. Klenke, 3rd version). Thus, If I know that an invariant distribution exists, I thought my attempt could prove the statement. What do you think? Thanks. – Enrico Aug 10 '23 at 07:26
  • 1
    Sorry, wrong counterexample, but irreducibility is not a sufficient condition for an invariant distribution to exist. – Math1000 Aug 10 '23 at 12:28
  • @Math1000 If now we suppose that $X$ is irreducible AND that an invariant distribution exists, are we able to prove the claim in my post? Is my attempt correct? Thanks for your time. – Enrico Aug 10 '23 at 15:43
  • 2
    It is not sufficient to know that $X$ is irreducible; consider $X_n$ on ${2,3,4,\dots}$, with $\mathbb{P}(X_{n+1} = i+1 | X_n = i) = 1- \frac{1}{i^2} = 1-\mathbb{P}(X_{n+1} = 2 | X_n = i)$. Then $X$ is clearly irreducible and aperiodic, but $\mathbb{P}_2(\tau_2^1 = \infty) = 1/2$. – Julius Aug 11 '23 at 11:31
  • @Julius How do you obtained $1/2$? Sorry if it is elementary stuff. I agree that it is not sufficient. And if I knew that an invariant distribution exists, could my above attempt holds? Thanks. – Enrico Aug 11 '23 at 11:39
  • Well, https://math.stackexchange.com/questions/18179/finding-value-of-the-infinite-product-prod-bigl1-frac1n2-bigr this answers the first question of my comment above. – Enrico Aug 11 '23 at 12:03
  • 1
    @Enrico $ \mathbb{P}2(\tau_2^1 = \infty) = \prod{i=2}^\infty \mathbb{P}2(X{i} = i+1 | X_{i-1} = i) = \prod_{i=2}^\infty (1-\frac{1}{i^2}) = 1/2 $ (the product telescopes when you write $(1-1/i^2)=(i+1)(i-1)/i^2$. – Julius Aug 11 '23 at 12:04
  • @Julius Thank you. So, If I knew that an invariant distribution exists, could my above attempt holds? What do you think? – Enrico Aug 11 '23 at 12:08
  • @Enrico It is not necessary to have an invariant distribution. The simple symmetric random walk on $\mathbb{Z}$ has no invariant distribution, but it is recurrent. With regards to your proof, I'm not quite convinced that the first hitting time is Geometrically distributed. The markov chain on two states where $1\rightarrow 2$ and $2 \rightarrow 1$, each with probability 1, has invariant distribution $\pi = (1/2,1/2)$, but the first hitting time of state 1 starting from $\pi$ is $\text{Bernoulli}(1/2)$. The issue is that the probability of hitting $x$ isn't independent in each step. – Julius Aug 11 '23 at 12:15
  • @Julius The first hitting time is thus finite a.s. if it is Bernoulli(1/2). Am I wrong? – Enrico Aug 11 '23 at 12:59
  • Let us continue this discussion in chat. – Julius Aug 11 '23 at 13:12

0 Answers0