1

Let $\{X_n\}$ be a Markov chain on $\mathbb{N}_0$ such that $p(x,x+1)=p(x,0)=1/2$ for all $x\in \mathbb{N}_0$. Find $E(T_0~|~X_0=1),$ where $T_0=\inf\{k\geq 0:~ X_k=0\}.$

Attempt. If $g(x)=E(T_0~|~X_0=x),~x\in \mathbb{N}_0$ then, according to one-step analysis:

$g(x)=\frac{1}{2}g(x+1)+\frac{1}{2}g(0)+1,~x\in \mathbb{N}$ and $g(0)=0$, that is $g(x+1)=2g(x)-2,~x\in \mathbb{N}$, with general solution $g(x)=c2^x+2,~x\in \mathbb{N}$. Since $g(0)=0$, we get $c=-2$, so $g(x)=2-2^{x+1},~x\in \mathbb{N}_0$, while $g(x)\geq 0$, a contradiction. Where seems to be the problem in the above solution?

Thank you in advance for the help.

Nikolaos Skout
  • 5,329
  • 2
  • 16
  • 40
  • 1
    You write $x \in \mathbb{N}$ for the equation $g(x + 1) = 2g(x) - 2$, but then invoke the condition $g(0) = 0$, which has nothing to do with the equation, since $x \in \mathbb{N}$. I think you need to use a different condition here. Any idea which one? – Ritz Feb 08 '17 at 13:39
  • My concern was exactly this, that is how to set $x=0$ to get $c$, since the equation (and the solution) holds for $x\in \mathbb{N}={1,2,\ldots}$. Any hint for the necessary condition needed? – Nikolaos Skout Feb 08 '17 at 19:23
  • 1
    Notice that the initial state $x$ does not change $T_0$, as long as $x \ge 1$. So, we conclude that $g(x) = g(y)$ for all $x,y \ge 1$. – Ritz Feb 09 '17 at 09:50

1 Answers1

1

Notice, that probability of reaching state $0$ from state $1$ in exactly $k$ steps is $\left(\frac{1}{2}\right)^k$ (the only way is $1\to2\to3\to...\to k \to 0 $), so $$E(T_0|X_0=1)=\sum_{n=1}^{\infty}\frac{n}{2^n}=2$$ A few days ago there was question about calculating similar sum: link

Community
  • 1
Jaroslaw Matlak
  • 4,895