0

I'm currently working on an practice question from my notes. But I'm not quite understanding the idea of how to prove that something is a Markov chain.

Let {$X_i$}, $i = 1,2,...$, be a Markov chain on $E =$ {$0,1,2,3$}, with transition matrix $$P= \begin{pmatrix} 1/4 & 0 & 1/2 & 1/4 \\ 0 & 1/5 & 0 & 4/5 \\ 0 & 1 & 0 & 0 \\ 1/3 & 1/3 & 0 & 1/3 \\ \end{pmatrix}$$ A new process is defined by $Z_n = 0$ if $X_n = 0$ or $1$ and $Z_n = X_n$ if $X_n = 2$ or $3$.

Firstly, how do I find $P(Z_{n+1}=2|Z_n=0, Z_{n-1}=2)$ and $P(Z_{n+1}=2|Z_n=0, Z_{n-1}=3)$? Could anyone show me how this is done?

Secondly, the question also requires me to answer if {$Z_n$} is a Markov chain.

The approach I would like to take to the second part of the question would be to show that the value of $Z_{n+1}$ only depends on $Z_n$ and not $Z_{n-1}$, hence in this case $$P(Z_{n+1}=2|Z_n=0, Z_{n-1}=2) = P(Z_{n+1}=2|Z_n=0, Z_{n-1}=3)$$

If this is true then it would imply that {$Z_n$} is Markov chain as it $Z_{n+1}$ is only dependent on $Z_n$ and not $Z_{n-1}$.

Am I on the right track? If not, how do I go about showing whether or not {$Z_n$} is a Markov chain?

Kexis Reeves
  • 389
  • 1
    For your first question: What information can you infer if you are given $Z_{n-1}=2$ and then $Z_n=0$? – Michael Oct 12 '15 at 03:00
  • Hmm, do I relate it back to X? Like for $Z_n = 0$, $X_n = 0$ or $1$ and for $Z_{n-1} = 2$, $X_{n-1} = 2$ – Kexis Reeves Oct 12 '15 at 03:13
  • 1
    Yes, you need to relate it back to $X_{n-1}$ and $X_n$. But in this case you can determine exactly the values of $X_{n-1}$ and $X_n$. – Michael Oct 12 '15 at 03:30
  • I'm lost here... I don't get how I can determine the values of $X_n$ and $X_{n-1}$ – Kexis Reeves Oct 12 '15 at 04:14
  • Why are you lost? I thought you already determined $X_{n-1}$. So you are selling yourself short. – Michael Oct 12 '15 at 04:18
  • 1
    If $X_{n-1}=2$ then does that mean that $X_n = 1$? (I arrived at this conclusion because given that $X_{n-1}$ = 2, then $X_n$ is definitely 1 according to the matrix) – Kexis Reeves Oct 12 '15 at 04:30
  • So, since you write "then $X_n$ is definitely 1 according to the matrix," you indeed answered your own question. So now you can calculate $Pr[Z_{n+1}=2|Z_n=0, Z_{n-1}=2]$. – Michael Oct 12 '15 at 04:54
  • That would be 0, and for the second equation where $Z_{n -1}=3$ it would be 1/2? In light of this result, is my approach to checking if {$Z_n$} is a Markov chain correct? – Kexis Reeves Oct 12 '15 at 05:00
  • See the answer here: http://math.stackexchange.com/questions/27507/transformation-of-state-space-that-preserves-markov-property/27538#27538 –  Oct 12 '15 at 19:16

0 Answers0