2

I have the following problem:

If I have a markov process with stationary distribution. The state space for the MP is integers. I also know that $P_{i,j}>0$ for all i and j. It is also given that $X_0=0$.

The question is how to show that $X_0^4,X_1^4,X_2^4,...$ is not necessarily a Markov process with stationary distribution by giving an example?

The solution I am thinking about involves the following example:

We know that $P_{i,j}>0$ for the original Markov process, which means I can go from state 0 to any other state with positive probability. Also I can go from any state to state 0 with positive probability.

However, in the new process, I cannot reach the negative states. But I am not sure how to incorporate this fact to give an example of why the new process need not to be a Markov process with stationary distribution.

I know that a Markov process need to satisfy two conditions:

  1. Markov Property
  2. Stationary Distribution

I think the trick is to find an example that violates the stationary distribution condition, but I still cannot figure it out.

Also, the question is asking for conditions that are sufficient for $X_0^4,X_1^4,X_2^4,...$ to be a Markov process.

For this part I am thinking that if I have the following conditions, then the new process would be a Markov process:

  1. $P_{i,j}>0$ for all $i,j\ge0$
  2. $P_{i,j}=0$ for all i $<$ 0

Do you agree with my reasoning? and do you have any idea how to give such an example? Thanks.

Solver
  • 173
  • Just to clarify: what is $P_{i,j}$? Usually, $P_{i,j}>0$ means that you can move from state $i$ to state $j$ in one step, not finitely many. Also, the sentence "Thus this Markov processes is recurrent." is wrong, but irrelevant for this question. –  Feb 24 '13 at 23:59
  • You are right. I noticed that I made a mistake in writing the problem, I'll correct it. Thanks. – Solver Feb 25 '13 at 00:37
  • I edited it, I think the wording now is more accurate. – Solver Feb 25 '13 at 00:41

1 Answers1

1

A Markov chain has a stationary probability distribution if and only if there is a positive recurrent state. Since all states communicate, they must all be positive recurrent. Now the return time to 0 is the same for both processes $X_n$ and $X^4_n$, thus $X^4_n$ would also have a stationary distribution provided that it is Markov.

The question here, and my answer here will help you decide when $X^4_n$ is Markov.

Community
  • 1