1

The definition that I have of a Transition Matrix for a Markov Chain is:

A transition matrix is regular if some power of it is positive.

Doesn't this mean though that in theory, you could keep calculating powers forever, because at some point one of the future transition matrices could become positive (assuming it's not already)?

Is there a more exact way checking? I have a transition matrix that I need to show is non-regular. For all the powers I've tried, it contains a 0, but for all I know, that could change if I kept checking forever.

The matrix:

$\begin{bmatrix}0.7 & 0 & 0.3\\0 & 1 & 0 \\ 0.2 & 0 & 0.8\end{bmatrix}$

2 Answers2

2

Solution 1: Denote by $P$ the transition matrix and by $P^n = (P^n(i,j))_{i,j=1,2,3}$ the $n$-th power of $P$. Let $(X_n)_{n \in \mathbb{N}}$ be a Markov chain with transition matrix $P$. It is known that

$$P^n(i,j) = \mathbb{P}^i (X_n=j), \tag{1}$$

i.e. $P^n(i,j)$ is the probability that the Markov chain starts at $x=i$ and moves to $x=j$ in $n$ steps. However, since $\mathbb{P}^2(X_1 = 2) = 1$, it follows (e.g. from the Markov property) that

$$\mathbb{P}^2(X_n = 2)=1$$

for all $n \in \mathbb{N}$. (Hint: Draw a graph of the chain!) This means that the Markov chain stays at $x=2$ whenever it starts at $x=2$. Therefore, $(1)$ is not satisfied for any $n \in \mathbb{N}$ for $i=2$, $j=1$.

Solution 2: Given two matrices $P,Q$ of the form

$$ P = \begin{pmatrix} p_{11} & 0 & p_{13} \\ 0 & 1 & 0 \\ p_{31} & 0 & p_{33} \end{pmatrix} \qquad \quad Q = \begin{pmatrix} q_{11} & 0 & q_{13} \\ 0 & 1 & 0 \\ q_{31} & 0 & q_{33} \end{pmatrix},$$

it is not difficult to see that $P \cdot Q$ is again of this form, i.e.

$$R := P \cdot Q = \begin{pmatrix} r_{11} & 0 & r_{13} \\ 0 & 1 & 0 \\ r_{31} & 0 & r_{33} \end{pmatrix}. \tag{2}$$

Since the transition matrix $P$ is of the form, we find that $P^2$ is also of this form and, by iteration, that $P^n$ is of the form $(2)$ for any $n \in \mathbb{N}$. Consequently, the matrix is not regular.

saz
  • 120,083
1

Your Markov chain is reducible, since there is no way to enter or leave state 2: State 2 does not communicate with either states 1 or 3.

In general, you might find a solution using the Perron-Frobenius theorem relating the eigenvalues of a matrix to its reducibility. For example, the matrix you gave us has the eigenvalue 1 with multiplicity 2. For a matrix to be irreducible, I think I remember the eigenvalue 1 must be simple.

Bernhard
  • 1,134
  • That sounds right. This is related to spectral clustering. If you have a graph represented in laplacian matrix form, the number of disconnected subgraphs (or connected components) is equal to the multiplicity of the eigenvalue 0. – william_grisaitis Apr 22 '21 at 04:10