1

We are learning about the exponential convergence of a Markov chain. We learned that if a Markov chain(represented by the transition matrix $P$) is a-periodic, irreducible and reversible then it converges exponentially and the rate of convergence(in term of the total variation norm)is the second largest eigenvalue of $P$.

I wonder if this is also true if $P$ is just a-periodic, irreducible and diagonalizable. If $P$ is diagonalizable then I can write $ P = S D S^{-1} \implies P^n = S D^n S^{-1} $ where $D$ is a diagonal matrix of the eigenvalues of $P$. We know that the magnitude of all eigenvalues is less than 1 except a single eigenvalue with a value of 1 which means that $ P^n = S D^n S^{-1} $ converges in the rate of the second largest eigenvalue of $P$.

Hope that someone can clear my confusion.

Eilif
  • 411
Tomer
  • 434

1 Answers1

1

Suggestion for this kind of problems: try to visualize the action of the linear operator in its eigenbasis.

Say $P$ is a diagonalizable $m\times m$ irreducible stochastic matrix. Let $\{\mathbf p,\mathbf v_2,\cdots\mathbf v_m\}$ be the $m$ eigenvectors associated with the eigenvalues $\{1,\lambda_2,\cdots\lambda_m\}$. As you state in the question, $|\lambda_j|<1 \quad \forall j$. Degeneracy in $\lambda_j$ allowed.

Apply $P^n$ to some probability distribution $\mathbf q = c_1\mathbf p+c_2\mathbf v_2 + \cdots + c_m\mathbf v_m$.

$$P^n \mathbf q = c_1\mathbf p+c_2 \lambda_2^n\mathbf v_2 + \cdots + c_m\lambda_m^n\mathbf v_m$$

As you can see, the distribution is converging exponentially to the steady-state distribution $\mathbf p$, with convergence rate determined by the largest $\lambda_j$ (second largest eigenvalue).

In a left stochastic matrix each column sums up to $1$, $\mathbf 1^T P=\mathbf 1^T$. Thus, $P \mathbf v_j = \lambda_j \mathbf v_j \Leftrightarrow \mathbf 1^T \mathbf v_j = \lambda_j \mathbf 1^T \mathbf v_j \Leftrightarrow \mathbf 1^T \mathbf v_j =0$ since $\lambda_j \neq 1$.

Together with, $\mathbf 1^T \mathbf q =1$ and $\mathbf 1^T \mathbf p =1$ we find $c_1 =1$.

G Frazao
  • 436