3

Following up on two previous questions: Expected number of steps for reaching a specific absorbing state in an absorbing Markov chain and Expected time till absorption in specific state of a Markov chain

Suppose you have an absorbing Markov chain with two absorbing states. You can calculate the probability of reaching either absorbing state, you can calculate the expected number of steps before being absorbed, you can construct a conditional matrix that removes an absorbing state, and you can calculate expected step count for that. For example, consider a minimal drunkard's walk with four states, A, B, C, and D, where A and D are absorbing. We have

$$ P = \begin{bmatrix}0 && 0.5 && 0.5 && 0 \\ 0.5 && 0 && 0 && 0.5 \\ 0 && 0 && 1 && 0 \\ 0 && 0 && 0 && 1 \end{bmatrix}, N = \begin{bmatrix}4/3 && 2/3 \\ 2/3 && 4/3\end{bmatrix}, NR = \begin{bmatrix}2/3 && 1/3 \\ 1/3 && 2/3\end{bmatrix} $$

Or if we eliminate column three

$$ P = \begin{bmatrix}0 && 1 && 0 \\ 0.5 && 0 && 0.5 \\ 0 && 0 && 1 \end{bmatrix}, N = \begin{bmatrix}2&&2\\1&&2\end{bmatrix}, NR = \begin{bmatrix}1\\1\end{bmatrix} $$

Skipping some of the other math, we eventually get P(reach A from B) = 2/3, P(reach D from B) = 1/3, E(steps starting from B) = 2, E(steps starting from B|end in A) = 3, and E(steps starting from B|end in D) = 4. But 2/3 * 3 + 1/3 * 4 is an expected 10/3 steps, not the 2 from the unconditional matrix.

Is there something wrong about the conditional matrix logic? Or else why does this appear to violate the law of total expectation?

StubbornAtom
  • 17,052
user3140990
  • 31
  • I think your matrix has C absorbing rather than A, but this does not really matter. But your elimination of the third column and row does not lead to the same walk conditioned on not visiting $C$ and so it should be less of a surprise that it produces different results. – Henry Jun 10 '20 at 16:51
  • @Henry I had actually implicitly meant that to be in canonical form, with the rows/columns ordered B, C, A, D. But my issue is that based on those other two questions, it looks like I'm doing my math correctly. But it violates my intuition that this would follow the law of total expectation. So if it produces different results, does that mean the other two answers are wrong? – user3140990 Jun 10 '20 at 17:55
  • With your version of the alphabet, let's suppose we count steps in a path from B to A as normal but steps in a path from B to D as $0$ steps; then the expected number of steps would be $\frac{10}{9}$. Now let's suppose we count steps in a path from B to D as normal but steps in a path from B to A as $0$ steps; this time the expected number of steps would be $\frac{8}{9}$. So the total expected number of steps would be $\frac{10}{9}+\frac{8}{9} =2$ as usual. – Henry Jun 10 '20 at 22:09
  • The expected number of steps on a path from B to A conditioned on that result would be $\frac{10/9}{2/3}=\frac53$ and the expected number of steps on a path from B to D conditioned on that result would be $\frac{8/9}{1/3}=\frac83$ and you would then get the total expected number of steps being $\frac53\times \frac23 +\frac83 \times \frac13=2$ as before. Your $3$ and $4$ are far too big – Henry Jun 10 '20 at 22:09
  • If it helps: $1 \times \frac1{2}+3 \times \frac1{2^3}+5 \times \frac1{2^5}+\cdots= \frac{10}{9}$ while $2 \times \frac1{2^2}+4 \times \frac1{2^4}+6 \times \frac1{2^6}+\cdots=\frac{8}{9}$ – Henry Jun 10 '20 at 22:17

1 Answers1

2

I expect OP doesn't care about this question anymore, but future readers might, so:

As discussed in the comments, your paradox is coming from an incorrect method used to calculate E(steps starting from X | end at Y) for a transient state X and absorbing state Y. In particular, the accepted answer on your previous question is incorrect. I've added a new answer on that question which explains a correct method (plus examples and sample Mathematica implementation), and that new method finds E[steps starting from B | end at A] = $\frac 5 3$ and E[steps starting from B | end at D] = $\frac 8 3$. In the comments I see Henry took advantage of this extreme simplicity of this Markov chain to obtain the same results "by hand". Anyway, using these corrected conditional expected values, we no longer run into the contradiction you asked about.

David Clyde
  • 5,251