7

The problem is stated as the following:

Suppose we have a man walking along a pavement, whose width is 5. For each step forward, the drunk man can either move to the left, or to the right. Both outcomes are equally as likely. However, when the drunk man encounters the 5th position, he must return to the 4th, and when the drunk man encounters the 1st position, he can either go left or right. If he chooses to go right, the drunk man falls off, and the game ends.

The question is now:

  • How far on average will the walker walk?
  • What is the probability of the walker returning home after moving K positions.

I'll try to solve the latter question, since I believe it's easier for me. First of all, we have to form our transition matrix T.

$$ T = \begin{pmatrix} 1 & 0 & 0 & 0 & 0 & 0 \\ 1/2 & 0 & 1/2 & 0& 0 & 0 \\ 0 & 1/2 & 0 & 1/2 & 0 & 0 \\ 0 & 0 & 1/2 & 0 & 1/2 & 0 \\ 0 & 0 & 0 & 1/2 & 0 & 1/2 \\ 0 & 0 & 0 & 0 & 1 & 0 \\ \end{pmatrix} $$

Where the rows are indexed as E,1,2,...,5 where E stands for "end", and thus the random walk terminates if the drunk man falls into this position.

To answer what the probability for the walker to arrive at home after K positions, we can note one obivous fact

  • If the K moves are odd, the probability of returning home is $0$.

We might need to use this later to check whether our answer is reasonable.

More generally, we can write the probability of returning home after $K$ positions, which we can denote as the event $H_K$, as:

$$ P(H_K) = \begin{pmatrix} 0& 0 & 0 & 1 & 0 & 0 \end{pmatrix} T^k \cdot \begin{pmatrix} 0& 0 & 0 & 1 & 0 & 0 \end{pmatrix} $$

However, I don't really know if there's any way to create a closed form expression for this probability given this. It's not really that easy to calculate $T^k$ for some general $k$ too.

I tried to plot the probabilities of the positions after 30 steps in Python, and got this graph:

enter image description here

Notice, position $1$ is what I previously labeled as position $E$. Position $2$ in the graph represents position $1$ in my previous definition, and so forth. I also tried to plot the probabilities after 100 steps, and it really looks some exponential decay regarding the probabilites for positions. So I assume there might be some easy expression to derive from this.

For the question regarding how far the walker will get on average, I've really no idea how to start. I'm thinking that I want to find the expected value of arriving at position $E$ given that our starting position is $3$. However, I don't really know how to express this in terms of our transition matrix $T$. I'd be glad if anyone could share their ideas for this problem.

Thanks in advance.

Tanamas
  • 1,837
  • 1
    For the first question, let $E_i$ be the expected walk length if the drunk starts at position $i$. Then $$E_1=1+\frac12E_2\E_2 =1+\frac12E_1+\frac12E_3\ \vdots\ E_5=1+E_4.$$ Solve the five equations. – MJD Dec 20 '22 at 11:49
  • @MJD I assume you used the law of total expectation to get the equations, or am I in the wrong? – Tanamas Dec 20 '22 at 13:27
  • I don't know the law of total expectation, but maybe? What I wrote matches the formula in Wikipedia that David Clyde linked to: $$E=(I-T)^{-1} \cdot \mathbf 1$$ where $\mathbf 1$ is a column of 1's. – MJD Dec 20 '22 at 13:35
  • @MJD Okay, I'll check the link! Thank you either way :) – Tanamas Dec 20 '22 at 13:39
  • 1
    @MJD I don't think $E$ is well defined in this case, since $(I-T)$ contains a row (the first one) with only $0$'s. Update: I now see that they used $Q$ in the wikipedia article instead of $T$, where $Q$ is just my $T$ without the first row and column. – Tanamas Dec 20 '22 at 13:54
  • 1
    Yes, I used the law of total expectation to derive the equations. – MJD Dec 20 '22 at 14:13

1 Answers1

4

This is called an absorbing Markov chain, because there's at least one "absorbing" state (impossible to leave once we arrive there) and all states can eventually reach an absorbing state. That Wikipedia page has a lot of good information about computing various properties of absorbing MCs.

How many steps will the walker take on average?

This is the "expected time until absorption". The Wikipedia article directly provides a formula here.

I reused my Mathematica script from this more complicated question and I found that the expected number of steps is $[9, 16, 21, 24, 25]$ depending where you start. That means if you start in position 1 then the expected number of steps is 9; if you start in position 2 then the expected number of steps is 16; and so on.

Solving the equations suggested by @MJD will be equivalent to using Wikipedia's formula here; the formula involves multiplying by a matrix inverse, and you'll instead end up just solving that linear system by hand. In this case the system is quite simple so it's totally doable to check the result that way if you want.

What's the probability of finishing in exactly $K$ steps?

For the probability of finishing in exactly $K$ steps, I think your idea using $T^K$ is correct. The $(j,0)$ entry of $T^K$ tells you the probability of moving from state $j$ to state $0$ in $\le K$ steps, so what you really want is $T^K_{j,0} - T^{K-1}_{j,0}$.

If you're willing to accept numerical solutions, then there's no problem here. You can compute $T^K$ in approximately $\log(K)$ matrix multiplications (see exponentiation by squaring) so you could answer the question even for extremely large $K$.

If you really want a closed form then you could observe that $T$ essentially provides you with a linear recurrence for the entries of $T^n$ in terms of the entries of $T^{n-1}$. In general could try to solve that recurrence using pencil-and-paper or using a CAS like Mathematica. I'm kinda suspicious this one doesn't have a pretty formula, but I also didn't work on it very hard. I did try asking Mathematica and absolutely nothing good came out:

enter image description here

Here are some sample numeric results in case you want to compare against answers from another source:

3       0.125
5       0.09375
7       0.078125
9       0.0683594
11      0.0610352
13      0.0549316
15      0.0495911
17      0.0448227
19      0.0405312
...
101     0.000661656
201     0.00000437789
David Clyde
  • 5,251
  • 1
    Thank you! I'll make sure to check the links regarding the absorbing Markov chains and compare the average walking distance with what I computed using the equations by @MJD. – Tanamas Dec 20 '22 at 13:33
  • 1
    @Tanamas Makes sense. I edited to add a code link + the actual answer for expected value instead of just the formula, so you can confirm you're getting the right result. – David Clyde Dec 20 '22 at 18:58