Expected length of a sequence that contains all words of a given length.

Question

Fix some alphabet $\Sigma$ and a positive integer $n$. What is the expected number of random letters drawn from $\Sigma$ until all length-$n$ words are present?

For example, let $\Sigma = \{0,1\}.$ Then the string "10" contains all possible 1-letter words, ie "0" and "1". The expected length for a random string is simply the coupon-collector problem with 2 cards... so the expected length is 3.

But it's more complicated when words can overlap. For the same alphabet and $n=2$ we can see that the string "00110" has all 2-letter words, and I claim that's the shortest string that does. But what's the expected length of a random string that contains all four strings "00", "01", "10", and "11"? The usual coupon-collector approach doesn't seem to work.

Answer 1

Well, here's a solution, but it's not pretty.

As noted in my answer to Cover time chess board (king), there is in principle no difficulty in answering this question. Calculating the expected cover time of the set of patterns $\cal S$ reduces to calculating the expected hitting time of every possible non-empty subset $A$ of $\cal S$:

$$\mathbb{E}(\text{cover time})=\sum_A (-1)^{|A|-1} \mathbb{E}(T_A)\tag1$$

These hitting times are defined by $T_A=\inf(n\geq 0: X_n\in A)$. These, in turn, can be found using a technique from Section 14.3 (Patterns III) of Problems and Snapshots from the World of Probability by Blom, Holst, and Sandell.

For instance with strings of length two we find $$\mathbb{E}(T_{00,01,10,11})=2$$ $$\mathbb{E}(T_{00,01,10})=\mathbb{E}(T_{01,10,11})=5/2,\quad \mathbb{E}(T_{00,10,11})= \mathbb{E}(T_{00,01,11})=9/4$$ $$\mathbb{E}(T_{00,01})=\mathbb{E}(T_{00,10})=\mathbb{E}(T_{00,11})=\mathbb{E}(T_{01,10})=\mathbb{E}(T_{01,11})=\mathbb{E}(T_{10,11})=3$$ $$\mathbb{E}(T_{00})=\mathbb{E}(T_{11})=6,\quad \mathbb{E}(T_{01})=\mathbb{E}(T_{10})=4$$

Plugging all this information into (1), we find that the expected time to see all four patterns $00,01,10$, and $11$ is $19/2=9.5$.

Just for fun, I've calculated the expected time to see all eight patterns of length three to be ${89259/ 3640}= 24.5217$.

Answer 2

Update: This is wrong - sorry, I misread the question, I understood that we were interested in getting all permutations (words with different letters).

---

For $n=2$ it's not hard to see that the expected length is 5 : expected length of first run (2) + expected length of second run (2) + 1. For $n>2$ it seems more difficult.

Applying the coupon-collector approach, one would say that the expected number of tries to collect the $m$-th permutation (among $n!$ permutations) is $1/p_m = n^n/(n!-m)$ and so

$$E_n = n^n \left( \frac{1}{1} + \frac{1}{2} + \dots +\frac{1}{n!} \right)=n^n H({n!}) $$

This, of course, is an approximation, because it assumes that each new letter of the sequence gives a new $n$-word, independent of the previous ones; but the overlapping must introduce some dependence. It's not clear, however, the sense of this dependence. And this, together with the fact that for large $n$ the probability of a word being a permutation (even for the "first coupon") is low, leads to think that the approximation might be reasonable. Numerical evidence sees to agree.

Eg: for $E_5 = 16777.7$, empirically I get $\approx 16730$.

---

Update: Actually we are not interested in getting all permutations ($n!$) but all words ($n^n$). The above coupen-collector approach can also be applied, giving us $E_n \approx n^n H(n^n) \sim n^{n+1} \log(n) $, but the independence assumption is less justified, and the result less precise.

We can regard each word as a vertex on a directed graph, or state of a doubly-stochastic Markov chain ($n$ equal non null entries in each row-column). We are asking then about the cover time of this chain. This could help to locate material. For example, here (section 2.3) it is stated as a known result (but references are missing) the asympotic

$$E_n \sim n^n \log(n)$$

Also, see here (page 11.3.3).