4

I'm trying to read Knuth's Notes on Open Addressing, and I don't quite follow the proof of Lemma 2.

The set-up We're thinking about hashing $k - 1$ keys into a size $N$ array, with collision resolution via linear probing. In particular, we're considering sequences of hash values $a_1, ..., a_{k-1}$ ($1 \leq a_i \leq N$). All we keep track of is variables $X_1, ..., X_N$, where $X_i = 0$ if the $i$th entry of the array is open, and $X_i = 1$ if the $i$th entry of the array is storing a key.

Example Suppose that $N = 5$, $a_1, a_2, a_3 = 1, 4, 1$. Then $X_1 = 1$, $X_2 = 1$, $X_3 = 0$, $X_4 = 1$, $X_5 = 0$.

Definition $\begin{bmatrix} n \\ k \end{bmatrix}$ is the number of sequences $a_1, ..., a_k$ ($1 \leq a_i \leq n$) such that no overflow step occurs. The overflow step is when a key hashes to an index near the end of the array, but because $X_{\text{hash}} = X_{\text{hash} + 1} = ... = X_N = 1$ already, we have to try to insert the key at the beginning of the array $X_1$.

In Lemma 1 Knuth proves that, for $0 \leq k \leq n + 1$, $\begin{bmatrix} n \\ k \end{bmatrix} = (n + 1)^k - k (n + 1)^{k - 1}$.

Definition $Q(N, k, t)$ is the number of sequences $a_1, .., a_{k - 1}$ ($1 \leq a_i \leq N$ satisfying (*): $X_{N - t - 1} = 0$, $X_{N - t} = ... = X_{N - 1} = 1$, $X_N = 0$.

Proof of lemma 2 Lemma 2 states that $$Q(N, k, t) = \begin{bmatrix} N - t - 2 \\ k - t - 1 \end{bmatrix} \begin{pmatrix} k - 1 \\ t \end{pmatrix}.$$

If $a_1, ..., a_{k-1}$ satisfies (*), then there is a subsequence, of length $t$, which falls into the range $X_{N - t}, ..., X_{N - 1}$. The complementary subsequence, of length $k - 1 - t$, falls into the range $X_1, ..., X_{N - t - 2}$. Knuth counts the number of each such subsequence, and the number of combining these two subsequences together to produce $a_1, ... a_{k - 1}$.

My question He seems to argue that

$$Q(N, k, t) = \begin{bmatrix} t \\ t \end{bmatrix} \begin{bmatrix} N - t - 2 \\ k - t - 1 \end{bmatrix} \begin{pmatrix} k - 1 \\ t \end{pmatrix}.$$

But $\begin{bmatrix} t \\ t \end{bmatrix} = (t + 1)^t - t(t+1)^{t-1} = (t + 1)^{t - 1} \not = 1$, so I don't see how the conclusion holds.

CruiskeenLawn
  • 43
  • 4
  • He says that by convention $\begin{bmatrix} n \ 0 \end{bmatrix}=1$. Did he forget also to say "by convention $\begin{bmatrix} n \ n \end{bmatrix}=1$" ? – Maczinga Mar 18 '17 at 17:30
  • @Maczinga I've now included the definition of $\begin{bmatrix} n \ k \end{bmatrix}$. It's not consistent with this definition to define $\begin{bmatrix} n \ n \end{bmatrix} = 1$ "by convention". – CruiskeenLawn Mar 18 '17 at 20:18
  • @D.W. Thank you for your feedback, I have edited the question so that, hopefully, it is now more self contained. – CruiskeenLawn Mar 18 '17 at 20:20

1 Answers1

3

I think that the notes have been reported incorrectly. Indeed CruiskeenLawn, you are right. I've found some very old notes here which report the correct result. Indeed, in your copy, they simply forgot to add $\left[\begin{matrix} t\\ t\end{matrix}\right]$ term. The Lemma you are looking for is on Page 2.

Maczinga
  • 460
  • 2
  • 8