3

Is the Simple Uniform Hashing Assumption (SUHA) sufficient to show that the worst-case time complexity of hash table lookups is O(1)?

It says in the Wikipedia article that this assumption implies that the average length of a chain is $\alpha = n / m$, but...

  • ...this is true even without this assumption, right? If the distribution is [4, 0, 0, 0] the average length is still 1.
  • ...this is a probabilistic statement, which is of little use when discussing worst case complexity, right?

It seems to me like a different assumption would be needed. Something like:

The difference between the largest and smallest bucket is bounded by a constant factor.

Maybe this is this implied by SUHA? If so, I don't see how.

aioobe
  • 217
  • 1
  • 9
  • A hashtable with $ [4, 0, 0, 0] $ probabilities, would generate an average chain length of $n$, not $n/m$, as it will chain all the elements together. Think of average chain length like this: what's the expected length of a chain that a random element $x$ would belong to. – Ameer Jewdaki Nov 25 '19 at 15:44
  • The average of 4, 0, 0, 0 is 1. Perhaps I expressed myself poorly in the question. Do you have a better way of putting it? – aioobe Nov 25 '19 at 16:04
  • it's the weighted average, in your example the weight of zero-length chains is zero, so the overall average is $4$ not $1$. – Ameer Jewdaki Nov 25 '19 at 16:13
  • plus, anything that holds in expection, i.e. $\alpha = n/m$, can be converted to a probabablistic bound by Markov inequality: $P(\text{query time of x}\ge t) \le E(\text{chain length of x})/t =n/(m t)$, – Ameer Jewdaki Nov 25 '19 at 16:16
  • Why is it the weighted average? I don't think it's correct to assume that one will only look up keys that exist in the table. Regarding probabilistic bounds, is that relevant when discussing worst-case time complexity? – aioobe Nov 25 '19 at 16:59
  • A nice supplementary answer to this question: https://cs.stackexchange.com/a/469/110729 – Chris Tang Sep 10 '20 at 14:07

3 Answers3

1

I think the answer is no, SUHA does not imply anything regarding worst-case time complexity.

The bottom line is that

  • the hashing is (even though uniform) still viewed as random, and
  • the keys are unknown.

Regardless of how small the probability is for all keys to hash to the same bucket, it's still a theoretical possibility, thus the theoretical worst-case is still O(n).

When discussing hash tables and complexity, I think this is mentioned briefly, regarded as irrelevant in practice and then the discussion moves on to expected run time complexity, and we're in O(1) land.

With the assumption mentioned in the question...

The difference between the largest and smallest bucket is bounded by a constant factor.

...(which is strictly stronger than SUHA) it can indeed be shown that the worst-case run time complexity is O(1). This assumption implies that even the largest bucket is within a constant factor of the load factor limit. This in turn means all linked lists are bounded by a constant and we get O(1) as worst case.

This assumption however is somewhat unreasonable. While it's quite probable that it holds true for any descent hash function and non-pathological set of keys, how would one guarantee such property? One would have to push this assumption further down and make constraints on what keys are allowed to be stored in the hash table and so on. This is what makes it unreasonable in a sense.

aioobe
  • 217
  • 1
  • 9
  • It is actually quite low probability that the difference between the largest and smallest bucket is bounded by a constant factor. – jbapple Nov 24 '19 at 20:13
  • How come? Could you elaborate? – aioobe Nov 24 '19 at 20:49
  • See https://cmurandomized.wordpress.com/2011/02/02/lecture-8-balls-and-bins/ – jbapple Nov 25 '19 at 14:42
  • I've read that page twice without finding any theorem of a formula for shortest path. I'm asking, because the expected length of the longest path, divided by the expected length of the shortest path could still be constant, right? – aioobe Nov 25 '19 at 15:14
1

Not only is SUHA insufficient to show that the largest bucket has size $O(\alpha)$, but if $m = n$, then there is a high probability that for any particular set of keys, the longest chain has length $\Theta(\lg n / \lg \lg n)$. There is a proof of this in CLRS and many proofs available in lecture notes.

jbapple
  • 3,380
  • 17
  • 21
  • But what's the expected length of the shortest chain? – aioobe Nov 24 '19 at 20:48
  • As long as you have fewer items than buckets, the expected length of the shortest chain is 0. – gnasher729 Nov 24 '19 at 20:49
  • Up until you have $\Omega(m \lg m)$ items, the expected length of the shortest chain is 0. This is called the "coupon collector's problem". – jbapple Nov 25 '19 at 14:45
  • Ok. Hmm.. This might be key, but I'm still unsure. It seems we're discussing the same thing in both comment threads here, but what I'm questioning here is basically that θ(lg n / lg lg n) for longest chain implies that it's not a constant factor more than the shortest chain. It quite possibly is, in fact I'd be surprised if it's not, but as far as I can see, the proof is not complete without that last bit. – aioobe Nov 25 '19 at 15:21
  • Are you saying that you think even the average chain is also $\Theta(\lg n / \lg \lg n)$? – jbapple Nov 25 '19 at 16:29
  • No. Let's assume for the sake of argument, that the smallest chain is expected (under SUHA) to be of length $\lg n / (\pi \lg \lg n)$ then the ratio between the smallest and largest bucket would still be constant, and worst-case complexity still O(1).

    I can't find a formula for the expected length of the shortest chain, but I think this would be needed to complete the proof properly.

     – aioobe Nov 25 '19 at 17:11
  • We're still both talking about the case $m = n$, right? In that case, since the longest chain is $\omega(1)$, the shortest chain is $0$, by the pigeonhole principle. Are you saying that you think the shortest non-empty chain is $\Theta(\lg n / \lg \lg n)$? – jbapple Nov 26 '19 at 04:34
  • @jbapple actually the $\tilde \Theta(\log n)$ result holds for universal hashing, i.e., full independence, the guarantee for SUHA is much weaker than that – Ameer Jewdaki Nov 26 '19 at 13:42
  • I suppose I should be saying $\Omega(\lg n / \lg \lg n)$. – jbapple Nov 26 '19 at 15:10
1

Let $h:[U]\to[m]$ be a uniform hash function, and $x_1,\dots,x_n\in[U]$ be the elements currently stored in the hash table, and we want to look up the query $q\in[U]$. Finally let $X_i$ be the indicator variable for the event that $x_i$ collides with $q$: $$X_i = \begin{cases}1 && h(x_i)=h(q)\\0 &&h(x_i)\ne h(q)\end{cases} \Rightarrow P(X_i=1) = \frac{1}{m}$$ Then the time $T$ to loop up $q$ is equal to the number of elements that collide with it: $$T=\sum_{i=1}^n X_i$$ By linearity of expectation we have: $$\mathbb{E} [T] = \sum_i\mathbb{E} X_i = \frac{n}{m}=\alpha$$ And finally by Markov's inequality: $$ P(T>t) \le \frac{\alpha}{t}$$ Now if we set $t:={\alpha}/{\delta}$, we will have: $$P(\text{look-up time}\ge \frac{\alpha}{\delta}) \le \delta$$ Think of $\delta$ as the failure rate. So for $\delta=0.1$, the query time is bounded by $10\alpha$ with probability $0.9$. If you strengthen the assumption about $h$ to a $k$-wise independent hash, the failure rate rouhgly drops to $P(\text{fail})\approx\delta^k$ and so you can have a much better complexity/failure-rate trade-off.

Now, a more conservative analysis is, what is the length of the longest chain in your table? In this case, we can define indicator variables $X_{i,j}$ for $i,j\in[n]$, that indicate the event that $x_i$ and $x_j$ collide. Clearly, $P(X_{i,j})=1/m$. Then the expected total number of collisions $S$ is: $$\mathbb{E} S = \sum_{i,j\le n} \mathbb{E} X_{i,j} = \binom{n}{2}\frac{1}{m}$$ If the maximum chain length is $\ell$, there is $\binom{\ell}{2}$ pairs of elements in that chain that collide with each other, implying $S\ge \binom{\ell}{2}$. We have: $$P(\text{max chain length}\ge \ell) \le P\left(S\ge \binom{\ell}{2}\right) \le\frac{ \mathbb{E} S} { \binom{\ell}{2}}\approx (n^2/m\ell^2) $$ If you set $\ell:=n^2/m\delta$ the probablistic bound is: $$P(\text{max chain length}\ge \ell) \le \delta \qquad \text{ for } \ell = \frac{n}{\delta \sqrt{m}}$$ So if $m\approx n$, your maximum length grows like $\ell\approx \sqrt{n}/\delta$ with $1-\delta$ probability. This is because the simple uniform hash assumption is really weak. The more independence you assume, the better the guarantee. For a universally uniform (meaning completely iid) hash function, you will get a logarithmic maximum chain $log(n)$ for $m\approx n$.

Ameer Jewdaki
  • 531
  • 2
  • 14
  • Nice. But this is a probabilistic analysis, right? I don't think it matters when discussing worst-case scenarios. – aioobe Nov 25 '19 at 18:54
  • Yes, that's what I tried to point out in the last paragraph. Typically, these kinds of probabilistic statements are the randomized equivalent of worst-case analysis for deterministic algorithms. In the actual worst-case analysis, there is always a non-zero probability that your hash function produces a chain of length $n$, but that would be very unlikely, and is not very meaningful. – Ameer Jewdaki Nov 25 '19 at 18:59