3

Someone recently claimed that there's an adversarial input for randomized quicksort; he referenced this paper. This defies my intuition because there are results that say that randomized quicksort runs in $O(n \log n)$ time with high probability. (See, for example, the book Randomized Algorithms by Motwani&Raghavan.) I am refering to the version of randomized quicksort that picks the pivot uniformly at random at each recursive call.

So my question is: are there indeed adversarial inputs for randomized quicksort? Please give an intuitive reason as to why such an input can exist.

Update. It seems to me that the adversary in the paper is granted unrealistic capabilities (generate array on the fly), which explains why it can make any quicksort perform poorly. Under reasonable assumptions I would say there is indeed no adversarial input for randomized quicksort.

user12344567
  • 261
  • 1
  • 8
  • 2
    Which results? Can you give a reference? Also, you should clarify what exactly is randomized, to which exact implementation you refer, and which inputs you want to allow. There are combinations that have adversarial inputs. – Raphael Apr 03 '16 at 13:55
  • @Raphael I saw this in the book Randomized Algorithms by Motwani&Raghavan. (In there it is one of the exercises to prove this; the RandQS one is supposed to analyze takes as input a set of distinct elements. We did a detailed proof of this in a course on randomized algorithms at the university.) – user12344567 Apr 03 '16 at 14:40
  • 1
    How about linking to the source of the claim on Stack Overflow? That would enable more useful, more informed answers. Also, when answering requests for clarification, please edit the question (don't just leave information in the comments; we want questions to stand on their own; people shouldn't have to read the comments). – D.W. Apr 04 '16 at 02:27

4 Answers4

8

There are implementations of Quicksort (the partitioning algorithm, specifically) which deal badly with duplicates. No matter how much you randomize -- shuffling the input, random choice of the pivot, random choice of the pivot with sampling, ... -- if all entries are the same (or there are only constantly many distinct values), these bad implementations will result in worst-case behaviour, always.

You can find details e.g. here and in the literature.

Note bene: The reason why many TCSists would claim the opposite with a passion it that they forget that most classical analyses are performed under permutation models. That does not give the whole truth, though, particularly if you use sorting in practice. Duplicates are relevant, for instance if you sort lists of binary values.

Community
  • 1
Raphael
  • 72,336
  • 29
  • 179
  • 389
  • Specifically, any 2-way split will put all the equal elements in one partition, thereby only reducing the recursion to n-1 elements. I suppose the equal elements could be split randomly between the two partitions to prevent that. – KWillets Apr 04 '16 at 06:44
  • @KWillets A canonical way to avoid this effect is to have a third partition for the pivot value on which you don't recurse. – Raphael Nov 01 '16 at 17:21
  • I'm well aware of that, but I think I was referring to how to make 2-way work. – KWillets Nov 01 '16 at 17:54
5

The analysis of randomized quicksort is independent of the input. While there might be better inputs and worse inputs, the upper bound given by the analysis works for every single input.

You can go over the proof in slides of Xi Chen, for example, who proves that whatever the input, randomized quicksort runs in expected time $O(n\log n)$. (I don't know whether your high probability claim is true as well, though it sounds plausible.)

Yuval Filmus
  • 276,994
  • 27
  • 311
  • 503
  • 1
    Yuval exactly, the result is independent of the input. So there is no adversarial input for randomized quicksort. – user12344567 Apr 03 '16 at 10:55
  • The expected running time might depend on the input (perhaps in this case it doesn't - I don't know), but the upper bound holds for all inputs. – Yuval Filmus Apr 03 '16 at 10:56
  • By might depend on the input you mean that the constants implied by the asymptotic notation might depend on the input? – user12344567 Apr 03 '16 at 10:57
  • That's one way to think about it, especially if you consider a family of inputs for infinitely many $n$ (otherwise asymptotic notation doesn't make much sense). It might even be that there exists a family of inputs for which the running time is $o(n\log n)$. – Yuval Filmus Apr 03 '16 at 10:59
  • "The analysis of randomized quicksort is independent of the input." -- that statement is wrong in its generality. It depends on which implementation is analyzed under which input model. – Raphael Apr 03 '16 at 13:56
  • @Raphael It also depends on the analysis. If your analysis is under worst case assumptions, then the guarantee indeed doesn't depend on the input. – Yuval Filmus Apr 03 '16 at 16:03
  • @YuvalFilmus The high probability claim is also true. $$Pr(T > (1+\delta)n\log n) \le e^{-\mathcal{O}(\delta)}$$ Where $T$ is the running time – Banach Tarski Apr 03 '16 at 16:49
  • @BanachTarski A high probability claim is one that holds with probability $1-o(1)$. In your case, we would have to choose $\delta = \omega(1)$, and then the time bound would be $\omega(n\log n)$. – Yuval Filmus Apr 03 '16 at 16:51
  • slides of Xi Chen mentioned in the answer are available through Wayback Machine here – mrtexaz Jun 20 '21 at 13:12
3

The mentioned StackOverflow question actually gives the source for this claim.

The source doesn't make this very clear, but the reason for this perceived mismatch between theory and practice is that in theory your comparison function is a mathematical function given ahead of time, but in practice it is a C function that can have side-effects and depend on previous side-effects.

Note that although the adversarial comparison function is non-deterministic in the sense that there is no mathematical function chosen ahead of time it corresponds to, it is still in a sense well-behaved: for any given execution of the program, there exists an order of elements such that the comparison function is compatible with it.

Community
  • 1
Rotsor
  • 131
  • 4
  • 1
    I think the paper they reference assumes the adversary has unrealistic capabilities. – user12344567 Apr 03 '16 at 15:03
  • Doing/depending on side-effects in C is hardly unrealistic. – Rotsor Apr 03 '16 at 15:08
  • Being able to generate the input on the fly is. You usually pass a filled array to the sorting function. – user12344567 Apr 03 '16 at 15:09
  • 1
    Ah, you are right that in the example they give in the paper they change the input array directly, which is not how you are supposed to invoke qsort. However, you can avoid that by introducing some sort of indirection: e.g. make your array contain pointers. Then your "input" doesn't get generated on the fly anymore. It's only your comparison function. :) – Rotsor Apr 03 '16 at 15:24
0

The killer adversary referenced builds an input that can make any quicksort go quadratic time provided that the pivot selection process responds the SAME every time the SAME input is used. In reality, a randomized pivot selection would have progressed to a different seed in the random series, so the next time the sort is called it will not cause quadratic time. It will cause quadratic only while it's building the input.

In other words, as soon as the killer input is built, it becomes not a killer. (in the case of random pivot selection).

S0lo
  • 1
  • 1
  • I don't get what you are trying to say. You start out by saying that there is a killer input in some cases, end end with that there are none in others. Please clarify. – Raphael Nov 08 '16 at 17:05
  • What I meant is that the killer input will NOT work for a quicksort that uses random pivot selection if that randomization starts with different seed – S0lo Nov 10 '16 at 08:48
  • Which always happens by the way. – S0lo Nov 10 '16 at 08:49
  • Just revise how random numbers are generated. For example, If you seed rand() with the same seed every time. It will generate exactly the same random series every time, in this case, the killer will work, because the killer input was built only for that series. But in practice that almost never happens because programmers usually only seed rand() (using srand()) once at the start of a program. – S0lo Nov 10 '16 at 08:59
  • You're wrong; please refer to my answer. – Raphael Nov 10 '16 at 15:41
  • Your phrase "You're wrong" is vague. Please explain. – S0lo Nov 14 '16 at 10:59
  • In comment to your answer, the adversary produces a killer permutation (no duplicates). So even if a quicksort is well built to handle duplicates, the killer will still cause quadratic time. Thats actually whats interesting about that paper if you understand it well. – S0lo Nov 14 '16 at 11:07