1

I am new to Advanced Algorithms and I have studied various samples on Google and StackExchange. What I understand is:

  1. We use $O(\log n)$ complexity when there is division of any $n$ number on each recursion (especially in divide and conquer).

  2. I know that for binary search, we have time complexity $O(n \log n)$, I understood $\log n$ is because each time it halves the full $n$ size number list in a recursive manner until it finds the required element. But why is it multiplied with $n$ even we just traverse half of the $n$ size element for each execution so why we multiply $\log n$ with $n$?

  3. Please give me any example explaining the complexity $O(n^2 \log n)$. I hope this will help me in understanding much better the above two questions.

Raphael
  • 72,336
  • 29
  • 179
  • 389
shekhar shekhawat
  • 11
  • 1
  • 1
    Have you looked in a textbook on the matter, esp. for the definitions? Have you checked out our reference questions? There is also this question. Finally, let it be said that complexity classes are not defined via algorithms of that runtime, and that any given algorithm may "be" in different classes depending on what you count (cf comparisons vs swaps for Quicksort). – Raphael Sep 16 '13 at 08:59
  • 1
    Maybe you just chose the wrong word but when you say, "We use $O(\log n)$ complexity," that suggests a significant misunderstanding. It suggests making a choice: for example, you might choose to implement an algorithm using linear programming, divide & conquer or some other method. But you don't choose the complexity of a given algorithm, just as you don't choose the prime factors of a given integer. The actual complexity of any specific algorithm is a fixed property of that algorithm. You might choose to use an algorithm with some given complexity but you can't choose the complexity per se. – David Richerby Sep 15 '13 at 20:37

1 Answers1

4

  1. You are correct in thinking binary search is $O(\log n)$, it shouldn't be multiplied by $n$.

  2. Popular (comparison-based) sorting algorithms are $O(n \log n)$.

  3. 3SUM (i.e. find 3 elements in an array that sums to zero) using binary search is $O(n^2 \log n)$.

    The pseudo-code:

    For each element
  For each other element
    Do a binary search for the 3rd element that will result in a zero sum.

    Although the problem can be solved in $O(n^2)$ in a different way, this should still serve as a decent example.

Explanation of merge-sort complexity:

Merge-sort, for example, splits the array into 2 parts repeatedly (similar to binary search), but there are some differences:

  • Binary search throws away the other half, where merge-sort processes both
  • Binary search consists of a simple $O(1)$ check at each point, where-as merge-sort needs to do an $O(n)$ merge. This should already make the $O(\log n)$ vs $O(n \log n)$ make sense.

For a quick check, ask how many work, on average, is done for each element.
Note that a single merge is linear time, thus $O(1)$ per element.
You recurse down $O(\log n)$ times and, at each step there's a merge, so each element is involved in $O(\log n)$ merges.
And there are $O(n)$ elements.
Thus we have a time complexity of $O(n \log n)$.

There is also the more mathematical analysis: (source)

Let $T(n)$ the time used to sort n elements. As we can perform separation and merging in linear time, it takes $cn$ time to perform these two steps, for some constant $c$. So,
$T(n) = 2T(n/2) + cn$.

From here you work your way down to $T(1)$, and the remaining terms gives you your $O(n \log n)$ running time. Or use the master theorem.

If both of these are unclear, it should be easy enough to find another resource explaining the complexity of merge-sort.

Bernhard Barker
  • 945
  • 6
  • 13
  • Thaks for the reply.Now i have the doubt that in merge sort we have complexity O(nlogn). I understood this 'logn' it is because of division of input array in to two halves each time but again why to multiply it with 'n' (may be this n complexity is for comparison among two elements but we compare only two elements at a time so why we multiply 'n' by 'logn') please explain this. – shekhar shekhawat Sep 14 '13 at 09:45
  • ^I suggest you to go through Introduction to algorithms by CLRS. Merge sort has been explained quite well in that. – avi Sep 14 '13 at 12:21
  • @shekharshekhawat Edited with an explanation of merge-sort. – Bernhard Barker Sep 15 '13 at 16:09
  • @shekharshekhawat Check out any introduction on algorithm analysis, e.g. CLRS or Sedgewick. There is a mathematical formalism to it. By applying this formalism repeatedly you will develop the intuition you search. There is no (reliable) shortcut. – Raphael Sep 16 '13 at 09:01