2

I read about Big-O notation with modular arithmetic. So, Time complexity is commonly estimated by counting the number of elementary operations performed by the algorithm, where an elementary operation takes a fixed amount of time to perform. Thus the amount of time taken and the number of elementary operations performed by the algorithm differ by at most a constant factor. Some algorithms have a worst-case time of $O(n)$ or $O(n^2)$ or $O(n^3)$.

What I want to know is what is the best time complexity of an algorithm? And why?

Mhsz
  • 145
  • 4
  • 2
    Are you trying to ask why we use big O notation (as the title seems to indicate) or what is the best running time? Smaller functions are better (indicating a faster algorithm) , with $O(1)$ being the smallest run time that makes sense in this context. – Joe Jun 07 '14 at 23:53
  • Mr joe thank you. If we have two time $20n^2$ and $n^2$ I know when we use the Big-O should be omitted coefficient. But practically when drawing functions note the existence of a difference between the two, can we say $n^2$ better than $20n^2$. Graphically – Mhsz Jun 08 '14 at 00:08
  • @mahmoud_zaiin You should update your question to stress this example. – Yuval Filmus Jun 08 '14 at 06:02
  • another point to YFs answer is that there are large open complexity class questions like P=?NP that even asymptotic notation (which is arguably somewhat "coarse-grained" for reasons YF enumerates) cant help with so far! in other words maybe "when" big complexity class separation questions are answered (which are fiendishly difficult & open for decades), more "fine-grained" complexity measures might be possible. one such measure might eg be "circuit gate count" etc. but it is so far intractable and not leading to major new insights than asymptotic notation on TM runtimes. – vzn Jun 08 '14 at 15:19
  • Closely related question: Justification for neglecting constants in Big O – Raphael Jun 08 '14 at 17:43

2 Answers2

7

The problem you are describing is indeed a shortcoming of asymptotic notation. Asymptotic notation has several advantages:

  1. It doesn't depend as much on the underlying computation model. For example, the microprocessor executing the compiled code shouldn't affect the running time by more than a constant.

  2. It avoids cumbersome exact results, for example instead of $n^2 + 3.5n\log n + 700\log n\lfloor \log\log n \rfloor$ we can write simply $O(n^2)$ (or even $\Theta(n^2)$).

  3. In many cases it is hard to analyze exactly the running time (even ignoring the first issue), but easy to come up with asymptotics. Standard examples are recursive algorithms. In those case, however, it is often possible to come up with first-order estimates, that is, perhaps we can say that the running time satisfies $T(n) \sim 2n^2$, which is more accurate than $T(n) = \Theta(n^2)$.

  4. Practically speaking, in many cases the constants hidden by the big O notation are "small" or at least "reasonable", and therefore for $n$ which is not too small, a $\Theta(n)$ algorithm will usually be faster than a $\Theta(n^2)$ algorithm.

  5. Big O estimates function as ball park heuristics, which allow us to guess which algorithm is better or which part of an algorithm needs optimization the most. If in doubt, however, it is better to use profiling.

On the other hand, in some cases the problem you're indicating is serious. A classical case is fast matrix multiplication, where the trivial $O(n^3)$ algorithm often beats Strassen's $O(n^{\log_2 7})$, and always beats all other algorithms (which have better asymptotic running time).

Summarizing, there is no easy answer. Asymptotic notation is a good model of reality in many cases, but not always. Still, it is useful since it is concise and allows quick comparison between different algorithms.

Yuval Filmus
  • 276,994
  • 27
  • 311
  • 503
3

The "best" time complexity of an algorithm is when it reaches the upper complexity bound for the given problem.

Let us take a trivial example: design an algorithm to count zeroes in a sequence of $N$ numbers.

Obviously, you need to "read" every number to get the answer (because you can't guess a number by reading the others), so that the complexity of any solution is at least $\Omega(N)$. If you are able to design an algorithm that takes $O(N)$ time, then you can stop searching for better solutions (at least in the asymptotic sense), your algorithm is optimal and can't be improved.

Asymptotic complexities are used mostly for mathematical tractability, and for "portability". Because they ignore numerous "premature" details in the analysis, simplify the equations (even though in many cases the developments remain extremely involved), and because they are independent of any architectural specifics of the hardware.

Asymptotic analysis is a powerful tool for algorithm designers to rate the performance of their solutions. Anyway, they are just a valid indication for sufficiently large problem sizes. In practice, nothing replaces benchmarking on representative data sets, and a theoretically poor algorithm can be the real winner.