-1

Regarding time complexity I've read conflicting things:

1) That it is worst case.

2) That is average case.

For example if I want to know the time complexity for inserting into an arbitrary point in a linked list ( not the beginning or the end ), on average it will take

(n/2) operations so we can drop the constant and say

it is

n.

So would Big O be n or would Big Omega be n. Or are they the same thing?

Raphael
  • 72,336
  • 29
  • 179
  • 389
cade galt
  • 141
  • 1
  • 1
  • 6
  • See http://cs.stackexchange.com/q/57/755 – D.W. Oct 09 '15 at 18:06
  • 3
    One thing I thought I would note that I often see missed. When you write out "$O(f(n))$" or "\Omega(f(n))" you are referring to a set of real valued functions. That is there is an actual mathematical object that these notations refer to, I.e. they are not just properties but actual sets. Hence "$\in$" is proper notation but "=" is a bit of an abuse. – Jake Oct 09 '15 at 18:45

2 Answers2

9

Big-$O$ and $\Omega$ are actually defined entirely separately from complexity theory. All they are is properties of functions.

$O$ is used for asymptotic upper-bounds of a function, and $\Omega$ is used for asymptotic lower bounds. If for some functions $f,g$ we have $f \in O(g)$, and $f \in \Omega(G)$, you can use big-theta notation, $f \in \Theta(G)$.

So, we can say something like $n^2 \in O(n^3)$, or $n^3 \in \Omega(n^2)$. That statement is entirely separate from what meaning we are attaching to $n^2$ or $n^2$.

For the formal definition of what we mean by "lower bound" or "upper bound", see here.

When we're analyzing an algorithm, there are a few cost functions we like to look at. One is the greatest time required for any input of size $n$ (worst case), one is average number of steps for an input of size $n$. There are numerous other things you can look at, such as the number of bits of memory used, the maximum number of cache misses, the number of queries to a database, etc.

The important thing is, all of these are just functions, and we can describe them with any of $O, \Omega, \Theta$. Big-O and friends describes their growth, but carries no meaning if we don't specify what function's growth we are describing.

Note that worst-case and upper bounds are different things. The "worst-case complexity" means, for any input of size $n$, what is the greatest time the algorithm takes. What's important here is, we're taking the worst over all inputs of size $n$, but imagining a function returns the exact worst case time for any input $n$.

On the other hand, saying that a function is an upper bound just means that we know the function is asymptotically less than that upper bound. So, you can still have an upper bound on the average case analysis (in fact, you usually will).

Joey Eremondi
  • 29,754
  • 5
  • 64
  • 121
  • worst case it would have to search through all n items. best case it would be the first item. So best case is 1, worst case is n, and average case is n/2. How does this relate to O(n) and Omega(n) and which one is average? – cade galt Oct 09 '15 at 18:34
  • 1
    Try rereading my post. You can describe the worst or average case with $O$ or $\Omega$, depending on whether you're talking about the upper bound or the lower bound, which is COMPLETELY SEPARATE from worst, best and average case. So there's worst case $O$, worst case $\Omega$, best case $O$, best case $\Omega$, average case $O$, average case $\Omega$ all as separate things you can look at. Also not, that average case is NOT always the average of best and worst case. – Joey Eremondi Oct 09 '15 at 18:49
  • 1
    You may want to add your answer to the question DW links above, or the duplicate I link. (Or parts to either?) – Raphael Oct 09 '15 at 22:40
3

$O$-notation is for upper bounds, $\Omega$-notation for lower bounds.

They can both be used to specify average and worst case bounds.

In your example it is both $O(n)$ and $\Omega(n)$, since both the upper and lower bounds (of the average case running time) are $n/2$.

Tom van der Zanden
  • 13,238
  • 1
  • 35
  • 54
  • worst case it would have to search through all n items. best case it would be the first item. So best case is 1, worst case is n, and average case is n/2. How does this relate to O(n) and Omega(n) and which one is average? – cade galt Oct 09 '15 at 18:34
  • Neither $O$ nor $\Omega$ "is" worst or average - they're separate notions. The worst case is $O(n)$ and $\Omega(n)$. The average case is also $O(n)$ and $\Omega(n)$. The best case is both $O(1)$ and $\Omega(1)$. – Tom van der Zanden Oct 09 '15 at 20:51
  • So this well-written cheat sheet - http://bigocheatsheet.com - chose to use Big O to describe operations and algorithmic behavior but it could have chosen to use Sigma just as well? – cade galt Oct 20 '15 at 19:16
  • @cadegalt $\Omega$ is omega, not $\sigma$ or $\Sigma$. Assuming all their bounds are tight, yes, they could have used omega (but, depending on your use case, omega isn't nearly as useful as big O). – Tom van der Zanden Oct 20 '15 at 19:22