7

I'm working out of the 3rd edition CLRS Algorithms textbook and in Chapter 3 a discussion begins about asymptotic notation which starts with $\Theta$ notation. I understood the beginning definition of:

$$\Theta(g(n)) = \{ f(n)\,|\, \exists\, c_1, c_2 > 0, n_0 \in \mathbb{N}: 0 \leq c_1 g(n) \leq f(n) \leq c_2 g(n)\ \ \forall n \geq n_0\}$$

But then on the next page the text says that:

The definition of $\Theta(g(n))$ requires that every member $f(n) \in \Theta(g(n))$ be asymptotically nonnegative, that is, that $f(n)$ be nonnegative whenever $n$ is sufficiently large. (An asymptotically positive function is one that is positive for all sufficiently large $n$.) Consequently, the function g(n) itself must be asymptotically nonnegative, or else the set $\Theta(g(n))$ is empty.

That last part about the how if the function is negative the set $\Theta(g(n))$ is empty and the general requirement of a positive function is sort of confusing. Can anyone out there clarify this definition for me and what it means, possible with an example, it would be much appreciated.

Raphael
  • 72,336
  • 29
  • 179
  • 389
Ockham
  • 263
  • 1
  • 2
  • 5
  • Is your problem covered by the answers to this question? – Raphael Sep 30 '12 at 16:59
  • 1
    I don't think the answer to this problem applies, but correct me if I'm wrong. The problem you posted is talking about little o and strict definitions, but I'm just interested in knowing why $\Theta$ requires nonnegative members – Ockham Sep 30 '12 at 17:29

2 Answers2

7

This is just a technicality. In asymptotic analysis we are only "really" interested in positive functions like $n^3$ or $n\log n$. However, if we want to be very formal and general, we could take into account non-positive functions (and that could turn it to be useful, see below). The definition of $\Theta$ states that $f(n) = \Theta(g(n))$ if from some point on, $f(n)/g(n)$ is bounded from both sides by constants, and furthermore $g(n) \geq 0$. (That's what you get if you unroll the definition.) In particular, if $f(n) = \Theta(g(n))$, then from some point on, $g$ is non-negative.

Here is an alternative definition of big $\Theta$. Suppose $f,g \colon \mathbb{N} \rightarrow \mathbb{N}$ are positive functions, that is $f(n),g(n)>0$. Then $f(n) = \Theta(g(n))$ if there exist positive constants $c_1,c_2$ such that $c_1 \leq f(n)/g(n) \leq c_2$. I don't know why the more complicated definition is presented in introductory texts.

What are the advantages of the more complicated definition? It can handle functions which have some non-positive values (there has to be a finite number of them). For example, this definition accommodates the (true) statement $n-10 = \Theta(2n-30\log n)$. Although functions encountered in practice are usually positive, sometimes negative functions could be encountered: for example, say we're interested in some genuine complicated function $k$, and we estimate it from below by a function $t$, which is however negative for small $n$.

Yuval Filmus
  • 276,994
  • 27
  • 311
  • 503
  • What exactly do you mean by the "more complicated definition"? Because it uses $n_0$? – phant0m Sep 30 '12 at 21:15
  • Note that the definition of CLRS as cited by the OP does not talk about the ratio of $f$ and $g$, and it is in fact not equivalent (without some lifting). – Raphael Sep 30 '12 at 21:47
  • @Raphael It is somewhat explained what cases the simplified definition does not cover in the last paragraph, but not exactly why. No, it may not talk about the ratio, but it's quite easy to see that this part alone is equivalent. – phant0m Sep 30 '12 at 21:50
  • @Raphael, I disagree. Both definitions are equivalent under the assumption that $f$ and $g$ are positive (exercise). – Yuval Filmus Oct 01 '12 at 03:59
  • @YuvalFilmus If we are talking about the same definitions, $f(x) = 1$ and $g(x) = \sin(x)$ is a counter-example. Any pair of functions for which the $\lim f/g$ does not exist breaks it. One has to use $\limsup$ resp. $\liminf$. – Raphael Oct 01 '12 at 09:01
  • @Raphael The function $\sin(n)$ is not positive. If you take $|\sin(n)|$ instead then $\sin(n) = \Theta(1)$ is false under both definitions, for the reason you mention: what value would you choose for $c_1$? – Yuval Filmus Oct 01 '12 at 13:52
  • @YuvalFilmus True enough; $0$ is a problem. Take $g(n) = 2 + \sin n$, then. By the definition given in the question, $g \in \Theta(1)$ ($c_1 = 1, c_2=3$) but the ratio does not have a limit. (The issue here is a rather arbitrary restriction to positive functions; the issue with the two definitions is another one which persists in either version.) – Raphael Oct 02 '12 at 17:23
  • @Raphael My definition also doesn't assume that $f(n)/g(n)$ approaches a limit, only that it is bounded from both sides. Also, I don't think the restriction to positive functions is arbitrary. Indeed, in the number-theoretic setting, you usually take absolute values, so your functions are always positive. E.g. by $f = O(1/n)$ we probably mean $|f|=O(1/n)$. – Yuval Filmus Oct 02 '12 at 22:26
  • @YuvalFilmus So we were talking about different definitions. I think you can construct cases where the ratio is not bounded (by constants) but the definition given here applies (something like $n+\log(\sin n)$). By "arbitrary" I did not not mean "without reason", but "without formal need". Asymptotic notation works on a broad range of functions, and even for $n\to c$ instead of $n \to \infty$ (even though I am not comfortable with that, either) -- if defined generally. – Raphael Oct 03 '12 at 08:42
  • @Raphael I was comparing my definition to the definition the OP posted. The OP's definition implies that the ratio is bounded (all assuming both functions are positive). It also implies that both functions are eventually non-negative. And I still contend that under the assumption that both functions are positive, both definitions are equivalent. – Yuval Filmus Oct 03 '12 at 12:24
3

Why $\Theta(g)$ may be empty

It follows directly from the definition.

$$\Theta(g(n)) = \{ f(n)\,|\, \exists\, c_1, c_2 > 0, n_0 \in \mathbb{N}: 0 \leq c_1 g(n) \leq f(n) \leq c_2 g(n)\ \ \forall n \geq n_0\}$$

The important part here is this: $ f(n) | \exists\, c_1 > 0: 0 \leq c_1 g(n)$

Obviously, the constraint on $ f$ (as of now, it does not even depend on $f$), states that $g$ multiplied with some positive constant $c_1$ must itself be positive (for large values of $n$, implied from here on).

Of course, if $g$ is not strictly positive, this constraint will prevent all possible functions $f$ from being a member of the set $\Theta(g)$.

It follows that such a set will be empty.

For all $f \in \Theta(g)$, $f$ is strictly positive

This also follows directly from the same part of the definition: $ f(n) | \dots : 0 \leq f(n) $

Obviously, if $f$ is not strictly positive, the condition is not met, hence no such $f$ can be contained in $\Theta(g)$.

Note: I'm not entirely sure what is unclear to you, because all you wrote is "confusing".

phant0m
  • 499
  • 3
  • 12