2

I learned that recursive Fibonacci is $O(2^N)$.

However, when I implement it and print out the recursive calls that were made, I only get 15 calls for N=5. What I am missing? Should it not be 32 or near abouts? It's like less then half!

$COUNT = 0

def fibonacci( n )
   $COUNT +=1
   return  n  if ( 0..1 ).include? n   # Base Case returns 0 or 1   
   fibonacci( n - 1 ) + fibonacci( n - 2 ) # Recursive case
end
Raphael
  • 72,336
  • 29
  • 179
  • 389
aarti
  • 131
  • 7
  • 1
    Big-O notation doesn't give you an equality, it gives you a (scaled) bounding function. Nothing says the scaling factor can't be less than 1. –  Jul 29 '15 at 22:59
  • @pjs I understand that Big(0) is an appx. But, the way fibonacci recursion time complexity was explained to me was that each node occurs twice 2x2x2.. n times = 2^N. I want to know what happened to the rest of the calls. – aarti Jul 29 '15 at 23:13
  • 3
    @500_error Either you misunderstood that person, or they did not know what they were talking about. There are $2^n$ nodes in the perfect binary tree which corresponds to the recurrence $f(n) = 2f(n-1)$ -- this is not what we have here (but an upper bound). – Raphael Jul 30 '15 at 07:39
  • 1
    for more info/ detail on how to pull off this basic complexity analysis exercise drop by [chat], but basically big-oh notation & complexity analysis is about characterizing a trend of points, not individual ones. – vzn Jul 31 '15 at 17:40
  • One problem is that O (2^n) is a correct upper bound, but a very bad one. Like "the top speed of my car is less than 250 miles per hour". Which is perfectly true, but a very bad upper bound. – gnasher729 Oct 31 '16 at 10:39

3 Answers3

13

You ask,

I have an $O(2^n)$ runtime, why do I not observe $2^n$ recursive calls for $n=15$?

There are many things wrong in the implied conclusion.

  1. $O(\_)$ only gives you an upper bound. The true behaviour may be of much smaller growth.
  2. Asymptotics (like $O(\_))$ only give you only something in the limit, that is you can only expect the bound to hold for large $n$.

  3. Since you use $O$ as though it was $\Omega$, note that a (lower) bound on runtime does not necessarily translate into a (lower) bound on recursive calls, each of which may take $\omega(1)$ time.

    Similarly, a tight upper runtime bound (which you don't have) may not be a tight upper bound on the number of recursive calls for the same reasons.

You can analyse exactly how many recursive calls you need. Just solve the recurrence

$\qquad\displaystyle \begin{align*} f(0) &= 1,\\ f(1) &= 1,\\ f(n) &= f(n-1) + f(n-2) + 1, \quad n \geq 3, \end{align*}$

which adds up $1$ for every recursive call your program makes. See here for more detail on how you get to such recurrences.

See here how to do that, or use computer algebra. The result is

$\qquad f(n) = 2F_{n+1} - 1$;

insert $n=5$ and you will get $f(n) = 15$. It's not that surprising that the Fibonacci numbers $F_n$ should show up. Inserting their known closed form, we get

$\qquad\displaystyle f(n) = \frac{2}{\sqrt{5}} \cdot \Bigl( \varphi^{n+1} - (1-\varphi)^{n+1} \Bigr) - 1$

with $\varphi = \frac{1 + \sqrt{5}}{2} \approx 1.62$ the golden ratio. This closed form of $f(n)$ implies

$\qquad\displaystyle f(n) \sim \frac{2}{\sqrt{5}} \cdot \varphi^{n+1} \approx 1.45 \cdot 1.62^n$

since $0< 1 - \varphi < 1 < \varphi$. We see that the $O(2^n)$-bound is rather loose -- it has exponential absolute and relative error.

Community
  • 1
Raphael
  • 72,336
  • 29
  • 179
  • 389
2

Double-recursion ends when n<=1, so it's more like 2^(n-1). As @pjs says, this doesn't change anything.

A function that take 0.1234 * (2^(n-3) + n^400) + 5000 cycles, or operations, or whatever, is still of the O(2^n) complexity class.

Peter Cordes
  • 1,055
  • 7
  • 15
  • 1
    It can be useful to analyse algorithms for exactly the number of operations, rather than just big-O complexity class, when finite problem sizes matter. :) –  Jul 29 '15 at 23:31
  • 1
    Complaint of terminology: big oh is rigorous, but it's a set of functions rather than a specific one. Perhaps the word you are looking for is exact rather than "rigorous". – Jake Jul 30 '15 at 07:52
1

In addition to the 3 reasons given by Raphael in his answer, regarding why the aymptotic cost upperbound $O(2^n)$ does not give you $2^n$ recursive calls for $n=15$, I think you should add a fourth one:

  1. Asymptotic costs are always defined up to a constant factor, when using Landau notation (big $O$, big $\Omega$, little $\omega$, etc.).

There is a reason for that: the cost unit is undefined. This makes analysis easier and somewhat support independent. As long as all primitive operations have a cost within a bounded interval, the relative cost of the various elementary steps of the algorithm can be ignored.

The price to be paid of course, is that the cost thus evaluated has no absolute meaning, only a relative one.

If we were to assume that:

  • $O(2^n)$ is not an upperbound (as indicated by big $O$) but an exact cost function (point 1 of Raphael);

  • $O(2^n)$ is not asymptotic but exact cost for all values of $n$ (point 2 of Raphael);

  • each recursive call takes constant time (point 3 of Raphael);

  • and that furthermore there are no cost other than the constant cost of each recursive call (not including sub-calls), which seems to be your implicit assumption,

we would still be unable to assert that for $n=15$ you should have $2^{15}$ calls, from the simple knowledge of $O(2^n)$ (ignoring how the formula was obtained, of course). The reason is the arbitrary constant factor that does not show in the formula.

At best you would know that there is a factor $k$ such that for each value of $n$ you should have $k2^n$ call. Thus, if you know that for $n=15$ you had $p$ recursive calls, that would imply that $p=k2^{15}$, which would allow you to compute the value of $k$, and then predict the number of calls for other values of $n$.

But this comes only after a lot of other assumptions, none of which is really true. With apologies for awkwardly attemtping to separate issues that really have meaning only together.

babou
  • 19,445
  • 40
  • 76