1

An algorithm terminates when the input size is less than $1$. For each iteration, there's $1/2$ chance that the input size gets halved, and $1/2$ chance it stays the same. What's the expected number of rounds to termination when the input is $n$?

I know that the intuitive answer of $\log_{4/3}n$ is not the correct answer, but I can't seem to convince myself why. Also, I was able to use Markov's inequality to upper bound the probability that the algorithm doesn't terminate after $k \log_{4/3}n$ rounds to be $\leq 1/n^{k-1}$, which is somewhat a good enough guarantee for runtime, but I'm just not sure the close form of the expectation, or if one exists, of the number of rounds.

Lee David Chung Lin
  • 7,043
Frank Zhang
  • 23
  • Thanks for replying! So I know that the intuitive answer of log_{4/3}n is not the correct answer, but I can't seem to convince myself why. Also, I was able to use Markov's inequality to upper bound the probability that the algorithm doesn't terminate after klog_{4/3}n rounds to be <= 1/n^{k-1}, which is somewhat a good enough guarantee for runtime, but I'm just not sure the close form of the expectation, or if one exists, of the number of rounds. – Frank Zhang Feb 12 '20 at 06:11
  • That's good stuff. I just made an edit on your behalf, see if you like it. In the future, when posting new questions please include those in the body of the question, so that people know where you're at and can work with you accordingly. – Lee David Chung Lin Feb 12 '20 at 07:19

1 Answers1

0

Given $n$ as a parameter, define a new parameter $ r\equiv \lceil \log_2 n \rceil$. For example, when $n = 8$ we have $r = 3$, and if $n = 10$ one will have seen at least $r = 4$ rounds at termination.

Denote the probability "the input size gets halved" as $p$. Here $p = 1/2$. Denote $X$ as the "total number of rounds when the algorithm terminates", then $X$ follows a Negative-Binomial distribution, with the following probability mass function:

$$\Pr\left\{ X = k\right\} = { k-1\choose r-1}p^r(1-p)^{k-r}~, \qquad k = r, r+1, r+2, \ldots$$ This is a standard discrete distribution, and it's expectation is well-known:

$$\mathbb{E}[X] = \frac{r}p = \frac{\lceil \log_2 n \rceil}p = 2\lceil \log_2 n \rceil$$ The above is the requested answer.

Below is a clarification in case of confusion.

There are at two common parametrizations of the Negative-Binomial distribution, besides simply switching $p \leftrightarrow q$ (and labeling which one as the "success"). Please be careful with the definition when reading materials from various sources.

Whenever you see the expectation expressed as $\mathbb{E}[Y] = r(1-p)/p$, it is parametrized as $$Y \equiv \text{number of (rounds of) failure till the termination at the $r$ th success}$$ as opposed to here $X$ being the total number of rounds. Namely, $X = r+Y$ which means $$\mathbb{E}[X] = \mathbb{E}[r+Y] = r+\mathbb{E}[Y] = r+\frac{r(1-p)}p =\frac{r}p$$ that is necessarily consistent.

Lee David Chung Lin
  • 7,043
  • Hi Lee, thanks a lot for answering, and that makes a lot of sense! On a related note, is it still possible to derive the expectation, if we don't get such nice numbers? Specifically, if we only know that, on expectation, our input size gets multiplied by 3/4, but not necessarily the distribution as above, is it still doable? Or does it depend on the distribution? If so, would you suggest a way to start from the distribution and deriving the expectation? Thanks again! – Frank Zhang Feb 12 '20 at 13:04
  • Okay, I have some ideas, but admittedly this is a bit beyond me. Why don't you post a new question about it? In the new post, you should link to this one, and please make sure the distinction is clear between the generalized (original?) question and this one. – Lee David Chung Lin Feb 12 '20 at 13:29
  • Also, you might want to add some attempts like quick-and-dirty simulation done by your own (with convenient assumptions), otherwise some people might view it as too general or lacking context (your effort). The point is, show people that you have tried to tackle it yourself, even if just a bit. – Lee David Chung Lin Feb 12 '20 at 13:33
  • Right now, I would formulate it in the following way. Given initial size $n_i$ and the threshold size $n_t$, the number of rounds is $\displaystyle R \equiv \min\Bigl{k:~\bigl(\prod\nolimits_i^k X_i \bigr) < n_t / n_i\Bigr}$, where $X_i$ are iid with support $[0,1]$ and $\mathbb{E}[X_i]=3/4$. The goal is to find $\mathbb{E}[R]$. – Lee David Chung Lin Feb 12 '20 at 13:42
  • Even with the distribution of $X_i$ specified (for example, to have mean $3/4$, one can let $X_i \sim \mathrm{Beta}[3,1]$ or $X_i \sim \mathrm{Unif}[1/2,1]$), the expectation of $R$ let alone the distribution of $R$ can be formidable. On the other hand, sometimes there are some tricks that just so happen to make things work. There also might be some limit-theorems and asymptotics that allow very general statements or inequalities. Like I said, this is beyond me, but I'm hopeful that experts on this site can give you something good. – Lee David Chung Lin Feb 12 '20 at 13:49
  • Anyway, here the definition of $R$ can be rewritten via taking logs and turn it from multiplicative to additive: $\displaystyle R = \min\Bigl{k:~\bigl(\sum\nolimits_i^k \log X_i \bigr) < \log n_t - \log n_i\Bigr}$, where $n_i > n_t$. In your case, $n_i = n$ and the threshold is $n_t = 1$. – Lee David Chung Lin Feb 12 '20 at 14:05
  • Thanks so much for all the contribution Lee! Here is the new thread, in case you are interested. https://math.stackexchange.com/questions/3543998/ – Frank Zhang Feb 12 '20 at 14:13