24

Suppose I have access to a fair coin. Is it possible to come up with a procedure that (1) returns TRUE with irrational probability (say $1/\sqrt{2}$) and FALSE otherwise, and (2) terminates in a finite amount of time?

I would think not, because at the end of the day I'm just assigning either TRUE or FALSE to sequences of coin flips, and any such assignment results in a rational probability. However, I don't think there's harm in asking: is there some extraordinarily clever way to extract irrational probabilities?

[Edit] Alternatively, what if we relax condition (2) to "terminates with probability 1"? (Thanks user6312!)

Elliott
  • 4,124
  • @Elliott: One can do it by relaxing your condition to "with probability $1$ terminates." – André Nicolas May 30 '11 at 23:13
  • Without knowing (or rembering) a lot about statistics, have you considered something like this: one might trow up a coin 10 x 10 times, count the number of heads for every 10 throws, calculate the average, variance, ... (or something like that) and test whether it is in some interval. I think I would find it more surprising if these probabilities were always rational. – Myself May 30 '11 at 23:17
  • @Myself: These probabilities can only be multiples of $2^{-100}$, since they are the probabilities of some subset of the power set of the independent and equiprobable elementary events, the coin flips. – joriki May 30 '11 at 23:23
  • @Joriki: Ah well indeed, thank you! – Myself May 30 '11 at 23:24
  • There is a procedure that is guaranteed to terminate in a finite amount of time - provided that you can make each coin flip take only half as much time as the preceding coin flip.... – Gerry Myerson May 30 '11 at 23:36
  • Thanks guys! It looks like the general idea is to set up an infinite sample space (this can't be done if we declare the maximum number of flips ahead of time, but it can be done so that the procedure terminates with probability 1). – Elliott May 30 '11 at 23:48
  • 1
    What if you are allowed to flip a needle? http://en.wikipedia.org/wiki/Buffon%27s_needle – Per Alexandersson May 31 '11 at 04:19
  • I was asked something very close to this question in a job interview and couldn't answer it. I came on here hoping for a proof that it was impossible. Great question and answers. – RoundTower Aug 24 '11 at 23:01

2 Answers2

25

It depends on what you mean by "terminates in a finite amount of time". This could mean:

  1. Always terminates in a finite amount of time, or

  2. Terminates in a finite amount of time with probability 1.

If you mean the former, you are correct that any event that depends on finitely many coin flips will have rational probability.

However, if you allow processes that terminate with probability 1, then any irrational probability is possible. For example, if you want an event with probability $1/\sqrt{2}$, simply interpret the sequence of coin flips as the digits of a binary number between 0 and 1, and check whether the resulting number is less than $1/\sqrt{2}$. With probability 1, you will be able to tell after some finite number of flips whether or not your number is less than $1/\sqrt{2}$, so you will almost surely have to flip only a finite number of coints.

Jim Belk
  • 49,278
  • 1
    Elliott, if the number of flips is bounded a priori, there are only a finite number of possibilities (no matter how they are defined), so the probability of any one of them is rational. – Dan Brumleve May 30 '11 at 23:45
  • 1
    indeed, any event that depends on finitely many coin flips will be a rational with denominator $2^n$. So, for example, probability 1/3 cannot be generated in that way. – GEdgar May 31 '11 at 00:25
  • What I love about this technique is that it works for rational non-power of two probabilities too. And I think its more efficient than naive rejection sampling. Reminds me a lot of http://en.wikipedia.org/wiki/Arithmetic_coding too. – Michael Anderson May 31 '11 at 06:33
  • GEdgar, to get 1/3, why can't you flip two, try again if HH happens, and return TRUE if TT happens? (Michael, is this what you mean by "naive rejection sampling"?) – Elliott May 31 '11 at 09:40
  • @Elliott: This is a good method. But, because of the "try again" provision, it can exceed your allowed number of tosses. – GEdgar May 31 '11 at 13:49
  • @Elliot - that's exactly what I mean by "naive rejection sampling". – Michael Anderson Jun 02 '11 at 09:24
  • @GEdgar - it can be shown that the expected number of coin tosses for naive rejection (and this arithmetic method too I suspect) to get a result is finite. And that the probability of needing more than N tosses is a negative exponential in N. I think that this is actually as good as you can guarantee for any number not expressible as a finite binary decimal. (e.g 1/5, 1/3 etc) – Michael Anderson Jun 02 '11 at 09:28
  • So, getting probability $1/3$ is exactly as difficult as getting probability $1/\sqrt{2}$ ? – GEdgar Jun 02 '11 at 18:10
  • 1
    +1 It seems that the expected number of coin flips is 2 (except when the target probability can be written as a rational with a denominator which is a power of 2, in which case a slightly lower expectation is possible). – Henry Jun 12 '11 at 00:20
10

Flip the coin until you get a tail. If the number of heads is prime, return TRUE. Otherwise, return FALSE.

Dan Brumleve
  • 17,796
  • I'm not looking forward to working out the exact probability, but I definitely believe that it's irrational! – Elliott May 30 '11 at 23:50
  • 1
    It is 0.00110101000101000101... The bits at prime indices are set (you may want to be more careful than me about off-by-one errors). It is irrational because primes are not periodic. – Dan Brumleve May 30 '11 at 23:51
  • 3
    How to prove this number is transcendental? – Dan Brumleve May 31 '11 at 00:20