0

Consider a coin where $P(\text{Head})=x$ where $0\leq x\leq 1$ and $x$ is a uniform random variable. After flipping the coin once, you get a head. What is the value of $P(\text{Head})$ on average?

I was very unsure about how to approach this question. I understand that based on the new information, $P(\text{Head})$ should be greater than $\frac{1}{2}$ on average because a coin that is biased towards heads is more likely to produce a head than a coin biased towards tails. However, I had no luck trying to calculate a weighted average.

Joe
  • 19,636

3 Answers3

2

Imagine that instead of flipping the coin, you draw a random number $r$ uniformly from $[0,1]$ and compare it to the coin’s probability $p$ of showing heads; the result is “heads” if $r\le p$. Now you want to know the expected value of $p$. This is the probability that yet another random number $s$ uniformly randomly drawn from $[0,1]$ is less than $p$.

So you have three random numbers independently uniformly drawn from $[0,1]$. A priori all $3!=6$ of their orders are equiprobable by symmetry. The given information that $r\le p$ (i.e. that the result of the toss was “heads”) is compatible with $3$ of these orders: $r\le p\le s$ or $r\le s\le p$ or $s\le r\le p$. In $2$ out of these $3$ equiprobable orders, $s\le p$, so the probability that $s\le p$, which is the desired expected value of $p$, is $\frac23$.

joriki
  • 238,052
  • I just revisited this problem and was shocked by the elegance of your solution. Thank you very much indeed! – Joe Aug 01 '20 at 02:37
  • @Joe: You're very welcome indeed! In that case, you might also be interested in my answers to these questions, where I used similar approaches: https://math.stackexchange.com/questions/3515829, https://math.stackexchange.com/questions/3514574, https://math.stackexchange.com/questions/3640533. – joriki Oct 08 '20 at 09:55
1

Imagine you have lots of coins with probabilities of heads uniform in $[0,1]$. You flip them all and keep the ones that come up heads. Now you are asked the average probability of the ones you keep. That means you keep the one that comes up heads $0.1$ of the time with probability $0.1$ and the one that comes up heads $0.9$ of the time with probability $0.9$. Can you draw the graph of chance of heads vs. probability of keeping? Now you want the average chance of heads over the region.

Ross Millikan
  • 374,822
  • Thank you for responding. So I think that the equation of the graph would be $y=x$. If the chance of heads coming up is $0.4$, for example, then the probability of it being kept is also $0.4$. This is because a coin will always be kept when it comes up as heads. I don't know where to go from here. – Joe May 07 '20 at 18:48
  • Yes, that is correct. Your pdf is proportional to this, but needs to be normalized so the integral is $1$. Then the expected value of $x$ is $\int_0^1 x\cdot pdf(x)\ dx$ – Ross Millikan May 07 '20 at 19:10
0

Unfortunately, the answers given here were a little bit too advanced for me, but I appreciate the efforts of @joriki and @Ross Millikan. Here is the approach that I ended up using. I'd be very pleased if someone could formalise it, although perhaps that too would also be too advanced!

So before considering the case of picking a random real number between $0$ and $1$, I thought about this scenario: consider the case where $P(\text{Head})$ could take only take $1$ out of $11$ possible values. They are $[0,0.1,0.2,0.3,\dots,0.9,1]$. I should add that each of these values is equally likely to be selected.

What is the chance of $P(\text{Head})$ being $0.1$ and a head coming up? Since these are $2$ independent events, $P[P(\text{Head})=0.1 \text{ and Head}] = \frac{1}{11} \times 0.1 = \frac{1}{11} \times \frac{1}{10} = \frac{1}{110}.$

Similarly, $P[P(\text{Head})=0.2 \text{ and Head}] = \frac{2}{110}$.

To try and make this scenario as intuitive as possible, I imagined what would happen if I flipped $110$ coins and everything occurs according to the theoretical probabilities.

For instance, there should be $2$ coins where $P(\text{Head})=0.2$ and the coins are showing heads.

Then, I discard the $55$ coins that are showing tails. There are $55$ coins remaining, all of which are showing heads. This means that $P[P(\text{Head})=0.2 \text{ and Head}]$ doubles from $\frac{2}{110}$ to $\frac{2}{55}$. Now I will consider the weighted average, again trying to make the process as intuitive as possible.

$$\frac{1(0.1)+2(0.2)+3(0.3)+\cdots+10(1)}{55}$$

So of the $55$ coins, $1$ will have $P(\text{Head})=0.1$, $2$ will have $P(\text{Head})=0.2$, and so on. Using a calculator, the weighted average ends up being $0.7$, which is not far from the true answer of $0.\overline{6}$ when you can pick any real number between $0$ and $1$.

I presume that as my approximation becomes more and more refined, e.g. I pick a value of $P(\text{Head})$ from $[0,0.01,0.02,\cdots,0.99,1]$, the answer tends towards $\frac{2}{3}$. This is where the formal aspects of mathematics hinder me, as I don't know if this is the proper way of approaching the real numbers. Perhaps someone can clarify.

Thank you for reading.

Joe
  • 19,636