3

If I have a continuous distribution of random variable $X$ then $P(X = k) = 0$ where $k$ is some specific value. I understand that this doesn't imply impossibility, as in the real world, no variable is completely continuous. We have limited precision for measuring physical phenomena, so while impossible from the math, it's physically possible.

But setting aside the "real-world" argument, if we're talking about probability theory in the context of measure theory, what does $P(A) = 0$ mean? If $A$ isn't impossible, what does $0$ mean, if anything? This post suggests that the $0$ assigned by the probability measure is a result of a limit, that $A$ is so unlikely that when performing a large number of trials, the frequency approaches $0$.

So then why not distinguish between $P(A) = 0$ and $P(A)$ approaches $0$ (like an infinitesimal) where the former represents $A$ being impossible, that is, $A = \emptyset$ and the latter meaning an event $A$ is possible, just extremely unlikely? Why does the probability measure assign $0$ to both?

Andrew Li
  • 4,554
  • 1
    I don't follow. What makes you think that such a distinction isn't already made? What makes you think that such a distinction should be visible based on the value that the probability measure assigns? – JMoravitz Apr 14 '20 at 21:04
  • @JMoravitz I haven't delved into measure theory that deeply, since I'm going based off my own curiosity. I think the reason why I think it should be visible is that that's the intuition, that impossible events and highly unlikely events are different. I'd love to understand if there's a reason or not why intuition isn't followed or if my intuition is wrong. – Andrew Li Apr 14 '20 at 21:08
  • This is like asking "why does the empty set have the same area as a single point in the plane?" – Maximilian Janisch Apr 14 '20 at 21:57

2 Answers2

1

This is my take on it, roughly:

The probabilities we assign have to be real numbers, so that we can make calculations with them basically. In real numbers, there is no way to distinguish between $0$ and an infinitesimal number. I am sure there are probably non-standard approaches to probability that may use infinitesimals, but this is not standard.

There is hope though. If we are to consider an outcome with $P(X=x)=0$ possible, it should mean that outcomes close to $x$ are also possible. This is exactly what is captured by a probability density function (PDF). A PDF is nonzero at $x$ if and only if the outcome $X=x$ is possible in that sense (up to some measure-theoretic nit-picking about sets of measure zero). This of course assumes that the outcomes live in some continuous space.

What then, if we had an isolated possible event? What I mean is, think for example of a dart throw. Let's say the dartboard is the unit disk $\{\Vert x\Vert\le 1\}$ in the plane. But then add a single point the disk, say $x=(2,0)$. If we throw a dart with some sort of uniform probability on this extended disk, then $P(X=(2,0)) = 0$ (where $X$ is the point we hit). Would you say it is possible to hit $(2,0)$? I don't think it makes sense to say that. I actually don't even think we could define a formal probability distribution, where $X=(2,0)$ is a possible outcome with probability $0$. On the other hand $P(X=(0,0))=0$, and this is clearly a possible event. All this corresponds to the PDF, which would have positive values on the unit disk, and $0$ elsewhere.

In short, we do distinguish between possible and impossible outcomes with PDF's. Therefore, we would not gain anything from distinguishing possible and impossible cases of $P(A)=0$.

EDIT: As Maximilian Janisch points out, we can actually (easily) make distributions like the one above. As a concrete example, let $Y\sim U([-1,1])$, and define $X=\begin{cases}Y & Y\ne0 \\ 2 & Y=0\end{cases}$. I can see that this breaks my argument a little, because now $X=2$ is possible just as much as $Y=0$ is.

In the end, I will maybe change my standpoint to this: In a continuous setting, it just isn't interesting to look at single points.

Milten
  • 7,031
  • 1
    We can definitely construct such a random variable: Take $Y=\text{Uniform}({\lVert x\rVert\le 1})$ and then $X=Y$ except at one point $\omega_0\in\Omega$ we have $X(\omega_0)=(2,0)$. Then $X\sim\text{Uniform}({\lVert x\rVert\le 1})$ but $(2,0)$ is a possible outcome of $X$ – Maximilian Janisch Apr 14 '20 at 21:55
  • 1
    @MaximilianJanisch You are right, of course, thank you. – Milten Apr 14 '20 at 21:56
  • So to distinguish between P(A) = 0 meaning impossible and P(A) = 0 meaning possible just extremely unlikely, one must turn to the PDF. So is it valid to say that the probability measure is not equipped to distinguished impossible events because its range is only $[0, \infty)$ and any nonzero probability would imply possibility? – Andrew Li Apr 14 '20 at 23:46
1

$\def\Fcal{\mathcal{F}}$

But setting aside the "real-world" argument, if we're talking about probability theory in the context of measure theory, what does $P(A) = 0$ mean?

In the context of measure theory, a fundamental concept in probability theory is called probability space, which consists of three elements:

  • A sample space $\Omega$;
  • A $\sigma$-field $\Fcal$ on the set $\Omega$;
  • A probability measure $P:\Fcal\to[0,1]$.

For any element $A\in\Fcal$, which is often called an event, it is associated with a unique real number $P(A)$ in $[0,1]$. The identity $P(A)=0$ simply means "the probability measure of the event $A$ is zero".

If $A$ isn't impossible, what does $0$ mean, if anything?

You are now confusing yourself going back and forth the "real world" and the mathematical model. If you are willing to "set aside the 'real-world' argument", then you will have to define what "A isn't impossible" mean mathematically first before asking anything meaningful about it.

So then why not distinguish between $P(A)=0$ and $P(A)$ approaches $0$ (like an infinitesimal).

There is no limiting process in the quantity $P(A)$, which is a constant. What you want is that $$ P(X=k) = \lim_{n\to\infty}P(k-\frac1n \leq X\leq k+\frac1n)=0\;. $$

To gain more "mathematical" intuition, you may find this recent 3Blue1Brown video helpful: Why “probability of $0$” does not mean “impossible”.

  • I happened to watch that 3b1b's video which prompted me searching about measure theory. So P(A) = 0 only means the element of the sigma algebra is assigned a measure zero by the probability measure. And that makes sense, there is no limiting process wrt P(A) in the context of measure theory. The takeaway is it's just assigned 0 and isn't bound by some intuition when it was developed, aside from the fact nonzero measures means possible? – Andrew Li Apr 14 '20 at 23:41
  • 1
    @AndrewLi: there is a limiting process: $ P(X=k) = \lim_{n\to\infty}P(k-\frac1n \leq X\leq k+\frac1n)=0 $, where you denote $A$ as the event $X=k$. It may be more helpful to think about the notation of length/area/volume in geometry. Consider the unit interval $[0,1]$, which has length $1$. If you randomly pick a point $x\in[0,1]$, which is of course a "possible" event, the length of a point is zero. You can view this from a "limiting process": $length({x}) = \lim_{n\to\infty}length([x-\frac1n,x+\frac1n]) = \lim_{n\to\infty}\frac{2}{n}=0$. –  Apr 15 '20 at 00:19