9

I'm a beginning student of Probability and Statistics and I've been reading the book Elementary Probability for Applications by Rick Durret.

In this book, he outlines the 4 Axioms of Probability.

  1. For any event $A$, $0 \leq P (A) \leq 1$.
  2. If $\Omega $ is the sample space then $P (\Omega) =1$.
  3. If $A$ and $B$ are disjoint, that is, the intersection $A \cap B = \emptyset$, then $$P(A\cup B) = P(A) + P(B)$$
  4. If $A_1, A_2,\ldots$, is an infinite sequence of pairwise disjoint events (that is, $A_i\cap A_j = \emptyset$ when $i \neq j $) then $$P\left(\bigcup_{i=1}^\infty A_i\right)=\sum_{i=1}^\infty P(A_i).$$

The book fails to explain why we need Axiom 4. I have tried searching on Wikipedia but I haven't had any luck. I don't understand how we can have a probability of disjoint infinite events. The book states that when the you have infinitely many events, the last argument breaks down and this is now a new assumption. But then the book states that we need this or else the theory of Probability becomes useless.

I was wondering if there were any intuitive examples of situations where this fourth axiom applies.

Why is it so important for probabilty theory? And why does the author state that not everyone believes we should use this axiom.

BCLC
  • 13,459
Brother John
  • 93

2 Answers2

9

Notice that if $A,B,C$ are pairwise disjoint, you have \begin{align} \Pr(A\cup B\cup C) & = \Pr(A\cup B)+\Pr(C) & & (\text{since }A\cup B\text{ is disjoint from }C) \\[6pt] & = \Pr(A) + \Pr(B) + \Pr(C) & & (\text{since }A\text{ is disjoint from }B) \end{align} and the same can be done with any finite sequence of pairwise disjoint events just by applying Axiom 3 repeatedly (or to put it another way, by mathematical induction).

However, there are assignments of numbers to subsets of a set that satisfy the first three axioms while failing to satisfy the fourth. The fourth axiom rules out such systems. Here is a simple example: Suppose $A\subseteq \{1,2,3,\ldots\}$. Let $$ P(A) = \lim_{n\to\infty} \frac{|A \cap \{1,\ldots,n\}|} n. $$ For some sets (for example the set of all integers whose first digit is $5$) this limit fails to exist. However, for pairwise disjoint sets $A$ and $B$ for which the limit does exist, the third axiom holds, as do the first two. But the fourth axiom fails: $$ P\left( \bigcup_{n=1}^\infty \{n\} \right) = P(\{1,2,3,\ldots\}) = 1 \ne 0 = \sum_{n=1}^\infty 0 = \sum_{n=1}^\infty P(\{n\}). $$ Hence the fourth axiom is not redundant.

Now suppose you throw a coin repeatedly until the first time you get a "head". What is the probability that the number $X$ of trials is even? It is $$ \Pr(X\in\{2,4,6,8,10,\ldots\}) = \sum_{n\text{ even}} \Pr(X=n). $$ There you use Axiom 4.

\begin{align} & \Pr(X\in\{2,4,6,8,10,\ldots\}) = \sum_{n\text{ even}} \Pr(X=n) \\[6pt] = {} & \sum_{n\text{ even}} \Pr(\text{“tail'' on first }n-1\text{ trials and “head'' on } n \text{th trial}) \\[6pt] = {} & \sum_{n \text{ even}} \left( \frac 1 2 \right)^n = \frac{1/4}{1-1/4} = \frac 1 {4-1} = \frac 1 3. \end{align}

Michael Hardy
  • 1
2

Another thing to point is that the fourth axiom gives us a way to deal with countably infinite sequence of events. In another word, it allows us to take limit! $$P\left(\bigcup_{i=1}^\infty A_i\right)=\sum_{i=1}^\infty P(A_i)=\lim_{n\to \infty}\sum_{i=1}^n P(A_i).$$ While the first three axioms can only deal with finite events.

lzstat
  • 411