5

In Grimmett and Stirzaker's Probability and Random Processes (section 1.3), for two disjoint events $A$ and $B$, we have that

$\mathbb{P} (A \cup B) = \mathbb{P}(A) + \mathbb{P}(B)$

From this statement, the authors 'jump' and state that $\mathbb{P}$ should be finitely additive, and further along in the text, they 'jump' again and state that $\mathbb{P}$ should be countably additive.

My questions:

  1. Why is $\mathbb{P}$ only finitely additive? Isn't it possible to keep adding disjoint events ad infinitum : $\mathbb{P} (A_1 \cup A_2 \cup \ldots) = \mathbb{P}(A_1) + \mathbb{P}(A_2) + \ldots$
  2. What is the difference between finitely additive and countably additive? I know that finitely additive just means I have a fixed number of events I need to add up but I am not sure of what countably additive means.

I have browsed the (many) other posts around this topic but they start discussing measure theory which I haven't studied yet in my course (I'm at undergrad level).

Asaf Karagila
  • 393,674
  • Finitely additive means that if you have any finite set of events $A_1,\ldots,A_n$ which are pairwise independent (i.e. $A_i\cap A_j=\varnothing$ for $i\neq j$) then $\mathbb(A_1\cup\ldots\cup A_n)=\mathbb P(A_1)+\ldots+\mathbb P(A_n)$. Countable additivity means that we can do that with any sequence of events (which are pairwise disjoint), $\langle A_n\mid n\in\Bbb N\rangle$, and not just finite sequences. – Asaf Karagila Apr 20 '14 at 06:38
  • So, $\mathbb{P}$ being countably additive answers my first question because, as you state, that property is not just for finite sequences. –  Apr 20 '14 at 06:47
  • Yes, it answers part of your questions. Except the important(?) part, why is it just finitely additive. I don't know about that. – Asaf Karagila Apr 20 '14 at 06:51
  • 1
    Looking at your comments here it seems that the focus of interest is finite probability spaces. In that case, there's no harm in requiring only finite additivity, since there are not infinitely many distinct (let alone pairwise disjoint) events. – Asaf Karagila Apr 20 '14 at 07:39

2 Answers2

1

If $P(A \cup B) = P(A) + P(B)$ whenever $A,B$ are disjoints sets, then

for $A_1 \ldots A_n$ disjoints sets, one can consider $P(A_1 \cup A_2 \cup \ldots \cup A_n) = P(A_1 \cup B) $ where $B = A_2 \cup \ldots \cup A_n$ is disjoint from $A$.

From the first property: $P(A_1 \cup B) = P(A_1) + P(B)$

Proceed in the same manner $n$ times (or apply induction) to obtain that $P(A_1 \cup A_2 \cup \ldots \cup A_n) = P(A_1) + P( A_2 ) + \ldots P (A_n)$

And that is finite aditivity.

Finite additivity does not imply countable aditivity. For instance consider on $(\mathbb{N} , \mathcal{P}(\mathbb{N})\,)$ be such that $$P(A) = \begin{cases}0 & \text{ if } A^c \text{ is finite}\\ 1 & \text{ otherwise }\end{cases}$$

P is finite aditive but is not countably aditive

Conrado Costa
  • 6,071
  • 3
    The final example here is not correct. For this function the probabilities of the even numbers, odd numbers, and natural numbers are all $1$, violating finite additivity. Instead, take $P(A)=1$ if $A^C$ is finite, $0$ otherwise (see https://math.stackexchange.com/questions/186280/finitely-but-not-countably-additive-set-function ). – Kevin P. Costello Oct 03 '17 at 17:58
  • @KevinP.Costello Isn't $P(\mathbb{N}) = 0$ for this definition? – Andrei Kh Feb 24 '20 at 20:45
1

You are right, but at that point in the book (one of my favorites), on page 5 (3rd ed) G&S are trying to introduce the ideas of finite and countable additivity in an intuitive manner. They give the simple case of $P(A\cup B) = P(A) + P(B)$ and then say, in effect, what is a natural generalization of this? What should we require of the concept of probability? And they give the rule for finite additivity (which is actually implied by the first sum).

Then they say, in effect, "Hey, we could generalize this further!", and they state countable additivity as a reasonable principle for the concept of probability.

Yes, finite additivity follows from countable additivity. Make most of the sets in the union empty, and you can derive finite additivity as a theorem from countable additivity. (But not vice versa: if you use finite additivity as your axiom, countable additivity doesn't follow, in general.)

G&S were just trying to introduce the ideas one step at a time. The "jumps" are pedagogical devices, not rigorous inferences.

Mars
  • 1,338