What is the meaning of "mutually exclusive outcome" in the classical definition of probability?

Question

I have known the definition of classical probability as follows:

If a random experiment $A$ results in $n$ mutually exclusive and exhaustive and equally likely outcomes, out of which $m$ of them are favorable to the happening of event $A$ then the probability of event $A$ is denoted by $P(A)$ and defined as $$P(A)=\frac{m}{n}$$

Easier texts just say that the sample space $S$ contains all possible outcomes of an experiment. And then they say if each of the outcomes are equally likely then probability is defined as $P(A)= \frac{|A|}{|S|}$. This definition simply avoids the use of the terms "mutually exclusive" and "exhaustive". I had seen this type of definition during the days of my middle school.

That the outcomes need to be exhaustive is clear to me. We are not supposed to leave out any outcome of an experiment from being included in the sample space.

But what I do not quite get the is meaning or use of the term "mutually exclusive" outcomes. When I had first seen the use of this term, I interpreted as follows suppose an experiment can result in each of the outcomes either in set $S_1=\{a,b,c\}$ or in set $S_2=\{b,c,d\}$ then the exhaustive sample space should be $S=\{a,b,c,d\}$ and the number of mutually exclusive outcomes are $n=|S_1|+|S_2|-|S_1 \cap S_2|$. i.e. I thought that we need to remove the possible overlap while considering the sample space as a whole.

Recently I came across a possible explanation of "mutually exclusive" outcomes as follows:

It means that the outcomes of an experiment are mutually exclusive [$\text{Outcome}_i \cap \text{Outcome}_j=\phi; \quad i\neq j$], i.e. an experiment cannot result in two outcomes simultaneously.

Can anyone point me out what is correct interpretation along with the intuition or mathematical logic/reasoning behind the same.

Answer 1

1. The sets $\{hh,ht\},\{th,tt\},\{hh,ht,th\}$ are called events, their elements $hh,ht,th,tt$ are called outcomes, and the single-outcome events $\{hh\},\{ht\},\{th\},\{tt\}$ are called elementary events.

   When a probability experiment is performed, a single outcome happens; an event is said to happen if it contains this outcome.

2. By definition, events being mutually exclusive means that their pairwise joint probabilities are $0,$ that is, each pair^҂ almost never or never both happen: $$\text{Events $X$ and $Y$ are mutually exclusive}\\\overset{\text{def}}{\iff}\\ P(X\cap Y)=0.$$

3. Clearly, $$\text{$X$ and $Y$ are disjoint (i.e., $X\cap Y=\emptyset)$}\\\implies\text{$X$ and $Y$ are mutually exclusive}.$$

   On the other hand,$$\text{$X$ and $Y$ are mutually exclusive}\\\kern.6em\not\kern-.6em\implies\text{$X$ and $Y$ are disjoint, i.e., $X$ and $Y$ both happening is impossible}.$$ For example, the events $(1,2]$ and $[2,3)$ are mutually exclusive but not disjoint.

^{҂ On the other hand, (mutual) independence for multiple events requires more than just pairwise independence.}

4. Recently I came across a possible explanation of "mutually exclusive" outcomes as follows:

   It means that the outcomes of an experiment are mutually exclusive $[\text{Outcome}_i \cap \text{Outcome}_j=\phi; \quad i\neq j],$ i.e. an experiment cannot result in two outcomes simultaneously.

   The author is referring to elementary events like $\{ht\}$ as “outcomes”.

   As such, since they are disjoint, from #3, they are indeed mutually exclusive.

5. When I had first seen the use of this term, I interpreted as follows:

   suppose an experiment can result in each of the outcomes either in set $S_1=\{a,b,c\}$ or in set $S_2=\{b,c,d\}$ then the number of mutually exclusive outcomes are $n=|S_1|+|S_2|-|S_1 \cap S_2|$. i.e. I thought that we need to remove the possible overlap while considering the sample space as a whole.

   For finite sets, $$|X\cup Y|=|X|+|Y|-|X\cap Y|;$$ in particular, $$\text{$X$ and $Y$ are disjoint}\iff |X\cup Y|=|X|+|Y|.$$ Generally, $$P(X\cup Y)=P(X)+P(Y)-P(X\cap Y);$$ in particular, from #3, $$\text{$X$ and $Y$ are mutually exclusive}\iff P(X\cup Y)=P(X)+P(Y).$$

Answer 2

An outcome is a set of atomic events (or atoms), such as $a, b, c$, and $d$ in your example.

Two outcomes--sets of atomic events--are "mutually exclusive" exactly when the two sets don't intersect, i.e. when the $A\cap B = \emptyset$. That is, your second definition of "mutually exclusive" is the correct, and general one.

In your example, $\{a,b,c\}$ and $\{d\}$ are mutually exclusive, as are $\{a,d\}$ and $\{b,c\}$. Your $S_1$ and $S_2$ are not mutually exclusive. Also $\{a\}, \{b\}, \{c\}$, and $\{d\}$ are mutually exclusive. And there are other examples using those four atomic events.

This gets a little bit trickier when the number of atomic events is infinite, even uncountably infinite, but the basic idea is the same: sets of events that don't intersect are mutually exclusive.

---

I suspect that part of what might be confusing you--I'm not sure--is that you're assuming that each atomic event is equally probable, and that you can therefore determine probabilities by counting them. It's not always the case though, that for example $\mathsf{P}(\{a\}) = \mathsf{P}(\{b\})$.

This is why the text you quoted refers to the outcomes considered as ones that are "equally likely"; that has to be specified for the indented claim to be correct.

For example, suppose $\mathsf{P}(\{a\}) = \mathsf{P}(\{b\}) = 1/3$, and $\mathsf{P}(\{c,d\})=1/3$. Suppose that the first two are the "favorable" events. Then there are $n=3$ "mutually exclusive and exhaustive and equally likely outcomes", of which $m=2$ of them are favorable, and the probability of a favorable outcome is 2/3, or more elaborately: $\mathsf{P}(\{a\})+\mathsf{P}(\{b\}) = 1/3 + 1/3$, even though $a$ and $b$ together only make up half of the four atomic events.