In frequentism, does every event have a probability?

Question

An experiment is a function $\mathrm{Ex}:\mathbb{N}\rightarrow\Omega$ with $\Omega$ representing possible outcomes, and $\mathrm{Ex}(n)$ representing the outcome of the $n^\mathrm{th}$ trial.

$A\subseteq \Omega$ is said to be an event if the following limit is defined: $$P(A):= \lim_{N\rightarrow\infty} \frac{|\{k|\mathrm{Ex}(k)\in A\}\cap\{1,2,\dots,N\}|}{N}$$

$P(A)$ is called the frequentist probability of $A$.

The axioms of probability (nonnegative, countably disjoint-additive, $P(\Omega)=1$) follow immediately from simple limit properties.

Say $\Omega=\mathbb{R}$, and we know that $P((-\infty,x])=F(x)$, some known distribution function. With this information, we can find the probabilities of all the Borel sets. (by applying the 'probability axioms')

In this situation, if $A$ is some non-Borel set of reals, can there be a meaningful value of $P(A)$?

• If we have $B\supset A$ and we know $P(B)=0$, then it's clear that $P(A)$ should be $0$.
• A satisfying answer would be a proof that the limit can't exist.
• Another possibility is that $P(A)$ may exist, we just don't know what it is with the given information. (i.e. you would have to know $\mathrm{Ex}$)

Answer 1

Even after your edits, you still seem to misunderstand the basic definition of an event:

1. Events $A$ live in a probability space $(\Omega, \mathcal{F},P)$, specifically $A \in \mathcal{F}$
2. However, before you can get a probability space, you need $(\Omega, \mathcal{F})$ to be a measurable space. This automatically disqualifies "pathological" sets from being events (like Vitali sets on the reals).
3. For $\Omega=\mathbb{R}$ you are typically taking $\mathcal{F}=\mathcal{B}(\mathbb{R})$, which also does not include non-measurable sets.

So, the bottom line is that for $P(A)$ to make sense, it must be part of a space with finite measure. Fore example, the Lebesgue measure over the reals does not qualify a valid probability measure.

As for a fundamental connection to frequentism: measure theory generalizes the concept of probability and frees it from any particular interpretation. Therefore, frequentism can be seen to arise from the measure theoretic formulation by equating the long run frequency with the probability measure. Its not the other way around. Also, Bayesians would have a very different sense of "intuitive" for their basis of probability.

Response to OP's new question

A great example (already provided by @Chris Janjigian) are the Vitali Sets under a uniform probability measure on $[0,1]$. This post has a great explanation of why such sets do not have a probability measure (or any measure for that matter). Unfortunately, the Vitali sets are highly abstract and so a concrete example is not possible. Another good example of a non-Borel set is here. How would you assign a probability to the sets defined in this latter post?

OK, now to your definition of frequentist probability:

---

First, your assertion that the axioms of probability follow from your definition is circular, and hence incorrect. Let's think about what sample paths of your limit would look like for finite $N$

$$P_N(A):= \frac{|\{k|\mathrm{Ex}(k)\in A\}\cap\{1,2,\dots,N\}|}{N}$$

You'll note that $P_n(A)$ is not a random variable, since $Ex(k)$ is a deterministic sequence. Therefore, the existence of $\lim\limits_{N \to \infty} P_n(A)$ depends critically on how you define $Ex(k)$.

I'll work with a simple, concrete example:

$\Omega := \{0,1\}$ and lets define a subset of the natural numbers as follows:

$$B:=\bigcup\limits_{i=1}^{\infty} \{ k \in \mathbb{N}:2^{2i-1} \leq k < 2^{2i}\}$$

Note that $|\{ k \in \mathbb{N}:2^{2i-1} \leq k < 2^{2i}\}|= 2^{2i}-2^{2(i-1)}-1$ so that we have a union of disjoint sets of consecutive natural numbers, with exponentially increasing cardinality.

The experiment function will be $\mathrm{Ex}(k)= 1$ if $k\in B$ and $0$ otherwise. And lets take $A=\{1\}$ as our subset of the sample space.

Now, lets try to figure out $P(A)$ using your definition:

$$P(A):= \lim_{N\rightarrow\infty}\frac{|\{k|\mathrm{Ex}(k)=1\}\cap\{1,2,\dots,N\}|}{N}$$

By the definition of $B$, $\{k|\mathrm{Ex}(k)= 1\}$ will be an oscillating sequence consisting of exponentially increasing subsequences of just $0$'s or just $1$'s. Thus, the limit will continually ocillate between $0$ and $1$ without converging. Hence, $P(A)$ is undefined.

The problem with your definition is that $\mathrm{Ex}(k)$ is arbitrary, and hence there is not guarantee that your limit will converge. In order to do that, you would need to $assume$ that each event occurs with a stable frequency $P(A)$...however, that would be question begging/circular, since you are using $\mathrm{Ex}(k)$ to define $P(A)$, but the convergence of the "experiment function" hinges on assumptions about the very frequencies you whose limit you are trying to derive.

This is what I was getting at by saying that frequentism is an interpretation of probability, not the definition of probability. The definition is given by Kolmogorov's axoioms, which are interpretation-neutral. They could just as easily apply to Bayesian "degrees of belief" or other, non-frequency-based weighting of outcomes. You can't boostrap your way to the axioms of probability using your definition of frequentest probability, as it implicitly assumes the existence of what it seeks to prove.