Uniform distribution on $\mathbb Z$ or $\mathbb R$

Question

I was assisting once the course in Probability Theory where students learnt quite quickly that there are ways to assign the uniform distribution to any finite set - or even subsets of $\mathbb R$ of a finite measure. They also clearly understtod the proof that there are no ways to put a uniform distribution (as a probability measure) on $\mathbb Z$ or a uniform distribution in a sense of a constant density on $\mathbb R$. But then I was asked the question:

Why is it impossible? What is intuitive meaning of this?

The only answer I could find that time refers to $\mathbb R$: well, the density is just a density w.r.t. Lebesgue measure - so why do we want the uniform distribution to have a constant density w.r.t. precisely Lebesgue measure? (as for me, if we omit the algebraic structure on $\mathbb R$, it would be just a measure).

However, I couldn't answer the question why there is no a uniform distribution on $\mathbb Z$ where the sentence "each value comes with equal probability" does have sense, I hope. Now this question bothers me myself and any help is appreciated.

Answer 1

If you have a uniform distribution $\mu$ on $\mathbb{R}$, it should assign the same value $v$ to $[z,z+1)$ for every $z\in\mathbb{Z}$. But if $v=0$, you have $\mu(\mathbb{R})=\sum_{z\in\mathbb{Z}}0=0$. On the other hand, if $v>0$ then $\mu(\mathbb{R})=\sum_{z\in\mathbb{Z}}v=\infty$. So $\mu(\mathbb{R})=1$ can not be when $\mu$ is countably additive, which is the usual assumption in probability. The same argument applies to the nonexistence of a uniform distribution on $\mathbb{Z}$, when you look at the probability assigned to each point.

Let's say that a uniform distribution on $\mathbb{Z}$ is a function $\mu:\mathcal{A}\to[0,1]$, with $\mathcal{A}$ being an algebra, such that, $\mu(\emptyset)=0$, $\mu(\mathbb{Z})=1$, $A,B\in\mathcal{A}$ and $A\cap B=\emptyset$ implies that $\mu(A\cup B)=\mu(A)+\mu(B)$ (so we require $\mu$ to be only finitely additive). Lastly, the uniformity is expressed by the property that for all $A\in\mathcal{A}$, $A+z=\{a+z:a\in A\}\in\mathcal{A}$ and $\mu(A)=\mu(A+z)$.

Under these assumptions, a uniform distribution on $\mathbb{Z}$ exists. Let $\mathcal{A}$ consist of all subsets of $\mathbb{Z}$ that are finite or have a finite complement. This is an algebra. Let $\mu(A)=0$ if $A$ is finite and let $\mu(A)=1$ if $A$ has a finite complement. You can verify that this satisfies all requirements we made on a uniform distribution. It is even possible to generate such a uniform distribution with $\mathcal{A}=\mathcal{P}(\mathbb{Z})$, by using the Hahn-Banach theorem. You can find some details here, together with other notions of uniformity.

So the non-existence of a uniform distribution on $\mathbb{Z}$ is essentially a matter of accepting countable additivity. There is no "elementary" reason.

Answer 2

Let's look at the case of $\mathbb Z$ first. A natural way to define the uniformity of a probability distribution on a discrete set like $\mathbb Z$ is to say that a random variable $X \in \mathbb Z$ is uniformly distributed iff, for any subsets $A, B \subset \mathbb Z$,

$$\|A\| = \|B\| \iff \mathrm P(X \in A) = \mathrm P(X \in B),$$

where $\|S\|$ denotes the number of elements in the set $S$.

Now, let's make the further assumption that there exists some constant $M \in \mathbb Z$ such that $\mathrm P(|X| \le M) = \varepsilon > 0$. (A probability distribution that did not satisfy this assumption would have to be a very curious beast indeed, since a random variable so distributed would exceed any finite bound with probability $1$.) Let $A = \{-M,\, \dotsc,\, M\}$. Then we can decompose $\mathbb Z$ into countably many disjoint subsets $A_k = A + (2M+1)k$ such that $\|A_k\| = \|A\| = 2M+1$ for all $k \in \mathbb Z$, and thus $\mathrm P(X \in A_k) = \varepsilon$ for all $k \in \mathbb Z$. But then

$$\mathrm P(X \in \mathbb Z) = \sum_{k \in \mathbb Z}\, \mathrm P(X \in A_k) = \sum_{k \in \mathbb Z}\, \varepsilon = \infty.$$

Indeed, we can even take a finite union of sufficiently many sets $A_k$ to obtain a finite set $B$ such that $\mathrm P(X \in B) \ge 1$, which is clearly absurd for a probability distribution.

Similarly, a natural property one would require of a random variable $X$ uniformly distributed over $\mathbb R$ would be that, for all $a,b,c,d \in \mathbb R$, $a \le b$, $c \le d$,

$$b-a = d-c \iff \mathrm P(X \in [a,b]) = \mathrm P(X \in [c,d]).$$

We can then carry out a similar argument as above to show that, if there exists an $M \in \mathbb R$ such that $\mathrm P(|X| \le M) = \varepsilon > 0$, then there exists a bounded set $B \in \mathbb R$ such that $\mathrm P(X \in B) \ge 1$.

Of course, this is not a complete proof of the impossibility of a uniform probability distribution over $\mathbb R$ (or $\mathbb Z$), since I haven't actually proven that there might not be a strange probability distribution for which $\mathrm P(|X| \le M) = 0$ for all $M \in \mathbb R$, yet somehow $\mathrm P(X \in \mathbb R) = 1$. But hopefully I've at least convinced you that, if a uniform probability distribution over $\mathbb R$ or $\mathbb Z$ were to exist, it would have to seriously violate our intuition about how a proper distribution ought to behave.

Answer 3

Think of it this way. You start dividing Z and R into N equal slices over a histogram. The overall probability has to sum to 1. So to know the probability to assign to each slice of the histogram you need to know the finite number of slices you are going to make. But Z and R are infinite in size. So you can't slice Z and R into any particular finite number of equal slices.

It is the equal size & equal probability of each slice that is the problem. Of course there are functions which have probability of 1 over infinity. For instance the normal distribution is defined for all values of X from -infinity to +infinity, and it sums to 1.

---

As an example consider dividing the set of integers into groups of 10. So if I have integers between 1 and 100, then I have 10 groups. Each group thus has a probability of 1/10 of being randomly chosen when I choose a random integer between 1 and 100.

But what happens if I extend set of integers to Z? I end up with an infinite number of groups and 1/infinity is undefined as far as probability goes. So I can't define a probability for a particular group of 10 being chosen.