0

!!! Layman here so please avoid complex math and answers.

Random (usually pseudorandom) events are usually characterized along these lines:

  1. Each outcome in a trial experiment must be i.i.d.; i.e. it has no effect on subsequent outcomes, thus individual outcomes cannot be predicted using past data as there is no obvious causal link
  2. Large sequences of outcomes are predictable, because they exhibit a pattern of stabilizing relative frequencies, such that no individual outcome is "preferred" and dominates the rest

The prevailing thought in probability theory (frequentism) is that stabilizing relative frequencies are an objective phenomenon, independent of human thought. This assumption has served statisticians, casinos and insurance companies well. What this basically implies is that large sequences of similar random events are consistent and their averages can be confidently predicted within a "sufficiently" large sample.

Why can we predict the averages of big samples of individually unpredictable random events?

Community
  • 1
vantage5353
  • 767
  • 2
    This seems to be a question about the philosophy of mathematics, and might also get good answers on Mathematics StackExchange. Since probability theory is central to science, it is also a question about the philosophy of science. I am not sure this is really a question about physics though. – Mark Mitchison Feb 03 '15 at 11:08
  • @MarkMitchison, assuming stabilizing relative frequencies is an objective phenomenon, I'd like to know why it is so. I need a more fundamental answer involving physical laws – vantage5353 Feb 03 '15 at 11:10
  • 1
    Yeah, I understand the point, and it is a very interesting question. Nevertheless, this is not "ObjectivePhenomena.StackExchange", for the very good reason that "objective phenomenon" is a slippery philosophical concept whose discussion bores the majority of physicists to death. In this particular case, I do not personally agree that "stabilising relative frequencies" should be called an objective phenomenon. It is a property of correlations between an observer and the observed system and depends on the model adopted. – Mark Mitchison Feb 03 '15 at 11:14
  • 2
    It looks as you are essentially asking why if an individual coin toss is random do we get the predictable result of 50% heads and 50% tails in the long term. Is that a fair summary? If so, this is the law of large numbers – John Rennie Feb 03 '15 at 11:23
  • @MarkMitchison OK. Can I somehow transfer the question if it is in inappropriate section? – vantage5353 Feb 03 '15 at 11:25
  • @JohnRennie No. I am asking why we observe LLN in the real world. – vantage5353 Feb 03 '15 at 11:26
  • 1
    I'm still not sure what you're asking. The Wikipedia article contains proofs of the LLN for truly random events. Are you asking why those proofs apply to real events that aren't truly random? – John Rennie Feb 03 '15 at 11:29
  • @JohnRennie LLN is mathematically proven, but can never be proven empirically. I'd like to know what is the physical cause of outcomes stabilizing in the long run and why we observe predictable patterns in large samples but not in small samples of random events. – vantage5353 Feb 03 '15 at 11:32
  • 1
    I don't think it's clear where this question should be transferred to, as the question itself isn't clear. If @user1891836 wonders why the LLN works in the real world, he is essentially wondering why the real world obeys laws, and we'd better send him off to Philosophy.SE. – MSalters Feb 03 '15 at 11:32
  • There is no physical cause. The LLN applies approximately to real systems because real systems are approximately perfectly random. Where the deviation from perfect randomness becomes significant the LLN doesn't apply, but then the probabilities become history dependant so you wouldn't expect it to. – John Rennie Feb 03 '15 at 11:35
  • @MSalters Your condescension aside, I am not questioning determinism; I just want to know if LLN is a fundamental law at all. You probably didn't get to the end of my question though... – vantage5353 Feb 03 '15 at 11:35
  • 1
    No, the LLN isn't a fundamental law. It's an emergent property. – John Rennie Feb 03 '15 at 11:35
  • @JohnRennie OK, John. Please define the necessary conditions for LLN to emerge. – vantage5353 Feb 03 '15 at 11:39
  • The events have to be (approximately) random. If you're asking for a mathematical treatise on how deviations from the LLN depend on the degree on non-randomness then you definitely want the Maths SE not the Physics SE. – John Rennie Feb 03 '15 at 11:40
  • 1
    @JohnRennie I am definitely not expecting complex math. I just can't understand how individually unpredictable events can have predictable long run averages. Hope that makes sense – vantage5353 Feb 03 '15 at 11:42
  • 4
    Now it sounds as if you want a heuristic justification for the LLN, rather than the admittedly esoteric proof given in Wikipedia. If so, that's a challenge because it seems intuitively obvious to most of us. – John Rennie Feb 03 '15 at 11:45
  • @JohnRennie So I simply have to accept that LLN is just in the nature of large samples of random events? I.e. It's characteristic of what makes randomness random? – vantage5353 Feb 03 '15 at 11:48
  • 3
    This could be on topic at Statistics.SE as well as Math.SE. Either way, it's not really a physics questions. – Kyle Kanos Feb 03 '15 at 13:40
  • @Martin Care for a chat? – vantage5353 Feb 03 '15 at 14:43
  • Possibly relevant: https://math.stackexchange.com/q/777493/96384 – Torsten Schoeneberg Oct 16 '22 at 02:45

3 Answers3

1

The single determining characteristic that is required for the emergence of the Law of large Numbers is that the various random events are independently random (or at least sufficiently so).

If I had a coin that I flip once, and then observe repeatedly, then those observations won't be independent. They'll be random, for sure: I cannot predict the result of the 2nd observation up front, but I can predict the result of all future observations after I observed the coin once. The LLN does not hold here.

But if I flip the coin as little as once every 1000 observations, then those observations are already sufficiently independent for the LLN to kick in. After all, the first observation now predicts only a minuscule fraction of the next million observations.

MSalters
  • 226
  • I guess it is just strange for me that large sequences of i.i.d events exhibit a predictable pattern consistently. If it cannot be deduced from other laws, I have to accept it as a basic property of randomness. – vantage5353 Feb 03 '15 at 12:20
  • @user1891836: The whole key is that there is no pattern ! A pattern implies repetition, which directly violates your "independendant" assumption. If there's a pattern of length 7, you do't need the law of large numbers; the number 7 is already large enough. – MSalters Feb 03 '15 at 12:23
  • Well, the pattern is not that you can predict the position of a certain outcome in a sequence, but that you can predict the distribution of outcomes in a sufficiently large sequence. After all, that's what casinos and insurance companies do. They don't know when an event will occur, but they expect it will occur at some time within a sample of a big enough size. – vantage5353 Feb 03 '15 at 12:27
  • @MSalters: I believe you're missing the point; I suspect the question sthat user1891836 wants to have answered is: Why is it legit to model a physical process like a coin toss as a random variable? Why should a series of such processes be treated equally (or at least all share an expectation value)?; the law of large numbers does not help with that as these are the assumptions that go into the theorem... – Christoph Feb 03 '15 at 12:59
  • @Christoph You put it way better than me! I am interested in precisely why it is reasonable that we assign equal probabilities to i.i.d events. Care for a chat? – vantage5353 Feb 03 '15 at 13:36
  • I already addressed the "independent"part of IID. The other part, Identically-Distributed literally means Equal Probabilities. We don't "assign" equal probabilities to IID events, that follows by definition. – MSalters Feb 03 '15 at 14:44
  • @user1891836: Help me. Are you asking why it is legit to make symbolic predictive mental models of physical things, and to include in those models a concept of probability, as a way of saying there are things we don't know, so our predictions are not fully specific? – Mike Dunlavey Feb 03 '15 at 15:22
  • @MikeDunlavey I can explain better in this discussion but to answer your question: yes to a degree. What bothers me is whether we can rely on an objective LLN or if probability is just an inference tool. – vantage5353 Feb 03 '15 at 15:31
1

Toss a fair coin. For a single drawing the expected proportion of heads is distributed as follows (for $0$ and $1$):

$$\frac12, \frac12$$

For ten drawings in a row, by simple probability computation (from $0$ to $10$):

$$\frac{1}{1024}, \frac{10}{1024}, \frac{45}{1024}, \frac{120}{1024}, \frac{210}{1024}, \frac{252}{1024}, \frac{210}{1024}, \frac{120}{1024}, \frac{45}{1024}, \frac{10}{1024}, \frac{1}{1024}$$

The probability of very skewed drawings, like 10 heads in a row, goes decreasing with the length of the sequence, and the more "compliant" distributions are much more likely.

So completely deviating means are still possible, but with a smaller and smaller probability. The probability distribution always tend to concentrate around the mean. (The standard deviation is divided by the square root of the sequence length.)

0

Note that as empirical scientists, physicists don't really need any more than the ability to make testable predictions to justify a theory.

According to our current understanding, the most basic laws of nature (quantum theories) are probabilistic. As far as we know, that's just how the world works, and if there's a deeper reason behind it, we haven't been able to find it.

The prime example for a probabilistic system due to quantum effects would be radioactive decay.

At macroscopic levels, there are several ways for probabilities to emerge, and here are two ways I thought of:

First, we can have a family of interacting periodic systems. If you look at them only for a short time, they appear random; but if you waited for the least common multiple of the periods (or any time 'long enough'), they'll behave perfectly regularly.

A second example would be chaotic systems, which are effectively unpredictable - but this does not mean that you're equally likely to find the system in any particular region of phase space.

Now, what happens when you have a large number of such systems? Then, we have results like the law of large numbers or the central limit theorem. From a practical point of view, in many cases, you just end up either with random noise or a bell curve.

Now, let's look at a particular example: Tossing a coin.

Let's assume we're operating under ideal conditions, and the result of any toss only depends on two parameters: The coin's spin and its velocity.

The result will be periodic in both initial conditions, and if you overlay a source of noise, it's easy to see how you can end up with 50/50 probabilities if the parameters are right. The randomness of the coin toss got pushed back to the randomness of the noise.

Christoph
  • 186
  • Can you please elaborate on the "interplay of periodic and chaotic systems" and how this can lead to the emergence of stabilizing relative frequencies? – vantage5353 Feb 03 '15 at 11:57
  • @user1891836: There are no stabilizing frequencies. After a coin comes up head, further throws are still equally random which means tail hasn't become more likely. Of course, the average effect of that first single head is diluted as you flip more and more coins. But that's back to LLN. – MSalters Feb 03 '15 at 12:01
  • @MSalters I understand the Gambler's fallacy and that probabilities remain the same each trial. However, large samples of i.i.d outcomes exhibit a peculiar pattern: the averages tend toward the expected value, even though individual outcomes cannot be predicted – vantage5353 Feb 03 '15 at 12:06
  • 1
    @user1891836: the deviation from the average doesn't tend towards the expected value - actually it increases as $\sqrt{N}$. However the ratio of the deviation to the number of trials does tend to zero because it varies as $\sqrt{N}/N = 1/\sqrt{N} \approx 0$ for large $N$. – John Rennie Feb 03 '15 at 12:10
  • @user1891836: see edit – Christoph Feb 03 '15 at 12:40
  • @Christoph You seem to have really understood my position. Care for a chat? – vantage5353 Feb 03 '15 at 13:38
  • @user1891836: sorry, not time for that, but I just re-wrote my answer to clarify what I was getting at – Christoph Feb 03 '15 at 14:42