1

So for regression to the norm it says if someone has a high score on a test (relative to the average) then they are likely to score lower and lower on each following test? This seems very counter intuitive.

  • 3
    No, they regress to their personal norm. An individual isn't a random sample of the entire class. – Thomas Andrews Dec 27 '14 at 03:40
  • 2
    Also, regression to the norm doesn't mean that a fair coin that flips four heads in a row is more likely to get tails next... – Thomas Andrews Dec 27 '14 at 03:42
  • Related: http://math.stackexchange.com/questions/433492/regression-towards-the-mean-v-s-the-gamblers-fallacy –  Dec 27 '14 at 05:05

4 Answers4

1

As with all probability problems the setup is very important, perhaps you heard of the problem of 3 coin flips, if we know 2 of them are heads what is the probability the third is heads? Which has probability 1/4 This is because the third coin to fall heads is not fixed. Likewise consider if you were to pick a high score from the class. For any given person they are much more likely to be picked if they did relatively well for themselves on the test rather than relatively poorly (with respect to their own average). Another way of thinking about this is consider a class with 10001 people 10000 average 70 on tests which we call group B and 1 that averages 90 which we call group A. You pick a paper with grade 80 is it more likely it came from group A or B?. Of course B is the natural answer which means that this person is likely to score lower than 80 on the next test.

Hao S
  • 288
1

It's usually called regression to the mean in my experience (and you will get a lot more relevant hits from the math.stackexchange "search" box if you use that term rather than "regression to the norm").

Looking only at the sequence of scores earned by a single individual, the only regression to the mean you are likely to see (other than the obvious effects of gaining or losing skills over time) is regression to that individual's own mean score. A very bright student might tend to score two standard deviations above the population mean (which I'll abbreviate as $+2\sigma$), but (depending on the test) they might occasionally score $+2.5\sigma$ (on a good day) or $+1.5\sigma$ (on a bad day). Having gotten what is (for them) a much-better-than-usual or much-worse-than-usual score one time, the next time they take the test they are likely to get something closer to their usual score.

Looking at the entire population, however, the group of students who scored $+2.5\sigma$ on any given day will contain a larger percentage of students who tend to score $+2\sigma$ but who had a good day than students who tend to score $+3\sigma$ but who had a bad day, simply because $+3\sigma$ students are much rarer. Hence when you retest you will likely see that among all the students who scored $+2.5\sigma$ on the first test, the majority will score worse on the retest. But you will also have a new crop of students (a different set of individuals) who score $+2.5\sigma$ on the retest just because they are more alert than usual or simply got lucky that day.

David K
  • 98,388
0

Suppose every day I take a $12$ question multiple choice test where each question has $4$ possible answers, and I always choose my answer randomly. On average, I should expect to get $3$ correct answers per test. So, for example, getting $9$ or more correct answers is unlikely.

Suppose when I took the test yesterday, I got $9$ correct answers. Is it still unlikely that today, when I retake the test, I get $9$ or more correct answers?

Edit: If the assumption that the student guesses randomly bothers you, consider the following formulation: I take an $n$ question test each day, and answer each question correctly with probability $p$ (perhaps reflecting that there is some probability that I focused on certain types of questions over others, or that I forgot some definition, or $\ldots$, etc.). On average, I should expect to answer $np$ problems correctly on each test.

Now, suppose I answer $n$ problems correctly on the first such test. What is the probability that I do worse on the next test? We can compute $$\operatorname{Pr}[\text{score} < n] = 1 - \operatorname{Pr}[\text{score} = n]$$ $$= 1 - (1-p)^n$$ So, for $p < 1 - 2^{-n}$, $$\operatorname{Pr}[\text{score} < n] > \frac{1}{2} > \operatorname{Pr}[\text{score} = n]$$ and I'm more likely than not to score worse on the next test.

Put another way: if, without changing my study habits or understanding the material any better, I suddenly score much higher than usual on a test, then I probably just got really lucky. Since I don't usually get really lucky (otherwise, it wouldn't be called luck), I should expect to score something closer to average (lower) next time.

  • Strants the problem with your answer is that when I pick a guy with 9 correct answers there is no way of knowing that he got 9 right by random guessing, I think its very strange if that becomes your immediate conclusion. Your answer does not address the confusion if we take a student that scored well why be so pessimistic about their ability? – Hao S Dec 27 '14 at 20:54
  • @HaoSun My point isn't that the student had to have guessed: that would indeed be silly. Instead, I'm arguing that, if we assume that the student's scores for each test come from the same underlying random process (here, guessing was just a simple random process), then if a student gets an unlikely result on the first test (in particular, a result which is higher than average), then it is likely that the result of the next test will be 'more likely' than the first unlikely result (in particular, the second result will be closer to the average, so lower, than the first). –  Dec 27 '14 at 22:39
-1

They would score lower and lower, and then higher and higher, and then lower and lower, etc. such that the mean is realized over large $n$. If a higher score is indicative of some real improvement in the student's average ability (as you suggest) you would need to alter the mean to reflect this.

Chris
  • 863
  • This is incorrect. If we assume that the student's scores are coming from some random distribution with mean $\mu$, then we should expect that after a large number of tests, a single 'lucky' test score should affect the mean less, but the student won't (under the assumptions of independent tests) score lower to compensate in some way. See, in particular, this comment. –  Dec 27 '14 at 05:02
  • Whatever the random distribution is there must be some action to balance the high test score. More importantly if a student continues to do very well on tests it may indicate the parameters of the distribution need to be adjusted. – Chris Dec 27 '14 at 05:27
  • So yes it is not that they are more likely to have a lower score the next day, just that if the distribution is still correct there will eventually be some counterbalance. – Chris Dec 27 '14 at 05:38
  • Even that is not precisely correct. Suppose $x_1, x_2, \ldots, x_n$ are independent random samples taken from a mean $0$, variance $1$ normal distribution, and let's say we know that $x_1 = 5$. Since $E[x_i] = 0$ for each $i$, we have that $$E\left[\frac{1}{n}\sum_{i=1}^n x_i \mid x_1 = 5\right] = \frac{5}{n}$$ For large $n$, this will certainly be close to zero (the true expectation of the distribution), but it won't be 'counterbalanced' by somehow having some of the $x_i$'s be more negative. The comment I linked above has further discussion. –  Dec 27 '14 at 05:46
  • Right I completely agree with the comment you have linked as well. I feel like I may not have communicated my point well. I'm simply trying to say that in the real world when we model a problem it is possible that we have the wrong model and this may cause us to change the parameters of our problem. If we have the correct model then over time our data will fit nicely. – Chris Dec 27 '14 at 15:26
  • Ah, OK. I agree that if the average we observe doesn't match our expectation, then we should rethink our model. I guess I was concerned when you said they would score lower, then higher, then lower, since this would seem to indicate that score were not independent (which, in the real work, might be true, but I think for the sake of this question its probably better to assume they are independent). –  Dec 27 '14 at 23:15