1

I cannot see why to use the "sample standard deviation" instead of the classic "standard deviation" (the "population") ever, most explanations I find are just prescribing it instead of actually demonstrating it.

It is usually prescribed that when you have a sample of some values, from a population of values, you must use the "sample standard deviation" whose formula changes the original standard deviation formula, dividing by $n-1$, instead of dividing by $n$. The classic standard deviation formula is then limited to be used only when you have the whole population, de facto deprecating it for most uses, although still keeping a lot of ambiguity because it is usually not initially specified whether some argument or experiment is working with one or the other, they just say "standard deviation".

There are many problems with this, but these are the main ones I am thinking of right now:

  1. Although it seems to make sense to think that making that little -1 change adds certain "security" by making the result wider, and therefore betting for a wider deviation, the use of -1 instead of any other number $-x$, seems completely arbitrary.

  2. Even though it is said that the distortion of $n-1$ will be "diluted" as the size of the sample grows, there is an issue of lack of precision present even when the sample is enormous: if the sample grows to become $100\%$ of the population, then it will not be equal at all to the population standard deviation, because $x / n$ can never be equal to $x / (n-1)$.

  3. Most of the inferences made from the studies of a sample are generalizations that are then thought to be properties of the whole population. These inferences are made by looking and measuring the sample, the same way we would look at and measure the whole population if we had it. However in this case, this is not what is being done.

  4. This use is usually prescribed but an actual mathematical demonstration, showing its deductive logical validity, is usually absent, and I have not been able to find one yet.

Perhaps this modified standard deviation is valid, but with the given arguments and usual prescription of it, I am not seeing the validity, and I do feel it very counter intuitive.

So, considering statistical analysis is the art of combining intuition with mathematics, if I do not find it intuitive, and they do not demonstrate it, then it is difficult to accept it.

Prem
  • 9,669
algo
  • 21

0 Answers0