Why do we divide by $\sqrt{N}$ to find the standard deviation?

Question

The standard deviation of a population $[a_1, a_2, \dots a_N]$ is defined as follows: $$ \sigma = \sqrt{\frac{\Sigma_{i=1}^{N} (a_i - \mu)^2}{N}} = \frac{\sqrt{\Sigma_{i=1}^{N} (a_i - \mu)^2}}{\sqrt{N}} $$

The numerator is the (Euclidean) length of the following vector: $$ [a_1 - \mu, a_2 - \mu, \dots, a_n-\mu] $$ We may imagine that this vector lives in some $N$-dimensional configuration space which describes the dispersion of the population.

As we go into higher dimensions, this vector will naturally get longer, because we are adding more coordinates without changing the values of the existing coordinates. So clearly we have to divide by some scaling factor to normalize this between different population sizes.

Why is that scaling factor the square root of $N$ instead of some other expression? Does it have anything to do with the $\sqrt{m}$ that appears in this problem?

Answer 1

In statistics, variance is defined as the average of the deviation from the average of a given set of data points. Hence, variance can be found using the following equation using $\sigma^2$ as the mathematical representation of variance: $$\sigma^2=\frac{\sum_{i=1}^{N}{(a_i-\mu)^2}}{N}$$

Standard Deviation, by its very definition, is just the square root of $\sigma^2$, or simply $\sqrt{\sigma^2}=\sigma$. Hence, in mathematical terms, the value of Standard Deviation is provided by a $\sigma$. When using the above equation and deriving the equation for Standard Deviation, one gets: $$\sigma=\sqrt{\frac{\sum_{i=1}^{N}{(a_i-\mu)^2}}{N}}=\frac{\sqrt{\sum_{i=1}^{N}{(a_i-\mu)^2}}}{\sqrt{N}}$$

This is why the scaling factor of $\sqrt{N}$ can be seen in the equation for Standard Deviation.

Answer 2

For the same reason why we divide by $N$ to find the mean.

Standard deviation is, on average how much the population deviates from the average.

Mathematically, $\sigma^2$ is the average of $(a_i -\mu)^2$.