2

I understand intuitively why spread would be additive, but not precisely why variances add rather than, say, the standard deviations.

And how does this relate to the pythagorean theorem/euclidean distance? It appears the SDs can be treated as vectors perpendicular to (i.e. independent from) each other and the length of the vector when you sum them is equal to the SD when you sum the distributions. (This seems similar to how the SD itself can be seen as the euclidean distance of summed perpendicular deviations, divided by $\sqrt{n}$).

(To be clear, I am not asking why variances can be added but not SDs, I am asking why variances are added rather than SDs (or anything else), so this is not a duplicate question)

Thanks so much!

Edit: I've gotten a great simple algebraic answer to my question, which is probably as far as you can go, but if anyone has some insight into the intuition behind it, that would be greatly appreciated too. It's probably like the pythagorean theorem and doesn't have a satisfying intuition, but I'd be more than happy with an explanation for why the standard deviations act like the sides of a right-angled triangle!

Khai
  • 21
  • Can you clarify what did you mean by "summing distributions"? Did you want to say that you have random variables $\xi_1,...,\xi_n$ and you ask about $\mathbb{V}ar[\xi_1+...+\xi_n]$? – perepelart Mar 23 '23 at 10:41
  • Your question is why the variance of the sum of two variables of zero covariance is the sum of their variances. This is a special case of the bilinearity of covariances, which has an interesting interpretation. – J.G. Mar 23 '23 at 11:04
  • Actually, variance does not add in the general case. It adds in the special case where you are adding independent variables. – David K Mar 23 '23 at 13:44
  • 1
    @perepelart Thanks for pointing out the inaccuracy, corrected! – Khai Mar 24 '23 at 16:48

1 Answers1

1

I am asking why variances are added rather than SDs (or anything else)

The simply truth of this question is that variances (unlike standard deviations) are the only measure of "dispersion" that adds when adding random variables. Suppose we are given independent random variables $X$ and $Y$. Then by the definition of variance $$ \begin{aligned} \mathsf{Var}(X+Y) &=\mathsf E(X-\mu_X+Y-\mu_Y)^2\\ &=\mathsf E(X-\mu_X)^2+2\mathsf E(X-\mu_X)(Y-\mu_Y)+\mathsf E(Y-\mu_Y)^2. \end{aligned} $$ By the definition of variance (and covariance) we then get $$ \mathsf{Var}(X+Y)=\mathsf{Var}(X)+\underbrace{2\mathsf{Cov}(X,Y)}_{=0}+\mathsf{Var}(Y)=\mathsf{Var}(X)+\mathsf{Var}(Y). $$ Taking the square root of both sides gives $$ \mathsf{Std}(X+Y)=\sqrt{\mathsf{Var}(X)+\mathsf{Var}(Y)}\neq \mathsf{Std}(X)+\mathsf{Std}(Y). $$

So the fact that we add variances (and not standard deviations or any other measure of dispersion) is simply a consequence of the definitions of such objects.

Aaron Hendrickson
  • 6,042
  • Normally I'm not a fan of algebraic understanding of concepts, but this is simple enough to provide some useful intuition, thanks a lot! – Khai Mar 24 '23 at 18:13