6

Assume $n$ variables $\{X_i\}_{i=1}^n$ are independently drawn from the same Gaussian distribution. Then, we define the empirical mean by $\bar{X}=\frac{1}{n}\sum_{i=1}^nX_i$ and variance by $S^2=\frac{1}{n-1}(X_i-\bar{X})^2$.

My question is that if $\bar{X}$ and $S^2$ are independent of each other? under which condition?

olivia
  • 1,514
  • You should really stick with standard notation and write $\overline{X}$ and $S^2$. – Ian Apr 03 '17 at 02:23
  • For normal data only, the sample mean $\bar X$ and the sample variance $S^2$ are (stochastically) independent random variables. This is not trivial to show; I have seen proofs using MGFs and others using matrix algebra. See. [I say 'stochastically' independent because they are clearly not 'functionally' independent (as $\bar X$ appears in the definition of $S^2$).] For data from other distributions, $\bar X$ and $S^2$ are dependent. – BruceET Apr 03 '17 at 04:05
  • 1
    Thank you very much. They are independent, so for the expectation $E[e^{(\bar{X}+S^2)}]$, it can be calculated by $E[e^{\bar{X}}]E[e^{S^2}]$? – olivia Apr 03 '17 at 04:30
  • 1
    @olivia Yes, exactly. – NCh Apr 03 '17 at 04:33
  • 1
    it is really amazing! @NCh – olivia Apr 03 '17 at 04:34
  • 1
    @BruceET : To show that $\operatorname{cov} \left(\overline{X}, X_i-\overline{X} \right) = 0$ is trivial. Perhaps showing that in a jointly Gaussian context that implies independence is more work, but one should at least say what my first sentence above says. See my answer below. – Michael Hardy Apr 15 '17 at 18:27

2 Answers2

7

The link in my Comment proves that $\bar X$ and $S^2$ are independent for normal data. This is a unique property of the normal family of distributions.

For intuition (only), here are plots of $S$ against $\bar X$ for many samples of size $n = 5$ from the (a) standard normal, (b) standard exponential, and (c) $\mathsf{Beta}(.1, .1)$ distributions, respectively. Each point in each graph represents one sample of size five. Examples (b) and (c) were chosen because they display obvious patterns of dependence. [In (c), $\rho(\bar X, S)= 0,$ but nonlinear association is clear.]

enter image description here

BruceET
  • 51,500
3

Under the assumptions you've stated they are independent. Maybe the quickest way to see that is to show that $\overline X$ and the vector $(X_1-\overline{X}, \ldots, X_n-\overline{X})$ are independent.

First find $\operatorname{cov}\left( \overline{X}, X_i - \overline{X} \right).$ You will see that it is $0$. Jointly Gaussian random variables whose covariance is $0$ are independent.

Michael Hardy
  • 1