Sample Mean and Sample Variance are Independent, if $X_i$ is not an iid.

Question

We know that if $X_i$ is from iid normal distribution $N(\mu,\sigma^2)$, then the sample variance ($S^2$) is independent with the sample mean ($\bar{X}$). But now, if $X_i$ is not iid, rather, $X_i$ is from a normal distribution $N(\mu, \Sigma)$, where $$\Sigma = \sigma^2 \begin{bmatrix} 1 & \rho & \dots & \rho \\ \rho & 1 & \dots & \rho \\ \vdots \\ \rho & \rho & \dots & 1 \end{bmatrix}. $$ whether the sample mean and the sample variance are still independent? How can we show that?

Answer 1

Suppose $X=(X_1,X_2,\ldots,X_n)^T$ has a multivariate normal distribution $N_n(\mu\mathbf1_n,\Sigma)$ where $\Sigma=\sigma^2\left[(1-\rho)I_n+\rho\mathbf1_n\mathbf1_n^T\right]$. Here $\rho\in \left(-\frac1{n-1},1\right)$ and $\mathbf1_n$ is a column vector of all ones.

One way to prove independence of $\overline X=\frac1n\sum\limits_{i=1}^n X_i$ and $S^2=\frac1{n-1}\sum\limits_{i=1}^n (X_i-\overline X)^2$ is pretty much the same as was done here for $\rho=0$ (the particular case when $X_i$'s are i.i.d $N(\mu,\sigma^2)$).

Here is a brief proof sketch:

Transform $X\mapsto Y=AX$ where $A$ is an $n\times n$ orthogonal matrix given by

$$A= \begin{bmatrix}\frac1{\sqrt n} &\frac1{\sqrt n} & \frac1{\sqrt n}& \frac1{\sqrt n}& \cdots &\frac1{\sqrt n} \\ \frac1{\sqrt 2} & \frac{-1}{\sqrt 2} &0 & 0 & \cdots & 0 \\ \frac1{\sqrt 6} & \frac1{\sqrt 6} &\frac{-2}{\sqrt 6} &0 &\cdots & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots \\ \frac1{\sqrt {n(n-1)}} & \frac1{\sqrt {n(n-1)}} & \frac1{\sqrt {n(n-1)}} & \frac1{\sqrt {n(n-1)}} & \cdots & \frac{-(n-1)}{\sqrt {n(n-1)}} \end{bmatrix}$$

Then $X\sim N_n(\mu\mathbf1_n,\Sigma)$ would imply $Y\sim N_n(\mu A\mathbf1_n,A\Sigma A^T)$.

Verify that $A\Sigma A^T$ turns out to be a diagonal matrix:

$$A\Sigma A^T=\sigma^2 \operatorname{diag}\left(1+(n-1)\rho,1-\rho,\ldots,1-\rho\right)$$

So if $Y=(Y_1,Y_2,\ldots,Y_n)^T$, then $Y_i$'s are independent normal with $Y_1=\sqrt n\overline X$ and $\sum\limits_{i=2}^n Y_i^2=(n-1)S^2$. Hence conclude.

This would also give you the exact distribution of $\overline X$ and $S^2$ in this setup.