1

I'm trying to work through the proof given here that the sample mean and sample variance of a random sample $X_1, X_2, ..., X_n \sim N(\mu,\sigma^2)$ are independent.

The part I can't seem to follow is the assertion that the Jacobian of the transformations...

$$ Y_1 = \bar{X} \\ Y_2 = X_2 - \bar{X} \\ \vdots \\ Y_n = X_n - \bar{X} \\ $$

is equal to $n$. I must be missing something simple, but when I create the matrix of partial derivatives, the determinant seems to always be one?

$$ \det\begin{bmatrix} \frac{dY_1}{d\bar{X}} & \frac{dY_1}{dX_2} & \dots & \frac{dY_1}{dX_n} \\ \frac{dY_2}{d\bar{X}} & \frac{dY_2}{dX_2} & \dots & \frac{dY_2}{dX_n} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{dY_n}{d\bar{X}} & \frac{dY_n}{dX_2} & \dots & \frac{dY_n}{dX_n} \\ \end{bmatrix} = \det\begin{bmatrix} 1 & 0 & \dots & 0 \\ -1 & 1 & \dots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ -1 & 0 & \dots & 1 \\ \end{bmatrix} = 1 $$

How am I setting up the Jacobian incorrectly? Thanks in advance!

Em.
  • 15,981
Ben Southgate
  • 173

2 Answers2

2

Comments: This result is true only for normal data.

The idea is to do an orthogonal transformation in which one of the new random variables is a function of $\bar X$ and the other $n-1$ are a function of $S^2$. For a multivariate normal, orthogonality is equivalent to independence.

You might start with $n = 2$ where the proof is especially simple. Let $Y_1 = X_1 + X_2$ and $Y_2 = X_1 - X_2.$ (A 45 degree rotation.) If $X_1$ and $X_2$ are independent, then so are $Y_1$ and $Y_2.$ It is obvious that $\bar X$ is a function of $Y_1,$ and not difficult to show that $S^2$ is a function of $Y_2$.

Illustration of independence of $\bar X$ and $S$ for normal, but not for nonnormal data: In each of the panels below 30,000 samples of size 5 are simulated, standard normal on the left and $Beta(.5, .5)$ on the right. In both cases, sample correlations are consistent with $\rho = 0$ (by symmetry). For normal data, $\bar X$ and $S$ are independent, and for beta data, they are obviously not. (The figure at right illustrates that a 5-D hypercube has been 'squashed' onto two dimensions. $Beta(.5,.5)$ puts most of its probability near corners, edges and (hyper)faces, images of some of these, after 'squashing', are discernible.)

enter image description here

In case you want it, the R code is shown below.

 par(mfrow=c(1,2))
  m = 30000;  n = 5
   DTA = matrix(rnorm(m*n), nrow=m)  # each row a sample of size n
   a = rowMeans(DTA);  s = apply(DTA, 1, sd)  # sample mean and sd
   cor(a,s)
   ## -0.0066071
   plot(a, s, pch=".", main="Normal: Mean vs SD")
  m = 30000;  n = 5
   DTA = matrix(rbeta(m*n, .5, .5), nrow=m)  # non-normal data
   a = rowMeans(DTA);  s = apply(DTA, 1, sd)  # sample mean and sd
   cor(a,s)
   ## -0.005415171
   plot(a, s, pch=".", main="BETA: Mean vs SD")
 par(mfrow=c(1,1))
BruceET
  • 51,500
0

This problem also shows up in the corersponding independence proof in Statistical inference by Casella and Berger (that's how I got here). Some extra confusion is added by the fact that the book at one point claims that the determinant is $1/n$, but the authors use the determinant $n$ at another point in the proof...

Since the transformation is from $X_1, ..., X_n$, the Jacobian should have elements $\frac{\partial x_i}{\partial y_i}$, and not the other way around (which is the way it is written in the question).

First we need to express $X_1, ..., X_n$ in terms of $Y_1, ..., Y_n$

$Y_1 = \bar{X} = \frac{1}{n}(X_1+...+X_2) \\ Y_2 = X_2 -\bar{X} = X_2 - Y_1\\ ...\\ Y_n = X_n - Y_1$

Clearly, $X_i = Y_1 + Y_n$ for $i \in (2, n)$, which gives us

$X_n = Y_1 + Y_n \\ X_{n-1} = Y_1 + Y_{n-1}\\ ...\\ X_1 =Y_1-\sum_{i=2}^n Y_i$

and thus the Jacobian will be an identity matrix with a leading column of ones and leading row of negative ones added.

For clarity, this is how it would look for $n=5$

$ J= \begin{bmatrix} 1 & -1 & -1 & -1 & -1 \\ 1 & 1 & 0 & 0 & 0 \\ 1 & 0 & 1 & 0 & 0 \\ 1 & 0 & 0 & 1 & 0 \\ 1 & 0 & 0 & 0 & 1 \\ \end{bmatrix}. $

To find the determinant of this Jacobian, we use two properties of the determinant. First, we can add any column or row to any other column or row without changing the value of the determinant. If we add each colum to the first, we end up with a matrix which consists of an identity matrix with the top row replaced by a row on the form $(n, -1, ..., -1)$, or in the $n=5$ case

$ J= \begin{bmatrix} 5 & -1 & -1 & -1 & -1 \\ 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ \end{bmatrix}. $

Now, if we use Lapalace's formula to expand along the first column we get

$\lvert J \rvert = n * \lvert I_{n-1} \rvert - 0 * \lvert . \rvert + ... - 0 * \lvert . \rvert$

where I have only written out the first minor (which is an identity matrix), since all the other ones will be multiplied by zero and disappear anyway.

Second, we use the result that the determinant of an identity matrix is always one, to obtain

$\lvert J \rvert = n.$

Oscar Oelrich
  • 1