28

This is solely a reference request. I have heard a few versions of the following theorem:

If the joint moment generating function $\mathbb{E}[e^{uX+vY}] = \mathbb{E}[e^{uX}]\mathbb{E}[e^{vY}]$ whenever the expectations are finite, then $X,Y$ are independent.

And there is a similar version for characteristic functions. Could anyone provide me a serious reference which proves one or both of these theorems?

saz
  • 120,083
nullUser
  • 27,877
  • 1
    I'm not at all sure that this is true. (Certainly it's true when "if" and "then" are interchanged.) – Michael Hardy Jan 26 '13 at 06:18
  • 1
    If this were true, every random variables with subexponential tails on both sides would be independent. Please reach a plausible statement. – Did Jan 26 '13 at 10:57
  • 1
    In this post a theorem with a more general result was presented: http://math.stackexchange.com/questions/1802289/does-eeitax-by-eeitaxeeitby-for-every-a-b-in-mathbbr-im – Carlos H. Mendoza-Cardenas May 29 '16 at 13:55

2 Answers2

37

Theorem (Kac's theorem) Let $X,Y$ be $\mathbb{R}^d$-valued random variables. Then the following statements are equivalent.

  1. $X,Y$ are independent
  2. $\forall \eta,\xi \in \mathbb{R}^d: \mathbb{E}e^{\imath \, (X,Y) \cdot (\xi,\eta)} = \mathbb{E}e^{\imath \, X \cdot \xi} \cdot \mathbb{E}e^{\imath \, Y \cdot \eta}$

Proof:

  • $(1) \Rightarrow (2)$: Straightforward, use $\mathbb{E}(f(X) \cdot g(Y)) = \mathbb{E}(f(X)) \cdot \mathbb{E}(g(Y))$
  • $(2) \Rightarrow (1)$: Let $(\tilde{X},\tilde{Y})$ be such that $\tilde{X}$, $\tilde{Y}$ are independent, $\tilde{X} \sim X$, $\tilde{Y} \sim Y$. Then $$\mathbb{E}e^{\imath \, (X,Y) \cdot (\xi,\eta)} \stackrel{(2)}{=} \mathbb{E}e^{\imath \, X \cdot \xi} \cdot \mathbb{E}e^{\imath \, Y \cdot \eta} = \mathbb{E}e^{\imath \tilde{X} \cdot \xi} \cdot \mathbb{E}e^{\imath \tilde{Y} \cdot \eta} = \mathbb{E}e^{\imath (\tilde{X},\tilde{Y}) \cdot (\xi,\eta)}$$ i.e. the characteristic functions of $(X,Y)$ and $(\tilde{X},\tilde{Y})$ coincide. From the uniqueness of the Fourier transform we conclude $(X,Y) \sim (\tilde{X},\tilde{Y})$. Consequently, $X$ and $Y$ are independent.

Remark: It is not important that $X$ and $Y$ are vectors of the same dimension. The same reasoning works if, say, $X$ is an $\mathbb{R}^k$-valued random variable and $Y$ and $\mathbb{R}^d$-valued random variable.

Reference (not for the given proof, but the result):David Applebaum, B.V. Rajarama Bhat, Johan Kustermans, J. Martin Lindsay, Michael Schuermann, Uwe Franz: Quantum Independent Increment Processes I: From Classical Probability to Quantum Stochastic Calculus (Theorem 2.1).

Community
  • 1
saz
  • 120,083
  • Do we need $X,Y \in L^1$? I don't see it being used in the proof. – nomadicmathematician Jun 13 '16 at 21:04
  • @takecare You are right; it's not needed for the proof. – saz Jun 14 '16 at 05:38
  • How do you know you can find such a pair of independent variables $(\tilde{X},\tilde{Y})$ – Ansel B Feb 03 '17 at 14:03
  • @AnselB Possibly $(\tilde{X},\tilde{Y})$ are defined on a different probability space; this is not a problem since everything is about distributions of random vectors (nothing pointwise here). – saz Feb 03 '17 at 14:41
  • Thank you for replying, but I am still a bit confused as to what justifies the existence of $\tilde{X}$ and $\tilde{Y}$. It looks to me that the same reasoning can be used to show that $\mathbb{E}e^{i(X\xi+Y\eta)}=\mathbb{E}e^{iX\xi}.\mathbb{E}e^{iY\eta}$ implies independence. Please tell me where I am going wrong and what justifies the existence. Thanks again. – Ansel B Feb 03 '17 at 20:49
  • 1
    @AnselB If $\mathbb{E}e^{i (X \xi + Y \eta)} = \mathbb{E}e^{i \xi X} \mathbb{E}e^{iY \eta}$ for all $\xi$, $\eta$, then $X$ and $Y$ are independent; that's exactly what the proof shows. If you prefer, think about like this: Denote by $\mathbb{P}_X$ and $\mathbb{P}_Y$ the distribution of $X$ and $Y$, respectively. Then the characteristic function of the product measure $\mu = \mathbb{P}_X \times \mathbb{P}_Y$ is given by $$\hat{\mu}(\xi,\eta) = \mathbb{E}e^{i \xi X} \mathbb{E}e^{i \eta Y}$$ which is, by assumption, also the characteristic function of $(X,Y)$. – saz Feb 04 '17 at 06:29
  • Since the Fourier transform (hence the characteristic function) determines a measure uniquely, this implies that $\mathbb{P}_{(X,Y)} = \mu = \mathbb{P}_X \times \mathbb{P}_Y$ which means that $X$ and $Y$ are independent. – saz Feb 04 '17 at 06:29
  • @saz Can we extend this theorem to any finite number of random variables? So that if 2. holds for $X_1, ..., X_n$ then all $X_i$s are independent. – nomadicmathematician Jun 03 '17 at 22:58
  • 1
    @takecare Yes... if $$\mathbb{E}\exp \left( i \sum_{j=1}^n \xi_j X_j \right) = \prod_{j=1}^n \mathbb{E}\exp(i \xi_j X_j)$$ for any $\xi_1,\ldots,\xi_n$, then the random variables are independent. – saz Jun 04 '17 at 04:45
  • @saz: You wrote : "By uniqueness of Fourier transform". Where did you use a Fourier transform ? (If I'm not mistaken, you are using Moment generating function, and not characteristic function, no ?) – John Nov 20 '19 at 08:18
  • 1
    @John I'm using characteristic functions (which is essentially nothing but the Fourier transform ... or the inverse Fourier transform, up to a constant, depending which definition you are using). Note that $\mathbb{R}^d \ni \xi \mapsto E(e^{i \xi X})$ is the characteristic function, not the moment generating function. – saz Nov 20 '19 at 14:02
  • Thank you, I was confused because on you post, your $i$ looks like $t$. Does the proof of $X,Y$ independents $\iff$ $\mathbb E[e^{\xi_1X+\xi_2Y}]=\mathbb E[e^{\xi_1 X}]\mathbb E[e^{\xi_2Y}]$ for all $\xi_1,\xi_2\in\mathbb R$ is rigorously the same ? – John Nov 20 '19 at 15:29
  • @John Well, not exactly, for instance you need that $X$ and $Y$ have exponential moments of all order. This is not necessary if you work with characteristic functions. – saz Nov 20 '19 at 16:48
1

Builidng on the answer by saz. If X and Y have a joint density, here is another proof for (2)⇒(1): By the inverse Fouriour transform: $$f_\mathbf{X}(\mathbf{x})=\frac{1}{(2\pi)^n}\int_{R^n}{e^{-j\mathbf{v'x}}\phi_\mathbf{x}(\mathbf{v})d\mathbf{v}}$$

where x and v are vertical vectors, and in this case, vector $\mathbf{x} = [x\ y]'$, vector $\mathbf{v} = [v_1\ v_2]'$

Therefore, $$f_{XY}(x,y)=\frac{1}{(2\pi)^2}\iint{e^{-j(v_1x+v_2y)}\phi_{XY}(v_1,v_2)}dv_1dv_2\\=\frac{1}{2\pi}\int{e^{-j(v_1x)}\phi_{X}(v_1)}dv_1\frac{1}{2\pi}\int{e^{-j(v_2y)}\phi_{Y}(v_2)}dv_2\\=f_X(x)f_Y(y)$$

And the joint probability density function (pdf) equals to the product marginal pdf's is the definition of independence for continuous random variables. This method should work for discrete random variables as well.

Shuang Liu
  • 11