How does one define the Fourier transform of a probability distribution?

Question

Say $p_X$ and $p_Y$ are two probability distributions on a $m$ element set.

Then I see an equality written as,

$$\sqrt{m} \vert \vert p_X - p_Y \vert \vert _2 = \sqrt{ \sum_{k=0}^{m-1} \vert \hat{p_X} (k) - \hat{p_Y} (k)\vert ^2 } $$

Can someone help derive and understand this? I don't know what is the defintion and meaning of these $\hat{p}$ quantities on the right.

What is the meaning of this identity?

It is likely that $\hat{p}_X$ is the probability mass function, rather than a result of Fourier transform. Then $\hat{p}_X(k)$ is just the probability that $X = k$. — Budenn, Jul 07 '15 at 19:46
This seems to be a definition of a distance between probability distributions (close to, but different from Hellinger distance), not an equality. — Budenn, Jul 08 '15 at 08:25
This identity is simply Parseval's theorem in the case of the discrete Fourier transform. For a proof of this, see here: http://math.stackexchange.com/questions/346894/prove-of-the-parsevals-theorem-for-discrete-fourier-transform-dft — Chester, Jul 08 '15 at 13:55
@Did Then the answer by Chester below is probably the correct one. Should I delete the comments? — Budenn, Jul 08 '15 at 20:14

Chester · Answer 1 · 2015-07-08T19:55:03.243

To answer your title question, you are dealing with a function whose domain is: $\{0, 1, \ldots, m-1\}$. The probability distribution can be represented by a vector, $p_X \in \mathbb{R}^m$ with the usual 2 norm.

Since this is discrete, finite function, its Fourier transform is then found using the discrete Fourier transform. This is defined as

$$ \hat{p}_X(k) = \sum_{j=0}^{m-1} p_X(j)\exp(-2\pi ikj/m) , \;\; k=0, 1, \ldots, m-1 $$

As I mentioned in the comments, the identity you ask about is called Parseval's theorem. This energy preservation identity (up to a scale factor) is true for the discrete Fourier transform and the continuous transform - or any unitary transformation. To be clear, the identity you've written states:

$$\sqrt{m}||p_X-p_Y||_2 = ||\hat{p}_X - \hat{p}_Y||_2$$

How does one define the Fourier transform of a probability distribution?

1 Answers1