2

I am studying the theory of probability with 'Jaynes style', i.e. not using all the theoretical substratum of measure theory and Lebesgue integral. I would like to understand if there exists in this 'limited' theoretical framework a rigorous proof (under the necessary hypotheses, even if a little more stringent) of the fact that:

$$\mathbb{E}\{g(X)\}=\int_{x=-\infty}^\infty g(x)f_X(x)\mathrm{d}x$$

where the integral is a Riemann integral, the state of knowledge about $X$ is described by the density function $f_X(x)$ and $g$ is a non-bijective function (continuous, differentiable, or what we need). From Wikipedia page https://en.wikipedia.org/wiki/Law_of_the_unconscious_statistician I understand that such a proof exists, and it apparently involves two things (which I don't even know, but which I will study):

  • Sard's theorem,
  • coarea formula.

I would like to ask, please, your help regarding where to study a serious proof of that formula because I am having difficulties finding it on the internet. If you perhaps have a link, a document, etc...

Thank you in advance.

Nameless
  • 81
  • The usual approach is to see that it holds when $g$ has finite range, then for arbitrary nonnegative $g$, approximate $g$ by a function with finite range. The usual approach is much more elementary than the Coarea formula. – Mason Dec 06 '23 at 16:44

1 Answers1

2

I think this might be a duplicate, but I can't find the right target.

You can find a proof in §4.3 of Grimmett and Stirzaker's Probability and Random Processes. First, you reduce to the case that $g$ is non-negative (by putting $g = g_+ - g_-$ where $g_\pm$ are non-negative). Then you prove that if $Y$ is non-negative with density function $f$ and CDF $F$, $E(Y) = \int_0^\infty 1-F(x) dx$ by doing $$\int_0^\infty 1-F(x) dx = \int_0^\infty P(Y>x) dx = \int_0^\infty \int_{x}^\infty f(y) dy dx $$ and changing the order of integration gives $\int_0^\infty \int_{0}^y f(y) dx dy = \int_0^\infty yf(y) dy = E(Y)$.

Then, for non-negative $g$, $E(g(X)) = \int_0^\infty P(g(X) > x)dx = \int_0^\infty \int_{g^{-1}((x, \infty))} f_X(y) \; dy dx$. Swap the order of the integrals to get $\int_0^\infty \int_0 ^{g(y)} f_X(y) dx dy = \int_0^\infty g(y)f_X(y) dy$, i.e. LOTUS.

The non-rigorous part here is the changing of the order of integration, for which you need a Fubini-type theorem. If you want to avoid the Lebesgue integral/measure theory you can find a reference in this question.

Matthew Towers
  • 10,854