10

It is common to see conditional distributions specified in stats like:

$$(X \mid \mu = t) \sim \mathcal{N} (t, 1)$$

(Of course, we can also use some other distribution here)

How do you prove that such a conditional probability actually exists, in terms of a regular conditional probability? And is there some condition on the underlying probability space?

Proofs and/or references appreciated.

Community
  • 1
simonzack
  • 1,701
  • 13
  • 28
  • I think this is the result you need: Theorem B.18 in the Appendix of the following lecture notes http://www.math.wisc.edu/~seppalai/courses/735/notes.pdf – Calculon Aug 07 '15 at 11:30
  • @Calculon Isn't that proving the existence of a conditional probability given P? How do we construct P here? – simonzack Aug 09 '15 at 15:27
  • your interpretation of the result is correct but i don't think constructing $P$ is an issue here. if you take your space to be the real line, which would be sufficient for your example, then you satisfy all the conditions for the theorem to work. – Calculon Aug 09 '15 at 15:35
  • @Calculon Still don't get how these two relate, can you make it a bit more explicit? I think $X$, $\mu$, $P$ all need to be constructed here. – simonzack Aug 09 '15 at 16:01
  • I am not sure if I am about to say is correct. But if you pick an arbitrary (continuous) prior distribution for $\mu$ (one whose support contains $t$), you will have a joint distribution of $X$ and $\mu$. There will be some probability space $(\Omega,\mathcal{B},P)$ that accomodates $X$ and $\mu$ with the prescribed joint distribution. From that point on you just apply the theorem with $\sigma(\mu)$ as your sub sigma-algebra $\mathcal{A}$. – Calculon Aug 09 '15 at 17:01
  • For future readers, the complete proof can be found here. – Analyst Nov 02 '22 at 23:15

1 Answers1

5

On a polish space $(\Omega,\mathcal{F},P)$, a regular conditional distribution exists, that is, given any $\mathcal{G}$ sub-sigma algebra there is a function $K: \Omega\times \mathcal{F} \to [0,1]$ such that

1) $K(\omega, \cdot)$ is a probability measure

2) $K(\cdot, A)$ is $\mathcal{G}$ measurable

3) $K(\cdot, A) = \Bbb{E}[1_A \mid \mathcal{G}]$ $P$ almost surely

Moreover, if $\mathcal{H}\subset \mathcal{G}$ is countably generated ($\mathcal{H} = \sigma(H_n, n \in \Bbb{N})$) then the following substitution principle holds:

4) There is a null set $N\in \mathcal{G}$: $$K(\omega,A) = 1_A(\omega) \quad A \in \mathcal{H}, \omega\in \Omega \setminus N$$

An interesting case occurs when $\mu$ is a random variable taking values on $\Bbb{R}$ since $\sigma(\mu)$ is countably generated we have that

$$K(\omega,\{\omega'\mid \{\mu(\omega') = \mu(\omega)\}) = 1, \quad \omega\in \Omega \setminus N$$

So almost surely $K(\omega, \cdot)$ is concentrated on the level set $\{\mu(\omega') = \mu(\omega)\}$

In the case you are dealing with, you know further (by assumption) that if $\mu(\omega )= t$ $$K(\omega, A) = \int_A \frac{1}{\sqrt{2\pi}}\exp\big(\frac{(x-t)^2}{2}\big)\, dx$$ that is, $K(\omega, \cdot) \sim N(\mu(\omega, 1))$ almost surely.

References for this can be found in

Parthasarathy, K. R., Probability Measures on Metric Spaces 1967.

Karatzas and Schreve - Brownian motion and stochastic calculus [chap 5, pag 306] 1991

and in the notes pointed out by Calculon:Theorem B.18 in the Appendix of the following lecture notes math.wisc.edu/~seppalai/courses/735/notes.pdf

Conrado Costa
  • 6,071