If $X_n \to X $ in probability then $f(X_n) \to f(X)$ in probability for a Borel function $f$

Question

$(X_n)_n$ is a sequence of identically distributed random variables, $f:\mathbb{R} \to \mathbb{R}$ a Borel function.

Prove that if $X_n$ converges in probability to $X,$ then $f(X_n)$ converges in probability to $f(X).$

Answer 1

Following up on a line of reasoning suggested by MikeG and Danny Pak-Keung Chan:

Let $\mu$ be the law of the $X_n$ (which is also the law of $X$). Fix $\epsilon, \eta > 0$. By Lusin's theorem applied to the measure $\mu$, there is a compact set $K \subset \mathbb{R}$ with $\mu(K) > 1-\eta$ on which which $f|_K$ is continuous. It is thus even uniformly continuous, so there exists $\delta > 0$ such that for every $x,y \in K$ with $|x-y| < \delta$, we have $|f(x)-f(y)| < \epsilon$.

As such, for any given $n$, if we are to have $|f(X_n) - f(X)| \ge \epsilon$, then we must either have $X_n \notin K$, $X \notin K$, or $|X_n - X| \ge \delta$. Thus by union bound $$\begin{align*}P(|f(X_n) - f(X)| \ge \epsilon) &\le P(X_n \notin K) + P(X \notin K) + P(|X_n - X| \ge \delta)\\ &\le 2 \eta + P(|X_n - X| \ge \delta).\end{align*}$$ Since $X_n \to X$ in probability, letting $n \to \infty$ we conclude $$\limsup_{n \to \infty} P(|f(X_n) - f(X)| \ge \epsilon) \le 2\eta$$ and $\eta > 0$ was arbitrary.

---

A previous of the version asked whether, if $X_n \to X$ almost surely, we can conclude $f(X_n) \to f(X)$ almost surely. The answer is no. For simplicity, work on the group $(S^1,+)$. You can take it back to $\mathbb{R}$ if you like by identifying $S^1$ with $[0,1)$ and $+$ with addition mod 1.

Let $E \subset S^1$ be a closed nowhere dense set of positive Lebesgue measure, e.g. a generalized Cantor set, and $f = 1_E$. I claim there exists a sequence $t_n \in S^1$ with $t_n \to e$, the identity element, for which $\liminf_{n \to \infty} 1_E(x+t_n) = 0$ for every $x$. Once this is shown, we can let $X$ be uniformly distributed on $S^1$ and let $X_n = X + t_n$. Clearly every $X_n$ is uniformly distributed on $S^1$ and $X_n \to X$ a.s., but $\liminf f(X_n) = 0$ a.s., so $f(X_n) \not\to f(X)$ on the event $\{X \in E\}$, which has positive probability.

To prove the claim, note that $U=E^c$ is open and dense. So for any integer $m>0$, the sets $U-s$ where $d(s,e) < 1/m$ form an open cover of $S^1$. By compactness, there is a finite subcover $U-s_{m,1}, \dots, U-s_{m, k_m}$. This means that for every $x$ and every $m$, we have $x + s_{m,i} \in U$ for some $i$. Let $t_n$ be the sequence $$(s_{1,1}, \dots, s_{1, k_1}, s_{2,1},\dots, s_{2, k_2}, \dots).$$
Since $d(s_{m,i}-e) < 1/m$, we have $t_n \to e$. And by construction, for every $x \in S^1$, there are infinitely many $t_n$ such that $x + t_n \in U$. This means that $\liminf_{n \to \infty} 1_E(x+t_n) = 0$.

Answer 2

This is an alternative proof for the problem, where we will suppose that $X,X_n$ take values in a separable metric space $(U,d),f:U \to \mathbb{R}$ measurable function.

Consider $\mathcal{K}=\{E \in \mathcal{B}(U),1_E(X_n) \to^P1_E(X)\}.$ We will prove that $\mathcal{K}$ is a $\sigma$-algebra containing $\mathcal{B}(U).$

Clearly $U \in \mathcal{K}$ and $\mathcal{K}$ is a $\pi$-system. Let $E,F \in \ \mathcal{K}$ such that $E \subset F.$ Write $1_{F-E}=1_F-1_E$ to conclude that $F-E \in \mathcal{K}.$ Taking an increasing sequence $(E_n)_n$ in $\mathcal{K}$ and letting $E=\bigcup_nE_n.$

So, $ \lim_k1_{E_k}=1_E.$ Notice that for all $n,k \in \mathbb{N}$ $$P(|1_E(X_n)-1_E(X)|>\epsilon) \leq P(|1_E(X_n)-1_{E_k}(X_n)|>\epsilon/3)+P(|1_{E_k}(X_n)-1_{E_k}(X)|>\epsilon/3)+P(|1_{E_k}(X)-1_E(X)|> \epsilon /3)$$ In other word, using the fact that $X_n$ are identically distributed, $$P(|1_E(X_n)-1_E(X)|>\epsilon) \leq P(|1_{E_k}(X_n)-1_{E_k}(X)|>\epsilon/3)+2P(|1_{E_k}(X)-1_E(X)|> \epsilon /3)$$ so $\limsup_n P(|1_E(X_n)-1_E(X)|>\epsilon)=0$ and $E\in \mathcal{K}.$

So $\mathcal{K}$ is a $\sigma$-algebra.

If $E$ is a closed of $U,$ then by considering $f_k(x)=\frac{1}{(1+d(x,E))^{k}}$ (which is continuous on $U$ and $\lim_k f_k=1_E$) then $$P(|1_E(X_n)-1_E(X)|>\epsilon) \leq P(|1_E(X_n)-f_k(X_n)|>\epsilon/3)+P(|f_k(X_n)-f_k(X)|>\epsilon/3)+P(|f_k(X)-1_E(X)|> \epsilon /3)$$

so, $\limsup_nP(|1_E(X_n)-1_E(X)|>\epsilon)=0$ this means that $E \in \mathcal{K}$ and $\mathcal{B}(U)=\mathcal{K}.$

Now if $f:U \to \mathbb{R}$ is a measurable function, then it can be approximated by a sequence of simple function $\phi_k$ and from the above for all $k \in \mathbb{N},\phi_k(X_n) \to^P \phi_k(X)$ and again since $$P(|f(X_n)-f(X)|>\epsilon) \leq P(|f(X_n)-\phi_k(X_n)|>\epsilon/3)+P(|\phi_k(X_n)-\phi_k(X)|>\epsilon/3)+P(|\phi_k(X)-f(X)|>\epsilon/3),$$ so $\limsup_nP(|f(X_n)-f(X)|>\epsilon)=0.$

In this proof we didn't need to use the continuity on a compact (uniform continuity).

Separability is supposed just to ensure that $(X,X_n)$ is indeed a random variable $(\mathcal{B}(U^2)=\mathcal{B}(U) \otimes \mathcal{B}(U))$

If $f(X_1) \in L^p$ and $X_n \to ^{L^p} X$ then $f(X_n) \to ^{L^p} f(X).$

The statement is not true for almost sure convergence.