📂Mathematical Statistics

Convergence in Probability Implies Convergence in Distribution

Theorem¹

Given a random variable $X$ and its sequence $\left\{ X_{n} \right\}$, $$ X_{n} \overset{P}{\to} X \implies X_{n} \overset{D}{\to} X $$

---

• $\overset{P}{\to}$ denotes convergence in probability.
• $\overset{D}{\to}$ denotes convergence in distribution.

Explanation

In simpler terms, it’s much easier to have convergence in distribution than exact convergence. Understanding a random variable as a function in its own right should make this concept not too difficult to grasp.

Proof

Strategy: A trick of diving events into two parts and setting up inequalities is used, and it’s recommended to get used to this as you’ll encounter it frequently in your studies. It’s okay if it’s hard at first, understanding doesn’t always come immediately. Don’t get discouraged – try reading through multiple times to grasp the concept.

---

For any arbitrary positive number $\epsilon > 0$, $$ \begin{align*} F_{X_{n}}(x) =& P[X_{n} \le x] \\ =& P[ \left\{ X_{n} \le x \right\} \cap \left\{ | X_{n} - X | < \epsilon \right\} ] +P[ \left\{ X_{n} \le x \right\} \cap \left\{ | X_{n} - X | \ge \epsilon \right\} ] \end{align*} $$ Looking closely at the first term $P[ \left\{ X_{n} \le x \right\} \cap \left\{ | X_{n} - X | < \epsilon \right\} ]$, $$ \begin{align*} & |X_{n}-X|<\epsilon \\ \implies& X<X_{n} + \epsilon \\ \implies& X<X_{n} + \epsilon \le x+ \epsilon \\ \implies& X< x+ \epsilon \end{align*} $$ While for the second term, $$ P[ \left\{ X_{n} \le x \right\} \cap \left\{ | X_{n} - X | \ge \epsilon \right\} ] \le P[\left\{ | X_{n} - X | \ge \epsilon \right\} ] $$ Organizing yields, $$ \begin{align*} F_{X_{n}}(x) \le & P[X \le x + \epsilon] + P[\left\{ | X_{n} - X | \ge \epsilon \right\} ] \\ =& F_{X}(x+\epsilon) + P[\left\{ | X_{n} - X | \ge \epsilon \right\} ] \end{align*} $$ Taking the limit $\displaystyle \lim_{n \to \infty}$ on both sides gives $\displaystyle \lim_{n \to \infty} P[\left\{ | X_{n} - X | \ge \epsilon \right\}=0$, hence $$ \limsup _{n \to \infty} F_{X_{n}}(x) \le F_{X}(x+\epsilon) $$ Having determined an upper bound, finding a lower bound by the same method shows $$ F_{X}(x-\epsilon) \le \liminf _{n \to \infty} F_{X_{n}}(x) \le \limsup _{n \to \infty} F_{X_{n}}(x) \le F_{X}(x+\epsilon) $$ Since $\epsilon$ is any arbitrary positive number, for $\epsilon \to 0$, at every continuous point $x \in C_{F_{X}}$, $$ \lim _{n \to \infty} F_{X_{n}}(x) = F_{X}(x) $$ thus, $X_{n}$ converges in distribution to $X$.

■