📂Mathematical Statistics

Convergence in Probability Implies Convergence in Distribution

Theorem¹

For a random variable $X$ and a sequence of random variables $\left\{ X_{n} \right\}$, $$ X_{n} \overset{P}{\to} X \implies X_{n} \overset{D}{\to} X $$

---

• $\overset{P}{\to}$ denotes convergence in probability.
• $\overset{D}{\to}$ denotes convergence in distribution.

Explanation

To restate it in intuitive terms, converging only in distribution is much easier than converging exactly. If you accurately understand a random variable itself as a function, this should not be hard to accept.

Proof

Strategy: A trick of splitting an event into two parts to set up an inequality is used here. If you study in this field, you will see it often, so even if it is difficult, we recommend trying to get used to it. It is naturally hard the first time you see it, so do not get discouraged even if you do not understand it, and try to grasp it by reading it several times.

---

For an arbitrary positive number $\epsilon > 0$, $$ \begin{align*} F_{X_{n}}(x) =& P[X_{n} \le x] \\ =& P[ \left\{ X_{n} \le x \right\} \cap \left\{ | X_{n} - X | < \epsilon \right\} ] +P[ \left\{ X_{n} \le x \right\} \cap \left\{ | X_{n} - X | \ge \epsilon \right\} ] \end{align*} $$ Here, looking closely at the first term $P[ \left\{ X_{n} \le x \right\} \cap \left\{ | X_{n} - X | < \epsilon \right\} ]$, $$ \begin{align*} & |X_{n}-X|<\epsilon \\ \implies& X<X_{n} + \epsilon \\ \implies& X<X_{n} + \epsilon \le x+ \epsilon \\ \implies& X< x+ \epsilon \end{align*} $$ Meanwhile, for the second term, $$ P[ \left\{ X_{n} \le x \right\} \cap \left\{ | X_{n} - X | \ge \epsilon \right\} ] \le P[\left\{ | X_{n} - X | \ge \epsilon \right\} ] $$ so rearranging gives $$ \begin{align*} F_{X_{n}}(x) \le & P[X \le x + \epsilon] + P[\left\{ | X_{n} - X | \ge \epsilon \right\} ] \\ =& F_{X}(x+\epsilon) + P[\left\{ | X_{n} - X | \ge \epsilon \right\} ] \end{align*} $$ Taking the limit $\displaystyle \lim_{n \to \infty}$ on both sides, since $\displaystyle \lim_{n \to \infty} P[\left\{ | X_{n} - X | \ge \epsilon \right\}=0$, $$ \limsup _{n \to \infty} F_{X_{n}}(x) \le F_{X}(x+\epsilon) $$ Having obtained the upper bound this way, obtaining the lower bound by the same method gives $$ F_{X}(x-\epsilon) \le \liminf _{n \to \infty} F_{X_{n}}(x) \le \limsup _{n \to \infty} F_{X_{n}}(x) \le F_{X}(x+\epsilon) $$ Since $\epsilon$ was set as an arbitrary positive number, as $\epsilon \to 0$, at every point $x \in C_{F_{X}}$ where $F_{X}$ is continuous, we have the pointwise convergence $$ \lim _{n \to \infty} F_{X_{n}}(x) = F_{X}(x) $$ and therefore $X_{n}$ converges in distribution to $X$.

■