Proof of the Mixing Theorem in Probability Theory
Theorem
Let the space $S$ be both a metric space $( S , \rho)$ and a measurable space $(S,\mathcal{B}(S))$.
The following are all equivalent:
- (1): $P_{n} \overset{W}{\to} P$
- (2): For every bounded, uniformly continuous function $f$, there exists $\displaystyle \int_{S} f dP_{n} \to \int_{S}f d P$
- (3): For every closed set $F$, there exists $\displaystyle \limsup_{n\to\infty} P_{n}(F) \le P(F)$
- (4): For every open set $G$, there exists $\displaystyle P(G) \le \liminf_{n\to\infty} P_{n}(G)$
- (5): For every $A$ such that $P(\partial A) = 0$, there exists $\displaystyle \lim_{n\to\infty} P_{n}(A) = P(A)$
Description
Portmanteau is an English word meaning ‘made up of a variety of elements’ or ‘hybrid’. Translating this directly as a hybrid theorem is not very smooth, so traditionally, it is read similar to [폴ㅌ맨퉈] and translated as a hybrid theorem because it is difficult to guess the meaning by that alone. The hybrid theorem is actually generalizable not just to probability measures but also to finite measures $\mu$, and it provides an equivalent condition for weak convergence of measures, making it a very important theorem.
Proof
Strategy: To prove this, we introduce the following notation. We recommend reading the detailed explanation.
- For elements $x \in S$ and subsets $A \subset S$, and $\delta >0$ $$ \rho (x, A) := \inf \left\{ \rho (x,a) : a \in A \right\} $$
$$ A^{\delta} := \left\{ x \in S : \rho (x, A) < \delta \right\} $$
- For some fixed $F \subset S$ $$ \begin{align*} f_{\delta}(x) :=& \left( 1 - \rho (x, F) / \delta \right)^{+} \\ =& \begin{cases} 1 &, x \in F \\ 1 - \rho (x,F)/\delta &, x \notin F \land x \in F^{\delta} \\ 0 &, x \notin F^{\delta} \end{cases} \end{align*} $$
Part 1. $(1) \implies (2)$
This is trivial by the definition of weak convergence.
Part 2. $(2) \implies (3)$
$$f_{\varepsilon}(x) : = \left( 1 - \rho ( x, F) / \varepsilon \right)^{+} $$ If $f_{\varepsilon}$ is defined as above, then $f_{\varepsilon}$ is bounded and uniformly continuous. Also, for all $\varepsilon > 0$, there exists $I_{F}(x) \le f_{\varepsilon}(x) \le I_{F}^{\varepsilon} (x)$, so $$ \int_{S} I_{F} dP_{n} \le \int_{S} f_{\varepsilon} dP_{n} \le \int I_{F^{\varepsilon}} dP_{n} $$ Where $\displaystyle P_{n}(F) = \int_{F} dP_{n} = \int_{S} 1_{F} dP_{n}$, thus $$ P_{n}(F) \le \int_{S} f_{\varepsilon} dP_{n} $$ Taking both sides by $\displaystyle \limsup_{n \to \infty}$, since $f_{\varepsilon}$ was bounded and uniformly continuous, according to $(2)$ $$ \begin{align*} \displaystyle \limsup_{n \to \infty} P_{n}(F) \le & \limsup_{n \to \infty} \int_{S} f_{\varepsilon} dP_{n} \\ =& \int_{S} f_{\varepsilon} dP \\ \le & P\left( F^{\varepsilon} \right) \end{align*} $$ Taking both sides by $\displaystyle \lim_{\varepsilon \to 0}$, by the continuity from above of measures $$ \begin{align*} \limsup_{n \to \infty} P_{n}(F) =& \lim_{\varepsilon \to 0} \limsup_{n \to \infty} P_{n}(F) \\ \le & \lim_{\varepsilon \to 0} P \left( F^{\varepsilon} \right) \\ =& P\left( \overline{F} \right) \end{align*} $$ If $F$ is a closed set, then because $\overline{F} = F$ $$ \limsup_{n\to\infty} P_{n}(F) \le P(F) $$
Part 3. $(3) \iff (4)$
Let $G:= F^{c}$, then $G$ is an open set $$ \begin{align*} & \displaystyle \limsup_{n\to\infty} P_{n}(F) \le P(F) \\ \iff & -P(F) \le -\limsup_{n\to\infty} P_{n}(F) \\ \iff & 1 -P(F) \le 1 -\limsup_{n\to\infty} P_{n}(F) \\ \iff & P(G) \le \liminf_{n\to\infty} \left[ 1 -P_{n}(F) \right] \\ \iff & P(G) \le \liminf_{n\to\infty} P_{n}(G) \end{align*} $$
Part 4. $(3),(4) \implies (5)$
Let’s briefly review interior, closure, and boundary.
$A^{\circ} \subset A \subset \overline{A}$Here, the interior $A^{\circ}$ is the largest open subset of $A$, and the closure $\overline{A}$ is the smallest closed superset of $A$. Also, the boundary $\partial A = \overline{A} \setminus A^{\circ}$ of $A$ is naturally disjoint from $A^{\circ}$.
According to $(3)$ $$ \limsup_{n \to \infty} P_{n}(A) \le \limsup_{n \to \infty} P_{n}\left( \overline{A} \right) \le P \left( \overline{A} \right) $$ According to $(4)$ $$ P \left( A^{\circ} \right) \le \liminf_{n \to \infty} P_{n} \left( A^{\circ} \right) \le \liminf_{n \to \infty} P_{n}(A) $$ Since $P \left( \partial A \right) = 0$, $P \left( A^{\circ} \right) = P(A) = P \left( \overline{A} \right)$ holds, $$ P(A) = P \left( A^{\circ} \right) \le \liminf_{n \to \infty} P_{n}(A) \le \limsup_{n \to \infty} P_{n}(A) \le P \left( \overline{A} \right) = P(A) $$ Thus, $$ \lim_{n \to \infty} P_{n}(A) = P(A) $$
Part 5. $(5) \implies (1)$
Let $g \in C_{b}(S)$, in other words, $g$ be a bounded and continuous function as defined in $S$. Let us define $\nu$ for $A \in \mathcal{B}(S)$ as follows. $$ \nu (A) := P \left( g^{-1} \left( A \right) \right) $$ Since $g$ is bounded, for all $x \in S$, we can choose $a$ and $b$ that satisfy $a \le g(x) \le b$. Here, $$ D := \left\{ \alpha : \nu ( \left\{ \alpha \right\} ) = 0 \right\} $$ If we consider $$ D^{c} = \left\{ \alpha : \nu ( \left\{ \alpha \right\} ) > 0 \right\} = \bigcup_{n=1}^{\infty} \left\{ \alpha : \nu ( \left\{ \alpha \right\} ) > {{1} \over {n}} \right\} $$ If a natural number $n \in \mathbb{N}$ is fixed, $ \left\{ \alpha : \nu ( \left\{ \alpha \right\} ) > {{1} \over {n}} \right\}$ must be a finite set because $\nu ( \mathbb{R}) < \infty$. If it is not a finite set, it means that there are infinitely many $\alpha$ that satisfy $\displaystyle \nu ( \left\{ \alpha \right\} ) > {{ 1 } \over { n }}$, contradicting $\nu ( \mathbb{R}) < \infty$. Therefore, $D^{c}$ is a countable union of finite sets, and thus, there can be at most countably many such $\alpha \in [a,b]$ that satisfy $\nu \left( \left\{ \alpha \right\} \right) > 0$.
Now, we can choose $t_{0} , \cdots , t_{m}$ that satisfies the following three conditions:
- (i): $a = t_{0} < t_{1} < \cdots < t_{m} = b$
- (ii): $\nu \left( \left\{ t_{i} \right\} \right) = 0$
- (iii): $t_{i} - t_{i-1} < \varepsilon$
Upon setting as $A_{i} = g^{-1} \left( [ t_{i-1} , t_{i} ) \right)$, $A_{i} \in \mathcal{B}(S)$ holds, and $\displaystyle \bigcup_{i=1}^{m} A_{i} = S$. Meanwhile, since the preimage of a continuous function preserves openness and closedness, $g^{-1} \left( ( t_{i-1}, t_{i}) \right)$ is an open set in $S$, and $g^{-1} \left( [ t_{i-1}, t_{i}] \right)$ is a closed set in $S$. Also, the interior $A_{i}^{\circ}$ of $A_{i}$ is its largest open subset, and the closure $\overline{A_{i}}$ is its smallest closed superset, so $$ g^{-1} \left( ( t_{i-1}, t_{i}) \right) \subset A_{i}^{\circ} \subset A_{i} \subset \overline{A_{i}} \subset g^{-1} \left( [ t_{i-1}, t_{i}] \right) $$ Whereas in condition (ii), since it was $\nu \left( \left\{ t_{i} \right\} \right) = 0$ hence, $$ P \left( A_{i}^{\circ} \right) = P \left( A_{i} \right) = P \left( \overline{A_{i}} \right) $$ This means $P \left( \partial A_{i} \right) = 0$, so according to the assumption $(5)$, $\displaystyle \lim_{n\to\infty} P_{n}(A_{i}) = P(A_{i})$ holds. $$ h(x) := \sum_{i=1}^{m} t_{i-1} 1_{A_{i}} (x) $$ Now, let’s define a new function $h$ as above. $h$ becomes a simple function with $m$ finite values, and from condition (iii), we can see that $h(x) \le g(x) \le h(x) + \varepsilon$. $$ \begin{align*} \left| P_{n}(g) - P(g) \right| =& \left| \int_{S} g dP_{n} - \int_{S} g dP \right| \\ =& \left| \int_{S} (g-h) dP_{n} + \int_{S} h dP_{n} - \int_{S} h dP + \int_{S} (h-g) dP \right| \\ \le & \left| \int_{S} (g-h) dP_{n} \right| + \left| \int_{S} h dP_{n} - \int_{S} h dP \right| + \left| \int_{S} (h-g) dP \right| \\ \le & \varepsilon + \left| \sum_{i=1}^{m} t_{i-1} \int_{S} 1_{A_{i}} P_{n} - \sum_{i=1}^{m} t_{i-1} \int_{S} 1_{A_{i}} P \right| + \varepsilon \\ \le & 2 \varepsilon + \left| \sum_{i=1}^{m} t_{i-1} \left[ P_{n}(A_{i}) - P(A_{i}) \right] \right| \end{align*} $$ Meanwhile, since $\displaystyle \lim_{n\to\infty} P_{n}(A_{i}) = P(A_{i})$, when $n \to \infty$, $P_{n}(g) \to P(g)$ holds. Since $g$ is bounded and continuous, by the definition of weak convergence, $P_{n} \overset{W}{\to} P$ holds.
■