Conditional Probability of Random Variables Defined by Measure Theory
Definition
Given a probability space $( \Omega , \mathcal{F} , P)$.
- When $\mathcal{G}$ is a sub sigma field of $\mathcal{F}$, the conditional probability of an event $F \in \mathcal{F}$ with respect to $\mathcal{G}$ is defined as $$ P(F | \mathcal{G}) := E ( \mathbb{1}_{F} | \mathcal{G}) $$.
- The conditional density of $Y$ when $f_{Y | X =x}$ is defined as follows and given $X=x$ is $$ f_{Y | X = x} (y | X = x) := {{\partial } \over {\partial y }} P( Y \le y | X = x) $$.
- You may ignore the term probability space if you haven’t encountered measure theory yet, but understanding this post without any knowledge of measure theory is nearly impossible.
- That $\mathcal{G}$ is a sub sigma field of $\mathcal{F}$ means that both are sigma fields of $\Omega$, with $\mathcal{G} \subset \mathcal{F}$.
Explanation
Conditional probability introduced with measure theory is defined by the conditional expectation.
Meanwhile, for the smallest sigma field $\sigma (X) = \left\{ X^{-1} (B) : B \in \mathcal{B}(\mathbb{R}) \right\}$ generated by the random variable $X$ with respect to $\Omega$, we use the following familiar notation. $$ E(Y|X) := E \left( Y | \sigma (X) \right) $$ And within the parentheses of probability or expectation, $Y \le y$ denotes the following event. $$ (Y \le y) := \left\{ \omega \in B : Y(B) \le y , B \in \mathcal{B}(\mathbb{R}) \right\} \in \mathcal{F} $$ Let’s derive the conditional probability $\displaystyle f_{Y | X = x} (y | X = x) = {{ f(x,y) } \over { f_{X} (x) }}$ using these notations.
Derivation
Given the condition of the expected value of conditional probability, $P(Y \le | X ) = E \left( \mathbb{1}_{(Y \le y)} | X \right) = E \left( \mathbb{1}_{(Y \le y)} | \sigma (X) \right)$ is obviously $\sigma (X)$-measurable. Naturally, it is assumed that $X$, $Y$ have a joint density $f(x,y) := f_{(X,Y)} (x,y)$.
For all Borel sets $B \in \mathcal{B}(\mathbb{R})$ and $F = X^{-1}(B)$, $$ \begin{align*} \int_{F} P(Y \le y | X ) dP =& \int_{F} E \left( \mathbb{1}_{(Y \le y)} | X \right) dP \\ =& \int_{F} \mathbb{1}_{(Y \le y)} dP \\ =& \int_{F} \mathbb{1}_{(Y \le y)} \mathbb{1}_{F} dP \\ =& E \left( \mathbb{1}_{(Y \le y)} \mathbb{1}_{F} \right) \\ =& \iint \mathbb{1}_{F} \mathbb{1}_{(Y \le y)} f(x,u) du dx \\ =& \int_{x \in F} \int_{-\infty}^{y} f(x,u) du dx \\ =& \int_{x \in F} \int_{-\infty}^{y} {{ f(x,u) } \over { f_{X} (x) }} f_{X} (x) du dx \\ =& \int_{x \in F} \int_{-\infty}^{y} {{ f(x,u) } \over { f_{X} (x) }} du f_{X} (x) dx \\ =& E \left( \int_{-\infty}^{y} {{ f(X,u) } \over { f_{X} (X) }} du \right) \\ =& \int_{F} \int_{-\infty}^{y} {{ f(X,u) } \over { f_{X} (X) }} du dP \end{align*} $$
Properties of Lebesgue Integration: $$ \int_{A} f dm = 0 \iff f = 0 \text{ a.e.} $$
To conclude, since $\displaystyle \int_{F} P(Y \le y | X ) dP = \int_{F} \int_{-\infty}^{y} {{ f(X,u) } \over { f_{X} (X) }} du dP$, almost surely $$ P(Y \le y | X ) = \int_{-\infty}^{y} {{ f(X,u) } \over { f_{X} (X) }} du $$ Finally, according to the Fundamental Theorem of Calculus, $$ \begin{align*} f_{Y|X=x} ( y | X=x ) =& {{ \partial } \over { \partial y }} P(Y \le y | X=x ) \\ =& {{ f(x,y) } \over { f_{X} (x) }} \text{ a.s.} \end{align*} $$
■