logo

Conditional Probability of Random Variables Defined by Measure Theory 📂Probability Theory

Conditional Probability of Random Variables Defined by Measure Theory

Definition

Given a probability space (Ω,F,P)( \Omega , \mathcal{F} , P).

  1. When G\mathcal{G} is a sub sigma field of F\mathcal{F}, the conditional probability of an event F∈FF \in \mathcal{F} with respect to G\mathcal{G} is defined as P(F∣G):=E(1F∣G) P(F | \mathcal{G}) := E ( \mathbb{1}_{F} | \mathcal{G}) .
  2. The conditional density of YY when fY∣X=xf_{Y | X =x} is defined as follows and given X=xX=x is fY∣X=x(y∣X=x):=∂∂yP(Y≤y∣X=x) f_{Y | X = x} (y | X = x) := {{\partial } \over {\partial y }} P( Y \le y | X = x) .
  • You may ignore the term probability space if you haven’t encountered measure theory yet, but understanding this post without any knowledge of measure theory is nearly impossible.
  • That G\mathcal{G} is a sub sigma field of F\mathcal{F} means that both are sigma fields of Ω\Omega, with G⊂F\mathcal{G} \subset \mathcal{F}.

Explanation

Conditional probability introduced with measure theory is defined by the conditional expectation.

Meanwhile, for the smallest sigma field σ(X)={X−1(B):B∈B(R)}\sigma (X) = \left\{ X^{-1} (B) : B \in \mathcal{B}(\mathbb{R}) \right\} generated by the random variable XX with respect to Ω\Omega, we use the following familiar notation. E(Y∣X):=E(Y∣σ(X)) E(Y|X) := E \left( Y | \sigma (X) \right) And within the parentheses of probability or expectation, Y≤yY \le y denotes the following event. (Y≤y):={ω∈B:Y(B)≤y,B∈B(R)}∈F (Y \le y) := \left\{ \omega \in B : Y(B) \le y , B \in \mathcal{B}(\mathbb{R}) \right\} \in \mathcal{F} Let’s derive the conditional probability fY∣X=x(y∣X=x)=f(x,y)fX(x)\displaystyle f_{Y | X = x} (y | X = x) = {{ f(x,y) } \over { f_{X} (x) }} using these notations.

Derivation

Given the condition of the expected value of conditional probability, P(Y≤∣X)=E(1(Y≤y)∣X)=E(1(Y≤y)∣σ(X))P(Y \le | X ) = E \left( \mathbb{1}_{(Y \le y)} | X \right) = E \left( \mathbb{1}_{(Y \le y)} | \sigma (X) \right) is obviously σ(X)\sigma (X)-measurable. Naturally, it is assumed that XX, YY have a joint density f(x,y):=f(X,Y)(x,y)f(x,y) := f_{(X,Y)} (x,y).

For all Borel sets B∈B(R)B \in \mathcal{B}(\mathbb{R}) and F=X−1(B)F = X^{-1}(B), ∫FP(Y≤y∣X)dP=∫FE(1(Y≤y)∣X)dP=∫F1(Y≤y)dP=∫F1(Y≤y)1FdP=E(1(Y≤y)1F)=∬1F1(Y≤y)f(x,u)dudx=∫x∈F∫−∞yf(x,u)dudx=∫x∈F∫−∞yf(x,u)fX(x)fX(x)dudx=∫x∈F∫−∞yf(x,u)fX(x)dufX(x)dx=E(∫−∞yf(X,u)fX(X)du)=∫F∫−∞yf(X,u)fX(X)dudP \begin{align*} \int_{F} P(Y \le y | X ) dP =& \int_{F} E \left( \mathbb{1}_{(Y \le y)} | X \right) dP \\ =& \int_{F} \mathbb{1}_{(Y \le y)} dP \\ =& \int_{F} \mathbb{1}_{(Y \le y)} \mathbb{1}_{F} dP \\ =& E \left( \mathbb{1}_{(Y \le y)} \mathbb{1}_{F} \right) \\ =& \iint \mathbb{1}_{F} \mathbb{1}_{(Y \le y)} f(x,u) du dx \\ =& \int_{x \in F} \int_{-\infty}^{y} f(x,u) du dx \\ =& \int_{x \in F} \int_{-\infty}^{y} {{ f(x,u) } \over { f_{X} (x) }} f_{X} (x) du dx \\ =& \int_{x \in F} \int_{-\infty}^{y} {{ f(x,u) } \over { f_{X} (x) }} du f_{X} (x) dx \\ =& E \left( \int_{-\infty}^{y} {{ f(X,u) } \over { f_{X} (X) }} du \right) \\ =& \int_{F} \int_{-\infty}^{y} {{ f(X,u) } \over { f_{X} (X) }} du dP \end{align*}

Properties of Lebesgue Integration: ∫Afdm=0  ⟺  f=0 a.e. \int_{A} f dm = 0 \iff f = 0 \text{ a.e.}

To conclude, since ∫FP(Y≤y∣X)dP=∫F∫−∞yf(X,u)fX(X)dudP\displaystyle \int_{F} P(Y \le y | X ) dP = \int_{F} \int_{-\infty}^{y} {{ f(X,u) } \over { f_{X} (X) }} du dP, almost surely P(Y≤y∣X)=∫−∞yf(X,u)fX(X)du P(Y \le y | X ) = \int_{-\infty}^{y} {{ f(X,u) } \over { f_{X} (X) }} du Finally, according to the Fundamental Theorem of Calculus, fY∣X=x(y∣X=x)=∂∂yP(Y≤y∣X=x)=f(x,y)fX(x) a.s. \begin{align*} f_{Y|X=x} ( y | X=x ) =& {{ \partial } \over { \partial y }} P(Y \le y | X=x ) \\ =& {{ f(x,y) } \over { f_{X} (x) }} \text{ a.s.} \end{align*}

■

See also