logo

Probability Distributions under Conditional Probability in Mathematical Statistics 📂Mathematical Statistics

Probability Distributions under Conditional Probability in Mathematical Statistics

Definition

  1. For a discrete random variable X1,X2,⋯ ,XnX_{1}, X_{2}, \cdots , X_{n}, the following p2,⋯ ,n∣1p_{2, \cdots , n \mid 1}, given X1=x1X_{1} = x_{1}, is called the joint conditional probability mass function of X2,⋯ ,Xn X_{2}, \cdots , X_{n}: p2,⋯ ,n∣1(x2,⋯ ,xn∣X1=x1)=p1,⋯ ,n(x1,x2,⋯ ,xn)p1(X1=x1) p_{2, \cdots , n \mid 1} ( x_{2} , \cdots ,x_{n} \mid X_{1} = x_{1} ) = {{ p_{1, \cdots , n}(x_{1} , x_{2} , \cdots , x_{n}) } \over { p_{1}( X_{1} = x_{1} ) }}
  2. For a continuous random variable X1,X2,⋯ ,XnX_{1}, X_{2}, \cdots , X_{n}, the following f2,⋯ ,n∣1f_{2, \cdots , n \mid 1}, given X1=x1X_{1} = x_{1}, is called the joint conditional probability density function of X2,⋯ ,Xn X_{2}, \cdots , X_{n}: f2,⋯ ,n∣1(x2,⋯ ,xn∣X1=x1)=f1,⋯ ,n(x1,x2,⋯ ,xn)f1(X1=x1) f_{2, \cdots , n \mid 1} ( x_{2} , \cdots ,x_{n} \mid X_{1} = x_{1} ) = {{ f_{1, \cdots , n}(x_{1} , x_{2} , \cdots , x_{n}) } \over { f_{1}( X_{1} = x_{1} ) }}
  3. When a function uu for X2,⋯ ,XnX_{2} , \cdots , X_{n} is given, the following, given X1=x1X_{1} = x_{1}, is called the conditional expectation of u(X2,⋯ ,Xn)u( X_{2}, \cdots , X_{n} ): E[u(X2,⋯ ,Xn)∣X1=x1]=∫−∞∞⋯∫−∞∞u(x2,⋯ ,xn)f2,⋯ ,n∣1(x2,⋯ ,xn∣X1=x1)dx2⋯ ,dxn \begin{align*} & E \left[ u \left( X_{2} , \cdots , X_{n} \right) \mid X_{1} = x_{1} \right] \\ =& \int_{-\infty}^{\infty} \cdots \int_{-\infty}^{\infty} u (x_{2} , \cdots , x_{n}) f_{2 , \cdots , n \mid 1} (x_{2} , \cdots, x_{n} \mid X_{1} = x_{1}) dx_{2} \cdots , dx_{n} \end{align*}

Theorem

  • [1] Conditional Variance: Var⁡(X2∣X1=x1)=E[(X2−E(X2∣X1=x1))2∣X1=x1]=E(X22∣X1=x1)−[E(X2∣X1=x1)]2 \begin{align*} \operatorname{Var} (X_{2} | X_{1} = x_{1}) =& E \left[ \left( X_{2} - E (X_{2} \mid X_{1} = x_{1}) \right)^{2} \mid X_{1} = x_{1} \right] \\ =& E \left( X_{2}^{2} \mid X_{1} = x_{1} \right) - \left[ E(X_{2} \mid X_{1} = x_{1}) \right]^{2} \end{align*}
  • [2]: E[E(X2∣X1)]=E(X2)E \left[ E (X_{2} | X_{1}) \right] = E (X_{2} )
  • [3]: If Var⁡(X2)\operatorname{Var}(X_{2}) exists, then Var⁡[E(X2∣X1)]≤Var⁡(X2)\operatorname{Var} \left[ E \left( X_{2} \mid X_{1} \right) \right] \le \operatorname{Var} (X_{2})

Explanation

As was the case in the curriculum level, conditional probability and conditional expectation belong to the most challenging sections to calculate in mathematical statistics. Putting everything else aside, the computations inevitably become more numerous because it involves multivariate variables. Of course, the concept of conditionality is worth this complexity. On the other hand, unlike mathematical statistics, which mainly relies on calculus, the studies progress to probability theory based on measure theory, making the calculations considerably more straightforward. The main point is, ‘do not ignore it, but do not obsess over it either.’

See Also