Probability Distributions under Conditional Probability in Mathematical Statistics
Definition
- For a discrete random variable $X_{1}, X_{2}, \cdots , X_{n}$, the following $p_{2, \cdots , n \mid 1}$, given $X_{1} = x_{1}$, is called the joint conditional probability mass function of $ X_{2}, \cdots , X_{n}$: $$ p_{2, \cdots , n \mid 1} ( x_{2} , \cdots ,x_{n} \mid X_{1} = x_{1} ) = {{ p_{1, \cdots , n}(x_{1} , x_{2} , \cdots , x_{n}) } \over { p_{1}( X_{1} = x_{1} ) }} $$
- For a continuous random variable $X_{1}, X_{2}, \cdots , X_{n}$, the following $f_{2, \cdots , n \mid 1}$, given $X_{1} = x_{1}$, is called the joint conditional probability density function of $ X_{2}, \cdots , X_{n}$: $$ f_{2, \cdots , n \mid 1} ( x_{2} , \cdots ,x_{n} \mid X_{1} = x_{1} ) = {{ f_{1, \cdots , n}(x_{1} , x_{2} , \cdots , x_{n}) } \over { f_{1}( X_{1} = x_{1} ) }} $$
- When a function $u$ for $X_{2} , \cdots , X_{n}$ is given, the following, given $X_{1} = x_{1}$, is called the conditional expectation of $u( X_{2}, \cdots , X_{n} )$: $$ \begin{align*} & E \left[ u \left( X_{2} , \cdots , X_{n} \right) \mid X_{1} = x_{1} \right] \\ =& \int_{-\infty}^{\infty} \cdots \int_{-\infty}^{\infty} u (x_{2} , \cdots , x_{n}) f_{2 , \cdots , n \mid 1} (x_{2} , \cdots, x_{n} \mid X_{1} = x_{1}) dx_{2} \cdots , dx_{n} \end{align*} $$
Theorem
- [1] Conditional Variance: $$ \begin{align*} \text{Var} (X_{2} | X_{1} = x_{1}) =& E \left[ \left( X_{2} - E (X_{2} \mid X_{1} = x_{1}) \right)^{2} \mid X_{1} = x_{1} \right] \\ =& E \left( X_{2}^{2} \mid X_{1} = x_{1} \right) - \left[ E(X_{2} \mid X_{1} = x_{1}) \right]^{2} \end{align*} $$
- [2]: $E \left[ E (X_{2} | X_{1}) \right] = E (X_{2} )$
- [3]: If $\text{Var}(X_{2})$ exists, then $\text{Var} \left[ E \left( X_{2} \mid X_{1} \right) \right] \le \text{Var} (X_{2})$
Explanation
As was the case in the curriculum level, conditional probability and conditional expectation belong to the most challenging sections to calculate in mathematical statistics. Putting everything else aside, the computations inevitably become more numerous because it involves multivariate variables. Of course, the concept of conditionality is worth this complexity. On the other hand, unlike mathematical statistics, which mainly relies on calculus, the studies progress to probability theory based on measure theory, making the calculations considerably more straightforward. The main point is, ‘do not ignore it, but do not obsess over it either.’