📂Mathematical Statistics

The Equivalence Between Two Normally Distributed Random Variables Being Independent and Having a Covariance of Zero

Theorem

$$X_{1} \sim N ( \mu_{1} , \sigma_{1} ) \\ X_{2} \sim N ( \mu_{2} , \sigma_{2} )$$ Surface $$X_{1} \perp X_{2} \iff \text{cov} (X_{1} , X_{2} ) = 0$$

Description

Generally, being uncorrelated does not imply independence. However, if there is an assumption that the distributions follow a normal distribution, then having a covariance of $0$ guarantees independence.

Proof

$( \implies )$

$$M_{X_{1}} (t_{1} ) = \exp \left[ \mu_{1} t_{1} + {{1} \over {2}} \sigma_{1} t_{1}^{2} \right] M_{X_{2}} (t_{2} ) = \exp \left[ \mu_{2} t_{2} + {{1} \over {2}} \sigma_{2} t_{2}^{2} \right]$$ If $\sigma_{12} : = \text{cov} (X_{1} , X_{2} )$ and $\sigma_{21} : = \text{cov} (X_{2} , X_{1} )$ then $$M_{X_{1} , X_{2}} (t_{1} , t_{2} ) = \exp \left[ \mu_{1} t_{1} + \mu_{2} t_{2} + {{1} \over {2}} \left( \sigma_{1} t_{1}^{2} + \sigma_{12} t_{1} t_{2} + \sigma_{21} t_{2} t_{1} + \sigma_{2} t_{2}^{2} \right) \right]$$ Since $X_{1} \perp X_{2}$, it follows that $M_{X_{1} , X_{2}} (t_{1} , t_{2} ) = M_{X_{1}} (t_{1} ) M_{X_{2}} ( t_{2} )$. Therefore, $$\begin{align*} & \exp \left[ \mu_{1} t_{1} + \mu_{2} t_{2} + {{1} \over {2}} \left( \sigma_{1} t_{1}^{2} + \sigma_{12} t_{1} t_{2} + \sigma_{21} t_{2} t_{1} + \sigma_{2} t_{2}^{2} \right) \right] \\ =& \exp \left[ \mu_{1} t_{1} + \mu_{2} t_{2} + {{1} \over {2}} \left( \sigma_{1} t_{1}^{2} + \sigma_{2} t_{2}^{2} \right) \right] \end{align*}$$ summarizing, we get $\sigma_{12} + \sigma_{21} = 0$. Meanwhile, since $\text{cov} (X_{1}, X_{2}) = \text{cov} (X_{2}, X_{1})$, $\sigma_{12} = \sigma_{21}$, and the system of equations $\begin{cases} \sigma_{12} + \sigma_{21} = 0 \\ \sigma_{12} - \sigma_{21} = 0 \end{cases}$ has only the solution $\sigma_{12} = \sigma_{21} = 0$.

---

$( \impliedby )$

Since $\sigma_{12} = \sigma_{21} = 0$, $$\sigma_{12} t_{1} t_{2} = \sigma_{21} t_{2} t_{1} = 0$$ Therefore, $$M_{X_{1} , X_{2}} (t_{1} , t_{2} ) = M_{X_{1}} (t_{1} ) M_{X_{2}} ( t_{2} )$$ and, the fact that the joint moment generating function can be separated implies independence, thus $X_{1} \perp X_{2}$.

■