📂Probability Distribution

Derivation of F-distribution from Two Independent Chi-squared Distributions

Theorem

If two random variables $U,V$ are independent and it is assumed that $U \sim \chi^{2} ( r_{1})$, $V \sim \chi^{2} ( r_{2})$ then $${{ U / r_{1} } \over { V / r_{2} }} \sim F \left( r_{1} , r_{2} \right)$$

Explanation

If two data follow the Chi-squared distribution and are independent, it might be possible to explain their ratio using distribution theory. In statistics in general, it is assumed that the squared standardized residuals follow the Chi-squared distribution, which is why the F-test is commonly used. While the proof itself is not crucial, understanding why the F-test is used in many analyses provides vital insight, making it an essential fact for students of mathematical statistics.

Derivation¹

Strategy: Direct deduction using the joint density function of Chi-squared distributions.

Definition of Chi-squared distribution: A continuous probability distribution for a given degree of freedom $r > 0$ that has the following probability density function $\chi^{2} (r)$ is called the Chi-squared distribution. $$f(x) = {{ 1 } \over { \Gamma (r/2) 2^{r/2} }} x^{r/2-1} e^{-x/2} \qquad , x \in (0, \infty)$$

Definition of F-distribution: A continuous probability distribution for a given degree of freedom $r_{1}, r_{2} > 0$ that has the following probability density function $F \left( r_{1} , r_{2} \right)$ is called the F-distribution. $$f(x) = {{ 1 } \over { B \left( r_{1}/2 , r_{2} / 2 \right) }} \left( {{ r_{1} } \over { r_{2} }} \right)^{r_{1} / 2} x^{r_{1} / 2 - 1} \left( 1 + {{ r_{1} } \over { r_{2} }} x \right)^{-(r_{1} + r_{2}) / 2} \qquad , x \in (0, \infty)$$

---

Since $U,V$ are independent, the joint density function with respect to $u,v \in (0,\infty)$ is as follows. $$h(u,v) = {{ 1 } \over { \Gamma \left( {{ r_{1} } \over { 2 }} \right) \Gamma \left( {{ r_{2} } \over { 2 }} \right) 2^{(r_{1} + r_{2})/2} }} u^{r_{1}/2 - 1} v^{r_{2}/2 - 1} e^{-(u+v)/2}$$ Now, if we set $\displaystyle W:= {{ U/r_{1} } \over { V / r_{2} }}$ and $Z := V$, then $u = (r_{1}/r_{2})zw$ and $v = z$, thus $$\left| J \right| = \begin{vmatrix} (r_{1}/r_{2})z & (r_{1}/r_{2})w \\ 0 & 1 \end{vmatrix} = (r_{1}/r_{2})z \ne 0$$ Therefore, the joint density function of $W,Z$ with respect to $w,z \in (0,\infty)$ is $$g(w,z) = {{ 1 } \over { \Gamma \left( {{ r_{1} } \over { 2 }} \right) \Gamma \left( {{ r_{2} } \over { 2 }} \right) 2^{(r_{1} + r_{2})/2} }} \left( {{ r_{1} z w } \over { r_{2} }} \right)^{{{ r_{1} - 2 } \over { 2 }}} z^{{{ r_{2} - 2 } \over { 2 }}} \exp \left[ - {{ z } \over { 2 }} \left( {{ r_{1} w } \over { r_{2} }} + 1 \right) \right] {{ r_{1} z } \over { r_{2} }}$$ The marginal density function $g_{1}$ of $W$ is set to $\displaystyle y:= {{ z } \over { 2 }} \left( {{ r_{1} w } \over { r_{2} }} + 1 \right)$, thus $$\begin{align*} g_{1} (w) =& \int_{-\infty}^{\infty} g(w,z) dz \\ =& \int_{-\infty}^{\infty} {{ 1 } \over { \Gamma \left( {{ r_{1} } \over { 2 }} \right) \Gamma \left( {{ r_{2} } \over { 2 }} \right) 2^{(r_{1} + r_{2})/2} }} \left( {{ r_{1} z w } \over { r_{2} }} \right)^{{{ r_{1} - 2 } \over { 2 }}} z^{{{ r_{2} - 2 } \over { 2 }}} \exp \left[ - {{ z } \over { 2 }} \left( {{ r_{1} w } \over { r_{2} }} + 1 \right) \right] {{ r_{1} z } \over { r_{2} }} dz \\ =& \int_{-\infty}^{\infty} {{ (r_{1} / r_{2})^{r_{1} / 2} w^{r_{1}/2 - 1} } \over { \Gamma \left( {{ r_{1} } \over { 2 }} \right) \Gamma \left( {{ r_{2} } \over { 2 }} \right) 2^{(r_{1} + r_{2})/2} }} \left( {{ 2y } \over { {{ r_{1} } \over { r_{2} }} w + 1 }} \right)^{{{ r_{1} + r_{2} } \over { 2 }} - 1} e^{-y} \left( {{ 2 } \over { {{ r_{1} } \over { r_{2} }} w + 1 }} \right) dy \\ =& {{ \Gamma \left( {{ r_{1} + r_{2} } \over { 2 }} \right) \left( {{ r_{1} } \over { r_{2} }} \right)^{r_{1} / 2} } \over { \Gamma \left( {{ r_{1} } \over { 2 }} \right) \Gamma \left( {{ r_{2} } \over { 2 }} \right) }} {{ w^{r_{1}/2 - 1} } \over { \left( 1 + {{ r_{1} } \over { r_{2} }} w \right)^{(r_{1} + r_{2}) / 2} }} \\ =& {{ 1 } \over { B \left( r_{1}/2 , r_{2} / 2 \right) }} \left( {{ r_{1} } \over { r_{2} }} \right)^{r_{1} / 2} w^{r_{1} / 2 - 1} \left( 1 + {{ r_{1} } \over { r_{2} }} w \right)^{-(r_{1} + r_{2}) / 2} \end{align*}$$ Therefore $$W \sim F (r_{1} , r_{2})$$

■