📂Probability Distribution

Proof That the Square of a Standard Normal Distribution Follows a Chi-Squared Distribution with 1 Degree of Freedom

Theorem

If $X \sim N(\mu,\sigma ^2)$, then $$ V=\left( { X - \mu \over \sigma} \right) ^2 \sim \chi ^2 (1) $$

---

• $N \left( \mu , \sigma^{2} \right)$ is the normal distribution with mean $\mu$ and variance $\sigma^{2}$.
• $\chi^{2} \left( 1 \right)$ is the chi-squared distribution with $1$ degree of freedom.

Explanation

As a theorem, its generalization, Student’s theorem, is more widely used.

Anyone studying statistics should always take it for granted, as a fact, that the square of a standard normal distribution follows a chi-squared distribution. When some data can be assumed to follow a normal distribution, if the variance of the standardized data is excessively large or small, one can immediately suspect that something is wrong. Naturally, it is applied in many statistical tests, and having or lacking theoretical intuition about it makes a world of difference.

On the other hand, thinking about it the other way around, rather than first recalling the definition of the chi-squared distribution and then exploring its properties, it is more sensible that one would discover the chi-squared distribution while studying what distribution the square of data following a standard normal distribution—presumably the square of residuals—follows.

Proof¹

Let $\displaystyle W := {(X-\mu) \over \sigma }$, then $W \sim N(0,1)$.

Definition of the standard normal distribution: The normal distribution $N \left( 0,1^{2} \right)$ with the following probability density function is called the standard normal distribution. $$ f(z) = {{ 1 } \over { \sqrt{2 \pi} }} \exp \left[ - {{ z^{2} } \over { 2 }} \right] $$

Let $F$ be the cumulative distribution function of $V$, then $$ \begin{align*} F(v) =& P(V \le v) \\ =& P \left( W^2 \le v \right) \\ =& P \left( \sqrt{v} \le W \le \sqrt{v} \right) \\ =& \int_{-\sqrt{v}}^{\sqrt{v}} { 1 \over \sqrt{ 2 \pi } } e^{-{{w^2} \over 2}} dw \\ =& 2 \int_{0}^{\sqrt{v}} { 1 \over \sqrt{ 2 \pi } } e^{-{{w^2} \over 2}} dw \end{align*} $$ Substituting as $w := \sqrt{x}$ gives $$ F(v) = 2\int_{0}^{v} { 1 \over \sqrt{ 2 \pi } } e^{-{{x} \over 2}} {1 \over {2 \sqrt{x} } } dx $$ By the fundamental theorem of calculus, the probability density function $f$ of $v$ is $$ f(v) = F ' (v) = { 1 \over {\sqrt{ 2 \pi } } } e^{-{{v} \over 2}}{ 1 \over {v^{1 \over 2}} } $$

Euler’s reflection formula: $$ {\Gamma (1-x) \Gamma ( x )} = { {\pi} \over {\sin \pi x } } $$

By the reflection formula, $\displaystyle \sqrt{\pi} = \Gamma \left( {{ 1 } \over { 2 }} \right)$, so $$ f(v) = { 1 \over { \Gamma ({1 \over 2}) 2^{1 \over 2} } } v^{ - {1 \over 2} } e^{-{{v} \over 2}} $$

Definition of the gamma distribution: For $k, \theta > 0$, the continuous probability distribution $\Gamma ( k , \theta )$ with the following probability density function is called the gamma distribution. $$ f(x) = {{ 1 } \over { \Gamma ( k ) \theta^{k} }} x^{k - 1} e^{ - x / \theta} \qquad , x > 0 $$

In summary, $V$ has the probability density function of the gamma distribution $\displaystyle \Gamma \left( {{ 1 } \over { 2 }} , 2 \right)$.

Relationship between the gamma distribution and the chi-squared distribution: $$ \Gamma \left( { r \over 2 } , 2 \right) \iff \chi ^2 (r) $$

Therefore, $\displaystyle \Gamma \left( {1 \over 2}, 2 \right) \sim \chi^2 (1)$ and $$ \left( { X - \mu \over \sigma} \right) ^2 \sim \chi ^2 (1) $$

■