📂Statistical Test

Pearson Chi-Square Test Statistic

Definition ¹

Let’s assume that in a multinomial experiment where $k$ categories are drawn with a probability of $p_{j} > 0$ each, categorical data obtained from $n$ independent trials is given. The frequency $O_{j}$ of data belonging to the $j$th category is referred to as the Observed Cell Count, while the expected value under the null hypothesis of the hypothesis test $E_{j}$ is called the Expected Cell Count. The test statistic $$ \mathcal{X}^{2} := \sum_{j=1}^{k} {{ \left( O_{j} - E_{j} \right)^{2} } \over { E_{j} }} $$ is known as the Pearson Chi-square Test Statistic.

Explanation

Hypothesis Testing

$\mathcal{X}^{2}$ is a typical test statistic that first-year students encounter, often causing them shock and awe as they were only familiar with the normal distribution or the binomial distribution. To youngsters with little experience, understanding the chi-square distribution without an intuitive grasp of data and statistical analysis is next to impossible, hence I will only explain it as far as one can understand just by looking at the formula.

1. In most cases, a large $\mathcal{X}^{2}$ indicates a discrepancy between the actual data and theoretical expectations. If we examine the numerator of the equation, we find that $\left( O_{j} - E_{j} \right)^{2} \ge 0$ becomes smallest precisely when it matches $O_{j} = E_{j}$, i.e., when the observed data perfectly aligns with the theoretically known probabilities $p_{j}$. The more these figures deviate, the indefinite growth of the numerator occurs.
2. Thus, $\mathcal{X}^{2}$ grows larger as the data increasingly deviates from the null hypothesis $H_{0}$, and typically, a Right-tailed Statistical Test is conducted when $\mathcal{X}^{2}$ is greater than $\chi^{2}_{1-\alpha}$, resulting in the rejection of the null hypothesis.
3. In short, a large $\mathcal{X}^{2}$ means “something is significantly wrong”. The chi-square distribution is used when one wants to quantify how much deviation or scatter exists.

The purposes of the Pearson Chi-square Test Statistic for categorical data notably include:

• Testing the goodness of fit of a group
• Testing the independence of a group
• Testing the homogeneity of a group

Theoretical Rationale

You reading this are likely beyond a freshman level.

It is known through Student’s theorem that the square of the normal distribution commonly assumed for the distribution of residuals is proportionally following the chi-square distribution. However, even for undergraduates with some mathematical statistics background, the form of $\mathcal{X}^{2}$ could still seem peculiar at a glance. It might seem empirical more than anything, especially because there’s no assumption about deviations following a normal distribution. Of course, statistics isn’t as haphazard, and the proper proof is provided by Pearson’s theorem, which justifies the chi-square nature of $\mathcal{X}^{2}$.

Pearson’s theorem: Assuming a random vector $\left( N_{1} , \cdots , N_{k} \right)$ for $n \in \mathbb{N}$ sample size and $k \in \mathbb{N}$ categories follows a multinomial distribution $M_{k} \left( n ; \mathbf{p} \right)$. Then, when $n \to \infty$, the following statistic $S$ converges in distribution to a chi-square distribution $\chi^{2} \left( k - 1 \right)$. $$ S = \sum_{j=1}^{k} {{ \left( N_{j} - n p_{j} \right)^{2} } \over { n p_{j} }} \overset{D}{\to} \chi^{2} \left( k-1 \right) $$

The multinomial experiment introduced in the definition precisely presupposes that our data follows a multinomial distribution, and according to Pearson’s theorem, given that the sample size is sufficiently large, it approximates a chi-square distribution with degrees of freedom $(k-1)$ subtracted by the number of categories $k$ from $1$. The proof of Pearson’s theorem is far from simple, yet for undergraduates, applying $\mathcal{X}^{2}$ in practice shouldn’t pose a significant problem, even without a deep theoretical understanding. Naturally, if you’re considering graduate school, it’s recommended to dedicate a day to studying until you can prove it by yourself.