Test of Homogeneity
Hypothesis Testing 1
Suppose that in a multinomial experiment, categorical data with $C$ categories were obtained from $R$ populations. Let $p_{ij}$ be the probability that an element from the $i = 1 , \cdots , R$th population belongs to the $j = 1 , \cdots , C$th category, and let the proportion vector of the $i$th population be denoted by $\mathbf{p}_{i} = \left( p_{i1} , \cdots , p_{iC} \right)$. The following hypothesis test using the Pearson chi-squared test statistic is called the test of independence.
- $H_{0}$: The proportions of elements in all populations are identical: $$\mathbf{p}_{1} = \cdots = \mathbf{p}_{R}$$
- $H_{1}$: There exists at least one population whose proportion of elements is not identical.
Test Statistic
The test statistic uses the Pearson chi-squared test statistic. $$ \mathcal{X}^{2} = \sum_{i=1}^{R} \sum_{j=1}^{C} {{ \left( O_{ij} - E_{ij} \right)^{2} } \over { E_{ij} }} $$ Here, the frequency $O_{ij}$ of the data belonging to both the $i$th population and the $j$th category is called the observed frequency, and the ratio $E := r_{i} c_{j} / n$ of the product $r_{i} c_{j}$ of the number of samples $r_{i}$ belonging to the $i$th population and the number of samples $c_{j}$ belonging to the $j$th category to the sample size $n$ is called the expected frequency. This test statistic follows a chi-squared distribution, whose degrees of freedom is $(R-1)(C-1)$, the product of the number of populations $R$ and the number of categories $C$ each reduced by $1$.
Explanation
Degrees of Freedom
$$ \begin{align*} p_{i} = \sum_{j1=1}^{C} p_{ij} = {\frac{ r_{i} }{ n }} \implies & \sum_{i=1}^{R} p_{i} = 1 \\ p_{j} = \sum_{i=1}^{R} p_{ij} = {\frac{ c_{j} }{ n }} \implies & \sum_{j=1}^{C} p_{j} = 1 \end{align*} $$ Considering each marginal probability, the above constraints follow, so if we know $R-1$ probabilities from each population and $C - 1$ probabilities from the categories, the remaining one is effectively determined. Therefore, the degrees of freedom must be their product $(R - 1) (C - 1)$.
| X\Y | y1 | y2 |
|---|---|---|
| x1 | 0000 | 0000 |
| x2 | 0000 | 0000 |
| x3 | 0000 | 0000 |
For example, the contingency table above represents data with $R = 3$ populations and $C = 2$ categories, and its degrees of freedom become $(3-1)(2-1) = 2$.
Homogeneity 2
In fact, the process of the test of homogeneity is exactly the same as the test of independence, and it can indeed be viewed as an extension of the test of independence, differing only in the null and alternative hypotheses and their interpretation. In the test of independence, two categories exist and, depending on their relationship, the row sums $r_{i}$ and the column sums $c_{j}$ are obtained randomly; but in the test of homogeneity, the sample size $r_{i}$ of each population is fixed, and $O_{ij}$ is regarded as being obtained randomly according to the proportion vector $\mathbf{p}_{i}$. That this proportion vector $\mathbf{p}_{i}$ is consistent regardless of $i$ means there is no particular difference across populations, and consequently it amounts to the choice of population and the proportion of categories being independent.
As a simple example, to determine whether there is a difference in grades between the two genders, male and female, in a single university course, one can construct a contingency table as follows and perform a test of homogeneity.
| Gender | A | B | C | D | F | Total |
|---|---|---|---|---|---|---|
| Male | 29 | 52 | 26 | 17 | 6 | 130 |
| Female | 6 | 75 | 98 | 62 | 9 | 250 |
- $H_{0}$: The grade proportions are identical regardless of gender.
- $H_{1}$: The grade proportions are not identical regardless of gender.
As can be seen, the grade category has five classes from A to F, so it is difficult to interpret the difference by simply fitting it to a 13:25 ratio. To check whether the grade proportions of males and females can be said to be statistically identical at a significance level $\alpha = 0.05$, since $R = 2$ and $R = 5$, we compare the test statistic $\mathcal{X}^{2}$ with $\chi_{1 - \alpha}^{2} (4) \approx 9.49$ obtained from the chi-squared distribution with degrees of freedom $(2-1)(5-1) = 4$. The actual computation yields $\mathcal{X}^{2} \approx 54.889$, which greatly exceeds $\chi_{1 - \alpha}^{2} (4)$, so we can conclude at a significance level of $5\%$ that there is a difference in grades according to gender.
What must be noted is that such a statistical interpretation should stop at ’not identical’. Of course, anyone with eyes can grasp in this hypothetical example that the males’ grades are superior, and one might say in passing ’the guys do better’, but such a judgment comes not from the test of homogeneity but from the intuition that ‘A is probably the best and F is probably the worst’. Even if such information were given, superiority was never mentioned in the null and alternative hypotheses of the test of homogeneity, and this is because the mathematical background actually is so.