logo

Test of Independence 📂Statistical Test

Test of Independence

Hypothesis Testing 1

In a multinomial experiment, assume we have categorical data with two characteristics XX and YY, where XX has RR categories and YY has CC categories obtained from nn independent trials. The following hypothesis test using the Pearson chi-square test statistic is called the test of independence.

  • H0H_{0}: The two categories are independent.
  • H1H_{1}: The two categories are dependent.

Test Statistic

The test statistic is the Pearson chi-square test statistic. X2=i=1Rj=1C(OijEij)2Eij \mathcal{X}^{2} = \sum_{i=1}^{R} \sum_{j=1}^{C} {{ \left( O_{ij} - E_{ij} \right)^{2} } \over { E_{ij} }} Here, the frequency OijO_{ij} of the data belonging to the ii-th category of XX and the jj-th category of YY is called the observed frequency. The product of the number of samples belonging to the ii-th category of XX, rir_{i}, and the number of samples belonging to the jj-th category of YY, cjc_{j}, divided by the sample size nn, E:=ricj/nE := r_{i} c_{j} / n, is called the expected frequency. This test statistic follows a chi-square distribution, where the degrees of freedom is the product of the number of categories of XX and YY, RR and CC, minus 11 each, giving (R1)(C1)(R-1)(C-1).

Explanation

Degrees of Freedom

For simplicity, let the ratio of the frequency OijO_{ij} of data belonging to the ii-th category of XX and the jj-th category of YY by the sample size nn be pijp_{ij}. pi=j1=1Cpij=rin    i=1Rpi=1pj=i=1Rpij=cjn    j=1Cpj=1 \begin{align*} p_{i} = \sum_{j1=1}^{C} p_{ij} = {\frac{ r_{i} }{ n }} \implies & \sum_{i=1}^{R} p_{i} = 1 \\ p_{j} = \sum_{i=1}^{R} p_{ij} = {\frac{ c_{j} }{ n }} \implies & \sum_{j=1}^{C} p_{j} = 1 \end{align*} Considering each marginal probability, we see that given these constraints, knowing R1R-1 probabilities from each category determines the remaining ones. Therefore, the degrees of freedom must be the product, (R1)(C1)(R - 1) (C - 1).

X\Yy1y2
x100000000
x200000000
x300000000

For example, the above contingency table represents data where XX has R=3R = 3 categories and YY has C=2C = 2 categories, giving a degrees of freedom of (31)(21)=2(3-1)(2-1) = 2.

Independence

Despite the seemingly complex explanations involving contingency tables and products of probabilities, a test of independence is essentially an extension of a goodness-of-fit test. In a goodness-of-fit test, the null hypothesis is that ’the given data was sampled to conform to theoretical probabilities’; in a test of independence, it means the theoretical probabilities simply manifest as the product of marginal probabilities for each category.

Independence of Events: Assume a probability space (Ω,F,P)(\Omega , \mathcal{F} , P).

  1. For P(B)>0P(B)>0, denote P(AB)=P(AB)P(B)\displaystyle P (A | B) = {{P(A \cap B)} \over {P(B)}} as the conditional probability of BB with respect to AA.
  2. If P(AB)=P(A)P(A | B) = P(A), i.e., P(AB)=P(A)P(B)P( A \cap B) = P(A) \cdot P(B), then A,BA, B are said to be independent.

In practice, the independence of events is defined such that the probability of both events occurring is the product of the probabilities of each event occurring separately. In the context of a test of independence, if the null hypothesis is true, then pij=pipjp_{ij} = p_{i} p_{j} should hold. Oijn=pij=pipj    Oij=npipj=nrincjn=ricjn=Eij \begin{align*} & {\frac{ O_{ij} }{ n }} = p_{ij} = p_{i} p_{j} \\ \implies & O_{ij} = n p_{i} p_{j} = n {\frac{ r_{i} }{ n }} {\frac{ c_{j} }{ n }} = {\frac{ r_{i} c_{j} }{ n }} = E_{ij} \end{align*} Significant differences between the observed frequency OijO_{ij} and the expected frequency EijE_{ij} mean a large value for the chi-square test statistic—indicating that the theoretical probability assumed under true independence leads to a smaller expected value X2\mathcal{X}^{2}. Thus, one would need to question the null hypothesis itself.

Example

Mendel’s Law of Inheritance

(Continuing from the goodness-of-fit test post)

mendel.jpg

Mendel’s law of inheritance states that in terms of color, yellow is dominant and green is recessive, and in terms of shape, round is dominant and wrinkled is recessive. When purebred yellow round peas are crossed with purebred green wrinkled peas, the first generation exhibits the dominant traits (yellow round). The second generation appears in a 3:1 ratio for both yellow to green and round to wrinkled, resulting in combined ratio of 9:3:3:1 overall2.

Round YellowWrinkled YellowRound GreenWrinkled Green
Observed Frequency77323123859

Assuming the observed frequencies of the second-generation peas are as shown, 3 the total sample size is n=1301n = 1301 and the number of categories is k=4k = 4. In a test of independence, a contingency table would be constructed as follows:

Color\ShapeRoundWrinkled
Yellow773231
Green23859

Let’s check if the two characteristics, color and shape, are independent at a significance level of α=0.05\alpha = 0.05. In hypothesis testing, the degrees of freedom for a chi-square distribution is not k1=3k-1 = 3 but R=2R = 2, as opposed to C=2C = 2, so that it’s (21)(21)=1(2-1)(2-1) = 1.

  • H0H_{0}: The experimental results show that color and shape are independent.
  • H1H_{1}: The experimental results show that color and shape are dependent.

The null and alternative hypotheses are as stated above, and the Pearson chi-square test statistic is calculated as E11=(773+231)(773+238)1301780.2029E12=(773+231)(231+59)1301223.7971E21=(238+59)(773+238)1301230.7971E22=(238+59)(231+59)130166.2029X2=i=1Rj=1C(OijEij)2Eij(773780.2029)2780.2029+(231223.7971)2223.7971+(238230.7971)2230.7971+(5966.2029)266.20291.31 \begin{align*} E_{11} =& {\frac{ (773 + 231) \cdot (773 + 238) }{ 1301 }} \approx 780.2029 \\ E_{12} =& {\frac{ (773 + 231) \cdot (231 + 59) }{ 1301 }} \approx 223.7971 \\ E_{21} =& {\frac{ (238 + 59) \cdot (773 + 238) }{ 1301 }} \approx 230.7971 \\ E_{22} =& {\frac{ (238 + 59) \cdot (231 + 59) }{ 1301 }} \approx 66.2029 \\ \mathcal{X}^{2} =& \sum_{i=1}^{R} \sum_{j=1}^{C} {{ \left( O_{ij} - E_{ij} \right)^{2} } \over { E_{ij} }} \\ \approx & {\frac{ \left( 773 - 780.2029 \right)^{2} }{ 780.2029 }} + {\frac{ \left( 231 - 223.7971 \right)^{2} }{ 223.7971 }} \\ & + {\frac{ \left( 238 - 230.7971 \right)^{2} }{ 230.7971 }} + {\frac{ \left( 59 - 66.2029 \right)^{2} }{ 66.2029 }} \\ \approx & 1.31 \end{align*} For the given significance level α=0.05\alpha = 0.05, the upper bound of the acceptance region for the chi-square distribution with degrees of freedom 11 is χ1α2(1)3.84\chi_{1 - \alpha}^{2} (1) \approx 3.84, and since χ1α2>X2\chi_{1 - \alpha}^{2} > \mathcal{X}^{2}, the null hypothesis cannot be rejected. This means that while we cannot support Mendel’s law of inheritance based on this data through a goodness-of-fit test, we can claim that the two characteristics are independent.

See Also


  1. Mendenhall. (2012). Introduction to Probability and Statistics (13th Edition): p602. ↩︎

  2. http://legacy.biotechlearn.org.nz/themes/mendel_and_inheritance/images/inheritance_of_multiple_traits_in_peas ↩︎

  3. 경북대학교 통계학과. (2008). 엑셀을 이용한 통계학: p269. ↩︎