Chi-square tests for homogeneity, independence, and goodness of fit

When data come in categories, we can compare the counts we observed with the counts we’d expect under a model. Let’s look at how that comparison works and how the question we’re asking determines which chi-square test to use.

Choose the chi-square test

Use a test for homogeneity to compare a categorical distribution across separately sampled groups. Use a test for independence when you sample one group and record two categorical variables for each observation. Use a goodness-of-fit test when you have one categorical variable and want to compare its counts with specified probabilities.

Use counts in mutually exclusive categories, with one count per observation. For inference, use random sampling or an appropriate randomized design and independent observations. When sampling without replacement, the 10% guideline helps justify treating observations as independent if the sample is at most 10% of its population. For these examples, check that each expected count is at least 5 so the chi-square approximation is appropriate.

Calculate expected counts, the statistic, and degrees of freedom

For homogeneity and independence, multiply the cell’s row and column totals and divide by the grand total. For goodness of fit, multiply the total count by the category’s probability under the null hypothesis. Let O and E be the observed and expected counts for a cell or category.

χ2=∑(O-E)2Edf=(r-1)(c-1)

The second line is for an r-by-c table. Count only the rows and columns of observations. For goodness of fit with k categories and probabilities specified in advance, use df=k-1. The p-value is the area to the right of the observed statistic under the chi-square curve for those degrees of freedom.

Chi-square distributions at three degrees of freedomAll three curves lie to the right of zero. With two degrees of freedom the curve decreases from its peak at zero. With four and eight degrees of freedom, the peaks move to the right.0481216df = 2df = 4df = 8Chi-square statistic
Changing the degrees of freedom changes the curve, so it also changes the right-tail probability for a given statistic.

Homogeneity example with pet preferences

Two groups of students are surveyed about whether they prefer cats, dogs, or another pet. The null hypothesis is that the distribution of preferences is the same for both groups. For the test, treat the two samples as random and independent.

Pet preferences by group
Group and preferenceObservedExpected
Male · Cats1925.00
Male · Dogs3434.44
Male · Other2215.56
Female · Cats2620.00
Female · Dogs2827.56
Female · Other612.44

There are 75 students in the male group and 60 in the female group. The total for cats is 45 out of 135 students. So the first expected count is 75·45135=25, and its contribution to the statistic is (19-25)225=1.44. The smallest expected count is about 12.44.

Use the unrounded expected counts to add all six contributions. For this 2-by-3 table, the degrees of freedom are (2-1)(3-1)=2.

χ2≈9.26p≈0.00975

The p-value is below 0.02, so we reject the null hypothesis at that level. It’s also just below 0.01. The samples give evidence that pet preference distributions differ between the groups.

Independence example with eye color and handedness

In a random sample of 1000 of a company’s 15000 employees, record eye color and handedness for each person. The null hypothesis is that the variables are independent. The sample is less than 10% of the population, and the smallest expected count is 9.38.

Eye color by handedness
Hand and eye colorObservedExpected
Left · Brown7274.48
Left · Blue3635.14
Left · Green2021.00
Left · Hazel129.38
Right · Brown460457.52
Right · Blue215215.86
Right · Green130129.00
Right · Hazel5557.62

The first expected count is 140·5321000=74.48. Add the eight contributions, and use (2-1)(4-1)=3 degrees of freedom.

χ2≈1.03p≈0.795

At α=0.05 we fail to reject independence. The sample gives insufficient evidence of an association.

Goodness-of-fit example with monthly cupcake sales

Suppose the model assigns an equal share of the year’s cupcake sales to each of the 12 months. There are 1020 sales, so the model gives 85 expected sales per month. The chi-square inference here assumes sales can be treated as independent observations under that model.

Cupcake sales by month
MonthObservedExpected
Jan6085
Feb8085
Mar6585
Apr7085
May8085
Jun10085
Jul14085
Aug12085
Sep9085
Oct9085
Nov6085
Dec6585

Use all 12 months. For example, the September contribution is (90-85)285=2585, and October gives the same contribution. The fixed probabilities give 12-1=11 degrees of freedom.

χ2=685085≈80.59df=11

The right-tail critical value at α=0.05 is about 19.68. Because 80.59>19.68, we reject the equal-month model under the stated assumptions.

Goodness-of-fit example with a dog-walking schedule

Marla’s model gives probabilities of 15%, 25%, 30%, 20%, and 10% for seeing one through five walkers working at a sampled time. She randomly samples 100 times. For this calculation, assume the sampling design lets us treat the observations at those times as independent.

Number of walkers working at sampled times
WalkersObservedExpected
11515
21625
33730
42420
5810

Every expected count is at least 10. For two walkers the contribution is (16-25)225=8125. Using all five categories gives

χ2=911150≈6.07df=5-1=4p≈0.194

At α=0.05 we fail to reject Marla’s model. These observations give insufficient evidence that the category probabilities differ from her model.