Background: Studies involving a large number of comparisons have a high likelihood of finding statistically significant associations by chance alone (Type 1 error). Genetic association studies are particularly prone to this pitfall. We tested the effect of multiple comparisons in a study of symptoms among subjects genotyped for mutations of the HFE hemochromatosis gene.
Methods: Two randomly selected groups were created from a dataset of 30,917 white adult subjects genotyped for the C282Y and H63D mutations of the HFE gene. Frequency of symptoms among subjects with different HFE genotypes was compared to sex-matched wild type controls in Random Group 1 (hypothesis generation). Statistically significant associations (p<0.05 by chi2) were then tested in Random Group 2 (hypothesis testing).
Results: A total of 101/1765 associations in men and 116/2015 associations in women were statistically significant in Random Group 1. Of these, 12 associations in men and 13 associations in women were also statistically significant in Random Group 2, 11 of which were specific to hemochromatosis. None of the remaining 14 associations (6 in men and 8 in women) involved symptoms with a biologically plausible relationship to hemochromatosis or iron overload.
Conclusions: Genetic association studies should be scrutinized for the possibility of Type 1 error.