• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of ajhgLink to Publisher's site
Am J Hum Genet. Jan 10, 2008; 82(1): 100–112.
Published online Jan 4, 2008. doi:  10.1016/j.ajhg.2007.09.006
PMCID: PMC2253956

Shifting Paradigm of Association Studies: Value of Rare Single-Nucleotide Polymorphisms

Summary

Currently, single-nucleotide polymorphisms (SNPs) with minor allele frequency (MAF) of >5% are preferentially used in case-control association studies of common human diseases. Recent technological developments enable inexpensive and accurate genotyping of a large number of SNPs in thousands of cases and controls, which can provide adequate statistical power to analyze SNPs with MAF <5%. Our purpose was to determine whether evaluating rare SNPs in case-control association studies could help identify causal SNPs for common diseases. We suggest that slightly deleterious SNPs (sdSNPs) subjected to weak purifying selection are major players in genetic control of susceptibility to common diseases. We compared the distribution of MAFs of synonymous SNPs with that of nonsynonymous SNPs (1) predicted to be benign, (2) predicted to be possibly damaging, and (3) predicted to be probably damaging by PolyPhen. Our sources of data were the International HapMap Project, ENCODE, and the SeattleSNPs project. We found that the MAF distribution of possibly and probably damaging SNPs was shifted toward rare SNPs compared with the MAF distribution of benign and synonymous SNPs that are not likely to be functional. We also found an inverse relationship between MAF and the proportion of nsSNPs predicted to be protein disturbing. On the basis of this relationship, we estimated the joint probability that a SNP is functional and would be detected as significant in a case-control study. Our analysis suggests that including rare SNPs in genotyping platforms will advance identification of causal SNPs in case-control association studies, particularly as sample sizes increase.

Introduction

The common-disease common-variant (CDCV) hypothesis1–4 has been the prevailing paradigm for case-control association studies for the past decade. Although the CDCV hypothesis1 originally defined common polymorphisms as those with a population frequency of ≥1%, in practice researchers often exclude single-nucleotide polymorphisms (SNPs) that have frequencies <5% from case-control association studies. The International HapMap Project was designed to improve the efficiency of case-control association studies and intentionally targeted SNPs with minor allele frequencies (MAFs) of ≥5%.5,6 Common SNPs (SNPs with MAF ≥5%) are preferentially queried in most case-control association studies for two major reasons: (1) the statistical power is not sufficient for rare SNPs when sample sizes are limited, and (2) common SNPs can significantly contribute to disease prevalence even if their effect on disease risk is modest.

Case-control association studies have led to the identification of several polymorphisms that affect a person's risk for common diseases, including Alzheimer's disease (APOE),7 type 2 diabetes (PPARG and KCNJ11),8–10 and several others.11–14 Furthermore, several common SNPs affecting cancer susceptibility have been identified.15–18 However, many of these currently identified SNPs have modest effects on cancer risk and have low reproducibility.19–23

It is also noteworthy that most of the cited studies were conducted with relatively small study samples (400–1000 study subjects). Recent technological advances enable genotyping of hundreds of thousands of SNPs in thousands of cases and controls (e.g., 24 and 25). A large sample size allows SNPs with MAF <5% to be analyzed. The dominance of the CDCV hypothesis has dissuaded genotyping companies from including rare SNPs in coding and promoter regions in their SNP genotyping panels. In this analysis, we evaluated the hypothesis that in large case-control association studies, targeting SNPs with MAF <5% is likely to be more effective than targeting common SNPs in detecting genetic susceptibility to common diseases, including cancer.

Material and Methods

Data Retrieval

We used the International HapMap database (rel22_Build36)26 to retrieve data on the distribution of MAFs for SNPs annotated as intronic, synonymous, or nonsynonymous by the dbSNP database.27,28 The HapMap data are subdivided into three groups or samples by race: whites of North European origin, Asians (Chinese and Japanese), and Yoruba in Ibadan, Nigeria. In this study, we separately analyzed CEPH and Yoruba samples because there is considerable variation in allele frequencies between them.29–31 We did not include Chinese and Japanese samples because different sample sizes were queried: If analyzed separately, the sample size is lower than those for CEPH and Yoruba; if combined, the sample size is bigger. A separate analysis was run with SNPs data from the ENCODE project.32 We obtained the ENCODE data by sequencing ten 500 kb regions in 48 individuals (16 from each group). The novel SNPs detected by sequencing were then genotyped in all 269 HapMap DNA samples.32

Data from the dbSNP database were used for the analysis of the relationship between MAF and the proportions of nsSNPs predicted to be protein disturbing. Not all SNPs reported in the dbSNP are true polymorphisms; according to some studies, the false-discovery rate might be as high as 10%.33,34 To decrease the proportion of false discoveries in our sample, we used only frequency-validated SNPs. Thus, 6158 frequency-validated nsSNPs from 3912 genes were used in the analysis.

SeattleSNPs Database

The SeattleSNPs project generated SNP data for samples from both European and African populations. At the time of our access (August 2007), the database contained sequencing data from 307 genes ranging in length from 3 Kb (ICAM4 gene) to 653 Kb (SEC15L2 gene). The SNP data were available for 24 African descent (AD) and 23 European descent (ED) subjects. The total number of SNPs detected in the analysis included 31505 intronic, 764 synonymous, and 720 nonsynonymous SNPs. We did not include deletions, insertions, and sites with more than two alleles in the analysis. The SNPs were identified by sequencing of genomic DNA and, therefore, provide unbiased representation of different types of SNPs in gene regions. Because the number of nonsynonymous SNPs was low in this sample, we subdivided SNPs in ten MAF categories with increments of 5%. Nonsynonymous SNPs were subdivided into two groups: (1) benign (B) and (2) possibly or probably damaging SNPs (Pos.D./Prob.D.). We combined the possibly and probably damaging SNPs together because overall there were only 214 damaging SNPs.

Intronic Ratio

We used the ratio of absolute numbers of nsSNPs to the absolute number of intronic SNPs in a given MAF bin (intronic ratio) to visualize the effect of purifying selection.35 A constant intronic ratio suggests that there are no differences in the intensity of purifying selection among MAF bins. Counts of the SNPs of different MAF categories for HapMap and SeattleSNPs samples are shown in Tables 1 and 2.

Prediction of Functional SNPs

NsSNPs that are likely to disturb protein structure or function can be predicted with bioinformatics approaches. Several bioinformatics tools for predicting the functionality of nsSNPs have been developed.36–38 In this study, we used SIFT and PolyPhen to evaluate the functional significance of SNPs because those methods are the most frequently used.36 SNPs predicted to be intolerant by SIFT were considered functional, and SNPs predicted to be tolerant were considered nonfunctional. For the PolyPhen-based prediction, possibly or probably protein-damaging SNPs were considered functional, and SNPs predicted to be benign were considered nonfunctional.

For estimating the relationship between MAF and the proportion of predicted protein-disturbing SNPs among nsSNPs, the nsSNPs were binned into 20 categories defined by MAF increments of 2.5%. For each MAF category, we computed the proportion of SNPs predicted to be protein disturbing. To compare MAF distributions for different types of SNPs, these were also were binned into 20 groups defined by MAF increments of 2.5%.

Radical and Conservative Missense Mutations

To stratify amino acid substitutions into radical and conservative, we adopted the classification system used by Dagan et al.39 In brief, all amino acids were subdivided into three groups according to their charge: positive (R, H, and K), negative (D and E), and uncharged (A, N, C, Q, G, I, L, M, F, P, S, T, W, Y, and V). The amino acids were further subdivided by volume and polarity: special (C), neutral and small (A, G, P, S, and T), polar and relatively small (N, D, Q, and E), polar and relatively large (R, H, and K), nonpolar and relatively small (I, L, M, and V), and nonpolar and relatively large (F, W, and Y). Any substitutions that moved an amino acid from one category to another were considered radical, whereas substitutions that did not change amino acid category were classified as conservative. We performed separate analyses for radical (totaling 3695) and conservative (totaling 2463) substitutions.

Statistical Analysis

Spearman's nonparametric correlation was used for estimation of the association between MAF and the proportion of nsSNPs predicted to be protein disturbing, P(F). We used logarithmic regression, P(F)=aln(MAF)+b, and linear regression, P(F)=aMAF+b, to fit the binned data by the least-squares method.

Statistical power was computed by assuming a case-control design with independent cases and controls, and the data were analyzed by an uncorrected chi-square test.40 The sample size was varied from 100 to 10,000 in increments of 100. The MAF was assumed to vary from 0 to 0.5 in increments of 0.025. Dominant and recessive models with genotypic risk ratios of 1.3 and 1.5 were considered. Critical p values of 0.05 were used.

Adjusting the Proportion of Functional nsSNPs by Sensitivity and Specificity of PolyPhen

The observed proportion of functional nsSNPs depends on the true proportion and on the sensitivity and specificity of the predicting method. Let Ptf be the true proportion of functional nsSNPs in a given MAF category (bin) and Pobs be the observed proportion of functional SNPs. For PolyPhen, the probability of identifying a SNP as functional when it is functional (sensitivity) is ~0.82, and the probability of identifying a SNP as functional when it is nonfunctional (1 specificity) is ~0.08.38 The observed proportion of functional SNPs, given that the true proportion is Ptf, can be computed as follows: Pobs = Ptf·0.82 + (1 − Ptf) 0.08; therefore, Ptf = (Pobs − 0.08)/0.74. We used the latter equation to adjust the estimated proportion of protein-disturbing nsSNPs for the sensitivity and specificity of PolyPhen. We were not able to find estimates of specificity and sensitivity for SIFT and, therefore, were not able to provide a similar correction for the proportion of SNPs predicted to be functional by SIFT.

Results

Distribution of SNPs by MAF

The International HapMap Project26 and the dbSNP database27,28 were used as sources of data. We retrieved data from the International HapMap Project on the distribution of MAFs in coding regions (rel22_Build36). A separate analysis was run with SNPs data from the Encyclopedia of DNA Elements (ENCODE) project.32 The ENCODE data were obtained by sequencing ten 500 kb regions in 48 individuals (16 from each group). The novel SNPs detected by sequencing were then genotyped in all HapMap DNA samples.32

We compared the proportion of SNPs in different MAF categories by using the ENCODE SNPs and all the HapMap SNPs (Figure 1). The total number of SNPs in the ENCODE data set was 93,149, and the total number of SNPs in the phase II HapMap database (rel22_NCBI_Build36) was 3,839,363 for the Centre d'Etude du Polymorphisme Humain (CEPH) population and 3,782,818 for the Yoruba population. Only parents were included in the analysis. SNPs were included in the analysis regardless of their MAF and binned into 20 MAF groups. We found that the proportion of SNPs with MAF <5% in the ENCODE data set (0.50 ± 0.01) was significantly greater than the proportion of rare SNPs in the entire HAPMAP data set (0.38 ± 0.01). Because ENCODE SNPs were identified by direct sequencing of a constant sample size, they are expected to be less biased than phase II HapMap SNPs for which the initial SNP discovery phase reflects a combination of pooled sequencing and direct sequencing conducted on varying number of subjects; this result strongly suggests that the majority of SNPs in the human genome have MAF <5%. The limited number of 45 individuals sequenced by ENCODE further suggests that the actual proportion of rare SNPs could be greater than 50%.

Figure 1
Distribution of SNPs from the Encyclopedia of DNA Elements and of All SNPs Reported in the International HapMap Database by Minor Allele Frequency

MAF and the Intronic Ratio Based on HapMap Data

The ratio of absolute numbers of SNPs of specific categories reported in the database (e.g., nsSNPs) to the absolute number of intronic SNPs (here referred to as the intronic ratio) can be used as a relative measure of selection.35 A constant intronic ratio across different MAF categories suggests that there are no differences in intensity of selection among MAF categories. An increased intronic ratio at low MAFs suggests purifying selection against nsSNPs. We computed intronic ratios for 20 MAF categories from the HapMap data for (1) nonsynonymous SNPs predicted to be probably damaging protein structure and function damaging (Prob.D.), (2) nonsynonymous SNPs predicted to be possibly damaging protein structure and function damaging (Pos.D.), (3) nonsynonymous SNPs predicted to be benign (B), and (4) synonymous SNPs (S). We used PolyPhen for prediction of functionality.38 The list of nsSNPs with prediction of functionality can be found in Table S1 available online.

Figures 2A and 2D show the intronic ratios for nsSNPs for CEPH (Europeans) and YRI (Africans) samples correspondingly. We found that the intronic ratio was nearly constant for nsSNPs with MAF >20%. However, for nsSNPs with MAF <10%, and especially for SNPs with MAF <5%, the intronic ratio increased sharply, suggesting a strong effect of purifying selection. We further found that the relationship between MAF and intronic ratio for benign SNPs was similar to that for synonymous SNPs. We also found an increased intronic ratio at lower MAFs for Prob.D. and Pos.D. SNPs, suggesting stronger purifying selection against these categories. The intronic ratio was increased for rare synonymous and benign SNPs for HapMap data set, but this effect could reflect a bias against genotyping of rare intronic SNPs.

Figure 2
Distribution of Intronic Ratios and MAFs for Various Types of the HapMap SNPs in Coding Regions of the Human Genome: S, B, Pos.D., and Prob.D. SNPs

Comparison of MAF Distributions of SNPs of Different Functional Types in the Coding Region Based on HapMap Data

Not all SNPs reported in the database are true polymorphisms; according to some studies, the false-discovery rate might be as high as 10%.33,34 To decrease the proportion of false discoveries in our sample, we used only frequency-validated SNPs. Thus, 6158 frequency-validated nsSNPs from 3912 genes were used in the analysis.

We found that the MAF distribution of the probably damaging SNPs (SNPs that are most likely to disturb protein structure and function) was left-shifted in CEPH and YRI samples (Figures 2B, 2C, 2E, and 2F). There was also a trend for SNPs that were likely to be functional to have lower MAF. The difference between SNP categories suggests that purifying selection shapes MAF distributions of SNPs in the coding region. We excluded intronic SNPs from this analysis of MAF distributions because of a bias against genotyping intronic SNPs; intronic SNPs are less likely to be chosen for genotyping by HapMap compared with the SNPs in the coding regions.

Intronic Ratios and MAF Distributions Based on SeattleSNPs Data

Figures 3A and 3D show the intronic ratios for SNPs identified by sequencing of genomic DNA (SeattleSNPs database) for the subjects of European and African descent correspondingly. We found that in European subjects in the group of rare SNPs (MAF ≤0.05) the intronic ratio for Pos.D./Prob.D. SNPs was 3.74 ± 0.18, which was significantly higher compared to the intronic ratio for both synonymous (1.23 ± 0.14) and benign (1.41 ± 0.17) SNPs (t test for S versus Pos.D./Prob.D. SNPs was 9.6, N = 32,269, p << 0.001). No significant differences were detected between benign and synonymous SNPs. Intronic ratios for these two types of SNPs were constant across MAF categories. Similar results were obtained for the subjects of African descent (Figure 3D).

Figure 3
Distribution of Intronic Ratios and SNPs by MAF Categories, SeattleSNPs Database

Figures 3B, 3C, 3E, and 3F show the distribution of MAF for intronic, synonymous, benign, and damaging SNPs among Europeans and Africans correspondingly. For Europeans, we found that the proportion of Pos.D./Prob.D. SNPs was highest in MAF 0–0.05 (0.78 ± 0.02); the proportion of benign SNPs was not significantly different from the proportion of synonymous SNPs, with 0.68 ± 0.02 and 0.67 ± 0.02, correspondingly. The proportion of intronic SNPs in the MAF category 0–0.05 (0.64 ± 0.01) tended to be lower compared to that of benign and synonymous SNPs. Similar results were obtained for the subjects of African descent (Figures 3E and 3F).

Relationship between MAF and the Proportion of Protein-Damaging SNPs

We analyzed the relationship between the MAF and the proportion of nsSNPs predicted to be protein damaging by PolyPhen (Figure 4A) (Spearman's correlation coefficient was −0.75, n = 25, and p < 0.001). The logarithmic regression of the observed proportion of functional nsSNPs on the MAF was P(F)=0.04ln(MAF)+0.17. In this case, logarithmic regression explained 79% of the variation and also fitted the data better than did linear regression, which explained 56% of the observed variation. For the PolyPhen method, we also adjusted the prediction curve by PolyPhen sensitivity and specificity as described in the Material and Methods.

Figure 4
Proportion of Nonsynonymous Single-Nucleotide Polymorphisms Predicted to be Protein Damaging Plotted against Minor Allele Frequency

A similar result was obtained for the proportion of the nsSNPs predicted to be protein damaging by sorting intolerant from tolerant (SIFT).41 MAF was negatively correlated with the proportion of SIFT-predicted protein-damaging nsSNPs (Spearman's nonparametric correlation coefficient was −0.73, n = 25, and p < 0.001) (Figure 4B). Logarithmic regression explained 74% of the observed variation and fitted the data better than did linear regression, which explained only 54% of the variation.

Obviously MAF is not the only indicator of the probability that a SNP is functional. The category of SNP is also associated with its functionality. For example, synonymous SNPs are less likely to be functional compared with nsSNPs. If we consider SNPs from a specific functional category (e.g., synonymous SNPs), the overall probability for SNPs from that category to be functional will vary from that for SNPs in other categories. Within a category, however, one can expect to see the same inverse relationship between MAF and the proportion of functional SNPs because purifying selection will drive down MAFs of functional SNPs. We compared the proportion of SNPs predicted to be functional separately for nsSNPs producing radical mutations and for nsSNPs producing conservative missense mutations (Figure 5). A radical missense mutation replaces wild-type amino acid with an amino acid that is chemically different, whereas conservative mutations replace wild-type amino acids with chemically similar ones. Therefore, the overall proportion of functional substitutions is expected to be greater among radical missense mutations than among conservative ones. We found that the overall probability that a SNP is functional is almost two times greater for nsSNPs producing radical missense mutations than for nsSNPs producing conservative missense mutations. We also found that the logarithmic-regression curves of the proportion of functional SNPs on MAF were very similar for these two types of SNPs, suggesting that the same factors influence MAF-functionality relationships for SNPs having different prior probabilities to be functional.

Figure 5
Conservative versus Radical Amino Acid Substitutions

Statistical Power to Detect Effects of Rare SNPs

Statistical power depends on many factors including effect size (usually expressed as the odds ratio [OR]), sample size, mode of inheritance (e.g., dominant or recessive), and MAF. The statistical power is generally lower for rare SNPs than for common SNPs of a similar effect size. Figure 6 illustrates the relationship between statistical power and the needed sample size for a series of SNPs with MAF ≤5%, assuming a dominant model. For a SNP with OR = 1.5 and MAF = 5%, a sample size of 1862 (931 cases and 931 controls) would be needed to achieve 80% statistical power to detect the effect at a p level of 0.05. For a SNP with OR = 1.5 and MAF = 2.5%, the required sample size would be 3420, and for MAF = 1%, the required sample size would exceed 8120. The power is very sensitive to the OR. With OR = 2 and MAF = 5%, the required sample size is 580 (290 cases and 290 controls). For the same OR and MAF = 2.5%, the required sample size is 1050, and for MAF = 1%, the required sample size is 2450. This shows that when the effect of rare SNPs is relatively high (OR ≥2), there is sufficient power to detect the effect of a rare SNP, even for a modest sample size of 1000.

Figure 6
Relationship between Statistical Power and Needed Sample Size

Power to Detect a True Association

Not all nsSNPs are functional and impart a potential to be disease associated. Statistical power predicts the probability that a SNP will be detected as significant conditional on its being functional, which we denote as P(S,F). Thus the joint probability that a SNP is significant and functional is expressed as P(S,F) = P(S|F)P(F), for which P(S,F) is the joint probability that a SNP is significant and functional, P(S|F) is the power, and P(F) is the probability that a SNP is functional. If we assume P(F) = 1, we will obtain the statistical power that is usually used to design case-control association studies. Our analysis demonstrated, however, that the probability that a SNP was functional was negatively correlated with MAF, whereas the probability that a SNP will be detected as significant was positively correlated with MAF. In other words, there is a trade-off between gain in probability of detection and loss in the proportion of functional SNPs when MAF increases. To account for this inverse relationship, we used the P(S,F) = P(S|F)P(F) formula to compute the joint probability that a SNP is functional and will be detected as significant. We defined this joint probability as the power to detect a true association (PDTA), which predicts statistical power when P(F)1 but depends on MAF.

Figure 7 gives a quantitative example of computing PDTA and shows that PDTA first increases, reaches a maximum, and subsequently decreases. The MAF at which PDTA is maximal is an important parameter for the design of a case-control study because it maximizes the chance that a functional SNP will be detected as significant. The MAF at which PDTA is maximal was denoted as the most powerful MAF (mpMAF).

Figure 7
Example of Computing Probability to Detect True Association and Most Powerful Minor Allele Frequency

For this and other computations of PDTA, we used a conservative assumption that the OR did not depend on MAF and that only the proportion of functional SNPs did. However, rare SNPs might disturb gene function to a greater extent than common SNPs and therefore have higher ORs. If this is true, statistical power should increase with the increasing rarity of SNPs compared with a model with a constant OR.

Like statistical power, PDTA depends on sample size, effect size, inheritance model, and MAF. PDTA's dependence on the sample size is important because sample size is one of the key parameters in case-control-study design. We computed PDTAs for a set of sample sizes and MAFs. Recessive and dominant models were analyzed (Figure 8). Ridges on the PDTA surfaces mark MAFs at the mpMAF, where the PDTA is maximal. The mpMAF ridge was much sharper for the dominant model than the recessive model, suggesting that a given deviation in MAF from the mpMAF leads to a much stronger decrease in the PDTA in the dominant model than in the recessive model.

Figure 8
Dependence of the Probability to Detect a True Association on Minor Allele Frequency and Sample Size

The results presented in Figure 8 suggest an inverse relationship between mpMAF and sample size. We further investigated the relationship between mpMAF and sample size by computing PDTAs and mpMAFs for dominant and recessive models with ORs of 1.3 and 1.5. We found that for all scenarios, mpMAF decreased as the sample size increased (Figure 9). For the dominant model, with a modest OR of 1.5, mpMAF was <5% when the sample size was ≥1500. For the recessive model, with an OR of 1.5, mpMAF was <5% when the sample size was ≥6000. We also found that, given the same sample size, mpMAF was higher for lower ORs (for a given sample size, mpMAF increased as OR decreased). mpMAFs were higher for the recessive model than for the dominant one.

Figure 9
Predicted Dependence of Most Powerful Minor Allele Frequency on the Sample Size

Discussion

Genetic Architecture of Common Disease

The number and penetrance of alleles affecting disease risk, i.e., the genetic architecture of a disease, directly affect the strategy for identifying polymorphisms that modulate disease susceptibility. Few theoretical analyses of the genetic architecture of common human diseases have been published.42–44 The expected number and distribution of disease alleles in the population depend on mutation rate, selection, and population demography. Mutation rate in this case means mutation rate for disease alleles. This rate depends on the number of the potential sites for deleterious disease-causing mutations in the disease-related gene and also on the number of disease genes in the genome. The disease mutation rate is higher than the nucleotide-substitution rate, which is estimated as ~10−8 mutations per nucleotide per generation.45,46 By assuming that (1) the disease mutation rate is ~10−6 disease-associated mutations per disease locus, that (2) a single dramatic expansion of the human population occurred approximately 70,000 years ago, and (3) that no genetic drift has occurred, Reich and Lander44 concluded that one or two common polymorphisms can explain genetic susceptibility to common human diseases. However, the analysis probably oversimplifies the real situation because it assumes no bottlenecks or effects of genetic drift for susceptibility mutations. Pritchard,42 who used stochastic modeling to estimate the level of genetic diversity for common diseases, concluded that “it is unlikely that any single mutation will constitute a large fraction of the susceptible class.” Simulation analysis of the genetic architecture of common human diseases by Peng and Kimmel47 demonstrated that mutation spectra are expected to be simple for a single-locus model. If, however, a common disease is caused by multiple loci, then a diverse allelic spectrum with rare causal alleles is predicted.

Recently, Kryukov et al.48 combined analysis of mutations causing human Mendelian diseases, of human-chimpanzee divergence data and the data on human genetic variation, and found that ~53% of new missense mutations have mildly deleterious effects. The authors also found that up to 70% of low-frequency missense alleles are mildly deleterious. Kotowski et al.49 used sequencing to identify rare polymorphisms in the PCSK9 gene controlling plasma levels of low-density lipoprotein cholesterol. The authors identified several rare nsSNPs with strong phenotypic effects on cholesterol level, providing support for the importance of including rare sequence variants in association studies.

As noted above, the major parameter that defines the diversity of disease alleles in a population is mutation rate per gene per generation. Unfortunately, there are no reliable estimates of this parameter. Reich and Lander44 estimated a mutation rate for disease-associated mutations as 3.2 × 10−6. Pritchard's42 estimate ranged from 2.5 × 10−6 to 1.3 × 10−4. Estimates based on the analysis of mutations reported in the human gene mutation database50,51 suggest that the mutation rate for slightly deleterious mutations is ~10−5. If the mutation rate for susceptibility alleles is ~10−5 or higher, it is likely that the genetic architecture of common diseases is diverse and that there are many susceptibility alleles in the population. Another factor that affects the genetic architecture of common disease is the number of genes contributing to genetic control of disease susceptibility. If several genes affect disease susceptibility, it is likely that many polymorphic susceptibility variants underlie disease risk. For many common human diseases, there are probably many susceptibility loci.

Cancer is a good example of a common disease with many loci affecting disease susceptibility. The development of cancer is a multistage process that involves genes important for cell-cycle control, cell proliferation, apoptosis, angiogenesis, and other cellular-pathway functions. Therefore, it is plausible to suggest that many causal SNPs modulate cancer susceptibility. We suggest that sdSNPs, which are subjected to weak purifying selection, are the major players in genetic control of susceptibility to many common diseases. However, cancer is predominantly a disease of late age when reproduction is mostly completed. Therefore, natural selection could not have affected the frequencies of alleles in cancer genes purely on the basis of their effect on cancer risk. On the other hand, genes that affect the risk of cancer did not evolve merely as cancer-risk genes; this function emerged relatively recently with the recent increase in life expectancy. Cancer suppressors and oncogenes play an important role in the control of the cell cycle, apoptosis, angiogenesis, and development processes that are under pressure of purifying selection. Therefore, protein-damaging mutations in cancer-related genes would be expected to be under the pressure of purifying selection and thus to have a lower population frequency.

SdSNPs and MAF

SdSNPs are not eliminated from the population because the reduction in fitness they cause is too small. If we assume that the observed population frequency of sdSNPs is a result of equilibrium between purifying selection and mutations, then the intensity of purifying selection against sdSNPs can be estimated on the basis of the classical formula q = μ/hs, where q is the equilibrium frequency of a mutant allele, μ is the mutation rate per generation, h is the dominance coefficient, and s is selection coefficients.52 Accepting that μ is ~10−8 (see 45 and 46) and h is ~0.1 (see 53) and assuming q = 0.05, the selection coefficient will be ~10−6 − 10−5. Genetic drift is expected to affect the population frequency of deleterious alleles when s << 1/Ne, where Ne is the effective population size. There is general agreement that for the human population, Ne is ~104.54 Therefore, it follows that selection and drift play a role in the distribution of MAFs for sdSNPs. This might also explain the existence of causal SNPs with high MAF for some common human diseases.11–14

The high prevalence of SNPs with low MAF among nsSNPs, higher intronic ratios for rare SNPs, and the inverse relationship between the proportion of protein-damaging SNPs and MAF strongly suggest that functional SNPs are under weak purifying selection and therefore tend to have lower MAFs. With respect to intronic ratio, we acknowledge that the increase of intronic ratio for low MAF can be a result of genotyping bias against intronic SNPs, especially those with low MAF when HapMap data are analyzed. It is also possible that some synonymous SNPs can be functional because of their effect on splicing or codon usage.55–58 Recent studies demonstrated that synonymous SNPs undergo a slight purifying selection.59,60

Results of our study are in agreement with other reports on the negative correlation between MAF and the proportion of functional SNPs.35,61 Cargill et al.35 analyzed 392 SNPs located in the coding regions of 106 genes and found that the proportion of nsSNPs was highest among SNPs with a low MAF. Wong et al.61 observed a similar relationship between MAF and the proportion of nsSNPs predicted to be protein damaging. The results of our analysis of the relationship between MAF and the proportion of functional SNPs are based on a much larger number of SNPs than that previously studied, and they are in agreement with previous studies and provide a more comprehensive picture of the relationship between MAF and the proportion of protein-damaging SNPs.

In this study, we used two data sets to retrieve MAF data: the HapMap and SeattleSNPs. The HapMap sample size is much larger compared to the SeattleSNPs sample size. However, the HapMap database is likely to underreport intronic SNPs, especially those with low MAF. This bias is probably the major source of the increased intronic ratio for benign and synonymous SNPs at low MAF category (Figures 2A and 2D). The intronic ratio was constant for benign and synonymous SNPs when the SeattleSNPs data were analyzed (Figures 3A and 3D). We cannot exclude also that a weak purifying selection against benign and synonymous SNPs might as well have contributed to the increased intronic ratio for benign and synonymous SNPs as suggested from the analysis of MAF distributions (Figures 3B, 3C, 3E, and 3F).

Our analysis was based on the assumption of the independence of SNPs. This is violated to some extent due to linkage disequilibrium (LD) between SNPs. If SNPs that are in strong LD tend to have similar MAFs, then the number of independent observations will be lower than the number of SNPs in the analysis. It is difficult, however, to imagine a biological phenomenon that can link MAFs of the SNPs on the basis of their position. We are not aware of any studies that address this phenomenon on a genome-wide level. Nevertheless, we have addressed this concern by analysis of singletons—single SNP per gene. In our data set, ~47% of nsSNPs are singletons. Those SNPs are unlikely to be in strong LD unless the genes are located very close to one another. The analysis conducted with singleton SNPs yielded very similar results in terms of MAF distribution between benign and possibly and probably damaging SNPs (data not shown).

Most SNPs in the Human Genome Are SNPs with MAF <5%

The MAF distribution of SNPs from the International HapMap Project shows that more than 40% of SNPs have MAF <5%. Because the International HapMap Project preferentially targeted common SNPs, the real proportion of rare SNPs is definitely higher than 40%. We estimate, on the basis of ENCODE data, that ~60% of SNPs have MAF <5%. This estimate is supported by Wong et al.,61 who sequenced 114 genes from the Environmental Genome Project.62,63 A total of 64, 38, and 12 genes were sequenced in 44, 90, and 450 individuals, respectively. Across this gene set, each base was evaluated on an average sample size of 84 individuals. The authors found that SNPs with MAF <5% constituted >60% of the total number of SNPs. If we include SNPs with MAF < < 1% in our analysis, the proportion of rare SNPs will be even higher. On the basis of practical considerations, we suggest that rare SNPs be considered those with MAF ≥0.5% and ≤5%. Indeed, current sample sizes do not allow effective detection and comparison of frequencies of SNPs with MAF <0.5%. Another reason for setting the lower limit at 0.5% for rare SNPs is that very rare SNPs would need extremely high penetrance (similar to that for dominant Mendelian mutations) to greatly affect the prevalence of disease.

Conclusions

We hypothesized that interindividual variation in susceptibility to common diseases is mainly caused by sdSNPs in genes implicated in disease pathways. Our hypothesis suggests that causal SNPs have low MAF; however, the low population frequency of the causal SNPs is compensated for by the large number of such SNPs in the genome. The deleterious effect of sdSNPs is deleterious enough to impair gene function and increase disease risk. It is, however, not strong enough for selection to totally eradicate them from the population (genetic drift, founder effects, and population bottlenecks are factors that help retain sdSNPs).

The principal difficulty in explaining common diseases by sdSNPs is that sdSNP might be too rare to explain the observed disease's prevalence. The high proportion of rare SNPs in the genome, however, can counter the low MAF of the causal SNPs. According to the Build 126 of the dbSNP database, there are >56,000 nsSNPs in the human genome. The real number of nsSNPs is probably higher because rare SNPs are underreported. If we assume that there are twice as many nsSNPs as there have been reported today64 and half of these nsSNPs are slightly deleterious with MAF <5%, and that there are ~24,000 genes in the human genome, there should be two to three rare sdSNPs per gene. The real number might be higher still because we did not consider promoter SNPs or SNPs located in sites important for splicing. This suggests that the effect of rare sdSNPs on disease prevalence can be substantial.

Our many rare SNPs hypothesis suggests that targeting rare SNPs in large case-control association studies has more power to detect causal SNPs than does targeting common SNPs. We found that there is a negative correlation between sample size and mpMAF, and this explains why most of the causal SNPs identified to date are common. Indeed, studies that have identified (and confirmed) causal SNPs used sample sizes of between 500 and 1000 subjects; for such sample sizes, the mpMAF ranges from 8% to 30%, depending on the OR and type of model. For case-control studies with sample sizes of ≥2000, the mpMAF is expected to be <5% (at least for the dominant model), suggesting that targeting rare SNPs in large studies might be a better strategy for identifying causal SNPs than targeting common SNPs, which are less likely to be functional. We conclude that targeting SNPs with MAF <5% in large case-control studies is a sound strategy to identify causal SNPs. Direct sequencing of candidate regions in a subset of cases and controls (e.g., 100 cases and 100 controls) can be used to identify rare SNPs. Those rare SNPs should be then genotyped in the whole sample with custom-designed chips.

We believe that targeting rare potentially functional SNPs (nsSNPs and SNPs located in promoter regions) can be a more appropriate strategy to understand the genetic architecture of many complex diseases compared with the strategy that targets common SNPs. Therefore, a practical recommendation from our analysis is the need for genotyping rare SNPs, especially those from the coding and promoter regions, into genotyping platforms. Another application of our analysis relates to assigning priors in Bayesian association analysis framework—SNPs can be assigned different prior weights depending on their MAFs, with higher weights being given to those with lower MAF.

In conclusion, we hypothesized that numerous rare functional SNPs are major contributors to susceptibility to common diseases, including cancer. The analysis of joint probability that a SNP is functional and that it will be detected as significant in a case-control study demonstrated that, for a given sample size, there is a MAF for which this joint probability is maximal—the most powerful MAF. We found that the larger the sample size, the lower the mpMAF, suggesting that for studies with large sample sizes (5000 and higher) targeting rare SNPs will be a better strategy for identifying causal SNPs than targeting common SNPs.

Web Resources

The URLs for data presented herein are as follows:

Supplemental Data

One table is available at http://www.ajhg.org/cgi/content/full/82/1/100/DC1/.

Supplemental Data

Document S1. Table S1:

Acknowledgments

This research was supported by the National Institute of Health grants ES09912 and PO1 CA 34936-14A1 and by the Genome Canada Grant.

References

1. Risch N., Merikangas K. The future of genetic studies of complex human diseases. Science. 1996;273:1516–1517. [PubMed]
2. Muller-Myhsok B., Abel L. Genetic analysis of complex diseases. Science. 1997;275:1328–1329. [PubMed]
3. Scott W.K., Pericak-Vance M.A., Haines J.L. Genetic analysis of complex diseases. Science. 1997;275:1327. [PubMed]
4. Long A.D., Grote M.N., Langley C.H. Genetic analysis of complex diseases. Science. 1997;275:1328. [PubMed]
5. The International HapMap Consortium The International HapMap Project. Nature. 2003;426:789–796. [PubMed]
6. The International HapMap Consortium Integrating ethics and science in the International HapMap Project. Nat. Rev. Genet. 2004;5:467–475. [PMC free article] [PubMed]
7. Strittmatter W.J., Roses A.D. Apolipoprotein E and Alzheimer's disease. Annu. Rev. Neurosci. 1996;19:53–77. [PubMed]
8. Altshuler D., Hirschhorn J.N., Klannemark M., Lindgren C.M., Vohl M.C., Nemesh J., Lane C.R., Schaffner S.F., Bolk S., Brewer C. The common PPARgamma Pro12Ala polymorphism is associated with decreased risk of type 2 diabetes. Nat. Genet. 2000;26:76–80. [PubMed]
9. Deeb S.S., Fajas L., Nemoto M., Pihlajamäki J., Mykkänen L., Kuusisto J., Laakso M., Fujimoto W., Auwerx J. A Pro12Ala substitution in PPARgamma2 associated with decreased receptor activity, lower body mass index and improved insulin sensitivity. Nat. Genet. 1998;20:284–287. [PubMed]
10. Florez J.C., Hirschhorn J., Altshuler D. The inherited basis of diabetes mellitus: Implications for the genetic analysis of complex traits. Annu. Rev. Genomics Hum. Genet. 2003;4:257–291. [PubMed]
11. Begovich A.B., Carlton V.E., Honigberg L.A., Schrodi S.J., Chokkalingam A.P., Alexander H.C., Ardlie K.G., Huang Q., Smith A.M., Spoerke J.M. A missense single-nucleotide polymorphism in a gene encoding a protein tyrosine phosphatase (PTPN22) is associated with rheumatoid arthritis. Am. J. Hum. Genet. 2004;75:330–337. [PMC free article] [PubMed]
12. Ogura Y., Bonen D.K., Inohara N., Nicolae D.L., Chen F.F., Ramos R., Britton H., Moran T., Karaliuskas R., Duerr R.H. A frameshift mutation in NOD2 associated with susceptibility to Crohn's disease. Nature. 2001;411:603–606. [PubMed]
13. Klein R.J., Zeiss C., Chew E.Y., Tsai J.Y., Sackler R.S., Haynes C., Henning A.K., SanGiovanni J.P., Mane S.M., Mayne S.T. Complement factor H polymorphism in age-related macular degeneration. Science. 2005;308:385–389. [PMC free article] [PubMed]
14. Edwards A.O., Ritter R., Abel K.J., Manning A., Panhuysen C., Farrer L.A. Complement factor H polymorphism and age-related macular degeneration. Science. 2005;308:421–424. [PubMed]
15. Li Y.Y., Xing J., Zhao L.S., Li Y.N., Wang Y.C., Zhang W.M. [Screening and analysis of coding SNPs of HLA-DQA1 gene involved in susceptibility for cervical cancer] Ai Zheng. 2006;25:906–910. [PubMed]
16. Andrew A.S., Nelson H.H., Kelsey K.T., Moore J.H., Meng A.C., Casella D.P., Tosteson T.D., Schned A.R., Karagas M.R. Concordance of multiple analytical approaches demonstrates a complex relationship between DNA repair gene SNPs, smoking and bladder cancer susceptibility. Carcinogenesis. 2006;27:1030–1037. [PubMed]
17. Webb T. SNPs: Can genetic variants control cancer susceptibility? J. Natl. Cancer Inst. 2002;94:476–478. [PubMed]
18. Schubert E.L., Lee M.K., Newman B., King M.C. Single nucleotide polymorphisms (SNPs) in the estrogen receptor gene and breast cancer susceptibility. J. Steroid Biochem. Mol. Biol. 1999;71:21–27. [PubMed]
19. Cantor C.R. The use of genetic SNPs as new diagnostic markers in preventive medicine. Ann. N Y Acad. Sci. 2005;1055:48–57. [PubMed]
20. Packer B.R., Yeager M., Burdett L., Welch R., Beerman M., Qi L., Sicotte H., Staats B., Acharya M., Crenshaw A. SNP500Cancer: A public resource for sequence validation, assay development, and frequency analysis for genetic variation in candidate genes. Nucleic Acids Res. 2006;34(Database issue):D617–D621. [PMC free article] [PubMed]
21. Packer B.R., Yeager M., Staats B., Welch R., Crenshaw A., Kiley M., Eckert A., Beerman M., Miller E., Bergen A. SNP500Cancer: A public resource for sequence validation and assay development for genetic variation in candidate genes. Nucleic Acids Res. 2004;32(Database issue):D528–D532. [PMC free article] [PubMed]
22. Horng J.T., Hu K.C., Wu L.C., Huang H.D., Lin F.M., Huang S.L., Lai H.C., Chu T.Y. Identifying the combination of genetic factors that determine susceptibility to cervical cancer. IEEE Trans. Inf. Technol. Biomed. 2004;8:59–66. [PubMed]
23. Ott J. Association of genetic loci: Replication or not, that is the question. Neurology. 2004;63:955–958. [PubMed]
24. Smyth D.J., Cooper J.D., Bailey R., Field S., Burren O., Smink L.J., Guja C., Ionescu-Tirgoviste C., Widmer B., Dunger D.B. A genome-wide association study of nonsynonymous SNPs identifies a type 1 diabetes locus in the interferon-induced helicase (IFIH1) region. Nat. Genet. 2006;38:617–619. [PubMed]
25. Breast Cancer Association Consortium Commonly studied single-nucleotide polymorphisms and breast cancer: Results from the Breast Cancer Association Consortium. J. Natl. Cancer Inst. 2006;98:1382–1396. [PubMed]
26. Thorisson G.A., Smith A.V., Krishnan L., Stein L.D. The International HapMap Project Web site. Genome Res. 2005;15:1592–1593. [PMC free article] [PubMed]
27. Sherry S.T., Ward M.H., Kholodov M., Baker J., Phan L., Smigielski E.M., Sirotkin K. dbSNP: The NCBI database of genetic variation. Nucleic Acids Res. 2001;29:308–311. [PMC free article] [PubMed]
28. Sherry S.T., Ward M., Sirotkin K. Use of molecular variation in the NCBI dbSNP database. Hum. Mutat. 2000;15:68–75. [PubMed]
29. Park J., Hwang S., Lee Y.S., Kim S.C., Lee D. SNP@Ethnos: A database of ethnically variant single-nucleotide polymorphisms. Nucleic Acids Res. 2007;35(Database issue):D711–D715. [PMC free article] [PubMed]
30. Sawyer S.L., Mukherjee N., Pakstis A.J., Feuk L., Kidd J.R., Brookes A.J., Kidd K.K. Linkage disequilibrium patterns vary substantially among populations. Eur. J. Hum. Genet. 2005;13:677–686. [PubMed]
31. Voight B.F., Kudaravalli S., Wen X., Pritchard J.K. A map of recent positive selection in the human genome. PLoS Biol. 2006;4:e72. [PMC free article] [PubMed]
32. ENCODE Project Consortium The ENCODE (ENCyclopedia Of DNA Elements) Project. Science. 2004;306:636–640. [PubMed]
33. Gabriel S.B., Schaffner S.F., Nguyen H., Moore J.M., Roy J., Blumenstiel B., Higgins J., DeFelice M., Lochner A., Faggart M. The structure of haplotype blocks in the human genome. Science. 2002;296:2225–2229. [PubMed]
34. Reich D.E., Gabriel S.B., Altshuler D. Quality and completeness of SNP databases. Nat. Genet. 2003;33:457–458. [PubMed]
35. Cargill M., Altshuler D., Ireland J., Sklar P., Ardlie K., Patil N., Shaw N., Lane C.R., Lim E.P., Kalyanaraman N. Characterization of single-nucleotide polymorphisms in coding regions of human genes. Nat. Genet. 1999;22:231–238. [PubMed]
36. Mooney S. Bioinformatics approaches and resources for single nucleotide polymorphism functional analysis. Brief. Bioinform. 2005;6:44–56. [PubMed]
37. Abushamaa A.M., Sporn T.A., Folz R.J. Oxidative stress and inflammation contribute to lung toxicity after a common breast cancer chemotherapy regimen. Am. J. Physiol. Lung Cell. Mol. Physiol. 2002;283:L336–L345. [PubMed]
38. Ramensky V., Bork P., Sunyaev S. Human non-synonymous SNPs: Server and survey. Nucleic Acids Res. 2002;30:3894–3900. [PMC free article] [PubMed]
39. Dagan T., Talmor Y., Graur D. Ratios of radical to conservative amino acid replacement are affected by mutational and compositional factors and may not be indicative of positive Darwinian selection. Mol. Biol. Evol. 2002;19:1022–1025. [PubMed]
40. Dupont W.D., Plummer W.D. Power and sample size calculations. A review and computer program. Control. Clin. Trials. 1990;11:116–128. [PubMed]
41. Ng P.C., Henikoff S. SIFT: Predicting amino acid changes that affect protein function. Nucleic Acids Res. 2003;31:3812–3814. [PMC free article] [PubMed]
42. Pritchard J.K. Are rare variants responsible for susceptibility to complex diseases? Am. J. Hum. Genet. 2001;69:124–137. [PMC free article] [PubMed]
43. Pritchard J.K., Cox N.J. The allelic architecture of human disease genes: Common disease-common variant…or not? Hum. Mol. Genet. 2002;11:2417–2423. [PubMed]
44. Reich D.E., Lander E.S. On the allelic spectrum of human disease. Trends Genet. 2001;17:502–510. [PubMed]
45. Kondrashov A.S. Direct estimates of human per nucleotide mutation rates at 20 loci causing Mendelian diseases. Hum. Mutat. 2003;21:12–27. [PubMed]
46. Nachman M.W., Crowell S.L. Estimate of the mutation rate per nucleotide in humans. Genetics. 2000;156:297–304. [PMC free article] [PubMed]
47. Peng B., Kimmel M. Simulations provide support for the common disease-common variant hypothesis. Genetics. 2007;175:763–776. [PMC free article] [PubMed]
48. Kryukov G.V., Pennacchio L.A., Sunyaev S.R. Most rare missense alleles are deleterious in humans: Implications for complex disease and association studies. Am. J. Hum. Genet. 2007;80:727–739. [PMC free article] [PubMed]
49. Kotowski I.K., Pertsemlidis A., Luke A., Cooper R.S., Vega G.L., Cohen J.C., Hobbs H.H. A spectrum of PCSK9 alleles contributes to plasma levels of low-density lipoprotein cholesterol. Am. J. Hum. Genet. 2006;78:410–422. [PMC free article] [PubMed]
50. Krawczak M., Cooper D.N. The human gene mutation database. Trends Genet. 1997;13:121–122. [PubMed]
51. Stenson P.D., Ball E.V., Mort M., Phillips A.D., Shiel J.A., Thomas N.S., Abeysinghe S., Krawczak M., Cooper D.N. Human Gene Mutation Database (HGMD): 2003 update. Hum. Mutat. 2003;21:577–581. [PubMed]
52. Crow J.F., Kimura M. Harper & Row; 1970. An Introduction to Population Genetics Theory.
53. Zhang X.S., Wang J., Hill W.G. Influence of dominance, leptokurtosis and pleiotropy of deleterious mutations on quantitative genetic variation at mutation-selection balance. Genetics. 2004;166:597–610. [PMC free article] [PubMed]
54. Fan J.B., Gehl D., Hsie L., Shen N., Lindblad-Toh K., Laviolette J.P., Robinson E., Lipshutz R., Wang D., Hudson T.J. Assessing DNA sequence variations in human ESTs in a phylogenetic context using high-density oligonucleotide arrays. Genomics. 2002;80:351–360. [PubMed]
55. Wicklow B.A., Ivanovich J.L., Plews M.M., Salo T.J., Noetzel M.J., Lueder G.T., Cartegni L., Kaback M.M., Sandhoff K., Steiner R.D. Severe subacute GM2 gangliosidosis caused by an apparently silent HEXA mutation (V324V) that results in aberrant splicing and reduced HEXA mRNA. Am. J. Med. Genet. A. 2004;127:158–166. [PubMed]
56. Xie J., Pabón D., Jayo A., Butta N., González-Manchón C. Type I Glanzmann thrombasthenia caused by an apparently silent beta3 mutation that results in aberrant splicing and reduced beta3 mRNA. Thromb. Haemost. 2005;93:897–903. [PubMed]
57. Pfarr N., Prawitt D., Kirschfink M., Schroff C., Knuf M., Habermehl P., Mannhardt W., Zepp F., Fairbrother W., Loos M. Linking C5 deficiency to an exonic splicing enhancer mutation. J. Immunol. 2005;174:4172–4177. [PubMed]
58. Denecke J., Kranz C., Kemming D., Koch H.G., Marquardt T. An activated 5′ cryptic splice site in the human ALG3 gene generates a premature termination codon insensitive to nonsense-mediated mRNA decay in a new case of congenital disorder of glycosylation type Id (CDG-Id) Hum. Mutat. 2004;23:477–486. [PubMed]
59. Carlini D.B., Genut J.E. Synonymous SNPs provide evidence for selective constraint on human exonic splicing enhancers. J. Mol. Evol. 2006;62:89–98. [PubMed]
60. Gorlov I.P., Kimmel M., Amos C.I. Strength of the purifying selection against different categories of the point mutations in the coding regions of the human genome. Hum. Mol. Genet. 2006;15:1143–1150. [PubMed]
61. Wong G.K., Yang Z., Passey D.A., Kibukawa M., Paddock M., Liu C.R., Bolund L., Yu J. A population threshold for functional polymorphisms. Genome Res. 2003;13:1873–1879. [PMC free article] [PubMed]
62. Wilson S.H., Olden K. The environmental genome project: Phase I and beyond. Mol. Interv. 2004;4:147–156. [PubMed]
63. Guengerich F.P. The Environmental Genome Project: Functional analysis of polymorphisms. Environ. Health Perspect. 1998;106:365–368. [PMC free article] [PubMed]
64. Taylor J.A., Xu Z.L., Kaplan N.L., Morris R.W. How well do HapMap haplotypes identify common haplotypes of genes? A comparison with haplotypes of 334 genes resequenced in the environmental genome project. Cancer Epidemiol. Biomarkers Prev. 2006;15:133–137. [PubMed]

Articles from American Journal of Human Genetics are provided here courtesy of American Society of Human Genetics
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • PubMed
    PubMed
    PubMed citations for these articles