If the promise of high-throughput population-genomic data is to be fulfilled, a number of analytical challenges will need to be met. High-throughput sequencing strategies lead to the unbalanced sampling of parental alleles (in nonselfing diploid species) and also introduce errors that can result in uncertainty about individual genotypic states (Johnson and Slatkin 2007; Hellmann et al. 2008; Lynch 2008; Jiang et al. 2009). Unless they are statistically accounted for, both types of problems will lead to elevated estimates of low-frequency alleles, especially singletons, potentially leading to exaggerated conclusions about purifying selection. Such aberrations are of concern because, for economic reasons, many plans for surveying large numbers of genomes envision the pursuit of relatively low (~2×) depth-of-coverage sequence for each individual, a sampling design that will magnify the variance of parental-allele sampling, while also rendering the error-correction problem particularly difficult (Lynch 2008).

Simple ad hoc strategies for dealing with light-coverage data sets include the restriction of site-specific analyses to individuals that by chance have adequate coverage for a high probability of biallelic sampling (e.g., Jiang et al. 2009). However, such an approach can result in the discarding of large amounts of data, increasing the lower bound on the frequency of alleles that can be determined (as a result of the reduction in sample size). In addition, although methods exist for flagging and discarding low-quality reads (Ewing and Green 1998; Ewing et al. 1998; Huse et al. 2007), such treatment is unable to eliminate other sources of error. With low-coverage data, singleton reads cannot be discarded without inducing significant bias, as such treatment will also lead to the censoring of true heterozygotes with unbalanced parental-allele sampling.

Thus, there is a clear need for novel methods for efficiently estimating nucleotide frequencies at individual sites, not only to maximize the information that can be extracted from population-level samples, but also to guide in the design of such surveys. In the latter context, questions remain as to the relative merits of increasing the number of individuals sampled vs. the depth of sequence coverage per individual, e.g., a 1000-genome project involving 2× coverage per individual, as opposed to a 500-genome project with 4× coverage, etc. Because many genetic analyses rely on low-frequency markers (SNPs), there is also a need to know the limits to our ability to estimate the frequencies of rare alleles, particularly in the face of error rates of comparable or even higher levels.

For the classical situation in which the genotypes of N individuals have been determined without error, assuming a population in Hardy–Weinberg equilibrium, the likelihood that the frequency of the leading allele (denoted as 1) is equal to p is simplywhere N₁₁, N₁₂, and N₂₂ are the numbers of times the three genotypes appear in the sample, and K is a constant that accounts for the multiplicity of orderings of individuals within the sample (which has no bearing on the likelihood analysis). The ML estimate of p, which maximizes L(p), has the well-known analytical solution (which also applies in the face of Hardy–Weinberg deviations) (Weir 1996).

Unfortunately, uncertainty in the identity of individual genotypes, resulting from incomplete sampling and erroneous reads, substantially complicates the situation in a random-sequencing project, as there are no longer just three categories of individuals. Rather, each site in each individual will be characterized by a specific read-frequency array (the number of reads for A, C, G, and T). This necessitates that the full likelihood be broken up into two components, the first associated with the sampling of reads within individuals and the second associated with the sampling of individual genotypes (as above). Assuming no more than two alleles per site, the data can then be further condensed as the analysis proceeds.

For example, letting 1 and 2 denote the putative major and minor nucleotides at a site in the population (with respective frequencies above and below 0.5) and 3 denote the error bin containing incorrect reads to the remaining two nucleotides, for any candidate major/minor nucleotide combination, each individual surveyed at the site can be represented by an array (n₁, n₂, n₃), where the three entries denote the numbers of reads at this site that coincide with the candidate major nucleotide, the minor nucleotide, and an erroneous read. Given the observed reads at the site for N individuals, our goal is to identify the major/minor nucleotides at the site and to estimate their frequencies. There are 12 possible arrangements of major and minor nucleotides, A/C, A/G,…, G/T, and the likelihoods of each of these alternatives must be considered sequentially.

We again start with the assumption of a population in Hardy–Weinberg equilibrium and further assume a homogeneous error distribution, such that each nucleotide is equally likely to be recorded erroneously as any other nucleotide with probability /3 (the total error rate per site being ). The composite-likelihood function for a particular read configuration is the sum of three terms, each the product of the probability of a particular genotype and the probability of the read configuration conditional on being that genotype,The first two terms in this expression denote the contributions to the likelihood assuming the genotype is homozygous for the major or minor alleles, with the functions _e(x, y; n, /3, 2/3) denoting joint probabilities of x errors to the alternative allele and y errors to the remaining two nonexistent alleles, assumed here to follow a trinomial distribution with n = n₁ + n₂ + n₃. The third term accounts for heterozygotes, with _e(n₃; n, 2/3) being the probability of n₃ errors to non-1/2 nucleotides, with the total error rate being reduced by one-third to discount internal (unobservable) errors, and p(n₁; n₁ + n₂, 0.5) being the probability of sampling n₁ copies of the major allele among the remaining (n − n₃) reads consistent with the designated genotype (Lynch 2008).

It is informative to start with the simple situation in which each individual is sequenced to the same depth of coverage (n), in which case the log likelihood for the entire data set, conditional on a particular candidate major/minor nucleotide pair, major-allele frequency (p), and error rate (), iswhere the unit of analysis is N(n₁, n₂, n₃), the number of individuals sampled with read configuration (n₁, n₂, n₃), and the summation is over all observed configurations. For sites with n× coverage, there are 1 + [n(n + 3)/2] possible read-array configurations for each major/minor nucleotide combination satisfying n = n₁ + n₂ + n₃. To obtain the ML solution, this expression needs to be evaluated over all possible combinations of major and minor nucleotides to determine the joint combination of major/minor allele identities, major-allele frequency (p), and error rate () that maximizes L.

To evaluate the behavior of Equations 2 and 3, stochastic simulations were performed by drawing random samples of N individuals from populations in Hardy–Weinberg equilibrium, with a constant sequence coverage per site n. Errors were randomly assigned to each read with probability , such that each nucleotide had a probability of being erroneously recorded as any other with probability /3. A relatively high error rate of = 0.01 was assumed to evaluate the situation under the worst possible conditions. The ML estimates for each simulation were obtained by thorough grid searches of the full parameter space for p and for each possible combination of major and minor alleles, for 250 to 500 replications.

The results of these analyses indicate that, provided the coverage is >1×, the proposed method yields estimates of p that are essentially unbiased, with the variance among replicate samples rapidly approaching the expectation based on individual sampling alone, p(1 − p)/(2N), once the coverage exceeds 4× (Figure 1). Small downward biases in the frequency estimates for rare alleles can arise when the sample size is low enough to cause rare-allele sampling to be extremely sporadic and the coverage is also low enough to cause sporadic sampling of errors. The problem is most severe with sequences with 1× coverage, as there is then no information within individuals on heterozygosity. However, even with 2× coverage, this bias is minor enough to be of little concern in most applications, as when it is likely to be of quantitative importance, it is also overwhelmed by the sampling variance. Moreover, as is demonstrated below, under the usual situation of variable coverage, even 1× sites are less problematic, as they supplement the information from sites with higher coverage.

The remaining issue is the extension of the above approach to sites with variable coverage among individuals. In this case, the absolute probabilities of the various configuration arrays are now functions of the probabilities of the various coverage levels, but the latter influence only the arbitrary constant in the log likelihood, so the approaches outlined above can be used without further modification, other than the summation over all configurations at all coverage levels in Equation 3. Evaluation of this procedure by computer simulation, assuming a Poisson distribution of read numbers across individuals and including 1×-covered sites in the analysis, yielded no substantive differences from the patterns illustrated in Figure 1. The estimator is essentially unbiased over all estimable allele frequencies (minor-allele frequencies >1/2N), and the sampling variance of the estimate asymptotically approaches p(1 − p)/(2N) at high coverages.

We may now inquire as to the optimal sampling strategy for accurate allele-frequency estimates under a fixed sequencing budget, which is assumed here to be proportional to T = Nn, the expected number of bases sequenced per site in the sample (with N individuals, each sequenced to average depth of coverage n). This effort function assumes that the bulk of the cost of data acquisition involves sequencing rather than the preparation/procurement of individual samples. Low coverage will magnify the problem of sampling variance within individuals, while also resulting in a fraction of individuals that are entirely lacking in observations at individual sites (~e⁻ⁿ under a Poisson distribution of coverages). On the other hand, under the constraint of constant Nn, increased sequencing depth per individual will result in a reduction in the number of individuals that can be assayed, thereby reducing the ability to detect rare alleles.

Two primary advantages of the proposed method are (1) its use of the full data set, which eliminates the need for arbitrary decisions on cutoffs for adequate coverage, and (2) its ability to internally separate the incidence of sequence errors from true variation, which eliminates the necessity of relying on external measures of the read-error rate that do not incorporate all sources of error. As the error rates assumed in a number of the simulations in this study are presumably near the upper end of what will occur in ultra-high-throughput methods now under development [although likely still too low in the case of ancient DNA samples (Briggs et al. 2007; Gilbert et al. 2008)], the fact that the estimates are nearly unbiased with asymptotic sampling variances defined by the number of individuals alone suggests that the ML method provides an optimal solution to the ascertainment of frequencies of both common and rare alleles, just as it does in the conventional situation where genotypes are assumed to be known without error (Weir 1996). Moreover, the fact that the proposed approach provides joint estimates of p and on a site-by-site basis provides a potentially strong advantage in situations where the error rate might be heterogeneous, e.g., dependent on the total sequence context.

With this methodology in hand, it should be possible to ascertain the form of the site-frequency spectrum for broad classes of genomic positions down to a very fine level. Indeed, provided the depth of coverage per site is sufficiently high, the proposed method has the potential to yield accurate allele-frequency estimates even when p is smaller than the error rate, as is often the case with disease-associated alleles. Thus, with large enough sample sizes of individuals, there are no significant barriers to estimating allele frequencies .

The ML allele-frequency estimator extends previous methods for estimating nucleotide heterozygosity (π) from high-throughput data from one or a small number of individuals (Johnson and Slatkin 2007; Hellmann et al. 2008; Lynch 2008; Jiang et al. 2009). For a given genomic region sequenced from multiple individuals without error, π is generally estimated as averaged over all sites, the terms containing 2N correcting for the bias associated with individual sampling, which induces variance in the estimate of p equal to p(1 − p)/(2N) (Nei 1978). It can be shown that when sequence errors are present, the sampling variance of is elevated to ~[1 + (4/3)²]p(1 − p)/(2N) + (1 − 4p)²/(3Nn), but with and , heterozygosities per nucleotide site will still be adequately estimated as .

A number of variants of the model assumed herein can be readily implemented. For example, it is straightforward to allow for non-Hardy–Weinberg conditions. For two alleles 1 and 2, this simply requires the incorporation of two unknown genotype frequencies (P₁₁ and P₁₂), with the third being constrained to be (1 − P₁₁ − P₁₂) and hence the estimation of just one additional unknown (Weir 1996). A simple likelihood-ratio test, contrasting the fit from the full model with that under Hardy–Weinberg assumptions, can then be used to evaluate whether sites are in Hardy–Weinberg equilibrium. By a similar extension, it is possible to test for interpopulation divergence by evaluating whether the likelihood of the full data set for multiple population samples is significantly improved by allowing for population-specific estimates of allele frequencies.

The assumption of a homogeneous site-specific error process can also be relaxed by allowing for unique error rates for any major/minor allele combination. Although there are 12 possible types of single-base substitution errors, for any particular major/minor nucleotide combination (under the two-allele model), a maximum of 4 are relevant to the specific likelihood expression: the reciprocal errors between the major and minor alleles and the movement of each of the two true nucleotides into the error bin. Again, the necessity of these types of embellishments can be evaluated by likelihood-ratio tests.

The overall analysis indicates that unless there is a need for specific information on each individual, from the standpoint of procuring accurate population estimates of allelic frequencies (and other associated parameters), there is little advantage in pursuing high-coverage data per individual. Indeed, given a fixed sequencing budget, an overemphasis on depth of coverage will result in a reduction in the accuracy of population-level parameters, as the optimal design for estimating most allele frequencies employs an average coverage of <1×, even though this magnifies the fraction of individuals with no data. Of course, if the ultimate goal is the procurement of accurate genotypic profiles for individuals (as, for example, in association-mapping studies), there is no substitute for relatively high coverages.