Logo of gseBioMed CentralBiomed Central Web Sitesearchsubmit a manuscriptregisterthis articleGenetics, Selection, Evolution : GSEJournal Front Page
Genet Sel Evol. 2011; 43(1): 38.
Published online Nov 9, 2011. doi:  10.1186/1297-9686-43-38
PMCID: PMC3354418

Deregressed EBV as the response variable yield more reliable genomic predictions than traditional EBV in pure-bred pigs

Abstract

Background

Genomic selection can be implemented by a multi-step procedure, which requires a response variable and a statistical method. For pure-bred pigs, it was hypothesised that deregressed estimated breeding values (EBV) with the parent average removed as the response variable generate higher reliabilities of genomic breeding values than EBV, and that the normal, thick-tailed and mixture-distribution models yield similar reliabilities.

Methods

Reliabilities of genomic breeding values were estimated with EBV and deregressed EBV as response variables and under the three statistical methods, genomic BLUP, Bayesian Lasso and MIXTURE. The methods were examined by splitting data into a reference data set of 1375 genotyped animals that were performance tested before October 2008, and 536 genotyped validation animals that were performance tested after October 2008. The traits examined were daily gain and feed conversion ratio.

Results

Using deregressed EBV as the response variable yielded 18 to 39% higher reliabilities of the genomic breeding values than using EBV as the response variable. For daily gain, the increase in reliability due to deregression was significant and approximately 35%, whereas for feed conversion ratio it ranged between 18 and 39% and was significant only when MIXTURE was used. Genomic BLUP, Bayesian Lasso and MIXTURE had similar reliabilities.

Conclusions

Deregressed EBV is the preferred response variable, whereas the choice of statistical method is less critical for pure-bred pigs. The increase of 18 to 39% in reliability is worthwhile, since the reliabilities of the genomic breeding values directly affect the returns from genomic selection.

Background

Genomic selection in pure-bred pigs can be implemented using a multi-step procedure. Effects of dense genetic markers are estimated using a reference population and these effects are used to predict genomic breeding values (GBV) of selection candidates [1]. Implementing a multi-step procedure relies on two prerequisites: 1) a response variable that summarises the genetic information for reference animals, and 2) a statistical method that associates the response variable to the marker information. The choice of response variable and statistical method may well depend on the data structure. Therefore, the challenge is to find a suitable response variable and statistical method that can handle pure-bred pig data.

Pure-bred pig data often have low and varying reliabilities of estimated breeding values (EBV). Some Duroc pigs in the Danish breeding scheme only have their own records, others only offspring records, while some have both - depending on the trait. Furthermore, only a subset of pigs are genotyped and these pigs tend to be closely related. Therefore, a response variable and a statistical method capable of handling such data are needed.

Among the possible response variables, EBV and deregressed EBV are the most promising to date. Garrick et al. [2] showed, at least in theory, that deregressed EBV with the parent average removed (hereafter referred to as "deregressed EBV") yield more accurate genomic breeding values than EBV for two reasons. First, deregressed EBV as the response variable results in less double-counting compared to EBV, because the proposed deregressed EBV excludes ancestral information. If both an offspring and it's parent are genotyped, the degree of double-counting decreases when using deregressed EBV as the response variable. Second, using EBV as the response variable results in double shrinkage of the genomic breeding values, particularly when the reliabilities of the EBV are low. In dairy cattle, Guo et al. [3] compared the two response variables on simulated data and found that deregressed EBV yielded slightly lower reliabilities. However, their simulated data were characterized by low degrees of double-counting. It appears that most pure-bred pig data, with EBV with low reliabilities and high degrees of double-counting, compared to Guo et al. [3], are more in line with the theoretical expectations. Therefore, this suggests that deregressed EBV as the response variable may yield higher reliabilities of genomic breeding values than EBV.

Three types of statistical models have been widely used in the literature [1,4-14]. These are models that assume the marker effects to be normally distributed, models that assume a thick-tailed distribution of marker effects, and models that assume a mixture of two distributions. The performance of the different models is predominantly affected by the number of QTL, the marker density and the genetic relatedness of the population. However, there is no unambiguous evidence that one model will yield more accurate genomic breeding values for pure-bred pig data for two reasons. First, mixture-distribution models have shown promising results for some traits, in particular traits under weak selection, possibly because these traits can be influenced to a greater extent by a few large QTL [9,15]. Second, low genetic relatedness between reference animals and validation animals appears to favour mixture-distribution models [14], presumably because mixture-distribution models utilize linkage disequilibrium more efficiently than normal and thick tailed distribution models. Probably for the same reason, Hayes et al. [9] concluded that thick-tailed models and models assuming normality were equally good only when data from a single cattle breed were analysed. It appears that for pure-bred pig data, for which there is strong selection on traits of interest and high relatedness between genotyped animals, none of the models would be favoured over the others. Therefore, the models assuming mixture-distribution, thick tailed distribution and normal distribution might yield similar reliabilities of genomic breeding values.

In summary, we reasoned that 1) deregressed EBV as the response variable yields higher reliabilities of the GBV compared to EBV, and that 2) normal, thick-tailed and mixture-distribution models yield similar reliabilities. To test these hypotheses, the three models and the two response variables were assessed for the reliabilities by which they could predict GBV for the two traits, daily gain and feed conversion ratio, in Danish Duroc pigs.

Methods

Procedure

Two response variables, EBV and deregressed EBV, and three statistical methods with normal, thick-tailed, and mixture-distributions, were assessed for their reliability to predict GBV for daily gain and feed conversion ratio in Danish Duroc pigs. The reliabilities were computed by splitting the genotyped animals into 1375 reference animals that were performance tested before October 2008 and 536 validation animals that were performance tested after October 2008 (further details are given below).

Data

The Duroc pigs were part of the genetic evaluation system in Denmark. All data were supplied by the Danish Agriculture and Food Council, Pig Research Centre.

Genotyping

A total of 1911 Danish Duroc pigs were genotyped using the Illumina PorcineSNP60 BeadChip (Illumina, San Diego, CA). A total of 26142 SNP markers and each of the animals met the following requirements. Each animal had a call rate greater than 0.95. Each marker was mapped to an autosome, had a minor-allele frequency greater than 0.05, a call-frequency score greater than 0.95, and a heterozygote frequency that did not deviate from Hardy-Weinberg expectations by more than 14pq, where p and q are the allele frequencies at the marker. The 14pq corresponds to a 1/4 standard-deviation unit when assuming a binomial distribution of alleles. For each animal and SNP-genotype combination, the GenCall score was greater than 0.65. Genotypes less than 0.65 were defined as missing. Animals with missing genotypes were allocated the population mean for the missing markers.

Performance test and pedigree

Daily gain (g/day) and feed conversion ratio (feed units/kg gain) were observed in the interval 30-100 kg live weight. Recordings were performed in the period 1992 to 2010. The pedigrees of the animals were traced back to 1984, and consisted of 345686 and 52537 animals for daily gain and feed conversion ratio, respectively. Both pedigrees included 373 unknown parents (base animals).

The reference data consisted of available records on October 1st 2008 and included 313068 and 23628 measurements of daily gain and feed conversion ratio, respectively. All of the 1375 genotyped animals in the reference data had their own records for daily gain, whereas only 898 genotyped animals had their own records for feed conversion ratio. There were 680 genotyped animals in the reference data that had more than four offspring records for daily gain and 633 genotyped animals that had more than four offspring records for feed conversion ratio.

The full data consisted of records available until May 2010 and included the records from the reference data and additional 32618 and 3344 measurements of daily gain and feed conversion ratio, respectively. All 536 genotyped validation animals were phenotyped for both daily gain and feed conversion ratio after October 1st, 2008.

Response variables

The EBV were calculated for daily gain and feed conversion ratio by single-trait animal models, and deregressed EBV were calculated by applying the procedure proposed by Garrick et al. [2].

The single-trait animal models were based on the routine evaluation model, with the regression effect of start weight, fixed effect of herd-week-section, random effect of pen and a random additive genetic effect. The model fitted to daily gain also included a fixed effect for sex and a random effect for birth litter. The variance components were estimated using REML. The heritabilities for daily gain and feed conversion ratio were 0.27 and 0.21. The 1375 reference animals had mean reliabilities for EBV of 0.62 (sd = 0.18) and 0.36 (sd = 0.12) for daily gain and feed conversion ratio, respectively. The software DMU [16] was used to estimate variances and predict the breeding values. Single-trait animal models were used because preliminary analyses showed that predictions of GBV were not improved with a bivariate model.

The deregression procedure of Garrick et al. [2] adjusts for ancestral information, such that the deregressed EBV only contains their own and the descendant's information on each animal. The deregression also eliminates shrinkage contained in the EBV, and therefore deregressed EBV behave as though they were observations with a heritability equal to the reliability of the deregressed EBV (reliabilities computed as in Garrick et al. [2]). Deregressed EBV have unequal variances and should be used in a weighted analysis. To ensure the quality of deregressed EBV, only animals with a deregressed reliability above 0.05 were included in the analysis. This resulted in 35 animals being removed from the analyses with deregressed EBV for feed conversion ratio.

Statistical methods to estimate marker effects

The marker effects were estimated by fitting linear, additive models to the response variables. The model was:

Y=μ+Xβ+ε,
(1)

where Y is a n × 1 vector of responses, with n being the number of reference animals. The mean is denoted μ, which is a scalar. The coefficient matrix for the markers, X, assume the values -1, 0 or 1, and has dimension n × p, where p is the number of markers. The vector of marker effects is denoted β and has dimension p × 1, whereas the vector of residuals, ε, has dimension n × 1, and is distributed ε~N(0,σε2W), where W is a diagonal matrix with elements w1, ..., wn.

For the analyses with EBV as the response variable, w1 = . . . = wn = 1, whereas for the analyses with deregressed EBV as the response variable, a weighted analysis was performed according to Garrick et al. [2]. The weight for the ith animal was wi=(1-h2)[(c+(1-ri2)ri2)h2], where c, the part of the genetic variance not explained by markers, was assumed to be 0.1, h2 was the heritability of the trait, and ri2 was the reliability of the deregressed EBV of the ith animal.

The statistical models studied differ by the distributional assumptions made on β, and by the shortness of presentation they are referred to by the corresponding inferential procedure. A brief description of each method is provided in the following.

GBLUP

The model for genomic BLUP (GBLUP) can be described as in equation (1), where the vector of marker effects are distributed β~N(0,σv2I). Alternatively, the model could be written as Y = μ + g + ε, where the genetic effects g~N(0,σv2XXT), i.e. a model with a genomic relationship matrix [7]. In this study, the latter form was used, where XXT for computational reasons was replaced by (X - P)(X - P)T with P containing the row means of X across animals. Variance components were estimated using REML, and the GBV were BLUP solutions from the mixed model equations. Software DMU [16] was used to fit GBLUP models.

Bayesian Lasso

The model for Bayesian Lasso assumes that the marker effects follow a double exponential distribution [12,17], which is a thick-tailed distribution. This is also referred to as a Laplace distribution. Thus, β ~ Laplace(0, λI) where λ is a scaling parameter. The MCMC software BayZ [18] was used for estimation. A Gibbs sampler was applied and a total of 40000 iterations of sampling was performed, with a burn-in of 10000 iterations.

MIXTURE

The model for MIXTURE assumes that the marker effects follow a mixture of two normal distributions with similar characteristics as the mixture model in Meuwissen [19]. The distributional assumption can be described as: β~π0N(0,σβ02I)+π1N(0,σβ12I), where π0 = 0.9, π1 = 0.1, and σβ02=0.01σβ12. The MCMC software BayZ [18] was used for estimation. A Metropolis-Hastings sampler was applied with a total of 40000 iterations of sampling and a burn-in of 10000 iterations.

Evaluation criterion

The predictive ability of each method was evaluated by the reliability of the GBV on the genotyped validation animals. To avoid use of overlapping information between the reference and validation animals [20], we based the validation on the own phenotype of genotyped validation animals, adjusted for fixed effects and non-genetic random effects. The computation of adjusted phenotypes for genotyped validation animals was based on the full data, and the adjusted phenotypes were the sum of EBV and the estimated residual errors (ĝ+ê). The reliability of the GBV, r(GBV)2, is the squared correlation r(GBV,g)2 between GBV and the vector of genetic effects, g, for genotyped validation animals. Using that the vector of residual errors, e, for these animals is independent of the GBV, and that g+eĝ+ê, we obtain a formula for the reliability of the GBV

r(GBV)2=r(GBV,g)2=r(GBV,g+e)2ωr(GBV,ĝ+ê)2ω,
(2)

where  ĝ  is the vector of EBV for validation animals,  ê  is the estimated residual vector, ω=(σe2+σa2)σa2, σe2 is the residual variance and σa2 is the additive genetic variance of the trait. The formula above for the reliability is similar to other formulas shown in the literature [14,21].

A test was performed to investigate whether the reliabilities of the GBV from the two methods were significantly different from each other within each trait. The test hypothesis was that the reliability of GBV from method A was equal to the reliability of GBV from method B, i.e. H0:r(GBVA,ĝ+ê)2ω=r(GBVB,ĝ+ê)2ω. Since ω is simply a constant for each trait, the test hypothesis can be reduced to H0:r(GBVA,ĝ+ê)=r(GBVB,ĝ+ê). However, ĝ+ê appears in both correlations and the two correlations are not independent within each trait. This issue of testing equality of correlated correlations was addressed by the Hotelling-Williams t-test [22,23], which was applied to each trait with a confidence level of 5%.

Results

Response variables

The response variable, deregressed EBV, resulted in higher reliabilities than EBV for both feed conversion ratio and daily gain (Table (Table1).1). For daily gain, the reliabilities were on average 35% higher for deregressed EBV with GBLUP, Bayesian Lasso and MIXTURE. The reliabilities for daily gain increased from approximately 0.25 to 0.34. For feed conversion ratio, the reliabilities were on average 18% higher for deregressed EBV for GBLUP and Bayesian Lasso, which approached significance (p-values between 0.05 and 0.11). Use of deregressed EBV instead of EBV increased reliabilities from approximately 0.16 to 0.19 for GBLUP and Bayesian Lasso. For the method MIXTURE, the deregressed EBV yielded 39% higher reliabilities than EBV, increasing reliabilities from 0.15 to 0.20.

Table 1
Reliabilities of GBV, r(GBV)2, based on the three statistical methods and the two response variables for daily gain and feed conversion ratio.

Statistical methods

The three statistical methods, GBLUP, Bayesian Lasso and MIXTURE did not yield different reliabilities (Table (Table1).1). For daily gain, the reliabilities ranged from 0.33 to 0.34 for deregressed EBV and from 0.25 to 0.26 for EBV. For feed conversion ratio, the reliabilities ranged from 0.19 to 0.20 for deregressed EBV and from 0.15 to 0.16 for EBV.

Discussion

The hypothesis that deregressed EBV yield higher reliabilities than EBV was supported. The increase in reliability of 18 to 39% for Duroc pigs is worthwhile, since the reliabilities of the genomic breeding values directly affect the returns from genomic selection. The hypothesis that the different statistical methods yielded similar reliabilities was also supported. Therefore, we believe that when applying genomic selection within a pure-bred pig population, deregressed EBV is the preferred response variable, while the choice of the statistical method is less critical.

The fact that the increase in reliabilities due to deregression was so large was surprising, since results reported by Guo et al. [3] suggested that EBV is the more suitable response variable. There are three possible reasons. In our study less information was available for the reference animals compared to Guo et al. [3], which implies that the expected double shrinkage was increased when using EBV as the response variable. The amount of information was more heterogeneous among the reference animals, and this favours deregressed EBV. The heterogeneity in amount of information was larger for daily gain than for feed conversion ratio, which also explains why the effect of deregression was larger for this trait. The combination of less information per reference animal and several parent-offspring relationships within the reference population increases double-counting further. If a genotyped parent has only one offspring, and this offspring is genotyped, the degree of double-counting is much larger than if both the parent and the offspring have many ungenotyped offspring. So, the combination of sire-son relationships with a low and heterogeneous amount of information in the reference data seems to favour deregressed EBV over EBV.

There are two reasons why the three statistical methods performed equally well. First, the traits have been subject to strong selection, which suggests that MIXTURE would not have an advantage over the other methods [9,15]. Second, we considered only one breed, in which the genotyped animals were closely related, implying that Bayesian Lasso and MIXTURE, which utilize linkage disequilibrium more efficiently, have no advantage over GBLUP [9,14]. Thus, the small difference between the statistical methods is caused by the pure-bred pig data with its high relatedness between genotyped animals and traits that have been subject to strong selection.

The results from this study may only partly apply to other species, traits and data structures. The positive effect of using deregressed EBV as the response variable will depend on the degree of double-counting, and the amount and heterogeneity of information for genotyped animals. However, since the deregressed EBV proposed by Garrick et al. [2] are theoretically more appropriate than EBV, we believe that deregressed EBV will be advantageous in most circumstances. In contrast, the conclusion about equal performances of the statistical methods is not generic. For traits controlled by large QTL, data from multiple breeds or distantly related genotyped animals, or when using a denser marker panel, MIXTURE and Bayesian Lasso could outperform GBLUP. The reason for this is that MIXTURE and Bayesian Lasso utilize linkage disequilibrium more efficiently. We believe that the positive effect of using deregressed EBV applies to most situations, whereas choosing the best statistical method is more sensitive to the particular situation.

The applied multi-step method has the benefit of being readily applicable in most breeding schemes. EBV with reliabilities are easily predicted, and can then be used to calculate deregressed EBV [2]. The deregression procedure itself is very simple and only requires little computing power. The association of marker effects with the response variable by GBLUP is relatively simple, since GBLUP is implemented in common breeding value estimation software. Bayesian Lasso and MIXTURE are available in various MCMC software, although particularly MIXTURE is sensitive to the prior information given, which makes the use of these methods less appealing. A final step, which has not been examined in the present study, would be to combine the genomic breeding values with the traditional pedigree information [24,25].

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

TO carried out most calculations and drafted the main parts of the manuscript, OFC created the EBVs from the animal model, made the GBLUP predictions, wrote significant parts of the manuscript and guided on statistical issues. MH guided in the structure of the manuscript, constructed the SNP data and wrote parts of the manuscript. BN, GS and PM helped conceive of the study and guided in the statistical analyses. All authors read and approved the final manuscript.

Acknowledgements

Two anonymous reviewers are thanked for their corrections. The authors gratefully acknowledge Luc Janss for help on the use of BayZ software, and David Lindsay for guidance on the structure of the paper. The work was funded by the Danish Ministry of Food, Agriculture and Fisheries (grant no. 3412-09-02505).

References

  • Meuwissen THE, Hayes BJ, Goddard ME. Prediction of total genetic value using genome-wide dense marker maps. Genetics. 2001;157:1819–1829. [PMC free article] [PubMed]
  • Garrick DJ, Taylor JF, Fernando RL. Deregressing estimated breeding values and weighting information for genomic regression analyses. Genet Sel Evol. 2009;41:55. doi: 10.1186/1297-9686-41-55. [PMC free article] [PubMed] [Cross Ref]
  • Guo G, Lund MS, Zhang Y, Su G. Comparison between genomic predictions using daughter yield deviation and conventional estimated breeding value as response variables. J Anim Breed Genet. 2010;127:423–432. doi: 10.1111/j.1439-0388.2010.00878.x. [PubMed] [Cross Ref]
  • Xu S. Estimating polygenic effects using markers of the entire genome. Genetics. 2003;163:789–801. [PMC free article] [PubMed]
  • Roso VM, Schenkel FS, Miller SP, Schaeffer LR. Estimation of genetic effects in the presence of multicollinearity in multibreed beef cattle evaluation. J Anim Sci. 2005;83:1788–1800. [PubMed]
  • Gianola D, Fernando RL, Stella A. Genomic-assisted prediction of genetic value with semiparametric procedures. Genetics. 2006;173:1761–1776. doi: 10.1534/genetics.105.049510. [PMC free article] [PubMed] [Cross Ref]
  • VanRaden PM. Efficient methods to compute genomic predictions. J Dairy Sci. 2008;91:4414–4423. doi: 10.3168/jds.2007-0980. [PubMed] [Cross Ref]
  • Pimentel E, Konig S, Schenkel F, Simianer H. Comparison of statistical procedures for estimating polygenic effects using dense genome-wide marker data. BMC Proceedings. 2009;3(Suppl 1):S12. doi: 10.1186/1753-6561-3-s1-s12. [PMC free article] [PubMed] [Cross Ref]
  • Hayes BJ, Bowman PJ, Chamberlain AC, Verbyla K, Goddard ME. Accuracy of genomic breeding values in multi-breed dairy cattle populations. Genet Sel Evol. 2009;41:51. doi: 10.1186/1297-9686-41-51. [PMC free article] [PubMed] [Cross Ref]
  • Su G, Guldbrandtsen B, Gregersen VR, Lund MS. Preliminary investigation on reliability of genomic estimated breeding values in the Danish Holstein population. J Dairy Sci. 2010;93:1175–1183. doi: 10.3168/jds.2009-2192. [PubMed] [Cross Ref]
  • Cole JB, VanRaden PM, O'Connell JR, Van Tassell CP, Sonstegard TS, Schnabel RD, Taylor JF, Wiggans GR. Distribution and location of genetic effects for dairy traits. J Dairy Sci. 2009;92:2931–2946. doi: 10.3168/jds.2008-1762. [PubMed] [Cross Ref]
  • de los Campos G, Naya H, Gianola D, Crossa J, Legarra A, Manfredi E, Weigel K, Cotes JM. Predicting quantitative traits with regression models for dense molecular markers and pedigree. Genetics. 2009;182:375–385. doi: 10.1534/genetics.109.101501. [PMC free article] [PubMed] [Cross Ref]
  • Gianola D, de los Campos G, Hill W, Manfredi E, Fernando R. Additive genetic variability and the Bayesian alphabet. Genetics. 2009;183:347–363. doi: 10.1534/genetics.109.103952. [PMC free article] [PubMed] [Cross Ref]
  • Habier D, Tetens J, Seefried FR, Lichtner P, Thaller G. The impact of genetic relationship information on genomic breeding values in German Holstein cattle. Genet Sel Evol. 2010;42:5. doi: 10.1186/1297-9686-42-5. [PMC free article] [PubMed] [Cross Ref]
  • Goddard ME, Hayes BJ, Meuwissen THE. In: Proceedings of the 9th World Congress on Genetics Applied to Livestock Production. 1-6 August, 2010, Leipzig. German Society for Animal Science, editor. 2010. Genomic selection in farm animal species - lessons learnt and future perspectives. paper 0701 E[ http://www.kongressband.de/wcgalp2010/assets/pdf/0701.pdf]
  • Madsen P, Jensen J. A users guide to DMU, version 6, release 4.7. Manual, Faculty of agricultural science, University of Aarhus; 2008.
  • Yi N, Xu S. Bayesian LASSO for quantitative trait loci mapping. Genetics. 2008;179:1045. doi: 10.1534/genetics.107.085589. [PMC free article] [PubMed] [Cross Ref]
  • Janss LLG. BayZ manual version 2.02. 2010. http://bayz.biz/
  • Meuwissen T. Accuracy of breeding values of 'unrelated' individuals predicted by dense SNP genotyping. Genet Sel Evol. 2009;41:35. doi: 10.1186/1297-9686-41-35. [PMC free article] [PubMed] [Cross Ref]
  • Amer PR, Banos G. Implications of avoiding overlap between training and testing data sets when evaluating genomic predictions of genetic merit. J Dairy Sci. 2010;93:3320–3330. doi: 10.3168/jds.2009-2845. [PubMed] [Cross Ref]
  • Legarra A, Robert-Granié C, Manfredi E, Elsen JM. Performance of genomic selection in mice. Genetics. 2008;180:611–618. doi: 10.1534/genetics.108.088575. [PMC free article] [PubMed] [Cross Ref]
  • Dunn OJ, Clark V. Comparison of tests of the equality of dependent correlation coefficients. J Amer Statistical Assoc. 1971;66:904–908. doi: 10.2307/2284252. [Cross Ref]
  • Revelle W. psych. Procedures for Psychological, Psychometric, and Personality Research. Northwestern University, Evanston, Illinois; 2010. http://personality-project.org/r/psych.manual.pdf [R package version 1.0-90]
  • Ducrocq V, Liu Z. Combining genomic and classical information in national BLUP evaluations. Interbull Bull. 2009;40:172–177.
  • VanRaden PM, Van Tassel CP, Wiggans GR, Sonstegard TS, Schnabel RD, Taylor JF, Schenkel FS. Invited review: reliability of genomic predictions for North American Holstein bulls. J Dairy Sci. 2009;92:16–24. doi: 10.3168/jds.2008-1514. [PubMed] [Cross Ref]

Articles from Genetics, Selection, Evolution : GSE are provided here courtesy of BioMed Central
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • MedGen
    MedGen
    Related information in MedGen
  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...