- We are sorry, but NCBI web applications do not support your browser and may not function properly. More information

- Journal List
- NIHPA Author Manuscripts
- PMC3091623

# Genomic-Enabled Prediction Based on Molecular Markers and Pedigree Using the Bayesian Linear Regression Package in R

## Abstract

The availability of dense molecular markers has made possible the use of genomic selection in plant and animal breeding. However, models for genomic selection pose several computational and statistical challenges and require specialized computer programs, not always available to the end user and not implemented in standard statistical software yet. The R-package BLR (Bayesian Linear Regression) implements several statistical procedures (e.g., Bayesian Ridge Regression, Bayesian LASSO) in a unifi ed framework that allows including marker genotypes and pedigree data jointly. This article describes the classes of models implemented in the BLR package and illustrates their use through examples. Some challenges faced when applying genomic-enabled selection, such as model choice, evaluation of predictive ability through cross-validation, and choice of hyper-parameters, are also addressed.

Prediction of genetic values is a central problem in quantitative genetics. Accurate predictions of genetic values of genotypes whose phenotypes are yet to be observed (e.g., newly developed lines) are needed to attain rapid genetic progress and to reduce phenotyping costs (e.g., Bernardo and Yu, 2007). Over many decades, such predictions have been obtained using phenotypic and family data, the latter usually represented by a pedigree. However, pedigree-based models do not account for Mendelian segregation, a term that in an additive model and in the absence of inbreeding explains as much as one half of the genetic variance. This sets an upper limit on the accuracy of estimates of genetic values of individuals without progeny. Dense molecular markers (MM) are now available in the genome of humans and of many plant and animal species. Unlike pedigree data, MM allow tracing back Mendelian segregation events at many points along the genome. Potentially, this information can be used to improve the accuracy of estimates of genetic values of newly developed lines.

Following the ground-breaking contribution of Meuwissen et al. (2001), genomic selection (GS) has gained ground in plant and animal breeding (e.g., Bernardo and Yu, 2007; Hayes et al., 2009; VanRaden et al., 2009; de los Campos et al., 2009; Crossa et al., 2010). In practice, implementing GS involves analyzing large amounts of phenotypic and MM data and requires specialized computer programs. The main purpose of this article is to show how the R-package (R Development Core Team, 2009) BLR (Bayesian Linear Regression, de los Campos and Pérez, 2010) can be used to implement several models for GS. A first version of the algorithms and the R-code was presented in de los Campos et al. (2009). The package was developed further and its performance was significantly improved by the first two authors of this article; the package and data set are available from the R website (http://www.r-project.org; verified 29 July 2010). We provide a brief overview of parametric models for GS and describe the type of models implemented in BLR. We show two applications that illustrate the use of the package and several features of the models implemented in it.

## Parametric Models for Genomic Selection

In parametric models for GS (e.g., Meuwissen et al., 2001), phenotypic outcomes, *y _{i}* (

*i*= 1,…,

*n*), are regressed on marker covariates

*x*(

_{ij}*j*= 1,…,

*p*) using a linear model of the form ${y}_{i}={\sum}_{j=1}^{p}{x}_{ij}{\beta}_{j}+{\epsilon}_{i}$, where β

*is the regression of*

_{j}*y*on the

_{i}*j*th marker covariate and

**ε**

*is a model residual. In matrix notation, the regression model is expressed as*

_{i}**y**=

**Xβ**+

**ε**, where

**y**= {

*y*},

_{i}**X**= {

*x*},

_{ij}**β**= {β

_{j}} and

**ε**= {ε

_{i}}. The number of molecular markers (

*p*) is usually larger than the number of observations (

*n*) and, because of this, estimation of marker effects via multiple regression by ordinary least squares (OLS) is not feasible. Instead, penalized estimation methods such as ridge regression (RR, Hoerl and Kennard, 1970), or the Least Absolute Shrinkage and Selection Operator (LASSO; Tibshirani, 1996) or Bayesian methods such as those of Meuwissen et al. (2001) or the Bayesian LASSO (BL) of Park and Casella (2008) (e.g., Yi and Xu, 2008; de los Campos et al., 2009) can be used to estimate marker effects.

In RR, estimates of the effects of MM are obtained as the solution to the following optimization problem:

Here, $\stackrel{~}{\lambda}\ge 0$ is a regularization parameter controlling the trade-offs between goodness of fit measured by the residual sum of squares, ${\sum}_{i=1}^{n}{\left({y}_{i}-{{\mathbf{x}}_{i}}^{\prime}\mathbf{\beta}\right)}^{2}$, and model complexity measured by the sum of squared marker effects (${\sum}_{j=1}^{p}{\beta}_{j}^{2}$). The first order conditions of the above optimization problem are satisfied by $\left[{\mathbf{X}}^{\prime}\mathbf{X}+\stackrel{~}{\lambda}\mathbf{I}\right]\widehat{\mathbf{\beta}}={\mathbf{X}}^{\prime}\mathbf{y}$; equivalently, $\widehat{\mathbf{\beta}}={\left[{\mathbf{X}}^{\prime}\mathbf{X}+\stackrel{~}{\lambda}\mathbf{I}\right]}^{-1}{\mathbf{X}}^{\prime}\mathbf{y}$. Relative to OLS, RR adds a constant, $\stackrel{~}{\lambda}$, to the diagonal of the matrix of coefficients; this makes the solution unique and shrinks estimates of marker effects towards zero, with the extent of shrinkage increasing as $\stackrel{~}{\lambda}$ increases (with $\stackrel{~}{\lambda}=0$ the solution to the above problem is the OLS estimate of marker effects).

From a Bayesian perspective, $\widehat{\mathbf{\beta}}$ can be viewed as the conditional posterior mode in a model with Gaussian likelihood and IID (independent and identically distributed) Gaussian marker effects, that is,

Above, ${\sigma}_{\beta}^{2}$ is the a priori variance of marker effects. The prior distribution of marker effects becomes increasingly informative and concentrated around zero as ${\sigma}_{\beta}^{2}$ decreases. The posterior mean and mode of **β** from [1] is equal to the ridge-regression estimate, with $\stackrel{~}{\lambda}={\sigma}_{\beta}^{-2}{\sigma}_{\epsilon}^{2}$, that is, $E\left(\mathbf{\beta}\mid y\right)=\widehat{\mathbf{\beta}}={\left[{\mathbf{X}}^{\prime}\mathbf{X}+{\sigma}_{\beta}^{-2}{\sigma}_{\epsilon}^{2}\mathbf{I}\right]}^{-1}{\mathbf{X}}^{\prime}\mathbf{y}$. Therefore, prediction of genetic values from [1] can be obtained as

Alternatively, from [1] and using properties of the multivariate normal distribution, one has:

Formulae [2a] and [2b] are equivalent and correspond to the Best Linear Unbiased Predictor (BLUP) under the model defined by [1]. Note that [2a] and [2b] require solving systems of *p* and *n* equations, respectively. Therefore, with *p* >> *n*, expression [2b] is computationally more convenient. However, unlike [2a], expression [2b] does not yield estimates of marker effects.

In RR-BLUP, estimates of marker effects are penalized to the same extent, and this may not be appropriate if some markers are located in regions not associated with genetic variance whereas others are linked to QTLs (Goddard and Hayes, 2007). To overcome this limitation, methods performing variable selection and shrinkage (e.g., LASSO) or Bayesian methods using marker-specific shrinkage of effects, such as methods BayesA and BayesB of Meuwissen et al. (2001) or the BL of Park and Casella (2008), have been proposed.

The BLR package implements Bayesian regression with marker-specific or marker homogenous shrinkage of estimates effects. The package allows inclusion of covariates other than markers and regressions on a pedigree as well. In BLR, phenotypes are expressed as follows:

where **y**, the response, is a (*n* × 1) vector (missing values are allowed); μ is an intercept; ${\mathbf{X}}_{\mathrm{F}}={\left\{{x}_{Fij}\right\}}_{n\times {p}_{F}}$, $\mathbf{Z}={\left\{{z}_{ij}\right\}}_{n\times {p}_{U}}$, ${\mathbf{X}}_{\mathrm{R}}={\left\{{x}_{Rij}\right\}}_{n\times {p}_{R}}$, and ${\mathbf{X}}_{\mathrm{L}}={\left\{{x}_{Rij}\right\}}_{n\times {p}_{L}}$ are incidence matrices for the vectors of effects **β**_{F}, **u**, **β**_{R}, and **β**_{L}, whose dimensions are *p*_{F}, *p*_{U}, *p*_{R}, and *p*_{L}, respectively. These vectors of effects differ with respect to the prior distributions assigned, as discussed later on. Finally, **ε** is a vector of model residuals assumed to be distributed as $\mathbf{\epsilon}\sim N\left(0,\mathrm{Diag}\left\{\frac{{\sigma}_{\epsilon}^{2}}{{w}_{i}^{2}}\right\}\right)$, where ${\sigma}_{\epsilon}^{2}$ is an (unknown) variance parameter and the *w _{i}*’s are (known) weights that allow for heterogeneous-residual variances. From these assumptions, the conditional distribution of the data, given the location effects, the residual variance and the weights, is:

Any of the elements on the right-hand side of [3], except μ and **ε**, can be excluded in BLR; by default, the program runs an intercept model, i.e., *y _{i}* = μ + ε

*. If weights are not provided, all weights are set equal to one.*

_{i}The Bayesian model is completed by assigning a prior to all model unknowns, $\mathbf{\theta}=\left\{\mu ,{\mathbf{\beta}}_{\mathrm{F}},\mathbf{u},{\sigma}_{\mathrm{u}}^{2},{\mathbf{\beta}}_{\mathrm{R}},{\sigma}_{{\beta}_{\mathrm{R}}}^{2},{\mathbf{\beta}}_{\mathrm{L}},{\tau}^{2},\lambda ,{\sigma}_{\epsilon}^{2}\right\}$. The intercept, μ, and the vector of “fixed” effects, **β**_{F}, are assigned flat priors, that is, *p*(μ, **β**_{F}) constant. This treatment yields posterior means of these unknowns that are similar to those obtained with OLS, provided that μ and **β**_{F} are the only effects included in the model.

The vector **u** is modeled using the standard assumptions of the infinitesimal additive model (e.g., Fisher, 1918; Wright, 1921; Henderson, 1975), that is, $p\left(\mathbf{u}\mid {\sigma}_{u}^{2}\right)=N\left(\mathbf{0},\mathbf{A}{\sigma}_{u}^{2}\right)$, where **A** is a positive-definite matrix (usually a numerator relationship matrix computed from a pedigree) and ${\sigma}_{\mathrm{u}}^{2}$ is an unknown variance, whose prior is a scaled inverse-**χ**^{2} density with degrees of freedom df_{u} and scale *S*_{u}, that is, ${\sigma}_{\mathrm{u}}^{2}\sim {\chi}^{-2}\left({\sigma}_{\mathrm{u}}^{2}\mid {\mathrm{df}}_{\mathrm{u}},{S}_{\mathrm{u}}\right)$; the hyper-parameters are user provided. In the parameterization used in BLR, $E\left({\sigma}_{\mathrm{u}}^{2}\mid {\mathrm{df}}_{\mathrm{u}},{S}_{\mathrm{u}}\right)={S}_{\mathrm{u}}{\left({\mathrm{df}}_{\mathrm{u}}-2\right)}^{-1}$. The multivariate normal prior assigned to **u** induces shrinkage of estimates of effects *u _{j}* toward zero and borrowing of information between levels of the random effect, $\left\{{u}_{j},{u}_{{j}^{\prime}}\right\}$.

The vector of regression coefficients **β**_{R} is assigned a Gaussian prior with variance common to all effects, that is, $p\left({\mathbf{\beta}}_{\mathrm{R}}\mid {\sigma}_{{\beta}_{\mathrm{R}}}^{2}\right)={\prod}_{j=1}^{{p}_{R}}N\left({\beta}_{\mathrm{R}j}\mid 0,{\sigma}_{{\beta}_{\mathrm{R}}}^{2}\right)$. This prior induces estimates that are the Bayesian counterpart of those obtained with Ridge Regression; we refer to this as “Bayesian Ridge Regression” (BRR). The variance parameter, ${\sigma}_{{\beta}_{\mathrm{R}}}^{2}$, is treated as unknown and is assigned a scaled inverse-χ^{2} prior density, that is, $p\left({\sigma}_{{\beta}_{\mathrm{R}}}^{2}\right)={\chi}^{-2}\left({\sigma}_{{\beta}_{\mathrm{R}}}^{2}\mid {\mathrm{df}}_{{\beta}_{\mathrm{R}}},{S}_{{\beta}_{\mathrm{R}}}\right)$ with degrees of freedom, df_{βR}, and scale, *S*_{βR}, provided by the user.

The vector of regression coefficients **β**_{L} is treated as in the Bayesian LASSO of Park and Casella (2008); the conditional prior distribution of marker effects, $p({\mathbf{\beta}}_{\mathrm{L}}\mid {\tau}^{2},{\sigma}_{\epsilon}^{2})$, is Gaussian with marker-specific prior variances, that is, $p({\mathbf{\beta}}_{\mathrm{L}}\mid {\mathbf{\tau}}^{2},{\sigma}_{\epsilon}^{2})={\prod}_{j=1}^{{p}_{L}}N\left({\mathbf{\beta}}_{\mathrm{L}j}\mid 0,{\sigma}_{\epsilon}^{2}{\tau}_{j}^{2}\right)$. This prior induces marker-specific shrinkage of estimates of effects, whose extent depends on ${\tau}_{\mathrm{j}}^{-2}$. The variance parameters, ${\tau}_{\mathrm{j}}^{2}$, are assigned exponential IID priors, $p\left({\mathbf{\tau}}^{2}\mid \lambda \right)={\prod}_{j=1}^{{p}_{L}}\mathrm{Exp}\left({\tau}_{j}^{2}\mid {\lambda}^{2}\right)$. Finally, in the BLR package, the prior distribution of the regularization parameter λ, *p*(λ), can be:

- a mass-point at some value (i.e., fixed λ),
- $p\left(\lambda \mid \mathrm{max},{\alpha}_{1},{\alpha}_{2}\right)\propto \mathrm{Beta}\left(\phantom{\mid}\frac{\lambda}{\mathrm{max}}\mid {\alpha}_{1},{\alpha}_{2}\right)$ (de los Campos et al., 2009).

With the above assumptions, the marginal prior of regression coefficients β_{L}* _{j}*, $p\left({\beta}_{\mathrm{L}j}\mid \lambda \right)=\int N\left({\beta}_{\mathrm{L}j}\mid 0,{\sigma}_{\epsilon}^{2}{\tau}_{\mathrm{j}}^{2}\right)\mathrm{Exp}\left({\tau}_{\mathrm{j}}^{2}\mid {\lambda}^{2}\right)\partial {\tau}_{\mathrm{j}}^{2}$, is Double-Exponential (DE). Figure 1 displays the Gaussian and Double-Exponential density functions of random variables with zero mean and unit variance. Relative to the Gaussian, the DE distribution places a higher density at zero and thicker tails, inducing stronger shrinkage of estimates for markers with relatively small effect and less shrinkage of estimates for markers with sizable effect.

Finally, the residual variance is assigned a scaled inverse-χ^{2} prior density with degrees of freedom, df_{ε}, and scale parameter, *S*_{ε}, provided by the user that is:

Collecting the aforementioned assumptions, the prior distribution in BLR is:

The prior distribution is indexed by several hyper-parameters; in the Appendix, we provide guidelines for choosing these parameters on the basis of prior expectations about the proportion of phenotypic variance that can be attributed to each of the components on the righthand side of [3].

The posterior distribution of model unknowns is proportional to the product of the likelihood and the prior distribution, that is:

This posterior distribution does not have a closed form; however, a Gibbs sampler can be used to draw samples from it. The Gibbs sampler is as in de los Campos et al. (2009) but extended to accommodate “fixed” effects and BRR.

## Using Bayesian Linear Regression

This section gives two examples that illustrate the use of the BLR package and describe features of the models. It is assumed that the reader is familiar with the R-language/environment. In Example 1, we study the impact of different shrinkage methods (BRR vs. BL) using simulated data. Example 2 illustrates how the package can be used to implement cross-validation (CV) using Bayesian methods. Cross-validation can be used for model comparison (e.g., to compare predictive ability of pedigree-based models versus marker-based models) or for selecting model parameters (e.g., λ in the BL). Both examples make use of a wheat data set made available with the package, whose main features are described next.

### Wheat Data Set

These data contain a historical set of 599 wheat lines from CIMMYT’s Global Wheat Program that were genotyped for 1447 Diversity Array Technology markers (DArT, Triticarte Pty. Ltd, Canberra, Australia; http://www.triticarte.com.au; verified 26 July 2010) and evaluated for grain yield (GY) in four macroenvironments. The dataset becomes available in the R environment by running the following R-code:

library(BLR) data(wheat)

Function library() loads the package, and data() loads datasets included in the package into the environment. The above code loads the following objects [type objects() in the R console to list them] into the environment: - **Y**, a matrix (599 × 4) containing the 2-yr average grain yield of each of these lines in each of the four environments (phenotypes were standardized to a unit variance within each environment); - **A** (599 × 599) is a numerator relationship matrix computed from a pedigree that traced back many generations. This relationship matrix was derived using the Browse application of the International Crop Information System (ICIS), as described in http://cropwiki.irri.org/icis/index.php/TDM_GMS_Browse; verified 26 July 2010 (McLaren, 2005); - **X** (599 × 1279) is a matrix with DArT genotypes; data are from pure lines and genotypes were coded as 0/1 denoting the absence/presence of the DArT. Markers with a minor allele frequency lower than 0.05 were removed, and missing genotypes were imputed with samples from the marginal distribution of marker genotypes, that is, *x _{ij}* ~ Bernoulli ( ${\widehat{p}}_{\mathrm{j}}$ ), where ${\widehat{p}}_{\mathrm{j}}$ is the estimated allele frequency computed from the nonmissing genotypes. The number of DArT MMs after edition was 1279. -

**sets**(599 × 1) is a vector that assigns observations to 10 disjoint sets; the assignment was generated at random. This is used later to conduct a 10-fold CV.

### Example 1: The Nature of Different Shrinkage Methods

As stated, BRR or BL use different priors for marker effects; this induces different types of shrinkage of estimates of such effects. The simulation presented in this section aims at illustrating these differences. Data were generated using marker genotypes from the wheat dataset (**X**) with marker effects and model residuals simulated as described below.

#### Data Simulation

Data were simulated under an additive model of the form, ${y}_{i}=\mu +{\sum}_{j=1}^{1279}{x}_{ij}{\beta}_{\mathrm{j}}+{\epsilon}_{ij}$, *i* = 1,…,599, where μ = 100 is an effect common to all individuals; {*x _{ij}*} are marker genotypes from a collection of wheat lines described previously;{β

*} are marker effects; and ε*

_{j}*~ N(ε*

_{i}*|0,1) are IID standard normal residuals. We assumed that most markers (1267) had a relatively small effect and that only a few markers (12) had a sizable effect. Specifically, marker effects were sampled from the following mixture model:*

_{i}
where *k* = 1279^{−1}. The R-code used to implement this simulation was:

set.seed(12345) data(wheat) p<-ncol(X) nQTL<-12 n<-nrow(X) b0<-rnorm(p,sd=1/sqrt(p)) isQTL<-seq(from=3,to=p,length=nQTL) b0[isQTL]<-rep(c(−1,1),times=nQTL/2)* runif(min=.5,max=.8,n=12) yHat0<-100+X%*%b0 e0<-rnorm(n,sd=1) y<-yHat0+e0

Function set.seed() initializes the random number generator; **X** is the matrix with information on molecular markers. Functions rnorm() and runif() generate random draws from the uniform and normal distributions, respectively. Figure 2 shows realized marker effects obtained with the above R-code. In this example, the sample variance of phenotypes (*y*) was 1.78, and the ratio of the sample variance of genetic values relative to phenotypic variance was 0.43.

#### Choice of Hyper-Parameters

The Appendix provides guidelines on how to choose values of hyper-parameters. It is assumed that the user has prior beliefs about the proportion of phenotypic variance that can be attributed to each of the components of the regression. In the simulation, the variance of phenotypes was about 1.78, and the variance of model residuals was 1. In practice, one does not know the true proportion of phenotypic variance that can be assigned to the genetic signal and model residuals, unless a precise estimate of heritability is available. Suppose our prior belief is that 50% of the phenotypic variance can be attributed to the genetic signal. Using df_{ε} = 3 and ${\sigma}_{\epsilon}^{2}=0.9$ = 0.9 in formula [1A] of the appendix, we set *S*_{ε} = 4.5. Values of hyper-parameters of the prior distribution of ${\sigma}_{{\beta}_{\mathrm{R}}}^{2}$ and λ^{2} can be chosen using formulae [3A] and [4A] in the Appendix. Using these formulae requires computing the sum of squares (over markers) of the average genotype, that is, ${\sum}_{j=1}^{p}{\stackrel{\u2012}{x}}_{\mathrm{j}}^{2}$, where ${\stackrel{\u2012}{x}}_{\mathrm{j}}^{2}={n}^{-1}{\sum}_{i=1}^{n}{x}_{\mathrm{i}\mathrm{j}}$. In the wheat dataset, this quantity is approximately 504; using this and df** _{βR}** = 3,

*V*

_{R}= 0.9 in formula [3A], we set ${S}_{{\beta}_{\mathrm{R}}}^{2}=\frac{4.5}{504}\approx 0.009$. Finally, 504 using ${\sigma}_{\epsilon}^{2}{V}_{\mathrm{L}}=1$ in formula [4A] of the Appendix, we find $\widehat{\lambda}\approx 32$. Choosing λ

^{2}~

*G*(λ

^{2}|

*r*= 2×10

^{−5},δ = 0.52) gives a prior density for λ that has high density and is relatively flat around $\widehat{\lambda}\approx 32$ (Fig. 3).

#### Fitting the Model

Using the aforementioned values of hyper-parameters, BL and BRR were fitted using the following R-code:

prior=list( varE=list(S=4.5,df=3), varBR=list(S=.009,df=3), lambda=list(type="random", value=30,shape=.52,rate=2e–5)) nIter<-60000 burnIn<-10000 fmR<-BLR(y=y,XR=X,nIter=nIter,burnIn=burnIn, thin=10,saveAt="R _ ",prior=prior) dput(fmR,file="fmR.out") fmL<-BLR(y=y,XL=X,nIter=nIter, burnIn=burnIn,thin=10, saveAt="L _ ",prior=prior) dput(fmL,file="fmL.out")

In the above code, **y** is the response vector, **X** is the matrix of genotypes, and nIter and burnIn define the number of iterations and burn-in period, respectively, used in the Gibbs sampler. The prior is provided as a list; type “help(BLR)” in the R console for more details. The BLR function returns a list with posterior means, posterior standard deviations, and Deviance Information Criterion (DIC, Spiegelhalter et al., 2002). Function dput() saves this list to the hard drive. The fitted model can then be retrieved using function dget(). In addition to returning posterior means and posterior standard deviations, as the Gibbs sampler runs, samples of the intercept, of the fixed effects, of the variance parameters and of λ are saved to the hard-drive using the thinning specified by the user (which is set to 10 by default).

#### Results of Example 1

We ran the processes in an Intel Xeon 5530 2.4 GHz Quad Core Processor (R was executed in a single thread) with 6 GB of RAM memory. With this dataset (599 subjects, 1279 markers) the process took about 1% of RAM memory. BRR took about 5.5 s for every 1000 iterations of the sampler; BL takes about twice as much time.

Figure 4 shows the estimated posterior density of the residual variance for each of the models; the prior density (up to a constant) is included as well. The posterior distributions moved away from the prior and were sharp. The estimated posterior means (standard deviation) of ${\sigma}_{\epsilon}^{2}$ were 1.00 (0.0883) and 0.930 (0.0814) for BRR and BL, respectively. These values are close to the true value of the parameter (one) and suggest that BL over-fitted the data slightly. The posterior standard deviation (SD) was 8% smaller in BL, and mixing of the residual variance was better in BRR. BL gives a slightly “less informative” posterior distribution and worse mixing for two reasons. First, because of use of marker-specific variances, the number of unknowns in BL is much larger than in BRR. Second, the BL has an extra level in which the regularization parameter (λ) indexing the prior distribution of the marker-specific variances is inferred from the data. In BRR, the counterparts of λ are the hyper-parameters indexing the prior assigned to ${\sigma}_{{\beta}_{\mathrm{R}}}^{2}$ ( df_{βR} and *S*_{βR} ) which are specified by the user.

The posterior mean of λ in BL was 20.1, and a 95th highest posterior density confidence region was bounded by [15.7, 26.2]. These results also indicate that the posterior distribution of λ moved away from the prior (Fig. 3), indicating that Bayesian learning takes place.

Measures of goodness of fit and model complexity [pD = estimated effective number of parameters, Spiegelhalter et al. (2002), and DIC] are included in the fitted object. This can be assessed with the following code:

fmR$fit ## Bayesian Ridge Regression fmL$fit ## Bayesian LASSO

Table 1 provides estimates of the log-likelihood evaluated at the posterior mean of model unknowns, $l\left(\stackrel{\u2012}{\theta}\right)$, the posterior mean of the log-likelihood, $\stackrel{\u2012}{l}(.)$, the estimated effective number of parameters, pD, and DIC for BRR and BL. The Bayesian LASSO fitted the data better, and had a higher estimated number of effective parameters; DIC, which balances goodness of fit and complexity, favored the BL.

The mean-squared error of estimates of marker effects, $\mathrm{MSE}\left(\mathbf{\beta},{\widehat{\mathbf{\beta}}}_{.}\right)={p}^{-1}{\sum}_{j=1}^{p}{\left({\beta}_{j}-{\widehat{\beta}}_{.j}\right)}^{2}$, where (.) can be either L or R, were 0.0046 and 0.0039 for BRR and BL, respectively. The mean-squared error of estimates of genetic values, $\mathrm{MSE}\left(\mathbf{X}\mathbf{\beta},\mathbf{X}{\widehat{\mathbf{\beta}}}_{.}\right)={n}^{-1}{\sum}_{j=1}^{n}{\left({\mathbf{x}}_{.i}\mathbf{\beta}-{\mathbf{x}}_{.i}^{\prime}{\widehat{\mathbf{\beta}}}_{.}\right)}^{2}$, were 0.3712 and 0.3116 for BRR and BL, respectively. Therefore, BL outperformed BRR in this simulation. The difference between methods is more pronounced at the level of marker effects (19% difference in MSE between methods for genetic values). Figure 5 shows scatter plots of the estimates of marker effects (left) and of genetic values (right) obtained with BL (horizontal axis) and BRR (vertical axis). The BRR shrinks estimates of markers with sizable effects to a larger extent than BL (see left panel in Fig. 5). On the other hand, the two models yielded similar predictions of genetic values (see right panel in Fig. 5). This occurs because with *p* >> *n*, one can arrive at similar predictions of genetic values either with a model where genetic values are highly dependent on a few markers with sizable effect (something that occurs in LASSO and, to a lesser extent, in BL) or with a model where a large number of markers make a small contribution to genetic values (something that occurs in RR-BLUP and BRR). Which model yields better prediction of genetic values will depend on the underlying architecture of the trait and on the available marker data.

### Example 2: Assessing Predictive Ability by Cross-Validation

Predicting genetic values of lines with yet-to-be observed phenotypes is a central problem in plant breeding programs. Such predictions can be used, for example, to decide which of the newly developed lines will be included in field trials or which of these lines will be parents for the next breeding cycle. Either of the models described above, BL or BRR, can be used to obtain these predictions. The rate of genetic progress will depend on how accurate such predictions are, i.e., on the ability of the model to predict future outcomes. Cross-validation (CV) methods can be used to assess predictive ability and for for tuning-up values of certain parameters. In this section, we illustrate how the regularization parameter of the BL, λ, can be chosen using CV methods, and compare the performance of this approach with that obtained with the fully Bayesian approach that consists of assigning a prior to λ. The comparison is made using the wheat dataset previously described.

#### Model

Here, the linear model is **y** = **1**μ + **X**_{L}**β**_{L} + **u** + **ε**. We chose values of hyper-parameters using formulae presented in the Appendix. As in Example 1, it was assumed a priori that 50% of the phenotypic variance (which equals to one because phenotypes were standardized to a unit variance) could be attributed to genetic values. With this, and using df_{ε} = 3 in formula [1A] in the Appendix, we obtained *S*_{ε} = 2.5. Further, we assumed a priori that one half of the variance of genetic values can be accounted for by the regression on markers, **X**_{L}**β**_{L}, and that the regression on the pedigree, **u**, accounted for the other half. With this, and using df** _{u}** = 3 and $\stackrel{\u2012}{a}=1.98$ in [2A], we obtained

*S*

_{u}= 0.63. Finally, using ${\sum}_{j=1}^{p}{\left({n}^{-1}{\sum}_{i=1}^{n}{x}_{\mathrm{L}\mathrm{i}\mathrm{j}}\right)}^{2}=504$ in formula [4A] in the Appendix, we obtained $\widehat{\lambda}=\sqrt{2{\sigma}_{\epsilon}^{2}{V}_{\mathrm{L}}^{-1}{\sum}_{j}^{{p}_{\mathrm{R}}}{\stackrel{\u2012}{x}}_{\mathrm{L}\mathrm{j}}^{2}}=\sqrt{2\frac{1}{2}\frac{4}{1}504}\approx 45$. Choosing λ

^{2}~

*G*(λ

^{2}|

*r*= 1×10

^{−5},δ = 0.52) gives a prior for λ that has a maximum and is relatively flat in the neighborhood of 45.

The R-code in Fig. 6 illustrates how this CV was performed for the first trait. The vector sets assign lines to folds of the CV. The code involves two loops: the outer loop runs over folds of the CV; the inner loop fits models over a grid of values of λ. For every fold in the outer loop, the phenotypes of approximately 60 lines are declared as missing values; the fitted model yields a prediction of the performance of these lines.

#### Results of Example 2

Figure 7 gives the estimated mean-squared error of predictive residuals (PMSE, vertical axis) versus values of the regularization parameter (λ), by environment. The vertical and horizontal dashed lines give the average (across 10 folds of the CV) estimated posterior mean of λ and the estimated PMSE obtained when a prior was assigned to λ (i.e., the fully-Bayesian LASSO). In all environments except E3, the curve relating PMSE and λ was U-shaped, with an optimum λ (minimum PMSE) near 20. However, the absolute value of the slope of the curve was higher for low values of λ, indicating that over-fitting, something that occurs with small values of λ, is more problematic. This may occur because the conditional expectation function in these models included two random components: the regression on markers and the regression on the pedigree. The presence of **u** in the linear model cannot prevent the over-fitting occurring when λ is “too small”. However, as values of λ increase (i.e., placing a higher penalty on the regression on markers), the contribution of the regression on the pedigree to the conditional expectation increases as well, preventing lack of fit. Environment 3 constitutes an extreme example of this; here the curve relating PMSE and λ looks L-shaped.

**...**

The posterior modes of λ were always considerably smaller than the prior mode (45), indicating that Bayesian learning took place. In all environments, the fully Bayesian treatment yielded posterior means of λ that were in the neighborhood of the optimal values found when models were run over a grid of values of this unknown. Also, predictive ability of models with random λ was as good as the best obtained when CV was used for choosing λ. These results suggest that, at least for these traits and this population, the fully Bayesian treatment, which consists of inferring λ from the data, yields good results.

## Conclusions

The BLR package allows fitting high-dimensional linear regression models including dense molecular markers, pedigree information, and covariates other than markers. The interface allows the user to choose models (e.g., BL versus BRR) and prior hyper-parameters easily. The algorithms implemented are relatively efficient and models with a modest number of markers (e.g., ~1000) can be fitted in a standard PC easily. The routines implemented in the package have also been used successfully in problems with larger numbers of molecular markers. For example, Weigel et al. (2009) used an earlier version of the package to fit models using data from the 50k Bovine Illumina Bead Chip, and our experience indicates that the software can be used with an even larger number of markers. Computational time is expected to increase linearly with the number of markers; the user also needs to be aware that marker information is loaded in the memory. Therefore, as the number of marker increases, so do the memory requirements.

In models for genomic selection, with *p* >> *n*, marker effects cannot be estimated uniquely from the likelihood. A unique solution can be obtained by using penalized estimation methods or, in a Bayesian framework, by assigning informative priors to marker effects. Because of lack of identification at the level of the likelihood, the choice of prior is expected to play a role. As illustrated in Example 1, different priors yield different estimates of marker effects. Although models per se cannot solve the intrinsic identification problem, one can use model comparison criteria, such as DIC, or the principle of parsimony, or CV methods, to choose among prior distributions of marker effects. Even if the choice of prior affects estimates of marker effects, it is still possible to obtain similar estimates of genetic values using different priors (see Example 1, or the simulation study presented in de los Campos et al., 2009). This occurs because, with *p >> n*, one can arrive at similar predictions either with a model where genetic values are highly dependent on some markers or with a model where all markers make a small contribution to genetic values.

Simulation studies (e.g., Habier et al., 2007) have suggested that models using marker-specific shrinkage of estimates of effects such as the BL or models Bayes A or Bayes B of Meuwissen et al. (2001) may capture linkage disequilibrium between markers and QTLs better than a BRR. Therefore, Habier et al. (2007) conclude, when estimates of effects are used to perform several rounds of selection and without retraining, those models should perform better than a BRR. However, as pointed out by Jannink et al. (2010), the superiority of these models over a BRR has not always been confirmed by real-data analysis.

Finally, estimates of marker effects obtained with BL or BRR could be used to assess the relative contribution of each region to genetic variability. However, one needs to be aware that the estimated marker effect reflects not only linkage between markers and genes affecting the trait but also the density of markers in the region. If a region containing a QTL has a high density of markers, the effect of the QTL is expected to be “distributed” across linked markers; conversely, if in the same region markers are sparse, estimated marker effects are expected to be larger in absolute value.

## Acknowledgments

The authors thank the numerous cooperators in national agricultural research institutes who carried out the Elite Spring Wheat Yield Trials (ESWYT) and provided the phenotypic data analyzed in this article. We also thank the International Nursery and Seed Distribution Units in CIMMYT-Mexico for preparing and distributing the seed and computerizing the data. Financial support by the Wisconsin Agriculture Experiment Station and grant DMS-NSF DMS-044371 to Gustavo de los Campos and Daniel Gianola and of NIH grant R01GM077490 to Gustavo de los Campos is acknowledged.

## Abbreviations

- BL
- Bayesian LASSO
- BLR
- Bayesian Linear Regression
- BLUP
- Best Linear Unbiased Prediction
- BRR
- Bayesian Ridge Regression
- CV
- cross-validation
- DE
- Double-Exponential
- DIC
- Deviance Information Criterion
- GY
- grain yield
- GS
- Genomic selection
- IID
- independent and identically distributed
- LASSO
- Least Absolute Shrinkage and Selection Operator
- MM
- molecular markers
- OLS
- ordinary least squares
- PMSE
- mean-squared error of predictive residuals
- RR
- ridge regression

## APPENDIX

The prior distribution of the BLR model is indexed by several hyper-parameters. We provide guidelines for choosing their values on the basis of beliefs as to what proportion of the variance of phenotypes can be attributed to each of the random components of the regression, that is, *u*_{i}, ${\sum}_{j=1}^{{p}_{\mathrm{R}}}{x}_{\mathrm{R}ij}{\beta}_{\mathrm{R}\mathrm{j}}$, ${\sum}_{j=1}^{{p}_{\mathrm{R}}}{x}_{\mathrm{L}\mathrm{i}\mathrm{j}}{\beta}_{\mathrm{L}\mathrm{j}}$ and ε_{i}. Other authors (e.g., Meuwissen et al., 2001) have discussed how to choose hyper-parameters in the context of models for genomic selection. However, the derivation used by those authors assumes that genotypes are random and marker effects are fixed quantities, while in fact the opposite is true in the Bayesian models that have been proposed (e.g., Gianola et al., 2009). Here, we derive formulae that are consistent with the standard treatment of marker and marker effects in Bayesian models for GS, where marker genotypes are observed quantities and marker effects are random unknowns. Unlike the formulae in Meuwissen et al. (2001), the ones presented here do not require making any assumption about the extent of linkage disequilibrium between markers.

#### Residual Variance

The prior distribution of the residual variance is indexed by two parameters, {*S*_{ε}, df_{ε}}. One can choose these parameters so that the prior mode of ${\sigma}_{\epsilon}^{2}$, $\frac{{S}_{\epsilon}}{{\mathrm{df}}_{\epsilon}+2}$ matches our prior beliefs about the variance of model residuals. In practice, df_{ε} can be chosen to be a small value, usually greater than two, to guarantee a finite prior expectation, e.g., df_{ε} = 3, then the prior scale can be:

where *V*_{ε} is chosen to reflect our expectation of the variance of model residuals. Formulas similar to that presented in this appendix can be derived using formulas for the prior expectation.

#### Variance of the Infinitesimal Effect

From the prior distribution, the variance of *u*_{i} is ${a}_{ii}{\sigma}_{u}^{2}$, where *a _{ii}* is the

*i*th diagonal element of

**A**, which in the absence of inbreeding equals one. If we let $\stackrel{\u2012}{a}$ be the average diagonal value of

**A**and

*V*be our prior expectation about $\stackrel{\u2012}{a}{\sigma}_{u}^{2}$, then we can choose df

_{u}*= 3 and set the prior scale to be:*

_{u}

#### Prior Variance of Marker Effects

The contributions of **X**_{R} and **X**_{L} to phenotypes is ${\sum}_{j=1}^{p}{x}_{ij}{\beta}_{\mathrm{j}}$. Here, (.) stands for R or L, depending on whether BRR or BL is being used. In regressions for GS, marker genotypes are fixed and marker effects are random. Furthermore, at the level of the marginal distribution, marker effects are IID; therefore, $\mathrm{Var}\left({\sum}_{j=1}^{{p}_{.}}{x}_{.ij}{\beta}_{.j}\right)={V}_{.}={\sum}_{j=1}^{p}{x}_{.ij}^{2}\mathrm{Var}\left({\beta}_{.j}\right)$. For the “average” genotype, this formula becomes ${V}_{.}=\left\{{\sum}_{j=1}^{p}{\stackrel{\u2012}{x}}_{.j}^{2}\right\}\mathrm{Var}\left({\beta}_{.j}\right)$ where *V*_{.} represent user’s beliefs about the variance that can be assigned to the regression on **X.**, and ${\stackrel{\u2012}{x}}_{.\mathrm{j}}$ denotes the average value of the *j*th column of **X**.

In BRR, the prior distribution of marker effects is Gaussian and $\mathrm{Var}\left({\beta}_{\mathrm{R}\mathrm{j}}\right)={\sigma}_{\beta}^{2}$. As before, df_{βR} can be chosen to have a relatively small value, e.g., df_{βR} = 3 then the prior scale can be set to be

where *V*_{R} is set to reflect our expectation of the variance of phenotypes that can be attributed to the regression on **X**_{R}.

In the BL, the marginal prior density of marker effects is Double-Exponential, and the prior variance of marker effects is $\mathrm{Var}\left({\beta}_{\mathrm{L}\mathrm{j}}\right)=2{\sigma}_{\epsilon}^{2}{\lambda}^{-2}$. Plugging this in

we obtain ${V}_{\mathrm{L}}=\left\{{\sum}_{j=1}^{{p}_{\mathrm{L}}}{\stackrel{\u2012}{x}}_{\mathrm{L}\mathrm{j}}^{2}\right\}2{\sigma}_{\epsilon}^{2}{\lambda}^{-2}$. Solving for λ, we get:

where ${\sigma}_{\epsilon}^{2}{V}_{\mathrm{L}}^{-1}$ is a noise-to-signal variance ratio. With [4A] we can choose a target value for the regularization parameter. Then we can choose hyper-parameters so that the prior has a mode and is relatively flat, in the neighborhood of $\widehat{\lambda}$. Note that when λ^{2} ~ *G*(*r*,δ), *p*(λ|*r*,δ) = *G*(λ^{2}|*r*,δ)2λ.

The above formulas constitute guidelines for choosing values of hyper-parameters. In practice, if Bayesian learning takes place, one would expect the posterior distribution to move away from the prior. Furthermore, with small *n*, it is always useful to check the sensitivity of inferences with respect to the choice of hyper-parameters.

## Footnotes

Permission for printing and for reprinting the material contained herein has been obtained by the publisher.

## References

- Bernardo R, Yu J. Prospects for genome-wide selection for quantitative traits in maize. Crop Sci. 2007;47:1082–1090.
- Crossa J, de los Campos G, Pérez P, Gianola D, Dreisigacker S, Burgueño J, Araus JL, Makumb D, Yan J, Singh R, Arief V, Banziger M, Braun H-J. Prediction of genetic values of quantitative traits in plant breeding using pedigree and molecular markers. Genetics. 2010 (in press) [PMC free article] [PubMed]
- de los Campos G, Naya H, Gianola D, Crossa J, Legarra A, et al. Predicting quantitative traits with regression models for dense molecular markers and pedigrees. Genetics. 2009;182:375–385. [PMC free article] [PubMed]
- de los Campos G, Pérez P. [verified 22 July 2010];BLR: Bayesian linear regression. R package version 1.1. 2010 http://www.r-project.org/ [PubMed]
- Fisher RA. The correlation between relatives on the supposition of Mendelian inheritance. Trans. R. Soc. Edinb. 1918;52:399–433.
- Gianola D, de los Campos G, Hill WG, Manfredi E, Fernando RL. Additive genetic variability and the Bayesian alphabet. Genetics. 2009;183:347–363. [PMC free article] [PubMed]
- Goddard ME, Hayes BJ. Genomic selection. J. Anim. Breed. Genet. 2007;124:323–330. [PubMed]
- Habier D, Fernando RL, Dekkers JCM. The impact of genetic relationship information on genome-assisted breeding values. Genetics. 2007;177:2389–2397. [PMC free article] [PubMed]
- Hayes BJ, Bowman PJ, Chamberlain AJ, Goddard ME. Invited review: Genomic selection in dairy cattle: Progress and challenges. J. Dairy Sci. 2009;92:433–443. [PubMed]
- Henderson CR. Best linear unbiased estimation and prediction under a selection model. Biometrics. 1975;31:423–447. [PubMed]
- Jannink JL, Lorenz AJ, Hiroyoshi I. Genomic selection in plant breeding: From theory to practice. Brief. Funct. Genomics. 2010;9:166–177. [PubMed]
- McLaren CG. The International Rice Information System. A platform for meta-analysis of rice crop data. Plant Physiol. 2005;139:637–642. [PMC free article] [PubMed]
- Meuwissen THE, Hayes BJ, Goddard ME. Prediction of total genetic values using genome-wide dense marker maps. Genetics. 2001;157:1819–1829. [PMC free article] [PubMed]
- Park T, Casella G. The Bayesian LASSO. J. Am. Stat. Assoc. 2008;103:681–686.
- R Development Core Team . R: A language and environment for statistical computing. R Foundation for Statistical Computing; Vienna, Austria: [verified 22 July 2010]. 2009. ISBN 3–900051–07–0, URL http://www.r-project.org/
- Spiegelhalter DJ, Best NG, Carlin BP, van der Linde A. Bayesian measures of model complexity and fit (with discussion) J. R. Stat. Soc. Ser. B Stat. Methodol. 2002;64:583–639.
- Tibshirani R. Regression shrinkage and selection via the LASSO. J. Royal Stat. Soc. B. 1996;58:267–288.
- VanRaden PM, Van Tassell CP, Wiggans GR, Sonstegard TS, Schnabel RD, et al. Invited review: Reliability of genomic predictions for North American Holstein bulls. J. Dairy Sci. 2009;92:16–24. [PubMed]
- Weigel K, de los Campos G, Vazquez AI, González-Recio O, Naya H, Wu XL, Long N, Rosa GJM, Gianola D. Predictive ability of direct genomic values for lifetime net merit of Holstein sires using selected subsets of single nucleotide polymorphism markers. J. Dairy Sci. 2009;92:5248–5257. [PubMed]
- Wright S. Systems of mating. I. The biometric relations between parents and offspring. Genetics. 1921;6:111–123. [PMC free article] [PubMed]
- Yi N, Xu S. Bayesian LASSO for quantitative trait loci mapping. Genetics. 2008;179:1045–1055. [PMC free article] [PubMed]

## Formats:

- Article |
- PubReader |
- ePub (beta) |
- PDF (1.2M)

- Genome-enabled prediction using the BLR (Bayesian Linear Regression) R-package.[Methods Mol Biol. 2013]
*de Los Campos G, Pérez P, Vazquez AI, Crossa J.**Methods Mol Biol. 2013; 1019:299-320.* - Predicting quantitative traits with regression models for dense molecular markers and pedigree.[Genetics. 2009]
*de los Campos G, Naya H, Gianola D, Crossa J, Legarra A, Manfredi E, Weigel K, Cotes JM.**Genetics. 2009 May; 182(1):375-85. Epub 2009 Mar 16.* - Genome-Wide Regression & Prediction with the BGLR Statistical Package.[Genetics. 2014]
*Pérez P, de Los Campos G.**Genetics. 2014 Jul 9; . Epub 2014 Jul 9.* - Genomic selection using regularized linear regression models: ridge regression, lasso, elastic net and their extensions.[BMC Proc. 2012]
*Ogutu JO, Schulz-Streeck T, Piepho HP.**BMC Proc. 2012 May 21; 6 Suppl 2:S10. Epub 2012 May 21.* - Additive genetic variability and the Bayesian alphabet.[Genetics. 2009]
*Gianola D, de los Campos G, Hill WG, Manfredi E, Fernando R.**Genetics. 2009 Sep; 183(1):347-63. Epub 2009 Jul 20.*

- Parametric and Nonparametric Statistical Methods for Genomic Selection of Traits with Additive and Epistatic Genetic Architectures[G3: Genes|Genomes|Genetics. ]
*Howard R, Carriquiry AL, Beavis WD.**G3: Genes|Genomes|Genetics. 4(6)1027-1046* - Accuracy of Estimation of Genomic Breeding Values in Pigs Using Low-Density Genotypes and Imputation[G3: Genes|Genomes|Genetics. ]
*Badke YM, Bates RO, Ernst CW, Fix J, Steibel JP.**G3: Genes|Genomes|Genetics. 4(4)623-631* - Genomic-enabled prediction with classification algorithms[Heredity. 2014]
*Ornella L, Pérez P, Tapia E, González-Camacho JM, Burgueño J, Zhang X, Singh S, Vicente FS, Bonnett D, Dreisigacker S, Singh R, Long N, Crossa J.**Heredity. 2014 Jun; 112(6)616-626* - Heteroscedastic Ridge Regression Approaches for Genome-Wide Prediction With a Focus on Computational Efficiency and Accurate Effect Estimation[G3: Genes|Genomes|Genetics. ]
*Hofheinz N, Frisch M.**G3: Genes|Genomes|Genetics. 4(3)539-546* - A reaction norm model for genomic selection using high-dimensional genomic and environmental data[TAG. Theoretical and Applied Genetics. Theo...]
*Jarquín D, Crossa J, Lacaze X, Du Cheyron P, Daucourt J, Lorgeou J, Piraux F, Guerreiro L, Pérez P, Calus M, Burgueño J, de los Campos G.**TAG. Theoretical and Applied Genetics. Theoretische Und Angewandte Genetik. 2014; 127595-607*

- Genomic-Enabled Prediction Based on Molecular Markers and Pedigree Using the Bay...Genomic-Enabled Prediction Based on Molecular Markers and Pedigree Using the Bayesian Linear Regression Package in RNIHPA Author Manuscripts. 2010; 3(2)106PMC

Your browsing activity is empty.

Activity recording is turned off.

See more...