• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of bioinfoLink to Publisher's site
Bioinformatics. Jan 1, 2009; 25(1): 83–89.
Published online Nov 7, 2008. doi:  10.1093/bioinformatics/btn579
PMCID: PMC2638932

Pan-specific MHC class I predictors: a benchmark of HLA class I pan-specific prediction methods

Abstract

Motivation: MHC:peptide binding plays a central role in activating the immune surveillance. Computational approaches to determine T-cell epitopes restricted to any given major histocompatibility complex (MHC) molecule are of special practical value in the development of for instance vaccines with broad population coverage against emerging pathogens. Methods have recently been published that are able to predict peptide binding to any human MHC class I molecule. In contrast to conventional allele-specific methods, these methods do allow for extrapolation to uncharacterized MHC molecules. These pan-specific human lymphocyte antigen (HLA) predictors have not previously been compared using independent evaluation sets.

Result: A diverse set of quantitative peptide binding affinity measurements was collected from Immune Epitope database (IEDB), together with a large set of HLA class I ligands from the SYFPEITHI database. Based on these datasets, three different pan-specific HLA web-accessible predictors NetMHCpan, adaptive double threading (ADT) and kernel-based inter-allele peptide binding prediction system (KISS) were evaluated. The performance of the pan-specific predictors was also compared with a well performing allele-specific MHC class I predictor, NetMHC, as well as a consensus approach integrating the predictions from the NetMHC and NetMHCpan methods.

Conclusions: The benchmark demonstrated that pan-specific methods do provide accurate predictions also for previously uncharacterized MHC molecules. The NetMHCpan method trained to predict actual binding affinities was consistently top ranking both on quantitative (affinity) and binary (ligand) data. However, the KISS method trained to predict binary data was one of the best performing methods when benchmarked on binary data. Finally, a consensus method integrating predictions from the two best performing methods was shown to improve the prediction accuracy.

Contact: kd.utd.sbc@leinm

Supplementary information: Supplementary data are available at Bioinformatics online.

1 INTRODUCTION

Cytotoxic T lymphocytes (CTL) play a central role in defeating intracellular infections with pathogens, such as viruses and certain bacteria. The CTL T-cell receptor (TCR) recognizes foreign peptides in complex with major histocompatibility complex (MHC) class I molecules on the surface of the infected cells. MHC class I molecules preferably bind and present nine amino acid long peptides, which mainly originates from proteins expressed in the cytosol of the presenting cell. In most mammals, MHCs exist in a number of different allelic variants each of which binds to a specific and very limited set of peptides.

During the last decades, a number of prediction methods have been developed to identify which peptides will bind a given MHC molecule (see review Lundegaard et al., 2007). Today, the best methods are able to predict MHC class I binding with very high accuracy (Lin et al., 2008; Peters et al., 2006). The human class I version of MHC, the human lymphocyte antigen (HLA), is expressed by genes at three loci (HLA-A, -B, and -C) at chromosome 6. Several hundred alleles expressing MHCs with slightly different amino acid sequence are hosted at each locus, and each product potentially binds a different set of peptides. Most of the available data of MHC:peptide binding has been originating from a limited number of alleles and thus prediction systems have been biased towards these alleles. Lately, the number of available binding data has increased significantly both in terms of amount of different peptides and the number of different MHC alleles. This fact has in itself influenced the MHC prediction systems that now cover a large number of different HLA alleles (Peters et al., 2006). More interestingly, however, it has enabled the development of new so called pan-specific algorithms that can predict peptide binding to alleles for which limited or even no experimental data are available (Jacob and Vert, 2008; Jojic et al., 2006; Nielsen et al., 2007; Zhang et al., 2005). A common feature of these pan-specific methods is that they go beyond the conventional single allele approach, and take both the peptide sequence and the MHC contact environment into account. A large set of publicly available allele-specific MHC class I predictors that predict peptide binding only to the alleles on which they have been trained have been benchmarked latest by Peters et al. (2006) and Lin et al. (2008). The pan-specific predictors, on the other hand, have not to date been evaluated on a large independent dataset. The Immune Epitope database (IEDB) (Sette et al., 2005) is growing rapidly, and a large number of MHC class I binding data has become available since the training of the publicly available online versions of the pan-specific predictors. In this work, 6533 such novel quantitative peptide binding affinity measurements covering 33 different human MHC class I alleles were collected. As the resources necessary for biochemical determination of MHC:peptide affinity are significant, many of the data-points are presumably selected either based on other biological evidence or by use of in silico predictions like the ones tested in this work. It is known for a fact that NetMHC has been used to predict a number of data subsequently measured and deposited in the IEDB database. To test if this bias will give any difference in evaluation performance, a second evaluation set of 5137 data points covering 27 HLA-I alleles was created by removal of data-points identified by the authors of the NetMHC and NetMHCpan methods. For evaluation of the ability to distinguish between ligands and non-ligands, a set of 566 HLA ligands covering 34 HLA class I alleles was created by download from the SYFPEITHI database. For each evaluation dataset, peptide data used in the training of the NetMHC-3.0 (Buus et al., 2003; Nielsen et al., 2003) and NetMHCpan-1.0 (Nielsen et al., 2007) methods was removed. The benchmark was run against adaptive double threading (ADT) (Jojic et al., 2006), the artificial neural network (ANN)-based NetMHCpan-1.0 (Nielsen et al., 2007) and the SVM-based: kernel-based inter-allele peptide binding prediction system (KISS) (Jacob and Vert, 2008) methods. These methods are all publicly available via web interfaces. In addition, the quality of the pan-specific predictors was compared with one of the best available conventional allele-specific MHC predictor: NetMHC-3.0 (Buus et al., 2003; Lundegaard et al., 2008; Nielsen et al., 2003).

In many areas where different predictors have been developed, it has been demonstrated how consensus methods defined by averaging over several different predictions can improve the prediction accuracy beyond that of each individual predictors (Bujnicki et al., 2001). This is often due to the fact that the methods have been trained on different data and thus have learned potentially different features of the sequence space. However, also methods trained on the exact same data can contribute with non-redundant information and thus gain superiority by combination, as shown for example in Nielsen et al. (2007), Nielsen et al. (2003) and Petersen et al. (2000). Although such so-called meta predictors have been tried several times for MHC class II prediction (Huang et al., 2006; Karpenko et al., 2008; Mallios, 2001), no such scheme has to our knowledge been published using the wide diversity of MHC class I predictors. To investigate the possible benefits of such an approach, the predictions of the individual methods were compared with a consensus method defined by averaging the allele-specific MHC NetMHC-3.0 and pan-specific NetMHCpan-1.0 predictors.

2 MATERIALS AND METHODS

2.1 Datasets

The 9mer peptides associated with quantitative binding data were retrieved from the Immune Epitope Database and Analysis Resource (IEDB).

All peptides used in the training of NetMHC and NetMHCpan methods were discarded. This resulted in a dataset of 6533 experimental measurements covering 33 HLA-I alleles. These data were released after a point of time when the methods NetMHC-3.0, NetMHCpan-1.0 and ADT were trained and published. By doing so, it can be assured that the data had not been used in training of the above methods. This dataset is named EvaluationSet-1. Table 1 (panel A) gives an overview of dataset EvaluationSet-1.

Table 1.
Overview of dataset EvaluationSet-1, dataset EvaluationSet-2 and the SYFPEITHI dataset EvaluationSet-3

There was a concern that a significant part of the data deposited into IEDB dataset was selected for measurement by the NetMHC prediction method. In this way the dataset could be biased and in favor of NetMHC and possibly NetMHCpan as well. To test this, all data submitted to the IEDB database by the authors of the NetMHC and NetMHCpan methods, were removed and this resulted in a dataset containing 5137 data points covering 27 HLA-I alleles. This dataset is called EvaluationSet-2. Table 1 (panel B) summarizes the data in dataset EvaluationSet-2. Alleles in both datasets containing less than 10 data points were not included in the evaluation. Note, that both EvaluationSet-1 and EvaluationSet-2 have a very high ratio of binding versus non-binding peptides (50% on average). This highly non-biological fraction of binders (due to the procedure used to select peptides for experimental validation) will make the predictive performance values lower than on the biological data, where the ratio of binders to non-binders is of the order 1–2% (Nielsen et al., 2003; Yewdell and Bennink, 1999).

2.2 Endogenously presented peptides

Both the NetMHCpan and ADT methods were trained only on quantitative peptide MHC binding data. The KISS method on the other hand was trained on qualitative binding versus non-binding data from a broad range of data sources including IEDB (Sette et al., 2005), SYFPEITHI (Rammensee et al., 1999), Los Alamos HIV database (http://www.hiv.lanl.gov) and MHCBN (Bhasin et al., 2003). To test if this inherent difference in the training data would make the NetMHCpan and ADT perform advantageous on quantitative data and the KISS method favor qualitative data, an evaluation set of HLA ligands were downloaded from the SYFPEITHI database (Rammensee et al., 1999). Only data restricted to HLA alleles covered by all methods and not included in the training of any of the three methods NetMHCpan, ADT and KISS were included. This set consists of 566 HLA ligands restricted to 33 different HLA-A and -B alleles. For every peptide, the source protein was found in the UniProt database (The UniProt Consortium, 2008). The source protein was split into overlapping 9mer peptide sequences, and all peptides except the annotated HLA ligand were taken as negative. When using this definition of positive and negative peptides, one has to take into account that some peptides will falsely be classified as negatives because the SYFPEITHI database is incomplete. Since the MHC class I molecules are very specific, binding only a highly limited repertoire of peptides, this misclassified proportion will, however, be very small (Yewdell and Bennink, 1999). Further, the KISS method does not provide prediction values for peptides included in the training. These peptides were removed from the negative set. For each protein–HLA ligand pair, the predictive performance was estimated as the AUC value (Swets, 1988). This benchmark dataset is referred to as EvaluationSet-3, and is summarized in Table 1 (panel C).

All benchmark datasets 1–3 are available online at http://www.cbs.dtu.dk/immunology/pan-eval.php

2.3 Prediction methods

All predictions were made using public web interfaces with default parameters of the method in question. Table 2 gives a summary of the different methods in terms of data resources and release dates.

Table 2.
Prediction methods

NetMHC-3.0: http://www.cbs.dtu.dk/services/NetMHC/

NetMHCpan-1.0: http://www.cbs.dtu.dk/services/NetMHCpan-1.0/

Adaptive Double Threading:

http://atom.research.microsoft.com/hlabinding/hlabinding.aspx

KISS: http://cbio.ensmp.fr/kiss/

Due to the late release date of the KISS method, it is likely that some overlap between the present evaluation set and the training set exists. Indeed close to 3% of the data in the EvaluationSet-1 are present in the training data for the KISS method. It is hence likely that the performance of the KISS method was slightly overestimated.

2.4 Criteria for performance

Several performance measures can be used to evaluate the prediction performance of a given method. As a first assumption the methods included in the benchmark give predictions that are linearly proportional to the logarithm of the actual measurement of affinity. The intuitive assessment is, therefore, to evaluate the strength of this correlation. Such a quantifiable relationship can be found with the Pearson correlation coefficient (Pearson CC) (Press et al., 1992). In cases where there is no linear relation between the prediction scores and the binding affinity, it is more appropriate to use the Spearman's Rank-order correlation (Spearman's RC) (Press et al., 1992). For an indication of how well the methods can separate peptide binders from peptide non-binders, the area under the receiving operator characteristics curve (AUC) (Swets, 1988) was calculated using the generally accepted affinity threshold of 500 nM.

2.5 Statistical analysis of results

Binomial tests were used to evaluate the statistical significance of the observed difference in predictive performance between the different methods. P-values <0.05 were taken to define a significant result. For each dataset, the number of alleles where one method outperformed one another was counted. Performing a one-tailed binomial test based on this count, and the number of alleles (excluding ties) in the dataset gives the P-value for accepting the null-hypothesis that the first method does not outperform the other. On the other hand, a significant P-value leads to the rejection of the null-hypothesis and to the acceptance of the alternative hypothesis, which states that the first method does consistently outperform the other. Note that some alleles in the benchmark data share highly similar binding specificities, making the assumption that predictions for different alleles are independently false. This could imply that the quoted P-values in some case should be interpreted with caution.

2.6 Consensus method

A consensus method was defined as the simple average of the raw log-transformed prediction scores from the NetMHC-3.0 and NetMHCpan-1.0 methods.

3 RESULTS

The results are divided into three sections according to the dataset used for testing: EvaluationSet-1, EvaluationSet-2, and the HLA ligand set in EvaluationSet-3. Within the first two sections, tests were conducted with the three different criteria for performance: Pearson's CC, Spearman's RC and AUC. For the HLA ligand data only AUC was applied.

3.1 EvaluationSet-1

Figure 1 summarizes the predictive performance of the different methods on the data in EvaluationSet-1.

Fig. 1.
Benchmark performance for EvaluationSet-1. The average predictive performance for the different methods included in the benchmark in terms of the Pearson's CC, the Spearman's RC and the AUC.

The figure suggests the ranking of the different methods as follows: Consensus > NetMHCpan-1.0 > NetMHC-3.0 > ADT > KISS.

The ADT method appears to achieve a higher performance value when measured in terms of the Spearman's RC compared with the Pearson's CC. This could be explained by the fact that the prediction made by the method is a kind of unitless energy, which might not be linearly correlated to the logarithm of the binding affinity. For such methods, the Spearman's RC and the AUC values are therefore more appropriate measures of predictive performance.

The details of the evaluation as estimated in terms of the AUC values is shown in Table 3 (data for the Pearson's CC and Spearman's RC is shown in Supplementary Table S1a and c, respectively).

Table 3.
The performance for EvaluationSet-1 measured in terms of the AUC

From the data in Table 3, it is clear that not one single method was consistently superior in predictive performance to all other methods. By way of example, the NetMHCpan-1.0 method outperformed NetMHC-3.0 for 20 of the 33 alleles, but had a lower or equal performance for the remaining 13 cases.

By performing one-tailed binomial tests on the difference in performance scores, the significance of such observed differences in predictive performance could be assessed. In doing so, the following ranking of the different methods as measured in terms of the AUC values was found. (Similar results were found for the Pearson's and Spearman's Rank correlation).

Consensus > NetMHCpan-1.0NetMHC-3.0 > ADTKISS Here ‘>’ indicates a significant difference (P < 0.05) and ‘≥’ indicates a non-significant difference. The methods can based on this be grouped into (Consensus), (NetMHCpan-1.0, NetMHC-3.0) and (ADT, KISS). The grouping indicates that the Consensus method performed significantly better than both NetMHCpan-1.0 and NetMHC-3.0, and both of the latter performed significantly better than any of the rest; ADT and KISS performed at the same level. Details of the statistical analysis are found in Supplementary Table S1b, d and f.

3.2 EvaluationSet-2

Figure 2 summarizes the predictive performance of the different methods on the data in EvaluationSet-2.

Fig. 2.
Benchmark performance for EvaluationSet-2. The average predictive performance for the different methods included in the benchmark in terms of the Pearson's CC, the Spearman's RC and the AUC.

The figure gives the average predictive performance for the different methods included in the benchmark in terms of the Pearson's CC, the Spearman's RC and the AUC. Details of the evaluation in terms of the AUC values are shown in Table 4 (further evaluation performance values are shown in Supplementary Tables S2a, c and e, respectively).

Table 4.
The performance for EvaluationSet-2 measured in terms of the AUC

For the results shown in Figure 2, the following rank of the different prediction methods was found

Consensus > NetMHCpan-1.0 > NetMHC0-3.0 > ADT > KISS

The significance of this ranking was assessed using a one-tailed binomial test. In terms of the AUC values, the following results were obtained;

Consensus ≥ NetMHCpan-1.0 > NetMHC-3.0 > ADT > KISS

where ‘>’ indicates a significant difference (P < 0.05) and ‘≥’ indicates a non-significant difference. The methods could therefore be grouped as (Consensus, NetMHCpan-1.0), (NetMHC-3.0), (ADT) and (KISS). For the EvaluationSet-2, the analysis thus did not show a significant difference between the Consensus and NetMHCpan-1.0 methods. However, all neural network methods (Consensus, NetMHCpan-1.0 and NetMHC-3.0) were, also for this benchmark, shown to significantly outperform both ADT and KISS methods. Details of the analysis are found in Supplementary Table S2b, d and f.

3.3 Training performance

It is clear from the performance values on the two evaluation sets, that the overall ranking of the different pan-specific methods included in the benchmark for quantitative predictions is Consensus,NetMHCpan-1.0, ADT and KISS. In order to investigate if the higher performance of the NetMHCpan-1.0 method is due to the quality and size of the training data, the predictive performance of the NetMHCpan-1.0 and ADT methods were compared when trained on an identical dataset. The dataset and data partitioning were taken from the work by Peters et al. (2006). The dataset consists of more than 29 000 quantitative HLA peptide-binding data covering 35 HLA-A and HLA-B alleles. The performance was estimated in terms of the AUC value for each allele. The performance values for the ADT method were taken from the ADT server web site. The average performance values were 0.92 and 0.87 for the NetMHCpan-1.0 and ADT methods, respectively. The NetMHCpan-1.0 achieved the highest predictive performance for 34 of the 35 alleles, making this difference highly statistically significant (P < 10−5, binomial test). The details of the analysis are given in Supplementary Table S3. This result strongly indicates that the performance gain of the NetMHCpan-1.0 method is not solely due to a difference in the training data, but also is caused by the algorithmic differences between the two methods. This analysis can further be extended to demonstrate the power of the pan-specific methods to interpolate binding motif information from neighboring HLA molecules, enabling prediction of binding motifs for HLA molecules for which no or only limited binding data exist. Such an analysis has been carried out in detail in the original NetMHCpan-1.0 publication (Nielsen et al., 2007), where it was demonstrated how a pan-specific prediction method could accurately be used to describe the binding motif for hereto uncharacterized HLA molecules. To give further support for the power of the pan-specific approach, Figure 3 shows the predictive performance of the NetMHCpan-1.0, ADT and NetMHC-3.0 methods when all methods are trained on the Peters et al. (2006) dataset and evaluated on the EvaluationSet-1. This figure displays a histogram of the predictive performance of the three methods for the different alleles in the evaluation set as a function of the number of peptide binders in the training data for each allele.

Fig. 3.
Histogram of the average predictive performance of the alleles in the EvaluationSet-1 as a function of the numner of binding pepdite in the training data for same allele. Each methods is trained on the data in the Peters et al. (2006) dataset.

The figure clearly demonstrates that for alleles characterized by a large set of peptide binders (>100) both single allele (NetMHC-3.0) and pan-specific (NetMHCpan-1.0) methods achieved similar predictive accuracy. For alleles characterized by no or limited binding data in the training, the figure on the other hand show the power of the pan-specific NetMHCpan-1.0 method. For these alleles, the single allele method, NetMHC-3.0 failed to produce accurate predictions whereas the pan-specific method, NetMHCpan-1.0, maintained a high accuracy. One extreme allele is the HLA-B *3901 that is absent from the training data. This allele is naturally unpredictable by the NetMHC-3.0 method, but is predicted with an accuracy of 0.26 by the NetMHCpan-1.0 method (data not shown). These results thus clearly demonstrate the power of the pan-specific methods to go beyond the data in the training set and accurately predict binding also for uncharacterized HLA molecules. Intriguingly, it seems that the lower predictive performance of the ADT method is consistent and independent of the number of peptide binding examples in the training data, suggesting that the linear algorithm underlying the ADT method has poorer predictive power compared with the higher order neural network algorithm underlying the NetMHCpan-1.0 method.

To allow for a comparison of the predictive performance of the different methods, a Table S4 was included in Supplementary Material comparing the predictive performance as stated in the original publication, and the corresponding evaluation performance for each allele in EvaluationSet-1. The analysis of the data demonstrates that all methods have around 10% drop in predictive performance when comparing the evaluation set to the training set.

3.4 HLA ligand data

Next, the performance on the HLA ligand EvaluationSet-3 was analyzed. The results of this analysis are shown in Table 5.

Table 5.
Predictive performance for the HLA ligand dataset EvaluationSet-3 dataset as measured in terms of the AUC.

From this table, it is apparent that the KISS method did perform significantly better when it came to HLA ligand identification compared with the prediction of quantitative peptide binding affinities. Taking the average performance per allele, the rank of the three methods is

NetMHCpan-1.0 > (≥) KISS > ADT

Where ‘>’ as before indicates a significant difference (P < 0.05), and ‘≥’ indicates a non-significant difference. The KISS method thus significantly outperformed the ADT method when it came to HLA ligand identification. When looking at the performance per protein, the NetMHCpan-1.0 method significantly outperformed both the other methods (>). However, when grouping the data for the different alleles did the KISS and NetMHCpan-1.0 methods achieve similar predictive performances (≥). There are many possible reasons for this difference in relative rank of the different methods depending on the type of evaluation data. The most important reason is probably the source of training data. The KISS method was trained on HLA ligand data, and thus implicitly incorporates both the potential bias in the data imposed by the experimental method used to detect the ligands as well as signals from other players in the MHC class I presentation pathway like TAP and proteasome. This additional information is not incorporated into the NetMHCpan-1.0 and ADT method, since these are trained only on in vitro derived quantitative peptide:MHC binding data.

4 DISCUSSION

The large amount of peptide binding data made available to the scientific community has recently made the development of so-called pan-specific MHC-binding methods possible. Such methods are capable of providing accurate prediction of the peptide binding strength to any MHC molecule of known protein sequence. Large-scale benchmark evaluations of prediction algorithms for peptide:MHC binding have been performed in several studies for allele-specific methods (method trained on single allele data) (Bui et al., 2005; Lin et al., 2008; Nielsen et al., 2003; Peters et al., 2006; Yu et al., 2002). These studies have provided a ranking of the vast list of methods publicly available for peptide:MHC binding prediction and thereby aiding the non-expert user in selecting which prediction method to use for a given task.

For pan-specific methods, however, no such benchmark has been performed, and it remains unclear to what extent the publicly available methods differ in accuracy. Also, it has not, in an independent evaluation, been evaluated to what extent pan-specific methods can compete with allele-specific methods for alleles already represented in the peptide binding databases.

Here, such a large-scale benchmark has been carried out trying to answer these questions. The predictive performance of three publicly available pan-specific peptide:MHC binding prediction methods NetMHCpan-1.0 (Nielsen et al., 2007), ADT (Jojic et al., 2006) and KISS (Jacob and Vert, 2008) was compared on a large set of quantitative peptide MHC binding data downloaded from the IEDB database. The data were released on the IEDB database post-training of the NetMHCpan-1.0, and ADT methods. Further, to test if the data in the IEDB database had a potential bias due to large data submissions from the authors involved in the NetMHC-3.0 and NetMHCpan-1.0 methods, a second benchmark set was designed excluding data submitted to the IEDB by these authors. In the benchmark, we further included one of the best available allele-specific MHC binding prediction methods, NetMHC-3.0 (Buus et al., 2003; Nielsen et al., 2003) as well as a consensus method defined by integrating the prediction scores from the NetMHCpan-1.0 and NetMHC-3.0 methods. The KISS method was trained on qualitative (categorized) data and in order to investigate to what extent this has implications for which type of the data this method best predicts a qualitative benchmark dataset consisting of HLA ligands downloaded from the SYFPEITHI database was designed.

In both quantitative benchmark calculations it was found consistently that the Consensus and NetMHCpan-1.0 methods perform better than any of the other pan-specific methods. Further, the NetMHCpan-1.0 method performed noticeably better than NetMHC-3.0. Interestingly, the top ranking quantitatively trained method did perform better or equal to the KISS method trained on binary data when tested on this type of data (EvaluationSet-3). The strong predictive power of the pan-specific prediction methods was clearly illustrated in the benchmark calculation on HLA ligand data. In this benchmark, a large fraction of the HLA alleles (42%) were unknown to the NetMHCpan-1.0 method. For these alleles the NetMHCpan-1.0 method achieved an average predictive performance of 0.96 in terms of the AUC value. This AUC value translates into a false positive ratio of 0.04, meaning that in a protein of length 200 amino acids less than 10 peptides will have to be tested in order to identify the ligand.

It is clear when comparing the NetMHC-3.0 and NetMHCpan-1.0 performances on a single allele basis (see Table 2 and Supplementary Material) that the latter had a superior performance especially in the cases where the training data for that particular allele was scarce. For the data in the EvaluationSet-2, it was for instance found that the five alleles with maximal difference in predictive performance between theNetMHC-3.0 and the NetMHCpan-1.0 methods (HLA-B*4002, HLA-B*5301, HLA-A*3002, HLA-B*4001 and HLA-B*5101) all had a number of peptide binders in the training dataset that was consistently less than 100. The NetMHC-3.0 method thus achieved its poorest predictions for alleles covered by limited data (an average of 66 in the examples shown here). Similar observations were found using the data in EvaluationSet-1. Earlier work has shown a similar result, namely that MHC binding prediction algorithms rely on a sufficient number (in the order of 100) of peptide binders to be available in order to achieve high predictive value (Yu et al., 2002). This illustrates very nicely the limits faced when deriving allele-specific predictive methods and strength of the pan-specific approaches like NetMHCpan-1.0, to benefit from data of related alleles.

Funding

National Institute of Health (contracts HHSN2662004 00025C, HHSN266200400083C, and HHSN26620040006C); sixth Framework program of the European commission (grant LSHB-CT-2003-503231).

Conflict of Interest: none declared.

Supplementary Material

[Supplementary Data]

REFERENCES

  • Bhasin M, et al. MHCBN: a comprehensive database of MHC binding and non-binding peptides. Bioinformatics. 2003;19:665–666. [PubMed]
  • Bui HH, et al. Automated generation and evaluation of specific MHC binding predictive tools: ARB matrix applications. Immunogenetics. 2005;57:304–314. [PubMed]
  • Bujnicki JM, et al. LiveBench-2: large-scale automated evaluation of protein structure prediction servers. PROTEINS: Structure, Function, and Genetics Suppl. 2001;5:184–191. [PubMed]
  • Buus S, et al. Sensitive quantitative predictions of peptide-MHC binding by a ‘Query by Committee’ artificial neural network approach. Tissue antigens. 2003;62:378–384. [PubMed]
  • Huang L, et al. A meta-predictor for MHC class II binding peptides based on naive Bayesian approach. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2006;1:5322–5325. [PubMed]
  • Jacob L, Vert JP. Efficient peptide-MHC-I binding prediction for alleles with few known binders. Bioinformatics. 2008;24:358–366. [PubMed]
  • Jojic N, et al. Learning MHC I-peptide binding. Bioinformatics. 2006;22:e227–e235. [PubMed]
  • Karpenko O, et al. A probabilistic meta-predictor for the MHC class II binding peptides. Immunogenetics. 2008;60:25–36. [PubMed]
  • Lin HH, et al. Evaluation of MHC class I peptide binding prediction servers: applications for vaccine research. BMC Immunology. 2008;9:8. [PMC free article] [PubMed]
  • Lundegaard C, et al. Modeling the adaptive immune system: predictions and simulations. Bioinformatics. 2007;23:3265–3275. [PubMed]
  • Lundegaard C, et al. NetMHC-3.0: accurate web accessible predictions of human, mouse and monkey MHC class I affinities for peptides of length 8-11. Nucleic Acids Res. 2008;36(Web Server issue):W509–W512. [PMC free article] [PubMed]
  • Mallios RR. Predicting class II MHC/peptide multi-level binding with an iterative stepwise discriminant analysis meta-algorithm. Bioinformatics. 2001;17:942–948. [PubMed]
  • Nielsen M, et al. Reliable prediction of T-cell epitopes using neural networks with novel sequence representations. Protein Sci. 2003;12:1007–1017. [PMC free article] [PubMed]
  • Nielsen M, et al. NetMHCpan, a method for quantitative predictions of peptide binding to any HLA-A and -B locus protein of known sequence. PLoS ONE. 2007;2:e796. [PMC free article] [PubMed]
  • Peters B, et al. A community resource benchmarking predictions of peptide binding to MHC-I molecules. PLoS Comput. Biol. 2006;2:e65. [PMC free article] [PubMed]
  • Petersen TN, et al. Prediction of protein secondary structure at 80% accuracy. Proteins. 2000;41:17–20. [PubMed]
  • Press WH, et al. Numerical Recipies in C: The Art of Scientific Computing. Cambridge: Cambridge University Press; 1992.
  • Rammensee H, et al. SYFPEITHI: database for MHC ligands and peptide motifs. Immunogenetics. 1999;50:213–219. [PubMed]
  • Sette A, et al. A roadmap for the immunomics of category A-C pathogens. Immunity. 2005;22:155–161. [PubMed]
  • Swets JA. Measuring the accuracy of diagnostic systems. Science. 1988;240:1285–1293. [PubMed]
  • The UniProt Consortium The universal protein resource (UniProt) Nucleic Acids Res. 2008;36(Database issue):D190–D195. [PMC free article] [PubMed]
  • Yewdell JW, Bennink JR. Immunodominance in major histocompatibility complex class I-restricted T lymphocyte responses. Annu. Rev. Immunol. 1999;17:51–88. [PubMed]
  • Yu K, et al. Methods for prediction of peptide binding to MHC molecules: a comparative study. Mol. Med. 2002;8:137–148. [PMC free article] [PubMed]
  • Zhang GL, et al. MULTIPRED: a computational system for prediction of promiscuous HLA binding peptides. Nucleic Acids Res. 2005;33:W172–W179. [PMC free article] [PubMed]

Articles from Bioinformatics are provided here courtesy of Oxford University Press
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • PubMed
    PubMed
    PubMed citations for these articles
  • Substance
    Substance
    PubChem Substance links

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...