TYPES OF OMICS DATA

Examples of the types of omics data that can be used to develop an omics-based test are discussed below. This list is by no means meant to be comprehensive, and indeed a comprehensive list would be impossible because new omics technologies are rapidly developing.

EMERGING OMICS TECHNOLOGIES AND DATA ANALYSIS TECHNIQUES

Many emerging omics technologies are likely to influence the development of omics-based tests in the future, as both the types and numbers of molecular measurements continue to increase. Furthermore, advancing bio-informatics and computational approaches are enabling improved analyses of omics data, such as greater integration of different data types. Given the rapid pace of development in these fields, it is not possible to list all relevant emerging technologies or data analytic techniques. A few illustrative developments are briefly discussed.

Advances in RNA sequencing technology are making possible a higher resolution view of the transcriptome. These new approaches could facilitate the development of more novel molecular diagnostics. In the future it may be possible to develop omics-based tests on the basis of small non-coding RNAs, RNA editing events, or alternative splice variants that were not measured using previous hybridization-based technologies such as microarrays. For example, analysis of miRNA (derived from RNA sequencing) shows great promise for clinical diagnostics (Moussay et al., 2011; Sugatani et al., 2011; Tan et al., 2011; Yu et al., 2008).

Similarly, DNA sequencing is making it possible to identify rare or previously unmeasured mutations that may have important clinical implications. Next-generation sequencing technologies hold tremendous promise for not only identification of complete DNA and RNA sequences, but also high-throughput identification of epigenetic and posttranscriptional modifications to DNA or RNA, respectively. For instance, new sequencing technologies can monitor a wide variety of epigenetic changes at the genomic scale, in addition to sequencing information.

However, it is important to note that because next-generation RNA and DNA sequencing produces even more measurements per sample than do traditional approaches, these new technologies add to the challenge of extremely high data dimensionality and the risks of overfitting computational models to the available data (see the section on Computational Model Development and Cross-Validation for a discussion of overfitting). Large meta-analyses of sequencing datasets collected at multiple sites may prove useful for overcoming these risks and aid in developing clinically useful omics-based tests.

The field of proteomics has benefited from a number of recent advances. One example is the development of selected reaction monitoring (SRM) proteomics based on automated techniques (Picotti et al., 2010). During the past 2 years, multiple peptides distinctive for proteins from each of the 20,300 human protein-coding genes have been synthesized and their mass spectra determined. The resulting SRMAtlas is publicly available for the entire scientific community to use in choosing targets and purchasing peptides for quantitative analyses (Omenn et al., 2011). In addition, data from untargeted “shotgun” mass spectrometry-based proteomics have been collected and uniformly analyzed to generate peptide atlases for plasma, liver, and other organs and biofluids (Farrah et al., 2011).

Meanwhile, antibody-based protein identification and tissue expression studies have progressed considerably (Ayoglu et al., 2011; Fagerberg et al., 2011); the Human Protein Atlas has antibody findings for more than 12,000 of the 20,300 gene-coded proteins. The Protein Atlas is a useful resource for planning experiments and will be enhanced by linkage with mass spectrometry findings through the emerging Human Proteome Project (Legrain et al., 2011).

Recently developed protein capture-agent aptamer chips also can be used to make quantitative measurements of approximately 1,000 proteins from the blood or other sources (Gold et al., 2010). For example, Ostroff et al. (2010) recently reported generation of a 12-protein panel from analysis of 1,100 plasma proteins that was shown to have promising clinical test characteristics for diagnosis of non-small cell lung cancers.

A major bottleneck in the successful deployment of large-scale pro-teomic approaches is the lack of high-affinity capture agents with high sensitivity and specificity for particular proteins (including variants due to posttranslational modifications, alternative splicing, and single-nucleotide polymorphisms or gene fusions). This challenge is exacerbated in highly complex mixtures such as blood, where the concentrations of different proteins vary by more than 10 orders of magnitude. One technology that holds great promise in this regard is “click chemistry” (Service, 2008), which uses a highly specific chemical linkage (generally formed through the Huisgen reaction) to “click” together low-affinity capture agents to create a single capture agent with much higher affinity. It also is feasible to combine computational algorithms for modeling protein structures and conformation to infer functional differences among alternative splice isoforms of proteins, including those involved in key cancer pathways (Menon et al., 2011).

Improving technologies for measurements of small molecules (Drexler et al., 2011) also is enabling the use of metabolomics for the development of candidate omics-based tests with potential clinical utility (Lewis and Gerszten, 2010). Promising early examples include a metabolomic analysis that identified a role for sarcosine, an N-methyl derivative of the amino acid glycine, in prostate cancer progression and metastasis (Sreekumar et al., 2009), metabolomic characterization of ovarian epithelial carcinomas (Ben Sellem et al., 2011), and an integrated metabolomic and proteomic approach to diagnosis, prediction, and therapy selection for heart failure (Arrell et al., 2011). Included within metabolomics is the emerging ability to more fully measure the lipids in a sample, a rich source of additional potential biomarkers (Masoodi et al., 2010). As with other omics data types, a lengthy, complex development path is necessary to establish a clinically relevant omics-based test from reports identifying metabolite concentration differences associated with a phenotype of interest (Koulman et al., 2009).

New technologies are emerging that will make it possible to obtain omics measurements (such as transcriptomics, proteomics) on single cells (Tang et al., 2011; Teague et al., 2010). Such detailed molecular measurements provide deep insight into the underlying biology of tissues, and potentially form a powerful basis for omics-based test development. However, as the resolution of these measurements increases, so too does the variability in the measurements due to the heterogeneity of cell states (Ma et al., 2011). Thus, while emerging omics technologies hold great potential for the development of omics-based tests, they also may exacerbate dangers of overfitting the computational model to the datasets.

Recent interest has focused on measuring multiple omics data types on a single set of samples, in order to integrate different types of molecular measurements into an omics-based test. Such multidimensional datasets have the potential to provide deep insight into biological mechanisms and networks, allowing for the development of more powerful clinical diagnostics. An encouraging example of simultaneous measurement of multiple types of omics data is the DNA-encoded antibody libraries approach (Bailey et al., 2007), which can measure DNA, RNA, and protein from the same sample. Another example is the analysis of histone modifications to identify potential epigenetic biomarkers for prostate cancer prognosis (Bianco-Miotto et al., 2010).

Approaches that integrate multiple omics data types within the same clinical test are expected to grow in importance as the number of simultaneous measurements that can be made continues to increase. While it is relatively straightforward to increase the number of genomic and transcriptomic measurements (because DNA and RNA have complementary binding partners), increasing the number of protein measurements is more challenging because of the need for high-affinity capture agents, as discussed previously in this section.

Systems approaches that integrate multiple data types in functionally based models can be advantageous for the development of omics-based tests. For instance, the analysis of omics measurements in the context of bio molecular networks or pathways can help to reduce the number of variables in the data by constraining the possible relationships between variables, ultimately leading to more robust and clinically useful molecular tests. General approaches for using prior biological knowledge to enhance signal in omics data include removing measurements that are believed to be noise or for which there is no support in the published biological literature (filtering), using pathway databases or other sources to guide model construction, and aggregating individual measurements, often across data types, to integrate multiple sources of evidence to support conclusions (Ideker et al., 2011). For example, in a study of prion-mediated neurodegeneration, data from five mouse strains and three prion strains were used to identify the transcripts, pathways, and networks that were commonly perturbed across all genetic backgrounds (Hwang et al., 2009; Omenn, 2009).

Datasets from genome-wide association studies, in which a set of cases and controls are sampled from a large population and genotyped and each mutation identified is evaluated for association with the phenotype of interest, also can be analyzed within the context of biological pathways in order to increase identification of disease-related mutations (Segre et al., 2010). The incorporation of evolutionarily conserved gene sets can lead to the identification of often unexpected factors in disease (McGary et al., 2010). Large-scale mechanistic network models (for example, for metabolic, regulatory, or signaling networks) may be used to identify biomarkers grounded in disease mechanisms (Folger et al., 2011; Frezza et al., 2011; Gottlieb et al., 2011; Lewis et al., 2010; Shlomi et al., 2011). Genomics, transcriptomics, proteomics, and metabolomics data can be combined with structural protein analysis in order to predict drug targets or even drug off-target effects (Chang et al., 2010). While computational models of bio-molecular networks for eventual clinical use are still in their infancy, their potential for providing stronger mechanistic underpinnings to omics-based test development is encouraging.

During the past 10 years, much of the effort to identify genes linked to disease and other conditions of biological interest has focused on genome-wide association studies. However, more recent work has successfully identified disease-causal genes using whole genome or exome sequencing (Ng et al., 2010; Roach et al., 2010). Such studies may prove to be very beneficial for the development of omics-based tests, and indeed such strategies are being used clinically today for the identification of the causal gene mutation resulting in unidentified and uncommon inherited disease states.

STATISTICS AND BIOINFORMATICS DEVELOPMENT OF OMICS-BASED TESTS

In recent years, a large number of papers have reported new omics-based discoveries and the development of new candidate omics-based tests: that is, computational procedures applied to omics-based measurements to produce a clinically actionable result. However, few of these candidate omics-based tests have progressed to clinical use (Ransohoff, 2008, 2009). Some of this discrepancy may be due to the inevitable time lapse of moving from initial identification of a candidate omics-based test to a precisely defined and validated test that can be used clinically.

However, more important are the many significant challenges in the formulation of appropriate research questions and in research design and conduct that confront the successful discovery of candidate omics-based tests, including the complexity of the data and the need for rigorous analyses, and the frequent lack of a plausible biological mechanism underpinning many of these discoveries. These challenges need to be addressed in order to realize the full clinical potential of omics research, taking into account issues specific to the field as well as broader principles of good scientific research.

Two primary scientific causes for failure of a candidate omics-based test to progress to clinical use are:

1. A candidate omics-based test may not be adequately designed for answering a specific, well-defined, and relevant clinical question. This crucial point is addressed in Chapters 3 and 4.
2. Omics-based discovery studies may not be conducted with adequate statistical or bioinformatics rigor, making it unlikely or even impossible that the candidate omics-based test will prove to be clinically valid or useful. This critical problem is addressed in the remainder of this chapter.

Figure 2-1 highlights the discovery and confirmation of a candidate omics-based test, the first component of the committee’s recommended test development and evaluation process. When candidate omics-based tests from the discovery phase are intended for further clinical development, several criteria should be satisfied and fully disclosed (for example, through publication or patent application) to enable independent verification of the findings (Recommendation 1), as discussed below. For the purpose of this discussion, the committee assumed that a clearly defined and clinically relevant scientific or clinical question or questions have been identified, and that an omics dataset from analyses of a set of patient samples, along with an associated clinical outcome for each patient, is available.

For example, an investigator may ask whether gene expression measurements could be used to predict recurrence in node-negative breast cancer samples in a way that is substantially more accurate than standard clinical prognostic factors, such as tumor size and grade. The investigator might have data consisting of gene expression measurements for breast cancer tissue samples obtained from patients with node-negative breast cancer, along with disease-free survival time for each patient following surgery. The goal would be to develop a defined assay method for data generation and a fully specified computational procedure¹ that can be used to reliably predict, on the basis of gene expression measurements on a new patient sample, whether a patient’s cancer will recur.

Before embarking on omics-based discovery, it is worth considering whether or not the test that will eventually be developed has a reasonable chance of demonstrating clinical validity and utility. For example, the sensitivity and specificity needed, particularly in light of the prevalence of the condition in the population to be tested, should be considered (see also Appendix A, page 209, for a discussion of sensitivity and specificity needs for an ovarian cancer screening test).

Several steps need to be followed to achieve this goal: (1) data quality control; (2) computational model development and cross-validation; (3) confirmation of the computational model on an independent dataset; and (4) release of data, code, and the fully specified computational procedures to the scientific community. Each of these is discussed below.

Step 4: Release of Data, Code, and the Fully Specified Computational Procedures to the Scientific Community

Once an omics-based measurement method and the locked-down computational procedures have been shown to perform well on an independent dataset (Step 3), the candidate omics-based test is ready to proceed to the test validation phase in which analytical and clinical/biological validity are assessed, as described in detail in Chapter 3. At this time, data and metadata used for development of the candidate omics-based test should be made available in an independently managed database (e.g., the database of Genotypes and Phenotypes [dbGaP]) in standard format (Recommendation 1b). dbGaP is designed to archive and distribute data from genome-wide association studies to examine genetic associations with phenotype and disease characteristics (Mailman et al., 2007; NLM, 2006).

In addition, computer code and fully specified computational procedures used for development of the candidate omics-based test should be made sustainably available (Recommendation 1c). For publicly funded research this means that computer code and fully specified computational procedures should be made publicly available either at the time of publication or at the end of funding. For commercially developed tests, code and fully specified computational procedures will be submitted to the Food and Drug Administration (FDA) for review if the developers are seeking FDA approval or clearance. In the case of a laboratory-developed test (LDT, see Chapter 3), publication is essentially required because laboratories need to justify the claims for the test. As indicated in chapter 5, the committee also recommends that journals require authors to make data, metadata, pre-specified analysis plans, computer code, and fully specified computational procedures publicly available (Recommendation 7a[ii]). (See also the discussion on data availability and transparency in Chapter 5.)

Ideally, the computer code that is released will encompass all of the steps of computational analysis, including all data preprocessing steps, that have been described in this chapter. All aspects of the analysis need to be transparently reported. If some aspect of the code or data cannot be made available, then the reason for this omission should be documented.

Release of data, code, and the fully specified computational procedures is important for independent verification of results. This recommendation reinforces the call for transparency in reporting made in several National Research Council reports (NRC, 2003, 2005, 2006). Others have also recently called for wider access to the full data, protocols, and computer codes for published omics studies (Ince et al., 2012; Ioannidis and Khoury, 2011; Morin et al., 2012), despite the known challenges to broad disclosure (Box 2-1). In the omics setting, release of this information is particularly crucial because:

BOX 2-1

Considerations in Data and Information Sharing. Test developers might be reluctant to share their data, code, and computational procedures out of concern that others may be able to capitalize on their research. Intellectual property protections provide (more...)

1.

The complex nature of the data and analyses can make it difficult to replicate results, but rigorous verification of results by the scientific community is necessary to ensure that candidate omics-based tests are scientifically and statistically valid; and

2.

If data are made available, then subsequent investigators can continue to conduct additional analyses to obtain future scientific insights.

Listed below are characteristics of candidate omics-based test concepts that pose risks for development of a spurious test. Some of these issues have already been mentioned. Others are not specific to omics-based test development, but are nonetheless extremely important for the development of such tests.

1.

High dimensionality of the data: Omics datasets are typically characterized by measurements (e.g., genes or gene products such as RNA or proteins) that are many orders of magnitude greater than the number of samples for which clinical data are available. This can lead to overfitting of the data unless proper measures, such as cross-validation combined with confirmation on an independent dataset, are performed. The initial MammaPrint studies were criticized for improper cross-validation and mixing training samples with the independent dataset used for confirmation (discussed in Appendix A).

2.

Biological plausibility: While lack of a known plausible biological mechanism should not, in itself, prevent an investigator from moving forward with a candidate omics-based test, such tests are more likely to endure further investigation if there is a plausible biological mechanism behind them. For instance, an effect modifier for breast cancer recurrence that is based on a set of genes known to be involved in breast cancer is more likely to hold up in further study than an effect modifier based on some set of genes not already known to be implicated in the disease. Better results can often be obtained by beginning the omics-based test development process using a subset of the omics measurements for which a plausible biological mechanism is available. For instance, there was a plausible biological mechanism behind the HER2 tests and Oncotype DX to motivate their initial clinical trials, but less so for the Duke, MammaPrint, and Ova1 tests (discussed in Appendix A and B). Bioinformatics methods to link transcript or protein expression changes to relevant signaling pathways or biological networks need to be deployed appropriately.

3.

Data variability unrelated to clinical outcome of interest: Often, a computational model developed on one dataset (Step 2) performs poorly on another independent dataset (Step 3). This can occur for a number of reasons, such as variability in patient population, sample preparation, time of sample collection, operator variability, etc. Hence, evidence of a computational model’s performance based only on the dataset used to train the model, even if cross-validation is properly performed, provides little evidence of the model’s suitability for future samples. A relevant example here is the OvaCheck case study, discussed in Appendix A, in which signals obtained on one dataset did not hold up when the analysis was applied to other independent sample sets (Baggerly et al., 2004).

4.

Need for multiple datasets: For the reasons just described, computational models that are fit on multiple datasets in Step 2 will tend to perform better later. In other words, investigators are urged to develop a computational model on omics datasets derived from specimens and associated clinical outcomes collected at multiple laboratories at multiple institutions, rather than fitting a model on just a single dataset. For instance, the 21-Gene Recurrence Score (Oncotype DX) case study (Appendix A) was developed using multiple independent datasets (Paik et al., 2004). In that case, data were analyzed by the same investigators, but different datasets were derived from different clinical trials at multiple institutions.

5.

Study design and batch effects: As in all areas of biomedical research, good study design is crucial. If the dataset used in Step 2 to develop the computational model resulted from poor experimental design (e.g., if the samples from patients whose cancers recurred were processed at a different time or by a different technician or in a different laboratory) then batch effects (Leek et al., 2010) can occur. This will lead to spurious signal, potentially resulting in a computational model that performs extremely well on the data on which it was developed (Step 2), but that will perform poorly on future patient samples (Step 3). A relevant example is the OvaCheck case study, discussed in Appendix A, in which peaks in the noise regions of the proteomic spectra could distinguish samples from controls and cancer, indicating batch effects (Baggerly et al., 2004).

6.

Computational procedure lock-down: It is crucial that at the end of Step 2, the fully specified computational procedures be locked down before progressing into confirmation on an independent test set in Step 3. For instance, simply reporting the set of genes included in the computational model underlying a transcriptomics-based test is insufficient, because this does not constitute a fully specified computational procedure. In the original Oncotype DX study, the researchers locked down the computational model after Step 2 and reported the fully specified computational procedures in the paper (Paik et al., 2004). In the Corus CAD case study, lock-down and the fully specified computational procedures were reported in the clinical validation paper (Rosenberg et al., 2010). The fully specified computational procedures for the AlloMap test were reported in Deng et al. (2006). In contrast, in the Duke studies, the genes used in the development of the computational model were reported, but the fully specified computational procedures were not; furthermore, it is likely that the computational procedures were not ever fully locked down before proceeding into Step 3 or further stages of omics-based test development, including clinical trials (see Appendix B for details).

7.

Role of biostatistics and bioinformatics experts: In a relatively new and evolving field such as omics, it is not possible to predict all the possible pitfalls that investigators may face in the discovery phase. The involvement of properly trained biostatistical or bioinformatics collaborators who are fully integrated in all aspects of the discovery and evaluation process can serve as an additional safeguard. The type of biostatistical expertise that is required may vary depending on the stage or phase of test development. For example, experts in developing computational models for omics-based tests may not have sufficient expertise in clinical trial design, and vice versa. This is relevant to the Duke case study (as discussed in Appendix B), in which there was a lack of continuity in bio statistics personnel and numerous errors were identified in the statistical methodology and analyses.

COMPLETION OF THE DISCOVERY PHASE OF OMICS-BASED TEST DEVELOPMENT

A candidate omics-based test should be defined precisely, including the molecular measurements, the computational procedures, and the intended clinical use of the test, in anticipation of the test validation phase (Recommendation 1d). There are enormous opportunities in the rapidly improving suite of omics technologies to identify measurements with potential clinical utility. However, there are significant challenges in moving from the initial identification of potentially relevant differences in omics measurements to validated and robust clinical tests. Among these challenges are risks of overfitting the data in the development of the computational model and the enormous heterogeneity among different studies of ostensibly the same disease states (for both technical and biological reasons). Going forward, transparency in the reporting of all aspects of the development of an omics-based test, including the measurements made, preprocessing techniques used, and the fully specified computational procedure, is critical. The release of sufficient metadata with publication is also key to the identification of candidate omics-based tests that work across multiple sites, which is necessary to generate increasingly robust omics-based tests to enhance patient care.

In the next phase of test development (analytical and clinical/biological validation, described in Chapter 3), the methods used to obtain the omics measurements from patient samples may be changed in order to establish a clinically feasible, inexpensive, and robust assay for implementation in clinical practice. However, the fully specified computational procedures defined in the discovery stage must remain locked down and unchanged in all subsequent test development steps. At the end of the validation phase in Chapter 3, the complete test method, including the methods for obtaining the omics measurements as well as the fully specified computational procedures, must be locked down before crossing the bright line to evaluate the test for clinical utility and use.

SUMMARY AND RECOMMENDATION

This chapter has outlined best practices for the discovery phase for omics-based test development. Because omics-based tests rely on interpretation of high-dimensional datasets, it is important to guard against over-fitting the data throughout the test development process. Overfitting due to lack of proper statistical methods can lead to a model that fits the training samples well, even though the model might perform poorly on independent samples not used in test development. The steps delineated in this chapter aim to prevent an overfit model from progressing to subsequent stages of test development. Cross-validation or a training set/test set approach can help reduce the risk of overfitting, but confirmation of all fully specified computational procedures and candidate omics-based tests on a blinded independent sample set is the “gold standard” for assessing the validity of any test. The importance of independent confirmation is also emphasized in the committee’s recommendations for funders (see Chapter 5), which urge funders to support this type of work. In addition, complex analyses of these large datasets highlight the need for availability of the data and code used for the discovery phase of omics-based test development, to enable independent verification of the findings. The result of the discovery process is a candidate omics-based test with locked-down computational procedures that is then moved into the test validation phase to assess analytical and clinical/biological validation, as described in Chapter 3.

RECOMMENDATION 1: Discovery Phase

When candidate omics-based tests from the discovery phase are intended for further clinical development, the following criteria should be satisfied and fully disclosed (for example, through publication or patent application) to enable independent verification of the findings:

a.

Candidate omics-based tests should be confirmed using an independent set of samples, not used in the generation of the computational model and, when feasible, blinded to any outcome or other phenotypic data until after the computational procedures have been locked down and the candidate omics-based test has been applied to the samples;

b.

Data and metadata used for development of the candidate omicsbased test should be made available in an independently managed database (such as dbGaP) in standard format;

c.

Computer code and fully specified computational procedures used for development of the candidate omics-based test should be made sustainably available; and

d.

A candidate omics-based test should be defined precisely, includ ing the molecular measurements, the computational procedures, and the intended clinical use of the test, in anticipation of the test validation phase.

REFERENCES

• Ahrens C. H., Brunner E., Qeli E., Basler K., Aebersold R. Generating and navigating proteome maps using mass spectrometry. Nature Reviews Molecular Cell Biology. 2010;11(11):789–801. [PubMed: 20944666]
• Arrell D. K., Zlatkovic Lindor J., Yamada S., Terzic A. K(ATP) channel-dependent metaboproteome decoded: Systems approaches to heart failure prediction, diagnosis, and therapy. Cardiovascular Research. 2011;90(2):258–266. [PMC free article: PMC3078801] [PubMed: 21321057]
• Ayoglu B., Haggmark A., Neiman M., Igel U., Uhlen M., Schwenk J. M., Nilsson P. Systematic antibody and antigen-based proteomic profiling with microarrays. Expert Review of Molecular Diagnostics. 2011;11(2):219–234. [PubMed: 21405972]
• Baggerly K. A., Morris J. S., Coombes K.R. Reproducibility of SELDI-TOF protein patterns in serum: Comparing datasets from different experiments. Bioinformatics. 2004;20(5):777–785. [PubMed: 14751995]
• Bailey R. C., Kwong G. A., Radu C. G., Witte O. N., Heath J.R. DNA-encoded antibody libraries: A unified platform for multiplexed cell sorting and detection of genes and proteins. Journal of the American Chemical Society. 2007;129(7):1959–1967. [PMC free article: PMC3677962] [PubMed: 17260987]
• Ben Sellem D., Elbayed K., Neuville A., Moussallieh F. M., Lang-Averous G., Piotto M., Bellocq J. P., Namer I.J. Metabolomic characterization of ovarian epithelial carcinomas by HRMAS-NMR spectroscopy. Journal of Oncology. 2011;2011:174019. [PMC free article: PMC3090613] [PubMed: 21577256]
• Bianco-Miotto T., Chiam K., Buchanan G., Jindal S., Day T. K., Thomas M., Pickering M. A., O’Loughlin M. A., Ryan N. K., Raymond W. A., Horvath L. G., Kench J. G., Stricker P. D., Marshall V. R., Sutherland R. L., Henshall S. M., Gerald W. L., Scher H. I., Risbridger G. P., Clements J. A., Butler L. M., Tilley W. D., Horsfall D. J., Ricciardelli C. Global levels of specific histone modifications and an epigenetic gene signature predict prostate cancer progression and development. Cancer Epidemiology, Biomarkers, & Prevention. 2010;19(10):2611–2622. [PubMed: 20841388]
• Chang R. L., Xie L., Bourne P. E., Palsson B.O. Drug off-target effects predicted using structural analysis in the context of a metabolic network model. PLoS Computational Biology. 2010;6(9):e1000938. [PMC free article: PMC2950675] [PubMed: 20957118]
• Compendia Bioscience, Inc. Compendia Bioscience: Cure Cancer with Genomic Data. 2012. http://www​.compendiabio.com/ (accessed February 23, 2012)
• Deng M. C., Eisen H. J., Mehra M. R., Billingham M., Marboe C. C., Berry G., Kobashigawa J., Johnson F. L., Starling R. C., Murali S., Pauly D. F., Baron H., Wohlgemuth J. G., Woodward R. N., Klingler T. M., Walther D., Lal P. G., Rosenberg S., Hunt S. for the CARGO Investigators. Noninvasive discrimination of rejection in cardiac allo-graft recipients using gene expression profiling. American Journal of Transplantation. 2006;6(1):150–160. [PubMed: 16433769]
• Drexler D. M., Reily M. D., Shipkova P.A. Advances in mass spectrometry applied to pharmaceutical metabolomics. Analytical and Bioanalytical Chemistry. 2011;399(8):2645–2653. [PubMed: 21107980]
• EBI (European Bioinformatics Institute). Data Resources and Tools. 2012. http://www​.ebi.ac.uk/ (accessed February 23, 2012)
• Fagerberg L., Stromberg S., El-Obeid A., Gry M., Nilsson K., Uhlen M., Ponten F., Asplund A. Large-scale protein profiling in human cell lines using antibody-based proteomics. Journal of Proteome Research. 2011;10(9):4066–4075. [PubMed: 21726073]
• Farrah T., Deutsch E. W., Omenn G. S., Campbell D. S., Sun Z., Bletz J. A., Mallick P., Katz J. E., Malmström J., Ossola R., Watts J. D., Lin B., Zhang H., Moritz R. L., Aebersold R. A high-confidence human plasma proteome reference set with estimated concentrations in PeptideAtlas. (M110.006353).Molecular and Cellular Proteomics. 2011;10(9) [PMC free article: PMC3186192] [PubMed: 21632744]
• Folger O., Jerby L., Frezza C., Gottlieb E., Ruppin E., Shlomi T. Predicting selective drug targets in cancer through metabolic networks. Molecular Systems Biology. 2011;7:501–527. [PMC free article: PMC3159974] [PubMed: 21694718]
• Frezza C., Zheng L., Folger O., Rajagopalan K. N., Mackenzie E. D., Jerby L., Micaroni M., Chaneton B., Adam J., Hedley A., Kalna G., Tomlinson I. P., Pollard P. J., Watson D. G., Deberardinis R. J., Shlomi T., Ruppin E., Gottlieb E. Haem oxygenase is synthetically lethal with the tumour suppressor fumarate hydratase. Nature. 2011;477(7363):225–228. [PubMed: 21849978]
• Gold L., Ayers D., Bertino J., Bock C., Bock A., Brody E. N., Carter J., Dalby A. B., Eaton B. E., Fitzwater T., Flather D., Forbes A., Foreman T., Fowler C., Gawande B., Goss M., Gunn M., Gupta S., Halladay D., Heil J., Heilig J., Hicke B., Husar G., Janjic N., Jarvis T., Jennings S., Katilius E., Keeney T. R., Kim N., Koch T. H., Kraemer S., Kroiss L., Le N., Levine D., Lindsey W., Lollo B., Mayfield W., Mehan M., Mehler R., Nelson S. K., Nelson M., Nieuwlandt D., Nikrad M., Ochsner U., Ostroff R. M., Otis M., Parker T., Pietrasiewicz S., Resnicow D. I., Rohloff J., Sanders G., Sattin S., Schneider D., Singer B., Stanton M., Sterkel A., Stewart A., Stratford S., Vaught J. D., Vrkljan M., Walker J. J., Watrobka M., Waugh S., Weiss A., Wilcox S. K., Wolfson A., Wolk S. K., Zhang C., Zichi D. Aptamer-based multiplexed proteomic technology for biomarker discovery. PLoS One. 2010;5(12):e15004. [PMC free article: PMC3000457] [PubMed: 21165148]
• Gottlieb A., Stein G. Y., Ruppin E., Sharan R. PREDICT: A method for inferring novel drug indications with application to personalized medicine. Molecular Systems Biology. 2011;7:496. [PMC free article: PMC3159979] [PubMed: 21654673]
• Honda K., Littman D.R. Annual Review of Immunology. Mar 24, 2011. 2011. The microbiome in infectious disease and inflammation. [Epub ahead of print]
• Hwang D., Lee I. Y., Yoo H., Gehlenborg N., Cho J. H., Petritis B., Baxter D., Pitstick R., Young R., Spicer D., Price N. D., Hohmann J. G., Dearmond S. J., Carlson G. A., Hood L.E. A systems approach to prion disease. Molecular Systems Biology. 2009;5:252. [PMC free article: PMC2671916] [PubMed: 19308092]
• Ideker T., Dutkowski J., Hood L. Boosting signal-to-noise in complex biology: Prior knowledge is power. Cell. 2011;144(6):860–863. [PMC free article: PMC3102020] [PubMed: 21414478]
• Ince D. C., Hatton L., Graham-Cumming J. The case for open computer programs. Nature. 2012;482:485–488. [PubMed: 22358837]
• Ioannidis J. P. A., Khoury M.J. Improving validation practices in “omics” research. Science. 2011;334(6060):1230–1232. [PMC free article: PMC5624327] [PubMed: 22144616]
• IOM (Institute of Medicine). Beyond the HIPAA Privacy Rule: Enhancing Privacy, Improving Health through Research. Washington, DC: The National Academies Press; 2009. [PubMed: 20662116]
• Kinros J. M., Darzi A. W., Nicholson J.K. Gut microbiome-host interactions in health and disease. Genomic Medicine. 2011;3(3):14. [PMC free article: PMC3092099] [PubMed: 21392406]
• Koulman A., Lane G. A., Harrison S. J., Volmer D.A. From differentiating metabolites to biomarkers. Analytical and Bioanalytical Chemistry. 2009;394(3):663–670. [PMC free article: PMC2865640] [PubMed: 19277615]
• Leek J. T., Scharpf R. B., Bravo H. C., Simcha D., Langmead B., Johnson W. E., Geman D., Baggerly K., Irizarry R.A. Tackling the widespread and critical impact of batch effects in high-throughput data. Nature Reviews Genetics. 2010;11(10):733–739. [PMC free article: PMC3880143] [PubMed: 20838408]
• Legrain P., Aebersold R., Archakov A., Bairoch A., Bala K., Beretta L., Bergeron J., Borchers C. H., Corthals G. L., Costello C. E., Deutsch E. W., Domon B., Hancock W., He F., Hochstrasser D., Marko-Varga G., Salekdeh G. H., Sechi S., Snyder M., Srivastava S., Uhlen M., Wu C. H., Yamamoto T., Paik Y. K., Omenn G.S. The human proteome project: Current state and future direction. (009993).Molecular & Cellular Proteomics. 2011;10(7):M111. [PMC free article: PMC3134076] [PubMed: 21742803]
• Lewis G. D., Gerszten R.E. Toward metabolomic signatures of cardiovascular disease. Circulation: Cardiovascular Genetics. 2010;3(2):119–121. [PMC free article: PMC3010400] [PubMed: 20407099]
• Lewis N. E., Schramm G., Bordbar A., Schellenberger J., Andersen M. P., Cheng J. K., Patel N., Yee A., Lewis R. A., Eils R., Konig R., Palsson B.O. Large-scale in silico modeling of metabolic interactions between cell types in the human brain. Nature Biotechnology. 2010;28(12):1279–1285. [PMC free article: PMC3140076] [PubMed: 21102456]
• Ma C., Fan R., Ahmad H., Shi Q., Comin-Anduix B., Chodon T., Koya R. C., Liu C. C., Kwong G. A., Radu C. G., Ribas A., Heath J.R. A clinical microchip for evaluation of single immune cells reveals high functional heterogeneity in phenotypically similar T cells. Nature Medicine. 2011;17(6):738–743. [PMC free article: PMC3681612] [PubMed: 21602800]
• Maas S. Gene regulation through RNA editing. Discovery Medicine. 2010;10(54):379–386. [PubMed: 21122469]
• Mailman M. D., Feolo M., Jin Y., Kimura M., Tryka K., Bagoutdinov R., Hao L., Kiang A., Paschall J., Phan L., Popova N., Pretel S., Ziyabari L., Lee M., Shao Y., Wang Z. Y., Sirotkin K., Ward M., Kholodov M., Zbicz K., Beck J., Kimelman M., Shevelev S., Preuss D., Yaschenko E., Graeff A., Ostell J., Sherry S.T. The NCBI dbGaP database of genotypes and phenotypes. Nature Genetics. 2007;39:1181–1186. [PMC free article: PMC2031016] [PubMed: 17898773]
• Masoodi M., Eiden M., Koulman A., Spaner D., Volmer D.A. Comprehensive lipidomics analysis of bioactive lipids in complex regulatory networks. Analytical Chemistry. 2010;82(19):8176–8185. [PubMed: 20828216]
• McGary K. L., Park T. J., Woods J. O., Cha H. J., Wallingford J. B., Marcotte E.M. Systematic discovery of nonobvious human disease models through orthologous phenotypes. Proceedings of the National Academy of Sciences. 2010;107(14):6544–6549. [PMC free article: PMC2851946] [PubMed: 20308572]
• McShane L. M. NCI Address to Institute of Medicine Committee Convened to Review Omics-Based Tests for Predicting Patient Outcomes in Clinical Trials; Washington, DC: Dec 20, 2010.
• Menon R., Roy A., Mukerjee S., Belkin S., Zhang Y., Omenn G.S. Journal of Proteome Research. 2011. Functional implications of structural predictions for alternative splice proteins expressed in HER2/ neu-induced breast cancers. [Epub ahead of print] [PMC free article: PMC3230717] [PubMed: 22003824]
• Morin A., Urban J., Adams P. D., Foster I., Sali A., Baker D., Sliz P. Research priorities. Shining light into black boxes. Science. 2012;336(6078):159–160. [PMC free article: PMC4203337] [PubMed: 22499926]
• Moussay E., Wang K., Cho J. H., van Moer K., Pierson S., Paggetti J., Nazarov P. V., Palissot V., Hood L. E., Berchem G., Galas D.J. MicroRNA as biomarkers and regulators in B-cell chronic lymphocytic leukemia. Proceedings of the National Academy of Sciences. 2011;108(16):6573–6578. [PMC free article: PMC3081030] [PubMed: 21460253]
• NCBI (National Center for Biotechnology Information). dbGaP. 2012. http://www​.ncbi.nlm.nih.gov/gap (accessed February 23, 2012)
• Ng S. B., Bigham A. W., Buckingham K. J., Hannibal M. C., McMillin M. J., Gildersleeve H. I., Beck A. E., Tabor H. K., Cooper G. M., Mefford H. C., Lee C., Turner E. H., Smith J. D., Rieder M. J., Yoshiura K., Matsumoto N., Ohta T., Niikawa N., Nickerson D. A., Bamshad M. J., Shendure J. Exome sequencing identifies MLL2 mutations as a cause of Kabuki syndrome. Nature Genetics. 2010;42(9):790–793. [PMC free article: PMC2930028] [PubMed: 20711175]
• NLM (National Library of Medicine). 2006. NIH launches dbGAP, a database of Genome Wide Association Studies. http://www​.nlm.nih.gov​/news/press_releases/dbgap_launchPR06​.html (accessed December 12, 2006)
• NLM. 2012. GEO: Gene expression omnibus. http://www​.ncbi.nlm.nih.gov/geo/ (accessed February 23, 2012)
• NRC (National Research Council). A Question of Balance: Private Rights and them Public Interest in Scientific and Technical Databases. Washington, DC: National Academy Press; 1999.
• NRC. Sharing Publication-Related Data and Materials: Responsibilities of Authorship in the Life Sciences. Washington, DC: The National Academies Press; 2003. [PubMed: 22649805]
• NRC. Catalyzing Inquiry at the Interface of Computing and Biology. Washington, DC: The National Academies Press; 2005. [PubMed: 20669460]
• NRC. Reaping the Benefits of Genomic and Proteomic Research: Intellectual Property Rights, Innovation, and Public Health. Washington, DC: The National Academies Press; 2006. [PubMed: 20669435]
• Omenn G. S. A landmark systems analysis of prion disease of the brain. Molecular Systems Biology. 2009;5:254. [PMC free article: PMC2671917] [PubMed: 19308093]
• Omenn G. S., Baker M. S., Aebersold R. Recent workshops of the HUPO Human Plasma Proteome Project (HPPP): A bridge with the HUPO CardioVascular Initiative and the emergence of SRM targeted proteomics. Proteomics. 2011;11(17):3439–3443. [PubMed: 21850651]
• Ostroff R. M., Bigbee W. L., Franklin W., Gold L., Mehan M., Miller Y. E., Pass H. I., Rom W. N., Siegfried J. M., Stewart A., Walker J. J., Weissfeld J. L., Williams S., Zichi D., Brody E.N. Unlocking biomarker discovery: Large scale application of aptamer proteomic technology for early detection of lung cancer. PLoS One. 2010;5(12):e15003. [PMC free article: PMC2999620] [PubMed: 21170350]
• Paik S., Shak S., Tang G., Kim C., Baker J., Cronin M., Baehner F. L., Walker M. G., Watson D., Park T., Hiller W., Fisher E. R., Wickerham D. L., Bryant J., Wolmark N. A multigene assay to predict recurrence of tamoxifen-treated, node-negative breast cancer. New England Journal of Medicine. 2004;351(27):2817–2826. [PubMed: 15591335]
• Petricoin E. F., Ardekani A. M., Hitt B. A., Levine P. J., Fusaro V. A., Steinberg S. M., Mills G. B., Simone C., Fishman D. A., Kohn E. C., Liotta L.A. Use of proteomic patterns in serum to identify ovarian cancer. Lancet. 2002;359(9306):572–577. [PubMed: 11867112]
• Picotti P., Rinner O., Stallmach R., Dautel F., Farrah T., Domon B., Wenschuh H., Aebersold R. High-throughput generation of selected reaction-monitoring assays for proteins and proteomes. Nature Methods. ProteomeXchange. 2012 Mission. 2010;7(1):43–46. http://www​.proteomexchange.org/ (accessed February 23, 2012) [PubMed: 19966807]
• Ransohoff D. F. The process to discover and develop biomarkers for cancer: A work in progress. Journal of the National Cancer Institute. 2008;100(20):1419–1420. [PubMed: 18840815]
• Ransohoff D. F. Promises and limitations of biomarkers. Recent Results in Cancer Research. 2009;181:55–59. [PubMed: 19213557]
• Roach J. C., Glusman G., Smit A. F., Huff C. D., Hubley R., Shannon P. T., Rowen L., Pant K. P., Goodman N., Bamshad M., Shendure J., Drmanac R., Jorde L. B., Hood L., Galas D.J. Analysis of genetic inheritance in a family quartet by whole-genome sequencing. Science. 2010;328(5978):636–639. [PMC free article: PMC3037280] [PubMed: 20220176]
• Rosenberg S., Elashoff M. R., Beineke P., Daniels S. E., Wingrove J. A., Tingley W. G., Sager P. T., Sehnert A. J., Yau M., Kraus W. E., Newby K., Schwartz R. S., Voros S., Ellis S. G., Tahirkhelli N., Waksman R., McPherson J., Lansky A., Winn M. E., Schork N. J., Topol E. J. and for the PREDICT (Personalized Risk Evaluation and Diagnosis In the Coronary Tree) Investigators. Multicenter validation of the diagnostic accuracy of a blood-based gene expression test for assessing obstructive coronary artery disease in nondiabetic patients. Annals of Internal Medicine. 2010;153(7):425–434. [PMC free article: PMC3786733] [PubMed: 20921541]
• Segre A. V., Groop L., Mootha V. K., Daly M. J., Altshuler D. Common inherited variation in mitochondrial genes is not enriched for associations with type 2 diabetes or related glycemic traits. PLoS Genet. 2010;6(8):e1001058. [PMC free article: PMC2920848] [PubMed: 20714348]
• Seppanen-Laakso T., Oresic M. How to study lipidomes. Journal of Molecular Endocrinology. 2009;42(3):185–190. [PubMed: 19060177]
• Service R. F. Chemistry. Click chemistry clicks along. Science. 2008;320(5878):868–869. [PubMed: 18487169]
• Shlomi T., Benyamini T., Gottlieb E., Sharan R., Ruppin E. Genome-scale metabolic modeling elucidates the role of proliferative adaptation in causing the Warburg effect. PLoS Computational Biology. 2011;7(3):e1002018. [PMC free article: PMC3053319] [PubMed: 21423717]
• Simon R., Radmacher M. D., Dobbin K., McShane L.M. Pitfalls in the use of DNA microarray data for diagnostic and prognostic classification. Journal of the National Cancer Institute. 2003;95(1):14–18. [PubMed: 12509396]
• Siva N. Myriad wins BRCA1 row. Nature Biotechnology. 2009;27:8.
• Sreekumar A., Poisson L. M., Rajendiran T. M., Khan A. P., Cao Q., Yu J., Laxman B., Mehra R., Lonigro R. J., Li Y., Nyati M. K., Ahsan A., Kalyana-Sundaram S., Han B., Cao X., Byun J., Omenn G. S., Ghosh D., Pennathur S., Alexander D. C., Berger A., Shuster J. R., Wei J. T., Varambally S., Beecher C., Chinnaiyan A.M. Metabolomic profiles delineate potential role for sarcosine in prostate cancer progression. Nature. 2009;457(7231):910–914. [PMC free article: PMC2724746] [PubMed: 19212411]
• Sugatani T., Vacher J., Hruska K.A. A microRNA expression signature of osteo-clastogenesis. Blood. 2011;117(13):3648–3657. [PMC free article: PMC3072882] [PubMed: 21273303]
• Tan X., Qin W., Zhang L., Hang J., Li B., Zhang C., Wan J., Zhou F., Shao K., Sun Y., Wu J., Zhang X., Qiu B., Li N., Shi S., Feng X., Zhao S., Wang Z., Zhao X., Chen Z., Mitchelson K., Cheng J., Guo Y., He J. A five-microRNA signature for squamous cell lung carcinoma (SCC) diagnosis and Hsa-miR-31 for SCC prognosis. Clinical Cancer Research. 2011;17(21):6802–6811. [PubMed: 21890451]
• Tang F., Lao K., Surani M.A. Development and applications of single-cell transcriptome analysis. Nature Methods. 2011;8(4 Suppl):S6–S11. [PMC free article: PMC3408593] [PubMed: 21451510]
• Teague B., Waterman M. S., Goldstein S., Potamousis K., Zhou S., Reslewic S., Sarkar D., Valouev A., Churas C., Kidd J. M., Kohn S., Runnheim R., Lamers C., Forrest D., Newton M. A., Eichler E. E., Kent-First M., Surti U., Livny M., Schwartz D.C. High-resolution human genome structure by single-molecule analysis. Proceedings of the National Academy of Sciences. 2010;107(24):10848–10853. [PMC free article: PMC2890719] [PubMed: 20534489]
• Tilg H., Kaser A. Gut microbiome, obesity, and metabolic dysfunction. Journal of Clinical Investigations. 2011;121(6):2126–2132. [PMC free article: PMC3104783] [PubMed: 21633181]
• UCSC (University of California, Santa Cruz). UCSC genome bioinformatics. 2012. http://genome​.ucsc.edu/ (accessed February 23, 2012)
• Weckwerth W. Metabolomics in systems biology. Annual Review of Plant Biology. 2003;54:669–689. [PubMed: 14503007]
• Wolf-Yadlin A., Sevecka M., MacBeath G. Dissecting protein function and signaling using protein microarrays. Current Opinion in Chemical Biology. 2009;13(4):398–405. [PMC free article: PMC2811369] [PubMed: 19660979]
• Yu S. L., Chen H. Y., Chang G. C., Chen C. Y., Chen H. W., Singh S., Cheng C. L., Yu C. J., Lee Y. C., Chen H. S., Su T. J., Chiang C. C., Li H. N., Hong Q. S., Su H. Y., Chen C. C., Chen W. J., Liu C. C., Chan W. K., Chen W. J., Li K. C., Chen J. J., Yang P.C. MicroRNA signature predicts survival and relapse in lung cancer. Cancer Cell. 2008;13(1):48–57. [PubMed: 18167339]
• Zhang G. F., Sadhukhan S., Tochtrop G. P., Brunengraber H. Metabolomics, pathway regulation, and pathway discovery. Journal of Biological Chemistry. 2011;286(27):23631–23635. [PMC free article: PMC3129142] [PubMed: 21566142]