Chapter  160:  Impact of Gene Expression Profiling Tests on Breast Cancer Outcomes

Literature on Key Questions

Key Question 1. What is the direct evidence that gene expression profiling tests in women diagnosed with breast cancer (or any specific subset of this population) lead to improvement in outcomes?

Direct evidence was defined as a study where the primary intervention is the use of a prognostic test (with therapeutic decisionmaking directed by the result) and the outcomes are patient morbidity, mortality and/or quality of life. No direct evidence was found in the published data on improvement of patients' outcomes due to such testing in women diagnosed with breast cancer, nor were there any randomized studies using the tests' predictions to manage patients. However, as described under Key Questions 3 and 4, some of the tests' supporting evidence was derived from past randomized controlled trials (RCTs) with prospectively gathered patient samples, giving them strong evidential value. Two ongoing RCTs, TAILORx and MINDACT (using Oncotype DX, and MammaPrint respectively), will provide further evidence allowing almost direct inference about the impact on patient outcomes.

Key Question 2. What are the sources of and contributions to analytic validity in these two gene expression-based prognostic estimators for women diagnosed with breast cancer?

In the field of gene expression there are no “gold standards” outside the technologies used in the tests under study, i.e., microarrays and RT-PCR. Consequently, a definitive evaluation of the analytic validity of expression-based tests is difficult. Evidence about operational characteristics was partial and limited to a few publications. A 2007 paper by Cronin and colleagues, on the analytic validity of Oncotype DX was the most detailed study for any of these tests so far, showing good performance for a number of analytic components of the assay. Data about the sources and contributions to variability of the tests and about their reproducibility was generally limited to analyses of few samples, and thus a complete evaluation of the impact of such variability on risk assessment was not available. Partial evidence about analytic validity was provided in the percentage of subjects whose samples were successfully analyzed with these tests, and those numbers were fairly good. Continuous monitoring of laboratory procedures and careful evaluation of the quality of the submitted specimens are major factors affecting test reliability.

Key Question 3. What is the clinical validity of these tests in women diagnosed with breast cancer?

• a

  How well does this testing predict recurrence rates for breast cancer compared to standard prognostic approaches? Specifically, how much do these tests add to currently known factors or combination indices that predict the probability of breast cancer recurrence, (e.g., tumor type or stage, age, ER, and human epidermal growth factor receptor 2 (HER-2) status)?

• b

  Are there any other factors, which may not be components of standard predictors of recurrence (e.g., race/ethnicity or adjuvant therapy), that affect the clinical validity of these tests, and thereby generalizability of results to different populations?

Clinical validity is defined as the degree to which a test accurately predicts the risk of an outcome (i.e., calibration), as well as its ability to separate patients with different outcomes into separate risk classes (discrimination). Clinical validity was documented to some degree for all three gene expression signatures. Oncotype DX was validated on a homogenous population of lymph node negative, ER positive patients all treated with tamoxifen, derived from an arm of an RCT, the National Surgical Adjuvant Breast and Bowel Project (NSABP-14). MammaPrint, on the other hand, was validated on samples from a clinical series with a wide range of clinical and treatment characteristics, and sometimes it was the signature and not the MammaPrint test itself that was validated. Data that made clear the incremental value of the test over standardized risk predictors using classical clinical factors, in the form of risk reclassification tables, was limited to Oncotype DX in one population, and for one of those predictors (Adjuvant! Online for MammaPrint). The evidence behind the two-gene test is quite heterogeneous, in that the specific manner in which the index was calculated differed in each, and only one examines the index that is to be used as part of the BCP (or H/I ratio) test in a study that was still using statistical methods to find optimal cut points, i.e., a training study. So the Oncotype DX test, which has been validated in exactly the form given to patients on clinically homogeneous samples with clear treatment implications, is regarded as the index with the strongest claim to clinical validity. It is not yet as clear to which populations MammaPrint best applies, and how much incremental value it would have within those clinically homogeneous populations above various standard predictors. Since the number of validation studies for any of the tests is still relatively small, more remains to be learned about stability between different populations of the relationship between expression-based score and the absolute observed risk. Essentially nothing is known about how specific characteristics of these populations might affect test performance.

While the H/I ratio test shows some promise, it must be regarded as still being in a developmental phase; it cannot yet be considered fully validated. It was not clear whether samples were processed by Quest Diagnostics, which hold the current license. There are a number of intriguing biological insights and plausible mechanisms to support the rationale for the test, but its consistent value in well-defined clinical settings has not yet been firmly established.

Key Question 4. What is the clinical utility of these tests?

• a

  To what degree do the results of these tests predict the response to chemotherapy, and what factors affect the generalizability of that prediction?

• b

  What are the effects of using these two tests and the subsequent management options on the following outcomes: testing or treatment related psychological harms, testing or treatment related physical harms, disease recurrence, mortality, utilization of adjuvant therapy, and medical costs.

• c

  What is known about the utilization of gene expression profiling in women diagnosed with breast cancer in the United States?

• d

  What projections have been made in published analyses about the cost-effectiveness of using gene expression profiling in women diagnosed with breast cancer?

Few studies addressed the clinical utility of Oncotype DX recurrence score (RS) in predicting the benefits of adjuvant chemotherapy, although the probability of recurrence represents an upper bound on the degree of absolute benefit. One fairly strong retrospective study produced preliminary evidence that the RS has predictive power in assessing the benefit of chemotherapy usage in ER-positive, lymph node negative breast cancer patients. This study was embedded within a large, well conducted RCT (National Surgical Adjuvant Breast and Bowel Project (NSABP B-20)). Some patients from the tamoxifen-only arm of the trial were in the training data sets for the Oncotype DX assay development, and this could potentially translate into a somewhat enhanced estimate of the discriminatory effect of Oncotype DX, although it is unlikely to eliminate entirely the effect seen here. Other studies produced preliminary evidence that the RS from the Oncotype DX assay has predictive power in assessing the likelihood of pathologic complete response after pre-operative chemotherapy with various drugs and regimens, although very limited sets of patients have been used. One study produced preliminary evidence that the RS cannot predict pathologic complete response after primary chemotherapy in advanced breast cancer patients.

One study produced preliminary evidence that the knowledge of the RS from the Oncotype DX assay can have an impact on the clinical management of patients diagnosed with ER positive, lymph node negative, and early breast cancer. However, it did not report specifically what the patients (or doctors) were told or understood about their absolute risk of recurrence, and therefore was minimally informative as to the actual risk thresholds used by women and their treating physicians, or whether absolute risks even entered into the decision.

There were no studies that addressed the clinical utility of the MammaPrint or H/I ratio tests.

Three published studies have addressed economic outcomes associated with use of the breast cancer gene expression tests. One study reported that using the 21-gene RT-PCR assay to reclassify patients who were defined by 2005 National Comprehensive Cancer Network (NCCN) criteria as low risk (to intermediate or high risk) would lead to an average gain in survival per reclassified patient of 1.86 years. The associated cost-utility of using recurrence score testing for this cohort was $31,452 per quality-adjusted life-year (QALY) gained. The analysis also reported that using the 21-gene RT-PCA assay to reclassify patients who were defined by 2005 NCCN criteria as high risk (to low risk) was cost saving. In a hypothetical population of 100 patients with characteristics similar to those of the NSABP B-14 participants, more than 90 percent of whom were NCCN-defined as high risk, using the 21-gene RT-PCR assay was expected to improve quality-adjusted survival by a mean of 8.6 years and reduce overall costs by about $203,000. However, the EPC team had only moderate confidence in the results of this analysis because the study was sponsored in part by the manufacturer of the 21-gene RT-PCR assay and the authors did not provide sufficient information about methodological and structural uncertainties as well as other potential sources of bias such as the derivation of the utility estimates. Furthermore, the 2007 NCCN guideline indicates that the use of chemotherapy in these patients is now considered optional, further diminishing the usefulness of these projections.

The second study reported that use of the 21-gene RT-PCR assay was associated with a gain of 0.97 QALYs and a cost-utility ratio of $4432 per QALY compared with use of tamoxifen alone, and a gain of 1.71 QALYs with net cost savings when compared with the chemotherapy and tamoxifen combination. However, the EPC team had little confidence in the results of this analysis, which was supported in part by the manufacturer, because the study did not meet many of the standards that the team used for appraising the quality of the analysis.

The third study compared the cost-effectiveness of the Netherlands Cancer Institute gene expression profiling (GEP) assay (MammaPrint) to the U.S. National Institutes of Health (NIH) guidelines for identification of early breast cancer patients who would benefit from adjuvant chemotherapy. The GEP assay was projected to yield a poorer quality-adjusted survival than the NIH guidelines (9.68 vs. 10.08 QALYs) and lower total costs ($29,754 vs. $32,636). To improve quality-adjusted survival, the GEP assay would need to have a sensitivity of at least 95 percent for detecting high risk patients while also having a specificity of at least 51 percent. The EPC team had confidence in the results of this analysis because it met most of the standards for appraising the quality of an economic analysis.

Based on the appraisal of these three studies, the overall body of evidence on economic outcomes was inconclusive.

Oncotype DX

• 1

  The role of the RS in guiding treatment of HER-2 positive patients is unclear, as most of these patients were classified in the high RS group in the initial trials.

• 2

  While awaiting the TAILORx results, the findings of the Paik 2006 study predicting treatment benefit need independent confirmation.

MammaPrint

• 1

  The prognostic value of the 70-gene signature has been assessed in different populations facing different therapeutic choices. In the analysis by van de Vijver and colleagues, 130 of the 295 patients received adjuvant therapy in a non-randomized fashion. Patients in the original development cohort were not treated, and Buyse validated the marketed assay in untreated patients. It is not yet clear which are the optimal patient populations for the use of this test, exactly what its performance is in those populations, and how many of its predictions would result in different therapeutic decisions. Larger independent validation studies in therapeutically homogeneous groups would be very valuable.

• 2

  There is no evidence for the degree to which this test predicts the benefit of adjuvant chemotherapy.

Breast Cancer Profile (H/I ratio) Test

• 1

  The BCP test is not yet as well validated as either of the other tests, with most of the supporting studies examining slightly different ways of either performing (e.g., different reference standards) or calculating the index. More work needs to be done documenting the risk discrimination and risk calibration of the marketed test in clinically homogeneous populations, as well as its incremental value.

• 2

  There is no evidence for the degree to which this test predicts the benefit of adjuvant chemotherapy.

In addition to the conclusions above, a series of other observations were made on the basis of what was learned in this investigation.

Assay Validation

In general, it is clear that validation studies need to deal with populations for whom the decision-making implications of various risk groupings are clear. For all tests except Oncotype DX, both validation and development studies have been on mixed populations, without sufficient sample sizes to stratify into large enough homogeneous groups to guide clinical decisionmaking. In addition, validation samples are often re-used by other investigators; the pool of such samples in the public domain needs to be greatly expanded.

Potential for Scale Problems

One problem that may be faced in the future is that of the consequences of an increase in demand for these tests. Whether the degree of accuracy seen in investigational settings can be maintained with increasing demands should be monitored by scientific or regulatory bodies.

Genetic Variability and Gene Expression

It is unknown whether gene expression profiles are more or less likely than more traditional biomarkers to be generalizable beyond the populations in which they were initially developed. Gene expression may reflect fundamental biological tumor features, and thus be relatively stable across ethnic groups. This speaks to the importance of validating these tests in populations with varying genetic background. Of particular interest will be the variation of the observed absolute risk in those populations, and its correlates.

The Need for Databases, Reproducibility, and Standards

Consideration should be given to the development of databases with complete data on each patient tested with these and future tests (absent identifiers). The data should include all the analyses performed, laboratory logs, the raw and processed data, and all the information about procedures and analyses that have been performed to produce a risk estimate from a tumor sample.

Where is the Field Going?

We can expect many new tests, as well as new uses for the assays that already exist. More genes might be added to the signatures, and in the particular case of MammaPrint this will be possible without changing the experimental procedures, since the array contains more genes than the ones that are incorporated in the 70-gene signature. In this regard, we might also expect other modifications: subsets of the current signatures might be proposed as alternatives to current clinical risk factors, or be proposed in different populations or for different purposes. For Oncotype DX, a natural evolution could be related to its use as an alternative to immunohistochemistry and/or pathology to evaluate tumor Grade, S-phase index, ER, progesterone receptor, and HER2 expression, since such genes are part of the set included in the assay. Reporting of individual gene expression results may also prove useful.

“Comparative Effectiveness” Studies

As these tests mature and proliferate, an important question will be how they compare to each other, and whether there is value in their combination. In the therapeutic domain, this has been called “comparative effectiveness” research. Such research has traditionally been difficult to fund by government or by industry, because it may not hold out as much therapeutic promise as new discoveries, and because industry understandably is not anxious to fund head-to-head comparisons with competitive products. This same dynamic could easily take hold in the risk prediction arena, with a proliferation of licensed prediction indices without any clear notion of what new ones are contributing over previous tests. In this perspective, development of future expression-based predictors should account for direct contrasts with “established” methods.

Gene Expression Profiling

Gene expression profiling (see Glossary, Appendix B) is an emerging technology for identifying genes whose activity may be helpful in assessing disease prognosis and guiding therapy. Gene expression profiling examines the composition of cellular messenger ribonucleic acid (RNA) populations. The identity of the RNA transcripts (see Glossary, Appendix B) that make up these populations and the number of these transcripts in the cell provide information about the global activity of genes that give rise to them. The number of mRNA transcripts derived from a given gene is a measure of the “expression” of that gene. Given that messenger RNA (mRNA) molecules are translated into proteins, changes in mRNA levels are ultimately related to changes in the protein composition of the cells, and consequently to changes in the properties and functions of tissues and cells in the body. However, only 2 percent of the genome (see Glossary, Appendix B) is translated into proteins, and little is known about how the expression of this 2 percent is controlled. The key intermediate is the transcriptome (see Glossary, Appendix B), which is made up of all the individual transcripts produced by the cell (see Figure 1).

Investigators have developed approaches to gene expression analysis that have led to substantial advances in our understanding of basic biology. Gene expression profiling has been applied to numerous mammalian tissues, as well as plants, yeast, and bacteria.¹⁰–¹⁴ These studies have examined the effects of treating cells with chemicals and the consequences of overexpression of regulatory factors in transected cells. Studies also have compared mutant strains with parental strains to delineate functional pathways. In cancer research, such investigation has been used to find gene expression changes in transformed cells and metastases, to identify diagnostic markers, and to classify tumors based on their gene expression profiles (see Glossary, Appendix B).¹⁵–¹⁸ The use of this approach for specific clinical problems, however, is relatively recent and poses several challenges related to the validity, reproducibility, and reliability required for use in diagnostic or predictive testing.

In recent years, gene expression profiling has been successfully used in breast cancer research. For instance, distinct subtypes of breast tumors (such as tumors expressing HER-2) have been identified as having distinctive gene expression profiles, representing diverse biologic entities associated with differences in clinical outcome.¹⁹–²³ Other investigators ²⁴ have found gene expression signatures (see Glossary, Appendix B) associated with the ER and lymph node status of patients, thus identifying subgroups of patients with different clinical outcomes after therapy. From such studies, investigators have proposed a number of gene expression profiles that could be used to classify prognosis. In a case-control study from the Netherlands Cancer Institute (Amsterdam, the Netherlands), one such gene profile, consisting of 70 genes, was developed using archived frozen tissue from 78 young, node-negative women with breast cancer.²¹ In this study, tumors from patients who suffered rapid relapses after primary therapy had gene expression profiles that were quite distinct from those who remained disease-free. These gene expression profiles were then applied to a second validation set of 295 frozen tissue specimens collected from young women (including 61 patients from the previous cohort), yielding very similar results.²⁵ Indeed, it appeared that this 70-gene profile more accurately predicted outcomes than did the traditional clinical criteria. Results from these preliminary studies further suggested that gene expression profiling may provide a powerful tool for estimating prognosis and the likelihood of benefit from selected therapeutic agents.

RT-PCR

RT-PCR is a molecular biology technique that combines reverse transcription with real-time PCR (see Glossary, Appendix B). This methodology allows the quantification of a defined RNA molecule. It is accomplished by reverse transcription of the specific RNA into its complementary DNA, followed by amplification of the resulting DNA using PCR. The quantification of the DNA produced after each round of amplification is accomplished by the use of fluorescent dyes that intercalate with double-stranded DNA, or by modified DNA oligonucleotide probes (see Glossary, Appendix B) that fluoresce when hybridized with complementary DNA.

In a PCR template, relative ratios of the product and reagent vary. At the beginning of the reaction, reagents are in excess, and template and product are present in low concentrations and do not compete with primer binding, so that the amplification proceeds at a constant, exponential rate. After this initial phase, the process enters a linear phase of amplification, and then in the late reaction cycles, the amplification reaches a plateau phase and no more product accumulates To achieve accuracy and precision, it is necessary to collect quantitative data during the exponential phase of amplification, since in this phase the reaction is extremely reproducible. In RT-PCR, this process is automated, and measurements are made at each cycle. Finally, several implementations of this technique allow multiple DNA species to be measured in the same sample (multiplex PCR), since fluorescent dyes with different emission spectra may be attached to the different probes. Multiplex PCR allows internal controls to be co-amplified with the target transcripts (see Glossary, Appendix B) and permits allele discrimination in single-tube, homogeneous assays (Figure 2).

This technique is extremely sensitive. The development of novel chemistries and instrumentation platforms has led to widespread adoption of real-time RT-PCR as the method of choice for quantifying absolute changes in gene expression. Moreover, this technique has become the preferred method for validating results obtained from microarray analyses and other techniques that evaluate gene expression changes on a global scale.

Microarrays

The analysis of gene expression by microarray technology is based on the Watson-Crick pairing of complementary nucleic acid molecules. In this technique, a collection of DNA sequences, called probes (see Glossary, Appendix B), are “arrayed” on a miniaturized solid support (microarray) and used to detect the concentration of the corresponding complementary RNA sequences, called targets (see Glossary, Appendix B), present in a sample of interest. The advancements made in attaching or synthesizing nucleic acid sequences to solid supports and robotics have allowed investigators to miniaturize the scale of the reactions, and it is now possible to assess the expression of thousands of different genes in a single reaction.²⁹–³¹

In the basic microarray experiment, RNA harvested from the sample of interest is labeled with a fluorescent dye and hybridized to the microarray, then incubated in the presence of RNA from a different sample labeled with a different fluorescent dye. In this two-color experimental design, samples can be directly compared to one another or to a common reference RNA, and their relative expression levels can be quantified. After hybridization, gray-scale images corresponding to fluorescent signals are obtained by scanning the microarray with dedicated instruments, and the fluorescence intensity corresponding to each gene investigated is quantified by specific software. After normalization, the intensity of the hybridization signals can be compared to detect differential expression by using sophisticated computational and statistical techniques (Figure 3).

Sources

Our comprehensive search plan included electronic and hand searching. On January 9, 2007, we ran searches of the MEDLINE^® and EMBASE^® databases, and on February 7, 2007, we searched the Cochrane database, including Cochrane Reviews and The Cochrane Central Register of Controlled Trials (CENTRAL), and CINAHL^®. All searches were limited to articles published in 1990 or later. This cut-off year was established based on the introduction date of the MeSH heading “gene expression profiling,” 2000, and the introduction date of the MeSH heading “gene expression,” 1990. Also, test searches of earlier dates returned limited and irrelevant results.

“Gray” literature was searched following a protocol that was reviewed and approved by EGAPP and the technical expert panel:

• 1

  Conference abstracts were reviewed using the same criteria as for journal articles but were only included if we felt we had a sufficient understanding of the underlying study and the data reported were critical enough to merit inclusion.

• 2

  Web sites for the gene profiling tests included in this review, Agendia (MammaPrint®) and Genomic Health (Oncotype DX™), were searched for additional information not available in the peer-reviewed literature.

• 3

  Agendia and Genomic Health, Inc. were contacted directly with requests for the following information:

  • a

    A listing of articles that applied to the analytic validity or clinical utility of the gene profiling test,

  • b

    Marketing materials on the gene profiling test, and

  • c

    Any pertinent unpublished data.

• 4

  We searched the Web site of the Food and Drug Administration (FDA) Center for Devices and Radiological Health for additional publicly available, unpublished information. ³⁷–³⁹

• 5

  A request was sent to the Center for Medical Technology Policy (CMTP) Gene Expression Profiling for Early Stage Breast Cancer Work Group to provide all background materials available on our study topic.

Search Terms and Strategies

Search strategies specific to each database were designed to enable the team to focus available resources on articles most likely to be relevant to the Key Questions. We developed a core strategy for MEDLINE, accessed via PubMed, based on an analysis of the MeSH terms and text words of key articles identified a priori. The PubMed strategy formed the basis for the strategies developed for the other electronic databases (see Appendix F).

Organization and Tracking of the Literature Search

The results of the searches were downloaded into ProCite^® version 5.0.3 (ISI ResearchSoft, Carlsbad, CA). Duplicate articles retrieved from the multiple databases were removed prior to initiating the review. We then reviewed the citations by scanning the titles, abstracts, and the full articles as described below (Figure 4).

Inclusion and Exclusion Criteria

The abstract review phase was designed to identify articles that reported on the analytic validity, clinical validity, and/or clinical utility of the gene expression profile tests of interest. Abstracts were reviewed independently by two investigators and were excluded only if both investigators agreed that the article met one of the following exclusion criteria:

• 1

  The study applied only to breast cancer biology;

• 2

  The study did not involve Oncotype DX or MammaPrint,

• 3

  The study did not involve original data or original data analysis;

• 4

  The study did not involve women;

• 5

  The study did not involve breast cancer patients;

• 6

  The study was not in the English language; or

• 7

  The study did not apply to the key questions.

We excluded letters to the editor and editorials when they did not present original data (usually in the form of electronic supplements in the case of letters). If a letter or editorial cited Some original data, it generally was not sufficiently original for consideration in this report. As mentioned earlier, the initial scope of this project did not include the H/I ratio test, and thus this test was not identified on the abstract review form (Appendix G, Abstract Review Form).

Abstracts were promoted to the article review level if both reviewers agreed that the abstract could apply to one or more of the key questions. Differences of opinion regarding abstract eligibility were resolved through consensus adjudication.

Oncotype DX™

Evidence about the analytic validity of Oncotype DX is available from two technical studies, Cronin et al, 2004,⁴⁴ and Cronin et al. 2007,⁴⁵) and from several clinical reports. Information about the overall success rate of the assay was documented in 9 studies (Chang, 2007,⁵⁵ Cobliegh, 2005,⁴⁷ Esteva, 2005,⁴⁸ Gianni, 2005,⁴⁹ Habel, 2006,⁵⁰ Mina, 2006,⁵¹ Oratz, in press,⁵² Paik, 2004,²⁸ and Paik, 2006⁵³). This success rate ranged from 78.9 percent to 98.9 percent, and only some of the studies provided detailed descriptions of the reasons for assay failure. Reported failures were mainly ascribed to an insufficient number of cancer cells in the specimens, to poor RNA quality, and in a few cases, to failure of the RT-PCR technique. A synopsis of this evidence is provided in Table 2.

Data on assay variability and reproducibility were available from 3 studies (Cronin, 2007,⁴⁵ Habel, 2006,⁵⁰ and Paik, 2004²⁸). These studies assessed the variability of repeated gene expression measurements using RNA from either the same or different FFPE blocks at repeated time points, and across different instruments and operators. Data reported in the study concerned the variability of individual genes in the assay as well as the RS reproducibility. Variability evidence was reported for 66 FFPE blocks from 22 distinct patients, and from repeated measurements of two aliquots of a pooled reference RNA. Overall, the standard deviation (SD) for the recurrence score was below 3 RS units, although the authors did not discuss the impact on risk stratification. This evidence is reported in Table 3.

Two studies addressed technical and operational aspects of analytic validity (Cronin, 2004,⁴⁴ and Cronin, 2007⁴⁵). The first study presented data about the development the assay procedures, comparing gene expression measurements between frozen tumor specimens and FFPE blocks. The optimization of the RT-PCR primers (see Glossary, Appendix B¹), and the normalization strategy were discussed. The second study addressed relevant analytic components of the assay, such as detection and quantification limits (limit of detection (LOD) and limit of quality (LOQ) respectively), amplification efficiency, linearity, dynamic range, accuracy, precision, and assay reproducibility. The available evidence is reported in Table 4.

Finally, eight studies (Chang, 2007,⁵⁵ Cobliegh, 2005,⁴⁷ Esteva, 2005,⁴⁸ Gianni, 2005,⁴⁹ Cronin, 2004,⁴⁴ Habel, 2006,⁵⁰ Mina, 2006,⁵¹ and Paik, 2004²⁸) compared gene expression measurements of specific individual genes (ER, progesterone receptor (PR), HER-2) to measurements of the corresponding proteins produced by those genes as obtained by other techniques, in particular immunohistochemistry (IHC). Such studies used various cycle thresholds (CTs) to define positivity for the genes (see Table 5). Overall agreement between RT-PCR and IHC proved generally good for ER (k statistics ranging from 0.80 to 1). In one study (Habel, 2006⁵⁰), agreement was low (0.49), although RT-PCR measurements were comparable to data available in the clinical records. In general, agreement for PR and HER-2 was moderate or poor. Such evidence is reported here for completeness (see Table 5), although it does not contain any relevant information about the assay as a whole.

Individual studies are briefly described below.

Cronin et al., 2004.⁴⁴ In this study, the authors discussed the primer (see Glossary, Appendix B) design optimization and expression level normalization necessary to obtain reliable RT-PCR measurements from archival FFPE samples, with the goal of establishing the reliability of their results with partially degraded RNA samples. The authors compared gene expression levels in 62 matched FFPE and frozen tissue specimens prepared from the same breast tumor. They showed that the relative expression profiles obtained from the two analyses were similar (correlation = 0.91, P value < 0.0001), although the magnitude of the measurements differed. They successfully corrected the differences using normalization based on the expression of five reference genes. Convincing evidence supporting the use of the implemented protocols in assessing gene expression levels from archival (i.e., formalin-fixed, paraffin-embedded) tumor specimens was shown.

The authors also analyzed several genes that were reported to show similar patterns in the literature²⁰ for co-expression,⁵⁴ and confirmed these correlations. Specifically, the expression of cytokeratin 5 and cytokeratin 17 (r = 0.85), LPL and RBP4 (r = 0.84), HER-2 and GRB7 (r = 0.71), ER1 and GATA3 (r = 0.6) were highly correlated.

Additionally, the authors compared RT-PCR measurements of ER, PR, and HER-2 (all components of Oncotype DX) expression to IHC analysis of corresponding protein levels, and to fluorescent in situ hybridization (FISH) analysis for HER-2 for a subset of 17 samples. The concordance among the different assays detecting the protein products of the genes and the relative RNA levels as measured by RT-PCR was high (94 percent, 84 percent, and 100 percent, respectively, see Table 5).

In summary, this study provided a foundation for the use of the Oncotype DX assay in archival tissue, although it did not contain data about the development of the RS (Appendix I, Evidence Tables 1, 2 and 3).

Paik et al., 2004.²⁸ In this clinical study, the authors reported data on the variability of the RS, and the overall success rate of the assay. The authors evaluated the reproducibility of the Oncotype DX assay within and between FFPE blocks from the same patient. The Oncotype DX assay was carried out on 5 serial sections from 6 different blocks from 2 distinct patients. Seventy nine blocks out of 754 were not analyzed due to insufficient tumor content, but RT-PCR was successful in 668 of the remaining 675 (98.9 percent) tissue blocks.

For the 16 genes considered in the RS, the SD of expression ranged from 0.07 to 0.21 expression units across serial sections from the same block. The within-block SD of the combined RS proved to be 0.72 RS units (with 95 percent CI: 0.55–1.04), while the within-patient SD, which included both among-block and within-block variation, proved to be 2.2 RS units. The impact of this variation on the risk stratification provided by the RS was not discussed in the paper. The difference between the low- and high-risk groups is 14 RS units, far larger than the standard deviations reported. Although ER, PR and HER-2 were also assessed by other techniques, the agreement of the measurement obtained by the different technologies was not reported.

In summary, this paper reported evidence about the fraction of tissue blocks that can be successfully typed by the Oncotype DX assay, as well as limited data about the reproducibility of the RS between different sections and FFPE blocks from the same patient. The impact of such variability on the risk stratification was not examined (Table 3, Appendix I, Evidence Tables 1, 2 and 3).

Esteva et al., 2005.⁴⁸ In this study, the authors evaluated the correlation of RS, both as a whole and broken into its components, with known standard prognostic markers in FFPE tumor specimens. Specifically, the relationship between RT-PCR and IHC for ER, PR, and HER-2 was examined. The concordance for PR status was poor (k of 0.48), high for ER (k = 0.81), and proved moderate for HER-2 (k = 0.60).

A logistic model using IHC HER-2 measurement as a quantal response indicated a significant (P < 0.0001) degree of correlation between IHC and RT-PCR. Sensitivity and specificity for HER-2 were also measured, using different RT-PCR cutoff points and positivity, and are reported in Table 5.

In summary, this paper reported evidence about the percentage of successfully-analyzed samples (67.7 percent, 149/220) in a large population from a single institution (M.D. Anderson Cancer Center) (Table 5, Appendix I, Evidence Tables 1, 2 and 3).

Cobleigh et al., 2005.⁴⁷ This study reports on the development of the 21-gene Recurrence Score assay (Oncotype DX), Duplicated gene expression measures were obtained by RT-PCR in archival FFPE tumor tissue blocks. An initial set of 192 genes (187 cancer-related and 5 controls) were analyzed and 16 additional candidate genes were added at a later time. Ninety-one point six percent (78/85) of samples were successfully analyzed

IHC-measured protein levels and RT-PCR mRNA levels for ER, PR, HER-2, and Ki-67/MIB-1 (a proliferation marker of cancer cells) were compared. The concordance was high for both ER (k = 0.83) and HER-2 (k = 0.67), somewhat lower for PR (k = 0.40), and poor for Ki-67 (k = 0.22). (Table 5, Appendix I, Evidence Tables 1, 2 and 3).

Gianni et al., 2005.⁴⁹ The authors of this paper evaluated the correlation of IHC-measured protein levels with RT-PCR mRNA measurements of ER and PR expression in tumors. The concordance was high for ER (k = 0.84; 95 percent CI, 0.71 to 0.96) and moderate for PR (k = 0.71; 95 percent CI, 0.56 to 0.86). This paper also reports preliminary evidence about the use of the Oncotype DX assay in fixed core biopsies from breast cancer patients. The percentage of successfully analyzed samples was 93.6 percent (89/95) (Table 5, Appendix I, Evidence Tables 1, 2 and 3).

Mina et al., 2006⁵¹ In this study, the authors evaluated the usefulness of FFPE core biopsies from a completed phase II trial in identifying genes that correlated with a response to primary chemotherapy. Out of the 70 patients enrolled in the study, 67 gave their consent, and specimens from 57 patients were available to perform gene expression analysis by RT-PCR. Out of these 57 patients, gene expression levels could be accurately measured in 45 patients. Failures were due either to low RNA yield (9 patients) or low tumor content in the biopsies (3 patients).

In this study the authors compared the expression levels of ER mRNA obtained by RT-PCR to ER protein expression as measured by IHC. Using a pre-defined cutoff of 6.5 CT, 64 percent of the 45 tumors were ER positive, while 36 percent were considered ER negative. ER expression by IHC correlated well with ER mRNA expression by RT-PCR (see Table 5), and only four of the 45 samples did not show agreement. The authors concluded that gene expression analysis on core biopsy samples was feasible. Data for PR, HER-2 and Ki-67 were not reported.

In summary, this paper reported preliminary evidence about the expression of some of the Oncotype DX assay genes in fixed core biopsies from breast cancer patients. The percentage of successfully analyzed samples was about 79 percent (45/57), raising concerns about the real feasibility in clinical settings (Table 5, Appendix I, Evidence Tables 1, 2 and 3).

Habel et al., 2006.⁵⁰ This study contains several results that are relevant for the overall analytic validity of the Oncotype DX assay. The authors cited two unpublished studies with data concerning the reproducibility of the RS. These studies analyzed, respectively, 60 blocks from a total of 20 distinct patients, and 49 core biopsies or resections from advanced breast cancer patients. In the first study the RS SD between different blocks from the same patient was 3.0 RS units, and less than 2.5 for 16 out of 20 patients. Similar results were claimed for the second study, although the actual data were not shown.

Finally, the authors compared the agreement of ER status, as obtained by RT-PCR, to the ER status reported in the medical records. A positive or negative classification was based on a CT cutoff point of 6.5. The RT-PCR failure rate was about 1 percent for specimens available after pathological review, and 7.9 percent of the samples were not assessable due to low tumor contents. In this study population, the concordance between RT-PCR and the medical chart information was only moderate (k = 0.49, 95 percent CI 0.41–0.56). In the multivariate models used in the following statistical analyses, the RT-PCR based ER status was used.

In summary, this paper reported a high percentage of successfully analyzed samples in a large population from a single institution and the reproducibility of the RS between different blocks from the same patient. The impact of such variability on the risk stratification was not addressed (Table 5, Appendix I, Evidence Tables 1, 2 and 3).

Paik et al., 2006.⁵³ In this clinical study the authors reported several results that can be used as indirect evidence for the overall analytic validity of the Oncotype DX assay. Particularly relevant, FFPE blocks with sufficient tumor content were available from 670 of the 2,299 eligible patients in the NSABP N-20 trial, and the RT-PCR assay was successful on 651 of the 670 patients (97.2 percent). (Appendix I, Evidence Tables 1, 2 and 3).

Cronin et al., 2007.⁴⁵ This study is the most extensive analysis to date of the analytic components of the Oncotype DX assay. Detection and quantification limits of the RT-PCR reactions, amplification efficiency, linearity, dynamic range, accuracy, precision, and assay reproducibility were investigated in serial dilution experiments, using a common RNA obtained by pooling 15 distinct RNA samples.

Detection and quantification limits proved to be well within the instrument's pre-specified CT unit limits for all the genes. Amplification efficiencies (100 percent efficiency means that the RT-PCR reaction products achieved perfect doubling) for the 16 cancer-related genes ranged from 75 percent to 112 percent, with an average of 96 percent, while the mean efficiency proved to be 88 percent for the reference genes, with a range from 75 percent to 101 percent.

Accuracy and precision studies were conducted at the target RNA concentration of 2 ng per assay well, which is what is used in the Oncotype DX assay. The mean percent bias from each gene target was -0.3 percent (ranging from -10 percent to 6 percent) for cancer-related genes, and 0.7 percent for reference genes (-1.5 percent to 3.3 percent), indicating 99 percent mean quantitative correctness at this assay condition. The CV averaged 5.7 percent for the cancer-related genes and 3.2 percent for reference genes. The implications of such variability for RS were not discussed.

Finally, individual gene and RS reproducibility were measured by performing repeated analyses across multiple days, operators, RT-PCR plates, RT-PCR instruments, and liquid-handling robots. Two operators obtained replicate CT measurements on two aliquots of a single RNA sample over the course of five days with three real time PCR instruments (7900HT instruments) and two liquid-handling robots. The study design allowed the estimation of all main effects, including operator, RT-PCR instrument, and liquid-handling robot. Total SD in CT measurements varied from 0.06 to 0.15 CT units across the 21 genes, and the upper bounds on 2-sided 95 percent confidence intervals for the CV were all within 10 percent. The authors reported that a maximum SD of 0.15 at a CT of 30 translates into a CV of 0.5 percent, allowing a 15 percent change in gene expression to be distinguished. The day-to-day SD for all 21 genes ranged from 0 to 0.055, the between-plate SD ranged from 0 to 0.09, while the within-plate SD ranged from 0.057 to 0.147. The standard deviation for the overall RS (total and within-plate) was 0.8 RS unit. The largest differences between operators, as well as between liquid handling robots and 7900HT instruments, were 0.5 CT units for each of the 21 Oncotype DX genes, while SD and CV for the RS were not reported.

In summary, this study presented extensively detailed results about several relevant analytic components of the assay (Table 4, Appendix I, Evidence Tables 1, 2 and 3).

Chang et al., 2007.⁵⁵ This clinical study reported several results that can be used as indirect evidence for the overall analytic validity of the Oncotype DX assay. Ninety-seven FFPE blocks from core biopsies were analyzed by the standard assay protocols, and the percentage of successfully analyzed samples was 82.4 percent.

In summary, this paper provides preliminary evidence about the use of the Oncotype DX assay in fixed core biopsies from breast cancer patients (Table 2, Appendix I, Evidence Tables 1, 2 and 3).

Oratz et al., in press.⁵⁶ This clinical study evaluated the impact of the Oncotype DX assay on clinical management, and also provided indirect evidence for the assay's overall analytic validity. Seventy-four FFPE blocks were analyzed by the standard assay protocols, and the percentage of successfully analyzed samples was 97.3 percent. No explicit eligibility criteria were used. The samples were included based on the request for analysis from the patient's clinician.

In summary this paper contains evidence about the use of the Oncotype DX assay on FFPE blocks from breast cancer patients (Table 2, Appendix I, Evidence Tables 1, 2 and 3).

MammaPrint®

Analytic validity and variability evidence for MammaPrint was available from two technical studies ( Ach, 2007,⁵⁷ and Glas, 2006⁵⁸) and information on the overall success rate of the assay was documented in just one study, Buyse, 2006⁵⁹(80.9 percent).

Data about variability and reproducibility were obtained in these studies using repeated gene expression measurements over time, within and across individual microarrays, across different laboratories, protocols instruments, and operators (see Tables 6, 7, 8). No comparisons were made between expression measurements of individual genes and their corresponding protein level by IHC.

The following is a brief description of each study.

Glas et al., 2006.⁵⁸ In this study the authors reported a summary of the results obtained during the development of the commercially marketed version of the 70-gene prognostic signature,^21,25 the expression array-based test known as MammaPrint. The authors evaluated and compared both technical aspects and the clinical validity of the assay using the originally published data (see Key Question 3).

MammaPrint uses a microarray accounting for 1,900 features (individual microarray locations where the probes are positioned), containing each of the 70 genes in the signature spotted in triplicates. In this paper the authors re-analyzed the data from the original series^21,25 using the new array, a dye-swap hybridization design, a different reference RNA and a different approach to computing gene expression levels. Triplicate measurements were obtained for each gene of the 70-gene signature and summarized by an error-weighted average, rather than the approach proposed by Hughes et al., 2000,⁶⁰ which was used in the original studies.

The results obtained with the new signature were comparable to the original results. Briefly, MammaPrint proved reproducible on the original development series²¹ (Pearson's correlation coefficient = 0.92 P value < 0.0001), and in a subset of the van de Vijver cohort²⁵ (Pearson's correlation coefficient of 145/151 lymph node-negative patients = 0.88, P value < 0.0001). The replication of the experiment within patients and along time suggested high reproducibility as well. In particular, the Pearson's correlation coefficient on 49 patients analyzed twice was 0.995, and no significant variability within individuals was found by an analysis of variance (ANOVA) for the 70-gene signature P value = 0.96).

Risk classification by MammaPrint is obtained by measuring the cosine correlation of individual patients' gene expression profiles to the mean gene expression profile obtained in the van't Veer²¹ series. The variability of such correlation was measured by repeated analysis of 3 patients over time and showed very small SDs (0,028, 0,028 and 0.027 respectively).

In summary, this study reported detailed data about the development of the MammaPrint assays as it is offered in clinical settings, as well as data about the reproducibility of the assay within a single laboratory (Tables 7 and 8, Appendix I, Evidence Tables 6, 7 and 8).

Buyse et al., 2006.⁵⁹ In this clinical study the authors reported several results that can be used as indirect preliminary evidence for the overall analytic validity of the MammaPrint® assay. Fresh frozen blocks from primary breast cancer patients collected in 5 distinct institutions were shipped for analysis to Agendia, and the percentage of successfully analyzed samples was 80.9 percent (326/403 patients) (Appendix I, Evidence Tables 6, 7 and 8).

Ach et al., 2007.⁵⁷ The inter-laboratory reproducibility of the MammaPrint assay was assessed in this paper. Results for the same set of four patients were obtained at three different sites and compared in order to assess the variation resulting from several important phases of analysis, including RNA amplification and labeling, hybridization and wash, and slide scanning. The same input RNA was used for all experiments.

In the first phase of the analysis, two laboratories, one in Amsterdam and one in California, amplified and labeled the RNA samples, then exchanged aliquots of the templates. Hybridization and slide scanning were performed at both locations and the scanned slides were then exchanged for re-analysis by the other laboratory. The same lot of labeling kits and microarrays were used at both sites. Technical replication variability was assessed by analyzing two separate slides in two different days. This experimental design allowed examination of both intra- and inter-laboratory variation.

The Pearson correlation coefficient across all technical replicates for all tumors analyzed proved to be above 0.983, indicating that the signals from replicate hybridizations correlated extremely well for genes expressed at all the measured intensity levels.

The reproducibility of laboratory scanning procedures was evaluated by scanning each of the 16 microarray slides at both sites. Signals for green fluorescent dye proved extremely reproducible, irrespective to the site of first hybridization and scan (Pearson correlation coefficient > 0.995, slope = 0.97), while signals for the red dye correlated less well and were always lower on the rescanned slide. The correlation of the 70-gene expression profile to the previously developed⁵⁹ mean signature⁵⁸ was computed for each dye-swapped pair of arrays and ANOVA was used to evaluate the variability by hybridization site, labeling site, and hybridization day. No significant differences were found between hybridization sites, or hybridization days (regardless of site), but two tumors showed a statistically significant difference (P value <0.05) between labeling sites. Variability due to the RNA labeling site was further confirmed for expression measurements of individual genes of the 70-gene expression profile, as well as on the 182 most highly expressed genes.

In the second phase of the study, the assay performance was evaluated by a third laboratory in Paris, France, using a different batch of arrays, reagents, and labeling kits, on the same four tumor RNAs, several months after the initial comparison. The 70-gene signature correlation values for each of the four tumors were compared by ANOVA analysis, and significant differences were found for two of the tumors, when stratified by labeling site (P values of 0.0004 and 0.01 respectively), whereas one tumor proved to be significantly different (P value, 0.016) by hybridization site. The authors predicted, but did not provide supporting data, that if variations in the washing protocols were introduced between laboratories, significant discrepancies in the 70-gene signature results would emerge. They concluded that while some sources of variation have measurable influence on individual microarray measurements, the overall impact on the 70-gene signature is low.

In summary, this study thoroughly investigated factors that could affect the reproducibility of the 70-gene signature within and across different laboratories. RNA labeling proved to be the largest contributor to inter-laboratory variation, but the authors did not address the impact of such factors on the classification of individual patients into different risk groups. The data (although from only four distinct patients) implies that results from MammaPrint testing cannot be compared across laboratories and that the test must be centralized (Tables 7 and 8, Appendix I, Evidence Tables 6, 7 and 8).

H/I Ratio

None of the studies reviewed here explicitly referred to the marketed H/I ratio (BCP assay). However, one publication described the analytic procedures involved with such test, Ma, 2006.⁶¹ The rest of the available analytic validity and variability evidence was specific to the way in which the two-gene ratio profile was computed in each clinical study, and did not contain direct information about the marketed test.

Three studies (Goetz 2006,⁶² Jerevall 2007,⁶³ Ma 2006⁶¹) reported the overall success rate of the analyses, one report, Jerevall 2007,⁶³ assessed the reproducibility between two different institutions, one assessed the correlation between RT-PCR and microarray based gene expression measurements for the two genes (HOXB13 and IL17RB), and one, Ma 2004,⁶⁴ study compared ER status by RT-PCR and IHC (see Tables 9, 10, and 11). No comparisons were made between expression measurements of HOXB13 and IL17RB transcripts and the corresponding proteins by IHC. For completeness, a brief description of individual studies follows.

Ma et al., 2004.⁶⁴ In this study the authors developed the HOXB13/IL17BR two-gene ratio signature. They identified differentially expressed genes associated with breast cancer recurrence in patients who were treated with tamoxifen, using gene expression arrays on whole mount as well as on laser micro-dissected (LMC) specimens. From a total of 5,475 genes selected because of their high variability across tumors, three differentially expressed genes proved to be common between the two analyses (macro-dissected specimens vs. LCM). These genes were HOXB13 (identified twice as AI700363 and BC007092), the 17B receptor IL17BR (AF208111), and EST AI240933.

HOXB13 was found to be over-expressed in tamoxifen recurrence cases, whereas IL17BR and AI240933 were over-expressed in tamoxifen non-recurrence cases. The authors confirmed relative gene expression by RT-PCR microarray analysis on 59 out of the 60 original patients. The Pearson correlation coefficient between array and RT-PCR results was 0.83 for HOXB13, and r = 0.93 for IL17BR. The RT-PCR-derived HOXB13/IL17BR ratios also highly correlated with its microarray-derived counterpart (0.83). The authors also evaluated by RT-PCR 20 additional ER-positive early-stage primary breast tumors from women treated with adjuvant tamoxifen monotherapy between 1991 and 2000. These were used as a validation set (see Key Question 3).

In summary, this study provides a foundation for the use of the H/I ratio signature in LMC FFPE specimens (Table 10, Appendix I, Evidence Tables 10, 11 and 12).

Ma et al., 2006.⁶¹ The authors developed the two-gene index concept in this study, based on the two-gene ratio they originally published in Ma et al, 2004.⁶⁴ New RT-PCR primers/probes for HOXB13 and IL17BR were used, and four reference genes were introduced for normalization. Total RNA was isolated from two 7-micrometer thick tissue sections for each sample, reverse transcribed into cDNA using a pool of gene-specific primers, and quantitated by TaqMan RT-PCR in duplicate in a 384-well plate. For each sample, CT values for the four reference genes were averaged and the relative expression level of each target gene was expressed as the difference from mean reference CT after Z-transformation. This resulting value is no longer a simple ratio, and is thus referred to as the two-gene index.

RNA for this study was prepared from cancer cells isolated by LCM from FFPE tissue microarray sections (see Glossary, Appendix B) of originally frozen tumor specimens. From 870 patients, 98.0 percent of samples were successfully processed (Table 9).

In this study the authors evaluated the concordance between ER and PR protein levels assessed by IHC and the corresponding gene expression measured by RT-PCR. Since the distributions were found to be bimodal for both genes, the midpoints between the two populations were used as cutoff points (2.5 CT for ER and 5.9 for PR). Both the ER (91 percent concordance; kappa = 0.83; P value = .0001), and PR (85 percent concordance; kappa = 0.70; P value = .0001) status proved to be highly concordant. According to the authors, this confirms the significant correlations between mRNA and protein levels for ER and PR and provided validation of their gene expression analysis.

In summary, this clinical study, in which the HOXB13-to-IL17BR index was developed, represents the foundation for using the two-gene ratio signature in tissue microarray FFPE specimens analysis (Table 11, Appendix I, Evidence Tables 10, 11 and 12).

Goetz et al., 2006.⁶² In this clinical study, FFPE tumors samples from 206 of 211 primary breast cancer patients were successfully processed by laser micro-dissection (LMC) prior to total RNA preparation. This study provides generic evidence about the analytic validity of the two-gene signature in primary breast cancer patients, as computed from LMC processed FFPE blocks (Table 9, Appendix I, Evidence Tables 10, 11 and 12).

Jerevall et al., 2007.⁶³ This paper quantified expression of HOXB13 and IL17BR (normalized to beta-actin) by RT-PCR in fresh frozen specimens from two distinct institutions in Sweden. RT-PCR reactions at the two institutions were performed using the same sets of primers and fluorescent probes, and two distinct instruments. Ninety-six percent of the 373 samples were successfully analyzed.

In summary, good reproducibility of the measurement between institutions was documented for each individual gene and the ratio (Pearson's correlation coefficient = 0.99, P value < 0.001) (Table 10, Appendix I, Evidence Tables 10, 11 and 12).

Oncotype DX

Paik et al., 2004.²⁸ This study was the first to validate the prognostic validity of Oncotype DX in a population independent from that used to develop the test. The population consisted of a sample of 668 (out of 2617) lymph node-negative, ER positive breast cancer patients from the tamoxifen-treated arm of the National Surgical Adjuvant Breast and Bowel Project (NSABP) Trial B-14. This 668-patient subset had enough analyzable tissue in paraffin blocks to be evaluated using the Oncotype DX assay, and was reported to be similar in baseline characteristics to the overall sample. A more complete sample was impossible because of sample unavailability or processing problems. In this study, the overall 10-year distant recurrence rate was 15 percent and the RS was significantly correlated with disease-free survival and overall survival (P<0.001 for both). The authors reported that RS alone was a better predictor of the distant recurrence risk than traditionally used predictors. In a multivariate model including age, tumor size grade, ER, PR, and HER, the RS Hazard Ratio was 2.81 (95 percent CI, 1.70–4.64, P<0.001, per 50 unit increase). Forty-four patients out of the 109 with small tumors (diameter less than 1 cm), were classified using Oncotype DX into the intermediate or high risk groups (Table 12, Appendix I, Evidence Tables 1, 2 and 4).

Esteva et al., 2005.⁴⁸ In this study the Oncotype DX assay was evaluated in a population of 149 patients treated at the MD Anderson Cancer Center between 1978 and 1995. These patients had been diagnosed with node-negative breast cancer and did not receive tamoxifen or chemotherapy, and had a median 18 year followup. The number of recurrences was not reported, and this study failed to find correlation between RS and distant breast cancer recurrence. ER, PR, and HER-2 showed no prognostic value, and well-differentiated tumors were correlated with worse survival than higher grade tumors, the reverse of expected. The population was unusual in that it received no treatment, and was different from the one used by Paik et al.²⁸ (Table 12, Appendix I, Evidence Tables 1, 2, and 4).

Cobleigh et al., 2005.⁴⁷ This report is the only study among the three used to develop the 21-gene Recurrence Score assay (Oncotype DX) to be published in a peer-reviewed journal. Seventy-eight breast cancer patients with more than 10 positive nodes from Rush University Cancer Center were studied, and 55 had recurred. Two hundred and fifty-five candidate genes were amplified with RT-PCR from FFPE tumor tissue obtained as long as 24 years ago. Twenty-two genes were significantly correlated with distant recurrence-free survival (DRFS) (unadjusted P value < 0.05). An RS was developed using these genes which very strongly predicted disease-free survival, but as this was training and not validation data, it has minimal evidential value in assessing Oncotype DX predictive properties (Table 12, Appendix I, Evidence Tables 1, 2 and 4).

Habel et al., 2006.⁵⁰ The Oncotype DX assay was used to assess the risk of breast cancer-specific mortality among women in a large case-control study population derived from fourteen Northern California Kaiser community hospitals with ER positive, node-negative breast cancer.

There were a total of 4,964 eligible patients, 220 had died and 570 were living controls. All were younger than 75 years old, diagnosed between 1985 and 1994, and had not been treated with adjuvant chemotherapy. For ER positive tamoxifen-treated patients, RS risk groups (as defined by pre-specified thresholds chosen by the test developers) showed similar 10-year risks of death from breast cancer (3 percent, 12 percent, and 27 percent respectively for low, intermediate, and high risk, groups) as Paik²⁸ reported for the NSABP B-14 patients. Multivariate analysis showed that RS and tumor size were significant and independent risk predictors of breast cancer death in both ER positive, tamoxifen-treated (hazard ratio per 50 units = 7.6, P<0.001) and untreated patients (RS hazard ratio per 50 units = 4.1, P<0.001). Tamoxifen-treated patients were shown to have a higher risk of death, and tumor grade proved to be a significant, independent predictor as well. The RS score showed some prognostic value in ER negative patients, although this group was too small to perform a reliable analysis.

ER status was missing from the medical record for a substantial proportion of patients in this study, and therefore ER status based on gene expression was used in the analysis. Cases and controls were matched with respect to tamoxifen treatment, so it was not possible to assess whether the RS was able to identify patients who are likely to respond to tamoxifen therapy. The performance of the Oncotype DX assay RS was not compared to standard risk stratification methods (e.g., St. Gallen, NIH criteria, or Adjuvant! Online) (Table 12, Appendix I, Evidence Tables 1, 2, and 4).

Paik et al., 2004,⁶⁵Bryant 2005,⁶⁶and Hornberger et al., 2005.⁶⁷ These posters showed the cross-classified risk predictions of the Oncotype DX assays compared to the risk stratifications using the 2004 NCCN and 2003 St. Gallen criteria, with the observed 10 year risks of relapse in the cross-classified strata. NCCN guidelines have since been modified, and the St. Gallen criteria did not accounted for HER-2. Patients came from the Paik⁶⁵ NSABP-14 validation cohort, N=668. Using the 2004 NCCN guidelines, the study indicated that of the 92 percent who were in the high-risk NCCN category, about half were reclassified as low-risk by RS, with a 10-year relapse risk of 7 percent (95 percent CI, 4–11 percent), which is similar to the risk observed in the low risk RS group, without the NCCN information⁶⁵. Finally, against the Adjuvant Online criteria, roughly 40 percent of those assessed to be at high risk (22 percent relapsed) were reclassified as having an 8 percent risk if they had a low RS score. These data, demonstrate that optimal predictions may come from a combination of expression predictors and standardized indices, although the latter contribute less than the RS to the risk estimate (Tables 13, 14, and 15).

MammaPrint

A synopsis of the clinical validity evidence presented in the following section is reported in Table 16. In the following section we will be distinguishing between MammaPrint, the marketed assay, and the gene expression profile which is the 70-gene signature originally published by van't Veer et al., in 2002.²¹

van't Veer et al., 2002.²¹ This study reported the development data for the 70-gene panel that is the basis for the MammaPrint test. A gene expression array containing 25,000 features was used to select genes associated with metastases-free survival at 5 years from surgery in 78 node negative patients, including 34 patients who recurred at 5 years and 44 who had not. Using the development of metastasis within 5 years as the first relapse event, 65 out of the 78 patients were correctly classified into good and poor prognosis groups by the 70-gene signature. Among the 13 misclassified patients, 5 patients with poor prognosis were in the good prognosis group, while 8 patients with good prognosis were classified in the poor prognosis group. Seventeen of 19 were correctly classified in the validation set.

The odds ratio (OR) to develop metastases within 5 years was 28, (95 percent CI, 7–107), while after leave-one-out cross-validation it was 15 (95 percent CI, 4–56). Using univariate analysis, the 70-gene signature performed better than tumor grade, size, patient age (less than 40years), ER status, and angioinvasion. Using multivariate analysis, the 70-gene signature was an independent predictor of metastases within 5 years, OR = 18 (95 percent CI, 3.3–94) (Tables 16 and 17, Appendix IEvidence Tables 6, 7 and 9).

van de Vijver et al., 2002.²⁵ This was the first major validation of the 70-gene signature as reported in van't Veer 2002 using the same protocol and approach. Banked tumor specimens from the Netherlands Cancer Institute were used from a consecutive series of 295 women with breast cancer, with a mix of lymph node positivity, ER status, chemotherapy, and tamoxifen treatment. Time to metastases, as well as overall survival (OS) were used as primary end points in survival models, and 61 patients in this cohort had been in van't Veer's²¹ original 78 patient training set.

Patients were young (less than 52 years) with small tumors (less than 5 cm). The 70-gene signature was shown to be associated with grade, size and ER positivity, with almost all of ER positive patients falling into the good prognosis category. Those with “good prognosis” 70-gene expression signatures had dramatically better 5-year (95 percent vs. 61 percent) and 10-year (85 percent vs. 51 percent) DRFS and OS (95 percent vs. 55 percent at 10 years) than the “poor prognosis” group. Multivariate analysis showed that the prognosis group, tumor size, and adjuvant chemotherapy were the strongest predictors of distant metastases. The “poor prognosis” signature had the largest hazard ratio = 4.6 (95 percent CI, 2.3–9.2). Analyses excluding the 61 previously-included patients produced similar results. Fourteen of the 115 “good signature” patients experienced a recurrence by 10 years, demonstrating that the “good prognosis” group may not be at low enough long-term risk to justify forgoing chemotherapy when the 70-gene signature is used alone.

The authors did not compare a regression-based predictor using only conventional variables with one including the 70 gene panel. However the authors demonstrated the prognostic value of the 70 gene index using survival curves stratified by the NIH and St. Gallen criteria, which showed substantial separation between 70-gene prognostic groups that were either low or high risk by those conventional indices. These stratified survival curves also showed that optimal prediction was achieved when the gene index and conventional predictors were combined (Table 16, Appendix I, Evidence Tables 6, 7, and 9).

Buyse et al., 2006.⁵⁹ This study compared the MammaPrint assay with the conventional combination risk predictors Adjuvant Online, Nottingham Prognostic Index, and St. Gallen. Patients were drawn from five distinct European institutions, in the context of an independent, multicenter validation study performed by the TRANS-BIG consortium. Gene expression in frozen tumor specimens from node negative patients younger than 60 years old who did not receive systemic adjuvant chemotherapy, and were diagnosed between 1980 and 1998 was characterized using the MammaPrint® assay. Final results were obtained for 302 out of 402 eligible patients. The median followup was 13.6 years, and the overall rate of distant metastasis was 25 percent.

The three primary end points of the study were time to distant metastases (TTM), DFS, and OS. The hazard ratios of the MammaPrint assay for TTM and OS were statistically significant after adjustment for St. Gallen, NPI and Adjuvant! On-line, but were generally far below (in the 1.5–2.5 range) that seen in the original validation cohort.^25,58 The partial explanation offered by the authors was that this study had a longer median followup time than the one used by the van de Vijver²⁵ cohort. Additionally, the authors introduced an interesting analysis showing the marked (3–6 fold) lowering of the hazard ratio for various endpoints when patients were artificially censored at increasing times, up to 10 years. Also, none of the ER positive patients reported in this study received hormonal therapy as did some of the original van de Vijver²⁵ cohort.

Specificity and sensitivity of the MammaPrint assay and the Adjuvant! algorithm were compared for distant metastases within 5 years and for death within 10 years. Similar sensitivities were found, but a higher specificity was demonstrated for MammaPrint. The areas under the Receiver operating characteristic (ROC) curves were comparable between MammaPrint and Adjuvant! (0.68 vs. 0.66 for distant metastases at 5 years). The use of alternative thresholds for the Adjuvant! Online results did not change the overall results, and Adjuvant! hazard ratios were greater than unity but not statistically significant when adjusted for the gene signature. Finally, there was no statistical heterogeneity in any outcomes between centers, suggesting that this prediction model has transportability across populations with possibly different genotypic patterns.

This study is particularly important in that it provided the first evidence for the degree of clinical validity of the MammaPrint assay distinct from the 70-gene signature. It provided insight into the impact of differing lengths of followup in validation cohorts, and concluded that the prognostic contribution was sizable. However, this study's predictions were made in the context of no adjuvant hormonal or chemotherapy treatment, thus its applicability to women over 60 years old and treated with tamoxifen is unknown⁶⁸ (Table 16, Appendix IEvidence Tables 6, 7, and 9).

Glas et al., 2006.⁵⁸ This study used the same patients as in the van't Veer,²¹ and van de Vijver²⁵ studies and compared the currently offered MammaPrint assay results to the results of the previous studies. RNA was available for all the 78 patients in the van't Veer series, but only 145 lymph node negative patients were available for reanalysis from the van de Vijver series. A different reference RNA was used, as well as a different quantification method, however odds ratios and hazard ratios were very similar. A total of 15 patients were incorrectly classified into discrepant risk categories. The results of the 70-gene signature used in the original cohorts therefore apply equally to the MammaPrint assay based on that signature (Table 16, Appendix IEvidence Tables 6, 7 and 9).

H/I Ratio

A synopsis of the clinical validity evidence presented in the following section is reported in Table 18.

Ma et al., 2004.⁶⁴ This study reported the development of the two-gene ratio predictor. The authors generated gene expression profiles with gene chips from whole and laser-capture microdissected (LCM) frozen tumor specimens from 60 ER positive, node positive or negative breast cancer patients all treated with adjuvant tamoxifen monotherapy. Twenty-eight of the cohort (46 percent) experienced a distant recurrence within 4 years and 54 percent had no recurrence by 10 years. Twenty-two thousand genes were screened in the whole tissue sections and in LMC samples for their ability to predict DFS. Only three genes were highly predictive of DFS in both tissue sets, with over-expression of HOXB13 predicting recurrence and over-expression of IL17BR predicting non-recurrence. These expression values were combined in the form of a ratio, which outperformed both existing biomarkers and either gene alone. The univariate OR (interquartile) was 10.2 (95 percent CI, 2.9–36), multivariate OR was 7.3 (95 percent CI, 2.1–26.3) with adjustment for tumor size, PR and ERBB2 (none statistically significant) in a logistic regression. Area under the receiver-operating-characteristic curve (AUCs) for the ratio were reported in the 0.8 range.

Next, the above analysis was repeated using just the two-gene ratio calculated by RT-PCR on 59 fresh-frozen samples from the training set along with 20 additional FFPE specimens to independently validate the ratio. Sixteen of these 20 were accurately predicted. The RT-PCR-measured expression was reported to have similar predictive power to that measured via gene arrays. No comparison with the full array of clinical predictors (e.g. tumor grade) or with standard combination predictors (e.g., Adjuvant!) was performed (Table 18, Appendix I, Evidence Tables 10, 11, and 13).

Reid et al., 2005.⁶⁹ In this paper the authors attempted to validate the two-gene ratio on an independent cohort of 58 patients with ER positive breast cancer. These patients had been treated with tamoxifen monotherapy, had larger tumors, a higher frequency of lymph node metastases (78 percent vs. 47 percent), and a higher HER-2 positivity (21 percent vs. 5 percent) than those in the Ma et al., 2004 study. Eighteen patients had distant recurrences within a median time of 31 months, and 40 had no recurrence after a median of 93 months (range 70–125). The expression of the genes HOXB13 and IL17BR was measured by RT-PCR and the association between their expression and outcome was assessed by use of univariate logistic regression, AUC, a two-sample t test, and a Mann-Whitney test. None of these analyses revealed any statistical relationship with outcome.

The authors then took the original data of Ma et al.⁶⁴ and applied standard supervised methods to this and to another independent data set with 99 similar patients.⁷⁰ They tried to estimate the classification accuracy obtainable by using two or more genes in a microarray-based predictive model, using linear discriminant analysis and extensive cross-validation. The authors failed to validate the two-gene ratio and found high error rates with two-gene predictors.

Overall, findings from this paper argued against the prognostic utility of the two-gene ratio in ER positive breast cancer patients treated with tamoxifen. However, it must be noted that a different part of the transcripts were assayed in the two studies and that differences could be due to the documented differences in the populations used, which were neither clinically nor therapeutically homogeneous, with small validation sets⁷¹ (Table 18, Appendix I, Evidence Tables 10, 11 and 13).

Goetz et al., 2006.⁶² To investigate the prognostic performance of the two-gene ratio, this study analyzed FFPE samples from 206 ER-positive patients treated in the tamoxifen-only arm of a Phase III randomized trial of tamoxifen alone versus tamoxifen plus fluoximesterone conducted through the NCCTG (North Central Cancer Treatment Group).⁶⁴ RT-PCR expression values for each gene were normalized using a standard curve (Appendix D) obtained by analyzing the human universal total RNA (Stratagen, La Jolla, CA), rather than the standard reference gene method, although the authors stated that control genes were not necessary to assess the expression ratio. The following end points were considered: RFS (time from randomization to any event of recurrence, contralateral breast cancer or death), DFS (time from randomization to any event of recurrence, or contralateral breast cancer, or other cancer, or death), and OS (time from randomization to death).

Cutoffs points that best predicted RFS, DFS and OS were identified: the optimal cut-off for the entire cohort was -1.85, corresponding to the 58th percentile, whereas the 59th percentile (-1.34) was used for the node-negative group (n = 130), and the 90th percentile (4.4) best discriminated in the node positive group (n = 86).

The ratio showed modest outcome prediction value in the entire cohort, with cross-validated hazard ratios near 1.5 and P values around 0.05, with the predictive value being restricted to the node-negative subset of patients (hazard ratios 1.7 to 2, P values = 0.04–0.06). In the node-positive group the ratio had no relationship to relapse or survival. The authors concluded that a high 2-gene expression ratio is associated with increased relapse and death in patients with node-negative, ER positive breast cancer treated with tamoxifen.

Overall this study provided some support of the two-gene ratio signature's prognostic value in ER positive, lymph node negative patients, but both the magnitude of that effect and the statistical support were modest, and the relevant cutoffs used for discrimination between high and low risk were optimized for each endpoint and patient subgroup. Hence, this is closer to a training than validation exercise (Table 18, Appendix I, Evidence Tables 10, 11 and 13).

Ma et al., 2006.⁶¹ This study examined a consecutive series of patients from Baylor University diagnosed between 1973 and 1993 with stage I or II breast cancer. The patients did not have distant spread, and non-relapsed cases had a median followup of 6.8 years. The authors reported data on the clinical validity of the two-gene expression index (HOXB13:IL17BR), which is the base of the H/I assay. A different normalization strategy (Table 1) from Ma et al., 2004⁶⁴ was applied to obtain this index. FFPE samples only yielded 852 analyzable cases out of 1,002 patients.

This population had 72 percent node negative, 73 percent ER positive, and 16 percent HER-2 positive patients, with an overall recurrence rate of 31 percent. A higher HOXB13:IL17BR index was associated with a higher risk of relapse (hazard ratio=1.5, P<0.001). In a stratified analysis, univariate Cox regression indicated that the HOXB13:IL17BR index was only significant in node-negative patients (hazard ratio = 1.6, P<0.001 vs. hazard ratio=1.2, P=0.1,) and further subsetting indicated that the interaction with node status was statistically significant for the HOXB13:IL17BR index (P= 0.02) only in ER positive patients. The HOXB13:IL17BR index correlated significantly with predictors of poor prognosis (i.e., HER-2 amplification, S-phase fraction, and number of positive lymph nodes) and correlated inversely with ER and PR expression.

The authors identified the optimal cut-off point for the index by analyzing a training set of ER-positive untreated patients (n=205), in order to obtain the smallest P value from a log-rank test in Kaplan-Meier survival analysis. The selected threshold (of about 1.0) was validated in a separate test set of untreated patients (n=103), and was also applied in the analysis of the tamoxifen-treated group of patients (n=122). Kaplan-Meier curves and univariate Cox regression analysis indicated that this cut point stratified patients into significantly different risk groups. Results from the Kaplan-Meier plots suggested that the prognostic power of the two-gene index was independent of tamoxifen therapy. The hazard ratio obtained in multivariate Cox proportional hazards regression, incorporating age, tumor size, S-phase fraction, PR status, and tamoxifen therapy, confirmed the prognostic role of the HOXB13:IL17BR index (hazard ratio=3.9, 95 percent CI = 1.5 to 10.3, P value = 0.007), in ER positive, node negative, patients irrespective of tamoxifen treatment. The index was also demonstrated to be a continuous predictor of DFS in untreated patients. The authors concluded that the two-gene index was a significant predictor of clinical outcome in ER positive, node-negative, patients regardless of tamoxifen therapy.

This study validated the two-gene ratio gene expression profile, developed the two-gene index, and provided preliminary evidence for its prognostic value. Classification probabilities were not presented, and its incremental value over conventional predictors was not reported, although some components of such predictors were included in the multivariate analyses (Table 18, Appendix I, Evidence Tables 10, 11 and 13).

Jansen et al., 2007.⁷² This clinical study evaluated the ability of the HOXB13-to-IL17BR expression ratio to predict DFS in breast cancer patients treated with tamoxifen. The HOXB13 and IL17BR expression levels were measured by RT-PCR in 1,252 primary breast tumor patients and normalized with respect to 3 housekeeping genes⁷³. The study population was a mix of ER-positive (73 percent), lymph node-positive (52 percent), tamoxifen-treated (14 percent), and chemotherapy-treated (17 percent) patients, with additional patients treated with tamoxifen or chemotherapy after relapse (55 percent). Patients with ER-positive tumors with node negative primary breast cancer (N = 468) were followed for DFS. Patients with recurrent breast cancer treated with first-line tamoxifen monotherapy (N = 193) were followed for progression free survival (PFS). This study used different populations, protocols, normalization strategy, and ratio thresholds than Ma et al. 2006.⁶¹

The study evaluated the relation between the HOXB13-to-IL17BR ratio and tumor aggressiveness in lymph node negative, ER positive patients who did not receive adjuvant systemic chemotherapy (N=468). Of these patients, 46 percent had a relapse during the followup period. The HOXB13-to-IL17BR ratio, as a univariate continuous variable, was significantly associated with a poor DFS (hazard ratio=1.6, P=0.02) and a poor OS (P<0.001, data not reported). When traditional factors were added to the model, the HOXB13-to-IL17BR ratio continued to contribute significantly to DFS and OS prediction, either as a continuous variable or after dichotomization according to published pre-specified thresholds⁶¹ (Table 18).

The same analysis was performed on ER-positive, lymph node-positive tumors from untreated patients, who were mainly enrolled in the early 1980's (n=151). Univariate analysis of the continuous HOXB13-to-IL17BR ratio was associated with a poor DFS and a poor OS. In the multivariate model for this population, the index was significantly associated with OS (P value = 0.001), but less strongly with DFS (P value = 0.065). The dichotomized index was not related to DFS (data not shown).

Finally, the authors evaluated the prognostic performance of the HOXB13-to-IL17BR ratio in 193 ER-positive primary breast tumors in relapsed patients treated with first-line tamoxifen monotherapy. Both univariate and multivariate analyses revealed that the ratio, continuous and dichotomized, was strongly associated with PFS (Table 18).

This study is by far the largest done so far concerning the potential value of the 2-gene ratio. It provided evidence of the clinical validity of the HOXB13-to-IL17BR ratio in ER positive, node negative patients who did not receive systemic adjuvant therapy, and also in ER positive relapsing patients whose relapse was treated with tamoxifen. However, the study was calculated and dichotomized in a somewhat different manner than in Ma et al., 2006.⁶¹ Additionally, comparisons were not provided with conventional combination risk indices, nor were classification probabilities provided for the models with and without the ratio. Therefore, incremental predictive values could not be accurately assessed. Although qualitative conclusions are not affected, there are some differences between the quantitative results reported in the text and tables (Table 18, Appendix I, Evidence Tables 10, 11 and 13).

Jerevall et al., 2007.⁶³ In this paper the authors investigated whether the two-gene ratio can predict the benefit of 2 versus 5 years of tamoxifen treatment in postmenopausal breast cancer patients, and also predict the ratio's prognostic value in systematically untreated pre-menopausal patients. Expression of HOXB13 and IL17BR were quantified by RT-PCR in tumors from 264 randomized postmenopausal patients and 93 systemically untreated premenopausal patients. The two study populations were collected as part of a collaborative study between two centers in Sweden, and 72 percent of the randomized patients were lymph node positive and 74 percent ER positive. To stratify the patients into risk groups the authors dichotomized the ratio using the median, a procedure and dichotomization differing from the approach used by Ma 2006.⁶¹ The results from the prediction of prolonged treatment benefit are reported under Key Question 4, Clinical Utility.

The ratio proved to be significantly correlated to tumor size, ER, PR, HER-2, Nottingham histologic grade (NHG), ploidy, and S-phase. ER, HER-2, S-phase and NHG correlations were mostly due to IL17BR, while PR and ploidy correlations showed contribution from both genes. The authors concluded that a lower expression of IL17BR, but not HOXB13, was correlated to several factors related to poor prognosis, and thus IL17BR might be an independent prognostic factor in breast cancer, and that HOXB13 may be correlated with tamoxifen resistance. However, the ratio had no prognostic value in ER negative postmenopausal patients and they were excluded from subsequent analyses.

In summary, this study produced additional developmental evidence of the prognostic value of the HOXB13-to-IL17BR ratio, and of the two individual genes, in ER positive breast cancer patients who received systemic adjuvant therapy. However, neither the patient profile nor the mode of calculation of the ratio were identical to previous studies, and the results differed from previous reports, as the ratio predicted for worse outcome in lymph node positive patients (Table 18, Appendix I, Evidence Tables 10, 11 and 13).

Oncotype DX

Currently a prospective randomized clinical trial, TAILORx, is underway with the goal of assessing the value of adjuvant chemotherapy among patients with mid-range RS results. However, one other published study does address the potential value of the RS in predicting chemotherapy benefit.

A synopsis of the clinical utility evidence presented in the following section is reported in Table 19.

Paik et al., 2006.⁵³ The authors used the Oncotype DX assay to investigate whether the RS was a predictor of the benefit from chemotherapy in ER-positive, lymph node negative, breast cancer patients. This study used 651 patients from the NSABP B-20 randomized trial and compared a group treated with both tamoxifen and chemotherapy with a group of patients who were randomized to tamoxifen only. Gene expression analysis was found to be correlated with chemotherapy benefit, defined in terms of 10-year distant recurrence-free survival (DRFS).

Kaplan-Meier analysis on all patients showed a significant benefit from the use of chemotherapy (P value = 0.02), however when the data was stratified by RS risk groups, only the high RS risk group of patients benefited from using chemotherapy (P value = 0.001).

When the authors used multivariate Cox proportional hazard analysis, findings about the benefit from chemotherapy use were unclear due to large confidence intervals in the low and intermediate RS risk groups (low RS risk group, RR=1.31; 95 percent CI: 0.46–3.78; intermediate RS risk group, RR = 0.61; 95 percent CI, 0.24 to 1.59). Patients classified in the high RS risk group, however, showed a significant benefit from the use of chemotherapy (RR=0.26; 95 percent CI: 0.13–0.53).

The authors also looked for interaction between each variable and chemotherapy treatment using separate likelihood ratio tests. The RS was the only significant interaction (P=0.038), with only slight statistical weakening when age, tumor size, tumor grade and site were added to the model individually (P values from 0.035 to 0.068). When RS was fit as a continuous score, there was not a clear threshold that predicted no benefit for chemotherapy.⁵³

Overall, this study produced preliminary, high-quality evidence that the RS from the Oncotype DX assay has clinical utility, i.e. predictive power in assessing the benefit of chemotherapy usage in ER-positive, lymph node negative breast cancer patients. The embedding of this study within a large, well conducted RCT was a strength. However, some patients from the tamoxifen-only arm of the NSABP B-20 trial were in the training data sets for the Oncotype DX assay. While the algorithm was trained for the outcome of recurrence and not chemotherapy benefit, optimization of recurrence prediction in one arm of this study could translate into a somewhat enhanced estimate of chemotherapy benefit, although it is unlikely to account for the large effect seen here. Finally, while the models could not sustain the inclusion of all possible clinical variables, they could have included a composite score, either standard risk predictors, or one tailored for the data set (Table 19, Appendix I, Evidence Tables 1, 2 and 4).

MammaPrint

No published studies evaluated the ability of the 70-gene signature for the main MammaPrint assay to predict chemotherapy benefit.

H/I Ratio

Jerevall et al., 2007.⁶³ This paper investigated whether the two-gene ratio can predict the benefit of 2 years versus 5 years of tamoxifen treatment in postmenopausal breast cancer patients, and also predict the prognostic value in systematically untreated premenopausal patients. Expression of HOXB13 and IL17BR were quantified by RT-PCR in tumors from 264 randomized postmenopausal patients and 93 systemically untreated premenopausal patients. The two study populations were collected as part of a collaborative study between two centers in Sweden, and 72 percent of the randomized patients were lymph node positive and 74 percent ER positive. To stratify the patients into risk groups the authors dichotomized the ratio using the median. Thus the normalization procedure and dichotomization differed from the approach used by Ma.⁶¹ The prognostic results from this study are reported under Key Question 3 (clinical validity).

Kaplan-Meier analysis of data from postmenopausal ER-positive patients demonstrated that a low HOXB13-to-IL17BR ratio was associated with a benefit to receiving 5 vs. 2 years of tamoxifen treatment (univariate P= 0.021; in KM analysis). There was no benefit (P=0.9) in patients who had a high ratio, which mainly appeared due to the low expression of HOXB13 genes (P= 0.010, in Kaplan-Meier analysis). The predictive significance of both the two-gene ratio and the HOXB13 gene alone was maintained using a Cox proportional hazard modeling, adjusting for tumor size, PR status, and lymph node status.

The authors concluded that the ratio, or even HOXB13 alone, could predict the benefit of prolonged endocrine therapy, and that a lower expression of IL17BR, given its correlation to poor prognosis, could be an independent prognostic factor.

Neither the patient profile nor the mode of calculation of the ratio were identical to previous studies (Table 21, Appendix I, Evidence Tables 10, 11 and 13). However this study produced additional developmental evidence about the prognostic utility of the HOXB13-to-IL17BR ratio, and of the two individual genes, in ER positive breast cancer patients who received systemic adjuvant therapy.

TAILORx (Trial Assigning IndividuaLized Options for Treatment (Rx))

The primary objective of TAILORx is to compare the DFS of women with previously-resected axillary-node-negative breast cancer who have an Oncotype DX RS of between 11 and 25 when treated with both adjuvant chemotherapy and hormonal therapy versus hormonal therapy alone. It should be noted that this range is lower on both ends than the standard “Intermediate” RS range, viz. 18–30. This represents a more conservative approach to the use of the RS than is suggested by current categories, in that subjects who agree to forego chemotherapy in this trial will be at lower risk than those in the current “low risk” RS group. The secondary objective is to determine if adjuvant hormonal therapy alone is sufficient treatment (i.e., 10-year distant DFS of at least 95 percent) for patients with an RS of less than or equal to 10.

This study will not provide direct evidence for the value of Oncotype DX, as all patients in the trial will receive the test. The trial results will indicate whether adjuvant chemotherapy is of value within the trial's intermediate RS range, and will serve as further validation of the absolute risk of recurrence in subjects with scores above and below the range. This will provide better estimates of the degree of benefit from utilization of the test, but will not directly examine what therapeutic choices would have been made and clinical outcomes incurred if only standard risk prediction tools were used. However, since standard risk prediction indices will be calculable, that information may be inferred. First results from this trial are expected in approximately 2013.

MINDACT (Microarray for Node-Negative Disease may Avoid Chemotherapy)

MINDACT is a multi-center, prospective, phase III randomized study comparing use of the MammaPrint assay with a common clinical-pathological prognostic tool, Adjuvant! Online, to select patients for adjuvant chemotherapy in node-negative breast cancer. Patients at low risk by both MammaPrint and standard clinical-pathological criteria will not receive chemotherapy, patients at high risk by both criteria receive chemotherapy, and patients with discordant criteria will be randomized to use either MammaPrint only or standard criteria to decide treatment (i.e., randomized to receive adjuvant chemotherapy or not). This will directly test whether the choice of chemotherapy guided by MammaPrint provides benefit over that guided by the Adjuvant! criteria.

Analytic Validity

Analytic validity evidence now exists for some of the operational/laboratory characteristics/procedures of this test, as well as about its reproducibility, although information about this latter point is limited to a few repeated analyses. These studies demonstrated that the reproducibility of the test across different samples of the same block, and samples from different blocks, is reasonably high.⁴⁵ The test involves not only the simple assessment of the RNA levels by RT-PCR, but also the preparation of the RNA, following a central review of the specimens shipped to Genomic Health to check for tumor content. No direct evidence is available about the sample preparation aspect of the test, although there is indirect evidence from peer-reviewed literature in the form of the overall success rate of extracting analyzable mRNA, which appears to be fairly high. Centralization is a current strength of Oncotype DX with regard to reproducibility, but additional scrutiny may be needed if other laboratories offer such testing in the future.

Clinical Validity

“Clinical validity” is defined here as the ability of a prediction test to accurately predict risk. Whether or not those risk predictions differ enough to justify its use in a clinical setting (i.e., whether discrimination is sufficient) is a second issue. The clinical validity of Oncotype DX has been evaluated in various settings. The first validation study²⁸ used tamoxifen-treated women with ER positive, lymph node negative breast cancer, from the randomized clinical trial NSABP B-14. This study independently validated the prognostic value of the RS, which had been previously tested in the tamoxifen-treated population of NSABP B-20. Perhaps the most important aspect of this population is that it was clinically and prognostically well defined, in that everyone was presumed to be eligible for chemotherapy, and all subjects had similar treatment (i.e., tamoxifen), making for a relatively clear interpretation of the results in terms of both treatment biology and clinical decisionmaking. Predictors of response on a specified therapy are not necessarily prognostic factors independent of that therapy, so studies which mix treated and untreated patients, or patients differently treated, can produce results that do not apply well to either. While this study took place in the past, all measurements were done concurrently independently of the outcome, and so has evidential value quite close to that of a concurrent prospective study. The main issues raised by the non-concurrency are whether the 668 subjects examined were a representative sample of the more than 2000 in the original study, and the degree to which the findings in tamoxifen-treated women will apply to aromatase inhibitor treated patients today, the role of HER-2 testing and treatment, and whether there was anything clinically relevant about how the early stage cancer was diagnosed (e.g., clinically or by mammogram) that might differ today.

While this study reported hazard ratios for the RS in the presence of clinical predictors, it did not provide predictions by the RS cross-classified by those of standardized combination risk predictors to see exactly how many women would be re-classified in risk strata that might change decisions regarding chemotherapy. This information was however presented in poster form in 2004⁶⁵ and it showed that the RS had considerable predictive power beyond that of the St. Gallen or NCCN risk stratification guidelines (n.b. St. Gallen did not include HER-2 at that time). Another poster⁶⁶ showed the same information for Adjuvant! Online (Tables 13 and 14, Chapter 3). All of these cross-classifications showed that the greatest contribution of the test was likely to be in the reclassification of patients from high to low risk, i.e., in reducing the number of patients who might unnecessarily undergo chemotherapy. It also showed that optimal predictions probably would be achieved with a combination of both expression and clinical predictors. It must be stressed that the cross-classified risk of patients in the low-risk RS groups in these posters does not represent the lowest attainable risk; they are an artifact of the “low risk” category threshold. Patients who had low absolute RS scores would be predicted to have lower risks than the “low risk” category average, and those lower risks are probably low enough for many women classified as high risk by other indices to change their treatment decisions. One very important remaining question is the degree to which the absolute observed risks in this population, particularly in these lowest risk groups, are similar to other populations, and to those whom it is currently being used, i.e. whether the calibration of the test predictions will vary. The low risk arm of the TAILORx trial, in which patients with a RS less than10 will not be treated with adjuvant chemotherapy, will help address this question.

A second large study looked at the clinical performance of the Oncotype DX assay to predict breast cancer death (at 10 years) in a community-based population of ER positive, node-negative patients treated with tamoxifen, confirming the B14 results among the tamoxifen treated patients, and showing predictive value, albeit lesser, in ER positive patients not treated with tamoxifen.⁵⁰ The Esteva study,⁴⁸ which showed no predictive value of RS in a small population of patients who received neither tamoxifen nor chemotherapy, showed such anomalous results with standard predictors (i.e. higher grade predicting better prognosis) that its results cannot be regarded as reliable. Finally, the Fan study,⁷⁹ while testing the RS signature measured by microarrays and not the actual Oncotype DX test based on RT-PCR, showed good discriminatory power in a relatively large, independent dataset, albeit with a heterogeneous mix of treatments, receptor and nodal status. This is the same dataset on which the MammaPrint signature was developed.

These studies in combination provide fairly strong support for the clinical validity of the Oncotype DX test over and above standard predictors, in a well defined population (ER positive, lymph node negative, tamoxifen treated) with clear treatment indication (adjuvant chemotherapy). Exactly how much it adds, however, exactly what proportion of these patients would benefit from its use, and the stability of the observed risk in the various risk categories in other (or current) populations, is not as clear. Discussion will continue below about its use in a clinical setting.

Clinical Utility

Clinical utility is the degree to which a test is predictive of treatment benefit, and hence is a critical foundation for the use of a test in clinical decisionmaking. Prognostic ability itself speaks to this to some degree, as it puts a ceiling on the degree of clinical benefit. For example, if the 10 year distant relapse rate is 5 percent, by definition additional treatment cannot provide more than a 5 percent absolute benefit, and background knowledge about treatment efficacy tells us it will be less. So if the risk of distant recurrence can be reliably established as low enough, this has clinical utility in itself.

However, it is of considerable value to have a direct estimate of the degree of treatment benefit. This can only be done reliably in the context of RCTs, prospectively or retrospectively, as they assess treatment effect in an unbiased manner. This was addressed by Paik et al. in their study of the correlation of the RS with the degree of adjuvant chemotherapy benefit in the context of the NSABP-20 trial.⁵³ This showed that the chemotherapy benefit in ER positive, node negative patients randomized to tamoxifen versus tamoxifen plus chemotherapy was almost entirely restricted to those in the high risk RS category. The CIs in the low and intermediate risk categories were wide and included the possibility of benefit whereas the CI for the high risk group was narrow and showed clear benefit. A statistical interaction was also found with patient age, although those data were not reported. The only caveat is that the tamoxifen arm of this population was part of the training set for the assay, although the outcome measure used in the training set was not treatment benefit. It is not clear whether the information in clinical predictors was optimally used (i.e., as continuous rather than dichotomous variables), but that is unlikely to have accounted for the degree of differential effect predicted by the RS. HER-2 positivity reportedly had no effect on the results. Several other studies evaluated the value of the RS information in different populations of patients to predict other correlates of treatment effect. For example, evaluation of pathologic response after preoperative chemotherapy^49,55 supports clinical utility, although that was evaluated in patients in whom chemotherapy was already determined to be necessary.

The NSABP-20 study probably represents as strong evidence as can be derived from already existing data regarding the clinical utility of the Oncotype DX test. While prospective confirmation of these findings are definitely needed as well as analysis of existing patient samples from other completed trials, this provides reasonable justification in the interim for the use of the test by women in this specific population.

Use in clinical decisionmaking. One published study has reported the impact of using the RS on clinical management,⁵⁶ and there have been examinations of the economic implications of testing.⁷⁵ In general, studies showing that physicians change recommendations or that woman change treatment decisions in response to their Oncotype DX risk category are minimally informative if the study is not designed to specifically explore the woman's risk thresholds for making that decision. The reported study does not specify what information was conveyed to the patients, i.e., a risk score, the risk category, or the risk itself. If the latter, the number they were told is important to know. In the absence of this information, it is not possible to know the threshold of risk below which most women (or any given proportion) would forego chemotherapy, or conversely, the risks at which they would choose it. In the absence of such information, it cannot be known whether the study is effectively examining compliance with physician recommendations, careful weighing of risks and benefits, or the effect of test marketing.

There are still uncertainties about the optimal use of this test in practice. First, while the cut-offs are valuable for test validation purposes, it is not clear whether the current thresholds actually correspond to the cutoffs that would be derived using a formal decision-analytic approach based on utility assessments. For an individual woman, a risk based on her exact RS value would be preferable, since by definition, those with RS scores near the upper boundary of the “low risk” range have a predicted risk higher than the average of the group, and those with low scores have lower risk. The fact that the boundaries used in the studies may not be optimal for decisionmaking is seen in the different cut-offs used by the TAILORx trial, in which the low-risk group is defined as RS less-than or equal-to10 instead of 18, and the high risk group is defined as greater than 25 instead of 30.

The second uncertainty is the optimal use of conventional predictors. While the RS has been shown to have more value than most predictors, the same studies show that clinical predictors retain predictive value, and clinical prediction models continue to evolve and improve. An improved prediction tool would involve a combination of the expression-based and clinical predictors, but this has not been systematically explored in any study, and absolute risks produced by regression models or stratified tables with all predictors included are generally not reported. As noted previously, cross-classification data using the most updated standardized clinical indices would be one form of such data, although those do not show the risk from combinations of the exact RS and clinical predictor score.

Cost-effectiveness. While our review highlights many gaps in what is known about the clinical utility of using gene expression profiling in women diagnosed with breast cancer, the review also revealed that little is known about the cost-effectiveness of using these tests. Once studies have demonstrated the clinical utility of these gene expression profiles, policy makers and health care providers will need information about the cost-effectiveness of those tests that have proven utility. Such information will be particularly important given the relatively high expected costs of the tests. Oncotype DX, for example, costs more than $3000 for each use of the test.

In our review, we found three published studies that have addressed economic outcomes associated with use of the breast cancer gene expression tests. One study reported that using the 21-gene RT-PCR assay to reclassify patients would be cost-effective for those who were defined by 2005 NCCN criteria as low risk ($31,452 per quality-adjusted life-year (QALY) gained) and would be cost-saving for those who were defined by NCCN criteria as high risk.⁶⁷ The EPC team had only moderate confidence in these projections because the study did not provide enough information about potential sources of bias in the analysis, allied with the fact that the study was supported by the manufacturer, which may introduce conflict of interest. The 2007 NCCN guideline now indicates that use of chemotherapy in these patients is optional, further diminishing the value of these projections.

The second study reported that use of the 21-gene RT-PCR assay was associated with a cost-utility ratio of $4432 per QALY compared with use of tamoxifen alone, and a gain of 1.71 QALYs with net cost savings when compared with chemotherapy plus tamoxifen.⁷⁵ The EPC team had little confidence in this analysis, which was supported by the manufacturer, because it did not meet many of the standards that were used for appraising the quality of the analysis.

The third study compared the cost-effectiveness of the Netherlands Cancer Institute gene expression profiling (GEP) assay (MammaPrint) to the U.S. National Institutes of Health (NIH) guidelines for identification of early breast cancer patients who would benefit from adjuvant chemotherapy. The GEP assay was projected to yield a poorer quality-adjusted survival than the NIH guidelines (9.68 vs. 10.08 QALYs) and lower total costs ($29,754 vs. $32,636). To improve quality-adjusted survival, the GEP assay would need to have a sensitivity of at least 95 percent for detecting high risk patients while also having a specificity of at least 51 percent. The EPC team had confidence in the results of this analysis because it met most of the standards for appraising the quality of an economic analysis.

Since the overall body of evidence is inconclusive about the economic outcomes associated with use of breast cancer gene expression tests, this is an area that will require further investigation. Future economic analyses of validated tests should take into consideration existing guidelines for the performance and reporting of such analyses.⁴¹ Ideally, the analyses should be performed by investigators who have not received financial support from manufacturers of the tests.

Questions Regarding the Clinical Validity and Utility of the Oncotype DX Assay

• 1

  Better information is needed about the predictions from combining the RS with current versions of standardized risk predictors, both in the form of cross-classification tables, and perhaps of regression-based combinations that optimize individual risk predictions. Formal development of cutoffs to optimize patient utility are also needed.

• 2

  While Oncotype DX exhibits a fair bit of risk discrimination (i.e., separating patients into different risk groups), the stability across different populations of the observed absolute risk in patients with a given risk score (i.e., calibration) needs further study. Of greatest interest is the observed risk in the lowest risk groups, since the absolute level of this risk is critical for informed decisionmaking, and patients may forego chemotherapy on the basis of this information.

• 3

  Data are currently available mainly for tamoxifen-treated patients and for those treated with cyclophosphamide-methotrexate-5-fluorocil chemotherapy. It is important to assess whether RS applies to other hormonal treatments such as aromatase inhibitors, as well as more contemporary chemotherapy regimens using taxanes and anthracyclines.

• 4

  It is not clear whether RS can be used to help guide treatment of HER-2 positive patients and additional studies are needed, as most of these patients were classified in the high RS group in the initial trials.

• 5

  While awaiting the TAILORx results, the findings of the Paik 2006⁵³ study predicting treatment benefit need independent confirmation, particularly for low and intermediate risk groups.

• 6

  Studies examining the use of Oncotype DX should provide women and physicians with quantitative risk information and report how this alters clinical decisionmaking. The manner in which this risk information is presented should also be studied.

Analytic Validity

This assay is the first microarray-based test introduced in the field. Two recent papers addressed issues related to the reproducibility of the test within laboratories, as well as across laboratories. Such evidence however was obtained from a limited number of patients and using a moderate number of replication experiments. Results showed a good reproducibility within a laboratory, and a good degree of agreement across laboratories, although RNA labeling emerged as a possible source of variation capable of affecting the results. Whether this issue has an impact on risk classification was not thoroughly investigated, and thus the portability of the result of the assay from one laboratory to another still remains open. A second relevant point is the fact that the only validation study using the MammaPrint assay showed that only about 80 percent of specimens from the field (in this case 5 different European institutions) were analyzable, raising some concern about the analysis of fresh-frozen specimens. As more patients are analyzed by this test, the overall success rate may change. Finally, it must be noted that although this technology requires fresh rather than paraffin embedded specimens, Agendia performs a central pathologic review of the specimens as is performed with FFPE samples at Genomic Health, before evaluation with the test.

Clinical Validity

Overall, published evidence supports MammaPrint as a better predictor of the 5-year risk of distant recurrence than traditionally used tumor characteristics or algorithms. However, the cohorts in whom it was developed and validated are more clinically heterogeneous than those used for the Oncotype DX test, with a mix of lymph node status, ER status and current treatment. Additionally, evidence was derived only from patients younger than 55 to 60 years of age. Even so, it is interesting that it had 80 percent concordance with the array-based RS classification when applied to the same patients, although it remains to be seen how well it predicts in cohorts with the same degree of clinical and treatment homogeneity as used in the Oncotype DX development, and which differ from its training set. Evidence about its value in comparison with clinical predictors was assessed in a collaborative study among 5 different institutions in Europe, where data were compared to standard clinical predictors like Adjuvant!.⁵⁹ The area under the receiver operating characteristic curves was 0.68 for MammaPrint and 0.66 for the Adjuvant! Score. Such estimates indicate that both methods have apparently similar and modest discriminatory power in absolute terms. Similar results were obtained also using the ten year overall survival end point. However, when Adjuvant! and MammaPrint were cross-classified against each other, Adjuvant! had no additional predictive value. Adjustment for other predictors (St. Gallen and the Nottingham Prognostic Index) had a minimal effect on the regression coefficient of MammaPrint score or its significance, but no other data were reported on their incremental value. Of note, no significant heterogeneity in the hazard ratio estimates was shown among centers, although original hazard ratio estimates were significantly higher than those obtained in this validation study. The validation cohort had longer time of observation, included older women, and excluded patients who received adjuvant therapy.

Clinical Utility

No studies evaluated clinical utility of this test.

Use in clinical practice. No studies explored the use of this test in clinical practice.

In summary, MammaPrint is the first commercialized microarray-based gene expression profile with a prognostic purpose. The underlying signature has been evaluated in approximately 700 patients, although MammaPrint itself has only been evaluated in one study of 307 untreated patients. A reanalysis of the original training data of the signature using the marketed test showed a net reclassification of only one patient of 78.⁵⁸ It is unclear what population of patients would derive benefit from use of the test, and what the magnitude of that benefit would be. Prospective data from trials like MINDACT will be extremely valuable. Overall, published evidence supports MammaPrint as a better predictor of the risk of distant recurrence than traditionally used tumor characteristics or algorithms, but its performance in therapeutically homogeneous populations is not yet known with precision, and it is unclear for how many women the lowest predicted risks are low enough to forgo chemotherapy. No evidence is available to permit conclusions regarding the clinical utility of MammaPrint to select women who will benefit from chemotherapy.

To conclude, the literature on the 70-gene signature includes numerous studies that focused more on its biological underpinning and less on the clinical implications of this gene expression profile, although it has now received FDA approval for clinical use. It has been shown that this signature is maintained along the cancer progression process.⁸² This profile was directly investigated by two different platforms (microarray and RT-PCR,⁴⁸), and was successfully re-implemented in two distinct microarray platforms, showing that it has a fair degree of analytic robustness.

Here we summarize open questions as well as research gaps found in the evidence about the clinical validity and utility of the MammaPrint assay and the 70-gene signature.

• 1

  The prognostic value of the 70-gene signature has been assessed in different populations facing different therapeutic choices. In the analysis by van de Vijver and colleagues, 130 of the 295 patients received adjuvant therapy in a non-randomized fashion. Patients in the original development cohort were not treated, and Buyse validated the marketed assay in untreated patients. It is not yet clear which are the optimal patient populations for the use of this test, exactly what its performance is in those populations, and how many of its predictions would result in different therapeutic decisions. Larger independent validation studies in therapeutically homogeneous groups would be very valuable.

• 2

  Previous comments noted in the Oncotype DX summary apply here as well, including the presentation of data regarding the test in combination with standard predictors, the use of risk categories instead of a continuous risk measure, and the importance of confirming the stability of the test's calibration in different populations.

Assay Validation

In general, it is clear that validation studies need to deal with populations for whom the decision-making implications of various risk groupings are clear. The studies examined herein have established the proof-of-concept that tumor gene expression has prognostic value, but for all tests except Oncotype DX, both validation and development studies have been on mixed populations, without sufficient sample sizes to stratify into large enough homogeneous groups to guide clinical decisionmaking. In addition, validation samples are often re-used by other investigators; the pool of such samples in the public domain needs to be greatly expanded.

Potential for Scale Problems

One problem that may be faced in the future is that of the consequences of an increase in demand for these tests. Scaling up the production could represent a challenge for the reproducibility and reliability of the tests in any setting, especially if more than one laboratory will offer the assays, since procedures to warrant inter-lab reproducibility will be needed. Not only analytical aspects will need monitoring, but also procedures involving specimen evaluation prior to testing. With a larger number of tests, for instance, the ability to reliably perform the central pathologic review might become an issue, while in the case of MammaPrint the availability of the current reference RNA could potentially become a limiting factor.

Genetic Variability and Gene Expression

It is unknown whether gene expression profiles are more or less likely than more traditional biomarkers to be generalizable beyond the populations in which they were initially developed. Gene expression may reflect fundamental biological tumor features, and thus be relatively stable across ethnic groups. However, gene expression patterns have also been associated with specific genetic mutations (i.e., BRCA1), indicating that specific DNA mutations or polymorphisms^21,99 may affect the performance of a signature. This speaks to the importance of validating these tests in populations with varying genetic background. Biological and genetic evidence potentially addressing these issues is expected to become available in the form of single nucleotide polymorphism (SNP) arrays coupled to expression arrays.

The Need for Databases, Reproducibility, and Standards

MammaPrint® is the first assay based on microarrays that has completed the path from the bench to FDA approval for clinical application. For data storage, the MIAME standards³² represent the basis for the proper collection and storage of microarray data, and should be used to develop procedures going forward for the archiving of the tests performed in real patients, much as databases have been developed to facilitate outcomes research to complement clinical trials. Consideration should be given to the development of databases with complete data on each patient (absent identifiers), including all the analyses performed, laboratory logs, the raw and processed data, and all the information about procedures and analyses that have been performed to produce a risk estimate from a tumor sample. These apply equally to the other two assays, differing only in the type of data that would be stored.

Where Is the Field Going?

The current evidence for the feasibility of such gene expression based tests in clinical settings, along with the demand for better tools to manage patients, is leading to both an evolution of the available tests, and the addition of novel alternative tests. The number of publications is growing, and several alternative signatures not considered here have already been proposed for breast cancer as well as for other neoplasms. We can expect many new tests, as well as new uses for the assays that already exist. More genes might be added to the signatures, and in the particular case of MammaPrint this will be possible without changing the experimental procedures, since the array contains thousands more genes than the ones that are incorporated in the 70-gene signature. In this regard, we might also expect other modifications: subsets of the current signatures might be proposed as alternatives to current clinical risk factors, or be proposed in different populations or for different purposes. For Oncotype DX, a natural evolution could be related to its use as an alternative to immunohistochemistry and/or pathology to evaluate tumor Grade, S-phase index, ER, PR, and HER-2 expression, since such genes are part of the set included in the assay. Reporting of individual gene expression results may also prove useful. A great deal more work needs to be done on the prediction of therapeutic benefit, which is the ultimate goal of all such tests.

“Comparative Effectiveness” Studies

The emphasis in virtually all of the papers and in our evidence assessment is on the establishment of the value of each of these predictors over standard clinical predictors. However, as gene expression tests mature and proliferate, an important question will be how they compare to each other, and whether there is value in their combination. In the therapeutic domain, this has been called “comparative effectiveness” research. Such research has traditionally been difficult to fund by government or by industry, because it may not hold out as much therapeutic promise as new discoveries, and because industry understandably is not anxious to fund head-to-head comparisons with competitive products. This same dynamic could easily take hold in the risk prediction arena, with a proliferation of licensed prediction indices without any clear notion of what new ones are contributing over previous tests. Development of future expression-based predictors should make clear their incremental value over pre-existing methods. In the absence of better oversight of test development, physicians and patient are likely to be awash in new tests that all claim to offer similar guidance, or perhaps new guidance in previously neglected clinical subsets, with no way to sort out those claims.