• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of jcoHomeThis ArticleSearchSubmitASCO JCO Homepage
J Clin Oncol. Dec 1, 2012; 30(34): 4223–4232.
Published online Oct 15, 2012. doi:  10.1200/JCO.2012.42.6858
PMCID: PMC3504327

Publication of Tumor Marker Research Results: The Necessity for Complete and Transparent Reporting

Abstract

Clinical management decisions for patients with cancer are increasingly being guided by prognostic and predictive markers. Use of these markers should be based on a sufficiently comprehensive body of unbiased evidence to establish that benefits to patients outweigh harms and to justify expenditure of health care dollars. Careful assessments of the clinical utility of markers by using comparative effectiveness research methods are urgently needed to more rigorously summarize and evaluate the evidence, but multiple factors have made such assessments difficult. The literature on tumor markers is plagued by nonpublication bias, selective reporting, and incomplete reporting. Several measures to address these problems are discussed, including development of a tumor marker study registry, greater attention to assay analytic performance and specimen quality, use of more rigorous study designs and analysis plans to establish clinical utility, and adherence to higher standards for reporting tumor marker studies. More complete and transparent reporting by adhering to criteria such as BRISQ [Biospecimen Reporting for Improved Study Quality] criteria for reporting details about specimens and REMARK [Reporting Recommendations for Tumor Marker Prognostic Studies] criteria for reporting a multitude of aspects relating to study design, analysis, and results, is essential for reliable assessment of study quality, detection of potential biases, and proper interpretation of study findings. Adopting these measures will improve the quality of the body of evidence available for comparative effectiveness research and enhance the ability to establish the clinical utility of prognostic and predictive tumor markers.

INTRODUCTION

Predictive and prognostic tumor markers are playing an increasingly important role in personalized oncologic patient care.1 These markers range from conventional single-protein-, RNA-, or DNA-based markers to molecular signatures based on multiplex assays. As the number of available markers continues to increase and to result in substantial expenditure of health care dollars, there is a pressing need to perform critical reviews of the body of evidence that supports claims of the clinical utility of these markers. Prognostic and predictive markers are used to guide clinical management of patients with established cancer diagnoses. Pure prognostic markers distinguish the clinical outcomes of subgroups of patients (eg, those who are positive v negative for the marker) in the absence of a future, considered therapy, assuming the patients will receive either no treatment or some selected base treatment (eg, local therapies such as surgery and/or radiation). A strong prognostic factor may be able to identify patients with cancers that are so likely to be cured with the base treatment that additional therapy is not needed, even if it has activity. When further therapy is deemed necessary, predictive markers identify patient subpopulations that will or will not derive substantial benefit from promising new targeted therapies. For example, estrogen receptor and human epidermal growth factor receptor 2 (HER2) status in breast cancer predict benefit or resistance to endocrine and anti-HER2 therapies, respectively.2,3 More recent investigations have demonstrated that ALK translocations in lung cancer and the absence of KRAS mutations in colorectal cancers indicate benefit from crizotinib4 and anti–epidermal growth factor receptor antibodies,5,6 respectively.

Given their critical importance in making clinical decisions, prognostic and predictive tumor markers should be subject to the same evidence-based medicine standards as other types of medical interventions and practices. Evidence-based medicine relies on access to complete and accurate information to draw reliable conclusions. We review the current state of efforts to enhance the quality and transparency of reporting of tumor marker studies.

TUMOR MARKER RESEARCH AND TRANSLATION TO THE CLINIC: IMPORTANT SEMANTICS

A clear consensus on definitions of terms is essential to understanding how to translate tumor marker research to standard clinical practice. First, it is important to delineate the intended clinical use of the marker; for example, distinguishing between prognostic and predictive roles. Other uses include risk categorization in unaffected individuals, screening for occult malignancy, differential diagnosis, and monitoring. Moreover, for a tumor marker to be used in making clinical management decisions, issues related to analytic and clinical validity, clinical utility, study designs and analysis, and comparative effectiveness research must be fully understood. Improved reporting strategies are critical to achieving this necessary level of understanding.

Validity Versus Utility

The term “validation” is widely used, but it means different things in different contexts.7 Recently, the Evaluation of Genomic Applications in Practice and Prevention (EGAPP) working group, convened by the Centers for Disease Control and Prevention, has designated three important terms that describe necessary steps in developing genetic and other types of markers: analytic validity, clinical validity, and clinical utility.8 Analytic validity refers to analytic accuracy, reliability, and reproducibility related to the marker assay or test in hand. A report on the analytic evaluation of the Oncotype DX assay9 provides an example of the types of performance characteristics that should be assessed to establish analytic validity. The National Cancer Institute's Cancer Diagnosis Program has also proposed guidelines for factors to be considered in the evaluation of assay performance10. Clinical validity is the demonstration that the test has a suitably strong association with a clinical outcome of interest. The clinical validity of the proliferation marker Ki67 as a prognostic indicator in breast cancer has been demonstrated in many studies, but clinical utility has not yet been established because of lack of harmonization of assay methodology, uncertain reproducibility, and lack of consensus on specific situations in which the marker might reliably inform clinical decisions.11 We have added an additional term, biologic validity which, although not proposed by the EGAPP working group, implies that the marker might also be associated with a biologic feature or end point, perhaps but not necessarily associated with clinical outcomes. Gene expression profiling studies have identified biologically distinct subgroups of diffuse large B-cell lymphoma that include germinal center B-cell–like and activated B-cell–like diffuse large B-cell lymphoma.12 These subgroups have been identified consistently across multiple studies and have been shown to associate with survival, even with changes in the assay platform13 and in a new treatment era with the addition of rituximab to standard chemotherapy.14 Although biologic validity for these subgroups has been established, potential clinical utility remains under active investigation.15 Regardless, although both are important components of tumor marker development, neither clinical nor biologic validity alone implies that the marker should be used to direct patient care. Most importantly, clinical utility implies that use of the marker test to direct patient care has been shown to result in a favorable balance of benefits to harm, leading to improved outcomes compared with nonuse of the marker test. Improvement in outcome may relate to overall survival, disease-free survival, quality of life, or cost of care.7,16

Prospective Versus Prospective-Retrospective Tumor Marker Studies

Ideally, as with new therapeutics, the clinical utility of a predictive or prognostic tumor marker would be established with a high level of evidence generated in large, prospective trials. Several possible trial designs for prospectively testing tumor marker utility have been proposed.17,18 In many respects, establishing the clinical utility of a marker can be a greater challenge than establishing efficacy of a drug. Such trials are large, time consuming, and expensive, and only a few have been or are being conducted, such as TAILORx [Trial Assigning Individualized Options for Treatment (Rx)],19 MINDACT [Microarray in Node-Negative and 1-3 Node-Positive Disease May Avoid Chemotherapy],20 and RxPONDER [Rx for Positive Node, Endocrine Responsive Breast Cancer],21 to test multianalyte assays in breast cancer.

One advantage of tumor marker research compared with drug investigations is the ability to perform studies by using archived specimens linked to clinical annotation. However, this approach is a double-edged sword, since it also makes it easier to perform poorly designed and improperly controlled studies. If not properly planned, conducted, analyzed, and reported, such studies may demonstrate clinical/biologic validity, but they usually provide low levels of evidence to support clinical utility. Even worse, they may produce completely spurious false-positive or false-negative results. Recently, Simon et al22 have proposed a hierarchy of studies using archived specimens that produce varying levels of evidence. For studies using archived specimens, they propose that the best option is what they have termed a “prospective-retrospective” study. In this type of study, specimens that are collected, processed, and archived during the course of a prospective trial are analyzed retrospectively to test the clinical utility of a tumor marker. Such a trial can be a prospective registry of patients treated and observed uniformly to evaluate prognosis or a prospective, randomized treatment trial to address the predictive role of a tumor marker with the therapeutic strategy under investigation. These registries and trials collect clinical data prospectively by using trial-quality methodology. They also prospectively collect, process, and store tissue or other samples, which usually results in a higher percentage of available, high-quality specimens than in studies using specimen banks for which the specimens have been collected ad hoc (so-called “studies of convenience”).

Taken together, these conditions maximize the chances that tumor marker studies conducted by using prospectively collected, trial-quality specimens provide more accurate results and approximate high levels of evidence for clinical utility. Nonetheless, a critical requirement of a prospective-retrospective study is that the design and analytic approach proposed for the tumor marker investigation must be prospectively planned in writing. If planned, performed, and analyzed properly, prospective-retrospective studies can provide a high level of evidence for the utility of a marker (Level IB in the terminology of Simon et al22) under certain conditions (Table 1). The requirement for multiple validating studies is uncertain, even for therapeutic agents. For example, although the US Food and Drug Administration and guidelines bodies have on occasion accepted a single prospective, randomized controlled trial to introduce a new drug into clinical practice, meta-analyses of multiple clinical trials are preferable for determining true clinical utility.2326 Simon et al have proposed the requirement that a prospective-retrospective study be confirmed by at least one additional prospective-retrospective study (with similar results) to establish clinical utility of a marker, compared with the need to conduct only a single truly prospective clinical trial that directly addresses clinical utility of the marker. This requirement is necessary because often there is limited statistical power to evaluate a marker's clinical utility within a single trial originally designed to answer a treatment efficacy question, and there might be heterogeneity among the patient populations or marker assays used in different studies. For example, if a single, properly powered, prospective clinical trial is considered sufficient for recommending use of a marker in clinical practice, Simon et al suggest that at least two prospective-retrospective studies with consistent results, that use the same or a similar assay for the same marker, and that are conducted by using specimens from similar trials or trial arms, be required to demonstrate clinical utility of a marker.

Table 1.
Requirements for a Marker-Based Test to Reach Level IB Evidence of Clinical Utility Based on Prospective-Retrospective Studies

Comparative Effectiveness

Recently, the generalizability of results from prospective randomized clinical trials for standard clinical care has come into question.27 Prospective trials are generally conducted under idealized conditions, for example, with potentially restrictive eligibility criteria that would exclude patients who are at higher risk of complications or adverse outcomes because of comorbid conditions. The term “comparative effectiveness research” has been proposed to designate whether results generated in such a pristine setting are applicable to the real world of typical patients seen in everyday practice.28

There is some ambiguity in how the terms “comparative effectiveness research” and “evidence-based medicine” are used. We view evidence-based medicine as referring to the process of synthesizing available research evidence to draw conclusions about how to address specific clinical problems. We view comparative effectiveness research as encompassing broader considerations such as cost and feasibility,29 thereby placing a higher emphasis on real-world settings and recognizing the impact of health care decisions at both the individual and population levels. Regardless of whether one is doing comparative effectiveness research or practicing evidence-based medicine, the quality of the evidence is of paramount importance. We hold the view that without complete and transparent reporting of studies, one can neither assess the quality of the evidence nor properly interpret it. Therefore, the ambiguity of these terms does not affect our conclusions about the importance of good reporting.

Comparative effectiveness considerations apply to all aspects of clinical care, including both therapeutics and diagnostics. Indeed, for real-world comparative effectiveness, one could argue that the tumor marker arena offers a better opportunity than that for therapeutic agents, since many specimen banks and associated clinical data annotation used to study tumor markers are collected within standard-of-care situations. However, this same circumstance also produces considerable risk of bias. Such specimens are often collected, processed, and stored without regard to careful preanalytic quality assurance and quality control, and the clinical data are usually gathered retrospectively and without the careful scrutiny and auditing that is used for clinical trials. Thus, although such banks may provide useful analytic and clinical/biologic validity during tumor marker development, they are rarely useful for determining true clinical utility or comparative effectiveness.

THE PROBLEM: THERE ARE FEW TUMOR MARKERS WITH CLINICAL UTILITY

Regrettably, most tumor marker studies fail to rigorously address analytic validity or clinical utility. Rather, most publications simply demonstrate clinical validity with a marker assay that is often poorly described and not demonstrated to be accurate, reliable, and reproducible outside of the respective research laboratory that developed it. Few tumor marker tests have been studied with sufficient rigor to generate the kind of high-level evidence needed to determine whether they have clinical utility.22 Indeed, 40 years into the remarkable biologic observations from the revolution in molecular biology and a decade after initiation of the “-omics” era,29a only a few tumor markers are supported by evidence that would be considered Level 1. For example, the only tissue-based markers recommended for breast cancer by the Tumor Marker Guidelines Committee of the American Society of Clinical Oncology (ASCO) are estrogen and progesterone receptor testing for decisions about delivery of endocrine therapy, HER2 testing for anti-HER2 therapy, and the 21-gene recurrence score and urinary plasminogen activator and plasminogen activator inhibitor 1 assay to determine prognosis.30 The situation in other solid tumors is even more dismal. In colorectal cancer, testing for KRAS mutations is recommended to select patients for treatment with antibodies against epidermal growth factor receptor.5,6 In lung cancer, testing for ALK translocations, which signify likelihood of sensitivity to crizotinib,4 and testing for EGFR mutations, which are associated with benefit from tyrosine kinase inhibitor therapy, are also recommended.31 Although many other promising markers have been reported for prognosis and prediction of benefit from both targeted and routine therapies, few have advanced beyond the clinical validity phase.

OBSTACLES IN TRANSLATING TUMOR MARKER RESEARCH TO THE CLINIC

Several factors have impeded translation of findings from tumor marker research into clinically useful tests that meet the requirements of comparative effectiveness research for demonstrating clinical utility. These factors include an uncertain regulatory environment, inadequate reimbursement incentives, and a lack of structure for design and conduct of the studies necessary to achieve the high levels of evidence demanded by guidelines bodies, third-party payers, and patients and clinicians. These issues have been discussed in several other publications.16,22,3240 However, a little recognized but major problem is the disorganization, incompleteness, and lack of transparency that characterize the publication environment for tumor marker studies. If tumor markers are to achieve the kind of clinical utility associated with new therapeutics, and if they are to be scrutinized with the tools of comparative effectiveness research, it is essential that researchers, journal editors and reviewers, guidelines and technical assessment panels, and clinicians and their patients have a clear understanding of what has not been reported or published, and even within published papers, what factors may have led to biases that could affect the results or interpretation of the reported findings.

Reporting Biases

Biased reporting of studies is a major threat to the reliability of comparative effectiveness research to determine the clinical utility of tumor markers. There is ample evidence for medical study reporting bias and outcome reporting bias41 and strong evidence for publication bias in tumor marker studies in particular.4244 Therefore, it is important to recognize these types of biases and for authors, editors, and reviewers to insist that these biases be minimized as much as possible.

We distinguish between three main types of reporting bias. The first arises from submission and acceptance bias, which is commonly called “publication bias,” or more accurately, “nonpublication” bias.45 In this case, authors may decide not to report negative studies at all; if they do, those negative studies are much less likely to be published in highly regarded, high-impact journals. For example, in a meta-analysis of the tumor suppressor protein TP53 as a prognostic marker in head and neck cancer, substantial differences in the magnitude and statistical significance of observed prognostic effects were noted when comparing published studies to unpublished studies with retrievable data.46

The second type of reporting bias has been termed “within-publication selective reporting.”4749 This type of bias occurs when an author elects to selectively report results for only a subset of study outcomes that were actually analyzed. Just as in the case of nonpublication bias, this practice is particularly misleading if the decision to report the result for a particular outcome is influenced by the statistical significance of or consistency with prior studies or expectations. Within-publication selective reporting can take many forms: reporting results only for time-to-event end points for which statistically significant associations with the marker are observed rather than reporting results for the most clinically relevant end point, failure to report results of multivariable analyses adjusting for standard variables unless the marker maintains its statistical significance, and reporting results based on optimized cut points that are selected to minimize the P value for the test of association between the dichotomized marker value and outcome.33 In a meta-analysis of studies that examined proliferation markers in early-stage breast cancer,50 more than a dozen different cut points ranging from 0% to 30% were applied among 26 studies that used a Ki67 assay based on the MIB1 antibody alone, which raises questions about whether some of the studies might have reported results for optimized cut points. Unfortunately, authors often fail to provide a rationale for their selection of cut point, so this potential bias often goes undetected.

A third type of bias results from a practice designated as “incomplete study reporting.”47,48 In this case, the authors fail to provide sufficient detail regarding study design, conduct, and analysis within the papers accepted for publication. Reproducibility is one of the hallmarks of the scientific method, and complete and transparent reporting of investigational design and methodology is universally required for basic and clinical trial publications. However, for a variety of reasons, crucial factors in design and methodology of tumor marker studies, including preanalytic issues related to collection, processing, and storage of specimens, assay accuracy and variability, selection of patients for study, and critical details about their treatment and follow-up are often poorly documented, if at all. Henry and Hayes51 provide an example of how incomplete reporting of specific types of chemotherapies received by patients with breast cancer could lead to different conclusions about the direction of the association between HER2 status and efficacy of chemotherapy. Incomplete reporting that omits certain details might occur independently of any knowledge of results; nonetheless, missing details may lead to incorrect interpretation of study results which may lead to biased analyses and would prevent other researchers from reproducing the study findings.

STRATEGIES TO IMPROVE REPORTING OF TUMOR MARKER STUDIES

Biases due to nonpublication, selective reporting, and incomplete reporting of tumor marker studies are common and insidious.52 Collectively, these practices result in exaggerated claims of the significance of findings, complicate efforts to perform systematic reviews, confound analysis of clinical utility, and prevent estimates of comparative effectiveness.4244,46,53,54 However, several strategies for avoiding selective reporting have been proposed, and these should be widely understood and followed by authors, editors, and reviewers so that tumor marker research gains the same legitimacy as basic and clinical therapeutic research. We maintain that acceptance of these simple steps will lead to better and more personalized oncologic patient care.

Establishment of a Prospective Registry of Tumor Marker Studies to Decrease Nonpublication Bias

Federal legislation was passed in late 1977 to mandate a registry for both federally and privately funded clinical trials of experimental treatments so that information about treatment options under investigation in clinical trials would be widely accessible in a form understandable to the general public.55 In response to that mandate, the ClinicalTrials.gov searchable database was developed and launched online in February 2000. Each trial entered into the database is assigned a unique identifier, and essential information is captured, including a trial summary, eligibility criteria, interventions, study design, recruitment status, and sponsors. Usage of the system rose sharply beginning in 2004 when the International Committee of Medical Journal Editors (ICMJE) instituted a policy requiring that a clinical trial would have to be registered in ClinicalTrials.gov before the first participant was accrued for it to be considered for publication.56 Updated legislation resulting from the 2007 US Food and Drug Administration Amendments Act requires that investigators enter results for the primary and secondary trial end points when they become available. A recent cross-sectional analysis of trial publication after registration in ClinicalTrials.gov provided evidence that the trials database could help identify a substantial number of unpublished studies.57

A public registry specifically for tumor marker studies does not currently exist. Indeed, because archived specimens are often collected and stored for unspecified future use outside a specific written study protocol, it has been difficult to know what tumor marker studies are planned, in progress, or completed. In an effort to provide some transparency to this situation, an international panel of researchers has called for initiation of a comprehensive marker study registry in oncology, and a prototype web-based system is currently under development.58,59 At the very least, this registry will incorporate prospectively conducted tumor marker trials, such as MINDACT,20 TAILORx,19 and RxPONDER.21 A study summary and identifier, patient characteristics, a listing of all markers tested and assay technologies used, and a study contact are some of the data elements initially planned for inclusion in the database. Over time, it is hoped that prospective-retrospective studies would be listed in the registry, and perhaps eventually registration at time of study inception would become a requirement for publication of the study results. If implemented, such a registration system could substantially improve the ease of identification of unpublished marker studies.

Standards for Reporting Within Published Manuscripts to Decrease Biased and Inadequate Reporting

In addition to trial registries, reporting guidelines have been developed for several major types of health research studies to address the problems of incomplete and selective reporting within publications.49,60 In 1994 independent efforts led to two proposals for standards for reporting of randomized controlled trials.61,62 In 1996, representatives from both working groups reconciled these efforts to generate the now widely used Consolidated Standards of Reporting Trials (CONSORT) Statement.63,64 The CONSORT Statement comprises a flow diagram and checklist of items to report that describe the flow of patients through the study, primary and secondary end points, prespecified hypotheses, key aspects of the study design and methods, the prespecified statistical analysis plan, and results. The CONSORT Statement has subsequently undergone revision6569 and elaboration.70 Efforts to develop similar reporting guidelines for several other types of health research studies have followed the successful CONSORT model.49,60

Complete and transparent reporting for prognostic and predictive marker studies requires attention to the relevant elements of the CONSORT guidelines as well as other aspects of these studies, including issues surrounding specimens and marker assays. Two reporting guidelines highly relevant to prognostic and predictive tumor marker studies are the Biospecimen Reporting for Improved Study Quality (BRISQ)7173 guidelines and the Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK) guidelines.74

BRISQ

Unrecognized preanalytic issues can fundamentally alter the results obtained for a tumor marker. Thus, translating study results into reliable clinical care can be challenging, and comparative effectiveness research demands that preanalytic issues be considered and addressed appropriately. These factors can include the organ or tissue site from which a specimen was obtained (eg, tissue, blood, secretions), the type of specimen (eg, core biopsy, fine-needle biopsy, excision), and whether the specimen was stored frozen or fixed, and if so, the type of fixative, time and duration of fixation, how it was stored, duration of storage, and whether it was manipulated (eg, freezing and thawing). These elements are critical to the analytic validity of any tumor marker test, and one cannot assume that an assay developed under one set of preanalytic conditions will pertain to or remain valid in another. Thus, it is essential that these issues are considered and reported so that other investigators can reproduce exciting new exploratory data (the hallmark of scientific endeavor), and more important, so that the results can be translated into clinical utility with a high degree of confidence.

Recently, a group of experts has developed the BRISQ criteria—a check list that authors can submit either as part of their original manuscript or as supplementary material to be made available online.7173 BRISQ consists of three tiers of elements. Tier 1 items are necessary to report and include essential issues such as the organ(s) and/or tissues from which the biospecimens were derived, the manner in which the biospecimens were stabilized and preserved, the disease status (or for controls, the lack of it), and other critical preanalytic features that may fundamentally affect technical conduct of the assay in question (Table 2). Tier 2 items are advisable to report and include data elements that are slightly less crucial or less likely to be available in the biospecimens' annotation (eg, the demographics of the patient population and the method of enrichment for relevant components). Tier 3 includes additional factors that are not as likely to influence research results or are unlikely to be available (eg, environmental factors to which patients were exposed or the type of storage container in which the biospecimens were kept). Adhering to the BRISQ guidelines when reporting marker studies will allow for better assessment of any biases inherent in the types of specimens used in the study and will provide a more realistic sense of the feasibility of collecting the specimens required to successfully perform the marker assay.

Table 2.
BRISQ Tier 1 Reporting Items

REMARK

In 2005, a set of guidelines was proposed by another international committee of experts in the tumor marker field. At this writing, these REMARK guidelines have been published and endorsed by seven highly respected journals.7480 Indeed, the Journal of Clinical Oncology has written instructions for potential authors that “submissions [to the Journal] need to be REMARK compliant … JCO will assign higher priority to those biomarker papers that, in addition to satisfying REMARK criteria, have validated the prognostic or predictive value of the biomarker in an independent data set that was not used for initial biomarker identification and characterization. Without such independent biomarker validation, the reproducibility of the findings may be difficult to assess. This is especially important for those papers that suggest clinical utility for a given biomarker. In select circumstances, discovery of a novel biomarker not previously reported, especially if derived from a prospective analysis, might not always require independent validation if the research is hypothesis generating and the marker has convincing biologic relevance.”81

The REMARK guidelines are modeled after the CONSORT criteria but were developed to be specific to tumor marker studies, with the goal of increasing the transparency and completeness of information provided by those studies. They list 20 reporting elements under four main headings: Introduction (one), Materials and Methods (10), Results (seven), and Discussion (two)— that include relevant information concerning study design, prespecified hypotheses, patient and specimen characteristics, assay methods, statistical analysis methods, and recommendations for specific types of results that are useful to report (Table 3). Taken together, the REMARK criteria are designed to permit an accurate understanding of the analytic validity, clinical/biologic validities, and clinical utility, if any, of a tumor marker.

Table 3.
REMARK Checklist

Regarding analytic issues, the REMARK guidelines call for description of the types of specimens used and preanalytic variables affecting those specimens (which are covered in more detail in BRISQ) and precise descriptions of the laboratory methods used to perform the assays of interest. Information about the required specimen collection and processing procedures for reliable performance of the assay and difficulty and robustness (sensitivity, specificity) of the assay method is critical for determining the feasibly of using the marker in real-world clinical settings. To put the study in its proper clinical context, REMARK guidelines request patient clinical characteristics, pathologic diagnoses, and the treatments received (necessary to distinguish prognostic and predictive markers), and the clinical setting in which they are delivered.

The REMARK criteria and recommended study profile47,48,52 stress the importance of thorough and transparent reporting of statistical analysis methods to allow for assessment of appropriateness of approaches used and to facilitate efforts to reproduce the results. This emphasis on reporting statistical analysis methods is also motivated in part by the observational nature of many tumor marker studies and their frequent inclusions of multiple exploratory analyses that increase the chances of generating spurious findings. Therefore, REMARK criteria specify that reports should explicitly identify all the markers and end points that were initially examined in the study, state any prespecified hypotheses, and clearly describe all analysis methods used to produce the results.

REMARK criteria also emphasize the need for comprehensive reporting of analysis results, including reporting of estimated effects accompanied by measures of their uncertainty, relationships between the markers and standard clinical and pathologic variables, and estimated marker effects after adjustment for these standard variables. These factors permit assessment of the contribution of prognostic and predictive markers above and beyond information already available for use in clinical decision making, an important goal of comparative effectiveness research.

Although the original scope of the REMARK recommendations was primarily the reporting of studies that evaluated the prognostic value of a single marker, the guidelines are relevant to predictive studies, to studies investigating more than one marker (eg, multivariable classification functions or indices), and to studies evaluating prognostic or predictive factors other than those that are marker-based. Not only is the REMARK checklist82 useful for evaluating an individual tumor marker study report, it can be a helpful tool for collecting information from multiple studies for the purpose of a comparative effectiveness analysis to ultimately determine clinical utility.

The REMARK working group has recently published an elaboration of the REMARK guidelines,47,48 and readers are referred to that publication for a more comprehensive discussion of the REMARK reporting elements and specific examples of good reporting. The REMARK criteria elaborations more thoroughly describe the scope and level of detail recommended for each of the REMARK reporting items, explain the rationale behind the need to report each item, and encourage use of a study profile as a format for presenting the key information that fulfills the checklist reporting elements.

SUMMARY

Taken together, the use of a tumor marker study registry, and adherence to BRISQ and REMARK guidelines will result in more complete and transparent reporting of tumor marker studies, thus making it easier to assess study quality, inherent biases, and relevance to a given clinical setting. Table 460 summarizes the multiple challenges associated with conducting comparative effectiveness research for prognostic and predictive markers in oncology. Many are related to deficient reporting of tumor marker studies, and we believe that these challenges can be minimized with the described strategies. Institution of these measures is feasible but will require concerted effort and support from members of the research community, study funders, regulatory agencies, payers, and patients. Support will arise from recognition of the potential for prognostic and predictive markers to improve patient care and outcomes as well as from an understanding of the need for more informed overviews on which evidence-based conclusions about the real-world utility of prognostic and predictive markers can be based. We strongly urge editors of major journals that publish oncologic research to mandate that these criteria be fulfilled and transparently included in all manuscripts at the time of submission.

Table 4.
Strategies to Facilitate Comparative Effectiveness Studies of Prognostic and Predictive Markers in Oncology

Footnotes

Supported by a grant from Fashion Footwear Charitable Foundation of New York/QVC Presents Shoes on Sale (D.F.H.).

Authors' disclosures of potential conflicts of interest and author contributions are found at the end of this article.

D.F.H. has two patent applications pending regarding use of circulating tumor cells.

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

Although all authors completed the disclosure declaration, the following author(s) indicated a financial or other interest that is relevant to the subject matter under consideration in this article. Certain relationships marked with a “U” are those for which no compensation was received; those relationships marked with a “C” were compensated. For a detailed description of the disclosure categories, or for more information about ASCO's conflict of interest policy, please refer to the Author Disclosure Declaration and the Disclosures of Potential Conflicts of Interest section in Information for Contributors.

Employment or Leadership Position: None Consultant or Advisory Role: None Stock Ownership: Daniel F. Hayes, Oncimmune Honoraria: None Research Funding: Daniel F. Hayes, Pfizer, Novartis, Veridex, Janssen Pharmaceutica Expert Testimony: None Other Remuneration: None

AUTHOR CONTRIBUTIONS

Conception and design: All authors

Collection and assembly of data: All authors

Data analysis and interpretation: All authors

Manuscript writing: All authors

Final approval of manuscript: All authors

REFERENCES

1. Schilsky RL. Personalizing cancer care: American Society of Clinical Oncology presidential address 2009. J Clin Oncol. 2009;27:3725–3730. [PubMed]
2. Wolff AC, Hammond ME, Schwartz JN, et al. American Society of Clinical Oncology/College of American Pathologists guideline recommendations for human epidermal growth factor receptor 2 testing in breast cancer. J Clin Oncol. 2007;25:118–145. [PubMed]
3. Hammond ME, Hayes DF, Dowsett M, et al. American Society of Clinical Oncology/College of American Pathologists guideline recommendations for immunohistochemical testing of estrogen and progesterone receptors in breast cancer. J Clin Oncol. 2010;28:2784–2795. [PMC free article] [PubMed]
4. Shaw AT, Yeap BY, Solomon BJ, et al. Effect of crizotinib on overall survival in patients with advanced non-small-cell lung cancer harbouring ALK gene rearrangement: A retrospective analysis. Lancet Oncol. 2011;12:1004–1012. [PMC free article] [PubMed]
5. Allegra CJ, Jessup JM, Somerfield MR, et al. American Society of Clinical Oncology provisional clinical opinion: Testing for KRAS gene mutations in patients with metastatic colorectal carcinoma to predict response to anti-epidermal growth factor receptor monoclonal antibody therapy. J Clin Oncol. 2009;27:2091–2096. [PubMed]
6. Blanke CD, Goldberg RM, Grothey A, et al. KRAS and colorectal cancer: Ethical and pragmatic issues in effecting real-time change in oncology clinical trials and practice. Oncologist. 2011;16:1061–1068. [PMC free article] [PubMed]
7. Institute of Medicine: Washington, DC: The National Academies Press; 2010. Evaluation of Biomarkers and Surrogate Endpoints in Chronic Disease.
8. Teutsch SM, Bradley LA, Palomaki GE, et al. The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) Initiative: Methods of the EGAPP Working Group. Genet Med. 2009;11:3–14. [PMC free article] [PubMed]
9. Cronin M, Sangli C, Liu ML, et al. Analytical validation of the Oncotype DX genomic diagnostic test for recurrence prognosis and therapeutic response prediction in node-negative, estrogen receptor-positive breast cancer. Clin Chem. 2007;53:1084–1091. [PubMed]
10. National Cancer Institute: Performance Standards Reporting Requirements for Essential Assays in Clinical Trials. http://www.cancerdiagnosis.nci.nih.gov/scientificPrograms/pacct/assay_standards.htm.
11. Dowsett M, Nielsen TO, A'Hern R, et al. Assessment of Ki67 in breast cancer: Recommendations from the International Ki67 in Breast Cancer working group. J Natl Cancer Inst. 2011;103:1656–1664. [PMC free article] [PubMed]
12. Alizadeh AA, Eisen MB, Davis RE, et al. Distinct types of diffuse large B-cell lymphoma identified by gene expression profiling. Nature. 2000;403:503–511. [PubMed]
13. Rimsza LM, Leblanc ML, Unger JM, et al. Gene expression predicts overall survival in paraffin-embedded tissues of diffuse large B-cell lymphoma treated with R-CHOP. Blood. 2008;112:3425–3433. [PubMed]
14. Fu K, Weisenburger DD, Choi WW, et al. Addition of rituximab to standard chemotherapy improves the survival of both the germinal center B-cell-like and non-germinal center B-cell-like subtypes of diffuse large B-cell lymphoma. J Clin Oncol. 2008;26:4587–4594. [PubMed]
15. Dunleavy K, Pittaluga S, Czuczman MS, et al. Differential efficacy of bortezomib plus chemotherapy within molecular subtypes of diffuse large B-cell lymphoma. Blood. 2009;113:6069–6076. [PMC free article] [PubMed]
16. Hayes DF, Bast RC, Desch CE, et al. Tumor marker utility grading system: A framework to evaluate clinical utility of tumor markers. J Natl Cancer Inst. 1996;88:1456–1466. [PubMed]
17. Sargent DJ, Conley BA, Allegra C, et al. Clinical trial designs for predictive marker validation in cancer treatment trials. J Clin Oncol. 2005;23:2020–2027. [PubMed]
18. Freidlin B, McShane LM, Korn EL. Randomized clinical trials with biomarkers: Design issues. J Natl Cancer Inst. 2010;102:152–160. [PMC free article] [PubMed]
19. Sparano JA, Paik S. Development of the 21-gene assay and its application in clinical practice and clinical trials. J Clin Oncol. 2008;26:721–728. [PubMed]
20. Cardoso F, Van't Veer L, Rutgers E, et al. Clinical application of the 70-gene profile: The MINDACT trial. J Clin Oncol. 2008;26:729–735. [PubMed]
21. Hayes DF. Targeting adjuvant chemotherapy: A good idea that needs to be proven! J Clin Oncol. 2012;30:1264–1267. [PubMed]
22. Simon RM, Paik S, Hayes DF. Use of archived specimens in evaluation of prognostic and predictive biomarkers. J Natl Cancer Inst. 2009;101:1446–1452. [PMC free article] [PubMed]
23. Henderson IC, Berry DA, Demetri GD, et al. Improved outcomes from adding sequential Paclitaxel but not from escalating Doxorubicin dose in an adjuvant chemotherapy regimen for patients with node-positive primary breast cancer. J Clin Oncol. 2003;21:976–983. [PubMed]
24. Early Breast Cancer Trialists' Collaborative Group (EBCTCG), Peto R, Davies C, et al. Comparisons between different polychemotherapy regimens for early breast cancer: Meta-analyses of long-term outcome among 100,000 women in 123 randomized trials. Lancet. 2012;379:432–444. [PMC free article] [PubMed]
25. Lenzer J. FDA committee votes to withdraw bevacizumab for breast cancer. BMJ. 2011;343:d4244. [PubMed]
26. Montero AJ, Escobar M, Lopes G, et al. Bevacizumab in the treatment of metastatic breast cancer: Friend or foe? Curr Oncol Rep. 2012;14:1–11. [PMC free article] [PubMed]
27. Naik AD, Petersen LA. The neglected purpose of comparative-effectiveness research. N Engl J Med. 2009;360:1929–1931. [PMC free article] [PubMed]
28. Lyman GH, Hirsch B. Comparative effectiveness research and genomic personalized medicine. Personalized Medicine. 2010;7:223–227.
29. Congressional Budget Office: Washington, DC: Congressional Budget Office; 2007. Research on the comparative effectiveness of medical treatments: Issues and options for an expanded federal role.
29a. Institute of Medicine: Washington, DC: National Academies Press; 2012. Evolution of Translational Omics: Lessons Learned and the Path Forward.
30. Harris L, Fritsche H, Mennel R, et al. American Society of Clinical Oncology 2007 update of recommendations for the use of tumor markers in breast cancer. J Clin Oncol. 2007;25:5287–5312. [PubMed]
31. Keedy VL, Temin S, Somerfield MR, et al. American Society of Clinical Oncology provisional clinical opinion: Epidermal growth factor receptor (EGFR) mutation testing for patients with advanced non-small-cell lung cancer considering first-line EGFR tyrosine kinase inhibitor therapy. J Clin Oncol. 2011;29:2121–2127. [PubMed]
32. Simon R, Altman DG. Statistical aspects of prognostic factor studies in oncology. Br J Cancer. 1994;69:979–985. [PMC free article] [PubMed]
33. Altman DG. Suboptimal analysis using ‘optimal' cutpoints. Br J Cancer. 1998;78:556–557. [PMC free article] [PubMed]
34. Altman DG, Lyman GH. Methodological challenges in the evaluation of prognostic factors in breast cancer. Breast Cancer Res Treat. 1998;52:289–303. [PubMed]
35. Pepe MS, Etzioni R, Feng Z, et al. Phases of biomarker development for early detection of cancer. J Natl Cancer Inst. 2001;93:1054–1061. [PubMed]
36. Moskowitz CS, Pepe MS. Quantifying and comparing the accuracy of binary biomarkers when predicting a failure time outcome. Stat Med. 2004;23:1555–1570. [PubMed]
37. Song X, Pepe MS. Evaluating markers for selecting a patient's treatment. Biometrics. 2004;60:874–883. [PubMed]
38. Altman DG, Riley RD. Primer: An evidence-based approach to prognostic markers. Nat Clin Pract Oncol. 2005;2:466–472. [PubMed]
39. Simon R. Development and validation of therapeutically relevant multi-gene biomarker classifiers. J Natl Cancer Inst. 2005;97:866–867. [PubMed]
40. Simon R. Roadmap for developing and validating therapeutically relevant genomic classifiers. J Clin Oncol. 2005;23:7332–7341. [PubMed]
41. Dwan K, Altman DG, Arnaiz JA, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One. 2008;3:e3081. [PMC free article] [PubMed]
42. McShane LM, Altman DG, Sauerbrei W. Identification of clinically useful cancer prognostic factors: What are we missing? J Natl Cancer Inst. 2005;97:1023–1025. [PubMed]
43. Kyzas PA, Denaxa-Kyza D, Ioannidis JP. Quality of reporting of cancer prognostic marker studies: Association with reported prognostic effect. J Natl Cancer Inst. 2007;99:236–243. [PubMed]
44. Ioannidis JP, Khoury MJ. Improving validation practices in “omics” research. Science. 2011;334:1230–1232. [PubMed]
45. Song F, Parekh S, Hooper L, et al. Dissemination and publication of research findings: An updated review of related biases. Health Technol Assess. 2010;14:iii. ix-xi, 1-193. [PubMed]
46. Kyzas PA, Loizou KT, Ioannidis JP. Selective reporting biases in cancer prognostic factor studies. J Natl Cancer Inst. 2005;97:1043–1055. [PubMed]
47. Altman DG, McShane LM, Sauerbrei W, et al. Reporting recommendations for tumor marker prognostic studies (REMARK): Explanation and elaboration. BMC Med. 2012;10:51. [PMC free article] [PubMed]
48. Altman DG, McShane LM, Sauerbrei W, et al. Reporting recommendations for tumor marker prognostic studies (REMARK): Explanation and elaboration. PLoS Med. 2012;9:e1001216. [PMC free article] [PubMed]
49. Altman DG, Moher D, Schulz KF. Improving the reporting of randomized trials: The CONSORT statement and beyond. Stat Med. in press. [PubMed]
50. Stuart-Harris R, Caldas C, Pinder SE, et al. Proliferation markers and survival in early breast cancer: A systematic review and meta-analysis of 85 studies in 32,825 patients. Breast. 2008;17:323–334. [PubMed]
51. Henry NL, Hayes DF. Uses and abuses of tumor markers in the diagnosis, monitoring, and treatment of primary and metastatic breast cancer. Oncologist. 2006;11:541–552. [PubMed]
52. Mallett S, Timmer A, Sauerbrei W, et al. Reporting of prognostic studies of tumour markers: A review of published articles in relation to REMARK guidelines. Br J Cancer. 2010;102:173–180. [PMC free article] [PubMed]
53. Malats N, Bustos A, Nascimento CM, et al. P53 as a prognostic marker for bladder cancer: A meta-analysis and review. Lancet Oncol. 2005;6:678–686. [PubMed]
54. Kyzas PA, Denaxa-Kyza D, Ioannidis JP. Almost all articles on cancer prognostic markers report statistically significant results. Eur J Cancer. 2007;43:2559–2579. [PubMed]
55. McCray AT, Ide NC. Design and implementation of a national clinical trials registry. J Am Med Inform Assoc. 2000;7:313–323. [PMC free article] [PubMed]
56. De Angelis C, Drazen JM, Frizelle FA, et al. Clinical trial registration: A statement from the International Committee of Medical Journal Editors. N Engl J Med. 2004;351:1250–1251. [PubMed]
57. Ross JS, Mulvey GK, Hines EM, et al. Trial publication after registration in ClinicalTrials.gov: A cross-sectional analysis. PLoS Med. 2009;6:e1000144. [PMC free article] [PubMed]
58. Andre F, McShane LM, Michiels S, et al. Biomarker studies: A call for a comprehensive biomarker study registry. Nat Rev Clin Oncol. 2011;8:171–176. [PubMed]
59. Biomarkers Studies Registry: Home page. http://win.biomarkerregistry.org.
60. Equator Network: Resource centre for good reporting of health research studies. http://www.equator-network.org.
61. [No authors listed]: Call for comments on a proposal to improve reporting of clinical trials in the biomedical literature: Working Group on Recommendations for Reporting of Clinical Trials in the Biomedical Literature. Ann Intern Med. 1994;121:894–895. [PubMed]
62. [No authors listed]: A proposal for structured reporting of randomized controlled trials: The Standards of Reporting Trials Group. JAMA. 1994;272:1926–1931. [PubMed]
63. Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials: The CONSORT statement. JAMA. 1996;276:637–639. [PubMed]
64. Consolidated Standards of Reporting Trials (CONSORT): CONSORT Statement home page for transparent reporting of trials. http://www.consort-statement.org.
65. Moher D, Jones A, Lepage L. Use of the CONSORT statement and quality of reports of randomized trials: A comparative before-and-after evaluation. JAMA. 2001;285:1992–1995. [PubMed]
66. Moher D, Schulz KF, Altman D. The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA. 2001;285:1987–1991. [PubMed]
67. Moher D, Schulz KF, Altman DG. The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet. 2001;357:1191–1194. [PubMed]
68. Moher D, Schulz KF, Altman DG. The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials. Ann Intern Med. 2001;134:657–662. [PubMed]
69. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: Updated guidelines for reporting parallel group randomized trials. Ann Intern Med. 2010;152:726–732. [PubMed]
70. Altman DG, Schulz KF, Moher D, et al. The revised CONSORT statement for reporting randomized trials: Explanation and elaboration. Ann Intern Med. 2001;134:663–694. [PubMed]
71. Moore HM, Kelly AB, Jewell SD, et al. Biospecimen reporting for improved study quality (BRISQ) J Proteome Res. 2011;10:3429–3438. [PMC free article] [PubMed]
72. Moore HM, Kelly AB, Jewell SD, et al. Biospecimen reporting for improved study quality (BRISQ) Cancer Cytopathol. 2011;119:92–101. [PubMed]
73. Moore HM, Kelly A, Jewell SD, et al. Biospecimen reporting for improved study quality. Biopreserv Biobank. 2011;9:57–70. [PMC free article] [PubMed]
74. McShane LM, Altman DG, Sauerbrei W, et al. Reporting recommendations for tumor marker prognostic studies. J Clin Oncol. 2005;23:9067–9072. [PubMed]
75. McShane LM, Altman DG, Sauerbrei W, et al. Reporting recommendations for tumor marker prognostic studies (REMARK) J Natl Cancer Inst. 2005;97:1180–1184. [PubMed]
76. McShane LM, Altman DG, Sauerbrei W, et al. REporting recommendations for tumour MARKer prognostic studies (REMARK) Br J Cancer. 2005;93:387–391. [PMC free article] [PubMed]
77. McShane LM, Altman DG, Sauerbrei W, et al. REporting recommendations for tumor MARKer prognostic studies (REMARK) Nat Clin Pract Oncol. 2005;2:416–422. [PubMed]
78. McShane LM, Altman DG, Sauerbrei W, et al. REporting recommendations for tumour MARKer prognostic studies (REMARK) Eur J Cancer. 2005;41:1690–1696. [PubMed]
79. McShane LM, Altman DG, Sauerbrei W, et al. REporting recommendations for tumor MARKer prognostic studies (REMARK) Breast Cancer Res Treat. 2006;100:229–235. [PubMed]
80. McShane LM, Altman DG, Sauerbrei W, et al. Reporting recommendations for tumor marker prognostic studies (REMARK) Exp Oncol. 2006;28:99–105. [PubMed]
81. American Society of Clinical Oncology: Journal of Clinical Oncology Information for Contributors: Manuscript Submission Requirements—Biomarker Submissions (REMARK Guidelines) http://jco.ascopubs.org/site/ifc/requirements.xhtml.
82. Equator Network: Reporting recommendations for tumour marker prognostic studies (REMARK) http://www.equator-network.org/resource-centre/library-of-health-research-reporting/reporting-guidelines/remark.

Articles from Journal of Clinical Oncology are provided here courtesy of American Society of Clinical Oncology
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...