NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Little J, Wilson B, Carter R, et al. Multigene Panels in Prostate Cancer Risk Assessment. Rockville (MD): Agency for Healthcare Research and Quality (US); 2012 Jul. (Evidence Reports/Technology Assessments, No. 209.)

  • This publication is provided for historical reference only and the information may be out of date.

This publication is provided for historical reference only and the information may be out of date.

Cover of Multigene Panels in Prostate Cancer Risk Assessment

Multigene Panels in Prostate Cancer Risk Assessment.

Show details


Topic Development

The McMaster University Evidence-based Practice Center (MU-EPC) engaged with representatives of Evaluation of Genomic Applications in Practice and Prevention (EGAPP) to seek clarification on the intended uses for the evidence report and for future recommendations. Subsequently, a Technical Expert Panel (TEP) was assembled, whose membership was nominated by the Evidence-based Practice Center and approved by the Agency for Healthcare Research and Quality (AHRQ). The TEP advised MU-EPC on aspects of the Key Questions (KQs), which were then revised to reflect the intent of the report from the perspective of AHRQ and EGAPP.

Analytic Framework

Figure 1 depicts the KQs within the context of the study selection criteria described in the following section. In general, the figure illustrates how the use of single nucleotide polymorphisms (SNP) test panels may result in different types of intermediate and final outcomes, including adverse events.

Figure 1 is a flow diagram depicting the use of multi-gene panels involving single nucleotide polymorphisms for prostate cancer risk assessment. The key questions are illustrated within the context of the PICOTS described in the previous section. The diagram reads temporally from left to right. On the left ,pre-analytic factors, analytic factors, and post-analytic factors are the scope of key question 1. Pre-analytic factors lead to asymptomatic or with prostate cancer. Analytic factors and asymptomatic or with prostate cancer lead to genetic test. Harms of testing is the subject of key question 3 and arises from genetic testing. Post-analytic factors and genetic test lead to predicted risk for outcome(s) which is the focus of key question 2. Predicted risk for outcomes lead to the outcomes of benefits, harms of testing and subsequent treatment decisions, and costs. These three outcomes are the topic of key question 3. Intermediate outcomes related to benefits include sensitivity, specificity, and screening rates. Final outcomes for benefits include mortality decrease and lifestyle changes. Intermediate outcomes for harms include missed tumours and delayed PSA test where as final outcomes consist of mortality increase and psychological stress.

Figure 1

Use of multigene panels involving SNPs for prostate cancer risk assessment.

Search Strategy

Studies were limited to those published in English, from the beginning of each database to October 2011. The following databases were searched: MEDLINE®, Cochrane CENTRAL, Cochrane Database of Systematic Reviews, and EMBASE. Strategies used combinations of controlled vocabulary (medical subject headings, keywords) and text words (see Appendix A).

Review was limited to commercially available SNP panels. The commercial availability of a test panel was defined as a clinical test offered (or soon to be offered) by a certified laboratory, or licensed or certified kit reagent test panels sold for use by clinical service laboratories within continental North America. To identify potential test panels for review, the following sources of information were used: PubMed, the Genetests Web site (now, grey literature, and letters to companies. Grey literature was identified through searching the Web sites of relevant specialty societies and organizations, Health Technology Assessment agencies (Hayes Inc. Health Technology Assessment), guideline collections, regulatory information (i.e., United States Federal Drug Agency, Health Canada, Authorized Medicines for European Community), clinical trial registries (i.e.,, Current Controlled Clinical Trials, Clinical Study Results, World Health Organization (WHO) Clinical Trials), grants and federally funded research (i.e., National Institute of Health (NIH), HSRPROJ), abstracts and conference proceedings (i.e., Conference Papers Index, Scopus), and the New York Academy of Medicine's Grey Literature Index. On behalf of the authors, the Scientific Resource Center directly contacted 40 companies known to provide either test services or diagnostic reagents potentially relevant to the key questions, in an effort to elicit unpublished sources of information.

Review of reference lists of included studies was undertaken. Any potentially relevant citations were cross-checked with our citation database. Any references not found were retrieved and screened at full text. Study authors were contacted to request details of relevant unpublished data.

Study Selection

Studies without a quantitative component were excluded (e.g., editorials, commentaries, notes, and qualitative studies). No restrictions were placed on study setting, minimum sample size, or duration of followup.


For all KQs, the eligible intervention was a commercially available (or soon to be available) test panel with at least two SNPs, at least one of which must have been validated in a genome-wide association (GWA) study. The criterion of having been validated in a GWA study was imposed because many associations with candidate genes have not been found to be replicated.154,155 We operationalized this criterion by checking the list of included SNPs against the list presented in Table 1, which was developed by reviewing the original articles indexed in the National Human Genome Research Institute GWA catalogue.184 Validation required observation of association in one or more independent data sets with a significance level of p<10-5. Studies of single gene tests, and/or panels which were not commercially available, were excluded. A test panel was defined by the list of SNPs (or other genetic sequence analytes) included in the assay. The included SNPs could be either informative (i.e., provide test results utilized in the interpretation of the result), or be controls used to assist in determining the accuracy and conclusiveness of the test result.

Table 3 summarizes the eligibility criteria by KQ.

Table 3. Eligibility criteria.

Table 3

Eligibility criteria.

Data Abstraction

Relevant fields of information were abstracted from individual studies by trained data abstractors using standardized forms and a reference guide. Prior to performing the data abstraction, a calibration exercise was conducted using a random sample of two included studies. Key study elements were reviewed by a second person (study investigator) with respect to outcomes, seminal population characteristics, and characteristics of the intervention. Disagreements were resolved by consensus.

Data were abstracted on study characteristics, SNP panels, metrics specific to each KQ, and other relevant data. Abstracted data included study characteristics (author and publication year, study objective, study design, setting, location, dates of data collection, and source of study funding) as well as details of the study participants (eligibility, sources and methods of selection, and number assessed for eligibility). Information was also abstracted about SNPs (number genotyped, type of laboratory, genotyping method and if done blind to participant status, call rate, concordance rate for duplicate samples, other quality control checks, Hardy Weinberg equilibrium information, rs (reference SNP) number and chromosomal region by model, method for handling SNPs in analysis, and other variables included in SNP panel). Analysis data was abstracted that included: method of constructing SNP panel, method for validating SNP panel, missing data, measures used to evaluate SNP panel (e.g., odds ratios (ORs) by risk score, area under the receiver operator characteristics curve (AUC), ΔAUC, maximum test accuracy, and cross-validation consistency). Data for results was abstracted as follows: number of participants included in analysis, mean age and standard deviation by group, ethnicity, first-degree family history of prostate cancer, prostate-specific antigen (PSA), Gleason score, pathologic stage (Tumor, Nodes, Metastases [TNM]), aggressive disease (definition and proportion of cases with aggressive disease), risk score, AUC, ΔAUC, other measure, subgroup analysis, results of validation if relevant (see Appendix B).

Assessment of Analytical Validity of Individual Studies

Information indicative of the rigor of assessment of analytical validity in individual studies was also abstracted and considered. Examples of sources of technical variation included:

  1. Pre-analytic phase: sample collection and handling, storage of sample, transport time, patient characteristics (age, race, ancestry, family health, etc.), patient preparation, other patient related attributes;
  2. Analytic phase: type of assay platform used and its reliability, specific analytes evaluated in the panel (specification of alleles, genes, or biochemical analytes), genotyping methods used, inclusion of relevant alleles), the type of software used to analyze and call SNPs (determination of positive or negative conclusion) of the test, and post-hoc review to ensure the result is correct (looking and reviewing the batch) was considered; and,
  3. Post-analytic phase: type of quality controls utilized, difficulty of interpretation, method of test interpretation and application, reporting protocols, post-test interpretation, contents of the report, and counseling information provided to the patient.

Assessment of Methodological Quality of Individual Studies

The methodological quality was interpreted to include primarily elements of risk of bias (systematic error) related to the design and conduct of the study.

Assessment of Studies Relating to Analytic Validity

As there were no studies that solely provided data on analytical validity, quality assessment was not performed.

Assessment of Studies Relating to Clinical Validity

We selected the Newcastle-Ottawa Scale (NOS)185 to assess risk of bias for observational studies (case-control and cohort). The study design elements evaluated with this tool include: selection of the study population, appropriate means for measuring exposures (case-control studies) and outcomes (cohort studies), and comparability of groups (controlling for confounding). We also selected some items from the QUADAS186 to evaluate the risk prediction aspect of the included studies.


Applicability was assessed by considering the key attributes of the population, intervention, comparator, and outcome in the context of a wider spectrum of patients in primary care settings that would likely benefit from these interventions in “real-world” conditions.

Rating the Body of Evidence

The overall strength of the body of the evidence was assessed using the AHRQ Strength of Evidence (SOE) approach.187 There are several factors that influenced the overall strength of the evidence:

  1. Study limitations (predominately risk of bias criteria);
  2. Type of study design (experimental versus observational);
  3. Consistency of results (degree to which study results for an outcome are similar; i.e. variability is easily explained, range of results is narrow);
  4. Directness of the evidence (assesses whether interventions can be linked directly to the health outcomes); and,
  5. Precision (degree of certainty surrounding an effect estimate for a specific outcome).

Publication Bias

Although the search strategy was comprehensive there is always the potential for publication bias. To help address publication bias, the Scientific Resource Centre (SRC) was asked to contact companies in an attempt to locate unpublished trials. No information was received from any of the companies.

Data Synthesis

A qualitative descriptive approach was used to summarize study characteristics and outcomes. Multiple publications for the same study were grouped together and treated as a single study, with the most current data reported for the presentation of summary results. Standardized summary tables explaining important study and target population characteristics, as well as study results, were created. Quantitative synthesis and subgroup analyses were not performed because of lack of comparability of the studies.

For KQ1, the analysis focused on assembling evidence that the SNP panels measured what they were intended to measure (i.e., their performance as assays). The metrics of primary interest were sensitivity, specificity, positive and negative predictive values, diagnostic OR, and the type of risk prediction (quantitative or qualitative) provided by the test, with the gold standard represented by some other form of genotyping. Because of the anticipated scarcity of relevant studies, we also scrutinized the reports for findings related to laboratory quality assurance (e.g., reliability (repeated sample testing), within and between laboratory precision, the time interval for testing, the proportion of specimens providing a conclusive result, failure rates for usable results, proportion of inconclusive results resolved, and more general evidence of external or internal quality control programs).

For KQ2, the focus of the analysis was on how well the SNP panels appeared to perform in classifying individuals in terms of the outcomes of interest (prostate cancer occurrence, detection, mortality, or stage/aggressiveness of cancer). The primary metrics were clinical sensitivity, specificity, positive and negative predictive value, positive and negative likelihood ratios, and AUC, and/or c-statistic.

For KQ3, the analysis assembled and evaluated the findings relating to the processes of care, health outcomes, harms, and economic aspects of using the SNP-based panels in practice. The range of relevant metrics was dependent on primary study design and the outcomes reported. For the economic analyses, direct and indirect cost estimates of the use of SNP-based panels were reviewed, and all cost-effectiveness and cost utility metrics were included.

Peer Review Process

Experts in the field were asked to act as peer reviewers for the draft report. They represented stakeholder groups including physicians, researchers and other professional representatives with knowledge of the topic. Additional peer reviewers included the Task Order Officer (TOO), associate editors, and members of the AHRQ internal editorial staff. The peer reviewer comments on the draft report were considered by the EPC in preparation of the final report. The responses to the peer reviewers were documented and will be published three months after the publication of the final evidence report.


  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (1.4M)

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...