Chapter  145:  Genomic Tests for Ovarian Cancer Detection and Management

Literature on Key Questions

Question 1: What is the evidence that ovarian cancer genomic tests performed in a typical clinical laboratory actually measure what they are purported to measure? The published data on clinical laboratory performance suggests that currently available radioimmunoassays for single gene products have acceptable reproducibility and reliability, although even this level of variability may have some impact on clinical interpretation of results, especially when comparing relatively small serial changes, or levels close to the discriminatory threshold.

There is insufficient evidence to estimate how newer technologies such as microarrays or protein profiles would perform in a “typical clinical laboratory.”

Question 2: What is the sensitivity and specificity of genomic tests in detecting ovarian cancer in asymptomatic and symptomatic women, including high-risk women? In general, single gene products other than CA-125 have not been shown to be useful in the diagnosis of ovarian cancer, either in symptomatic or asymptomatic women; the sensitivity of CA-125 in screening populations is approximately 80 percent. Small sample sizes, lack of detail on the prediagnosis history of patients, and an unrealistically high prevalence of ovarian cancer in the majority of studies make it difficult to assess how any of these tests would perform in clinical practice.

Estimating the clinical value of more complex tests, using multiple gene and/or protein markers, is even more difficult. Studies of protein expression, in particular, are limited by lack of consensus on appropriate statistical methods, small sample sizes with substantially higher prevalences of ovarian cancer than would be found in the general population, spectrum bias, lack of reproducibility, and uncertainty about the specificity of the biological processes resulting in the observed protein patterns.

Question 3: What is the evidence that genomic testing to detect ovarian cancer in asymptomatic women, including high-risk women, changes clinical management and leads to improved clinical outcomes? We did not identify any evidence on the value of tests other than CA-125 to detect ovarian cancer in asymptomatic women. CA-125 has not been shown to improve ovarian cancer mortality or quality of life; in series of women with mutations of BRCA1 and BRCA2, screening with CA-125 and transvaginal ultrasound does not appear to prevent development of advanced stage ovarian cancer.

Question 4: What is the evidence that genomic testing in women with clinical suspicion of ovarian cancer or with already-diagnosed ovarian cancer changes clinical management and leads to improved health outcomes? Although there is a reasonable amount of data on the association between genomic tests, particularly CA-125, and the likelihood of different clinical outcomes, we did not identify any studies which provided evidence for changes in management leading to improved outcomes based on the results of the tests, other than for CA-125. Based on the results of another evidence report, CA-125 is helpful in distinguishing malignant from benign masses in postmenopausal women.

Question 5: What are the harms of using genomic tests for ovarian cancer prevention and management? The majority of the available literature focuses on BRCA1/2 testing and rarely describes results specifically for ovarian cancer. In the few studies that did, concerns about the risk of ovarian cancer were considerably less than for breast cancer; it is unclear whether testing for genetic markers of ovarian cancer susceptibility alone has different implications compared to testing for genes which affect both breast and ovarian cancer risk.

Conceptually, the harms of testing for genetic susceptibility for ovarian cancer should be no different than testing for genetic susceptibility of other cancers; the main issues are the effectiveness and potential risks of prevention strategies in those who are identified as high risk (primarily the risks of prophylactic oophorectomy), and issues related to reproduction. Similarly, the qualitative harms of the use of genomic tests for screening, diagnosis, and management - the psychological effect of a potential cancer diagnosis, the risks of diagnostic and therapeutic procedures including laparotomy, the harms of a false negative result leading to delayed or inappropriate management - are not conceptually different for genomic tests than for other types of tests, such as imaging; the main difference lies in the quantitative risks of these events, which in turn are determined by the sensitivity and specificity of the test and the pretest probability of disease.

Question 6: Has direct-to-consumer and direct-to-physician marketing of genomic tests for ovarian cancer increased the “appropriate” use of these tests? We identified two studies which compared utilization of BRCA1/2 tests for breast and ovarian cancer susceptibility before and after an advertising campaign; in both cases, utilization was compared in cities where the campaign was put in place to geographically distant cities where there was no formal campaign. The studies suggested increased utilization of testing, and one study found that the positive predictive value of testing declined after the campaign, but there was no way to judge whether the changes in testing were “appropriate.”

Modeling Results

The model is able to approximate reported age-specific incidence and mortality from ovarian cancer under both assumptions about natural history. At a given value for test sensitivity, screening was less effective in reducing mortality in a model assuming direct transition from Stage I to Stage III than one assuming that all cancers progress to Stage II prior to Stage III. However, screening frequency was much more important than test sensitivity; even at a test sensitivity of 99 percent, screening frequencies of less than 12 months are needed to reduce ovarian cancer mortality by more than 50 percent. At these high screening frequencies, positive predictive values are less than three percent, even for a test with specificity of 99 percent.

Limitations of the Report

The report did not include non-English publications. We did not formally attempt to estimate pooled sensitivity and specificity for tests because of heterogeneity of study design. Because many of the parameters in the natural history of ovarian cancer are unknown, any model will require assumptions and imputation of key parameters; calibrating a cohort model to cross-sectional data may result in errors in the imputation of these parameters because of unmeasured cohort effects in the cross-sectional data.

Limitations of the Literature

Common limitations of the literature included: failure to adequately describe relevant patient characteristics; small sample size with subsequent wide confidence intervals for estimates of sensitivity and specificity; unrealistically high prevalences of ovarian cancer; a spectrum of disease severity which does not reflect screening populations; lack of reproducibility for complex statistical algorithms; potentially inappropriate choices for cases and controls in initial developmental studies; and underlying assumptions about the natural history of ovarian cancer that may not reflect the actual biology of the disease.

Primary Prevention in the Entire Population

Primary prevention can be achieved either through medical or surgical treatment which preserves the ovaries but reduces the incidence of ovarian cancer, or by removal of the ovaries themselves.

Although oral contraceptives and tubal ligation have consistently been associated with reduction in ovarian cancer in epidemiological studies,⁵ the use of these measures as prophylaxis has never been prospectively tested in an adequately designed and powered trial; given the relative rarity of ovarian cancer, as well as the rarity of some of the serious side effects of oral contraceptives, such as an increased risk of deep vein thrombosis, such a trial may ultimately not be feasible.

Although primary peritoneal carcinomatosis, a condition which histologically and clinically is almost identical to ovarian cancer,⁶ can occur after removal of the ovaries, it appears to be rare in average-risk women.⁷ Bilateral oophorectomy in perimenopausal women undergoing hysterectomy for other causes has traditionally been recommended for prevention of ovarian cancer; however, this practice has also not been subjected to rigorous prospective study. A recent decision analysis suggests that, based on the available evidence, the potential harms from the other effects of oophorectomy may outweigh the benefits of ovarian cancer prevention.⁸

Primary Prevention in Women at Increased Risk for Developing Ovarian Cancer

This strategy depends on two things: the availability of a test for ascertainment of individuals at increased risk for developing ovarian cancer, and the availability of effective primary preventive treatment.

Although no randomized trials have been conducted, several observational studies suggest that women with an inherited predisposition to developing ovarian cancer who undergo prophylactic oophorectomy are at reduced risk of developing ovarian cancer compared to the expected incidence in this population.⁹–¹¹ Observational data also suggests that oral contraceptive use reduces ovarian cancer incidence in high-risk groups.^{12, 13}

Secondary Prevention through Screening

Unlike cervical cancer, where screening has proven remarkably effective, no screening test has proven effective in reducing ovarian cancer mortality. Physical examination using the bimanual pelvic examination,¹⁴ serum testing using the tumor marker cancer antigen 125 (CA-125), and imaging using vaginal ultrasound¹⁵ have all proven ineffective; the U.S. Preventive Services Task Force (USPSTF) gives a D recommendation to current methods for screening for ovarian cancer (at least fair evidence that the practice is ineffective or that harms exceed benefits). Additional studies are currently being conducted.

Secondary Prevention through Screening in Women at High Risk

As with primary prevention, this strategy is dependent on both effective screening methods and the ability to accurately determine who is at “high risk.” Screening, including more frequent screening, has not resulted in a reduced ovarian cancer incidence, or a substantial shift in stage distribution of detected cancers, in high-risk groups.¹⁶–¹⁹

Improved Therapy after Diagnosis of Ovarian Cancer

Identification of women who are particularly likely to respond to specific therapies, or identification of new targets for therapy, could lead to improved survival and quality of life in women with ovarian cancer. Although there is much ongoing research into possible targets for therapy, ovarian cancer therapy lags behind therapy for breast cancer, where identification of particular molecular targets appears to be effective.²⁰ This category could also include tests that help distinguish particular types of ovarian cancer from other types, and to distinguish primary ovarian cancer from cancer metastatic to the ovary from other sites, since misclassification could lead to relatively less effective therapy.

Tests Measuring Single Gene Products

These tests measure the concentration or presence/absence of proteins which are associated with the presence of ovarian cancer. The classic example of this type of test is CA-125, a protein for which several validated, commercially available assays are available. Levels of CA-125 are increased in patients with ovarian cancer compared to normal subjects, and the test is useful in discriminating benign from malignant masses in postmenopausal patients.¹⁴ Typically, these tests are for proteins detectable in serum, although, in some cases, tests may be peformed in fluid aspirated from an ovarian mass or the peritoneal cavity, or immunohistochemistry stains may be performed on ovarian or tumor tissue.

Tests for Variations in DNA

Tests for inherited or acquired mutations (e.g., breast cancer genes 1 and 2 [BRCA1/2]) in single genes can potentially identify patients at higher risk for developing cancer. Alternatively, mutations in some genes in the cancer itself may indicate greater or lesser likelihood of responding to a given therapy, or of developing side effects with a given therapy. In addition, changes in the overall pattern of the genome, such as loss of heterozygosity, are characteristic of many cancers, and potentially have a role in diagnosis.²¹ Finally, epigenetic changes (reversible changes to DNA and chromatin, such as the addition or subtraction of methyl groups), are currently under active investigation in a variety of cancers, including ovarian cancer.²²–²⁴

Gene Expression

Quantitative or semi-quantitive measurement of the expression (either higher or lower than normal) of particular genes in serum or tumor tissue has the potential for help in diagnosis (either as a screening tool or in discrminating particular subtypes of cancer), or potentially to aid in targeted therapy; for example, overexpression of human epidermal growth factor receptor 2 (HER-2) in breast cancer predicts responsiveness to therapy with an antibody against the receptor, trastuzimab.²⁰ Both single genes, and patterns of expression of multiple genes using technologies such as microarray, can be helpful. The introduction of high-throughput technology has facilitated the search for patterns of expression associated with specific outcomes, allowing simultaneous comparison of multiple genes in specimens from patients with and without the outcome. Studies of gene expression may also serve as the basis for identification of single gene products which can subsequently be evaluated as markers for screening, diagnosis, or management guidance.

Proteomics/Protein Characterization

Finally, quantification of protein patterns, typically in serum, can be performed using mass spectroscopy; one of the more common techniques is surfance-enhanced laser desorption inonization time-of-flight (SELDI-TOF).²⁵ As with multiple gene expression, protein patterns can be compared between patients with and without a given outcome of interest, or used to identify single markers.

Sources

The primary source of literature was MEDLINE^® (1966–May 2006). Searches of this database were supplemented by reviews of reference lists contained in all included articles and in relevant review articles and meta-analyses.

Search Strategies

The basic search strategy used the National Library of Medicine's Medical Subject Headings (MeSH) key word nomenclature developed for MEDLINE.^® Searches were limited to articles published in English. The exact search string used is given in Appendix A.^*The three searches yielded a total of 1,303 citations, whose records were maintained in a ProCite (Thompson ISI ResearchSoft, Berkeley, CA) database.

Abstract and Full-text Screening

Paired researchers from the Duke research team independently reviewed abstracts and classified each as “include” or “exclude” according to study-specific criteria, which they also developed. Abstracts were included if at least one of the paired reviewers recommended that it be included. A total of 552 abstracts were included for the further full-text review stage. Interrater reliability for include/exclude decisions at the abstract screening stage was tested by having seven pairs of readers review 813 abstracts. Agreement was good to excellent (kappa 0.36 to 0.75).

At the full-text review stage, the paired researchers independently reviewed a set of the articles and indicated a decision to “include” or “exclude” the article for the data-abstraction stage. When a pair of reviewers arrived at different opinions about whether to include an article, they were asked to reconcile the difference. Detailed inclusion and exclusion screening criteria were developed by research question and are described immediately below.

Screening Criteria

Abstracts were included for full-text review if they met the criteria described below, or if insufficient information was provided to judge whether they met the criteria. Articles were included for abstraction if full-text review showed that all criteria were met.

An article was included if it pertained to:

• (1)

  Epithelial ovarian cancer or primary peritoneal carcinomatosis; and

• (2)

  Genomics as defined by AHRQ for this project to mean any gene-based test used for predicting risk of developing disease, screening, diagnosis of disease, disease management, or prognosis only in strategies for the prevention of ovarian cancer morbidity and mortality. These included single gene products (e.g., cancer antigen 125 [CA-125]); genetic variations (e.g., breast cancer genes 1/2 [BRCA1/2] ); gene expression (e.g, human epidermal growth factor receptor 2 [HER2]/neu); and either single or multiple genes (e.g., microarrays) and protein expression (e.g., mass spectroscopy of multiple proteins in sera of patients with ovarian cancer compared with controls).

We included tests that:

• (1)

  Detect the presence of inherited mutations or gene polymorphisms which increase the risk of development of ovarian cancer;

• (2)

  Genes, RNA, or protein markers which are present or produced (or are present or produced in greater quantity) only in cells that have already undergone the transformation to cancer, and which can be used to detect asymptomatic or symptomatic cancers; and

• (3)

  Genes or proteins which may help predict the response to specific types of therapy, or themselves be targets of specific therapies.

We excluded the following:

• (1)

  Studies on BRCA1/2 screening and identification of risk covered in an earlier AHRQ evidence report;³¹

• (2)

  Studies on CA-125 screening and diagnosis covered in earlier AHRQ evidence reports;^{14, 30}

• (3)

  Studies involving only germ cell or stromal ovarian cancer, or non-ovarian primary;

• (4)

  Studies where patients are not the denominator;

• (5)

  Studies involving a cell line only;

• (6)

  Studies where reported data do not allow construction of a 2-by-2 table.

Summaries of the results of the abstract screening and full-text review are provided in Tables 3 and 4. A list of excluded articles, with reasons for exclusion, is provided in Appendix B.^*

Table 5 depicts the specific tests and clinical application of the tests covered by the included articles.

Test Characteristics and Confidence Intervals

For test characteristics, a Microsoft Excel® spreadsheet was developed that calculated appropriate test characteristics (sensitivity, specificity, negative predictive value, positive predictive value) for individual studies if studies provided enough data to input (a) values for individual cells of a 2-by-2 table, (b) the prevalence of disease and values for sensitivity and specificity, or (c) sufficient data to solve for two equations involving sensitivity, specificity, or predictive values. Ninety-five percent confidence intervals were automatically estimated using the approximate formula for proportions:

, where p = point estimate of proportion, N = total sample size.

Ovarian Cancer Model

Model description. We developed a Markov model to estimate the life expectancy for asymptomatic women who are considered candidates for potential prevention and screening strategies for ovarian cancer. The model tracks a hypothetical cohort of 40-year-old women over their lifetimes and compares the impact of one of six strategies for the prevention of ovarian cancer on cancer incidence, mortality, and overall life expectancy.

Simulation model. Women enter the Markov model (Figure 3), which follows the women's natural history of ovarian cancer; the probabilities of each event can be modified based on different strategies for primary prevention, screening, or targeted treatment (see below). Each month women are at risk for developing ovarian cancer. Over time, the cancer could progress through the different stages of ovarian cancer; we assumed that death from ovarian cancer was always preceded by diagnosis. Women with cancer could be detected either through a screening program or through clinical symptoms and diagnosis. Once detected, women undergo a laparotomy, and those who survive undergo treatment for their cancer. Cancer survival is based on the stage at diagnosis.

Historically, cancer progression has been modeled as a serial progression through clinical stages - Stage I is followed by Stage II, Stage II is followed by Stage III, and Stage IV follows Stage III. This conceptual model has worked well with cervical cancer, but it is not clear that using this overall “model” for ovarian cancer is appropriate for the following reasons:

• The main purpose of cancer staging is to identify groups of patients who have similar prognosis; this allows comparability in comparing treatment results in both prospective and retrospective studies. Although the concept that stages also represent biological progression is attractive, it is not necessarily true, and, at least in the case of the ovarian cancer staging system of the International Federation of Obstetrics and Gynecology (FIGO),³³ plays no role in the development and validation of a staging system.

• Cervical cancer is, in many ways, unique among human cancers: it has a single cause (persistent infection with certain types of human papilloma virus); exposure to this cause in most people occurs within a relatively narrow time frame (roughly ages 15 to 25, the times of highest sexual activity with multiple partners); and the most common type of cancer is a squamous type, which primarily spreads through direct extension. In contrast, the cause or causes of ovarian cancer are unclear, duration of exposure is unclear, and, most importantly, the pattern of spread and metastases is quite different.

• By definition, Stage I ovarian cancer is limited to the ovary, Stage II involves the ovary and other organs in the pelvis, and Stage III, the most common stage at diagnosis, involves organs in the upper abdomen, including the large and small bowel, the omentum, the diaphragm, and other peritoneal surfaces. Peritoneal fluid constantly circulates, and it is not uncommon for loops of small bowel to come in contact with the ovary. In order for the “conceptual model” requiring an intervening Stage II prior to development of Stage III to be correct, one has to assume that cancer cells on the surface of the ovary must necessarily spread to the uterus or other pelvic organs before they can spread to areas in the upper abdomen via transport in peritoneal fluid or via direct contact with small bowel. We postulate that a scenario where a certain unknown proportion of ovarian cancers progress directly from Stage I to Stage III is at least as plausible a scenario.

Given this uncertainty about the clinical progression of ovarian cancer, we therefore modeled the progression under two alternative assumptions: (1) that ovarian cancer needs to progress from Stage I to Stage II before progressing to Stage III; and (2) that a proportion of ovarian cancer progresses directly from Stage I to Stage III. We evaluated how these two competing assumptions about the natural history of ovarian cancer affect the required stage progression and mortality rates and the estimated life expectancies of the alternative prevention strategies.

We assumed that women who have survived their detected cancer for 5 years are to be considered disease-free and to have mortality equal to that of the general population. Each month, a woman may also choose to have a benign oophorectomy, reducing her risk of ovarian cancer. Throughout their lifetimes, all women are at risk for age-specific mortality unrelated to ovarian cancer. We also included age-specific rates for bilateral oophorectomy, under the assumption that women without ovaries are not at risk for developing ovarian cancer; we did not specifically model the possibility of primary peritoneal carcinomatosis in these women.

Arrows in Figure 3 depict possible transitions between states. Note that one version of the model allows transition directly from Stage I (confined to the ovaries) to Stage III (metastases to the upper abdomen).

Data sources. We obtained age-specific estimates of ovarian cancer incidence, mortality, stage distribution, and survival from the National Cancer Institute's Surveillance, Epidemiology, and End Results (SEER) online database Cancer Query System (http://seer.cancer.gov/canques).

Estimates for other-cause mortality were obtained by subtracting age-specific ovarian cancer mortality from age-specific all-cause mortality for women, using U.S. lifetables available from the National Center for Health Statistics (www.cdc.gov/nchs/deaths.htm).

Estimates for age-specific oophorectomy rates were obtained from AHRQ's Nationwide Inpatient Sample, using ICD-9 codes for bilateral oophorectomy, bilateral salpingoophorectomy, or removal of remaining ovary or remaining tube and ovary (http://hcup.ahrq.gov/HCUPnet.asp).

Software. We constructed the model and performed all analyses using DATA Pro 2006 (Williamstown, MA: TreeAge Software, Inc).

Prevention strategies. We modeled six clinical strategies of prevention for ovarian cancer (Figure 4):

• (1)

  The baseline strategy of no screening or prevention (NoScreen) where women are identified with ovarian cancer only through development of clinical symptoms.

• (2)

  A primary prevention strategy (PrimaryPrevention) where women undertake a hypothetical method of primary prevention which reduces their incidence of ovarian cancer.

• (3)

  An interval screening strategy (IntervalScreen) where women are screened at recurrent intervals for ovarian cancer using a hypothetical test. Women identified through screening could benefit from early treatment.

• (4)

  A genetic screening strategy where women are tested for a specific genetic mutation and if positive undergo primary prevention for ovarian cancer (Genetic&PrimaryPrevention). The overall population risk for ovarian cancer is unchanged; we varied incidence in those with and without the putative mutation.

• (5)

  A genetic screening strategy where women are tested for a specific genetic mutation and if positive they undergo screening for ovarian cancer at recurrent intervals (Genetic&IntervalScreen).

• (6)

  A strategy where women once identified with ovarian cancer are tested for a hypothetical marker which allows targeted treatment for ovarian cancer (TargetTx). Women who are positive for the marker and undergo the targeted treatment experience greater survival.

Approach. Because the majority of the literature on genomic testing does not allow definitive conclusions about the relative effectiveness of different strategies using different tests, we adapted a “generic” approach to comparison of different strategies.

We chose as a goal a 20 percent reduction in ovarian cancer death, similar to the reductions targed for other cancers in the Healthy People 2010 objectives. With this target, we used the calibrated models to explore the following clinical questions:

• (1)

  How effective would a primary prevention intervention need to be to reduce ovarian cancer deaths by 20 percent?

• (2)

  What combinations of test sensitivity and frequency result in at least a 20 percent reduction in mortality?

• (3)

  What combinations of (a) prevalence of a genetic mutation in the population and (b) relative risk associated with that mutation would result in the target 20 percent reduction in ovarian cancer deaths with either primary prevention (at various levels of effectiveness) or interval screening (at varying levels of sensitivity and frequency)?

• (4)

  How effective would a targeted treatment for ovarian cancer need to be (and in what proportion of the patient population would the marker for that treatment need to exist)? Note that we assume that targeted therapy would be equally effective across all stages of disease.

• (5)

  How do the test characteristics for targeted treatment or genetic screening affect the results?

• (6)

  How do the above results differ under the assumption that cancer must progress from Stage I to II and then III versus that assumption that ovarian cancer may progress directly from Stage I to Stage III?

• (7)

  What effect does the assumption about natural history have on the relative efficacy of screening?

• (8)

  What is the impact of attributable risk proportion on the potential efficacy of genetic risk factors?

Approach

We sought to identify articles that provided details on the performance of genomic tests in a laboratory setting, with an emphasis on laboratories providing results for clinical care. Because data on sensitivity and specificity are covered under Question 2, our emphasis in this question was on evidence related to analytic performance, such as:

• Test reproducibility, as measured by inter- and intra-assay coefficients of variation for quantitative tests, or measurements of observer variability for tests that require human observation (such as immunohistochemistry).

• Measurements of correlation with other tests, including previous generations of other tests.

• Quantification of variability between laboratories.

• Analytic sensitivity and specificity in comparison to a recognized reference standard.

We included only articles that specifically addressed the laboratory performance of genomic tests for ovarian cancer. Although specific assays may have documented analytic validity when used for other cancers, or other conditions, our focus was on ovarian cancer.

Results

Articles included for Question 1 are summarized in Evidence Table 1 (Appendix D ^*).

Radioimmunoassays for single gene products - cancer antigen 125 (CA-125). We identified six articles that compared the performance of a next-generation radioimmunoassay (RIA) for CA-125 (CA-125 II) from various manufacturers to earlier generation tests or to other RIAs for other tumor markers.³⁴–³⁹ All six studies reported high correlation coefficients with previous assays. All studies reported low inter- and intra-assay coefficients of variation (values generally less than 10 percent for inter-assay, less than 5 percent for intra-assay). Of note, two studies examined coefficients of variation at different levels of CA-125 and found changing variability with CA-125 levels. Fillela et al.,³⁸ using an automated analyzer, found coefficients of variation of 2.8 to 6.4 percent for “level 2” values of CA-125 (mean 47.1 U/mL), with values of 1.8 to 4 percent for “level 3” (mean 164.3 U/mL). Hubl et al.³⁴ reported slightly higher intra-assay coefficients of variation in mid-range (40 U/mL) compared to low-range values (10 to 20 U/mL). A third study³⁹ did not find an effect of concentration in the clinically relevant range. Because 35 U/mL is the most commonly used threshold for considering a CA-125 value suspicious for cancer, these results suggest that random variation in test results may have some impact on sensitivity and specificity at values close to the threshold. The clinical impact of this variability would ultimately depend on how values close to the threshold are managed.

Tuxen and colleagues performed serial measurements of CA-125 over the course of a year in 26 women with known ovarian cancer⁴⁰ and 31 healthy controls⁴¹ to assess the relative effect of analytic variability and inter- and intra-individual biologic variation on CA-125 levels. In women with cancer, analytic imprecision accounted for 12 percent of the variability in levels, intra-individual variations 24.0 percent, and inter-individual variations 43.6 percent; after accounting for this imprecision, the investigators estimated that a change of greater than 62.6 percent in the reference value would be needed in order to be statistically significant. Similar values were found in healthy controls, with imprecision being greater in premenopausal women (69.5 percent) compared with postmenopausal women (35.7 percent) due to variability in levels over the course of the menstrual cycle. The change in reference value required for significance after accounting for variation in the entire group was 50 percent.

One study³⁷ compared sensitivity and specificity of the new generation and first generation assays using 138 stored samples and found slightly higher sensitivity with the new assay (89.8 vs. 84.7 percent), and lower specificity (83.5 vs. 84.7 percent). However, the prevalence of cancer in the samples was much higher than would be expected in a typical clinical population, and the confidence intervals for the sensitivity and specificity estimates overlapped.

Radioimmunoassays for other single gene products. Few studies of other markers were performed in clinical laboratories. Hasholzner and colleagues³⁶ evaluated clinical laboratory performance of an RIA for cancer antigen 72-4 (CA-72-4); intra-assay coefficients of variation were 3.5 to 4 percent, and inter-assay coefficients of variation were 5 to 7.4 percent. Correlation between CA-125 levels and CA-72-4 levels was good for healthy controls (-0.066), but poor for serous ovarian cancer patients (0.576).

Tuxen and colleagues⁴¹ measured carcinoembryonic antigen (CEA) and tissue plasminogen activator (TPA) along with CA-125 in healthy controls in the study described above. For CEA, the change in reference values needed for significance after accounting for imprecision was 44.8 percent; for TPA, the value was 67.9 percent. Unlike with CA-125, menopausal status did not affect the degree of intra-individual variability.

Two studies reported research laboratory performance of two other single gene products and preliminary clinical validation. Riisbro and colleagues⁴² reported an inter-assay coefficient of variation of 7.6 percent, and an intra-assay coefficient of variation of 4.6 percent for an RIA for soluble urokinase plasminogen activator receptor. Using 129 stored serum samples, levels were correlated with malignancy and stage of disease, but not after adjusting for other variables. Thougaard and colleagues⁴³ compared three different antibodies targeted against tetranectin, with similar performance in terms of assay variability, differences in absolute levels of 10 percent or less, and similar correlations in ovarian cancer patients (n = 43); levels were observed to decrease as cancer stage worsened.

Two studies^{44, 45} reported on the performance of RIAs developed after identification of candidate single gene products identified after using microarrays to identify overexpressed genes. These studies will be discussed below under “Microarrays.”

Other assays. Sapi and colleagues⁴⁶ reported a method for removing peripheral lymphocytes from blood samples or ascitic fluid in order to measure telomerase activity. After this method, telomerase activity was observed in 8 of 8 patients with Stage IV ovarian cancer, 7 of 20 patients with Stage III, and 0 of 30 controls. CA-125 levels were higher in patients with positive telomerase assays.

Single gene mutation/polymorphism. Janatova and colleagues⁴⁷ evaluated the performance of Spreadex Polymer NAB (electrophoresis gels) in patients with known breast cancer gene 1/2 (BRCA1/2) mutations (n = 13) and 13 controls; the technique successfully identified mutations only in those subjects with known mutations; all patients with known mutations had mutations detected.

Wen and colleagues⁴⁸ compared microarray with gel-based DNA sequencing for identifying mutations in the p53 gene in 108 patients with ovarian cancer. Mutations were detected by both methods in 57 cancers, and no mutations by both methods, for a concordance of 81 percent.

Microarrays. Only one study specifically examined test performance in a clinical laboratory. Zarrinkar et al.⁴⁹ compared high-throughput microarray using parallel analysis to single sample assays in specimens from 31 patients with known ovarian cancer and found a high level of correlation (0.980).

Two studies reported preliminary data from research laboratories on candidate single-gene products identified initially through microarray studies.^{44, 45} Hellstrom and colleagues⁴⁵ compared the performance of an antibody to human epididymis protein 4 (HE4) to CA-125 in 121 subjects, of whom 37 (30.6 percent) had ovarian cancer. Reported sensitivities for HE4 were better for HE4 then for CA-125 at a fixed specificity of 96 percent, but confidence intervals were quite wide. Mok and colleagues⁴⁴ examined the performance of prostatin in 201 subjects, 64 (31.8 percent) of whom had ovarian cancer. Prostatin levels correlated poorly with CA-125 levels; sensitivity of prostatin was less than that of CA-125 at the same specificity of 94 percent, but the combination of the two markers had a sensitivity of 92 percent at the same level of specificity.

Proteomics. Although we identified 10 studies that looked at protein expression in serum as a potential biomarker for ovarian cancer,⁵⁰–⁵⁹ all were performed in research laboratories. Because several of these studies have attracted wide attention in the media, we will discuss them in more detail here.

Petricoin et al.,⁵⁷ created a proteomics-based genetic algorithm with cluster analysis to distinguish between ovarian cancer and non-ovarian cancer serum samples using a training set of 50 ovarian cancers and 50 healthy controls from a high-risk population. The new algorithm was then tested using a validation set consisting of 50 ovarian cancers and 66 non-cancers, some with benign ovarian cysts, benign gynecologic disease, or benign non-gynecologic disease. The algorithm successfully classified 50/50 cancers (sensitivity = 100 percent) and 63/66 non-cancers (specificity = 95 percent) in the validation set. The study has two major limitations. First, the proteins used to distinguish cancers from non-cancers were not identified, leading to questions of whether proteins of interest were actually produced by tumor cells or by other inflammatory responses in the tumor's microenvironment. Although the reported positive predictive value is 94 percent in the study, the low prevalence of ovarian cancer (1 in 2,500) in the general population would reduce the positive predictive value of proteomic screening to less than one percent in a screening population.

These investigators subsequently published three datasets online as the Clinical Proteomics Program Databank (http://home.ccr.cancer.gov/ncifdaproteomics/ppatterns.asp). The first dataset (2-16-02) consists of 100 control, 100 ovarian cancer, and 16 benign disease samples run on a Ciphergen H4 ProteinChip array. Ovarian Dataset 4-3-02 consists of the same samples run on Ciphergen WCX2 ProteinChip array. Ovarian Dataset 8-7-02 contains serum profiles run on Ciphergen WCX2 ProteinChip array of 162 ovarian cancer patients subdivided into stages and 91 non-cancer control subjects.

Sorace et al.⁵⁵ analyzed Ovarian Dataset 8-7-02 using a training set containing 45 controls and 80 cancers. A 2-sided Wilcoxon test was used to compare intensity between controls and cancers at different mass-to-charge (M/Z) values. A subset of M/Z values that resulted in the lowest Wilcoxon p-values was selected, and stepwise discriminant analysis was used to determine the subset of M/Z values that best discriminated cancers from controls. Classification rules were then used on the remainder of the patient data (test set). Three classification rules were developed, all with sensitivity > 90 percent and specificity > 90 percent when applied to the test set. The authors expressed concerns over the existence of highest discriminatory ability in the M/Z < 500 range, where data are traditionally discarded due to increased “noise.” They hypothesized several explanations for these findings including very low molecular weight (MW) biomarkers such as LPA, low MW degradation products of higher MW macromolecules, and systematic processing error.

Li et al.⁵⁹ analyzed all three Clinical Proteomics Program Datasets using two different approaches: support vector machine statistical testing (SVM-ST) and support vector machine with genetic algorithm (SVM-GA). Datasets were not split into training and validation sets; instead, a leave-out-one cross validation was used. Sensitivity and specificity for analysis of Dataset 2-16-02 were lower than in the analysis by Petricoin et al.⁵⁷ of the same data (0.79 and 0.80 for SVM-ST, and 0.96 and 0.948 for SVM-GA, respectively). Sensitivity and specificity were improved with analysis of the other two datasets, achieving 100 percent sensitivity and 100 percent specificity using SVM-GA to analyze Dataset 8-7-02. The authors were unable to reproduce the sensitivity and specificity reported by Petricoin et al. when training an SVM with the discriminatory features identified in the latters' paper.

Zhang et al.⁵³ performed a multicenter study to analyze serum proteomic expression profiles using Ciphergen ProteinChip in 153 patients with epithelial ovarian cancers, 42 with other ovarian cancers, 166 with benign pelvic masses, and 142 healthy controls. Results were cross-validated against different subsets of the data to identify biomarkers. Three biomarkers (apolipoprotein A1, transthyretin, both down-regulated in ovarian cancers, and a fragment of human inter-alpha trypsin inhibitor, upregulated) were identified and immunoassays performed on serum from another subset of patients. Levels of these three biomarkers were included in a multivariate model to predict malignancy, and the model was tested on a validation set consisting of 138 ovarian cancers and 63 healthy controls. The resulting model had a sensitivity and specificity of 0.775 and 0.968, respectively, in distinguishing cancer from healthy controls. The authors also created a model incorporating CA-125 with the three markers, which improved on the specificity of CA-125 alone. The discovered biomarkers were all acute phase reactants deemed unlikely to be released by tumor cells. Controls were not age-matched and were significantly younger (median test and validation sets 39 and 44) compared to cancer patients (median 52 and 57).

Kozak et al.⁶⁰ analyzed serum from 109 ovarian cancers, 19 patients with benign disease, and 56 healthy donors using the Ciphergen ProteinChip SAX2. Samples were divided into training and test sets. Proteins differentially expressed were identified using t-test and Wilcoxon rank sum tests. Three biomarker protein panels were then developed: SBP (five markers), VBP I (five markers), and VBP II (four markers). Multivariate logistic regression was used to develop panels with the best predictive value. Sensitivity and specificity were 0.957 and 0.826, respectively, for SBP; 0.815 and 0.949 for VBP I; and 0.728 and 0.949 for VBP II. Test sets were employed. Panels correctly identified early stage disease with variable sensitivity. Individual discriminatory proteins were not identified.

Although all studies reported good discrimination for the particular protein profile studied, there were several recurrent issues that limit the ability to draw inferences about potential clinical applicability:

• Technical issues with the assay. For example, Conrads and colleagues⁵⁸ noted that “comparisons...revealed that the variation in mass spectra (overall amplitude, total record count and deviation between ovarian cancer cases and control samples) was statistically indistinguishable from the variance within the process itself, as indicated by the serum reference standard.” Sorace and Zahn,⁵⁵ in an analysis of a dataset used by several other groups, found sensitivity and specificity of 100 percent in a training set, but noted that much of the discrimination of the profile lies in the region of the spectroscopy results with low mass-to-charge ratios. They note that this region is problematic both because of technical issues of measurement and because differences in protein profiles in this region may result from processes independent of cancer.

• Varying analytic methods. No consistent methodology was used. Given the complexity of the data and the variety of methods used, it is difficult to draw consistent conclusions about performance. Li and colleagues⁵⁹ found marked variability in results using similar statistical methods on different datasets, as well as using different statistical methods on the same dataset.

• Unrealistically high prevalence of ovarian cancer. The majority of the studies compared serum samples from known ovarian cancer patients to healthy controls, using relatively small datasets of 100 to 200 subjects, with a prevalence of cancer of 30 to 50 percent. Although repeated sampling and resampling was performed in all of these studies, the prevalence of cancer was still substantially higher than it would be in a screening population (approximately 0.05 percent). Only one study⁵⁹ provided estimates for the positive predictive value within a screening population; these estimates were in general at least an order of magnitude lower than the results based on the original dataset.

Discussion

The majority of the literature we identified that specifically addressed issues of clinical laboratory performance in ovarian cancer dealt with radioimmunoassays of single gene products, with CA-125 being the most common product. Test reproducibility and validity is in general quite good for these assays, although a series of Danish studies by Tuxen and colleagues suggests that both inherent laboratory variation and biological variation should be considered when considering thresholds for determining clinically relevant changes in concentrations of these markers. In addition, coefficients of variation for CA-125 are generally greatest when levels are in the range of the most commonly used discriminatory threshold of 35 U/mL, suggesting that this irreducible imprecision may have some impact on sensitivity and specificity in practice.

We did not identify any relevant literature on the clinical laboratory performance of other types of genomic tests. Although there were numerous articles describing research laboratory performance, the relevance of these studies to widespread clinical practice is uncertain. In particular, the prevalence of ovarian cancer in studies of potential proteomic patterns as predictors of early stage ovarian cancer is at least an order of magnitude higher than the likely prevalence in the general population.

Summary

The published data on clinical laboratory performance suggests that currently available radioimmunoassays for single gene products have acceptable reproducibility and reliability, although even this level of variability may have some impact on clinical interpretation of results, especially when comparing relatively small serial changes, or levels close to the discriminatory threshold.

There is insufficient evidence to estimate how newer technologies such as microarrays or protein profiles would perform in a “typical clinical laboratory.”

Approach

We sought to identify articles that provided details on the sensitivity and specificity of genomic tests in a clinical setting. We separately reviewed studies intended for screening purposes, both in the general population and in women identified as high risk based on family history and/or BRCA testing, and studies used for diagnostic purposes, either in women with symptoms or women with a diagnosed mass.

Other Evidence Reports

Asymptomatic women - average risk. The systematic review conducted for the U.S. Preventive Services Task Force (USPSTF)³⁰ concluded that annual CA-125 screening had an estimated sensitivity of 80 percent, with false positive rates of 0.1 to 0.6 percent, based on three studies with small numbers of cancers and variable, relatively short, followup durations; the estimated positive predictive value for screening was 1 percent for women called for additional testing, and 15 percent for women undergoing surgery.

The evidence report on management of adnexal masses¹⁴ found that the majority of studies did not report results separately for women with asymptomatic masses compared with those who had masses detected because of symptoms. Of note, the report also found an extremely low sensitivity (less than 50 percent) of the bimanual pelvic examination as both a screening test and an initial diagnostic test.

Asymptomatic women - high risk. The BRCA1 and BRCA2 systematic review for the USPSTF³¹ did not specifically address the sensitivity and specificity of genomic tests in this setting.

Symptomatic women - average risk. Again, the literature on the use of diagnostic tests, including genomic tests, does not provide useful information on differences in test performance in symptomatic versus asymptomatic women. The evidence report on management of adnexal masses¹⁴ found an approximate sensitivity of 78 percent for CA-125 in the diagnosis of cancer in adnexal masses, with an approximate specificity of 78 percent; both sensitivity and specificity were higher in postmenopausal women. Other genomic tests (all single gene products) reviewed included TAG-92, cancer antigen 19.9 (CA-19.9), and CEA; all had sensitivities lower than the pooled estimates for CA-125. There were few studies examining combination testing; those that did failed to find improved discrimination compared to CA-125 alone.

Symptomatic women - high risk. The adnexal mass evidence report¹⁴ did not identify any studies uniquely in high-risk populations.

Results

Articles included for Question 2 are summarized in Evidence Table 2 (Appendix D ^*).

Asymptomatic women. We did not identify any studies of genomic tests other than CA-125 that provided evidence of sensitivity and specificity as primary screening tests for ovarian cancer in asymptomatic women. The one major study published subsequent to the USPSTF review reported the initial baseline results of the National Cancer Institute Prostate, Lung, Colon, and Ovarian (PLCO) screening trial.⁶¹ In this study, over 28,000 women aged 55 or older were screened with transvaginal ultrasound and CA-125; 402 women (1.4 percent) had an abnormal CA-125. Of the 19 invasive cancers, four had normal CA-125 levels for a sensitivity of 78.9 percent (95 percent confidence interval [CI], 60.6 to 97.3 percent), and a specificity of 98.7 percent (95 percent CI, 98.5 to 98.8 percent), consistent with previous studies in postmenopausal women.

Only one study⁶² provided sufficient detail about patient characteristics to be able to ascertain test performance of a genomic test (in this case, vascular endothelial growth factor [VEGF]) as a diagnostic tool in asymptomatic women identified with a pelvic mass through screening; sensitivity was 55.9 percent, and specificity 55.3 percent, too low to be considered useful as a second line diagnostic test. All of the other studies that included women with a pelvic mass failed to report the proportion of women with a mass who had presented on the basis of symptoms, or on the basis of asymptomatic detection of a mass through a pelvic examination or imaging study; this limitation is shared by the majority of the literature on diagnosis of ovarian cancer in women with masses.¹⁴

Symptomatic women - single gene products. The majority of studies identified were retrospective studies that compared serum or, in some cases, tissue from women with known ovarian cancer to serum from women with benign adnexal masses and/or asymptomatic women. There were more than two studies identified for only two markers, CA-72-4 and VEGF.

Table 6 shows results for CA-72-4. In general, sensitivity is poor. Although specificity is better (85 percent or higher), the high positive predictive values observed in these studies are a reflection of the high prevalence of cancer in the study populations. In a screening setting, where prevalence is likely to be less than one percent, positive predictive values would be much lower.

Table 7 shows results for VEGF. Although sensitivity was in general somewhat higher than for CA-72-4, specificity was somewhat lower.

Table 8 shows results for other single gene products. In general, there is a trade-off between sensitivity and specificity. Common limitations of these studies included failure to adequately characterize the study population (such as underlying risk factors, menopausal status, and how the patients presented to the health system); small numbers (as reflected in the wide confidence intervals of the sensitivity and specificity estimates); and prevalence of ovarian cancer much higher than would be expected in many clinical settings, especially screening settings. These limitations preclude meaningful synthesis or direct comparisons between tests.

Symptomatic women - DNA variations. We did not identify any studies that allowed estimation of sensitivity and specificity of inherited or acquired mutations in detecting ovarian cancer.

Symptomatic women - gene expression. We identified one study that reported on the sensitivity of cytological tests of ascitic fluid for the presence of a series of genes believed to be activated in ovarian cancer;⁹⁸ although specificities were universally high, sensitivities ranged from 8 to 60 percent, with wide confidence intervals for both values. These low sensitivities were even more striking given the high prevalence of ovarian cancer in the samples (61 percent).

We identified one study that used immunohistochemistry for a range of gene products in the diagnosis of ovarian cancer in ovarian tissue;⁹⁹ performance for different markers ranged widely. Because there were only 20 ovarian cancer patients (out of a total of 70), confidence intervals were wide. In addition, no data were provided on the reproducibility of the assay or interpretation of results.

Another study measured cytokeratin 19 (CK19) expression in peripheral blood mononuclear cells;¹⁰⁰ false-positive rates were quite high in both discriminating cancer from benign ovarian tumors (specificity 28.6 percent) and cancer from normal controls (specificity 40.0 percent). Again, confidence intervals were very wide.

No other studies directly reported the sensitivity and specificity of gene expression patterns identified through the use of microarray technology.

Symptomatic women - proteomics. Studies of protein profiles as a potential tool for early diagnosis of ovarian cancer have attracted considerable attention recently. However, all of the identified studies examined test performance using databases; none have been tested in a clinical population.

Discussion

In general, single gene products other than CA-125 have not been shown to be useful in the diagnosis of ovarian cancer, either in symptomatic or asymptomatic women. Small sample sizes, lack of detail on the prediagnosis history of patients, and an unrealistically high prevalence of ovarian cancer in the majority of studies make it difficult to assess how any of these tests would perform in clinical practice.

Estimating the clinical value of more complex tests (those using multiple gene and/or protein markers) is even more difficult. Studies of protein expression, in particular, are limited by lack of consensus on appropriate statistical methods, small sample sizes with substantially higher prevalences of ovarian cancer than would be found in the general population, lack of reproducibility, and uncertainty about the specificity of the biological processes resulting in the observed protein patterns. Most importantly, none have been tested in clinical populations.

Approach

We searched for articles related to the use of genomic tests in screening for ovarian cancer in asymptomatic women, including any studies that focused on screening in women previously identified as being at greater risk on the basis of family history or other genomic tests. We excluded studies of CA-125 that were previously reviewed in a report for the USPSTF;³⁰ we did, however, review studies published subsequent to the USPSTF review.

Other Evidence Reports

In the review of ovarian cancer screening for the USPSTF,³⁰ studies did consistently show a greater prevalence of Stage I ovarian cancer among women screened with CA-125 (based on small numbers of cancers), but there were no data on the impact of screening on mortality.

In the BRCA1 and BRCA2 review,³¹ there were “limited” data on the efficacy of intensive screening among carriers, and no prospective studies of chemoprevention (especially oral contraceptives) or tubal ligation, although some suggestion from observational studies that both of those interventions might reduce ovarian cancer risk. In three retrospective studies and one cohort study, prophylactic oophorectomy reduced the risk of ovarian cancer by 85 to 100 percent, although the authors of the review noted that the confidence interval for risk reduction crossed 1.0 in the prospective study.

Results

We did not identify any studies of the use of genomic tests for screening asymptomatic women in any risk group that met our inclusion criteria.

Discussion

To date, no test has been shown to have acceptable sensitivity and specificity for screening for ovarian cancer, or to reduce the morbidity and mortality associated with ovarian cancer. Because definitive diagnosis of ovarian cancer requires surgery, a high level of specificity is needed in order to minimize the costs and potential complications of unnecessary surgery; although screening in high-risk groups could theoretically have better outcomes (because a higher pretest probability of cancer should result in better positive predictive values), this has not been demonstrated in adequately designed studies. One study of 1,610 women at increased risk because of family history found a positive predictive value for ovarian cancer of less than five percent (3 of 61 abnormal tests),¹⁰¹ a value similar to that observed in the 28,000 average-risk women in the PLCO study (3.7 percent).⁶¹ The ability to detect early cancer in these populations, even with intensive screening, may be limited; a study of 291 high-risk women who were screened every 6 months with ultrasound and CA-125 detected early stage ovarian cancer in only one of the eight women who developed ovarian or peritoneal cancer during 10 years of followup; five of the eight had had normal screening tests within 6 months of diagnosis.¹⁸ Other studies in similar populations have reported similar findings.^{16, 19} The degree to which the lack of effectiveness of screening is due to insufficient test sensitivity rather than the inherent biology of ovarian cancer is discussed further in the section on modeling.

Summary

We found no articles on the use of genomic tests (other than CA-125) for detecting ovarian cancer in asymptomatic women, regardless of risk group.

Approach

Studies included in this section reported data on the association of genomic test results with either a change in clinical management or a health outcome related to a particular management strategy. For example, genomic tests whose results were associated with response to therapy are included here. We did not, however, include studies that related genomic tests strictly to prognosis, for example, describing survival differences based on genomic test results. Similarly, if genomic test results were associated with staging data, we did not include these studies here despite the fact that staging may in turn be used to select treatment. Our notion is that we were primarily interested in how genomic testing could inform clinical management beyond usual clinical staging, which is already routinely used to guide therapy.

Results

We found no studies that compare two groups of women, one of which underwent genomic testing and one of which did not. Ideally, such a study would be prospective with random allocation to the groups. In fact, we did not encounter any non-randomized comparative studies, either prospective or retrospective, that compare management or health outcomes in two such groups. Therefore, the following review considers only uncontrolled studies describing the association of test-positive and test-negative women with management and health outcomes. This design limits the certainty with which one might infer that applying the test in clinical practice could result in improved management decisions or health outcomes compared to not applying the test.

The description of evidence is divided first between studies of women with a clinical suspicion of ovarian cancer versus women with already diagnosed ovarian cancer. Within these groups, we will discuss the influence of genomic tests on specific management decisions or health outcomes. Summaries of all the included articles are provided in Evidence Table 3 (Appendix D ^*).

Women with clinical suspicion of ovarian cancer. We found no studies that describe evidence regarding change in management or health outcome resulting from use of genomic testing in women with a clinical suspicion of ovarian cancer. A large number of studies discussed under Question 2 describe the diagnostic accuracy of genomic tests in women with a clinical suspicion of ovarian cancer (based on symptoms), but none of these studies, nor any others screened in the literature search, described clinical management changes or health outcomes resulting from these tests.

Women with already diagnosed ovarian cancer. The most studied use of genomic tests was for predicting or detecting response to treatment after debulking therapy and adjuvant chemotherapy. The studies were of two types. First, several studies sought to predict which patients would have a favorable response to chemotherapy (e.g., complete or partial response vs. stable or progressive disease). A second goal of studies was to predict, among women who appeared to have no evidence of disease on clinical evaluation, who would have evidence of disease on second-look laparotomy (SLL). Finally, several studies related genomic test results at the time of primary debulking surgery with the ability to achieve optimal cytoreduction.

Predicting favorable response to chemotherapy. We found six studies describing the association between CA-125 and favorable response to chemotherapy;^{79, 102}–¹⁰⁶ these studies used a wide range of threshold values, from 10 to 500 U/mL. In addition, the following tests were described in one study each: human kallikrein 6 (hK6);⁷⁹ low-density lipoprotein receptor-related protein (LRP), multidrug resistance protein (MRP), and P-glycoprotein (Pgp);¹⁰⁷ multidrug resistance gene 1 (MDR-1), MRP-1, and MRP-2;¹⁰⁸ TP53;¹⁰⁹ c-erb-B2;¹¹⁰ and human kallikrein 10 (hK10).¹¹¹

Table 9 and Figure 5 show the sensitivity and specificity of these tests for predicting response to chemotherapy. Estimates of sensitivity are widely scattered and range from 12 to 100 percent; specificity ranged from 0 to 84 percent. The only study to report diagnostic performance that exceeded that of the studies of change in CA-125^{102, 104}–¹⁰⁶ was one of MDR-1;¹⁰⁸ however, it is important to note that this relatively small study (n = 27) did not report estimates from the other markers assessed in the study (MRP-1 and MRP-2), but noted only that there was no association between those tests and response to chemotherapy.¹⁰⁸ No other studies have replicated this finding.

Predicting residual disease on SLL. A second goal of studies was to predict, among women who appeared to have no evidence of disease on clinical evaluation, who would have evidence of disease on SLL. A test might be clinically useful if it could predict with a high sensitivity which patients with clinically undetectable disease might have cancer progression on SLL. Such a test might obviate the need for SLL or at least improve the accuracy of clinical staging. CA-125 is one marker used to detect early recurrence.

We found five studies describing the association between CA-125 and positive disease on SLL.¹¹²–¹¹⁶ Two studies described cancer-associated serum antigen (CASA).^{113, 114} In addition, the following markers were described in one study each: Cathespin-D and nm23;¹¹⁷ p53, murine double minute protein (Mdm2), and Bcl-2;¹¹⁸ interleukin 6 (IL-6);⁸¹ cytokeratin fragment 21-1 (CYFRA 21-1);¹¹⁹ tetranectin (TN);¹¹³ and TPS.¹¹⁵

Three reports classified patients with microscopic disease as disease-negative,^{81, 117, 118} while the remaining studies classified macroscopic or microscopic disease at SLL as disease-positive.

Table 10 shows the sensitivity of a positive test for identifying patients with positive SLL. The sensitivities for CA-125 range from 6.4 to 57.9 percent. Thresholds for a positive test vary from 10 U/mL to 35 U/mL, but there is no clear relationship between cut-point and sensitivity that explains the differences between studies. While most studies of CA-125 fell in the lower left quadrant of the receiver operating characteristic (ROC) space (Figure 6), representing low sensitivity but high specificity, the other genomic tests reported displayed greater variability, particularly in specificity. The higher average sensitivity was associated with lower specificity.

Studies also differ with regard to the separation in time between the time at which the marker was measured and the time of SLL. The immunohistochemical tests are based on surgical samples, while serum markers were measured after surgery, after adjuvant chemotherapy, and immediately prior to SLL.

Predicting ability to perform optimal cytoreduction. Several studies evaluated genomic tests for their ability to predict whether optimal cytoreduction (by surgical debulking) was possible. Definitions for optimal cytoreduction were identical between studies, based on the Gynecology Oncology Group criteria,¹²¹ requiring no residual tumor masses > 1 cm at debulking surgery.

Table 11 and Figure 7 show the sensitivity and specificity of tests for predicting which patients will achieve optimal cytoreduction at primary surgical debulking.

Discussion

The vast majority of the available literature on the use of genomic tests in the management of patients with ovarian cancer consists of serum measurement of single-gene products, particularly CA-125, as predictors of (a) initial response to chemotherapy; (b) complete resolution of disease (i.e., negative SLL); or (c) the ability to perform optimal cytoreduction. There is also a substantial literature that reports on the association of various genomic tests with prognosis, but the majority of these studies were excluded because they did not describe patient management.

The studies that sought to predict initial response to chemotherapy were generally performed in unselected women with ovarian cancer, not just those with optimal debulking, for example. Although, in theory, there is significant benefit, both clinically and in the research setting, of being able to predict who will respond to chemotherapy, we did not identify any studies that demonstrated this; there were no studies, for example, that compared different chemotherapeutic regimens based on test results.

Studies evaluating the association of genomic test results with second-look surgery were commonly limited to women who had appeared to have a complete response to initial debulking and chemotherapy. In these studies serum markers were often measured close to the time of SLL (e.g., at the end of chemotherapy or immediately prior to SLL); however, immunohistochemical tests were usually based on tissue obtained at the time of primary surgery. SLL is sometimes used to evaluate women who appear clinically to have had a complete response, since other techniques, such as CA-125 and imaging, are fairly insensitive for very small disease. However, most of the data suggests that there is no substantial survival benefit to SLL, even if residual tumor is removed; a Gynecologic Oncology Group non-randomized study reported a difference in median survival of only 1 month.¹²⁸ Thus, the SLL might be more properly though of as a test for monitoring disease or the outcome of treatment (a reference standard) rather than as a therapeutic option itself; the potential benefit of better sensitivity at detecting residual disease would be the ability to avoid the need for SLL altogether (and its concomitant cost and morbidity).

Finally, the prediction of optimal debulking is potentially helpful; for example, patients might benefit by referral to particularly expert surgeons, or to research protocols. Patients unlikely to obtain optimal debulking could be selected for neoadjuvant chemotherapy to improve the likelihood of debulking success; however, this strategy has not been tested.¹²⁹ As with tests for response to chemotherapy, there are, as yet, no studies prospectively demonstrating improved patient outcomes from such a strategy.

Summary

Although there is a reasonable amount of data on the association between genomic tests, particularly CA-125, and the likelihood of different clinical outcomes, we did not identify any studies that provided evidence for changes in management leading to improved outcomes based on the results of the tests.

Approach

The nature of the potential harms associated with genomic testing in ovarian cancer varies depending on the potential application of the test:

Testing for increased risk of ovarian cancer. Potential harms associated with testing for inherited or acquired genetic changes that are associated with increased risk of ovarian cancer include:

• The harms associated with the management of women who have positive results. These include complications of primary preventive therapy (for example, surgical complications from prophylactic oophorectomy) and sequelae of the therapy (loss of fertility, premature menopause). For strategies involving more frequent screening, the impact on time and any discomforts associated with the screening test need to be considered.

• The effects on quality of life and other psychological measures of a diagnosis that provides knowledge of an increased risk of disease, with little direct evidence for the benefit of management strategies.

• The potential impact on decisions about childbearing for inherited mutation.

Screening for early ovarian cancer, diagnosis of ovarian cancer. The main potential harm associated with the use of genomic tests for screening for ovarian cancer or for the diagnosis of ovarian cancer is the risk of a false-positive test result, with potential for anxiety about the diagnosis, as well as the risks of definitive diagnostic surgery.¹⁴

Testing for targets for specific therapy. The main potential harm associated with testing for specific targets is a false-positive result, which would lead to inappropriately exposing a patient to the risks of the targeted treatment, provision of ineffective treatment, and delayed start of potentially more effective therapy.

We searched for studies that described these classes of adverse outcomes for any type of genomic testing, excluding studies covered in previous reviews of the potential harms of screening using CA-125 and BRCA1/2 conducted for the USPSTF,^{14, 30, 31} but including articles published after the inclusion dates for these reviews.

Other Evidence Reports

The review of ovarian cancer screening for the USPSF did not identify any specific articles describing harms of screening, but pointed out the low positive predictive value of available tests and the large number of unnecessary surgeries.³⁰ The adnexal mass evidence report was unable to draw any conclusions about the potential harms of false-positive surgeries because of limitations in the literature, primarily a failure to distinguish the preoperative indication for surgery from the postoperative findings.¹⁴

The review of BRCA1/2 testing identified relatively few studies addressing the harms of testing.³¹ Only one study reported complications of prophylactic oophorectomy in carriers (4 of 80 women). Quality-of-life studies which involved only prophylactic oophorectomy (as opposed to prophylactic mastectomy with or without oophorectomy) were inconclusive.³¹

Results

We did not identify any articles that specifically described the harms associated with genomic testing for ovarian cancer. In the PLCO study, 62 of 402 women with an elevated CA-125 underwent surgical biopsy; of these, 16 had any neoplasm, with 13 (3.7 percent) having an invasive cancer.⁶¹ The paper did not report whether there were any complications from these surgeries.

We identified four studies of BRCA1/2 testing published since the USPSTF review (see Evidence Table 4, Appendix D ^*).

McInerney-Leo and colleagues published two papers from the same large study. In the first paper,¹³⁰ they reported data on a variety of psychological measures, including depression, self-esteem, and cancer-related distress, in 212 adult members of families with documented BRCA1/2 mutations before and after counseling and possible testing. Intrusive thoughts, depressive symptoms, and breast and ovarian cancer worries improved in subjects with negative test results, but there was no change in those with positive results, or in those who declined testing.

In the second paper, McInerney-Leo and colleagues¹³¹ measured the impact of BRCA1/2 testing on family relationships using a validated index in the same study population. Interestingly, subjects who declined testing had more positive changes than those who accepted testing. In those who accepted testing, there was a non-significant trend towards decreased expressiveness among family members in those who had an abnormal test result.

Two studies reported measures separately for ovarian and breast cancer in women at risk for BRCA1 and BRCA2 mutations. In the first, Bish and colleagues¹³² collected data on a variety of quality-of-life measures in 203 subjects undergoing counseling regarding BRCA1 and 2 mutation testing because of family histories, and found that (1) worry about ovarian cancer was significantly less than worry for breast cancer; (2) worry about ovarian cancer was highest in women with a personal history of cancer, independent of the degree of risk or results of testing and (3) there was no overall change in worry about ovarian cancer in response to testing.

Claes and colleagues¹³³ performed a similar study in 71 similar subjects. As in the Bish study, there were differences in responses by cancer type: risk perception and distress were higher for breast cancer than for ovarian cancer. After testing, distress related to ovarian cancer was higher in carriers then in non-carriers, but was not signficiantly different from baseline levels by 12 months posttesting. Women who underwent prophylactic oophorectomy had decreased levels of concern, but higher levels of somatic symptoms.

Discussion

Only two articles specifically reported outcomes relevant to ovarian cancer. The majority of the available literature focuses on BRCA1/2 testing, but does not report results separately for ovarian and breast cancer outcomes. The differences observed in the studies we did identify suggest that testing for genetic markers of ovarian cancer susceptibility alone may have different implications compared to testing for genes that affect both breast and ovarian cancer risk.

For the most part, the potential harms associated with the use of genomic tests in screening, diagnosis, and management of ovarian cancer are no different than those of other tests, such as imaging: the risks of false-positive results leading to unnecessary and potentially dangerous treatment, as well as the psychological effects of a cancer diagnosis; and the risks of false-negative results leading to delayed diagnosis and therapy, with a potential for a poorer prognosis. The types of risks are similar - the only potential difference between genomic tests and other modalities lies in the quantitative risk of false-negative and false-positive results, which in turn depends on test sensitivity, specificity, and the pretest probability of disease. Higher quality evidence about the test characteristics of genomic tests for ovarian cancer should allow better estimation of these types of harms. The one type of test that might have implications similar to those for markers of increased risk would be a test for an inherited polymorphism that affected the likelihood of response to therapy; if such a polymorphism also affected the likelihood of responses or side effects for therapies for other conditions, the longer term implications for the patient and her family would have to be considered.

For tests of markers of increased risk, such as inherited mutations, there are some additional potential harms, namely, the impact of knowledge of increased risk when optimal options for reducing that risk, either through primary or secondary prevention, are unclear. Specifically, in the case of ovarian cancer, bilateral oophorectomy in premenopausal women affects childbearing potential and induces premature menopause; given the uncertainty about optimal methods for hormone replacement, this option may be even more confusing for some women. Because of varying degrees of penetrance, even the estimate of increased risk associated with a given mutation is subject to a fairly wide degree of uncertainty. Issues surrounding heritability in subsequent generations may also be important to some women. In this case, although the literature on BRCA testing is helpful in identifying some of these issues, there is a greater need for providing results specifically for ovarian cancer-related issues in studies of markers which affect the risk of several types of cancers. The available literature suggests that, for most women with BRCA mutations, breast cancer is a greater concern than ovarian cancer. Given the differences in both quantitative risk and the types of risks associated with testing, diagnosis, and prophylaxis, results need to be provided specifically for ovarian cancer-related outcomes in studies of BRCA testing.

Summary

The literature on the harms of genomic tests for ovarian cancer is sparse, with the majority of the available literature on psychological impacts of testing consisting of studies of women tested for susceptibility to both breast and ovarian cancer. Future studies will need not only to identify the short-term psychological impact of different test results, but also provide data on the outcomes of strategies used to reduce the risk of ovarian cancer in patients who undergo testing for susceptibility. The theoretical harms of genomic tests in the setting of screening, diagnosis, and management of ovarian cancer are similar to those for other types of tests and should ultimately be estimated based on better evidence for test characteristics and the effectiveness of management strategies based on test results.

Approach

We searched for articles that specifically measured responses by providers and/or patients to direct advertising campaigns. We also considered alternative sources of data on the nature and volume of direct-to-consumer and direct-to-provider marketing and identified methodological issues involved with utilizing these alternative sources.

Results

We did not identify any articles specifically targeting ovarian cancer genomic testing. We identified two articles that investigated the impact of a single advertising campaign for BRCA1/2 testing, targeted at women at high risk for breast or ovarian cancer.^{134, 135}

From September 2002 through February 2003, the U.S. manufacturer of BRCA1/2 testing conducted a pilot direct-to-consumer marketing campaign in Atlanta, Georgia, and Denver, Colorado. The campaign was targeted at women aged 25 to 54 with personal or family histories of breast and/or ovarian cancer, along with their healthcare providers. Television, radio, and print advertisements were generated to raise awareness about BRCA1/2 testing and to motivate women to ask their providers about how testing might help assess risk and change management. Providers received information and patient support materials prior to the beginning of the campaign. Although this marketing campaign was not designed as a research study, two groups were able to take advantage of the campaign and design studies to assess the impact of the campaign on test utilization.

In a study conducted by the Centers for Disease Control and Prevention (CDC) and the state health departments of Colorado, Georgia, North Carolina, and Washington,¹³⁴ investigators conducted a survey of providers and consumers in the two pilot cities and two comparison metropolitan areas (Raleigh-Durham, NC, and Seattle, WA). From April 21 through May 20, 2004, a 51-question consumer telephone survey was conducted using random telephone numbers, with a target response of 1,600 women. Questions included family history; campaign awareness; interest in genetic testing for BRCA1/2; cancer concerns; and interactions with providers, family, and friends. A 35-question survey and monetary survey were mailed to providers (randomly selected to be proportionately representative of family practice, internal medicine, obstetrics/gynecology, and oncology) on May 1, 2003; the target response was approximately 1,600.

One thousand and six hundred and thirty-five (1,635) women completed the survey, for a response rate of 45 percent; most were non-Hispanic white women with more than high school education and a median age of 40 years. Women in the pilot cities were significantly more likely to have heard of the test, but no significant differences were observed in stated knowledge about genetic testing, concern about risk for breast and ovarian cancer, or proportion of women who had talked with someone about genetic testing. Family histories were similar among those who expressed an increased interest compared to those who did not.

One thousand and fifty-four (1,054) providers completed the survey (66 percent response rate). Providers in pilot cities were significantly more likely to report that they and their patients had been exposed to an advertisement about genetic testing and to report an increase in the number of patients asking about testing, asking for genetic counseling, and requesting testing. The number of tests ordered increased significantly in the pilot cities as well, although the number of referrals for counseling did not increase. Provider knowledge about testing did not differ between cities, but knowledge did differ between specialists, with obstetricians/gynecologists and oncologists having higher levels of knowledge.

Limitations noted by study investigators included lack of data on non-responders; the potential for bias because of the low response rate among consumers; a relatively short lag time between the advertising campaign and the survey, which might have been insufficient to allow all those interested in testing to undertake and complete the process; lack of availability of data on the number of tests actually performed and the appropriateness of those tests because of the proprietary nature of the tests; and lack of data on the appropriateness of education, counseling, and testing ordered by providers.

A separate study conducted by investigators at Kaiser Permanente Colorado compared utilization of testing before and after the advertising campaign to similar time periods using data from the Henry Ford Health System in Detroit, Michigan.¹³⁵ Utilization assessment was through electronic records. The investigators noted a 240 percent increase in the number of referrals for genetic testing in Colorado during the advertising campaign compared to a similar time period 1 year prior to the campaign (from 144 referrals per average membership to 499 referrals per average membership), while no change was seen in Detroit (53 and 52 referrals per average membership during the two time periods).

Interestingly, although the absolute number of women with 10 percent or greater pretest probability of a mutation increased during the advertising campaign, the proportion of all referrals with a high pretest probability decreased from 69 percent to 48 percent in Denver, while no change was seen in Detroit. An increase in referrals from non-physician providers was noted.

The authors noted the difference between self-reported patient behavior in the CDC report and their observations; possible explanations included inaccurate self-report, differences in interest in testing between the general population and women with prepaid access to healthcare, concurrent education efforts by Kaiser Permanente, and discussion among women at workplaces or other settings with common insurance coverage.

Discussion

We identified only two relevant articles on the impact of direct-to-consumer advertising on utilization of genomic tests, both involving the same advertising campaign. One study found evidence of increased awareness of the test covered in the advertising campaign among consumers, but no self-reported increase in knowledge or intention to get tested. Conversely, providers reported the perception of an increased number of patients discussing and requesting testing and reported ordering more tests. The second study used administrative data to measure test utilization within a managed care organization before and after the campaign and found an increase in the number of tests ordered, with a decrease in the proportion of women with a high pretest probability of a mutation.

There are a number of methodological issues involved in assessing the impact of advertising on genetic test utilization:

Definition of “appropriate” use of testing. Possible definitions include:

• Use of the test only in those women with characteristics similar to those for whom the benefit of the test has been conclusively demonstrated, preferably through a randomized trial.

• Use of the test only in those women with characteristics that meet criteria agreed upon by expert consensus.

• Use of the test only in women who receive unbiased counseling on the state of knowledge regarding the benefits and harms of the test and who, based on their personal preferences, wish to have the test.

• Use of the test only in women for whom use of the test is estimated to result in an acceptable cost-effectiveness ratio.

Given the state of the literature on genomic tests in ovarian cancer, consensus opinions on appropriateness are likely to be the only criteria available for the near future.

Measuring test utilization. There are several challenges to measuring test utilization:

• As the CDC researchers noted, data on tests performed in private laboratories may not be publicly available; companies may view these data as proprietary information which, if available, would put them at a disadvantage in a competitive marketplace.

• Test utilization data could be obtained through administrative data, such as from a managed care organization. The ability to link to clinical data could help in estimating “appropriateness” - estimation of pretest probability of BRCA1/2 mutations by the Kaiser researchers is an example of this. However, this type of data is also often proprietary and not readily accessible to outside researchers. In addition, depending on the data source, there may be issues about the generalizability of results, since presumably utilization of tests and other resources would be more tightly constrained within a managed care organization. Also, as the Kaiser researchers point out, women enrolled in managed care plans may be different in many respects from the general population.

Quantifying direct-to-consumer advertising. Challenges to measuring and quantifying direct-to-consumer advertising include:

• Estimating exposure to various advertising in various types of media requires complex survey methodology.

• Measurement of the impact of non-advertising coverage in the media, such as coverage in news reports of scientific meetings, journal publications, or other forums (such as congressional hearings), needs to be considered.

• It is possible that other publicly available information could provide some insight into advertising; for example, annual reports of publicly traded companies might provide a breakdown of marketing expenses, although the extent to which specific inferences could be drawn about the nature of these marketing expenses is unclear.

Quantifying direct-to-physician advertising. Methdological challenges in measuring direct-to-physician advertising include:

• Quantifying advertising in journals would require either hand searches or access to records from a specific journal. This would be particularly difficult for non-peer-reviewed journals, which are frequently not maintained in medical libraries, but which may represent a significant portion of the literature.

• Quantifying other types of “advertising,” such as sponsorship of symposia at meetings, exhibits at meetings, sponsorship of continuing education activities at local hospitals, etc.

Study design. Even with a clear definition of “appropriate use” and methods for measuring utilization and advertising, there remain additional methodological issues. For example, although randomized trials would be ideal, it is difficult to imagine how one would be feasible. Before-and-after studies, with “exposed” and control populations, as were done with BRCA1/2 testing, appear to be the most practical, but require considerable planning, including, ideally, advance notice about the advertising campaign.

Summary

There are considerable methodological issues involved in determining the effect of direct-to-consumer and direct-to-physician marketing on the appropriate or inappropriate use of tests. We identified two studies of a single advertising campaign which suggested increased utilization of testing in the near term after a direct-to-consumer campaign, but provided little information on the appropriateness of the testing. The decrease in pretest probability observed after the advertising campaign suggests that, for some types of genomic tests, there may be a decrease in positive predictive value in response to advertising.

Model Validation

Lifetime cancer incidence and mortality. Figures 8–11 and Table 12 show the results of these calibrations under our two natural history assumptions of disease progression.

Both models closely approximate the Surveillance, Epidemiology, and End Results (SEER) lifetime risk for women at age 40 of 1.44 percent; there is a slight underestimation of mortality risk, primarily for women 85 and older (which is likely due to the assumption of constant stage-specific probability of diagnosis; because of age-specific variations in access to care, prevalence of conditions mimicking ovarian cancer, etc., this assumption may be incorrect). Lifetime risk of dying from ovarian cancer within the SEER data is 1.13 percent; model predictions are 1.06 percent.

Stage distribution. Table 12 illustrates overall and age-specific stage distributions from SEER and the two models.

Although many of the parameters of the model are known (stage-specific mortality, age-specific mortality from other causes, etc.), there are several parameters which are “unknowable unknowns.” In particular, we can never know with any degree of precision two key variables: (1) the rate at which an ovarian cancer at a given stage will proceed to the next stage; and (2) the probability that, within a given time period, a woman with a given stage of ovarian cancer will have her cancer detected and thus become an incident case. These probabilities must be imputed from available data - we know what the cancer incidence and stage distribution should be, and the values for these “unknowable” parameters are adjusted until a reasonable approximation is achieved. Table 13 shows the current values for these imputed probabilities.

The wider variation in stage distribution observed between age groups within the SEER data compared to model predictions may be due to any of several factors:

• (1)

  There may be age-specific variations in the rates of progression between stages; for example, age-related changes in hormonal status or immune function may affect the likelihood of metastasis.

• (2)

  There may be age-specific variations in the rates of detection within stages; for example, older women may have their cancer detected at later stages because of less access to physicians, different thresholds for seeking care for symptoms, or delayed diagnosis because the non-specific symptoms of ovarian cancer frequently mimic other conditions common in older women.

• (3)

  SEER data is cross-sectional, and the model is simulating a cohort. There may be unmeasured cohort effects in exposure to risk factors for ovarian cancer, competing risks, etc., which are not captured.

Future versions of the model will explore these possibilities by allowing the probabilities of progression and detection to vary with age. Allowing either more rapid progression to Stage III/IV or lower probability of detection among older women, in particular, would result in a greater proportion of advanced stage disease and higher mortality rates, and result in a closer match to SEER data. However, the greater “precision” of such an approach must be balanced against the risks of introducing inaccuracies by “overfitting” a cohort model which does not incorporate potential cohort effects into cross-sectional data.

Impact of Different Strategies

In order to compare the relative impact of different strategies and the conditions under which they would be effective, sensitivity analyses were conducted to determine the values for specific parameters that would result in a 20 percent reduction in ovarian cancer mortality. Table 14 summarizes these results.

Addressing the original questions:

Summary

• The current model closely approximates lifetime cancer incidence, mortality, and stage distribution. Differences between observed and predicted values for age-specific stage distribution and mortality will require further imputation of values for stage-specific progression and detection probabilities.

• Strategies which seek to identify high-risk groups are likely to have relatively small impact on overall ovarian cancer mortality, even if they are highly successful in reducing mortality in the risk group.

• Therapies after diagnosis which are based on genomic targets may reduce mortality substantially, but only if the targets are common and treatment highly effective.

• Reductions in ovarian cancer mortality of greater than 50 percent through screening require testing at intervals of 12 months or less, and are relatively independent of test sensitivity. Conversely, the number of false-positive results is quite high at these screening intervals unless test specificity is quite high.

• These findings suggest that the failure to identify effective strategies for ovarian cancer screening may be due at least in part to the natural history of the disease, rather than the failure of the tests evaluated.

Description of the Patient Population

Many of the issues identified in the evidence report on adnexal mass¹⁴ were found in this literature as well. The majority of the papers reviewed failed to adequately describe the patient population; in particular, for those studies that included women with both benign and malignant ovarian disease, the manner in which the mass was originally detected and the subsequent evaluation can affect the probability of underlying disease, and thus predictive values. Depending on study design, prevalence may also indirectly affect estimates of sensitivity and specificity, especially in cases where women who have a negative test do not undergo the reference standard evaluation. Thus, even though the performance of a given test may vary depending on whether the patient is symptomatic or asymptomatic, the failure of studies to describe this aspect of their population makes drawing inference about applicability in specific clinical settings difficult.

Another common shortcoming was the failure of many studies to describe potential differences in results stratified by age or menopausal status. Given the clear and widely recognized relationship between age and ovarian cancer risk, as well as the effect of menopausal status on the prevalence of biological processes that may affect the levels of some tumor markers such as cancer antigen 125 (CA-125), we believe that this should be standard in all studies of potential ovarian cancer tests. This is especially true for studies of complex phenomena such as multiple gene or protein expression patterns, where the discovery process is based on identifying differences in patterns between populations; if some of the identified differences between cancer patients and controls in gene expression or protein profiles are in fact differences related to aging, menopausal status, or other processes unrelated to ovarian cancer, the ultimate sensitivity and specificity of tests based on these pattern recognitions may be substantially worse than in preliminary reports.

Sample Size

Few of the studies we reviewed included a priori sample size calculations, and use of confidence intervals for parameter estimates was uncommon. Our calculated confidence intervals were, for the most part, quite wide.

Prevalence of Ovarian Cancer in Samples

The majority of the studies we reviewed included prevalence of ovarian cancer of 30 to 60 percent. In a screening setting, this is several orders of magnitude higher than the observed prevalence in screening studies in the U.S. of 0.05 to 0.2 percent.¹⁴ This higher prevalence leads to falsely decreased confidence intervals for the estimates of sensitivity. Even more importantly, as only one author⁵⁹ pointed out, this prevalence lowers the positive predictive value of tests to substantially less than 5 percent.

The prevalence in studies of the diagnostic use of a test would be expected to be higher, but, as discussed above, how much higher is dependent on the age, menopausal status, symptom status, etc., of the patient. Failure to describe these characteristics prevents assessment of how closely the study population reflects a likely clinical population.

Stage Distribution within Samples

Given that most ovarian cancer presents in later stages, the stage distribution of samples used for test development and validation is likely to be skewed towards later stages. Any abnormalities identified may be more common in advanced cancers than early cancers; since the goal of screening is to identify early stage (or even preinvasive cancers), the sensitivity for tests derived from these types of samples may be quite lower in real-world settings. This is especially true for tests which identify simultaneous multiple changes in a variety of markers (such as gene arrays or studies of protein patterns) without a clear understanding of the underlying biological significance of the changes. Changes associated with late stage cancer may not be seen in early cancers.

Biological and Observer Variability in Test Results

Many studies, especially of tests for single gene products, reported measures of assay reproducibility. However, we identified only one series of studies^{40, 41} that reported on the impact of both test and biological variability on interpretation of test results, in this case the significance of changes in CA-125 levels. Since both test reproducibility and biological variations may affect test characteristics (especially in applications where serial measurements are used to make clinical decisions), documentation of these effects for other tests should be required.

Similarly, for studies of multiple gene or protein expression, demonstration of reproducibility of results by different groups using similar analytic approaches is necessary; the available evidence suggests that reproducibility is still an issue.^{55, 59}

Use of Tests for Decisionmaking in Management

The majority of the studies we identified on the use of genomic tests in patients already diagnosed with ovarian cancer reported on associations between test results and certain clinical outcomes, such as lack of response to chemotherapy, positive second look laparotomy, or length of overall or disease-free survival. We did not identify any studies which explicitly discussed how these results could be used, let alone any studies which formally tested the impact of use of the tests on patient outcomes. For example, if the results of a genomic test indicate a greater likelihood of failure to respond to standard chemotherapy, should that patient be offered only experimental therapies, or comfort care, rather than undergoing the effects of therapy which is unlikely to work? There are obvious ethical and feasibility issues involved in designing studies of such an approach, but if a patient will undergo the same therapy regardless of the results of the genomic tests, there seems to be little clinical value to performing the test. It is possible that there is some value to the patient and her family in having greater information about the probability of various outcomes, even if therapy is not affected, but this should be demonstrated using appropriate study designs and instruments.

Selection of Cases and Controls in Initial Test Development

The most common approach to initial test development in the studies we reviewed was to use serum or tissue from patients with cancer and compare results to a comparison group with no disease, or with non-ovarian cancer disease. In some cases, an attempt was made to discriminate between normal women, women with early stage ovarian cancer, and women with late stage ovarian cancer, in the hopes of identifying markers of early stage disease.

There are several implicit assumptions involved with this type of study design:

• If attempts are not made to discriminate between stages, then the assumption is that all cancers, regardless of stage, exhibit a similar pattern. However, if there are changes in gene and/or protein patterns which are associated with advancing stage, failure to examine differences between stages may affect test accuracy.

• If stages are examined differently, then the assumption is that all of the advanced stage cancers must have “looked” like the early stage cancers at some point. However, if “early” stage cancers represent cancers which are biologically different, rather than an early, necessary step in the development of ovarian cancer,¹³⁶ then identification of markers of early stage cancer may not result in substantial reductions in mortality.

• There are no other factors other than ovarian cancer itself which can explain observed differences between cancers and controls. The effect of potential confounders such as age, menopausal status, or other factors on gene or protein patterns would affect test specificity substantially.

Ultimately, the ideal approach is to use prospectively collected sera to attempt to identify markers for those patients who subsequently developed advanced stage ovarian cancer, an approach which may be achievable in some of the large ongoing studies of ovarian cancer screening.¹³⁷

Natural History of Ovarian Cancer

The search for better screening tests for ovarian cancer has been based on the implicit assumption that ovarian cancer progresses through a series of stages in a fashion analogous to that of cervical cancer. Alternative models are biologically plausible, and, as demonstrated by our simulation models, mathematical models can be “fitted” to match reported data under both alternatives. Our modeling suggests that, even under a model that assumes that all cancers progress through Stage II, screening at intervals more frequent than every 12 months is needed to reduce mortality by greater than 50 percent, even with a highly sensitive test, and that such screening would have a very low positive predictive value, even with a highly specific test.

If this is the case, then alternative methods for reducing ovarian cancer morbidity and mortality, such as improved methods for primary prevention and improved therapies, may ultimately offer more promise than the search for the Pap test equivalent in ovarian cancer.

Modeling

We were able to closely approximate reported ovarian cancer incidence and mortality using a simulation model. This model can be used to identify test and treatment characteristics that would result in substantial reductions in ovarian cancer mortality. The most striking finding of the model is that the effect of screening frequency in achieving large-scale reductions in ovarian cancer mortality is greater than that of test sensitivity; achieving mortality reductions greater than 50 percent requires screening frequencies of less than 12 months. This is problematic for several reasons. First, a pilot study suggests that women are unlikely to be compliant with more frequent screening intervals.¹³⁸ Second, more frequent screening results in lower overall positive predictive value, even with a highly specific test. Finally, if effective primary prevention strategies are identified which lower the incidence of ovarian cancer, the positive predictive value of screening will be lowered to an even larger extent.