NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Bast RC Jr, Kufe DW, Pollock RE, et al., editors. Holland-Frei Cancer Medicine. 5th edition. Hamilton (ON): BC Decker; 2000.

Cover of Holland-Frei Cancer Medicine

Holland-Frei Cancer Medicine. 5th edition.

Show details

Chapter 78Outcomes Assessment

, MD, MSc.

Over the past decade, there has been an explosion of interest in measuring the outcomes of medical care. The science of measuring outcomes and of integrating that process into the routine care of patients has come to be known as the “outcomes movement.” Outcomes is an imprecise term that has different meanings in different contexts. In the narrowest sense, outcomes are what patients experience as a result of disease and its treatment. Often, the discipline of outcomes assessment is interpreted more broadly and encompasses, in addition, the study of how patients are treated, determinants of treatment choice, quality of care, costs of delivering medical care under various conditions, and optimal allocation of society’s and individual institutions’ limited health-care resources. This blending of clinical and policy perspectives is the hallmark of outcomes assessment.

Historic Perspective

In the classic paradigm of quality assessment in health care, three types of information can be considered: structure, process, and outcome. Structure describes the environment, including the physical plant and resources available; process is what is done for patients; and outcome is what is accomplished for patients.1 There were early proponents of measurement of outcomes as the primary indicator of quality, most notably, Ernest Codman, a Boston surgeon and zealous advocate of the “end-results idea” of linking specific interventions with their effects on patients.2 However, until recently, measures of structure and process were the dominant methods of assessing the quality of medical care.

In the 1980s two important observations resulted in rekindled interest in the assessment of outcomes. First, efforts at health-care cost containment focused on reimbursement mechanisms were failing to produce the hoped for decreases in the escalating proportion of the gross domestic product spent on health care. It was recognized that the primary cause of the cost crisis was not so much the price of health care, but rather, the increasing volume and intensity of medical services.3 Second, a number of investigators examined large, administrative databases and found substantial geographic variability in the use of various medical interventions, without associated differences in medical outcomes.4–6 These observations laid the groundwork for what Arnold Relman labeled the “Third Revolution in Medical Care,” namely, the “Era of Assessment and Accountability.”7

The most tangible effect of this revolution was the creation in 1989 of the federal Agency for Health Care Policy and Research (AHCPR). In order to achieve its mission of generating and disseminating information to improve the delivery of health care, AHCPR funded a series of Patient Outcome Research Teams (PORT) studies.8 These large-scale, 3- to 5-year clinical studies were designed to evaluate the effectiveness, in clinical settings, of health-care interventions.8 The early PORTs relied heavily on existing large databases. It became clear, however, that collection of primary data, through randomized trials or prospective, longitudinal studies, was often required to adequately assess the effectiveness of new and established medical interventions. Over time, AHCPR and the outcomes research community have placed increasing emphasis on measuring and intervening to improve the “appropriateness” and quality of care.9 Reflecting the growing emphasis on quality of care within the agency, the AHCPR was renamed the Agency for Health Care Research and Quality in late 1999.

The recognition of the importance of outcomes in determining the value of medical treatments has not been limited to the federal government. Insurers also use outcome data in making decisions about what treatments to include in benefit packages. Increasingly, managed care organizations are also taking institutional outcomes into account in contracting decisions with hospitals and other providers.

Outcome Measures

Disease Outcomes

In outcomes research as well as clinical trials, the effectiveness of cancer therapy is judged primarily in terms of its impact on the course of the disease. The most important outcome to consider in evaluating a cancer treatment in a clinical trial or clinical practice setting is survival. Disease-free survival may also be a useful end point to consider, especially in the adjuvant setting. In its position paper on outcomes of cancer therapy for technology assessment and cancer treatment guidelines, the American Society of Clinical Oncology (ASCO)10 distinguishes between patient outcomes, such as survival, and cancer outcomes, such as tumor response and biomarker measurements. The report concludes that while cancer outcomes may be useful in the drug development process, technology assessments and guidelines should be based primarily on evaluations of the impact of treatment on patient outcomes.

Quality of Life

A full assessment of the outcomes of cancer treatment involves a consideration of its impact on not only length of life but also quality of life. Quality of life is a multi-dimensional concept that includes physical, social, and psychological functioning. It is particularly important to consider quality-of-life outcomes when treatment is given with palliative intent or when toxic therapy is likely to yield only a modest survival benefit. In treatment decisions involving trade-offs between length and quality of life, it may be useful to measure quality-adjusted survival, a single outcome that incorporates both length and quality of life (Fig. 78.1). Calculation of quality-adjusted survival requires that quality of life be measured in such a way that the product of length and quality of life is meaningful, for example, that 1 year of life at a quality of x is as desirable as 6 months of life at a quality of 2x. Measures of quality of life that have this property are called utilities.11 Utilities are defined as the quantitative measure of the strength of a person’s preference for an outcome.12 By convention, utilities are measured on a scale of 0 to 1, in which 0 represents death and 1 represents excellent health. They differ from more familiar measures of quality of life, in that they reflect how a patient values a state of health, not just the characteristics of the health state. The terms utilities, values, and preferences are sometimes used interchangeably.

Figure 78.1. Quality-adjusted survival, measured in quality-adjusted life years (QALYs), represents the area under a curve of survival versus quality of life, measured in utilities.

Figure 78.1

Quality-adjusted survival, measured in quality-adjusted life years (QALYs), represents the area under a curve of survival versus quality of life, measured in utilities.

Cost and Cost-Effectiveness

The goal of any health economic analysis is to determine whether the cost of a particular intervention is justified by the health benefits it produces. The question is usually framed by asking not how much it costs to deliver a particular treatment, but how much more it costs to provide that treatment than the most reasonable alternative.13 This alternative may be a “no treatment” strategy, but this does not necessarily mean it is a “no cost” strategy.

All economic analyses examine the difference in cost between alternative strategies; they differ in how they measure the benefits resulting from those strategies13–15 (Table 78.1). The four basic types of economic analysis measure these benefits in four different ways. A cost-minimization study simply assesses the additional cost of one strategy in comparison with another and therefore implicitly assumes that the two treatments produce comparable benefits. Because alternative medical interventions rarely produce truly equivalent outcomes, this type of analysis generally does not suffice as a complete economic evaluation of competing interventions. Usually, one wants to know whether the additional benefit conferred by the more expensive treatment is sufficient to justify the additional cost.

Table 78.1. Types of Economic Analysis.

Table 78.1

Types of Economic Analysis.

Cost-benefit analyses answer this question by assigning a dollar value to the health outcome in order to determine whether the incremental benefit of one treatment over another, measured in monetary terms, is greater than or equal to the incremental cost. Cost-effectiveness analyses, in contrast, measure the benefits of health-care interventions in units of medical effect. For example, the cost-effectiveness of combination chemotherapy, compared with single agent therapy, for a given disease could be assessed by calculating the additional cost (in dollars) per additional patient reaching the 5-year disease-free survival mark. However, one of the goals of cost-effectiveness analysis is to facilitate resource allocation decisions between interventions to treat or prevent different diseases. Cost-effectiveness data are much more useful if health benefits are measured in units that are common across diseases. Years of life saved is the most frequently used measure. Cost-effectiveness ratios are, therefore, usually expressed in terms of dollars per year of life saved.

Medical interventions affect not only length of life but also quality of life. Cancer cure may be bought at the expense of substantial treatment-related morbidity. Conversely, palliative therapy may bring marked relief of symptoms, even if it does not lengthen life dramatically. Cost-utility analysis, a specific type of cost-effectiveness analysis, takes into account the impact of a health intervention on quality of life as well as length of life. Most commonly, this is done by assessing health benefits in terms of quality-adjusted survival, measured in quality-adjusted life years (QALYs) (see Fig. 78.1). The units of a cost-utility ratio are, thus, dollars per QALY.

Study Designs

Cohort Studies

Outcomes research emphasizes the effectiveness, rather than the efficacy, of medical interventions. Efficacy, defined as the results of an intervention in carefully selected patients treated under controlled conditions, is best measured by a phase III trial. Effectiveness, in contrast, is a measure of the impact of routine medical interventions in all patients. It is assessed by observing the outcomes of care, typically in a cohort study design.

In a cohort study, subjects are not assigned to a particular treatment or intervention by the investigator, as they are in a randomized trial. Instead, treatment is chosen in the course of routine care, and subjects are then monitored to determine outcomes.16 The advantages of a cohort study design include its comparatively low cost, the ability to evaluate the effects of treatments in types of patients who are under-represented in clinical trials, such as the elderly or patients with comorbid diseases, and the opportunity to evaluate the effectiveness of a treatment when it is given under routine conditions. The major disadvantage of the cohort study design is “confounding,” the influence of factors that affect both treatment choice and outcome. For example, if patients with extensive comorbid diseases are consistently treated with a less aggressive regimen and have a poorer outcome, one might mistakenly conclude that the regimen is less effective, when, in fact, those patients fared poorly simply because of a higher burden of comorbidity. Prospective enrollment of patients with careful attention to pretreatment measurement of potential confounding factors, and use of techniques, such as multivariate analysis to control for them, enhances the validity of a cohort study.

One particular type of cohort study that deserves specific mention is the large database study. Databases amassed for administrative purposes, such as the Medicare billing of the Health Care Financing Administration (HCFA), may contain fairly extensive information on not only the costs of care but also the location and nature of medical treatments received by patients. These databases have been used effectively to study the outcomes associated with common medical treatments in huge cohorts of patients treated in the community. They have been especially useful in detecting the influence of factors such as geographic location, race, and age on treatment choice and outcome. The principal limitation of this data set as a research resource has been addressed by the creation of linkages between HCFA files and tumor registry records from the Surveillance, Epidemiology and End Results (SEER) program.17 These linked files allow researchers to examine the costs and patterns of care among patients classified not only by type of cancer but also stage. Even this data set has limitations, however. It cannot be used to examine the care of younger patients who are not covered by Medicare, nor does it provide detailed information on the specific treatment regimens used.

Clinical Trials

Phase I and II trials, while an essential component of the drug development process, are not particularly helpful in evaluating the outcomes of new or established therapies. Phase III trials, in contrast, are an essential source of information on the outcomes of alternative treatments. If the trial is sufficiently large, randomization eliminates the problem of confounding by distributing all known and unknown factors that may influence outcomes equally among arms. The primary limitation of randomized controlled trials as a source of outcome data is that the results may not be generalizable. It is well documented that the tiny proportion of patients who participate in clinical trials are not representative of the population as a whole, in terms of race, socioeconomic factors, or level of comorbidity.18,19 In addition, clinical trials often dictate procedures for staging, treatment, and evaluation of outcomes that differ dramatically from practices that would be adopted in routine clinical care. “Large simple trials” are one solution to enhancing the generalizability of outcome data from clinical trials.20,21 Oncology has lagged behind other disciplines, such as cardiology, in incorporating such trials into its clinical research portfolio.

The value of randomized trials in generating outcome data may also be augmented by broadening the array of end points to include quality of life and costs. Randomization enhances the validity of conclusions regarding the effects of treatment on these outcomes, just as it does for survival and other biologic outcomes. If costs, for example, are related to pretreatment characteristics and those characteristics also influence treatment choice, then it may be difficult or impossible to determine the independent impact of treatment choice on cost in a cohort study design. In addition, surveys to elicit quality-of-life and economic data are expensive, and it may be more efficient to piggyback them onto clinical trials than to collect data through another mechanism. The medical record reviews, audits, data management, and follow-up that occur in a clinical trial may be used to collect information on economic end points, for example, at relatively little additional cost.

Decision Analysis

For many medical decisions, some empiric data on the effectiveness of alternative therapies are available, but the definitive experimental and observational studies have not been done. Decision analytic models may be very helpful in aggregating the available data under these circumstances and in pinpointing areas where additional research is needed.22,23 The first step in performing a decision analysis is to construct a computer model in the form of a decision tree which diagrams all the possible downstream outcomes of a particular treatment choice (Fig. 78.2). Probabilities are then assigned to each branch of this tree. These probabilities come from relevant clinical trials or cohort studies; when no data are available, expert judgement may be used. Outcomes are then assigned to each branch of the tree. A variety of outcomes may be used in decision analysis, including survival, quality-adjusted survival, and cost. The model is then analyzed to determine which treatment yields the best overall outcome. The most important step in decision analytic modeling is sensitivity analysis. Each estimate in the model is varied, singly and in combination, to determine how sensitive the results are to any particular estimate. Sensitivity analysis is analogous to tests of significance in empiric studies, in that it indicates how much confidence one should have in the conclusions.

Figure 78.2. Example of a simple decision tree.

Figure 78.2

Example of a simple decision tree. Adapted from Moore and colleagues. The tree models the choice, represented by a square “decision node,” between treatment with prophylactic intravenous immunoglobulin (IVIG) in chronic lymphocytic leukemia (more...)

Cost-Effectiveness Analysis

A comprehensive cost-effectiveness analysis requires data on how the alternative treatments being compared affect (1) length of life, (2) quality of life (measured in utilities), and (3) costs.24 Often, cost is determined by measuring resource use (e.g., hospital days, drugs) and multiplying each unit of resource by an estimate of its cost. Ideally, costs rather than charges are used. In economic terms, costs are a measure of the resources consumed to provide a service. Charges may include some degree of profit and also are influenced by regulation and cost shifting between departments and are, therefore, poor proxies for costs.25 A full accounting of the direct medical costs of a particular medical intervention includes not only the drugs, physicians fees, and hospital days required to deliver that intervention but also the downstream cost consequences of treatment side effects, the morbidity avoided if the disease is successfully treated, and the additional diseases and medical problems that occur in patients who live longer as a result of the original treatment (Table 78.2). Depending on the perspective of the analysis, direct nonmedical costs for items such as transportation to the hospital and family care, as well as indirect costs in the form of wages gained or lost because of disease and its treatment, may also be included.15,16,24

Table 78.2. Net Health-Care Costs of an Intervention.

Table 78.2

Net Health-Care Costs of an Intervention.

A variety of sources may be used to generate each of the categories of data involved in estimating cost-effectiveness (Table 78.3). In the past, most cost-effectiveness analyses were done using decision analytic techniques to combine data on effectiveness, drawn from published studies or expert judgment, with cost data, often estimated from a single institution’s experience in treating similar patients. Increasingly, data for cost-effectiveness analyses are being collected prospectively alongside cancer clinical trials. The pharmaceutical industry has taken the lead in these studies, spurred by Australia’s requirement that evidence of cost-effectiveness be included in requests to the public health service to add new drugs to the list of medications approved for payment as well as the practice of regulators in many European countries of relying on economic data to set drug prices.27 Several of the cooperative groups, including the Cancer and Leukemia Group B (CALGB) and Southwestern Oncology Group (SWOG) are following suit and are conducting phase III trials, in which resource use data and bills of study patients are being collected prospectively.28 These data will be used to estimate how much more it costs to treat patients on one arm of the study than the other and will be combined with clinical data from the trials to generate estimates of the cost-effectiveness of one treatment in comparison with the other.

Table 78.3. Sources of Data for Cost-Effectiveness Analysis.

Table 78.3

Sources of Data for Cost-Effectiveness Analysis.

Outcomes Studies in Oncology

Patterns and Quality of Care

While not outcomes studies in the narrowest sense, analyses of the nature and determinants of treatment choice among patients with a particular diagnosis are fundamental to the outcomes movement. Using administrative billing data collected by the federal government and other payers, as well as data from the SEER program and other tumor registries, a number of investigators have shown that geographic location, hospital type and size, and nonmedical patient characteristics may influence cancer treatment choice. For example, recent large database studies have found substantial geographic variations in the use of breast conserving surgery for early breast cancer,29–39 and radical prostatectomy for early prostate cancer.32 Distressingly, the patterns and quality of care have also been found to vary by gender,33 race,34–39 and socioeconomic status.33,37,38,40,41 One particularly disturbing study found that patients’ health insurance status predicted for stage at diagnosis and survival, stage for stage, in breast cancer.42 Several studies have demonstrated that older and non-Caucasian men receive less aggressive therapy for prostate cancer, differences that persist after controlling for stage, comorbidity, and treatment site.43,44 Racial disparities in cancer treatment have also been demonstrated in early non–small cell lung cancer, with African-Americans less likely to undergo curative resection and to experience poorer survival as a result.39

These studies have relied on secondary data sources and have, therefore, been unable to examine whether these differences reflect unmeasured clinical characteristics, physician prejudices, or patient preferences. In-depth studies of the decision-making process have suggested that the primary determinant of cancer treatment choice is physicians’ recommendations.45 These recommendations, in turn, may be heavily influenced by physicians’ specialties. For example, in a study in which physicians were presented with hypothetical scenarios describing patients with prostate cancer, physicians generally favored the modality in which they were trained, urologists opting for surgery, radiotherapists favoring radiotherapy, and medical oncologists dividing their votes equally between the two treatments.46

The experience of providers has also been shown to influence the outcomes of the care they provide. The data supporting this so-called “volume–outcome” relationship are strongest for technically complex procedures, such as pneumonectomy and pancreatectomy.47 But there is a growing body of literature suggesting that in many cases, specialized providers who treat large numbers of patients with a particular condition achieve superior short- and long-term outcomes.48

Effectiveness Studies

Little systematic research on the effectiveness of alternative cancer therapies in the community setting have been performed. Several recent and ongoing large-scale prospective cohort studies, however, should help elucidate not only the determinants of treatment choice but also the outcomes of treatment in the routine care setting for several of the most common solid tumors. For example, two large observational studies of outcomes of early prostate cancer found much higher rates of sexual and urinary dysfunction after radical prostatectomy than had been reported in prior case series from single institutions or surgeons.49–53 Notably, high levels of impotence were found even among men who had undergone nerve-sparing prostatectomy.50,51 The results of these cohort studies are particularly helpful since efforts to mount randomized trials in prostate cancer have been marked by very poor accrual.

Cost-Effectiveness Studies

Until recently, the majority of cost-effectiveness studies in cancer were economic studies of screening. In the past few years, a number of studies of the cost-effectiveness studies of various treatments for cancer have appeared in the literature. This literature has been summarized in a recent comprehensive review;54 the cost-effectiveness ratios for a subset of those studies are shown in Table 78.4.55–66 There is no absolute standard for what constitutes a reasonable number of dollars per year of life saved. A range for this threshold has been established by examining cost-effectiveness ratios for interventions that are generally regarded by society as reasonable and those that are generally regarded as inordinately expensive for the degree of benefit produced. The conclusion is that interventions costing under approximately $50,000 per year of life saved are cost effective, and those costing over $100,000 are cost ineffective. Cost-effectiveness ratios falling between these levels are in a gray zone. Applying this standard to the results shown in Table 78.4, it is evident that cost-effectiveness ratios for cancer treatment are much more sensitive to the magnitude of the benefit produced than to the price tag of the therapy. In general, cancer therapies that result in clinically meaningful improvements in patient outcomes have been shown to be cost effective, regardless of the cost of the treatment.

Table 78.4. Examples of Published Cost-Effectiveness Analyses in Cancer.

Table 78.4

Examples of Published Cost-Effectiveness Analyses in Cancer.

What is the relevance of cost-effectiveness analyses to the practicing clinician? In the United States, though not necessarily in all the developed countries, it is generally believed that the physician should function as the patient’s advocate in these discussions and should offer any treatment likely to be of net benefit, regardless of the cost to society. But some clinicians serve other roles as well, as administrators and policy makers, jobs in which they need to consider whether the costs of various health-care services are justified by the benefits they produce. And all practitioners must have some understanding of issues of cost and cost-effectiveness if they are to have any voice in the debate over allocation of health-care dollars and other resources.

Outcomes Management

Outcomes management describes the process by which studies of the outcomes of care are used to improve the delivery of medical services. Paul Ellwood has characterized outcomes management as a way “to help patients, payers, and providers make rational medical care–related choices based on better insight into the effect of these choices on the patient’s life.”67 To accomplish this goal, he argues, outcomes management requires four components: (1) guidelines that physicians can use in selecting appropriate interventions; (2) systematic measurement of the functioning and well being of patients; (3) pooling of clinical and outcome data on a massive scale; and (4) analysis and dissemination of the results of the database.

Clinical practice guidelines are at the heart of outcomes management. These standards, referred to by some as “cookbook” medicine, specify how patients in particular clinical circumstances should be treated.68–72 Guidelines dictate what the treatment should be; “critical paths,” in contrast, specify how that therapy should be delivered, including the frequency of follow-up visits and testing, the tasks to be assumed by physicians and nurses, and so forth. Guidelines are designed to eliminate variation in patterns of care that represent deviations from what is believed to be the most effective and cost-effective therapy for a given disease. How much emphasis to place on consideration of cost in the development of guidelines is an area of great controversy.

The Institute of Medicine has laid out criteria for selecting topics for guidelines which take into account the prevalence of the problem; the burden of illness imposed by the condition; cost; variability in practice; potential to improve health outcomes; and potential to reduce costs.73 Clearly, many cancer diagnoses fulfil these criteria. Therefore, it is not surprising that cancer clinical practice guidelines are proliferating.

Although AHCPR has published only one guideline directed at cancer treatment, on management of cancer pain, other groups have been more active in developing guidelines for cancer care. After each of its Consensus Development Conferences, the National Institutes of Health (NIH) issues statements distilling the presentations of experts to identify areas of consensus, emerging trends in the clinical literature, and areas requiring further investigation. These reports include both definitive statements about what constitutes optimal care in areas where consensus was reached, and recommendations regarding therapeutic approaches to be considered in other areas. Recently, the trend has been away from such consensus-based guidelines to an evidence-based approach, in which guidelines are explicitly grounded in a comprehensive review of the relevant literature graded for the quality of the evidence presented. The ASCO has produced a series of highly rigorous evidence-based guidelines on the use of hematopoietic colony– stimulating factors,74,75 use of tumor markers in breast and colorectal cancer,76,77 breast cancer surveillance,78,79 treatment of non–small cell lung cancer,80 and colorectal cancer surveillance.81 More comprehensive, but less rigorously evidence-based guidelines have been developed by several cancer provider networks. The nearly 100 guidelines of the National Comprehensive Cancer Network (NCCN), for example, provide treatment pathways for 94% of cancers in the United States.82 Patient versions of several of these guidelines have also been developed in a collaborative effort by the NCCN and the American Cancer Society.

Under pressure to control costs and minimize unwarranted variability in care, many institutions are developing their own guidelines for the treatment of common cancers. To be successful, such an effort requires a substantial commitment in time and money, an investment which may be beyond the resources of some institutions. However, the involvement of the ultimate users of a guideline in its development and the tailoring of the guideline to local conditions helps ensure buy-in by clinicians. Proponents of institutional guidelines argue that as a result, these guidelines may produce more sustained effects on practice patterns than the guidelines promulgated by national bodies have been able to achieve.83

Summary

At the heart of the outcomes movement is a very simple idea: the value of the health care we deliver should be determined by a systematic examination of patient outcomes. The novelty of this approach lies largely in its emphasis on how patients fare in routine clinical care, not just in clinical trials, and in the broadening the array of outcomes to include quality of life and costs as well as biologic end points. To accomplish this goal, methods are required which are relatively unfamiliar to cancer clinical researchers and practitioners, including observational cohort studies, decision analysis, and cost-effectiveness analysis. In comparison with the finely honed, sophisticated study design and statistical methods that have been developed for use in clinical trials, many of these techniques are relatively untested. Further methodologic work is clearly needed. However, there is a more fundamental challenge that must be met if the outcomes movement is to deliver on the grandiose promises made by many of its proponents. Physicians, patients, and health-care institutions must agree to participate in data collection efforts on a massive scale. Assessments of outcomes must be integrated into the daily routine of caring for cancer patients. The resulting data must be shared and continuously analyzed to create a feedback loop that produces continuous modification and improvement of clinical practice guidelines. In the short term, interest in outcomes assessment and management is being driven largely by a desire to control health-care costs. If this “Third Revolution in Medical Care” is successful, however, the real pay-off will be higher quality medical care for cancer patients.

References

1.
Donabedian A. The role of outcomes in quality assessment and assurance. QRB Qual Rev Bull. 1992;18:356–360. [PubMed: 1465293]
2.
Codman EA. The shoulder: rupture of the supraspinatus tendon and other lesions in or about the subacromial bursa. Boston, MA: Thomas Todd Company; 1934.
3.
Roper W L. Perspectives on physician-payment reform: the resource-based relative value scale in context. N Engl J Med. 1988;319:865–867. [PubMed: 3045555]
4.
Wennberg J E, Freeman J L, Culp W J. Are hospital services rationed in New Haven or over-utilized in Boston? Lancet. 1987;1:1185–1189. [PubMed: 2883497]
5.
Wennberg J E, Freeman J L, Shelton R A, Bubolz T A. Hospital use and mortality among Medicare beneficiaries in Boston and New Haven. N Engl J Med. 1989;321:1168–1173. [PubMed: 2677726]
6.
Wennberg J E, Mulley A G, Hanley D. et al. An assessment of prostatectomy for benign urinary obstruction. Geographic variations and the evaluations of medical care outcomes. JAMA. 1988;259:3027–3030. [PubMed: 2452906]
7.
Relman A M. Assessment and accountability. The third revolution in medical care. N Engl J Med. 1988;319:1220–1222. [PubMed: 3173460]
8.
Youngs MT, Wingerson L. AHCPR profile, the 1996 medical outcomes and guidelines sourcebook. New York, NY: Faulkner and Gray; 1995.
9.
Lee S J, Earle C C, Weeks J C. Outcomes research in oncology: history, conceptual framework and trends in the literature. J Natl Cancer Inst. 2000;92:195–2040. [PubMed: 10655436]
10.
Outcomes Working Group, Health Services Research Committee, American Society of Clinical Oncology. Outcomes of cancer treatment for technology assessment and cancer treatment guidelines. J Clin Oncol. 1996;14:671–679. [PubMed: 8636786]
11.
Tsevat J, Weeks J C, Guadagnoli E. et al. Using health-related quality of life information: clinical encounters, clinical trials, and health policy. J Gen Intern Med. 1994;9:576–582. [PubMed: 7823230]
12.
Torrance G W. Measurement of health state utilities for economic appraisal. A review. J Health Econ. 1986;5:1–30. [PubMed: 10311607]
13.
Detsky A S, Naglie I G. A clinician’s guide to cost-effectiveness analysis. Ann Intern Med. 1990;113:147–154. [PubMed: 2113784]
14.
Eisenberg J M. Clinical economics. JAMA. 1989;262:2879–2886. [PubMed: 2509745]
15.
Weeks J C. Taking quality of life into account in health economic analyses. J Natl Cancer Inst Monogr. 1996;20:23–27. [PubMed: 8750463]
16.
Hennekens CH, Buring JE. Epidemiology in medicine. Boston, MA: Little, Brown; 1987.
17.
Potosky A L, Riley G F, Lubitz J D. et al. Potential for cancer related health services research using a linked Medicare-tumor registry database. Med Care. 1993;31:732–748. [PubMed: 8336512]
18.
Svensson C K. Representation of American blacks in clinical trials of new drugs. JAMA. 1989;261:263–265. [PubMed: 2909024]
19.
Hunninghake D B, Darby C A, Probstfield J L. Recruitment experience in clinical trials: literature summary and annotated bibliography. Control Clin Trials. 1987;8:6S–30S. [PubMed: 3326716]
20.
Yusuf S, Collins R, Peto R. Why do we need large, simple randomized trials? Stat Med. 1984;3:409–422. [PubMed: 6528136]
21.
Peto R, Collins R, Gray R. Large scale randomized evidence: large, simple trials and overviews of trials. J Clin Epidemiol. 1995;48:23–40. [PubMed: 7853045]
22.
Richardson W S, Detsky A S. Users’ guides to the medical literature. VII. How to use a clinical decision analysis. JAMA. 1995;273:1610–1613. [PubMed: 7745775]
23.
Weinstein MC, Fineberg HV. Clinical decision analysis. Philadelphia, PA: W.B. Saunders; 1980.
24.
Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-effectiveness in health and medicine: report of the panel on cost-effectiveness in health and medicine. New York, NY: Oxford University Press; 1996.
25.
Finkler S A. The distinction between costs and charges. Ann Intern Med. 1982;96:102–109. [PubMed: 7053682]
26.
Weinstein M C, Stasson W B. Foundations of cost-effectiveness analysis for health and medical practices. N Engl J Med. 1977;296:716–721. [PubMed: 402576]
27.
Bloom B S. Issues in mandatory economic assessment of pharmaceuticals. Health Aff (Millwood) 1992;11:197–201. [PubMed: 1483640]
28.
Nelson H, Weeks J C, Weiand H S. Proposed phase III prospective randomized trial comparing laparoscopic-assisted colectomy versus open colectomy for colon cancer. Monogr Natl Cancer Inst. 1995;19:51–56. [PubMed: 7577206]
29.
Nattinger A B, Gottlieb M S, Veum J. et al. Geographic variation in the use of breast-conserving treatment for breast cancer. N Engl J Med. 1992;326:1102–1107. [PubMed: 1552911]
30.
Osteen R T, Steele G D, Menck H R, Winchester D P. Regional differences in surgical management of breast cancer. CA Cancer J Clin. 1992;42:39–43. [PubMed: 1728336]
31.
Farrow D C, Hunt W C, Samet J M. Geographic variation in the treatment of localized breast cancer. N Engl J Med. 1992;326:1097–1101. [PubMed: 1552910]
32.
Lu-Yao G L, McLerran D, Wasson J, Wennberg J E. An assessment of radical prostatectomy time trends, geographic variation and outcomes. JAMA. 1993;269:2633–2636. [PubMed: 8487445]
33.
Ganz P A. Age and gender as factors in cancer therapy. Clin Geriatr Med. 1993;9:145–155. [PubMed: 8443732]
34.
Michalski T A, Nattinger A B. The influence of black race and socioeconomic status on the use of breast-conserving surgery for Medicare beneficiaries. Cancer. 1997;79:314–319. [PubMed: 9010104]
35.
Ball J K, Elixhauser A. Treatment differences between blacks and whites with colorectal cancer. Med Care. 1996;34:970–984. [PubMed: 8792784]
36.
Burns R B, McCarthy E P, Freund K M. et al. Black women receive less mammography even with similar use of primary care. Ann Intern Med. 1996;125:173–182. [PubMed: 8686974]
37.
Klabunde C N, Potosky A L, Harlan L C, Kramer B S. Trends and black/white differences in treatment for nonmetastatic prostate cancer. Med Care. 1998;36:1337–1348. [PubMed: 9749657]
38.
Earle C C, Venditti L N, Neumann P J. et al. Who gets chemotherapy for metastatic lung cancer? Chest. 2000;117:1239–1246. [PubMed: 10807806]
39.
Bach P B, Cramer L D, Warren J L, Begg C B. Racial differences in the treatment of early-stage lung cancer. N Engl J Med. 1999;341:1198–1205. [PubMed: 10519898]
40.
Hislop T G, Olivotto I A, Coldman A J. et al. Variations in breast conservation surgery for women with axillary lymph node negative breast cancer in British Columbia. Can J Public Health. 1996;87:390–394. [PubMed: 9009395]
41.
Lantz P M, House J S, Lepkowski J M. et al. Socioeconomic factors, health behaviors, and mortality. JAMA. 1998;279:1703–1708. [PubMed: 9624022]
42.
Ayanian J Z, Kohler B A, Abe T, Epstein A M. The relation between health insurance coverage and clinical outcomes among women with breast cancer. N Engl J Med. 1993;329:326–321. [PubMed: 8321261]
43.
Bennett C L, Greenfield S, Aronow H. et al. Patterns of care related to prostate cancer. Cancer. 1991;67:2633–2641. [PubMed: 2015564]
44.
Schapira M M, McAuliffe T L, Nattinger A B. Treatment of localized prostate cancer in African-American compared to caucasian men: less use of aggressive therapy for comparable disease. Med Care. 1995;33:1079–1088. [PubMed: 7475418]
45.
Siminoff L A, Fetting J H, Abeloff M D. Doctor-patient communication about breast cancer adjuvant therapy. J Clin Oncol. 1989;7:1192–1200. [PubMed: 2671280]
46.
Moore M J, O’Sullivan B, Tannock I F. How expert physicians would wish to be treated if they had genitourinary cancer. J Clin Oncol. 1988;6:1736–1745. [PubMed: 3054004]
47.
Begg C B, Cramer L D, Hoskins W J, Brennan M F. Impact of hospital volume on operative mortality for major cancer surgery. JAMA. 1998;280:1747–1751. [PubMed: 9842949]
48.
Hillner B E, Smith T E, Desch C E. Hospital and physician volume or specialization and outcomes in cancer treatment: importance in quality of cancer care. J Clin Oncol. 2000;18:2327–2340. [PubMed: 10829054]
49.
Talcott J A, Rieker P, Clark J A. et al. Patient-reported symptoms after primary therapy for early prostate cancer: results of a prospective cohort study. J Clin Oncol. 1998;16:275–283. [PubMed: 9440753]
50.
Stanford J L, Feng Z, Hamilton A S. et al. Urinary and sexual function after radical prostatectomy for clinically localized prostate cancer. JAMA. 2000;283:354–360. [PubMed: 10647798]
51.
Talcott J A, Rieker P, Propert K J. et al. Patient-reported impotence and incontinence after nerve-sparing radical prostatectomy [see comments] J Natl Cancer Inst. 1997;89:1117–1123. [PubMed: 9262249]
52.
Walsh P C. Radical prostatectomy, preservation of sexual function, cancer control. The controversy. Urol Clin North Am. 1987;14:663–673. [PubMed: 3314061]
53.
Catalona W J, Bigg S W. Nerve-sparing radical prostatectomy: evaluation of results after 250 patients. J Urol. 1990;143:538–543. [PubMed: 2304166]
54.
Smith T J, Hillner B E, Desch C E. Efficacy and cost-effectiveness of cancer treatment: rational allocation of resources based on decision analysis. J Natl Cancer Inst. 1993;85:1460–74. [PubMed: 8360929]
55.
Weeks J C, Tierney M R, Weinstein M C. Cost effectiveness of prophylactic intravenous immunoglobulin in chronic lymphocytic leukemia. N Engl J Med. 1991;325:81–86. [PubMed: 1904989]
56.
Levine M N, Drummond M F, Labelle R J. Cost-effectiveness in the diagnosis and treatment of carcinoma of unknown primary origin. Can Med Assoc J. 1985;133:977–987. [PMC free article: PMC1346409] [PubMed: 3933808]
57.
Hillner B E, Weeks J C, Desch C E, Smith T E. Pamidronate in prevention of bone complications in metastatic breast cancer: a cost-effectiveness analysis. J Clin Oncol. 2000;18:72–79. [PubMed: 10623695]
58.
Hillner B E, Smith T J. Efficacy and cost effectiveness of adjuvant chemotherapy in women with node-negative breast cancer. N Engl J Med. 1991;324:160–168. [PubMed: 1898533]
59.
Lee S J, Anasetti C, Kuntz K. et al. The costs and cost-effectiveness of unrelated donor bone marrow transplantation for chronic phase chronic myelogenous leukemia. Blood. 1988;92:4047–4052. [PubMed: 9834208]
60.
Churchill D N, Lemon B C, Torrance G W. A cost-effectiveness analysis of continuous ambulatory peritoneal dialysis and hospital hemodialysis. Med Decis Making. 1984;4:439–500.
61.
Edelson J T, Weinstein M C, Tosteson A N A. et al. Long-term cost-effectiveness of various initial monotherapies for mild to moderate hypertension. JAMA. 1990;263:407–413. [PubMed: 2136759]
62.
Desch C E, Lasala M R, Smith T J, Hillner B E. The optimal timing of autologous bone marrow transplantation in Hodgkin’s disease after chemotherapy relapse. J Clin Oncol. 1992;10:200–209. [PubMed: 1531067]
63.
Smith T J, Hillner B E. The efficacy and cost-effectiveness of adjuvant therapy of early breast cancer in premenopausal women. J Clin Oncol. 1993;11:771–776. [PubMed: 8478670]
64.
Jaakkimainen L, Goodwin P J, Pater J. et al. Counting the costs of chemotherapy in a National Cancer Institute of Canada trial in nonsmall-cell lung cancer. J Clin Oncol. 1990;8:1301–1309. [PubMed: 2166142]
65.
Welch H G, Larson E B. Cost-effectiveness of bone marrow transplantation in acute nonlymphocytic leukemia. N Engl J Med. 1989;321:807–812. [PubMed: 2505077]
66.
Goodwin P J, Feld R, Evans W K, Pater J. Cost-effectiveness of cancer chemotherapy: an economic evaluation of a randomized trial in small-cell lung cancer. J Clin Oncol. 1988;6:1537–1547. [PubMed: 2845011]
67.
Ellwood P M. Shattuck lecture—outcomes management. A technology of patient experience. N Engl J Med. 1988;318:1549–1556. [PubMed: 3367968]
68.
Cook D J, Guyatt G H, Laupacis A, Sackett D L. Rules of evidence and clinical recommendations on the use of antithrombotic agents. Chest. 1992;102(4 Suppl):305S–311S. [PubMed: 1395818]
69.
General Accounting Office. Practice guidelines: the experience of medical specialty studies. GAO/PEMD-91-11. February 1991.
70.
Woolf S H. Practice guidelines, a new reality in medicine. Arch Intern Med. 1992;152:946–952. [PubMed: 1580720]
71.
Field MJ, Lohr KN. Clinical practice guidelines: directions for a new program. Committee to Advise the Public Health Service on Clinical and Practice Guidelines. Washington, D.C.: National Academy Press; 1992.
72.
Kapp M B. Cookbook” medicine. A legal perspective. Arch Intern Med. 1990;150:496–500. [PubMed: 2310271]
73.
Institute of Medicine. Setting priorities for clinical practice guidelines. Washington, D.C.: National Academy Press; 1995.
74.
American Society of Clinical Oncology. Recommendations for the use of hematopoietic colony-stimulating factors: evidence-based, clinical practice guidelines. J Clin Oncol. 1994;12:2471–2508. [PubMed: 7964965]
75.
American Society of Clinical Oncology. Update of recommendations for the use of hematopoietic colony-stimulating factors: evidence-based practice guidelines. J Clin Oncol. 1996;14:1957–1960. [PubMed: 8656266]
76.
American Society of Clinical Oncology. Clinical practice guidelines for the use of tumor markers in breast and colorectal cancer. J Clin Oncol. 1996;14:2843–2877. [PubMed: 8874347]
77.
American Society of Clinical Oncology. 1997 update of the recommendations for the use of tumor markers in breast and colorectal cancer. J Clin Oncol. 1998;16:793–795. [PubMed: 9469371]
78.
American Society of Clinical Oncology. Recommended breast cancer surveillance guidelines. J Clin Oncol. 1997;15:2149–2156. [PubMed: 9164230]
79.
Smith T E, Davidson N E, Schapira D V. et al. American Society of Clinical Oncology 1998 update of the recommended breast cancer surveillance guidelines. J Clin Oncol. 1999;17:1080–1082. [PubMed: 10071303]
80.
American Society of Clinical Oncology. Clinical practice guidelines for the treatment of unresectable non-small-cell lung cancer. J Clin Oncol. 1997;15:2996–3018. [PubMed: 9256144]
81.
Desch C E, Benson A B, Smith T E. et al. Recommended colorectal cancer surveillance guidelines by the American Society of Clinical Oncology. J Clin Oncol. 1999;17:1312–1321. [PubMed: 10561194]
82.
Winn R J, Brown N H. The NCCN guidelines development process. Oncology. 1999;13:28–32.
83.
Browman G P, Levine M N, Mohide E A. et al. The practice guideline development cycle: a conceptual tool for practice guidelines development and implementation. J Clin Oncol. 1995;13:502–512. [PubMed: 7844612]
© 2000, BC Decker Inc.
Bookshelf ID: NBK20918

Views

  • PubReader
  • Print View
  • Cite this Page

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...