NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Institute of Medicine (US) Roundtable on Evidence-Based Medicine; Yong PL, Saunders RS, Olsen LA, editors. The Healthcare Imperative: Lowering Costs and Improving Outcomes: Workshop Series Summary. Washington (DC): National Academies Press (US); 2010.

Cover of The Healthcare Imperative

The Healthcare Imperative: Lowering Costs and Improving Outcomes: Workshop Series Summary.

Show details

8Knowledge Enhancement


While medical insights emerging from ongoing clinical care and research in emerging areas such as proteomics and genomics have contributed to a rapidly expanding pool of information about health and human disease, the sheer volume of data has overwhelmed the cognitive capacity of humans (Stead, 2007). Yet the ability to apply the information emerging from scientific journal publications, research studies, specialty society guidelines, and recommendations from health agencies like the Centers for Disease Control and Prevention (CDC) at the point of care is critical to helping patients and their clinicians decide on the best care options to pursue at any moment in time. In the American Recovery and Reinvestment Act of 2009, significant resources have been devoted to both developing the knowledge needed to assist patients, providers, payers, and purchasers in their decision-making processes, and to creating the infrastructure needed to improve the quality of care delivered. Expanding on these investments in comparative effectiveness research and health information technology, speakers in this session focus on the essential strategies to enable more efficient generation and application of knowledge during the care process. In particular, they highlight tools for generating high quality, consistent treatment, with a focus on the medically complex; timely, independent, and understandable evidence; reliable, sharable, and secure clinical records; protected but accessible data; and patient-centered care.

Highlighting the potential for evidence-based guidelines to assist clinicians in applying state-of-the-art knowledge to clinical care, Lucy A. Savitz of Intermountain Healthcare discusses how evidence-based care process models (CPMs) have enabled improvements in care quality with simultaneous cost savings. Savitz illustrates the success of this strategy by discussing outcomes with implementation of the care of febrile infants evidence-based CPM, explaining that infant stays have dropped to an average of 36 hours from 69 hours previously, readmissions have decreased, and adverse events, including preventable bacterial infections, have fallen significantly. While suggesting that Intermountain’s protocols could be adopted across different models of care delivery, she additionally discusses the larger challenge of sustainability of savings beyond initial implementation.

Rainu Kaushal of Weill-Cornell Medical College posits that electronic health records (EHRs) are defined very differently across the country and that looking at the benefits and costs of interventions involving EHRs necessitates building a common language. However, she asserts that EHRs possess many benefits, including connecting physicians and other health-care providers in the interest of furthering quality care. Citing some recent studies on EHRs, Kaushal reviews estimates of cost savings from implementation of the various components of EHRs that range up to $77 billion annually and projects even greater savings from long-term chronic disease prevention and management. However, she underscores that the estimates described are restrained by the limited availability of primary data and consequent heavy reliance on expert estimates. She also suggests that the critical cofactors needed for successful implementation and use of EHRs include financial support, technical support (i.e., regional extension center services), and refinement of standards.

Carolyn M. Clancy of the Agency for Healthcare Research and Quality (AHRQ) outlines the work of AHRQ in advancing comparative effectiveness research (CER). She describes CER as a powerful tool in providing the information needed to drive improvement in clinical care. Not only can it assist clinicians and patients in deciding on the best care option at a particular time, but, with the translation of research findings into practical tools, CER additionally promises to address many inequities in health care. She concludes by describing AHRQ’s goals of (1) ensuring that effectiveness data are more widely used, and (2) promoting an open and collaborative approach to comparative effectiveness, which have been facilitated with the support of $300 million in federal dollars dedicated to AHRQ specifically for CER.

Peter K. Smith of Duke University describes the importance of enhancing clinical data as a knowledge utility. Employing the metaphor of the Christmas tree to describe medical records today, he compares current medical records to a tree riddled with a multitude of ornamental information in apparent disarray. Smith suggests that today’s medical record is less a knowledge utility to guide practice and more a tool for controlling malpractice liability and for driving proper billing and payment. Shedding light on the 9,000 fee schedule codes necessary for physician payment, he expresses the view that the goals of the expansive clinical regulatory requirements may well be misaligned and possibly contrary to effective healthcare delivery.


Lucy A. Savitz, Ph.D., M.B.A.

Intermountain Healthcare1

Intermountain Healthcare is a nonprofit health system based in Salt Lake City, Utah. Serving the healthcare needs of Utah and southeastern Idaho residents, Intermountain’s system of 21 hospitals, more than 700 physicians, 130 clinics, and an owned health plan (Select Health) provides clinically excellent medical care at affordable rates across the full rural-urban continuum. The system has been recognized as a national leader in high-performance healthcare delivery (Bohmer, 2009; Staines, 2009).

This paper leverages the experience of Intermountain Healthcare to provide background and examples of evidence-based CPMs that have reduced costs while maintaining and improving quality of care for our patients.

The Problem

Intermountain’s clinical investigators have long recognized the need to reduce variation in compliance with evidence-based guidelines together with making these guidelines adequately explicit. A guideline is a systematic statement of policy rules or principles, representing state-of-the-art knowledge, that often direct a clinician in where to go but do not necessarily specify how to get there. Conversely, protocols or CPMs are precise and detailed plans for the study of a medical problem and/or for a regimen of therapy, indicating how to get there. An adequately explicit protocol or CPM provides enough detail to lead different clinicians to the same patient-specific decision via a reproducible clinical decision method. Clinical decision support tools can then include all ways in which healthcare knowledge is represented in health information systems. The advantages of evidence-based CPMs are that they:

  • Provide readily accessible references and allow access to knowledge in guidelines that have been selected for use in a specific clinical context,
  • Often improve the clarity of a guideline,
  • Can be tailored to a patient’s clinical state, and
  • Propose timely decision support that is specific for the patient.

Key components of our strategy for developing such protocols or CPMs are to:

  • Identify the problem;
  • Establish the evidence base; and
  • Develop, test, and implement using quality improvement tools (e.g., Six Sigma—define, measure, analyze, improve, control—Plan, Do, Study, Act).

Evidence-Based Care Process Model

The development of evidence-based CPMs at Intermountain Healthcare is anchored in our clinical programs—primary care, pediatrics, women and newborns, intensive medicine, cardiovascular, surgical services, oncology, and behavioral health. Clinical programs are staffed with a medical director, nurse administrator, statistician, and support team that includes information technology and finance personnel. Clinical program workgroups identify problems and work to develop, test, and implement evidence-based CPMs in a phased approach as warranted. Bohmer (2009) provides a detailed description of clinical programs and the role of the Institute for Health Care Delivery.2

Five example Intermountain Healthcare evidence-based CPMs are provided in Table 8-1 together with indication of clinical program, cost drivers impacted, observed cost savings, and scope. While many more CPMs have been implemented across our clinical programs,3 this sample demonstrates that the common cost drivers targeted are reduced length of stay, readmissions, and emergency room (ER) visits. Unfortunately, given perversities in our current reimbursement system, it will also be necessary to incentivize nonintegrated systems of care to coordinate and minimize financial penalties when achieving such cost savings, shifting care across the continuum from inpatient to outpatient service.

TABLE 8-1. Example Evidence-Based Care Process Models at Intermountain Healthcare.


Example Evidence-Based Care Process Models at Intermountain Healthcare.

Carrie Byington’s work provides a specific example in terms of the care of the febrile infant evidence-based CPM. Byington practices at Inter-mountain’s Primary Children’s Medical Center and is a clinical leader in our Pediatric Clinical Program and a Professor of Pediatrics and Infectious Disease at the University of Utah. Impetus for Byington’s problem identification (Byington et al., 2004) in addressing care of the febrile infant is attributable to knowledge that:

  • Expert guidelines were published in 1993 to address the issues of diagnostic testing and hospitalization for febrile infants;
  • Existing guidelines, developed before 1990, provide no information regarding viral diagnostic testing or management of infants with confirmed viral illness who make up the majority of the group;
  • Our ability to rapidly diagnose viral illness has changed significantly; and
  • Physicians need guidance regarding the appropriateness of viral diagnostic tests and implications of positive/negative test results on risk for serious bacterial infection.

The University of Utah/Intermountain evidence-based CPM was developed using an evidence base derived from prospective research together with a Six Sigma process. We further were aware that fever in infants 1 to 90 days of age is one of the most common reasons for medical encounters (i.e., 20 percent of physician visits and 58 percent of all ER visits at Primary Children’s Medical Center). Fever of 38°C or higher is associated with serious bacterial infection—bacteremia, meningitis, and urinary tract infection, with the latter being the most common serious bacterial infection. We documented that only 49 percent of febrile infants managed in hospital-based outpatient facilities had both a complete blood count and a urine analysis, as recommended by guidelines. This understanding of the problem led Byington’s team to conduct the following analyses:

  • Reanalyze Rochester Criteria and risk for serious bacterial infection.
  • Analyze age and risk for serious bacterial infection.
  • Analyze viral diagnostic testing and risk for serious bacterial infection.
  • Analyze complete blood count and urine analysis as predictors for serious bacterial infection.
  • Analyze missed serious bacterial infection.

Sixteen peer-reviewed publications document this foundational work to develop and guide clinicians in an adequately explicit protocol that incorporates state-of-the-art medical knowledge and more newly developed laboratory testing capabilities. From this, the work in developing the evidence base led to the development, testing, and spread of a CPM and standing orders involving complete blood count and urine analysis of febrile infants (approximately $60/patient tested), which seems minor relative to the estimated cost of $1 million for a missed case of meningitis. Key quality measures used to monitor the process and clinical compliance with the CPM include the following:

  • Receive core laboratory tests and viral testing as indicated.
  • Admit patients at high risk for serious bacterial infection as indicated by CPM threshold.
  • Give appropriate antibiotics per CPM.
  • Stop antibiotics within 36 hours for febrile admission with bacterial negative cultures.
  • Length of stay must be 42 hours or less.

Appropriate evaluation in 100 percent of infants is our goal. Figure 8-1 presents change in the median length of stay observed across four facilities over time, documenting achieved decreases in length-of-stay variation. These data depict replication in adult ER settings outside the children’s hospital setting where the evidence-based CPM was developed. Targeted cost drivers include: patient volume, prolonged/unnecessary length of stay, reduced repeat ER visits and readmissions; decreased morbidity and mortality; and minimized family burden. The overall effect on costs of the care of the febrile infant CPM includes:

FIGURE 8-1. Median length of stay (LOS) for febrile infant admissions with negative cultures by admission year.


Median length of stay (LOS) for febrile infant admissions with negative cultures by admission year.

  • Getting infants from a 3-day (mean 69-hour stay) to a 36-hour stay, this saves $3,000/infant (note that societal vs. payer savings would be much larger; also this represents a loss in hospital revenues given reimbursement perversity);
  • Ten percent or less of infants develop a fever in the first 90 days;
  • Decreased readmission; and
  • Less adverse events, including preventable bacterial infection.

The care of the febrile infant evidence-based CPM has been tested at four Intermountain Healthcare facilities with similar quality and cost results. Byington and the lead author have been awarded a grant to expand this evidence-based CPM and document the cost effectiveness across all Intermountain Healthcare facilities. Further, this evidence-based CPM was adopted by the American Board of Pediatrics for maintenance and certification requirements.


Evidence-based health care does not spread automatically (Dopson and Fitzgerald, 2005); diffusion of such innovations will require national attention, training, and perhaps national priority setting. As described by Bohmer (2009) and Staines (2009), Intermountain Healthcare provides:

  • Supportive infrastructure and culture for improvement;
  • Commitment from leadership; and
  • Necessary staff training, education, and feedback.

We have also documented the value of clinical decision support tools to accommodate use of evidence-based CPMs across our clinical programs.

The question at hand is how much of the Intermountain Healthcare savings can be realized by adopting clinics, hospitals, and health systems. As shown by the Institute for Healthcare Improvement in its 100,000 Lives Campaign (Wachter and Pronovost, 2006), improvements can be realized across disparate systems of care when there is a focused effort; we have also seen others show improvements from adopted models with grant funding. Alan Morris has reported replicable results across disparate care settings for his eProtocol-insulin (i.e., another example)—at the LDS Hospital, National University Hospital in Singapore, University of Virginia, and Baystate Health (Morris et al., 2008). The real, unanswered question is how sustainable these cost savings are beyond the focused implementation initiative and/or grant award.

Potentially Achievable Results

There are several tangible results that can be potentially achieved via recognition of the evidence-based CPM strategy. These include

  • Widespread adoption of febrile infant evidence-based CPM at Intermountain Healthcare and beyond;
  • Demonstrated value of evidence-based CPMs leading to application/modeling in other areas;
  • Electronic medical record vendors building capacity to integrate clinical decision support for evidence-based CPMs;
  • Useful outcomes comparisons supported through collaboratives or focused national efforts led by government agencies such as the Centers for Medicare & Medicaid Services (CMS) or AHRQ; and
  • Documentation of significant national cost savings.

Policy Options

Several policy options follow from consideration of the effect of evidence-based CPMs as a strategy to reduce costs while maintaining and improving the quality of care. These include requiring research translation to include evidence-based CPMs based on comparative effectiveness research results; providing resources to create a clearinghouse for evidence-based CPMs that can be implemented in electronic medical records (minimizing waste and duplicated efforts and encouraging modeling); and eliminating reimbursement perversities in the system via payment reform and/or the provision of financial incentives.


Rainu Kaushal, M.D., M.P.H., and Lisa M. Kern, M.D.

Weill-Cornell Medical College

Health care in the United States today is plagued by high costs, substantial fragmentation, and poor quality. Communities spending the most money on health care may actually have the lowest quality of care (Gawande, 2009). At the same time, the communication burden on individual physicians is increasing; for example, an individual physician must interact with 99 other physicians in 53 different practices for every 100 Medicare patients (Pham et al., 2009).

EHRs are a potentially powerful tool to address economic, quality, communication, and structural delivery issues in health care today. The National Alliance for Health Information Technology defined an EHR in 2008 as an

Electronic record of health-related information on an individual that conforms to nationally recognized interoperability standards and that can be created, managed, and consulted by authorized clinicians across more than one healthcare organization. (National Alliance for Health Information Technology, 2008)

An important aspect of this definition is interoperability, which is the electronic exchange of information across multiple healthcare providers, thereby improving information access with improvements in efficiency and quality, as well as resultant decreases in costs and fragmentation of care. However, it should be noted that the inclusion of interoperability was not consistently incorporated into definitions of EHRs prior to this. In addition, EHRs have clinical decision support, which are electronic alerts and reminders provided to a physician at the point of care to improve medical decision making. Most EHRs include applications for ordering medications and tests, referred to as computerized physician order entry (CPOE) in the inpatient setting or e-prescribing in the outpatient setting. EHRs frequently incorporate additional support to improve the accuracy of billing and coding.

EHRs target several large healthcare cost drivers, including preventive care delivery, chronic care management, transitions of care, medications, radiology testing, and laboratory testing. In each of these areas, EHRs can improve the efficiency, quality, and/or safety of care. Furthermore, EHRs can facilitate clinical data access and retrieval for quality reporting, public health surveillance, and research. Finally, EHRs can be critical methods of restructuring healthcare delivery, as for example, through the medical home model.

Savings from Use of Electronic Health Records

National Estimates

There is an increasing amount of data regarding potential national and local savings from the use of EHRs. In 2005, both Walker and colleagues from the Center for IT Leadership and Hillestad and colleagues from RAND came to similar estimates of cost impacts on widespread adoption of EHRs despite different assumptions and methodologies. Assuming universal adoption over 5 years, Walker estimated $77 billion annual savings from implementation and use of interoperable EHRs (Walker et al., 2005). Hillestad also estimated $77 billion in annual savings, assuming 90 percent nationwide adoption of interoperable EHRs (Hillestad et al., 2005). Both studies relied on a review of expert opinions and the available published literature. Hillestad included further savings projections, estimating an additional $147 billion savings per year from long-term chronic disease prevention and management. Over 15 years, cumulative net hospital efficiency and safety savings could be $371 billion, and physician practice savings could be $142 billion. The net savings estimates could double with the inclusion of chronic disease prevention and management.

However, a Congressional Budget Office (CBO) report released in May 2008, titled Evidence on the Costs and Benefits of Health Information Technology, challenged both the Center for Information Technology Leadership and RAND studies as guides for legislative proposals (CBO, 2008). The CBO stated that both studies estimated “potential” rather than “likely” impact and overrelied on studies demonstrating positive effects. They also felt that several specific assumptions were overly optimistic in each study.

Most recently, in June 2009, the Commonwealth Fund issued a report authored by Nuzum and colleagues summarizing financial benefits of various healthcare reform policies by presenting estimates from the CBO and the Lewin Group (Nuzum et al., 2009). The CBO estimated that the Health Information Technology for Economic and Clinical Health (HITECH) Act, which is part of the 2009 American Recovery and Reinvestment Act, will expend $20 billion to achieve 70 percent EHR adoption for hospitals and 90 percent for physicians and will save Medicare and Medicaid $13 billion over 10 years (CBO, 2009). In an earlier estimate in 2008, the CBO had estimated $4 billion in federal savings over 10 years through a bonus (for the first 5 years)/penalty (for the next 5 years) system for using/not using EHRs, and $61 billion in savings over 10 years if EHR use were required for Medicare participation (CBO, 2008).

The Lewin group report suggested $70 billion in federal savings and $180 billion in total health system savings over 10 years through requiring electronic reporting of key health outcomes to qualify for payment updates, thereby achieving a predicted 96 percent use of EHRs.

These reports suggest very significant savings opportunities from interoperable EHRs nationally. The savings may be even more significant when measurements of other effects of EHRs are included. Notably, the savings are dependent both on the widespread adoption of EHRs and the effective incorporation of interoperability.

Local and Regional Estimates

In addition to these national estimates, there have been some setting-specific estimates. In 2003, Wang and colleagues used a simulation model to estimate the costs and benefits of EHR adoption in a generic primary care setting (Wang et al., 2003). The model relied on data from the authors’ institution and literature reviews, although data were somewhat limited in 2003. They estimated a net benefit over 5 years of $86,400 per provider, driven by reductions in paper chart pulls and transcription, reduced adverse drug events, more economically efficient prescribing, reduced laboratory and radiology ordering, and increased revenue or reduced losses brought about by improved billing accuracy.

In 2004, Barlow and colleagues used a pre–post study design to examine the economic effect of implementing an EHR in a multicenter, multispecialist ambulatory care organization in central Utah (Barlow et al., 2004). Their study included the savings achieved by more efficient records management and billing. However, it is not clear what, if any costs, were factored into the analysis. They estimated savings of $8.2 million over 5 years for a 59-physician multispecialty group practice.

In 2005, Miller and colleagues published an in-depth case series (retrospective pre–post design) examining the experience of EHR adoption in the context of solo or small group practices (Miller et al., 2005). The authors included comprehensive lists of relevant costs and benefits and collected their data through extensive interviews and direct observation. They concluded that practices generally recover their EHR costs in 2.5 years and then accrue approximately $33,000 in savings per provider annually.

Finally, in 2007 Grieger and colleagues used a pre–post study to evaluate the implementation of an EHR in six ambulatory care practices affiliated with an academic medical center in Rochester, New York (Grieger et al., 2007). The largest component of savings they estimated was through reduced chart pulls and reduced staffing needs. They found recapture of initial costs within 16 months and estimated ongoing annual savings of $10,000 per provider.

Other studies have focused on specific aspects of EHRs, such as CPOE or e-prescribing. Kaushal and colleagues, using published studies and institutional expert opinions, estimated financial savings of $950,000 annually for a tertiary academic hospital through the use of CPOE (Kaushal et al., 2006). The CPOE system elements that resulted in the greatest savings were renal dosing guidance, nursing time use, specific drug guidance, and adverse drug event prevention. A Massachusetts-based report suggested that the average community hospital accrued annual savings of $2.7 million from a CPOE system with robust clinical decision support (Massachusetts Technology Collaborative & New England Healthcare Institute, 2009). Finally, Fischer and colleagues examined changes in prescribing behavior and savings resulting from the use of a handheld e-prescribing device with formulary decision support. They suggested savings of $845,000 per 100,000 patients each year through improved formulary compliance (Fischer et al., 2008).

Challenges and Limitations of Estimates

These setting-specific studies, and to a lesser degree the national estimates, highlight the significant variations among and limitations of the efforts to quantify the impact of EHRs. Among the confounding factors in any analysis of this body of literature are the heterogeneity of assumptions along several domains, including the perspective taken (e.g., federal government or an individual provider); unit of analysis; study design; time horizon; characterization of the EHR (e.g., included applications, implementation versus actual use, and level of decision support); and characterization of interoperability (e.g., amount of coded data, architecture, and external data sources). Furthermore, important contextual issues, such as payment systems, are rarely addressed.

The difficulties are compounded by limited availability of primary data and consequently a heavy reliance on expert estimates. Because successful and broad-based adoption and use of EHRs is difficult—requiring extensive capital, workflow redesign, and technical support—we have a dearth of primary data to drive cost and savings estimates. Achieving interoperability may be even harder as few successful community-wide data exchanges exist nationally.


Currently, only 7.6 percent of hospitals and 13 percent of office practices have EHRs (DesRoches et al., 2008; Jha et al., 2009). Fortunately, those rates may soon change for the better. There are significant national investments in interoperable EHRs underway. The American Recovery and Reinvestment Act stipulates a minimum investment of $19 billion (Steinbrook, 2009; U.S. House of Representatives and U.S. Senate, 2009). Kaiser invested $3 billion in KP Health Connect (Garrido et al., 2004). In New York State, the HEAL NY program is investing $250 million with significant private matching funds, and other states are also making substantial investments (New York State Department of Health, 2006).

As above, implementation of EHRs and the realization of interoperability are challenging, consistent with the low adoption rates today. Some critical cofactors for successful implementation and use of EHRs include financial support, technical support (i.e., regional extension center services), and refinement of standards. Supporting interoperability is a significant task unto itself; while many existing EHRs have the technical capacity to be interoperable, very few are actively exchanging data. Successful interoperability will depend on further development of state and federal policies, including those focused on privacy and security, development of community-wide governance for health information exchange, and technical development by vendors.


Carolyn M. Clancy, M.D.

Agency for Healthcare Research and Quality

We are in a climate of growing concern about healthcare spending—spending that already exceeds $2 trillion per year and is projected to keep growing at alarming rates (Elmendorf, 2009). Some of the major drivers of this spending have been discussed already in compendium, and among them include the wide variation in clinical care, uncertainty about best practices, and pervasive issues of quality, safety and equity. While striving to provide the right treatment for the right patient at the right time, the potential impact of comparative effectiveness research (CER) translates into the need for practical tools for patients and physicians alike that can inform the decision-making process.

Which Is Better?

The application of CER findings has inspired concerns but has not always been well defined in the national discussion. The question of its role is in many ways like the basic question for baseball enthusiasts of “Which is better—an aluminum bat or a wooden bat?” The best answer to that question is another question: “Better for what?” as aluminum bats are often better for younger children, while professional players tend to use wooden bats. This playful sports analogy illustrates the point of CER: the findings have less often declared affirmatively that one option is superior to another. More often than not, the answer is more nuanced, just as clinical decision making is complex.

The Role of AHRQ

AHRQ has emerged as a frontrunner in defining the role of CER and in framing the debate in a different light. Our mission is to improve the quality, safety, efficiency, and effectiveness of health care for all Americans. Over the past few years, we have worked effectively with congressional mandates to forward this mission with growing support from the national healthcare community. Our work has included patient safety and health information technology, among others.

As we look at CER, the landscape we find is quite ad hoc. It is a well-intentioned, albeit uncoordinated, effort that is hindered both by limited capacity to do research and by even less capacity to translate the research into meaningful, usable applications. It is here where we would argue that the opportunity for CER has fallen most tragically short.

AHRQ has been very encouraged by the Institute of Medicine’s (IOM’s) recent elaboration of CER as a public good. The notion is a powerful one for the healthcare discussion. CER as a public good provides healthcare decision makers—patients, clinicians, purchasers, and policy makers—access to the latest, unbiased evidence-based information about treatment options. And, as such, it helps inform choices and, where possible, is closely aligned with the sequence of decisions patients and clinicians face.

AHRQ is one of three major partners identified by the American Recovery and Reinvestment Act of 2009 to invest in the development of CER. In addition to $800 million for the National Institutes of Health and for the Secretary of Health and Human Services, AHRQ receives $300 million to support its efforts.

The investment in CER by the American Recovery and Reinvestment Act is a significant down payment on the kind of scientific infrastructure we need to develop and sustain a healthcare system that is information rich and patient focused. In late June 2009, both the IOM and the Federal Coordinating Council independently defined the CER agenda as broader than the specific focus on clinical interventions. Instead, and again this is where AHRQ has focused its attention on CER, the agenda was one that included the context and the care delivery interventions necessary to make it feasible for patients and clinicians to use information to drive decisions.

AHRQ envisions CER as providing the information needed to drive improvement in clinical care by:

  • Providing information that can be used on the front lines of treatment;
  • Helping to make decisions more consistent, transparent, and rational;
  • Ensuring the effectiveness data is more widely used; and
  • Promoting an open and collaborative approach to comparative effectiveness.

Building Scientific Infrastructure to Support Reform

We believe AHRQ’s role in defining CER is building the scientific infrastructure needed for health reform. AHRQ has operationalized the Effective Healthcare Program into three major areas in order to build this foundation: evidence synthesis, evidence generation, and evidence communication and translation.

In the first area, we have been working on systematically reviewing, synthesizing, and comparing existing evidence on treatment effectiveness. In the second, we have sought to close some of the knowledge gaps in the research and to accelerate practical studies. We have worked closely with partners such as the DECIDE network and the Centers for Education & Research on Therapeutics to generate evidence from large clinical databases.

But the most exciting area, and perhaps the most relevant to the broader definition of CER, is our work with the John M. Eisenberg Center for Clinical Decisions and Communications Science. Here, we have been aggressively addressing the challenges of communicating scientific information in plain language to policy makers, patients, and their families. While there have certainly been “patient guides” for many years, available resources are often better resources for doctors who are looking for quick information outside of their specialties than they are accessible resources for patients. These past efforts have not provided patients with the tools to translate information and with a framework to look at the information that supports informed decision making. Faced with an array of options with different benefits and harms, how does the patient think about what is right for him or her? AHRQ continues to work on developing these types of consumer guides for patients and tackling the question of dissemination that broadens the scope of influence beyond reliance on our formidable partners, like the National Business Group on Health, American Association of Retired Persons (AARP), and the Consumers Union, among others.

Consistent with our focus, we will be investing the $300 million appropriated to AHRQ in the American Recovery and Reinvestment Act in evidence synthesis and generation, evidence communication and translation, and continued support for methods, training, and data development.

  • Evidence synthesis and generation We have had standing announcements to researchers for career development and other types of training awards in order to build research capacity. Additionally, we are investing in large, pragmatic prospective studies that include underrepresented populations in significant numbers as an effort to address systematic reviews that have relied on studies that, despite federal policies for inclusion, have samples of minorities too small to yield any definitive findings relevant to those populations.
  • Evidence communication and translation We will continue our commitment to this work and increase our investment to innovative broad dissemination and translation.
  • Continued support for methods, training, and data development The unprecedented investment in CER provided by the Recovery Act coincides with an unparalleled public investment in health information technology. The latter will accelerate broad adoption of electronic health records and other applications by practitioners, hospitals, and others. In addition, broad adoption can help transform care delivery to a platform for discovery and rapid translation of scientific findings. In short, these investments set the stage for achieving the Learning Health System envisioned by the IOM Roundtable on Value & Science-Driven Health Care—a vision integral to achieving high quality, affordable care for all Americans.


In the end, CER is an essential tool in a much larger toolkit; CER is necessary but not sufficient. CER is descriptive rather than prescriptive: it does not make policy, and it does not make healthcare decisions. What it can do—and now more than ever there is a coalition around this broader purpose—is to weigh the evidence and present it in ways that help patients, clinicians, and other stakeholders make the best possible decisions about their healthcare choices. In doing so, CER can also help us identify what is known and not known (e.g., where the gaps in our research base are) and help direct attention to shifting the landscape from ad hoc environment to a coordinated and focused effort on driving and improving healthcare choices at the individual patient–doctor level.


Peter K. Smith, M.D.

Duke University

Clinical data are critical to the maintenance and improvement of health for individual patients and to the advancement of our systematic understanding of the treatment of acute and chronic diseases. Yet the medical records that hold the promise for centralizing all of these clinical data fall short of the task, instead bringing together excess information and obscuring other critical information that impedes better care and better research.

The current medical record can be likened to a Christmas tree with the data elements being ornaments. Our current “Christmas tree” is trimmed with a vast excess of ornaments placed in disarray, with duplication in many areas and obvious critical gaps. This is the result of the uncontrolled growth of clinical data demand without reference to clinical data utility, with too many stakeholders who are not aligned with the objective of relevant knowledge creation. Put another way, clinical data are not easily converted into knowledge to guide therapy for individuals and to evaluate the effects of new treatment options.

The clinical data in the medical record include demographics and patient identifiers; history and symptoms; lab and test results; medications; and ongoing observations and records of treatment. These data can then be translated at the clinical level into knowledge that supports the care of each patient by each doctor, as well as the many handoffs between healthcare professionals that may occur over time. The medical record is a knowledge utility, but in order for it to best perform in that regard, we need comprehensive restructuring of our clinical data collection (Fiesta, 1993; Joint Commission, 2008).

Three major opportunities for this comprehensive restructuring lie in physician documentation requirements for physician payment; nursing documentation related to hospital and nursing liability claims; and nursing documentation requirements for home healthcare payment. Major change in the data collection in these areas (documentation requirements) would not only improve healthcare delivery, but also add to our systemic knowledge while reducing healthcare costs.

Physician Documentation Requirements for Payment

There are over 9,000 Current Procedural Terminology (CPT) codes that describe the physician fee schedule, which defines physician work, practice, and liability reimbursement. Looking at one evaluation and management code, 99223, reveals just how complex and onerous these codes can be at the clinical level. Code 99223 describes the highest level initial inpatient visit and was performed 5,696,413 times for Medicare beneficiaries in 2007 (AMA, 2008).

To be paid, this visit must be composed of a comprehensive history, physical exam, and review of systems and also include high-level medical decision making (AMA, 2009). Specifically, 9 of 14 elements must be included in the physical exam. Furthermore, each of those nine elements must have at least two specific points addressed. In the case of the cardiovascular system, there are at least seven possible bullet points from which to choose. These include palpation of the heart (location, size, thrills), auscultation of the heart with notation of abnormal sounds and murmurs, and assessment of lower extremities for edema and/or varicosities.

All in all, 18 clinical elements must be recorded, and each may contain many further individual data elements beyond the two levels described above. When considering the entire service, more than 100 identifiable clinical data elements are required. And, there is no absolute requirement that the chosen elements be related to the medical necessity diagnosis. Indeed, there are no requirements that the recorded information be relevant in either a positive or negative sense to the patient or an identified problem.

For this code, the physician is allotted 20 minutes to document the clinical data, coordinate care with other providers, and counsel the patient. The utility of these data are therefore limited by a variety of inherent problems, even if entered into an electronic record. The likelihood that these data will be accurate, relevant, and comprehensive is vanishingly small. Consequently, conversion of these data into knowledge that can be transmitted to other caregivers is also limited.

The drivers of these limitations and of this incredibly complicated system governing a physician visit are inherent in the design of the system. The evaluation and management database is designed to ensure that a sufficient amount of physician work is performed to justify payment and to allow auditing to result in payment denial, payment reduction, or prosecution for fraud. Clinical relevance and utility are secondary and dispensable concerns rather than goals. The onerous nature of the documentation requirements has led to the creation of electronic systems designed to meet these requirements and to the aggregation of meaningless, irrelevant, and inaccurate clinical data.

Nursing Documentation to Reduce Vulnerability Liability Claims

“If it is not in the medical record, it didn’t happen.” This quotation is a major driving force determining the clinical data elements recorded by our nation’s nursing workforce (Joint Commission, 2005). Unfortunately, it seems a foregone conclusion that the philosophy underlying this statement will continue in the next phase of healthcare reform. Our tort system will continue to distort healthcare delivery and significantly impair the collection of clinical data and the creation of medical knowledge (Iglehart, 2004). The medical record will serve more as an instrument for post hoc adjudication rather than as a prospective knowledge utility.

Our nurses have become scribes in a clinical data system whose burden is stultifying and whose only objective is to record everything that happened and everything that did not happen. As a result, the information in the hospital record is frequently unincorporated into care transitions, and even less so relied upon by physicians to augment their understanding of the patient’s response to therapy (Miller and Miller, 2007). A review of the hospital documentation for an 8-hour nursing shift for relevant information takes at least 30 minutes, much of which is spent parsing an electronically generated boilerplate for any real care information.

Numerous studies have shown that nursing documentation is a large component of the working time of the average nurse, that the proportion of time is increasing, and that the information recorded is poorly utilized (Brooks, 1998; Korst et al., 2003; Pabst et al., 1996; Smeltzer et al., 1996; Trossman, 2001; Upenicks, 1998; Urden, 1997). The most recently available information indicates that 35 percent of nursing work hours are devoted to documentation and only 7 percent to patient assessment and determining vital signs (Hendrich et al., 2008) (Figure 8-2). Documentation is expensive, costing an estimated $146 billion annually in the United States (Table 8-2). A 60 percent reduction in documentation time would result in sufficient savings to pay for all physician services to Medicare beneficiaries. Or, of course, we could reinvest that time in increased patient assessment and care for our patients, who currently experience only a fraction of the benefit that could be provided if nurses could more fully apply their experience and training (Blachly and Young, 1998; Brunt et al., 1999; Hendrich et al., 2009; IOM, 2004a, 2004b; LaDuke, 2001).

FIGURE 8-2. Distribution of nursing work time per shift (percentage, minutes).


Distribution of nursing work time per shift (percentage, minutes).

TABLE 8-2. Annual U.S. Healthcare Spending.


Annual U.S. Healthcare Spending.

Nursing Documentation to Enable Home Healthcare Payments

Another area where the administrative burden of data collection and codification may have significant impact on cost and quality is in home health care with OASIS data. On August 4, 1999, Centers for Medicare & Medicaid Services (CMS) (then the Health Care Financial Administration [HCFA]) mandated “the use, collection, encoding, and transmission of Outcome and Assessment Information Set (OASIS) for home health agencies” (HCFA, 1999). OASIS data must be collected and submitted on admission, resumption of care after an inpatient stay, recertification every 60 days, transfer, and at discharge. In its most recent rendition, now proposed for enhancement, there is a 175-page date specification file describing the 375 required data lines (OASIS data specifications, n.d.).

For example, nurses are asked to assess what level of cognitive function their patients exhibit (Data Element M1700), whether and how patients exhibit confusion (Data Element M1710), and whether and how they exhibit anxiety (Data Element M1720). While, these data are clinically important and relevant, they are also intrusive (Moffit, 1999; Moffit et al., 1999), as well as nondiagnostic. Furthermore, they represent less than 1 percent of the required data in OASIS, but a patient and family interview by a skilled healthcare provider would be required to answer them.

Indeed, the process is so complex and detailed that an industry has evolved to train users to complete this process to their advantage. The national estimate for training, assessment data collection, and training new staff is over 15,000,000 hours for 2009.4


These three examples all represent failings of the bureaucratic systems in health care to remain focused on patient care. Instead, the goals are aligned more with preventing fraud and reducing liability than achieving better health outcomes and increasing patient satisfaction. The failings discussed here could be addressed, in large measure, by a uniform patient problem list and a national patient identifier. Any change in our healthcare system is doomed to failure unless the physician and nursing problem lists are merged, codified, and placed in a national repository for each patient who is in turn identified unequivocally. Longitudinal reference to these clinical data is the key to understanding the safety and efficacy of our healthcare system. Furthermore, a major reduction in the number of clinical data elements collected is in order, with the requirements tested for clinical relevance and research relevance. All three examples share the common theme that the goals of our expansive bureaucratic regulation are at best misaligned and at worst contrary to effective healthcare delivery. Addressing these problems means the difference between delivering affordable and high-quality health care and continued and escalating failure.


  1. AMA (American Medical Association) Relative Value Update Committee Database, 2009. Ver. 2. Chicago, IL: AMA; 2008.
  2. AMA (American Medical Association) CPT assistant archives 1990–2008. Chicago, IL: AMA; 2009.
  3. Barlow S, Johnson J, Steck J. The economic effect of implementing an EMR in an outpatient clinical setting. Journal of Healthcare Information Management. 2004;18(1):46–51. [PubMed: 14971079]
  4. Blachly B, Young HM. Reducing the burden of paperwork. Journal of Gerontological Nursing. 1998;25(6):16–20. [PubMed: 9709065]
  5. Bohmer R. Designing Care, Aligning the Nature and Management of Health Care. Boston: Harvard Business Press; 2009.
  6. Brooks J. An Analysis of Nursing Documentation as a Reflection of Actual Nurse Work. MEDSURG Nursing. 1998;7(4):189–198. [PubMed: 9782888]
  7. Byington C, Enriquez F, Hoff C. Serious bacterial infections in febrile infants 1 to 90 days old with and without viral infections. Pediatrics. 2004;113(6):1662–1666. [PubMed: 15173488]
  8. CBO (Congressional Budget Office) Evidence on the costs and benefits of health information technology. Washington, DC: Congress of the United States; 2008.
  9. CBO (Congressional Budget Office) Letter to the chairman of the Committee on Ways and Means from Robert a Sunshine, Acting Director of CBO. 2009
  10. DesRoches CM, Campbell EG, Rao SR, Donelan K, Ferris TG, Jha A, Kaushal R, Levy DE, Rosenbaum S, Shields AE, Blumenthal D. Electronic health records in ambulatory care—A national survey of physicians. New England Journal of Medicine. 2008;359(1):50–60. [PubMed: 18565855]
  11. Dopson S, Fitzgerald L. Knowledge to action? Evidence-based health care in context. New York: Oxford University Press; 2005.
  12. Elmendorf DW. Options for Controlling the Cost and Increasing the Efficiency of Health Care. 2009. [accessed September 4, 2009]. http://www​​/99xx/doc9911/02-25-Health​_Insurance.pdf.
  13. Fiesta J. Charting—one national standard, one form. Nursing Management. 1993;24(6):22, 24. [PubMed: 8510851]
  14. Fischer MA, Vogeli C, Stedman M, Ferris T, Brookhart MA, Weissman JS. Effect of electronic prescribing with formulary decision support on medication use and cost. Archives of Internal Medicine. 2008;168(22):2433–2439. [PubMed: 19064827]
  15. Garrido T, Raymond B, Jamieson L, Liang L, Wiesenthal A. Making the business case for hospital information systems—A Kaiser Permanente investment decision. Journal of Health Care Finance. 2004;31(2):16–25. [PubMed: 15839526]
  16. Gawande A. The cost conundrum. The New Yorker. Jun 1, 2009. online.
  17. Grieger D, Cohen SH, Krusch DA. A pilot study to document the return on investment for implementing an ambulatory electronic health record at an academic medical center. Journal of the American College of Surgeons. 2007;205(1):89–96. [PubMed: 17617337]
  18. HCFA (Health Care Financing Administration) Medicare and Medicaid programs: Mandatory use, collection, encoding, and transmission of Outcome and Assessment Information Set (OASIS) for home health agencies. Federal Register. 1999;64(117):32984. [PubMed: 10558574]
  19. Hendrich A, Chow MP, Goshert WS. A proclamation for change: Transforming the hospital patient care environment. Journal of Nursing Administration. 2009;39(6):266–275. [PubMed: 19509601]
  20. Hillestad R, Bigelow J, Bower A, Girosi F, Meili R, Scoville R, Taylor R. Can electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Affairs (Millwood) 2005;24(5):1103–1117. [PubMed: 16162551]
  21. Iglehart J. The Malpractice Morass: Symbol of Societal Conflict. Health Affairs (Millwood) 2004;23(4):7–8.
  22. Jha AK, DesRoches CM, Campbell EG, Donelan K, Rao SR, Ferris TG, Shields A, Rosenbaum S, Blumenthal D. Use of electronic health records in U.S. hospitals. New England Journal of Medicine. 2009;360(16):1628–1638. [PubMed: 19321858]
  23. Joint Commission. Health care at the crossroads: Strategies for improving the medical liability system and preventing patient injury. 2005. [accessed November 13, 2009]. http://www​.jointcommission​.org/NR/rdonlyres​/167DD821-A395-48FD-87F9-6AB12BCACB0F​/0/Medical_Liability.pdf.
  24. Joint Commission. Improving America’s Hospitals. The Joint Commission; Oakbrook Terrace: 2008.
  25. Kaushal R, Jha AK, Franz C, Glaser J, Shetty KD, Jaggi T, Middleton B, Kuperman GJ, Khorasani R, Tanasijevic M, Bates DW. Return on investment for a computerized physician order entry system. Journal of the American Informatics Association. 2006;13(3):261–266. [PMC free article: PMC1513660] [PubMed: 16501178]
  26. Korst LM, Eusebio-Angeja AC, Chamorro T, Aydin CE, Gregory KD. Nursing documentation time during implementation of an electronic medical record. Journal of Nursing Administration. 2003;33(1):24–30. [PubMed: 12544622]
  27. LaDuke S. Online nursing documentation: Finding a middle ground. Journal of Nursing Administration. 2001;31(6):283–286. [PubMed: 11417165]
  28. Massachusetts Technology Collaborative & New England Healthcare Institute. Saving lives, saving money in practice: Strategies for computerized physician order entry in Massachusetts hospitals. 2009. [accessed August 26, 2009]. http://web3​.streamhoster​.com/mtc/cpoe2009.pdf.
  29. Miller R, Miller B. The Santa Barbara County Care Data Exchange: What Happened? Health Affairs (Millwood) 2007;26(5):w568–w580. [PubMed: 17670775]
  30. Miller RH, West C, Brown TM, Sim I, Ganchoff C. The value of electronic health records in solo or small group practices. Health Affairs (Millwood) 2005;24(5):1127–1137. [PubMed: 16162555]
  31. Morris AH, Orme J Jr, Truwit JD, Steingrub J, Grissom C, Lee KH, Li GL, Thompson BT, Brower R, Tidswell M, Bernard GR, Sorenson D, Sward K, Zheng H, Schoenfeld D, Warner H. A replicable method for blood glucose control in critically ill patients. Critical Care Medicine. 2008;36(6):1787–1795. [PubMed: 18520641]
  32. Moffit RE. HCFA’s latest assault on patient privacy. Washington, DC: Heritage Foundation; 1999.
  33. Moffit RE, Applebaum P, Brown KM, Pyles J, Welch R. How the Medicare Bureaucracy threatens patient privacy. Washington, DC: Heritage Foundation; 1999.
  34. National Alliance for Health Information Technology. Report to the Office of the National Coordinator for Health Information Technology on Defining Key Health Information Technology Terms. 2008
  35. New York State Department of Health. Request for grant applications: Heal NY—Phase 1 health information technology grants. 2006. [accessed March 8, 2008]. http://www​.health.state​
  36. Nuzum R, Mika S, Schoen C, Davis K. Finding Resources for Health Reform and Bending the Health Care Cost Curve. New York: The Commonwealth Fund; 2009.
  37. OASIS data specifications. Data specifications. [accessed November 13, 2009]. n.d. http://www​​/OASIS/04_Data-Specifications.asp.
  38. Pabst MK, Scherubel JC, Minnick AF. The impact of computerized documentation on nurses’ use of time. Computational Nursing. 1996;14(1):25–30. [PubMed: 8605657]
  39. Pham HH, O’Malley AS, Bach PB, Saiontz-Martinez C, Schrag D. Primary care physicians’ links to other physicians through Medicare patients: The scope of care coordination. Annals of Internal Medicine. 2009;150(4):236–242. [PMC free article: PMC3718023] [PubMed: 19221375]
  40. Smeltzer CH, Hines PA, Beebe H, Keller P. Streamlining documentation: An opportunity to reduce costs and increase nurse clinicians’ time with patients. Journal of Nursing Care Quality. 1996;10(4):66–77. [PubMed: 8783547]
  41. Staines A. Institutional strategies for holistic improvement. Institute for Health Care Delivery; July 29, 2009; Salt Lake City, UT. 2009.
  42. Stead W. Beyond expert-based practice. Institute of Medicine (IOM) annual meeting; Washington, DC. 2007.
  43. Steinbrook R. Health care and the American Recovery and Reinvestment Act. New England Journal of Medicine. 2009;360(11):1057–1060. [PubMed: 19224738]
  44. Trossman S. The documentation dilemma: Nurses poised to address paperwork burden. The American Nurse. 2001;33(5):1, 9, 18.
  45. Upenicks V. Work sampling: Assessing nursing efficiency. Nursing Management. 1998;29(4):27–29. [PubMed: 9697490]
  46. Urden IR. Working sampling: A decision-making tool for determining resources and work redesign. Journal of Nursing Administration. 1997;27(9):34–41. [PubMed: 9300013]
  47. U.S. House of Representatives and U.S. Senate. The American Recovery and Reinvestment Act of 2009 (ARRA) Washington, DC: U.S. Government Printing Office; 2009.
  48. Wachter R, Pronovost P. The 100,000 lives campaign: A scientific and policy review. The Joint Commission Journal on Quality and Patient Safety. 2006;32(11):621–627. [PubMed: 17120921]
  49. Walker J, Pan E, Johnston D, Adler-Milstein J, Bates DW, Middleton B. The value of health care information exchange and interoperability. Health Affairs (Millwood) 2005;(Suppl Web Exclusives):W5–10. W15–18. [PubMed: 15659453]
  50. Wang SJ, Middleton B, Prosser LA, Bardon CG, Spurr CD, Carchidi PJ, Kittler AF, Goldszer RC, Fairchild DG, Sussman AJ, Kuperman GJ, Bates DW. A cost-benefit analysis of electronic medical records in primary care. The American Journal of Medicine. 2003;114(5):397–403. [PubMed: 12714130]

The author wishes to give special thanks to Institute for Health Care Delivery Research analysts Erick Henry, Craig Gale, Karen Valentine, Thomas French, and Pascal Briot for providing summary results of clinical program CPMs. The author would also like to thank Brent James, Director of the Institute and Chief Quality Officer at Intermountain Healthcare, for his vision and leadership in providing analytic infrastructure to clinical programs as well as guidance in placing Intermountain Healthcare’s quality improvement work on the national agenda. Finally, thanks to Byington and Morris for agreeing to share their work that exemplifies the impact of evidence-based CPMs.

Interested readers should evaluate the mandated Paperwork Reduction Act publication’s contention that no additional burden will occur with the implementation for OASIS-C (CMS, 1995).



The author wishes to give special thanks to Institute for Health Care Delivery Research analysts Erick Henry, Craig Gale, Karen Valentine, Thomas French, and Pascal Briot for providing summary results of clinical program CPMs. The author would also like to thank Brent James, Director of the Institute and Chief Quality Officer at Intermountain Healthcare, for his vision and leadership in providing analytic infrastructure to clinical programs as well as guidance in placing Intermountain Healthcare’s quality improvement work on the national agenda. Finally, thanks to Byington and Morris for agreeing to share their work that exemplifies the impact of evidence-based CPMs.


See http:​//intermountainhealthcare​.org/about​/quality/institute/Pages/home.aspx.


See http:​//intermountainhealthcare​.org/about​/quality/institute/clinicalmanagement​/Pages/home.aspx.


Interested readers should evaluate the mandated Paperwork Reduction Act publication’s contention that no additional burden will occur with the implementation for OASIS-C (CMS, 1995).

Copyright © 2010, National Academy of Sciences.
Bookshelf ID: NBK53928
PubReader format: click here to try


  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (5.4M)

Related information

  • PMC
    PubMed Central citations
  • PubMed
    Links to pubmed

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...