Elements of Quality

Quality comprises three elements:

• Structure refers to stable, material characteristics (infrastructure, tools, technology) and the resources of the organizations that provide care and the financing of care (levels of funding, staffing, payment schemes, incentives).
• Process is the interaction between caregivers and patients during which structural inputs from the health care system are transformed into health outcomes.
• Outcomes can be measured in terms of health status, deaths, or disability-adjusted life years—a measure that encompasses the morbidity and mortality of patients or groups of patients. Outcomes also include patient satisfaction or patient responsiveness to the health care system (WHO 2000).

Structural measures are the easiest to obtain and most commonly used in studies of quality in developing countries. Many evaluations have revealed shortages in medical staff, medications and other important supplies, and facilities, but material measures of structure, perhaps surprisingly, are not causally related to better health outcomes (Donabedian 1980). Although higher technology or a more pleasant environment may be conducive to better-quality care, the evidence indicates only a weak link between such structural elements and better health outcomes (Donabedian 1988). The notable exceptions are cases in which physical improvements either increase access to primary care in very poor settings or increase the volume of a clinical procedure, such as cataract surgery, that is specifically linked to better health outcomes (Javitt, Venkataswamy, and Sommer 1983). At best, however, structure is a blunt approximation of process or outcomes; structural improvements by themselves rarely improve the health of a population.

Process, by contrast, can be measured with every visit to a provider. Measuring process is difficult, however, particularly in developing countries. The private nature of the doctor-patient consultation, a lack of measurement criteria, and the absence of reliable measurement tools have limited the ability to assess process (Peabody, Tozija, and others 2004). However, new methods are being developed that can provide valid measurements of clinical practice (Thaver and others 1998). In addition, evidence-based clinical studies have steadily revealed which process measures lead to better health outcomes. This combination of ubiquity, measurability, and linkage to health outcomes makes the measurement of process the preferred way to assess quality.

Although good outcomes are the objective of all health actions, outcomes alone are not an efficient way to measure quality for two reasons. The first is the quality conundrum. A patient may receive poor-quality care but may recover fully, or a patient may receive high-quality care for an illness such as cerebral malaria and still not recover. Second, adverse health outcomes are relatively rare and obviously do not occur with every encounter.

The classic framework of structure-process-outcome is well established. However, in recent years the concept of quality has been expanded to include specific aims for improvement. For example, the Institute of Medicine's (2001) landmark report, Crossing the Quality Chasm, broadens the concept to include other, more contextual elements to illuminate how process changes can improve care. It focuses on six aims: patient safety, effectiveness, patient centeredness, timeliness, efficiency, and equity (see box 70.1).

Box 70.1

The Institute of Medicine's Six Elements of Quality. Patient safety. Are the risks of injury minimal for patients in the health system? Effectiveness. Is the care provided scientifically sound and neither underused nor overused? Patient centeredness. (more...)

Quality Assessment Perspectives

We can look at the Institute of Medicine's aims from two perspectives: patient perception, and technical or professional assessment. Patients' perceptions of quality depend on their individual characteristics and affect their compliance, follow-up decisions, and long-term lifestyle changes (Zaslavsky and others 2000). Interpersonal relationships, cultural appropriateness, and gender sensitivity—long thought to be luxuries of wealthier countries—are also major determinants of patient access and utilization in developing countries. These findings have led to the inclusion of patient satisfaction and patient responsiveness as outcome measures.

Technical assessment concerns whether providers meet normative standards for appropriateness of care or adherence to explicit evidence-based criteria. Although patient perception or satisfaction is important, researchers increasingly rely on objective, evidence-based quality criteria that can be more readily linked to better health outcomes at both the individual and the population levels.

Population-Level Considerations

Quality is typically assessed through the interaction between individual doctors and patients. However, emerging evidence shows that the average quality of care given by groups of doctors and other providers is an important determinant of overall community health status. For example, in a cross-sectional analysis in the former Yugoslav Republic of Macedonia, researchers found not only that patients' heath status was significantly higher in areas where quality was higher but also that the overall self-reported health status of those members of the general population who had not recently received care was higher (Peabody, Tozija, and others 2004).

Our quality-of-care framework supports these findings. When process is improved among groups of providers, the aggregate improvement in quality leads to better health outcomes for the entire patient population. In addition, resources can be allocated among clinical interventions based on actual effectiveness and the overall impact of care on the population. For example, cancer chemotherapy may be available and may prolong the lives of cancer patients. However, it may result in fewer lives saved than the expansion of coverage of directly observed treatment short-course coverage for tuberculosis patients.

Legal Mandates, Accreditation, and Administrative Regulations

Legal mandates, accreditation, and administrative regulations affect quality by controlling entry into the practice of health care. These policies include the licensing of professionals and facilities, their accreditation or certification to perform certain procedures, and the formal delineation of functions that various types of health workers can legally perform. Although these policies assume that providers' prior qualifications are good predictors of actual performance in health care delivery, there is little evidence that such policies have a positive effect on process or outcomes. They are more successful at barring unqualified persons from practicing than at ensuring quality among those who are allowed to practice. A review of health sector regulations in Tanzania and Zimbabwe, for example, revealed that the regulations primarily control entry into the market and ensure a minimum standard of quality (Kumaranayake and others 2000).

Hospital accreditation, with its periodic reviews of health facility performance standards, can potentially provide ongoing regulatory pressure for improvement. To date, research has not demonstrated that hospital accreditation programs are linked to improvements in health outcomes. In a randomized controlled trial of a hospital accreditation program in the KwaZulu-Natal province of South Africa, researchers showed a conclusive link between the implementation of the program and improvements in the accreditation standard indicators. However, they were unable to link those indicators to improvements in health outcomes (Salmon and others 2003).

Sharing Information on Quality Improvement Technology

Worldwide interest in quality has given rise to new professional bodies, scientific publications, and institutions dedicated to sharing ideas and innovations in quality improvement. Organizations such as the Robert Wood Johnson Foundation, the Nuffield Trust, and the Institute for Healthcare Improvement cultivate ideas for improvement, bring people and organizations together to learn from each other, and take action to achieve results. Although the sharing of information on quality health care practices has long been an established part of provider education and training networks, the sharing of information on successful systemwide policies for process improvements could potentially accelerate the scale-up of quality practice.

One organization active in developing countries is the Council on Health Research for Development (COHRED), which promotes, facilitates, and evaluates the Essential National Health Research strategy in such countries as Benin, the Arab Republic of Egypt, and Indonesia. COHRED aims to develop a system of effective health research to improve health services, including quality of care. The Quality Assurance project funded by the U.S. Agency for International Development has studied and shared information about quality in the developing world since 1990. Under the Quality Assurance project umbrella, researchers have studied and implemented quality measurement and improvement interventions and have used these case studies to develop a library of tools and articles to promote global quality improvement.

Public-Private Provision of Care

In most health care systems, a professional regulatory framework governs the network of civil servants delivering health care. These civil servants operate alongside autonomous, self-governed, private providers—independent for-profit physicians and health clinics and nonprofit nongovernmental organizations (NGOs). Two conclusions arise from the often heated debate about the right balance between public and private services. First, private practitioners provide a significant amount of care in developing countries. Second, though there is no one prescription for striking the right public-private mix, in some cases the public regulatory framework has led to private provision of higher-quality care. The government of Senegal successfully contracted with community-based groups for preventive nutrition services. Eighteen months after nutrition services were implemented, severe malnutrition disappeared among children age 6 to 11 months (Marek and others 1999). The success of the program has led to its expansion nationwide.

Targeted Education and Professional Retraining

Continuing medical education is a common approach to improving clinical practice, but it neither changes clinical practice nor advances health outcomes (Davis and others 1995). Newer techniques—targeted education, case-based learning, and interactive and multimodel teaching techniques—have had some success. In Guatemala, distance education targeting diarrhea and cholera case management increased accurate assessment and classification of diarrhea cases by 25 percent. Rehydration did not improve, however, and improvements in counseling were insignificant (Flores, Robles, and Burkhalter 2002). In Tanzania, training staff in the control of acute respiratory infections of young children yielded reductions in under-five mortality within two years (Mtango and Neuvians 1986).

Organizational Change

In recent years, organizational change in the health care system has been shown to influence quality of care and to further the six aims of the Institute of Medicine by focusing on the continual design and redesign of systems. The emphasis is on developing organizational and individual capabilities where they most profoundly affect the process of care. Design and redesign interventions assume that simply adding a new resource or a new process in isolation will not improve care because better care is the product of many processes working together. Although change interventions have not been widely used in the developing world because they require large investments to plan and implement, four related models of organizational change have been successful in changing provider practice in developing nations:

• Total Quality Management in health care Advances in business management practices to continually design and redesign systems for quality improvement have been effectively adapted for health systems. In Total Quality Management, also known as Continuous Quality Improvement, teams use mutually reinforcing techniques in a cycle of planning, implementing, evaluating, and revising to improve the quality of clinical and administrative processes. These techniques include process mapping, statistical quality control, and structured team activities. In rural Bihar, India, private practitioners who treat sick children were provided with standard case-management information, were given feedback on their performance, and were tracked and monitored over time. This strategy produced significant improvements in practitioners' case-management skills (Chakraborty, D'Souza, and Northrup 2000). In Malaysia, anesthesia safety has been improved through the implementation of consensus-based protocols that emphasize (a) communication among the operating, recovery, and ward team members; (b) individual feedback; and (c) frequent monitoring to identify areas for improvement (Tan 1999).
• Collaborative Improvement Model The early success of Total Quality Management techniques has given rise to a related model, the Collaborative Improvement Model. It addresses broad and complex systemic processes within health care systems and has facilitated the scale-up of quality improvements. This model, designed to continuously improve organizational and individual performance, comprises four elements: definition of an aim, measurement, innovation, and testing to see whether the innovation meets the original aim. This approach strikes a pragmatic balance between the need for action and the need to be scientifically grounded. It has been used with success in Peru and the Russian Federation. In Peru, the collaborative improvement model was used by multidisciplinary teams in 41 clinics to design changes aimed at achieving world-class tuberculosis care. The preliminary results have led to impressive changes in the process of care, but it is too early to determine whether they have been effective in improving quality (Berwick 2004).
• Plan-Do-Study-Act cycle The Plan-Do-Study-Act (PDSA) cycle calls for action-oriented learning in quality improvement. Team members using the PDSA model design a quality-improvement intervention (plan), implement it on a small scale (do), evaluate the results (study), and implement or alter the intervention accordingly (act). Often multiple PDSA cycles are necessary before the appropriate improvement method can be identified. All improvement techniques that involve the design and redesign of systems use some form of the PDSA cycle. Successful scale-up of a PDSA prototype is possible with careful leadership oversight. A team of investigators in Russia's Tula province developed a series of successful interventions for adults who have poorly controlled hypertension. The interventions, which were started in 20 clinics, were expanded to 500 clinics within 18 months. The scale-up resulted in a sevenfold increase in patients receiving hypertension management at the primary care level and an 85 percent reduction in admissions for hypertension. In Tver province, the same group addressed problems related to prenatal care. They began with 5 hospitals and scaled up to cover all 42 hospitals and all maternity clinics in the province. The result was a 99 percent reduction in newborns with hypothermia and a reduction in pregnancy-induced hypertension from 44 percent to 6 percent (Berwick 2004). Although the experience of researchers implementing interventions that are based on system redesign in the developing world has been largely positive, it is not clear whether the resources and leadership exist to bring these interventions to scale through country or regional policies. Further evidence is needed concerning the real-world feasibility and cost-effectiveness of system redesign.
• Internal enabling environment Creating the right environment for change involves leadership and leadership training, clinicians empowered to make quality improvement decisions, and resources for quality improvement planning activities (Silimper and others 2002). The internal enabling environment in Costa Rica promoted strong leadership that led to the adoption of structural adjustment loans in the early stages of health sector reforms. The loans were used to maintain such public health programs as mother and child nutrition, even though public spending dropped and prices increased dramatically (Peabody 1996). An enabling environment can also be created by teams of individuals, each representing different stakeholder groups (physicians, nurses, staff members, patients, and so forth) or simply by a strong leader with an interest in teamwork and the resources to support a discrete quality improvement function for team members.

Training with Peer Review Feedback

In Mexico City, physician retraining on treatment of diarrhea, combined with the concurrent creation of a peer-review structure, decreased the use of antibiotics and increased the use of oral rehydration therapy (ORT). These improvements continued to be seen in a follow-up evaluation 18 months later (Gutierrez and others 1994). The approach has been effectively expanded to prescribing practices for rhinopharyngitis among primary care physicians, using an interactive training workshop and a managerial peer-review committee (Perez-Cuevas and others 1996).

Performance-based Remuneration

A potentially powerful instrument for accelerating quality improvements involves making payments directly to providers who meet quality standards that are based on process indicators associated with favorable patient outcomes. Systems that tie performance to remuneration use relatively small incentives—equivalent to 3 to 10 percent of the provider's total compensation. Performance-based remuneration has been successfully used in the United States to compensate both private and public providers (McBride, Neiman, and Johnson 2000).

Examples of performance-based incentives come from developing countries too. The Nicaraguan Ministry of Health has implemented a pilot program in six hospitals that offers an incentive bonus (a maximum average of 17 percent of hospital revenue) for facilities that achieve performance targets that include quality measures (Jack 2003). In Haiti, a performance-based payment scheme was set up for NGOs that provided services to the population. The scheme resulted in all three participating NGOs reaching target immunization coverage rates (Eichler, Auxila, and Pollock 2001). Thus, payment for specified and observable performance (in terms of provider effort, client coverage, or health impact on the population) can be usefully applied to NGOs and private providers.

The specific features of performance-based remuneration are crucial. A study evaluating the South African government's experience in contracting with private organizations to operate district hospitals found no cost savings—in fact, the government was spending more than if it provided the services itself. The contracting may have failed because remuneration was not based on specific process or outcome measures. Instead, the contractor's obligation, the methods of monitoring performance, and the sanctions for nonperformance were only minimally specified (Broomberg, Masobe, and Mills 1997).

High Volume of Care

Evidence exists that a high volume of care by individuals or institutions leads to better health outcomes (Habib and others 2004). Physician experience (learning) and practice (repetition) lead to fewer complications, less resource use, and better quality for a variety of procedures, such as cataract surgery and laparoscopy (Brian and Taylor 2001). More complex procedures, including endarterectomy, cancer surgery, and coronary bypass surgery, have shown similar effects.

Volume effects leading to better health outcomes are not confined to surgical procedures (Zgibor and Orchard 2004). Facilities specializing in the care of chronic diseases such as diabetes, myocardial infarction, and heart failure are also associated with better outcomes. Debate exists over how much of the volume effect is due to specialist care. The benefits of high-volume care persist, however, even after controlling for referral and case-mix biases. When carefully trained nonphysicians are substituted for physicians, volume effects persist but can be accomplished at significantly lower costs. In one study, nurse practitioners and physician's assistants were able to provide high-quality care for common outpatient conditions such as hypertension, diabetes, asthma, otitis media, pharyngitis, and back pain at substantially lower costs than that of physicians (Douglas and others 2004).

Performance-based Professional Recognition

Providers work in a community of peers in which professional status, prestige, and recognition are often as valuable as material rewards. Nonmonetary incentives, such as public recognition or disclosure, administrative privileges, and awards from professional organizations, can promote improvements in quality. Uganda, for example, implemented the Yellow Star Program as part of a broader health services improvement project. This program evaluated health facilities on a quarterly basis, using 35 indicators of technical and interpersonal quality, and awarded a large yellow star to facilities that scored 100 percent in two consecutive quarters. The star was then prominently displayed outside the facility.

The Mexican Ministry of Health has implemented a strategy that combines the accreditation and the training strategies discussed earlier with nonmonetary incentives. The National Crusade for Quality in Health Care introduces quality-oriented incentives to health facilities and medical schools. It also includes public recognition in an effort to encourage learning and to change practice. The National Crusade has already generated measurable improvements in the responsiveness of state-level health systems (Secretaría de Salud de Mexico 2003).

Both types of policies examined in this section are associated with better quality and better health outcomes—lower premature mortality and avoidable morbidity, increased patient satisfaction, and more health-seeking behaviors. When effective, these policies result in increased coverage rates, better prescribing patterns, and increased adherence to clinical guidelines. They can spell the difference between an individual's survival or death, between an individual benefiting from the encounter with the health sector or being harmed by it, and between an individual and society rising from poverty or sinking deeper into it.

Chart Abstraction

Chart abstraction, or review of the medical record, has long been used to measure technical quality. Such familiar quality evaluations as clinical audits, physician report cards, and profiles are based on chart abstraction. The core strength of the medical record is that it is ubiquitous and can generally be obtained after each encounter. Chart reviews, however, suffer from problems of legibility when notes are handwritten. Often they are generated for reasons other than recording the actual events of the clinical visit (legal protection or obtaining payments, for example) and thus lack crucial clinical details. One prospective study showed that charts identified only 70 percent of items performed during the clinical encounter (Luck and others 2000). In a related analysis, 6.4 percent of the items recorded in the chart were false and had never really occurred.

Where resources and infrastructure are sufficient, the electronic medical record (EMR) is becoming a priority for health systems worldwide. EMR technology promotes uniformity, legibility, and communication, which can lead to guideline use and reduce prescription errors. It also holds the promise of managing populations rather than individuals by aggregating patients into groups. However, the EMR has not always lived up to its potential. In many countries, some impressive successes have occurred—as have spectacular failures, costing billions of dollars (McConnell 2004). The great heterogeneity in record-keeping practices, problems with medical records (both paper and electronic), and costs of trained medical abstractors have led to a search for other reliable ways to measure quality.

Direct Observation and Recording of Visits

Direct observation and recording of visits is a commonly used approach in developing countries (Nolan and others 2001). Ethically, the provider and the patient must be informed of the observation or recording, which introduces participation bias because provider behavior may change as a result of being evaluated. In addition, trained observers are costly, and variation between observers is difficult to remedy.

Administrative Data

Administrative data, collected for purposes of managing the delivery of care, are available in all but the poorest settings. A data collection system, once established, is ubiquitous and can provide information on charges and many cost inputs. Administrative data, however, lack sufficient clinical detail to be useful in evaluating process. In a 2003 study, an incorrect diagnosis was recorded in the data 30 percent of the time (although the diagnosis was made correctly). Overall, these data reflected the actual clinical diagnosis only 57 percent of the time (Peabody, Luck, Jain, and others 2004). As information systems advance, accuracy problems may be mitigated, although the lack of adequate clinical detail will continue to limit the use of administrative data.

Standardized Patients

Standardized patients can be a gold standard for process measurement (Luck and Peabody 2002). Trained to simulate illness, standardized patients present themselves unannounced into a clinical setting to providers who have previously given their consent to participate in the study. At the conclusion of the visit, the standardized patient reports on the technical and interpersonal elements of process. Standardized patients are reliable over a range of conditions and provide valid measurements that accurately capture variation in clinical practice among providers over time. However, they are expensive and useful only for adult conditions and only those conditions that can be simulated. Thus, they are not practical for routinely evaluating quality.

Clinical Vignettes

Clinical vignettes were developed explicitly for measuring quality within a group of providers and evaluating quality at the population level. Vignettes are responsive to variation in quality, and providers readily accept them if they are given anonymously (Peabody, Luck, Glassman, and others 2004). More than 20 vignettes have been used in 13 countries around the world. They can be administered on paper, by computer, or over the Internet. Providers are typically presented with several cases. When process is being measured for many providers, each provider is presented with the same case or set of cases, thus eliminating the need for case-mix adjustment. The provider completing the vignette is asked to take a history, do an examination, order the necessary tests, make a diagnosis, and specify a treatment plan. The questions are open ended and include interactive responses that simulate the visit and evaluate the physician's knowledge. In two separate, prospective validation studies among randomly selected providers, vignettes consistently demonstrated greater predictive validity of process than did the abstracted medical record. Vignettes have been validated against the gold standard of standardized patient visits, and they reflect actual clinical practice, not just physicians' knowledge. Vignettes have several other advantages. Because exactly the same case can be given to many providers, vignettes are useful for comparison studies. They are also useful for pre- and postevaluations of policy interventions designed to improve quality. Finally, they are inexpensive to administer and straightforward to score, making them particularly useful in developing countries.

Better Treatment of Pneumonia in Children

Part of the high mortality from childhood pneumonia in the developing world can be explained by poor-quality care, which is defined as the inability either to accurately diagnose or to treat the disease. Our prototype intervention has two cost components: the cost of implementing an educational activity for providers and the cost of treating nonsevere and severe childhood pneumonia. The former component is based on a study and uses conservative high-end cost estimations (Kelley and others 2001); the latter is the midpoint from another study (Stansfield and Shepard 1990). The number of lives saved depends on the effect of the intervention—that is, the change in the percentage of cases diagnosed and treated; the prevalence rate of both types of pneumonia; the population covered by each provider; the case-fatality ratio; and the effectiveness of the treatment. Both the case-fatality ratio and the effectiveness ratio were fixed at middle values suggested by earlier work (Stansfield and Shepard 1990). For the other parameters, a large range of variation was considered, producing 450 scenarios. Finally, six impact levels were considered, which were based on two previous studies (Chakraborty, D'Souza, and Northrup 2000; Mtango and Neuvians 1986).

The analysis showed that, under average conditions, improving quality of care for conditions of acute respiratory illness can be very cost-effective. When the baseline quality is low and the disease prevalence is high, an intervention that raises quality has a cost-effectiveness ratio of US$132 to US$800 per life saved; if the policy intervention is ineffective or the prevalence of pneumonia is low, the average cost of saving a life could be more than US$2,000. When 60 percent of cases are already appropriately diagnosed and treated, the cost-effectiveness ratio rises to US$5,000 per life saved. ¹

Better Treatment of Diarrhea

Diarrhea remains one of the leading causes of childhood morbidity and mortality in the developing world. The diarrhea incidence rate among children in resource-constrained countries can reach six to seven episodes per year (Thapar and Sanderson 2004). ORT is the accepted standard of care for acute diarrhea. Unfortunately, a large proportion of cases are still treated with nonrehydration medication, including antibiotics and antidiarrheals. Improved diagnosis of dehydration and reduced use of unnecessary medications, however, lead to better outcomes.

Various interventions can make sizable changes in the diagnostic and prescribing patterns of providers. Verbal case review, combined with a package of additional intervention referred to as INFECTOM (Information, Feedback, Contracting with Providers to Adhere to Practice Guidelines, and Ongoing Monitoring), increased the proportion of cases treated correctly from 16 percent to 48 percent (Bloom, Canning, and Sevilla 2001). One study reports that small group, face-to-face interventions reduced antimicrobial prescriptions by 16 percent and antidiarrheal prescriptions by 7 percent among a group of providers treating acute diarrhea in Indonesia (Santoso, Suryawati, and Prawaitasari 1996). The same study showed that formal seminars reduced antimicrobial use by 10 percent and antidiarrheal use by 7 percent. On the basis of these studies, an average cost per intervention was used, ranging from US$25 to US$125.

The savings from switching to a less costly treatment (instead of antibiotics, for example) were subtracted from the direct costs that are related to implementing the training activity. Because other savings, such as those related to a lower use of inpatient services, were ignored, the estimates are conservative. Savings could be greater: Two years after an ORT unit was established at the Kamuza Central Hospital in Malawi, 50 percent fewer children with diarrhea were admitted to the pediatric ward, and those admitted required 56 percent less intravenous fluid for rehydration (Martines, Phillips, and Feachem 1990).

Again, the number of lives saved depends on the disease prevalence; the effect of the policy on treatment quality; the population covered by each provider; the average case-fatality ratio, which was set at 6 per 1,000 on the basis of Snyder and Merson (1982); and the effectiveness of the treatment. For the latter parameter, reductions in mortality rates following ORT treatment of 40 to 60 percent and reductions in effectiveness ratios of 5 to 100 percent have been reported (Shepard, Brenzel, and Nemeth 1986). Accordingly, the effectiveness ratio was set at 80 percent. As before, alternative values for the other parameters were adopted, generating 450 scenarios.

Educational interventions to improve the quality of care for treatment of diarrheal diseases are also highly cost-effective. In general, the cost of saving a life through educational interventions is less than US$500 and could be as low as US$14. Scenarios with high cost per life saved (more than US$6,000) are when prevalence rates are low or when implementation costs for quality-related interventions are high.

Although the data available to estimate the costs and benefits of health outcomes and process are limited, these simulations, combined with published reports of successful policy interventions, clearly show the cost-effectiveness of interventions that improve health outcomes through better quality of care. However, reliable measures of quality are necessary to design and evaluate these interventions.