Discounting and Age Weighting in the DALY Measure

This section briefly reviews the rationale and implementation of discounting and age weights in the standard DALY. To denote different choices for the discount rate and age weights, we use the notation DALYs(r, K), where r is the discount rate in percent (not a fraction as in the GBD 1990 study) and K is the age-weighting modulation factor, a parameter that allows uniform (K = 0) or the GBD nonuniform (K = 1) age weighting to be used. With this notation, DALYs(3,0) denotes the DALY with a 3 percent discount rate and uniform age weights as used in the Disease Control Priorities Project (DCPP) and DALYs(3,1) denotes the 3 percent discount rate and varying age weights as used in the GBD study. Similarly, we may refer to the DALY components of years of life lost due to premature mortality (YLL) and years of healthy life lost due to disability (YLD) as YLL(r, K) or YLD(r, K) using the same convention.

Discounting, Age Weights, and the YLL Loss Function

DALYs are calculated as the sum of YLL from a cause and the YLD for incident cases of the health condition (see chapter 3 for more details). Murray (1996) provides general formulas for YLL and YLD that allow the annual discount rate r and the age-weighting parameters (K, C, β) to be varied. When K is set equal to 1, then the DALY includes an age-weighting function of the form Cxe^−βx, where x is the age in years and β and C are constants. For the 1990 GBD study, Murray and Lopez chose β = 0.04. The value of β = 0.04 was chosen to give an age pattern similar to that seen in available empirical data. C is a parameter chosen to ensure that the total global DALYs are the same with and without age weighting, estimated at C = 0.1658 for the 1990 GBD study. Figure 5.1 illustrates the form of the age-weighting function for β = 0.02, 0.04, and 0.06. For the other two choices of β (0.02 or 0.06), the value of C was varied to ensure the same area under the curve from age 0 to 100 years.

Figure 5.1

Age-Weighting Function Incorporated into the DALY

The age-weighting function specifies the relative value of a year of life lived at different ages either for YLD or YLL estimates. To estimate the total years of life lost due to death at age x, the age-weighting function is integrated over all ages above x. Table 5.1 shows the resulting loss function for selected exact ages, also plotted in figure 5.2 for females. The male-female gap in YLL(0,0), 2.5 years at birth, is reduced to 0.1 years for YLL(3,1) (figure 5.3). Figure 5.4 shows the effect on YLL of varying the parameter β in the age-weighting function. Values of β higher than 0.04 give relatively greater weight to younger ages and less to older ages; values of β lower than 0.04 give relatively lower weight to younger ages and more to older ages.

Table 5.1

Standard Life Expectancies at Selected Exact Ages and Discounted YLL Due to a Death at Selected Ages.

Figure 5.2

Effect of Age Weighting and Discounting on the YLL per Death at Various Ages for Females

Figure 5.3

Effect of Age Weighting and Discounting on the Male-Female Gap in YLL per Death

Figure 5.4

Effect on YLL per Death of Varying the Parameter β in the DALY Age-Weighting function

Table 5.2 further examines the effects of varying the parameter β in the age-weighting function on the weights applicable at different ages. For the standard DALY, β = 0.04 implies a maximum age weight of 1.52 at age 25, and the age weight is greater than 1 over the range 8.4 to 54.2 years. Compare this with β = 0.03, which gives a maximum age weight of 1.29 at age 33.3 years with a prime age range (weight greater than 1) of 14.9 to 63.0 years. Note that the choice of β = 0.03 gives a prime age range that matches fairly typical ages for formal entry and exit from work in many societies (Mahapatra 2001). We do not consider variations in β further here. Sensitivity analyses for GBD 2001 that follow compare standard age weights (β = 0.4) with uniform age weights.

Table 5.2

Implications of Variation in Choice of Age-Weight Parameter β on the Age-Weighting Function.

Sensitivity of Burden of Disease and Injury Results to Variations in Key Parameter Values

This section examines the sensitivity of the DALY estimates for the global burden of disease in 2001 to alternative assumptions about the discount rate and age weighting. As discussed in chapter 3, the DALY measures the future stream of healthy years of life lost due to each incident case of disease or injury. It is thus an incidence-based measure rather than a prevalence-based measure. The GBD study applied a 3 percent time discount rate to years of life lost in the future to estimate the net present value of years of life lost. With this discount rate, a year of healthy life gained in 10 years' time is valued at 24 percent less than one gained now (note that the standard DALY uses an instantaneous 3 percent discount rate, which results in an annual discount factor that is slightly higher).

Table 5.3 summarizes the effects of varying the discount rate and age weights. Changes in the discount rate and age weights have little effect on the proportion of the burden in males and females. However, changes in the discount rate have an important effect on the proportion of the burden due to nonfatal outcomes (YLD), on the age distribution of the burden, and on the distribution of the burden by broad cause group. When the discount rate is set to zero, the proportion of burden due to YLD is just over a quarter of the total burden. When the discount rate is set to 3 percent, then 36 to 38 percent of the burden is due to YLD, depending on whether age weights are also applied.

Table 5.3

Comparison of the Effects of Changing the Discount Rate (r) and the Age-Weighting Factor (K) on the Composition of DALYs(r, K), 2001.

Similarly, a nonzero discount rate significantly reduces the importance of the burden of disease or injury in children. This effect is more pronounced in low- and middle-income countries, where children bear a disproportionately large share of the total burden (figure 5.5). Because of the differences in the cause structure of the disease burden by age, these effects also influence the overall distribution of DALYs by broad cause group for low- and middle-income countries. In contrast, for high-income countries, while some changes in the age distribution of the burden are apparent, the choice of discounting (and age weights) has relatively little influence on the broad cause group breakdown of the total burden of disease (figure 5.5).

Figure 5.5

Effects of Changing the Discount Rate and Age Weighting on DALYs' Broad Cause and Age Composition, 2001

The effects of introducing nonuniform age weights are generally much smaller than the effects of introducing nonzero discounting. A comparison of the discounted DALYs with and without age weighting in table 5.3 shows that the main effect is on the age distribution of the disease burden. For both high-income and low- and middle-income countries, age weights reduce the importance of the share of the burden borne by older people. In low- and middle-income countries, people aged 60 years and older suffer 21 percent of the total burden of disease and injury. This declines to 13 percent when nonuniform age weights are used. As shown in the second part of figure 5.5, the effects of discounting and age weighting on the age structure of the burden of disease largely offset each other for older ages, so that for DALYs(0,0) and DALYs(3,1) the share of the burden for those aged 60 years and older is quite similar. Overall, the importance of Group I conditions (communicable diseases, maternal and perinatal conditions, and nutritional deficiencies) is also slightly enhanced by age weighting and that of Group II conditions (noncommunicable diseases) is reduced. The effects on Group III (injuries) are relatively minor.

Figure 5.6 compares the rank order of causes contributing to the global burden of disease measured using DALYs(3,1) and DALYs(3,0). The introduction of nonuniform age weights has the most impact on neuropsychiatric disorders, such as bipolar disorder, panic disorder, and obsessive-compulsive disorder, whose prevalence is greatest in younger and middle-aged people. Age-weighted DALYs give less importance to causes whose burden falls predominantly on older ages.

Figure 5.6

Relationship between the Rank Order of Causes of the Global Burden Using DALYs(3,1) and DALYs(3,0) in 2001

Figure 5.7 compares ranks for causes measured using undiscounted DALYs(0,0) and discounted DALYs(3,0), both with uniform age weights (K = 0). A zero discount rate gives greater importance to causes with a larger burden at younger ages, such as whooping cough (pertussis) and meningitis, and lower importance to causes predominantly affecting older ages. However, the different choices of discount rates and age weights do not cause any large changes in the rank ordering of diseases and injuries, which is to a large degree anchored in absolute differences in the burden arising from large differences in prevalence and mortality levels across causes.

Figure 5.7

Relationship between the Rank Order of Causes of the Global Burden of Disease in 2001, Using Uniform Age Weights and 3 Percent Discounting and No Discounting

Table 5.4 compares DALYs(3,0) with DALYs(3,1) and DALYs(0,0) in more detail according to the second level of cause disaggregation within a group. These more detailed results confirm the major conclusions outlined earlier on the impacts of discounting and age weighting. DALYs(0,0) give greater weight to perinatal conditions (the International Classification of Diseases [ICD] category of conditions arising in the perinatal period) and respiratory infections, which primarily affect young children, than either of the discounted forms of DALYs. In contrast, the age-weighted DALYs(3,1) give more weight than DALYs(3,0) to causes that predominantly affect younger adult ages, such as neuropsychiatric conditions and injuries. DALYs(3,0) give greater weight than either DALYs(3,1) or DALYs(0,0) to causes that predominantly affect older people, such as cardiovascular diseases and cancers.

Table 5.4

Effects of Changing the Discount Rate (r) and the Age-Weighting Factor (K) on the Second-Level Cause Group Composition of DALYs(r, K), 2001 (percentages of total DALYs).

Figure 5.8 summarizes the effects of changing the discount rate and age weighting on the global rankings for the top 20 causes of the burden of disease in 2001. The left-hand column shows the rankings for causes measured using DALYs(3,0) as used for the DCPP. The middle column is for DALYs(3,1), as used by WHO to present the GBD analysis. The principal difference is that the use of DALYs(3,0) results in relatively greater importance being placed on chronic diseases of middle and older ages, such as ischemic heart disease and stroke, and somewhat lesser on HIV/AIDS, road traffic accidents, congenital anomalies, and other disorders affecting children and younger adults. Undiscounted DALYs, shown in the right-hand column, give proportionately greater importance to conditions affecting children, such as malaria and measles.

Figure 5.8

Effects of Changing the Discount Rate and Age Weighting on Global Rankings for the Top 20 Causes of the Burden of Disease, 2001

Sensitivity of Risk Factor Estimates to Variations in Key Parameter Values

Figures 5.9 to 5.11 examine the sensitivity of the burden of disease attributable to each of the 19 risk factors discussed in chapter 4 to key DALY discounting and age-weighting parameters for the world, for low-and-middle-income countries, and for high-income countries. The figures plot the attributable disease burden estimated by altering one key parameter against the baseline of DALYs(3,0) used in chapter 4. To allow comparability, all burdens attributable to risk factors are shown as a proportion of the total global or regional disease burden, which is itself estimated with the corresponding parameters.

Figure 5.9

Effects of Changes in Key DALY Parameters on Proportion of the Global Disease Burden Attributable to Risk Factors

Figure 5.11

Effects of Changes in Key DALY Parameters on Proportion of the Regional Disease Burden Attributable to Risk Factors for High-Income Countries

Figure 5.10

Effects of Changes in Key DALY Parameters on Proportion of the Regional Disease Burden Attributable to Risk Factors for Low- and Middle-Income Countries

Including age weighting, DALYs(3,1), increases the relative health consequences of risks that affect people in young and middle ages (alcohol use, illicit drug use, and unsafe sex) and lowers the relative contribution of those risks that result in death in older ages (high blood pressure, high cholesterol, low fruit and vegetable intake, overweight and obesity, physical inactivity, and smoking). In addition, the burden of disease attributable to childhood and maternal underweight increases as a proportion of the total global or regional burden of disease. This increase probably reflects a relative reduction in the total burden of those diseases that affect older adults, and hence a relative increase in the total burden of those diseases that affect young children. Because childhood and maternal underweight is a risk factor for this latter group of diseases, its attributable burden as a share of the total global or regional disease burden increases.

Removing discounting, DALYs(0,0), results in a large relative increase in the disease burden attributable to risk factors that affect young children, including childhood underweight; indoor smoke from household use of solid fuels; unsafe water, sanitation, and hygiene; vitamin A deficiency; and zinc deficiency. This is mirrored by a decrease in the disease burden attributable to the risk factors for diseases that affect adults, because the total burden of the chronic diseases affected by these risks is reduced. This effect is more noticeable in the low- and middle-income countries than in the high-income countries, where childhood mortality is low and the overall share of the disease burden is less sensitive to discounting.

Sensitivity to key DALY parameters differed in the low- and middle-income countries and the high-income countries. The burden of disease attributable to risk factors for chronic diseases in adults (high blood pressure, high cholesterol, low fruit and vegetable intake, overweight and obesity, physical inactivity, and smoking) was more sensitive to these parameters in low- and middle-income countries than in high-income countries because deaths attributable to these risks occurred at younger ages in the former. By contrast, the burden of disease attributable to alcohol was much more sensitive to age-weighting in the high-income countries because many of the hazards of this risk, especially those related to injuries and neuropsychiatric conditions, occur among younger adults in this group of countries.

Uncertainty Analysis of the Global Burden of Disease Estimates

The 2001 GBD study estimated mortality and the burden of disease for a comprehensive set of disease and injury causes and for all regions of the world, including regions with limited, incomplete, and uncertain data. To allow users of the information to assess whether the information uncertainty range is compatible with the purpose at hand, providing some analysis and guidance on levels of uncertainty is important (Murray, Mathers, and Salomon 2003). This is difficult to do, because apart from the large number and disparate nature of the data sources used (see chapter 3), information or knowledge about the quality of and potential biases in the data is often limited. This and the following sections provide an overview of initial efforts to quantify the uncertainty associated with the estimation of deaths by cause, with disability weights, and with epidemiological estimates of incidence and prevalence for GBD 2001.

Uncertainty Estimates for All-Cause Mortality and Life Expectancies

Chapters 2 and 3 describe methods for estimating life tables for each of 192 WHO member states. For those countries with vital registration data projected using time series regression models on the parameters of the logit life table system, we accounted for uncertainty around the regression coefficients by taking 1,000 draws of the parameters using the regression estimates and variance-covariance matrix of the estimators. For each of the draws, we calculated a new life table. In cases where additional sources of information provided information on the limits of uncertainty ranges around ₅q₀ (the mortality risk for children under five years of age) and ₄₅q₁₅ (the mortality risk for adults between the ages of 15 and 60), the 1,000 draws were constrained so that each life table produced estimates within these specified ranges. The range of 1,000 life tables produced by these multiple draws reflects some of the uncertainty around the projected trends in mortality, notably, the imprecise quantification of systematic changes in the logit parameters over the time period captured in available vital registration data.

For countries that did not have time series data on mortality by age and sex, the following steps were undertaken. First, point estimates and ranges around ₅q₀ and ₄₅q₁₅ for males and females were developed on a country-by-country basis as described in chapter 2 and elsewhere (Lopez and others 2002). For countries where the ₅q₀ estimate for 2001 was based on an analysis of available data sources for earlier years, such as surveys and censuses, the uncertainty range for ₅q₀ was typically dominated by the uncertainty resulting from the scatter of survey-based direct and indirect estimates of child mortality for earlier years and the uncertainty in extrapolation of the trend to 2001, rather than the sampling error associated with individual estimates. For countries without usable information on levels of adult mortality, ₄₅q₁₅ was estimated, along with uncertainty ranges, based on regression models of ₄₅q₁₅ versus ₅q₀ as observed in a set of almost 2,000 life tables judged to be of good quality. Estimated levels of child and adult mortality were then applied to a modified logit life table model, using a global standard, to estimate the full life table in 2001; HIV/AIDS deaths and war deaths were added to total mortality rates where necessary. Uncertainty ranges for HIV/AIDS were estimated as described elsewhere (Grassly and others 2004). In countries with substantial numbers of war deaths, estimates of their uncertainty range were also incorporated into the life table uncertainty analysis.

Figure 5.12 plots the final estimated uncertainty ranges for ₅q₀ and ₄₅q₁₅ for 192 WHO member states for males and females. Using Monte Carlo simulation methods, 1,000 random life tables were generated by drawing samples from normal distributions around these inputs with variances defined in reference to the defined ranges of uncertainty for ₅q₀ and ₄₅q₁₅. In countries where uncertainty around ₅q₀ and ₄₅q₁₅ was considerable because of a paucity of survey or surveillance information, the samples were drawn from wide distributions, but then constrained within prior specified maximum and minimum possible values for ₅q₀ and ₄₅q₁₅. For each country, the results of this analysis were 1,000 different simulated life tables that were then used to describe ranges around key indicators, such as life expectancy at birth and age- and sex-specific mortality rates.

Figure 5.12

Uncertainty Ranges for Child and Adult Mortality for WHO Member States, 2001

Figure 5.13 illustrates the resulting uncertainty ranges for life expectancy at birth for the World Bank regions (see map 1 inside the book's front cover). For high-income countries, where relatively complete death registration data are available, the uncertainty ranges for life expectancy at birth are around ±0.07 years for females and ±0.16 years for males. For regions such as Latin America and the Caribbean, where death registration data are available for most countries but are often incomplete, the uncertainty ranges are larger, typically around ±0.5 years. For regions with partial data on child mortality only, where adult mortality is predicted from child mortality, the uncertainty ranges are much larger, and for Sub-Saharan Africa are typically around ±5.0 years. Across the regions, this translates to considerable heterogeneity in uncertainty ranges for life expectancies at birth and for estimates of all-cause mortality levels.

Figure 5.13

Uncertainty in Average Life Expectancy at Birth, by Sex and DCPP Region, 2001

Uncertainty Estimates for Regional Mortality by Cause

We use a simulation approach to estimate uncertainty ranges for deaths by cause for GBD 2001. These uncertainty ranges take into account uncertainty in the expected number of total deaths (life table uncertainty); uncertainty in the estimated proportions of broad cause Groups I, II, and III (where relevant for countries without vital registration data or with incomplete coverage); uncertainty in the diagnosis of underlying cause; uncertainty arising from the miscoding of underlying cause; and fundamental Poisson uncertainty in the estimated death rate arising from the observation of a finite number of deaths in a fixed time interval. This analysis was carried out by country.

As described in the previous section, a total of 1,000 life tables were developed for each of the 192 WHO member states to quantify the uncertainty distribution of key life table parameters. We then used the age-specific mortality rates from the 1,000 life tables to estimate the uncertainty distribution for the expected number of total deaths for 2001. Uncertainty in the underlying cause attribution was estimated in terms of the relative uncertainty of the proportion of deaths due to each specific cause. The estimates of cause-specific relative uncertainty were based on advice from nosologists and experts in the area of cross-country mortality analysis on the general levels of uncertainty in the attribution of specific causes within Groups I, II and III, together with detailed advice on particular causes with known higher levels of attribution uncertainty according to the ICD. Information on the latter causes derives from comparative analyses across countries, across time periods, and across ICD revisions, together with information from a variety of country-specific coding quality studies involving recoding or dual coding of deaths and comparisons with the original attributed causes.

Based on this advice, for cause distributions derived from vital registration data coded using ICD-10 (the 10th edition of the ICD), we generally assumed that diagnostic uncertainty and coding uncertainty together resulted in approximate relative 95 percent uncertainty ranges of ±3 percent for Group I causes, ±7 percent for Group II causes, and ±2 percent for Group III causes. Larger uncertainty ranges were assumed for specific causes known to have greater levels of diagnostic or coding error; for WHO member states that have been using ICD-10 coding for less than three years; for member states still using ICD-9 coding (with particular attention to causes where coding rates between ICD-9 and ICD-10 are known to differ); and for member states using other cause coding systems or verbal autopsy methods, or where cause of death models were used to estimate death distributions across Groups I, II, and III. In the latter case, an additional relative uncertainty for the estimation of Group I, II, and III proportions was estimated from the prediction uncertainty ranges associated with the CodMod regression model (see chapter 3).

Uncertainty estimates also took into account the redistribution of general, cancer, cardiovascular, and injury ill-defined cause codes and incomplete coverage of vital registration data. The relative uncertainty range for each cause was then combined with the estimated uncertainty distribution for all-cause mortality to provide estimates of the uncertainty distributions of cause-specific mortality estimates for all ages and both sexes at the country level.

The analysis of uncertainty in cause of death estimates at the country level thus combines quantitative, country-specific information on uncertainty in all-cause mortality and, in some cases, also in major cause group distributions, together with quantified average relative uncertainty ranges for specific cause attributions based on expert advice and adjusted for specific causes and for country-specific information on data sources, type of cause information available, and indicators of data quality. Here we summarize these uncertainty estimates at the regional level to provide some indication of the range of uncertainty for cause-specific mortality estimates across the World Bank regions as reported in chapter 3. This requires some additional assumptions about the cross-country correlations in uncertainty distributions.

At one extreme, if all country-level estimates have uncorrelated uncertainty because they are derived from completely independent data sets, then even with high levels of uncertainty at the country level, there will be considerably less uncertainty at the regional or global level. At the other extreme, if the uncertainty in country-level estimates for a cause derives predominantly from a single source or assumption, for example, about the case fatality rate of malaria, that is applied in deriving each country estimate, then the uncertainty distributions will be highly correlated and the regional uncertainty will be of a similar relative magnitude as each of the country uncertainty ranges.

With respect to cross-country correlations for life table and cause of death estimates based on death registration data, we assumed that even though life table uncertainties would be uncorrelated, relative uncertainties in cause of death attribution for specific causes were likely to be correlated because of systematic errors in ICD coding practices across countries for specific causes. We arbitrarily set this correlation at 25 percent. For life table estimates not based on death registration data, we assumed some correlation in uncertainty because even though estimates of childhood mortality came from independent sources, the method for determining adult mortality was similar across countries. We therefore set this correlation at 50 percent.

We assumed that cross-country correlation for relative cause of death uncertainties in the absence of vital registration data would vary depending on the method of causal attribution. Attributions based on some data and country-level predictions or assumptions were assumed to have less correlation than those based simply on regional patterns. In the case of the latter, we set the correlation at 75 percent; in the former, we set it at 50 percent or 25 percent depending on the degree of independence of the underlying inputs. We assumed greater independence for cancers and maternal conditions and less independence for tuberculosis, HIV/AIDS, sexually transmitted infections, diarrheal diseases, childhood-cluster diseases, meningitis, tropical-cluster diseases, lower respiratory infections, and perinatal conditions. We set cross-country correlations for war and drug use disorders at 25 percent for all countries, including those with vital registration data, to reflect the different methods used to obtain estimates for these causes.

We derived 95 percent uncertainty intervals by cause for World Bank regions in 2001 from the foregoing assumptions using simulation methods. We constructed 1,000 draws with the required correlation structure between countries separately for each cause, and the 2.5th percentile and the 97.5th percentile of expected deaths were taken to be the lower and upper bounds of the corresponding uncertainty interval. Note that these ranges provide guidance on uncertainty in the underlying cause-specific death rates, as expressed in terms of expected deaths in the population in 2001. Uncertainty in population estimates is not included, and the uncertainty ranges relate to underlying death rates, not to the numbers of deaths that actually occurred in 2001.

Table 5.5 summarizes regional uncertainty ranges for total estimated deaths for selected causes for 2001. Uncertainty ranges for estimated all-cause deaths increase from around ±1 percent for high-income countries to (−15 percent, +21 percent) for Sub-Saharan Africa. For specific causes, regional uncertainty ranges are generally higher, except for those causes for which cause-specific mortality estimates were available based on country-specific data from cause-specific surveillance systems (see chapter 3). For example, the uncertainty range for HIV/AIDS deaths in Sub-Saharan Africa is somewhat narrower than the all-cause mortality range, reflecting the substantial database for these estimates from antenatal clinic surveillance data and seroprevalence surveys, albeit still with considerable uncertainty arising from issues around the representativeness of the available data and the assumptions relating to survival and case fatality rates (Grassly and others 2004).

Table 5.5

Estimated Total Deaths and 95 Percent Uncertainty Ranges for Selected Causes, by Region, 2001 (thousands).

For most other causes, uncertainty ranges are greater than for the all-cause mortality estimates, because additional uncertainty is associated with cause attribution, as described earlier. For example, the relative uncertainty ranges for ischemic heart disease range from around ±12 percent for high-income countries to (−24 percent, +34 percent) for Sub-Saharan Africa (table 5.5). While the uncertainty range for high-income countries may seem surprisingly large, it reflects not only uncertainty in overall mortality levels, but also uncertainty in the attribution of underlying cause and in the attribution of causes coded to cancer, cardiovascular, and injury ill-defined cause codes or to the ICD chapter for symptoms, signs, and ill-defined conditions. The proportion of deaths coded to these two groups of causes is surprisingly large for some high-income countries (Mathers and others 2005).

Figure 5.14 illustrates the relative insensitivity of the regional uncertainty ranges to the assumptions about cross-country correlation of uncertainty. The broad patterns of the uncertainty ranges for causes across regions provide useful additional guidance to policy makers in interpreting regional differences, particularly in judging which policy questions these estimates can help address and for which the uncertainty levels are too great to allow useful inferences.

Figure 5.14

Sensitivity of Uncertainty Ranges to Changes in Between-Country Correlation Assumptions

Uncertainty in Disability Weights

Although health state valuations are often treated as value parameters without uncertainty, we argue that unlike social choices such as the discount rate, no clear normative basis is available on which to assign relative values to the different dimensions of health that collectively define the universe of health states. Ideally, these values should be derived from empirical data among representative populations (Salomon and others 2003). Numerous challenges are associated with population-based data collection for the purpose of health state valuations, particularly given the broad scope of valuations required for a comprehensive assessment of disease burden. As a result, the current empirical base for disability weights remains well short of this ideal. Given the limitations in currently available information, an examination of the contribution of uncertainty around health state valuations to overall uncertainty in burden of disease estimates measured using YLD or DALYs is useful.

Conceptually, the basis for assigning disability weights to specific sequelae requires an understanding of (a) the distribution of health states among those living with the particular sequelae, where a health state is defined by the levels on the various dimensions that constitute health; and (b) a valuation function that provides a systematic way to aggregate across multiple dimensions of health in order to arrive at a single index value that captures the overall level of health associated with a given health state (Salomon and others 2003). While disability weights may vary across regions because of variation in either component, we have proposed elsewhere that for purposes of standardization and global comparisons, computing disability weights based on an average global valuation function is the most appropriate approach (Murray and others 2002). The need to understand variation in the distribution of health states among people living with given sequelae highlights the critical link between the epidemiological inputs of burden and the estimated disability weights.

In this section, we undertake a first analysis of the contribution of uncertainty in disability weights to uncertainty in the GBD DALY estimates. Given that the current set of disability weights reflects the accumulation of a wide array of different empirical inputs rather than the result of the comprehensive and standardized approach defined earlier as the ideal, we operationalize our analysis of uncertainty in terms of error around the disability weights by sequelae rather than in terms of the uncertainty arising from the constituent components, that is, the health state distributions and the valuation function. Based on this approach, the results offer guidance on the sensitivity of burden estimates to a certain degree of uncertainty around disability weights, but do not necessarily capture all sources of uncertainty and their covariance. As noted earlier, certain specific measurement methods for eliciting health state valuations, for example, the person trade-off or standard gamble, may have important normative implications that are orthogonal to the assessment of health levels. However, undertaking a sensitivity analysis that focuses on a specific measurement approach is not appropriate here, because the weights currently used in the GBD estimates have been derived from the synthesis of multiple data sources rather than from a single measurement method.

Because of the natural constraints on the range of values that disability weights may assume, we have incorporated normal distributions with constant variance in the space of the logit of disability weights. The logit transformation is given by logit(x) = ln[x/(1 − x)]. By allowing for normally distributed error in logit space, ranges in the natural space of valuations are constrained to fall between 0 and 1. We chose a value of 0.6 for the standard deviation of the logits, based on the standard deviations observed across the mean valuations by country for an array of conditions included in the WHO multicountry survey study from 2000–1 (with valuation modules implemented in 14 countries) (Salomon and others 2003; Ustun and others 2003). Although the variability in country means may reflect a range of different factors, including the possibility of real valuation heterogeneity, we use this value to approximate the average level of uncertainty around the set of disability weights used in the GBD study. A constant value in logit space yields absolute ranges that widen at the midpoint of the interval and narrow as the disability weight approaches 0 or 1 (figure 5.15). In relative terms, the uncertainty is greatest for the smallest disability weights and narrows as more severe weights are attained (figure 5.16).

Figure 5.15

Assumed 95 Percent Uncertainty Ranges for Disability Weights Based on Constant Variance Distribution for Logit of Disability Weight

Figure 5.16

Relative 95 Percent Uncertainty Ranges for Disability Weights Based on the Assumption of a Constant Variance Distribution for Logit of Disability Weight across All Disability Weights

To trace the implications of this uncertainty through to the calculation of DALYs(3,0) used in the DCPP, we took 100 draws from each of 622 independent normal distributions with a mean of 0 and a standard deviation of 0.6 (for the 622 sequelae included in the calculations). For each of the sequelae we applied a given sampled value as a perturbation of all age, sex, and region estimates of logit-transformed disability weights pertaining to that sequela, and recomputed YLD(3,0) based on the disability weight plus the random perturbation (after reversing the transformation for the sum). We estimated uncertainty ranges by taking the 2.5th and 97.5th percentiles across the 100 values of the various quantities of interest based on the random draws of error around the disability weights. This method implies the simplifying assumption that errors are uncorrelated between sequelae but perfectly correlated for all estimates within a sequela. In addition to YLD(3,0) numbers, we recomputed YLD ranks resulting from each set of sampled values, and also calculated DALY numbers and ranks by adding each YLD(3,0) draw to constant YLL(3,0) estimates by sequela. Our intent is only to provide an indication of the sensitivity of the YLD and DALY results to disability weight uncertainty. We did not attempt either to carry out a full empirically based analysis of this issue or to combine this source of uncertainty with mortality uncertainty and uncertainty in epidemiological estimates to give a comprehensive uncertainty analysis for the DALY estimates.

Table 5.6 presents the resulting uncertainty ranges for YLD and DALYs by cause for low- and middle-income countries. As would be expected, DALY uncertainty ranges due to disability weight uncertainty are generally largest for those causes dominated by YLD and smallest for those causes dominated by YLL. Uncertainty ranges are also large for those YLD-dominated causes with high prevalence and low disability weight (with high relative uncertainty), such as hearing loss and anemia. Figure 5.17 summarizes in graphical form the uncertainty in total DALYs for low- and middle income countries for the 20 highest-ranked causes.

Table 5.6

Estimated 95 Percent Uncertainty Ranges for YLD and DALYs Arising from Uncertainty in Disability Weights for Selected Causes for Low- and Middle-Income Countries, 2001 (thousands).

Figure 5.17

Estimated 95 Percent Uncertainty in Total DALYs(3,0) Due to Uncertainty in Estimation of Disability Weights, Top 20 Causes, Low- and Middle-Income Countries

Table 5.7 presents the resulting 95 percent uncertainty ranges for the 40 leading causes of the burden of disease in low- and middle-income countries. Taking into account uncertainty in disability weights does not result in significant uncertainty in the ranking of the top four causes, with only the third (ischemic heart disease) and fourth (HIV/AIDS) possibly changing places. The total estimated DALYs for these two causes differ by less than 2 percent, so this is not surprising. Among the other top 10 causes, the disability weight uncertainty could change the rankings of individual causes by up to two ranks, with the exception of depressive disorders, which could change by up to four ranks. This reflects both the high relative uncertainty in the disability weight for mild depression and the fact that YLD are responsible for almost all depression DALYs. Among conditions ranked 20th to 30th in table 5.7, uncertainty ranges for most ranks are relatively narrow with the exception of nonfatal, high-prevalence conditions such as hearing loss and osteoarthritis, where the uncertainty in rank may be as much as ± 15 places.

Table 5.7

Estimated 95 Percent Uncertainty Ranges Arising from Uncertainty in Disability Weights for the Top 40 Causes of the Burden of Disease in Low- and Middle-Income Countries, 2001.

This analysis confirms the importance of efforts to improve the measurement of disability weights for health states close to full health, that is, with disability weights close to zero, particularly for health states with high prevalence in many populations, such as mild to moderate sense organ impairment or mild to moderate anemia. Unfortunately, most of the available choice-based or trade-off methods involving comparison in some form with death or survival present greater cognitive challenges to respondents when applied to health states close to full health.

Uncertainty Arising from Epidemiological Estimates

Uncertainty in YLD estimates is mainly determined by the uncertainty in (a) epidemiological estimates for the prevalence and/or incidence of disability associated with specific causes or cause groups; and (b) disability weights arising from uncertainty in health state valuations and, in some cases, also in the disability severity distribution associated with a condition.

For a subset of the GBD causes, analysts carrying out reviews and analyses for the estimation of YLD also estimated levels of uncertainty in regional prevalences. These assessments took into account not only typical levels of measurement error in the input data sets, but also expert judgment on the degree of uncertainty arising from the lack of representativeness of the available data for each region. The resulting uncertainty ranges vary considerably across causes, ranging from relatively certain estimates for some causes such as polio, for which intensive surveillance systems are in place, to highly uncertain estimates for other causes such as osteoarthritis, where for some regions not a single usable dataset was found, and where for others the latest available data were decades old. The summary tables provided in chapter 3 for numbers of data sources used for YLD estimates by cause and region provide one indication of the relative uncertainty associated with YLD estimates for different causes.

For some causes, such as stroke and ischemic heart disease, YLD estimates were essentially derived from estimates of cause-specific mortality by means of models of regional variations in case fatality rates. In such cases, YLD uncertainty will be significantly higher than the uncertainty associated with cause-specific mortality estimates given the considerable uncertainty in case fatality rates for most low-and middle-income countries and in models used to infer the burden of nonfatal disease from mortality. YLD uncertainty will generally be greater than YLL uncertainty, and will also vary across causes according to both the typical uncertainty associated with the measurement of incidence or prevalence according to GBD case definitions and with the number and representativeness of available studies. For a subset of 16 major causes of YLD for which analysts estimated indicative uncertainty ranges, the typical uncertainty for regional prevalence estimates ranged from ± 10 percent to ± 90 percent, with a median value of ± 41 percent. Uncertainty ranges were generally higher for low- and middle-income countries than for high-income countries.

Uncertainty in the Disease Burden Attributable to Risk Factors

The assessments of the disease burden attributable to selected risk factors reported in chapter 4 are affected by additional sources of uncertainty, beyond the uncertainty in DALY estimates for specific disease and injury outcomes discussed earlier. A full uncertainty analysis of such burden estimates has not yet been carried out, but would involve assessment of the following additional types of uncertainty:

• uncertainty in the estimated distributions of population risk exposure;
• uncertainty in estimates of relative risks for cause-specific mortality and incidence associated with specific exposures, for which a significant source of uncertainty is the extrapolation of relative risks measured at other ages to older age groups;
• uncertainty associated with estimating joint effects of risk factors.

Uncertainty in exposure and in both the existence and magnitude of hazardous effect always affects quantitative risk assessment. In one taxonomy, risk assessment uncertainty can be divided into parameter uncertainty and model uncertainty (National Research Council 1994). Parameter uncertainty is often quantifiable using random variable methods, for example, uncertainty due to sample size or measurement error. Model uncertainty is due to gaps in scientific theory, measurement technology, and data. It includes uncertainty in causal relationships or the form of the exposure-response relationship (for instance, threshold versus continuous or linear versus nonlinear), the level of bias in measurement, and so on. Defined broadly, model uncertainty also includes extrapolation of exposure or hazard from one population to another. Model uncertainty dominates uncertainty in risk assessment, a result of difficulty in carrying out direct studies on exposure, hazard, and background disease burden. This has motivated innovative assumptions and extrapolations even in the case of the most widely studied risk factors like smoking (Peto and others 1992).

Uncertainty around disease causation (Evans 1978; Hill 1965) was, in practice, secondary to uncertainty around hazard size, for example, relative risk, because when causality was uncertain, the estimates of relative risk needed for risk assessment were also unknown or uncertain. For example, whether the relationships between physical inactivity and lower back pain or between alcohol and violence are causal has equivalent questions on the magnitude of hazard of each risk for the disease outcome. Collective scientific knowledge from disciplines such as social and behavioral sciences, physiology and neuroscience, and epidemiology would confirm the possibility of a causal relationship in the foregoing cases, but would shift the uncertainty to hazard size. As a result, for some risk factors, we could only quantify the contribution to a subset of disease outcomes because epidemiological studies did not provide enough information for all risk factor and disease pairs, even when the causal relationship was believed or suspected.

Estimates of hazard in individual epidemiological studies were adjusted for confounding as much as possible. Extrapolation of hazard from a limited number of studies to other populations has received less attention. While the robustness of proportional measures of risk has been confirmed for more proximal factors in studies across populations (Eastern Stroke and Coronary Heart Disease Collaborative Research Group 1998; Horton 2000; Law, Wald, and Thompson 1994), their extrapolation is an important source of uncertainty for more distal risks (such as childhood sexual abuse) or those whose effects are heterogeneous (for example, alcohol and injuries versus alcohol and cancer).

Direct exposure data for many risk factors are limited both because of measurement difficulties and because of underinvestment in risk factor surveillance. To allow maximum use of available data, such risk factors were represented using indirect or aggregate indicators, for instance, smoking impact ratio for accumulated hazards of smoking, weight-for-age for childhood undernutrition, and use of solid fuels for indoor air pollution. Furthermore, for some risks multiple data sources allowed limiting the range of exposure estimates. For example, in the absence of alcohol surveys, information on total alcohol production, trade, and unrecorded consumption provided upper bounds on the fraction of the population that would be in the highest consumption category. Finally, some of the risk factors examined in chapter 4 were represented using continuous exposure variables such as high blood pressure. Others used categorical variables, for example, indoor smoke from household use of solid fuels, childhood underweight, and physical inactivity, even though the health effects occur along a continuum. This choice reflected the availability of exposure data and hazard estimates in categories. In such cases, the contribution to disease within the baseline category would not have been captured.

In addition to uncertainty in exposure and hazard, the uncertainty of estimated population attributable fractions (PAFs) is determined by the analytical properties of the PAF relationship. In particular, the PAF relationship is an increasing concave function of relative risk and exposure level, approaching 100 percent asymptotically, that is, the rate of increase declines with increasing relative risk or prevalence (figure 5.18). Therefore, if a risk factor or group of risk factors individually or jointly account for large fractions of specific diseases, the PAFs are more robust to uncertainty in inputs. Finally, there is uncertainty in mortality and disease burden estimates to which the estimated PAF are applied (see the previous section).

Figure 5.18

PAF Sensitivity to Exposure and Relative Risks

The findings in chapter 4 should therefore be considered within the context of limited available data and viewed as subject to uncertainty, which varies across risk factors and geographical regions. For further discussion of sources and quantification of uncertainty for specific risk factors see Ezzati and others (2004).

Discussion

As described in chapter 3, the data requirements for adequate measurements of the global burden of disease are substantial and include information about age at death, cause of death, age-specific incidence of diseases and injuries, typical duration of life lived with the sequelae of diseases and injuries, and some quantification of the severity of disability assessed according to a common framework. While the ethical, philosophical, and conceptual issues involved in quantifying states of health other than perfect health are still very much a matter of debate, a substantial body of empirical evidence on the variations across individuals and populations in health state valuations is now available.

We have shown in this chapter that the distribution of the global burden of disease and the overall rankings of various conditions in terms of their contribution to it are largely insensitive to alternative assumptions about the discount rate and age weighting. The major effect of discounting and age weighting is to enhance the importance of neuropsychiatric conditions and sexually transmitted infections. While disease rankings are relatively unaffected, the share of the burden due to disability, the age distribution of the burden, and the distribution of the burden by broad cause group are sensitive to the discount rate but less affected by age weighting.

When compared with the discounted and age-weighted DALY used in the 1990 GBD study and the WHO updates for 2000–2, the DCPP's use of discounted but not age-weighted DALYs results in somewhat more weight being given to the chronic diseases of older ages and somewhat less weight being given to mental disorders and injuries, which affect younger adults disproportionately. Of the value choices incorporated into the standard DALYs(3,1), the nonuniform age weights have been the most controversial. Apart from the DCPP, a number of national burden of disease studies, including those in Australia and Canada (Mathers, Vos, and Stevenson 1999; Public Health Agency of Canada 2005), have chosen not to apply the nonuniform age weights, presumably on equity grounds. In contrast, some investigators concerned with the inequitable health burden of the low- and middle-income countries have argued for ignoring all deaths over a certain age on the grounds that they are not premature—an extreme form of age weighting (Williams 1997). Chapter 6 presents some empirical evidence in making the case for a stronger form of age weighting for infants and younger children, that is, age weights that depart further from unity than the standard age weighting used in the DALY.

Although the choices for discounting and age weighting do affect the cause and age distributions of the burden of disease to some extent, and the results of specific cost-effectiveness studies may be even more sensitive to these choices, we conclude that the uncertainty of the underlying epidemiological choices is vastly more consequential than these social preferences when interpreting the results of burden of disease analysis. The validity and reliability, and hence the utility, of burden of disease studies for public policy depend much more strongly on the quality and availability of the underlying epidemiological data.

The GBD study has been criticized for making estimates of mortality and burden of disease for regions with limited, incomplete, and uncertain data (Cooper and others 1998; Gupta, Sankaranarayanan, and Ferlay 1994). Murray and Lopez describe the GBD approach as a "`meta-synthesis,' or in other words, the construction of a comprehensive and comparable view of health problems using all available sources of information" (Murray and Lopez 1996b, p. 289). The incorporation of many types of information about a comprehensive set of causes of death and disability results in estimates that are much less likely to be biased than those that emerge from an examination of specific health conditions in isolation. It also avoids the tendency to assume that if no data are available or the data are highly uncertain, then there is no disease burden.

We argue that including uncertain results (with quantified uncertainty to the extent possible) is far preferable than leaving blank cells in tables intended to provide policy makers with an overall assessment of the burden of disease in populations. We maintain that providing large volumes of unsynthesized, biased, and incomplete data relating to population health does not generally allow policy makers to make the best use of such information. Unless they have considerable analytic resources of their own, the unsynthesized products of the research enterprise are of little help to decision makers, who will often then resort to decisions on the basis of ideology, of their own beliefs about what is important, or of political imperatives.

The quantities of interest for the GBD study are the underlying rates of incidence, remission, and mortality for defined causes for whole populations for a specified time period, and the assessment of these often requires synthesizing data from multiple studies or making adjustments for biases in relation to population, age groups, or time periods. A major source of uncertainty for the GBD estimates is the uncertainty associated with extrapolating from one or more subgroups to a regional population. For example, how representative of the incidence and prevalence patterns of dementia in Sub-Saharan Africa are two or three population-representative studies of rural or urban populations in specific regions of specific countries? The uncertainty associated with extrapolating from a set of studies in subpopulations to the regional population is related to potential systematic (selection) biases and is much more difficult to quantify than the uncertainty associated with stochastic variation due to sample size or measurement error.

Estimates of deaths from specific causes undergo continual revision as new data and syntheses become available, yet drawing a time cutoff is a necessary (if somewhat arbitrary) condition for preparing any volume such as this which reports comprehensive and consistent global and regional estimates of deaths and burden of disease (see also annex 6C). During 2001 WHO established the Child Health Epidemiology Reference Group (CHERG) to review and synthesise data on cause of deaths under age 5. While early CHERG results contributed substantially to the GBD analyses in this volume, much of their work became available well after the cutoff date for this publication. While CHERG has published revised estimates of the distribution of child deaths by cause (Bryce and others 2005), based on recent comprehensive reviews of epidemiological data, these analyses used cause categories not consistent with the GBD (including use of incompatible cause categories for neonatal and other child deaths), fewer cause categories than the GBD, and left study deaths assigned to ill-defined categories in the `Other' category. Additionally, at the date of writing, the CHERG evidence has not been brought into the GBD analytic and consistency framework, involving consistent mapping to causal categories and checking of internal consistency between incidence, prevalence and mortality estimates for specific causes.

To the extent that they can be compared with the GBD 2001 estimates, the WHO/CHERG estimates at the global level are differ substantially for tetanus (46% higher), lower respiratory infections (56% higher), and are somewhat lower for measles, malaria, low birthweight and noncommunicable diseases. It is not possible at this stage, to conclude whether or how much the WHO/CHERG analyses would modify the GBD 2001 results reported in this volume, when they are properly brought into the GBD analytic framework. However, they do give some indications that new evidence is becoming available for child deaths, and that uncertainty ranges for GBD estimates of child deaths may be greater for some causes than indicated by the analyses presented in this chapter.

The 1990 GBD study and GBD 2001 were both meta-syntheses of the available data, using the best models and tools available at the time, whose primary aims were to provide a comprehensive assessment of the current burden of disease. The assessment of trends between 1990 and 2001 is a much more difficult task, as discussed in chapter 2. The comparability of best point in time estimates is difficult to assess given changes in both the availability of data and in the methods used to synthesize those data for many of the causes. Murray, Mathers, and Salomon (2003) discuss this issue in more detail and conclude that to assess change or evaluate programs, extrapolating current levels of burden of disease from past measurements is inadequate, and that the assessment must include measurements carried out at both points in time or explicit measurement of the relevant trends or rates of change.

Conclusions

The 2001 GBD study uses a summary measure of population health, the DALY, that explicitly incorporates several important social values. This has the advantage that the effects of changing preferences can be readily explored through sensitivity analysis, as illustrated in this chapter. Another advantage of the burden of disease approach is that it entails a data audit, whereby the completeness, reliability, and consistency of routinely collected data are assessed and critical gaps in health data collection are identified. One implication is that periodic quality assessments of, say, routine cause of death data are needed to ensure their continued relevance and reliability for public policy (Mathers and others 2005). Another is the need for a more rational assessment of priority data for the health care sector that places greater emphasis on data collection and data linkage to facilitate burden of disease studies rather than on routine collection of statistics of limited relevance to public health. The burden of disease framework, based on the estimated distribution and duration of health states resulting from incident cases, would benefit greatly from wider availability of linked data sets on health outcomes and further longitudinal research into health state transition probabilities following on from specific disease or injury causes (Kelman and Bass 2002).

A major advance with GBD 2001 has been the systematic, though as yet incomplete, attempt to quantify uncertainty in both national and global assessments of the disease burden. This uncertainty must be taken into account when making cross-national comparisons, and needs to be carefully communicated and interpreted by epidemiologists and policy makers alike. Estimates of mortality in countries without functioning vital registration systems for causes of death will always be substantially more uncertain than those derived from systems where all deaths are registered and medically certified. The same may be said for the quantification of disability due to various conditions, where the gaps in data availability across countries are likely to be even more extreme than for mortality.

Despite the progress of the past decade, the incremental gains in advancing knowledge and understanding of global descriptive epidemiology have been modest. A globally coordinated research and development effort is urgently needed to devise and implement cost-effective approaches to data collection and analysis in poor countries that are targeted to their health development needs, and that can routinely yield comparable information of sufficient quality to establish how the disease and risk factor burden is changing in populations (Murray, Lopez, and Wibulpolprasert 2004).

Much can be learned about the health of populations from relatively modest investments in sample registration systems, provided these are designed to reliably measure the causes of death in sample areas and have sufficient resources to do so. China's Disease Surveillance Points system is a good example of what can be done to improve knowledge about disease and injury control priorities in low-income countries at a modest cost (Lopez 1998; Yang and others 2005). Greater investments in getting the descriptive epidemiology of diseases and injuries correct in poor countries will do vastly more to reduce uncertainty in disease burden assessments than philosophical debate about the appropriateness of social value choices. Just as the production of global and regional estimates should not create the impression that the descriptive epidemiology of disease and injury is reliably known, so the uncertainties around these estimates must not create the impression that not enough is known reliably enough to usefully inform health priorities and programs. Health intelligence is an essential ingredient of the health development process. Those engaged in collecting, analyzing, and disseminating population health information have a responsibility to develop this evidence base using novel methods that communicate what we do know, as well, if not more convincingly, than what we do not know.

Information for policy purposes will never be perfect, but good policy makers will want to benefit from all available information to guide priority setting and action. We might well take solace in the comments of a prominent medical statistician who once cautioned that "Making the best the enemy of the good is a sure way to hinder any statistical progress. The scientific purist who will wait for medical statistics until they are nosologically exact is no wiser than Horace's rustic waiting for the river to flow away" (Greenwood 1948, p. 28).

Acknowledgments

We wish to acknowledge stimulating discussions with and advice from Christopher J. L. Murray in the development of our approach and analytic tools relating to the analysis and estimation of uncertainty in mortality and GBD estimates. We also acknowledge the assistance of staff of the former Global Program on Evidence for Health Policy at WHO, who helped substantially in the analysis of uncertainty, namely: Doris Ma Fat, Brodie Ferguson, Mie Inoue, and Niels Tomijima. Finally, we thank two referees for extremely useful comments and suggestions that have substantially improved this chapter.

References

1. Anand S., Hanson K. Disability-Adjusted Life Years: A Critical Review. Journal of Health Economics. 1997;16(6):685–702. [PubMed: 10176779]
2. Barendregt J. J., Bonneux L., van der Maas P. J. DALYs: The Age-Weights on Balance. Bulletin of the World Health Organization. 1996;74(4):439–43. [PMC free article: PMC2486889] [PubMed: 8823967]
3. Bobadilla, J. L. 1996. "Priority Setting and Cost Effectiveness." In Health Policy and Systems Development: An Agenda for Research, ed. K Janovsky, 43–60. Geneva: World Health Organization.
4. Bryce J., Boschi-Pinto C., Shibuya K., Black R. E. the WHO Child Health Epidemiology Reference Group. WHO Estimates of the causes of death in children. Lancet. 2005;365(9465):1147–1152. [PubMed: 15794969]
5. Cooper R. S., Osotimehin B., Kaufman J. S., Forrester T. Disease Burden in Sub-Saharan Africa: What Should We Conclude in the Absence of Data? Lancet. 1998;351(9097):208–10. [PubMed: 9449884]
6. Dasgupta, P., K.-G. Mäler, and S. Barrett. 1999. "Intergenerational Equity, Social Discount Rates, and Global Warming." In Discounting and Intergenerational Equity, ed. P. R. Portney and J. P. Weyant, 51–77. Washington, DC: Resources for the Future.
7. Eastern Stroke Coronary Heart Disease Collaborative Research Group. Blood Pressure, Cholesterol, and Stroke in Eastern Asia. Lancet. 1998;352(9143):1801–7. [PubMed: 9851379]
8. Evans A. S. Causation and Disease: A Chronological Journey. American Journal of Epidemiology. 1978;108(4):248–55. [PubMed: 727194]
9. Ezzati, M., A. D. Lopez, A. Rodgers, and C. J. L Murray. 2004. Comparative Quantification of Health Risks: Global and Regional Burden of Disease Attributable to Several Major Risk Factors. Geneva: World Health Organization.
10. Fox-Rushby, J. A. 2002. Disability Adjusted Life Years (DALYS) for Decision-Making? An Overview of the Literature. London: Office of Health Economics.
11. Gold, M. R., J. E. Siegel, M. C. Weinstein, and L. B. Russell. 1996. Cost-Effectiveness in Health and Medicine. New York: Oxford University Press.
12. Grassly N. C., Morgan M., Walker N., Garnett G. P., Stanecki K. A., Stover J., Brown T., Ghys P. D. Uncertainty in Estimates of HIV/AIDS: The Estimation and Application of Plausibility Bounds. Sexually Transmitted Infections. 2004;80(Suppl 1):S31–S38. [PMC free article: PMC1765838] [PubMed: 15249697]
13. Greenwood, M. 1948. "Medical Statistics from Graunt to Farr: The Fitzpatrick Lectures for the Years 1941 and 1943." Cambridge, U.K.: Cambridge University Press.
14. Gupta P., Sankaranarayanan R., Ferlay J. Cancer Deaths in India: Is the Model-Based Approach Valid? Bulletin of the World Health Organization. 1994;72(6):943–4. [PMC free article: PMC2486736] [PubMed: 7867141]
15. Hill A. B. The Environment and Disease: Association or Causation? Proceedings of the Royal Society of Medicine. 1965;58:295–300. [PMC free article: PMC1898525] [PubMed: 14283879]
16. Horton R. Common Sense and Figures: The Rhetoric of Validity in Medicine (Bradford Hill Memorial Lecture 1999). Statistics in Medicine. 2000;19(23):3149–64. [PubMed: 11113950]
17. Kelman C. W., Bass A. J. Research Use of Linked Health Data: A Best Practice Protocol. Australian and New Zealand Journal of Public Health. 2002;26(3):251–5. [PubMed: 12141621]
18. King G., Tomz M., Wittenberg J. Making the Most of Statistical Analyses: Improving Interpretation and Presentation. American Journal of Political Science. 2000;44(2):341–55.
19. Kneese, A. V. 1999. "The Faustian Bargain." In The RFF Reader in Environment and Resource Management, ed. W. D. Oates, 55–60. Washington, DC: Resources for the Future.
20. Kruijshaar M., Barendregt J., Hoeymans N. The Use of Models in the Estimation of Disease Epidemiology. Bulletin of the World Health Organization. 2002;80(8):622–8. [PMC free article: PMC2567595] [PubMed: 12219152]
21. Law M. R., Wald N. J., Thompson S. G. By How Much and How Quickly Does Reduction in Serum Cholesterol Concentration Lower Risk of Ischaemic Heart Disease? British Medical Journal. 1994;308(6925):367–73. [PMC free article: PMC2539460] [PubMed: 8043072]
22. Lopez A. D. Counting the Dead in China. British Medical Journal. 1998;317(7170):1399–400. [PMC free article: PMC1114289] [PubMed: 9822388]
23. Lopez, A. D., O. Ahmad, M. Guillot, B. Ferguson, J. Salomon, C. J. L. Murray, and K. H. Hill. 2002. World Mortality in 2000: Life Tables for 191 Countries. Geneva: World Health Organization.
24. Mahapatra, P. 2001. "Local Age Preference, Age Weight, and Discounting Parameters for Computation of DALYs." In Estimating National Burden of Disease: The Burden of Disease in Andhra Pradesh 1990s, ed. P. Mahapatra. Hyderabad, India: Institute of Health Systems.
25. Mathers C. D., Ma Fat D., Inoue M., Rao C., Lopez A. D. Counting the Dead and What They Died from: An Assessment of the Global Status of Cause of Death Data. Bulletin of the World Health Organization. 2005;83(3):171–7. [PMC free article: PMC2624200] [PubMed: 15798840]
26. Mathers C. D., L Murray C. J., Lopez A. D. Epidemiological Evidence: Improving Validity through Consistency Analysis. Bulletin of the World Health Organization. 2002;80(8):611. [PMC free article: PMC2567596] [PubMed: 12219149]
27. Mathers, C. D., T. Vos, and C. Stevenson. 1999. The Burden of Disease and Injury in Australia. Canberra: Australian Institute of Health and Welfare. [PMC free article: PMC2566696] [PubMed: 11731817]
28. Morgan, M. G., and M. Henrion. 1990. Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis. Cambridge, U.K.: Cambridge University Press.
29. Murray, C. J. L. 1996."Rethinking DALYs." In The Global Burden of Disease, ed. C. J. L. Murray and A. D. Lopez, 1–98. Vol. 1 of Global Burden of Disease and Injury Series. Cambridge, MA: Harvard University Press.
30. Murray C. J. L., Acharya A. K. Understanding DALYs. Journal of Health Economics. 1997;16(6):703–30. [PubMed: 10176780]
31. Murray, C. J. L, and A. D. Lopez, eds. 1996a. The Global Burden of Disease: A Comprehensive Assessment of Mortality and Disability from Diseases, Injuries, and Risk Factors in 1990 and Projected to 2020. Vol. 1 of Global Burden of Disease and Injury Series. Cambridge, MA: Harvard University Press.
32. ———. 1996b. "The Global Burden of Disease in 1990: Final Results and Their Sensitivity to Alternative Epidemiological Perspectives, Discount Rates, Age-Weights, and Disability Weights." In The Global Burden of Disease, ed. C. J. L. Murray and A. D. Lopez, 247–93. Vol. 1 of Global Burden of Disease and Injury Series. Cambridge, MA: Harvard University Press.
33. Murray C. J., Lopez A. D., Wibulpolprasert S. Monitoring Global Health: Time for New Solutions. British Medical Journal. 2004;329(7333):1096–100. [PMC free article: PMC526127] [PubMed: 15528624]
34. Murray, C. J. L, C. D. Mathers, and J. A. Salomon. 2003. "Towards Evidence-Based Public Health." In Health Systems Performance Assessment: Debates, Methods, and Empiricism, ed. C. J. L. Murray and D. Evans, 715–26. Geneva: World Health Organization.
35. Murray, C. J. L, J. A. Salomon, C. D. Mathers, and A. D. Lopez. 2002. "Summary Measures of Population Health: Conclusions and Recommendations." In Summary Measures of Population Health: Concepts, Ethics, Measurement, and Applications, ed. C. J. L. Murray, J. A. Salomon, C. D. Mathers, and A. D. Lopez, p. 731–56. Geneva: World Health Organization.
36. National Research Council. 1994. Science and Judgment in Risk Assessment. Washington, DC: National Academy Press.
37. Parfit, D. 1984. Reasons and Persons. Oxford, U.K.: Clarendon Press.
38. Peto R., Lopez A. D., Boreham J., Thun M., Heath C. Mortality from Tobacco in Developed Countries: Indirect Estimation from National Vital Statistics. Lancet. 1992;339(8804):1268–78. [PubMed: 1349675]
39. Public Health Agency of Canada. 2005. "Population Health Impact of Disease in Canada." http://www​.phac-aspc​.gc.ca/phi-isp/summary_measures.html#3.
40. Salomon, J. A., C. J. L. Murray, T. B. Ustun, and S. Chatterji. 2003. "Health State Valuations in Summary Measures of Population Health." In Health Systems Performance Assessment: Debate, Methods, and Empiricism, ed. C. J. L. Murray and D. Evans, 409–36. Geneva: World Health Organization.
41. Ustun, T. B., S. Chatterji, M. Villanueva, L. Bendib, C. Celik, R. Sadana, N. Valentine, C. Mathers, J. P. Ortiz, A. Tandon, J. Salomon, Y. Cao, X. W. and C. J. L. Murray. 2003. "The WHO Multicountry Household Survey Study on Health and Responsiveness 2000–2001." In Health Systems Performance Assessment: Debates, Methods, and Empiricism, ed. C. J. L. Murray and D. Evans, 761–96. Geneva: World Health Organization.
42. Vose, D. 2000. Risk Analysis: A Quantitative Guide. New York: Wiley.
43. Williams A. Intergenerational Equity: An Exploration of the `Fair Innings' Argument. Health Economics. 1997;6(2):117–32. [PubMed: 9158965]
44. Yang G. H., Hu J., Rao K. Q., Ma J., Rao C., Lopez A. D. Mortality Registration and Surveillance in China: History, Current Situation, and Challenges. Population Health Metrics. 2005;3:3. [PMC free article: PMC555951] [PubMed: 15769298]