NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Meltzer DO, Hoomans T, Chung JW, et al. Minimal Modeling Approaches to Value of Information Analysis for Health Research [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2011 Jun. (Methods Future Research Needs Reports, No. 6.)

Cover of Minimal Modeling Approaches to Value of Information Analysis for Health Research

Minimal Modeling Approaches to Value of Information Analysis for Health Research [Internet].

Show details

Literature Review

We conducted a comprehensive review of both published, peer-reviewed literature and grey literature to identify and describe clinical research studies describing or applying minimal modeling approaches to VOI.

Search Strategy and Inclusion Criteria

We searched the MEDLINE database for English-language publications from January 1, 1990 to June 3, 2010, using the following exact search terms (in all fields): “value of information,” “value of additional information,” “value of information analysis,” “expected value of perfect information,” “EVPI,” “expected value of partial perfect information,” “EVPPI,” “Bayesian approach to uncertainty,” or “value of research.” All titles and abstracts of the search results were screened by two investigators (DM and JC) to identify potentially relevant studies.

Our grey literature search was limited to Internet sites of different health technology assessment (HTA) organizations and institutions in the United States, Canada, the U.K., Australia/New Zealand, The Netherlands, and Germany. Websites were searched for: (1) VOI methods guidance intended to aid authors in completing a HTA, and (2) examples of VOI applications in individual HTA and systematic review publications.

Studies were only included if they involved clinically related application or development of VOI analysis for estimating the value of research or prioritization of research. Studies focusing on the value of diagnostic testing to collect information to guide the treatment for individual patients did not meet our scope definition, and were not included in our review. In addition, the reference lists of relevant studies were checked.

Study Classification and Data Extraction

Based on the full-text reading, the investigators (DM, TH, JC, and AB) independently classified the publications as to whether (1) these involved “VOI theory/methods only,” “VOI theory/methods with application,” or “VOI application only,” and (2) whether the approach to the VOI calculations comprised the use of a full model, limited modeling, or no modeling (Table 1).

We summarized theory and methods for those studies that appeared to adopt a minimal modeling approach. For each VOI application in the studies that were classified “VOI theory/methods with application” or “VOI application only” based on a “limited modeling” or “no modeling” approach, we extracted data on: the limited modeling component; the approach to VOI calculations (Table 1); the application and its setting; the perspective of analysis; the incidence (or prevalence) of the disease or condition; the time horizon of the decision problem (relating to durability); the approach (if any) to implementation issues; the discounting of costs and effects; and the cost-effectiveness results and VOI results (per patient and per population).

All data extraction was done by one investigator (TH), while the other investigators (DM, JC, and AB) performed a check for accuracy and completeness of the extracted data. Any disagreements about the classification of publications and the data extraction were resolved by consensus. We undertook a descriptive synthesis of the review results and compared VOI calculations across studies.

Results

Identification of Studies

Figure 1 is a flowchart that summarizes the process and results of our literature review. The MEDLINE database search produced 230 hits, while 120 studies were identified as potentially relevant following the screening of the abstracts and titles. On the basis of full-text reading of the 116 papers that we were able to collect, we identified 18 studies as “VOI theory/methods only,” and 80 studies with some empirical application, including 24 studies with “VOI theory/methods with application(s)” and 56 “VOI application only.” In total, we found 4 studies describing minimal modeling approaches only,5, 8–10 while 8 VOI applications adopting a limited modeling approach were reported in 6 studies7,11, 12[a,b,c],13–15 versus 9 VOI applications with no modeling in 12 studies.4,16, 17[a,b], 18, 19[a,b], 20 [a,b]; 21–25

Figure 1 is a flow chart that summarizes the search and selection of articles from the VOI literature for the review. It begins with the total number of hits screening from both a MEDLINE search and a gray literature search, and ends with the number of studies that were included as each VOI category in the report. This figure is described further in the Section titled “Results, Identification of Studies” as follows: “Figure 1 is a flowchart that summarizes the process and results of our literature review. The MEDLINE database search produced 230 hits, while 120 studies were identified as potentially relevant following the screening of the abstracts and titles. On the basis of full-text reading of the 116 papers that we were able to collect, we identified 18 studies as “VOI theory/methods only,” and 80 studies with some empirical application, including 24 studies with “VOI theory/methods with application(s)” and 56 “VOI application only.” In total, we found four studies describing minimal modeling approaches only (references 5 and 8–10) while eight VOI applications adopting a limited modeling approach were reported in 6 studies (references 7,11, 12, and 13–15) versus 9 VOI applications with no modeling in 12 studies. (references 4, 16–25). In our grey literature search, 12 HTA organizations were identified, with only two organizations (i.e., NICE in the U.K. and CVZ in The Netherlands) providing a small amount of guidance for the use of VOI methods. A search of all HTA publications from these organizations for VOI analysis produced 22 hits, nine of which were previously identified in the MEDLINE search. Of the 13 new publications, we classified 12 as full VOI models, and one as minimal modeling with one application. Table 2 summarizes these searches by country.

Figure 1

VOI literature review flowchart.

In our grey literature search, 12 HTA organizations were identified, with only 2 organizations (i.e., NICE in the U.K. and the Dutch Health Care Insurance Board [CVZ] in The Netherlands) providing a small amount of guidance for the use of VOI methods. A search of all HTA publications from these organizations for VOI analysis produced 22 hits, 9 of which were previously identified in the MEDLINE search. Of the 13 new publications, we classified 12 as full VOI models, and one as minimal modeling with one application.26 Table 2 summarizes these searches by country.

Table 2. Results of grey literature search, by country.

Table 2

Results of grey literature search, by country.

Theory/Methods on Minimal Modeling

Four papers described theory/methods related to limited modeling5,8 or no modeling 9,10 without seeking to apply the theory/methods. Detsky (1985)8 described how the cost-effectiveness of a clinical trial could be calculated with effectiveness measured in terms of deaths prevented; because a more comprehensive outcome measure (such as QALYs) was not used, no complex decision model was needed, so the paper provides an example of how a minimal modeling approach might be applied. Meltzer (2001)5 focused on the development of bounding approaches such as the way in which burden of disease-type calculations and the expected value of perfect information could bound estimates of the value of information. Willan (2008)9 argued that the potential for imperfect implementation of health technologies should be accounted for, such as by calculating VOI with current implementation. Janssen and Koffijberg (2009)10 focused on the construction of VOI estimates with independent estimates of variability in benefits and costs.

VOI Applications using Minimal Modeling

Tables 3 and 4 provide details of the applications in the published studies that applied limited modeling and no modeling approaches to VOI calculations, respectively. In total, our review of the academic literature found 18 applications of VOI analysis in which either a limited modeling (50%) or no modeling (50%) approach was adopted. In the limited modeling studies, the modeling component commonly involved the approximation of patient survival or life expectancy, for example, using (declining) exponential distributions.

Table 3. Details of applications (i=9) in VOI studies [k=7] using “limited modeling” approaches to VOI.

Table 3

Details of applications (i=9) in VOI studies [k=7] using “limited modeling” approaches to VOI.

Table 4. Details of applications [k=9] in VOI studies [n=12] using “no modeling” approaches to VOI.

Table 4

Details of applications [k=9] in VOI studies [n=12] using “no modeling” approaches to VOI.

The majority of the minimal modeling studies involved pharmaceuticals or other clinical interventions, including surgical procedures12,13,25 and medical devices.14 Six VOI studies were conducted in the U.K., five in the United States, three in Canada, and two in The Netherlands, while four occurred across different jurisdictions (e.g., North America or the European Union). Five out of 18 VOI applications were undertaken from a societal perspective, as one would expect given the public characteristics of evidence collection.

As shown in Table 3, 7 of the 18 VOI applications were based on equation-based computations relying on parametric assumptions for the costs and effects of the health technologies under evaluation. This included two early studies by Detsky (1990)11 and Omenn (2001)16, and five recent and related studies by Willan and colleagues.9,17,19–21 Four studies, three by Townsend and colleagues12[a,b,c] and the Detsky (1990)11 study, adopted an alternative approach to minimal modeling in that the incremental cost-effectiveness ratios of future trials or evidence to be developed were calculated on the basis of prior information or elicitation of expert opinion of the cost and effects of the health technologies and costs of research. All the remaining eight VOI applications, reflecting the vast majority of the more recent studies, used simulation/bootstrapping of raw data on costs and effects (QALYs). Parametric simulation/bootstrapping techniques (in R, Microsoft Excel, or Stata) were often used to explore decision uncertainty and establish VOI measures in these studies.

Most VOI application studies reported outcomes in terms of person level estimates of the value of information (e.g., EVPI ). Population-level estimates of the value of information (e.g., pEVPI) were reported less commonly, despite that these measures establish the necessary and sufficient condition for decisionmaking about research funding. EVPI results varied from €2.1 ($2.89) to £1064 ($1675) per patient, while the population values for pEVPI ranged between €365,000 ($504,649) and $308 billion.* Where separate measures of benefits and costs were available, the value of partial information on benefits and costs was calculated in some cases.

Comparison of VOI Calculations Using Minimal Modeling

Perhaps not surprisingly, given that the minimal modeling approach to VOI is driven primarily by empirical concerns, most of the papers we found that discussed approaches we would classify as discussing minimal modeling approaches to VOI were empirical applications. Aside from the fact that the results of VOI studies vary with the uncertainty in the costs and effects of the health technologies under evaluation, the results of such analyses are often difficult to compare due to variation in the perspective and time horizon for the analysis, the size of population targeted and the actual use or implementation of health technologies in the specific settings. Moreover, difference may exist in the approach to decision analysis (e.g., nonparametric bootstrapping vs. parametric equation-based computations), the use of end outcomes (e.g., clinical events avoided vs. QALYs), and the threshold value for cost-effectiveness in the jurisdictions in which the VOI analysis is applied.

Some of these differences in study assumptions (such as discount rates and assumptions about the value of a QALY) almost certainly cannot even approximately be adjusted for based on the published results, which makes it nearly impossible to compare value of research calculations across studies. However, some differences across studies, such as the population studies (e.g., U.K. vs. U.S.), time horizon (e.g., 5 or 10 years), currency (e.g., $, £, €), and year can be fairly readily adjusted for. To demonstrate this, we attempted to compare results across minimal modeling studies (i.e., limited modeling and no modeling studies) for all studies possible. Populations from the different studies were normalized to reflect the U.S. population, with a horizon of 10 years and denominated in 2010 U.S. dollars using historical currency exchange rates and the general U.S. Consumer Price Index.

As shown in Tables 5 and 6, standardizing on these factors often had large effects on the value of research, suggesting the importance of developing approaches to standardize population level VOI analyses if they are to be compared to each other. Estimates of the standardized VOI varied from around $2 million to nearly $125 billion, with most studies distributed broadly across the range from $2 million to $600 million. The $125 billion study, analyzing uncertainty in the value of atypical antipsychotics in schizophrenia, reflects to large degree the fact that schizophrenia affects 1 percent of the population throughout their entire life, and the substantial uncertainty about the effects of these medications on both quality of life and costs.

Table 5. Comparison of VOI estimates across limited modeling studies.

Table 5

Comparison of VOI estimates across limited modeling studies.

Table 6. Comparison of VOI stimates across no modeling studies.

Table 6

Comparison of VOI stimates across no modeling studies.

Footnotes

*

Foreign currencies converted to $US using October 2010 exchange rates.

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (1.1M)

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...