NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Decision and Simulation Modeling in Systematic Reviews
Methods Research Reports
Investigators: Karen Kuntz, ScD, Francois Sainfort, PhD, Mary Butler, PhD, MBA, Brent Taylor, PhD, Shalini Kulasingam, PhD, Sean Gregory, MBA, Eric Mann, BA, Joseph M Anderson, BA, and Robert L Kane, MD.
Author Information and AffiliationsStructured Abstract
Purpose:
The purpose of this study is to provide guidance for determining when incorporating a decision-analytic model alongside a systemic review would be of added value for decisionmaking purposes. The purpose of systematic reviews is to synthesize the current scientific literature on a particular topic in the form of evidence reports and technology assessments to assist public and private organizations in developing strategies that improve the quality of health care and decisionmaking. However, there is often not enough evidence to fully address the questions that are relevant for decisionmakers. Decision models may provide added value alongside systematic reviews by adding a formal structure, which can be informed by the evidence.
Methods:
Our framework is informed by two sets of interviews and a focus group discussion; literature reviews to summarize best modeling practices and to profile the modeling literature; and an exploration of the feasibility of developing a database of published models. We interviewed Evidence-based Practice Center (EPC) members, some of whom have successfully incorporated models in EPC reports, to document lessons learned from those experiences. We also interviewed members of the U.S. Preventive Services Task Force (USPSTF) and cancer modelers who were involved in the recent efforts to use modeling with a systematic review to update USPSTF cancer screening guidelines, to evaluate the process of conducting a simultaneous systematic review and modeling exercise, and to evaluate stakeholder-perceived needs and whether needs were met. We reviewed and summarized the literature on best practices for modeling. This was supplemented by a focus group discussion with modeling experts to elicit, characterize, and precisely qualify best practices in decision and simulation modeling. These included: model formulation and characterization, model development and construction, handling and presentation of modeling assumptions, definition and presentation of parameters, outcomes to incorporate into the model, model analysis, model testing, validation, and implementation (including results presentation and communication). We developed a profile of the current modeling literature by conducting a systematic review of the medical literature and the grey literature to document publications that used a decision model and for what purpose (e.g., disease of interest, interventions evaluated). We also developed a prototype database to serve as a preliminary step in developing a resource that could be used to determine if, and how many, models exist on a particular disease of interest.
Results:
The resulting report consists of six chapters. Decision and Simulation Modeling Alongside Systematic Reviews provides an overview of models and describes the differences and synergies between systematic reviews and decision analysis. In Overview of Decision Models Used in Research, we provide a “scan” of the medical literature over the past 5 years in terms of the use of models in studies that compare intervention strategies using multiple sources of data. Use of Modeling in Systematic Reviews: The EPC Perspective documents the extent to which EPCs have incorporated models into data and presents results from key informant interviews with EPC members. We present a framework for deciding when a decision model can inform decisionmaking alongside a systematic review in Suggested Framework for Deciding When a Modeling Effort Should Be Added to a Systematic Review. Potential Modeling Resources explores several possible approaches to use when undertaking a modeling effort and discusses some of the challenges. Lastly, Best Practices for Decision and Simulation Modeling reviews the literature on best practices for modeling, supplemented by a focus group discussion with modeling experts, and lessons learned about the process of conducting a modeling exercise alongside a systematic review using recent experience with the USPSTF.
Conclusions:
We suggest a process for deciding when conducting a decision analysis in conjunction with a systematic review would be of value to decisionmakers.
Contents
- Preface
- Acknowledgments
- Technical Expert Panel
- Executive Summary
- Decision and Simulation Modeling Alongside Systematic Reviews
- Overview of Decision Models Used in Research
- Use of Modeling in Systematic Reviews: The EPC Perspective
- Suggested Framework for Deciding When a Modeling Effort Should be Added to a Systematic Review
- Potential Modeling Resources
- Best Practices for Decision and Simulation Modeling
- References and Included Studies
- Acronyms/Abbreviations
- Appendix A Search String and Results
- Appendix B Semistructured Interview Guide
- Appendix C Revised Discussion Guide
- Appendix D Verbatim Quotes for Key Themes
- Appendix E Types of Systematic Reviews
- Appendix F Model Documentation
- Appendix G Search Strategy for Best Practices Papers
- Appendix H Focus Group Instructions
- Appendix I Summary of Preliminary Findings Provided to Focus Group Participants
- Appendix J General Interview Outline – Modelers and USPSTF Members
- Appendix K Verbatim Quotes from Interviews of Modelers and USPSTF Members, by Theme
- Appendix L Excluded Studies
Prepared for: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services1, Contract No. 290-2007-10064-I. Prepared by: Minnesota Evidence-based Practice Center, Minneapolis, MN
Suggested citation:
Kuntz K, Sainfort F, Butler M, Taylor, B, Kulasingam, S, Gregory S, Mann E, Anderson JM, Kane RL. Decision and Simulation Modeling in Systematic Reviews. Methods Research Report. (Prepared by the University of Minnesota Evidence-based Practice Center under Contract No. 290-2007-10064-I.) AHRQ Publication No. 11(13)-EHC037-EF. Rockville, MD. Agency for Healthcare Research and Quality. February 2013. www.effectivehealthcare.ahrq.gov/reports/final.cfm.
This report is based on research conducted by the Minnesota Evidence-based Practice Center (EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 290-2007-10064-I). The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
The information in this report is intended to help health care decisionmakers—patients and clinicians, health system leaders, and policymakers, among others—make well-informed decisions and thereby improve the quality of health care services. This report is not intended to be a substitute for the application of clinical judgment. Anyone who makes decisions concerning the provision of clinical care should consider this report in the same way as any medical reference and in conjunction with all other pertinent information, i.e., in the context of available resources and circumstances presented by individual patients.
This report may be used, in whole or in part, as the basis for the development of clinical practice guidelines and other quality enhancement tools, or as a basis for reimbursement and coverage policies. AHRQ or U.S. Department of Health and Human Services endorsement of such derivative products may not be stated or implied.
None of the investigators have any affiliations or financial involvement that conflict with material presented in this report.
- 1
540 Gaither Road, Rockville, MD 20850; www
.ahrq.gov
- NLM CatalogRelated NLM Catalog Entries
- Review Adding decision models to systematic reviews: informing a framework for deciding when and how to do so.[Value Health. 2013]Review Adding decision models to systematic reviews: informing a framework for deciding when and how to do so.Sainfort F, Kuntz KM, Gregory S, Butler M, Taylor BC, Kulasingam S, Kane RL. Value Health. 2013 Jan-Feb; 16(1):133-9.
- Review Modeling and Simulation in the Context of Health Technology Assessment: Review of Existing Guidance, Future Research Needs, and Validity Assessment[ 2017]Review Modeling and Simulation in the Context of Health Technology Assessment: Review of Existing Guidance, Future Research Needs, and Validity AssessmentDahabreh IJ, Chan JA, Earley A, Moorthy D, Avendano EE, Trikalinos TA, Balk EM, Wong JB. 2017 Jan
- Review Updating Systematic Reviews[ 2007]Review Updating Systematic ReviewsShojania KG, Sampson M, Ansari MT, Ji J, Garritty C, Rader T, Moher D. 2007 Sep
- Review Understanding Health-Systems' Use of and Need for Evidence To Inform Decisionmaking[ 2017]Review Understanding Health-Systems' Use of and Need for Evidence To Inform DecisionmakingWhite CM, Sanders Schmidler GD, Butler M, Wang Z, Robinson K, Mitchell MD, Berkman N, Henderson J, Fiordalisi C, Hartling L, et al. 2017 Oct
- Review Improving Access to and Usability of Systematic Review Data for Health Systems Guidelines Development[ 2019]Review Improving Access to and Usability of Systematic Review Data for Health Systems Guidelines DevelopmentTotten AM, Smith C, Dunham K, Jungbauer RM, Graham E. 2019 Feb
- Decision and Simulation Modeling in Systematic ReviewsDecision and Simulation Modeling in Systematic Reviews
Your browsing activity is empty.
Activity recording is turned off.
See more...