NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Cover of Frameworks for Determining Research Gaps During Systematic Reviews

Frameworks for Determining Research Gaps During Systematic Reviews

Methods Future Research Needs Reports, No. 2

Investigators: , PhD, , MBBS, MPH, and , MS.

The Johns Hopkins University Evidence-based Practice Center
Rockville (MD): Agency for Healthcare Research and Quality (US); .
Report No.: 11-EHC043-EF

Structured Abstract

Research Objective:

Systematic reviews, in addition to summarizing the evidence, generally also discuss needs for future research. However, in contrast to the methods of the systematic review, future needs are not identified systematically. There is limited literature describing organizing principles or frameworks for determining research gaps. We developed and pilot-tested a framework for the identification of research gaps from systematic reviews.

Study Design:

We reviewed the research gaps identification practices of organizations involved with evidence synthesis. We contacted: (i) evidence-based practice centers (EPCs) (n=12) associated with the Agency for Healthcare Research and Quality (AHRQ) in the US and Canada, and (ii) other organizations around the world (n=64) that conduct systematic reviews, cost-effectiveness analyses, or technology assessments. Based on the responses, we developed a framework for identifying research gaps. We obtained feedback from two technical experts at our institution and pilot-tested this framework on two randomly selected EPC evidence reports. We also developed a simple, user-friendly worksheet with instructions to facilitate the use of the framework by investigators during or after a systematic review.

Population Studied:

Not Applicable.

Principal Findings:

Four (33.3%) EPCs and 3 (8.1%) of the other organizations reported currently using an explicit framework to determine research gaps. We did not identify one framework that captured all elements needed to determine and characterize research gaps. Variations of the PICO (population, intervention, comparison, outcomes) framework were most common. It is also important to classify the reason(s) for the gap to help determine how to address the gap. Therefore, we propose a framework that includes both the characterization of the gap using PICOS elements (also including setting) and the identification of the reason(s) why the gap exists. The framework allows investigators to classify reasons for the existence of a research gap as: (a) insufficient or imprecise information, (b) biased information; (c) inconsistency or unknown consistency, and (d) not the right information. We mapped each of these reasons to concepts from three commonly used evidence grading systems: the Grading of Recommendations Assessment, Development and Evaluation (GRADE); the United States Preventive Services Task Force (USPSTF); and the Strength of Evidence (SOE) used by EPCs. This allows leveraging of work already being completed during evidence grading. During pilot-testing, we identified challenges including difficulty in applying the framework for completed systematic reviews and differences in the specificity of research gaps abstracted by different users. These could be tackled with a priori discussions amongst investigators. Further testing should determine if these challenges are ameliorated if the framework is used during a systematic review.


We developed a framework to identify and characterize research gaps from systematic reviews. The framework provides for the classification of where and why the current evidence falls short.

Implications for Policy, Delivery, or Practice:

In synthesizing evidence, systematic reviews inform health-care decisions for patients, policymakers, and clinicians. Systematic reviews can also be invaluable for identifying research gaps, thus helping develop research agendas. This potential impact of systematic reviews has not been realized. Our framework provides for systematically identifying and characterizing research gaps from systematic reviews. This explicit identification of research gaps will help determine the type of research needed to address the goals of comparative effectiveness research.


Prepared for: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services1, Contract No. 290-2007-10061-I, Prepared by: The Johns Hopkins University Evidence-based Practice Center,Baltimore, MD

Suggested citation:

Robinson KA, Saldanha IJ, Mckoy NA. Frameworks for determining research gaps during systematic reviews. Methods Future Research Needs Report No. 2. (Prepared by the Johns Hopkins University Evidence-based Practice Center under Contract No. HHSA 290-2007-10061-I.) AHRQ Publication No. 11-EHC043-EF. Rockville, MD: Agency for Healthcare Research and Quality. June 2011. Available at:

This report is based on research conducted by the Johns Hopkins University Evidence-based Practice Center (EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 290-2007-10061-I). The findings and conclusions in this document are those of the author(s), who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.

The information in this report is intended to help health care researchers and funders of research make well-informed decisions in designing and funding research and thereby improve the quality of health care services. This report is not intended to be a substitute for the application of scientific judgment. Anyone who makes decisions concerning the provision of clinical care should consider this report in the same way as any medical research and in conjunction with all other pertinent information, i.e., in the context of available resources and circumstances.

This report may be used, in whole or in part, as the basis for research design or funding opportunity announcements. AHRQ or the U.S. Department of Health and Human Services endorsement of such derivative products or actions may not be stated or implied.

None of the investigators has any affiliations or financial involvement that conflicts with the material presented in this report.


540 Gaither Road, Rockville, MD 20850; www‚Äč

Bookshelf ID: NBK62478PMID: 21977524


Related information

Similar articles in PubMed

See reviews...See all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...