Table 1Step 1: Focused literature review—Summary of included articles with methods/frameworks used

Author (Year)RefTopic AreaRelevant Text From ArticleSummary of Method/Framework Used
Chou (2009)1Chronic non-cancer painTo assign an overall strength of evidence (good, fair, or poor), the number, quality, and size of the studies; consistency of results between studies; and directness of the evidence were considered.
  • Consistent results from a number of higher-quality studies across a broad range of populations support a high degree of certainty that the results of the studies are true (the entire body of evidence would be considered “good-quality”).
  • For a “fair-quality” body of evidence, results could be due to true effects or due to biases present across some or all of the studies.
  • For a “poor-quality” body of evidence, any conclusion is uncertain due to serious methodological shortcomings, sparse data, or inconsistent results.
Research gaps were identified as the key questions that were addressed by only "poor-quality" evidence.
Organizing Principle(s) - Key question.
Shepherd (2007)2
  • STIs in teenagers
  • Acute pain management in children and young people (28 days - 19 years)
The included literature was read, assimilated, and mapped to the appropriate nodes of the care pathway. This process was performed by one researcher and checked by a second. Additional areas for intervention along the care pathway were identified during this assimilation process. Validation of this process was undertaken by an independent reviewer. The quality of the guidelines was appraised narratively using the AGREE appraisal tool. Analysis of the populated care pathway and narrative map (the framework based on the care pathway) facilitated the identification of gaps in research evidence and policy literature. The gaps were identified by (i) analyzing those areas of the care pathway lacking appropriate guidelines/guidance or evaluation research, (ii) noting gaps cited within existing guidelines/guidance or evaluation research, and (iii) identifying poor-quality and out-of-date studies and guidelines/guidance.A care pathway was developed for each condition. Gaps were identified using the following three criteria to assess areas within the care pathway:
  • analyzing those areas of the care pathway lacking appropriate guidelines/guidance or evaluation research,
  • noting gaps cited within existing guidelines/guidance or evaluation research, and
  • identifying poor-quality and out-of-date studies and guidelines/guidance.
Organizing principle(s) - C are pathway.
Clarke (2007)3Not specificOne author read each record and categorized it on the basis of whether a recommendation was made as to the need for more research and, if so, whether suggestions were made regarding the specific types of intervention, participant, or outcome measures that should be assessed or included in future research. In addition, suggestions for a new, expanded, or updated systematic review were noted. Another author read each record to identify whether it mentioned a specific ongoing or planned study. The third author read each record, checked the assigned codes, and made the final decision on the coding of each record. Subsequently, details were obtained on the content of the “ongoing studies” sections. The number of studies listed for each review was counted and cross-checked by two of the authors.Uses the Implications for Research sections of all Cochrane reviews from one issue of the Cochrane Library (various topics). Reviews were categorized by whether a recommendation was made as to the need for more research and, if so, whether suggestions were made regarding the types of intervention, participant, or outcome measures that should be included in future research. In addition, suggestions for a new, expanded, or updated systematic review were noted. It was also noted whether reviews mentioned a specific ongoing or planned study. Subsequently, details were obtained on the content of the “ongoing studies” sections.
Organizing principle(s) - By types of intervention, types of participant, and types of outcome measures.
Johnson (2003)4InfertilityThe following information as collected from each review: the number of trials available for meta- analysis, the total number of trial participants available for meta-analysis and whether there was an answer to the primary clinical question-into which category from (A) to (C), below, the review fell: (A) Where there is evidence of effectiveness or harm from a metaanalysis of trial data. The term “relative effectiveness” was used when two interventions were compared and the term “effectiveness” was used when the treatment was compared with either placebo or no treatment. (B) Where there is insufficient evidence of effectiveness and the review authors have called for further research. (C) Where there is insufficient evidence of effectiveness and the review authors have not called for further research.Uses Cochrane reviews to identify areas of insufficient evidence. Gaps were identified if one of the following two criteria were fulfilled (based on review authors' definition of the sufficiency of evidence):
  • Where there is insufficient evidence of effectiveness and the review authors have called for further research.
  • Where there is insufficient evidence of effectiveness and the review authors have not called for further research.
Organizing principle(s) - Topic area.
de Vet (2001)5Chronic benign pain syndromesThe methodologic quality of the relevant reviews was assessed according to the method developed by Assendelft et al. on a 0–100 point scale. If reviews of reasonable (60–79 points) to good quality (≥80 points) were found, their conclusions (effective, not effective, inconclusive) were adopted. If only reviews of poor or moderate quality were found, a new systematic review was recommended. If no recent reviews of a specific topic were found, a search was made for randomized controlled trials (RCTs), using the strategy recommended by the Cochrane Collaboration. If there were more than five RCTs in the computerized databases mentioned above, a systematic review was recommended. If there were five RCTs or less, the following data were extracted from the abstract: the design (parallel or a crossover study), the sample size, whether the trial was really randomized, whether it was blinded, which interventions were compared, and the conclusions. If the conclusions were inconsistent, a new RCT was recommended. If the results appeared to be consistent, it was advised that the evidence of (in)effectiveness should be studied in detail in order to decide on the level of evidence. When the evidence from a small number of studies is convincing, it should be incorporated in a clinical guideline. A decision tree was used for this purpose for each intervention-syndrome combination (topic). This resulted in a list of topics for the nine chronic benign pain syndromes for which systematic reviews were recommended and a list of topics for which new RCTs were needed.A decision tree was generated to decide about the need for a new systematic review or the need for a new RCT for each topic. Lists of research gaps/needs were identified as areas of insufficient or inconsistent evidence. These lists were then prioritized.
Organizing principle(s) - Decision tree

Abbreviations: AGREE=appraisal of guidelines research and evaluation, RCT=randomized controlled trial, STI=sexually transmitted infection.

From: Results

Cover of Frameworks for Determining Research Gaps During Systematic Reviews
Frameworks for Determining Research Gaps During Systematic Reviews [Internet].
Methods Future Research Needs Reports, No. 2.
Robinson KA, Saldanha IJ, Mckoy NA.

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.