NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Buckley DI, Ansari M, Butler M, et al. The Refinement of Topics for Systematic Reviews: Lessons and Recommendations From the Effective Health Care Program [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2013 Jan.

Cover of The Refinement of Topics for Systematic Reviews

The Refinement of Topics for Systematic Reviews: Lessons and Recommendations From the Effective Health Care Program [Internet].

Show details


Topic refinement is one of several major stages in the process of producing a systematic review through the EPC Program; it bridges the initial stage of topic nomination and development and the latter stage of conducting the systematic review (see Figure 1). During topic nomination and development, a team of investigators reviews stakeholder-nominated topics and determines which nominations meet program inclusion criteria and should be recommended for topic refinement and systematic review. These recommendations are based on EPC Program principles, priority conditions, and specific selection criteria.5 Selected topics then undergo the topic refinement process addressed in this report.

This figure depicts three distinct phases of the development of topic in the EHC Program. Each phase is described by a short list of high-level details about each of these phases. The figure consists of three tiered boxes with down-pointing arrows on the left, each depicting a different phase and the transition to the next phase. From top to bottom, the phases are: Topic Nomination and Development, Topic Refinement and Systematic Review. The topic nomination and development phase appears in the top tier and consists of 4 bullet points: Identify topic; Feasibility scan; Apply EHC topic selection criteria; and Determine the appropriateness for systematic review development. The topic refinement phase appears in the middle tier and consists of 1 bullet point: See Figure 2 for full details of Topic Refinement. The systematic review phase is shown in the bottom tier and consists of 5 bullet points: develop protocol; selects studies; abstract, analyzes and synthesize data; apply EHC Methods Guidance for Effectiveness Reviews; and report the conclusions and the implications for decisionmaking.

Figure 1

Major stages in producing a systematic review.

The primary goal of topic refinement is to formulate research questions that can be addressed by a systematic review; the goal is not to answer the questions. A refined topic includes three principal elements: (1) clearly articulated population(s), intervention(s), comparator(s), outcome(s), timing, and setting(s) of interest—collectively referred to as the PICOTS;5,6 (2) well-written Key Questions that are precise, detailed, and clearly focused; and (3) an analytic framework that represents the relationships between the elements of the PICOTS and the Key Questions.7-10 The topic refinement process includes a number of steps that begin with preliminary materials from the initial topic nomination and development stage and end with the refined topic and summary report being sent to the systematic review team for use in developing the systematic review protocol. These steps are outlined in Figure 2.

This figures illustrates the steps in the process of topic refinement with details of the activities of each step. Generally, the topic refinement phase begins with the original nominated topic, and then is followed by initial topic refinement, key informant interviews, synthesis and reporting, and public reporting. Each step is described by a short list of high-level details about each of these phases. The figure consists of five tiered boxes with down-pointing arrows on the left, each depicting a different step and the transition to the next step. From top to bottom, the steps are: Original Nominated Topic; Initial Topic Refinement; Key Informant Interviews; Sythensis and Reporting; Public Posting.

Figure 2

The process of topic refinement. AHRQ = Agency for Healthcare Research and Quality; EPC = Evidence-based Practice Center; PICOTS = population, intervention, comparator, outcomes, timing, and setting

The steps of topic refinement fall into two main phases—an initial phase in preparation for interviews with Key Informants, and a second phase that starts with Key Informant interviews and includes subsequent refinement and reporting of the topic. The Topic Refinement Document (Appendix A) provides a template for preparing a Topic Refinement Summary Report in the initial refinement phase. This is used for the Key Informant interviews and contains a narrative on the background and context of the topic, provisional PICOTS, provisional Key Questions, a provisional analytic framework, and a list of issues to discuss with the Key Informants. In preparing this report, the topic refinement team will conduct a targeted literature scan and may consult with topical experts. The Key Questions reflect important decisional dilemmas faced by stakeholders and clearly define the logic and scope of the topic. The Key Questions and analytic framework are formulated around specified PICOTS of interest. Typically, topic nominations present the elements of the PICOTS in a general form. Therefore, refining and focusing the PICOTS is a critical task of topic refinement.

Through Key Informant interviews in the second phase of refinement, the team elicits input on issues that cannot be resolved with a limited literature search and/or that require the perspective, experience, or technical knowledge of experts or other stakeholders. The Key Informants' input is considered, synthesized, and, when appropriate, incorporated into modifications of the provisional Key Questions and analytic framework, all of which is then described in the topic refinement summary report. The refined PICOTS, Key Questions, and analytic framework are posted online for broader stakeholder input before finalizing refinement. This topic refinement process typically takes about 4 months.

A Note on Terminology

In this report, we use the term “preliminary” to refer to elements of a topic that are developed prior to the topic refinement process. This includes the proposed Key Questions formulated by the nominating stakeholder and/or the topic nomination and development team. We use the term “provisional” to refer to the elements of the initial topic refinement phase. These “provisional” elements are: (1) descriptions of the PICOTS of interest; (2) Key Questions for the systematic review; and (3) an analytic framework. These represent the first stage of refinement, based on the work of the topic refinement team, a scan of the literature, and input from topical experts. These elements are considered provisional because they still do not include the input of multiple Key Informant stakeholders, whose views, expertise and values may lead to further refinement. Finally, we use the term “refined” to refer to the elements of the topic in their modified form after the topic refinement team has considered and integrated input from stakeholders (Key Informants and/or public commentary).

Objectives of the Topic Refinement Work Group

AHRQ's EPCs have produced summary reports of the refinement of approximately 100 topics for systematic reviews, using the EPC Topic Refinement Document. However, while the Topic Refinement Document stipulates the required elements to be included in the Topic Refinement Summary Report, it provides only general guidance on how to actually conduct the various steps of the process. A previous methods paper presented some guidance for topic refinement in similarly general terms.5 With this guidance, EPCs have approached the details of topic refinement in a variety ways. This variation offered an opportunity to learn from the experience of different EPCs, to synthesize that experience into a more detailed description of the topic refinement process, and to generate more detailed guidance for this important stage in the production of systematic reviews through the EPC Program. To that end, AHRQ convened a work group to assess the topic refinement process and develop recommendations for effective approaches to topic refinement.

The objectives of the topic refinement work group were:

  1. To elaborate on the minimal and general description of topic refinement provided in the Topic Refinement Document, based on an assessment of the experience of various EPCs in conducting topic refinements.
  2. To articulate a set of guiding principles for the topic refinement process.
  3. Based on an assessment of the experience of various EPCs, to identify best practices and incorporate those practices into the more detailed description of topic refinement.

By producing a more detailed description of topic refinement, including guiding principles and best practices, we hope to provide useful guidance that will make the topic refinement process more consistent, deliberate, and transparent. However, we expressly did not seek to develop prescriptive recommendations to be uniformly applied in all cases. Topics vary in their requirements for refinement, and different investigators may use different but equally valid rationales to make different but equally valid topic refinement decisions. Therefore, we sought to articulate viable approaches to the numerous aspects of topic refinement and to discuss the relative advantages and disadvantages of different approaches. Rather than prescribing exactly how investigators should conduct every topic refinement, we sought to offer guidance to help EPC investigators make better decisions about how to approach topic refinement.


Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...