NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Stein E, Clarke JO, Hutfless S, et al. Wireless Motility Capsule Versus Other Diagnostic Technologies for Evaluating Gastroparesis and Constipation: Future Research Needs: Identification of Future Research Needs From Comparative Effectiveness Review No. 110 [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2013 May. (Future Research Needs Papers, No. 27.)
Wireless Motility Capsule Versus Other Diagnostic Technologies for Evaluating Gastroparesis and Constipation: Future Research Needs: Identification of Future Research Needs From Comparative Effectiveness Review No. 110 [Internet].
Show detailsThe aim of the Future Research Needs (FRN) project was to develop a prioritized list (or multiple lists) of research needs within the scope of the original systematic review, with considerations for potential research designs with sufficient detail for researchers and funders to use for developing research proposals or solicitations. As the resulting research is meant to improve healthcare decisions, stakeholders included patients/advocates, clinicians, and third-party payers.
The research needs were based on the research gaps identified in the systematic review writing process and contributed by the stakeholders. The methods for identifying evidence gaps and developing them into a prioritized list of research needs and feasible researchable questions involved the steps described in the immediately following subsections.
Identification of Evidence Gaps
A subset of seven of the original systematic review authors constituted the EPC FRN team. Evidence gaps were identified in the systematic review writing process based on the strength of evidence, applicability, and limitations of the systematic review. Evidence gaps were defined as parts of the systematic review Key Questions that had low strength of evidence or insufficient evidence. The FRN team met multiple times and circulated by email lists of potential questions to identify gaps with specific reference to study design and the PICOTS (lack of information or insufficient evidence for: sub-populations/whole populations; interventions, comparisons of interventions to each other; outcomes; timing of interventions or comparisons of interventions; and settings). The FRN team used this process to develop a list of research gaps to be presented to a stakeholder panel for review, as described in the following subsections.
Engagement of Stakeholders, Researchers, and Funders
Stakeholder Identification
We recruited a group of nine stakeholders to participate in the identification and prioritization of research gaps. We sought input from patients/advocates, clinical experts, and payers. Five stakeholders were chosen from the systematic review Key Informants and Technical Expert Panel members. These were chosen because of their expertise, familiarity with the systematic review, and because they had been particularly responsive and helpful with the systematic review process. In addition, four new participants were chosen to fill out the nine member stakeholder panel. Some of these were suggested by the systematic review investigators. We also searched websites of advocacy organizations to identify patient advocates who appear to be independent of payers and manufacturers according to the voting membership requirements and funding mechanisms of their organizations. The list was summarized in a table of their individual strengths and the list was presented to stakeholders who accepted an email invitation explaining the project and inviting them to participate signed a Conflict of Interest (COI) form declaring professional activities and financial ties relevant to the clinical area. It was made clear to them that accepting the invitation and returning the COI form constituted agreement to be identified as a stakeholder contributor to the final document. Manufacturers were not solicited to be part of the stakeholder panel, but they were informed of their ability to comment during the 4 week public posting period for this report.
Orienting Stakeholders
By emailed letters we provided the stakeholders with a description of the FRN Project, and how it is derived from and relates to the systematic review. We also sent them the draft of the executive summary of the systematic review. A Web link to the complete draft report was provided noting that the draft is only temporarily available, and that reading the executive summary should be sufficient to meaningfully contribute to the process to identify evidence gaps and FRN.
Stakeholder Engagement for Additional Gap Identification and Prioritization
We used an approach performed in two rounds of engagement with the stakeholders by means of emailed questionnaires.
Engagement Round 1. Gap List From Systematic Review and Preliminary Prioritization
The FRN team's list of research gaps, derived from the systematic review as described above, was presented to the stakeholders by email for review and for suggestions of additional gaps within the scope of the systematic review. They were instructed to carry out a preliminary prioritization of the gaps, including any additional gaps they added to the list. To perform this preliminary prioritization they were asked to use the criteria and ranking method described in the next subsection.
Criteria for Prioritization
Prior to engaging our stakeholders, we developed a draft framework consisting of criteria we considered important for prioritizing topics for future research. These draft criteria were adapted from AHRQ's Effective Health Care Program Topic Selection Criteria and included:
- Importance. The importance of the condition to patients (including consideration of whether that gap is of particular relevance to priority subpopulations such as pediatric patients, elderly patients, vulnerable and disparity populations)
- Impact. The extent to which new research with definitive findings could potentially impact decision-making by patients, providers, or policy makers
Other prioritization criteria of AHRQ/EHC were determined to be less useful or relevant for future research need prioritization.
Uncertainty is not a useful criterion, because all identified evidence gaps are uncertain by definition.
Feasibility of research on an evidence gap is a secondary concern that is independent of the need for evidence. Evidence gaps and the need for research to close such gaps are innate to the area of interest. They are gaps and needs regardless of whether research is possible or feasible. There is value in determining the absolute importance and potential impact of closing each of the gaps. The feasibility of carrying out the required research to close a gap is relative, depending on the difficulty of the research, funding sources, availability of adequate numbers of patients, incentives for researchers, the convenience of the needed length of followup, attractiveness of the research to researchers, etc. A question of the highest priority may secondarily be deemed worth pursuing in spite of the difficulty and cost of the research; whereas, similar difficulty and cost may render research on a lesser priority gap “unfeasible.” A funding source or research group with substantial resources may consider a research question feasible, while those with limited resources may not – circumstances beyond our knowledge or control. Therefore, we did not want relative feasibility to enter into the absolute priority decisions of the stakeholders. Nevertheless, we asked them to comment on feasibility, and we discuss it as a secondary aspect of the FRN questions.
We did not attempt to give our two criteria quantitative values or to break these major criteria into their multiple factors for individual weighting or priority ranking and combination by a mathematical formula. That would require validation of the weights. We did not consider that practical within the scope of this project. Summing multiple factors by an arbitrary mathematical formula would give an undue appearance of objectivity, accuracy and precision. Instead we instructed the stakeholders to consider these two major criteria, importance and impact, in their priority decisions.
In the round 1 questionnaire each stakeholder was presented with two lists of gaps, one for clinical questions from the systematic review, and another for methodological questions. The stakeholders were asked to choose their top five in each list and rank them by priority from 1 (highest) to 5, based on the criteria described above. Stakeholders were also asked to suggest additional gaps within the scope of the systematic review, if they were aware of any ongoing studies addressing a gap, and to comment on feasibility of research to address the gap.
In order to sort the gaps by priority, the ratings of 1 (highest) to 5 were inverted, so a priority rating of 1 corresponded to a point value of 5 for highest priority. Then these inverted individual stakeholder scores for each gap were summed and sorted from highest sum (highest priority) to lowest. If multiple gaps achieved the same sum, they were given the same priority rank. This priority sorted list was considered the preliminary priority ranked list of gaps.
Engagement Round 2. Final Prioritization
The FRN team incorporated the stakeholder comments and additional suggestions from engagement round 1 into a list of gaps for final prioritization. This list included the preliminary ranking from the previous round. Again this was two lists, one for clinical questions, and another for methodological questions. Each stakeholder was presented with these lists by email and asked to choose their top 6 choices in the clinical list and prioritize them from 1 (highest) to 6. And they were asked to choose their top seven from the methodology list and rate them from 1 (highest) to 7. We again asked them to base their ratings on the same criteria as in round 1, importance, and potential impact. Stakeholders were again asked if they are aware of any ongoing studies addressing the gaps (duplication), and they were able to comment on the feasibility of research addressing the gaps.
Top-Tier Future Research Needs
A global priority ranking of the gaps was calculated from the stakeholder individual ratings as described above. Appendix C shows the inverted scores, sums, and priority ranks. The global ranking was inspected by the EPC team to determine if there was an obvious cutpoint between a top tier of questions and the remainder. If the global ranking was a continuum with no apparent cutpoint, the top half of the gaps or the top 10, whichever is fewer, were to be chosen as the top tier and considered the high-priority FRN.
We also took into account the research needs that were prioritized highly by each stakeholder. After determining a preliminary top-tier cutpoint (Appendix C) according to the score sums as described, we reviewed and analyzed the individual stakeholder responses for the questions on either side of the cutpoint. We then assessed which questions had received top votes and next-to-top votes from each stakeholder, and we tracked how many of these number 1 and number 2 priorities each question near the cutpoint received. Using this method we verified that the sum score cutpoint we chose reflected the high-priority items that the stakeholders sought to identify (Appendix C). If a stakeholder scored an item highest priority it was placed on the high-priority list automatically. There were 9 designated top priority items in the Clinical Questions list, and 6 in the Methodology Questions list, for a total of 15.
Research Needs Development and Research Design Considerations
We developed questions addressing the FRN (top tier of knowledge gaps), including PICOTS information. These were circulated to the entire group by email, and discussed and further developed in multiple meetings of the FRN team. These FRN were presented to the stakeholders along with the corresponding evidence gaps in the second round of prioritization.
When we solicited information from stakeholders, we asked them to assess factors such as resource use, ethical considerations, data availability, recruitment or feasibility issues, and validity. We also offered them space to comment on any specific study designs relevant to a particular question. To complete the FRN report, we listed and assessed one to three study design suggestions for each of the final FRN, with limited textual description as suggested by the AHRQ guidance.11 We considered information from comments on each question that were provided to us by stakeholders via the initial questionnaire, and many of these were incorporated in the study design section.
Approach to External Literature Searches
To identify ongoing clinical trials that may address our FRN questions, we searched ClinicalTrials.gov, NIH Reporter, the Canadian Institutes of Health Research, the World Health Organization Clinical Trials Registry, and the European Union Clinical Trials Register since the search cutoff date of the systematic review. Our search terms are presented in Appendix A. Each article was reviewed by two people for inclusion, applying the same inclusion/exclusion criteria used in the comparative effectiveness systematic review. For each included trial, we abstracted the trial identification number, date of registry, the expected date of completion, the study name, status, medications compared, any published results, and determined the Key Question the study is likely to address (Appendix B).
Analytic Framework
We used an analytic framework to describe research gaps using the same format as for the systematic review (Figure 1).
- Methods - Wireless Motility Capsule Versus Other Diagnostic Technologies for Eva...Methods - Wireless Motility Capsule Versus Other Diagnostic Technologies for Evaluating Gastroparesis and Constipation: Future Research Needs
Your browsing activity is empty.
Activity recording is turned off.
See more...