Format

Send to

Choose Destination
Trials. 2016 Jan 16;17:32. doi: 10.1186/s13063-016-1158-y.

Use of PRECIS ratings in the National Institutes of Health (NIH) Health Care Systems Research Collaboratory.

Author information

1
Group Health Research Institute, Seattle, USA. karinjohnso@gmail.com.
2
National Cancer Institute, Bethesda, MD, USA. netagil@mail.nih.gov.
3
University of Pennsylvania Perelman School of Medicine, Philadelphia, USA. ldember@upenn.edu.
4
Kaiser Permanente Center for Health Research, Portland, OR, USA. gloria.d.coronado@kpchr.org.
5
National Cancer Institute, Bethesda, MD, USA. jerry.suls@nih.gov.
6
National Cancer Institute, Bethesda, MD, USA. dchamber@mail.nih.gov.
7
University of Washington, Seattle, WA, USA. srundell@u.washington.edu.
8
Kaiser Permanente Center for Health Research, Portland, OR, USA. David.H.Smith@kpchr.org.
9
National Cancer Institute, Bethesda, MD, USA. liub2@mail.nih.gov.
10
National Cancer Institute, Bethesda, MD, USA. taplins@mail.nih.gov.
11
National Heart, Lung, and Blood Institute, National Institutes of Health, Bethesda, MD, USA. stoneyc@nhlbi.nih.gov.
12
National Cancer Institute, Bethesda, MD, USA. farrellm@mail.nih.gov.
13
Department of Family Medicine, University of Colorado School of Medicine, Denver, USA. Russell.Glasgow@ucdenver.edu.

Abstract

BACKGROUND:

The National Institutes of Health (NIH) Health Care Systems Research Collaboratory (NIH Collaboratory) seeks to produce generalizable knowledge about the conduct of pragmatic research in health systems. This analysis applied the PRECIS-2 pragmatic trial criteria to five NIH Collaboratory pragmatic trials to better understand 1) the pragmatic aspects of the design and implementation of treatments delivered in real world settings and 2) the usability of the PRECIS-2 criteria for assessing pragmatic features across studies and across time.

METHODS/DESIGN:

Using the PRECIS-2 criteria, five pragmatic trials were each rated by eight raters. For each trial, we reviewed the original grant application and a required progress report written at the end of a 1-year planning period that included changes to the protocol or implementation approach. We calculated median scores and interrater reliability for each PRECIS domain and for the overall trial at both time points, as well as the differences in scores between the two time points. We also reviewed the rater comments associated with the scores.

RESULTS:

All five trials were rated to be more pragmatic than explanatory, with comments indicating that raters generally perceived them to closely mirror routine clinical care across multiple domains. The PRECIS-2 domains for which the trials were, on average, rated as most pragmatic on the 1 to 5 scale at the conclusion of the planning period included primary analysis (mean = 4.7 (range = 4.5 to 4.9)), recruitment (4.3 (3.6 to 4.8)), eligibility (4.1 (3.4 to 4.8)), setting (4.1 (4.0 to 4.4)), follow-up (4.1 (3.4 to 4.9)), and primary outcome (4.1 (3.5 to 4.9)). On average, the less pragmatic domains were organization (3.3 (2.6 to 4.4)), flexibility of intervention delivery (3.5 (2.1-4.5)), and flexibility of intervention adherence (3.8 (2.8-4.5)). Interrater agreement was modest but statistically significant for four trials (Gwet's AC1 statistic range 0.23 to 0.40) and the intraclass correlation coefficient ranged from 0.05 to 0.31. Rating challenges included assigning a single score for domains that may relate to both patients and care settings (that is, eligibility or recruitment) and determining to what extent aspects of complex research interventions differ from usual care.

CONCLUSIONS:

These five trials in diverse healthcare settings were rated as highly pragmatic using the PRECIS-2 criteria. Applying the tool generated insightful discussion about real-world design decisions but also highlighted challenges using the tool. PRECIS-2 raters would benefit from additional guidance about how to rate the interwoven patient and practice-level considerations that arise in pragmatic trials.

TRIAL REGISTRATIONS:

Clinicaltrials.gov trial registrations: NCT02019225 , NCT01742065 , NCT02015455 , NCT02113592 , NCT02063867 .

PMID:
26772801
PMCID:
PMC4715340
DOI:
10.1186/s13063-016-1158-y
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for BioMed Central Icon for PubMed Central
Loading ...
Support Center