Evaluating fitness-for-use of electronic health records in pragmatic clinical trials: reported practices and recommendations

Objective: To empirically explore how pragmatic clinical trials (PCTs) that used real-world data (RWD) assessed study-specific fitness-for-use.

Methods: We conducted interviews and surveys with PCT teams who used electronic health record (EHR) data to ascertain endpoints. The survey cataloged key concerns about RWD, activities used to assess data fitness-for-use, and related barriers encountered by study teams. Patterns and commonalities across trials were used to develop recommendations for study-specific fitness-for-use assessments.

Results: Of 15 invited trial teams, 7 interviews were conducted. Of 31 invited trials, 15 responded to the survey. Most respondents had prior experience using RWD (93%). Major concerns about EHR data were data reliability, missingness or incompleteness of EHR elements, variation in data quality across study sites, and presence of implausible or incorrect values. Although many PCTs conducted fitness-for-use activities (eg, data quality assessments, 11/14, 79%), less than a quarter did so before choosing a data source. Fitness-for-use activities, findings, and resulting study design changes were not often publically documented. Overall costs and personnel costs were barriers to fitness-for-use assessments.

Discussion: These results support three recommendations for PCTs that use EHR data for endpoint ascertainment. Trials should detail the rationale and plan for study-specific fitness-for-use activities, conduct study-specific fitness-for-use assessments early in the prestudy phase to inform study design changes before the trial begins, and share results of fitness-for-use assessments and description of relevant challenges and facilitators.

Conclusion: These recommendations can help researchers and end-users of real-world evidence improve characterization of RWD reliability and relevance in the PCT-specific context.