Format

Send to

Choose Destination
Drug Saf. 2013 Oct;36 Suppl 1:S95-106. doi: 10.1007/s40264-013-0101-3.

Empirical performance of a self-controlled cohort method: lessons for developing a risk identification and analysis system.

Author information

1
Janssen Research and Development LLC, 1125 Trenton-Harbourton Road, Room K30205, PO Box 200, Titusville, NJ, 08560, USA, ryan@omop.org.

Abstract

BACKGROUND:

Observational healthcare data offer the potential to enable identification of risks of medical products, but appropriate methodology has not yet been defined. The self-controlled cohort method, which compares the post-exposure outcome rate with the pre-exposure rate among an exposed cohort, has been proposed as a potential approach for risk identification but its performance has not been fully assessed.

OBJECTIVES:

To evaluate the performance of the self-controlled cohort method as a tool for risk identification in observational healthcare data.

RESEARCH DESIGN:

The method was applied to 399 drug-outcome scenarios (165 positive controls and 234 negative controls across 4 health outcomes of interest) in 5 real observational databases (4 administrative claims and 1 electronic health record) and in 6 simulated datasets with no effect and injected relative risks of 1.25, 1.5, 2, 4, and 10, respectively.

MEASURES:

Method performance was evaluated through area under ROC curve (AUC), bias, and coverage probability.

RESULTS:

The self-controlled cohort design achieved strong predictive accuracy across the outcomes and databases under study, with the top-performing settings exceeding AUC >0.76 in all scenarios. However, the estimates generated were observed to be highly biased with low coverage probability.

CONCLUSIONS:

If the objective for a risk identification system is one of discrimination, the self-controlled cohort method shows promise as a potential tool for risk identification. However, if a system is intended to generate effect estimates to quantify the magnitude of potential risks, the self-controlled cohort method may not be suitable, and requires substantial calibration to be properly interpreted under nominal properties.

PMID:
24166227
DOI:
10.1007/s40264-013-0101-3
[Indexed for MEDLINE]

Supplemental Content

Full text links

Icon for Springer
Loading ...
Support Center