Send to

Choose Destination
Pharmacoepidemiol Drug Saf. 2018 Jan;27(1):87-94. doi: 10.1002/pds.4340. Epub 2017 Nov 6.

Application and optimisation of the Comparison on Extreme Laboratory Tests (CERT) algorithm for detection of adverse drug reactions: Transferability across national boundaries.

Author information

Vigilance and Compliance Branch, Health Sciences Authority, Singapore.
Genome Institute of Singapore, Agency for Science and Technology, Singapore.
Department of Biomedical Informatics, Ajou University School of Medicine, Suwon, Republic of Korea.
Department of Biomedical Sciences, Ajou University Graduate School of Medicine, Suwon, Republic of Korea.
Academic Informatics Office, National University Health System, Singapore.
Saw Swee Hock School of Public Health, National University of Singapore, Singapore.
Yong Loo Lin School of Medicine, National University of Singapore, Singapore.
Department of Medical Statistics, Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, UK.
Health Services and Systems Research, Duke-NUS Medical School, Singapore.



The Singapore regulatory agency for health products (Health Sciences Authority), in performing active surveillance of medicines and their potential harms, is open to new methods to achieve this goal. Laboratory tests are a potential source of data for this purpose. We have examined the performance of the Comparison on Extreme Laboratory Tests (CERT) algorithm, developed by Ajou University, Korea, as a potential tool for adverse drug reaction detection based on the electronic medical records of the Singapore health care system.


We implemented the original CERT algorithm, comparing extreme laboratory results pre- and post-drug exposure, and 5 variations thereof using 4.5 years of National University Hospital (NUH) electronic medical record data (31 869 588 laboratory tests, 6 699 591 drug dispensings from 272 328 hospitalizations). We investigated 6 drugs from the original CERT paper and an additional 47 drugs. We benchmarked results against a reference standard that we created from UpToDate 2015.


The original CERT algorithm applied to all 53 drugs and 44 laboratory abnormalities yielded a positive predictive value (PPV) and sensitivity of 50.3% and 54.1%, respectively. By raising the minimum number of cases for each drug-laboratory abnormality pair from 2 to 400, the PPV and sensitivity increased to 53.9% and 67.2%, respectively. This post hoc variation, named CERT400, performed particularly well for drug-induced hepatic and renal toxicities.


We have demonstrated that the CERT algorithm can be applied across national boundaries. One modification (CERT400) was able to identify adverse drug reaction signals from laboratory data with reasonable PPV and sensitivity, which indicates potential utility as a supplementary pharmacovigilance tool.


adverse reaction; data mining; electronic medical records; laboratory abnormality; pharmacoepidemiology; pharmacovigilance; signal detection

[Indexed for MEDLINE]
Free full text

Supplemental Content

Full text links

Icon for Wiley Icon for London School of Hygiene and Tropical Medicine
Loading ...
Support Center