Send to

Choose Destination
J Biomed Inform. 2015 Dec;58 Suppl:S20-9. doi: 10.1016/j.jbi.2015.07.020. Epub 2015 Aug 28.

Annotating longitudinal clinical narratives for de-identification: The 2014 i2b2/UTHealth corpus.

Author information

School of Library and Information Science, Simmons College, Boston, MA, USA. Electronic address:
Department of Information Studies, State University of New York at Albany, Albany, NY, USA.


The 2014 i2b2/UTHealth natural language processing shared task featured a track focused on the de-identification of longitudinal medical records. For this track, we de-identified a set of 1304 longitudinal medical records describing 296 patients. This corpus was de-identified under a broad interpretation of the HIPAA guidelines using double-annotation followed by arbitration, rounds of sanity checking, and proof reading. The average token-based F1 measure for the annotators compared to the gold standard was 0.927. The resulting annotations were used both to de-identify the data and to set the gold standard for the de-identification track of the 2014 i2b2/UTHealth shared task. All annotated private health information were replaced with realistic surrogates automatically and then read over and corrected manually. The resulting corpus is the first of its kind made available for de-identification research. This corpus was first used for the 2014 i2b2/UTHealth shared task, during which the systems achieved a mean F-measure of 0.872 and a maximum F-measure of 0.964 using entity-based micro-averaged evaluations.


Annotation; De-identification; HIPAA; Natural language processing

[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for Elsevier Science Icon for PubMed Central
Loading ...
Support Center