Send to

Choose Destination
Front Neurosci. 2017 Jun 16;11:322. doi: 10.3389/fnins.2017.00322. eCollection 2017.

Inter-Rater Reliability of Preprocessing EEG Data: Impact of Subjective Artifact Removal on Associative Memory Task ERP Results.

Author information

Mental Illness Research Education Clinical Center, Edith Nourse Rogers Memorial Veterans Hospital (VHA)Bedford, MA, United States.
Department of Neurology, Massachusetts General HospitalBoston, MA, United States.
Biospective, Inc.Montreal, QC, Canada.
Quinnipiac Medical School, Quinnipiac UniversityNorth Haven, CT, United States.
Boston University School of Medicine, Boston UniversityBoston, MA, United States.
New Jersey Medical School, Rutgers UniversityNew Brunswick, NJ, United States.
Department of Mental Health, Tewksbury HospitalTewksbury, MA, United States.
Department of Neurology, Albert Einstein College of MedicineNew York, NY, United States.
Department of Neurosurgery, University of LouisvilleLouisville, KY, United States.
Ray Dolby Brain Health Center and California Pacific Medical Center Research Institute, California Pacific Medical CenterSan Francisco, CA, United States.
Department of Neurology, Center for Brain/Mind Medicine, Brigham and Women's HospitalBoston, MA, United States.
Harvard Medical School, Harvard UniversityBoston, MA, United States.


The processing of EEG data routinely involves subjective removal of artifacts during a preprocessing stage. Preprocessing inter-rater reliability (IRR) and how differences in preprocessing may affect outcomes of primary event-related potential (ERP) analyses has not been previously assessed. Three raters independently preprocessed EEG data of 16 cognitively healthy adult participants (ages 18-39 years) who performed a memory task. Using intraclass correlations (ICCs), IRR was assessed for Early-frontal, Late-frontal, and Parietal Old/new memory effects contrasts across eight regions of interest (ROIs). IRR was good to excellent for all ROIs; 22 of 26 ICCs were above 0.80. Raters were highly consistent in preprocessing across ROIs, although the frontal pole ROI (ICC range 0.60-0.90) showed less consistency. Old/new parietal effects had highest ICCs with the lowest variability. Rater preprocessing differences did not alter primary ERP results. IRR for EEG preprocessing was good to excellent, and subjective rater-removal of EEG artifacts did not alter primary memory-task ERP results. Findings provide preliminary support for robustness of cognitive/memory task-related ERP results against significant inter-rater preprocessing variability and suggest reliability of EEG to assess cognitive-neurophysiological processes multiple preprocessors are involved.


EEG/ERP; artifacts; inter-rater reliability; memory; preprocessing

Supplemental Content

Full text links

Icon for Frontiers Media SA Icon for PubMed Central
Loading ...
Support Center