Reporting Methodology of Neurosurgical Studies Utilizing the American College of Surgeons-National Surgical Quality Improvement Program Database: A Systematic Review and Critical Appraisal

Background: Use of large databases such as the American College of Surgeons-National Surgical Quality Improvement Program (ACS-NSQIP) has become increasingly common in neurosurgical research.

Objective: To perform a critical appraisal and evaluation of the methodological reporting for studies in neurosurgical literature that utilize the ACS-NSQIP database.

Methods: We queried Ovid MEDLINE, EMBASE, and PubMed databases for all neurosurgical studies utilizing the ACS-NSQIP. We assessed each study according to number of criteria fulfilled with respect to Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement, REporting of studies Conducted using Observational Routinely-collected Health Data (RECORD) Statement, and Journal of American Medical Association-Surgical Section (JAMA-Surgery) Checklist. A separate analysis was conducted among papers published in core and noncore journals in neurosurgery according to Bradford's law.

Results: A total of 117 studies were included. Median (interquartile range [IQR]) scores for number of fulfilled criteria for STROBE Statement, RECORD Statement, and JAMA-Surgery Checklist were 20 (IQR:19-21), 9 (IQR:8-9), and 6 (IQR:5-6), respectively. For STROBE Statement, RECORD Statement, and JAMA-Surgery Checklist, item 9 (potential sources of bias), item 13 (supplemental information), and item 9 (missing data/sensitivity analysis) had the highest number of studies with no fulfillment among all studies (56, 68, 50%), respectively. When comparing core journals vs noncore journals, no significant difference was found (STROBE, P = .94; RECORD, P = .24; JAMA-Surgery checklist, P = .60).

Conclusion: While we observed an overall satisfactory reporting of methodology, most studies lacked mention of potential sources of bias, data cleaning methods, supplemental information, and external validity. Given the pervasive role of national databases and registries for research and health care policy, the surgical community needs to ensure the credibility and quality of such studies that ultimately aim to improve the value of surgical care delivery to patients.