Refining the WHO Definition: Predicting Autopsy-Defined Sudden Arrhythmic Deaths Among Presumed Sudden Cardiac Deaths in the POST SCD Study
Associated Data
Abstract
Background:
Conventional definitions of sudden cardiac death (SCD) presume cardiac etiology. We studied World Health Organization (WHO) defined SCDs autopsied in the POstmortem Systematic InvesTigation of SCD (POST SCD) Study to determine whether premortem characteristics could identify autopsy-defined sudden arrhythmic death (SAD) among presumed SCDs.
Methods:
Between 2/1/11 and 4/1/16, we prospectively identified all 615 WHO-defined SCDs (144 witnessed) 18 to 90 years in San Francisco County for medical record review and autopsy via medical examiner surveillance. Autopsy-defined SADs had no extracardiac or acute heart failure cause of death. We used two nested sets of premortem predictors - an emergency medical system (EMS) set and a comprehensive set adding medical record data - to develop Least Absolute Selection and Shrinkage Operator models of SAD among witnessed and unwitnessed cohorts.
Results:
Of 615 presumed SCDs, 348 (57%) were autopsy-defined SAD. For witnessed cases, the EMS model (area under the receiver-operator curve (AUROC) 0.75 [0.67–0.82]) included presenting rhythm of ventricular tachycardia/fibrillation (VT/VF) and pulseless electrical activity (PEA), while the comprehensive (AUROC 0.78 [0.70–0.84]) added depression. If only VT/VF witnessed cases (n=48) were classified as SAD, sensitivity was 0.46 (0.36–0.57) and specificity was 0.90 (0.79–0.97). For unwitnessed cases, the EMS model (AUROC 0.68 [0.64–0.73]) included black race, male sex, age, and time since last seen normal, while the comprehensive (AUROC 0.75 [0.71–0.79]) added use of beta blockers, antidepressants, QT-prolonging drugs, opiates, illicit drugs, and dyslipidemia. If only unwitnessed cases <1 hour (n=59) were classified as SAD, sensitivity was 0.18 (0.13–0.22) and specificity was 0.95 (0.90–0.97).
Conclusions:
Our models identify premortem characteristics that can better specify autopsy-defined SAD among presumed SCDs and suggest the WHO definition can be improved by restricting witnessed SCDs to VT/VF or non-PEA rhythms and unwitnessed cases to < 1 hour since last normal, at a cost of sensitivity.
Introduction
Investigators have long sought an accurate and practical definition for sudden cardiac death (SCD). One of the most widely adopted definitions was developed by the World Health Organization (WHO), which defines SCD as sudden unexpected death either within 1 hour of symptom onset (witnessed), or within 24 hours of having been observed alive and symptom free (unwitnessed).1 As Hinkle and Thaler originally delineated in their classification of cardiac deaths in 1982, the primary utility of such a definition would be to identify sudden arrhythmic deaths (SADs).2 This notion has carried forth to the most recent 2016 American College of Cardiology/American Heart Association definition which defines SCD as “a natural death due to cardiac causes, heralded by abrupt loss of consciousness.”3 These conventional epidemiologic definitions are designed and operationalized with the intent of identifying those who died of fatal arrhythmias using information typically available at the time or after death, such as death certificates or emergency medical system (EMS) records. However, given the inherent sudden nature of these deaths, non-cardiac and non-arrhythmic etiologies which may have caused sudden death cannot be excluded without autopsy.
In the San Francisco POstmortem Systematic InvesTigation of Sudden Cardiac Death (POST SCD) Study, we systematically employed autopsy to identify SADs among all incident WHO-defined SCDs occurring countywide over a 3-year period.4 In that study, we demonstrated that only 55.8% of WHO-defined SCDs were actually autopsy-defined SAD after excluding non-arrhythmic and non-cardiac etiologies identified by postmortem investigation, including intracranial hemorrhage, occult overdose, acute heart failure, tamponade, or pulmonary embolism. In an attempt to better approximate true SAD without the high cost of autopsy, we used comprehensive premortem data from our POST SCD Study cohort to determine whether predictive models based on combinations of premortem variables typically used to define SCDs by conventional, retrospective criteria could be used to identify autopsy-defined SAD among presumed SCDs, and thereby refine the WHO definition to better specify SAD.
Methods
The authors declare that all supporting data and analytic methods are available within the manuscript and supplemental material.
Study Population
By California state law, all out-of-hospital deaths are reported to the Medical Examiner. In the San Francisco POST SCD Study, we used prospective surveillance of all out-of-hospital cardiac arrest deaths by the Medical Examiner to identify WHO-defined SCDs 18 to 90 years in San Francisco County over a 37-month period (Jan 2011-Feb 2014). In this period, total mortality in San Francisco County was 20,440 deaths, all of which were reviewed to ensure that we did not miss any WHO-defined SCDs. From a total countywide mortality of 20,440, 12,671 out-of-hospital, emergency department, and unexpected inpatient deaths were reported to the medical examiner: 2,021 were due to non-natural (e.g., trauma, homicide) causes, 2,012 did not meet age criteria, and 3,862 were inpatient, nursing home, or hospice deaths. Via daily screening of EMS records for the remaining 4,776 natural out-of-hospital or emergency department deaths, we identified 912 out-of-hospital cardiac arrest deaths, 896 (98%) of which were autopsied. After adjudication, including a comprehensive review of EMS and past medical records and family and investigator interviews, we excluded 371 of 912 out-of-hospital cardiac arrest deaths (40%) as non-sudden or non-cardiac pre-autopsy (i.e., did not meet WHO criteria for SCD) to arrive at a final cohort of 541 WHO-defined SCDs, 525 (97%) of which were autopsied.
Since the initial 3-year period, 90 additional WHO-defined SCDs were autopsied and added to our cohort from March 2014-April 2016. Comprehensive premortem data (EMS records, all available medical records, the Medical Examiner forensic scene investigation, and family, primary physician, and witness interviews) were then combined with systematic postmortem investigation (full autopsy, detailed cardiac examinations, toxicology, and histology) to adjudicate autopsy-defined SAD and non-arrhythmic sudden deaths among the 615 total WHO-defined (presumed) SCDs. Nearly all (94%) POST SCD cases had complete retrieval of all medical records up to date of sudden death. A more detailed description of the original POST SCD study design and methods including a flow chart of the study population has been reported previously.4 The study was approved by the UCSF IRB and had additional IRB approval at all 10 San Francisco County adult hospitals and 3 EMS agencies to obtain medical records.
Premortem Variable Sets
We evaluated two nested sets of premortem predictor variables that are commonly reported in other population-based studies of SCD to predict adjudicated SAD based on full autopsy, reflecting the varying breadth of information employed to adjudicate WHO-defined SCDs in these studies (Supplemental Materials: Supplemental Table 1). To develop and test a predictive model that may reflect studies or situations in which only EMS record alone are available,5 a more restrictive “EMS” set of 40 predictor variables included data available from EMS records: demographics (age, gender, race/ethnicity), symptoms, whether the death was witnessed, and presenting cardiac rhythm, if available. To develop and test a predictive model that may reflect cohort studies with access to more comprehensive resources in the determination of WHO-defined SCDs, a more extensive “comprehensive” set of 114 variables added information from the Medical Examiner forensic investigator report and medical chart review, including lifestyle factors, medical history and co-morbidities, medication use, and most recent electrocardiogram (ECG)/transthoracic echocardiogram (TTE) results if performed.
Model Selection and Validation
The cohort was divided into “witnessed” and “unwitnessed” sub-cohorts. Thus, four models in total were produced—with one EMS and one comprehensive model for each of the witnessed and unwitnessed sub-cohorts. In an initial screening step, variables with P>0.1 in unadjusted logistic regression with the outcome of autopsy-defined SAD were omitted from further consideration. We then used Least Absolute Selection and Shrinkage Operator (LASSO) modelling to develop final predictive models to distinguish autopsy-defined SAD from non-SAD among presumed SCDs.6 LASSO improves predictive accuracy by shrinking the regression coefficients for each variable in approximate inverse proportion to the information it provides. To protect against overfitting, we used the “one standard error rule,” focusing on the parsimonious model within one standard error of the larger model that minimizes cross-validated prediction error.
From each of the four resulting parsimonious models, we derived the predicted probability of autopsy-defined SAD for different clinically relevant scenarios, calculated the area under the receiver operating characteristic (AUROC) curve, with 95% confidence intervals, and tabulated test characteristics including sensitivity, specificity, positive predictive value, and negative predictive value for a range of possible probability thresholds, each with exact binomial confidence intervals. It should be noted that LASSO does not provide standard errors, confidence intervals, or P-values for the logistic model odds ratios.
Electrocardiogram and Echocardiogram Variables
Of the predictors we considered, only those derived from ECGs and TTEs had substantial numbers of missing values, with 311 (51%) of cases lacking an ECG and 473 (77%) lacking a TTE on record—not unexpected given the community-based nature of POST SCD. Missing values were singly imputed (i.e., assigned) as negative. To address sensitivity to this imputation, indicator variables were created for whether these tests were performed and were also considered in developing the models. If an ECG-based test result as well as the indicator variable for ECG (or a TTE-based result and the indicator variable for TTE) were included in the model, this would have the effect of distinguishing SAD rates for positive, negative, and missing test results.
Results
Study Population: POST SCD Cases
We included 615 WHO-defined SCDs, 348 of which (57%) were adjudicated as autopsy-defined SAD.4 EMS records were available for all cases and comprehensive medical records were retrieved for 94% of cases. Premortem characteristics of the subjects are shown in Table 1.
Table 1.
Premortem characteristics of witnessed and unwitnessed WHO-defined SCDs in POST SCD.
| Witnessed (n=144) | Unwitnessed (n=471) | |||||
|---|---|---|---|---|---|---|
| Non-SAD | Autopsy-defined SADs | P* | Non-SAD | Autopsy-defined SADs | P* | |
| N | 51 | 93 | 216 | 255 | ||
| Age, median (IQR) | 64 (55–72) | 60 (52–67) | .14 | 59 (51–71) | 65 (55–76) | <.01 |
| Male, (%) | 34 (67%) | 76 (82%) | .04 | 133 (62%) | 190 (75%) | <.01 |
| Race (%) | ||||||
| Asian | 11 (22%) | 19 (20%) | .87 | 34 (16%) | 56 (22%) | .09 |
| Black | 7 (14%) | 7 (8%) | .24 | 46 (21%) | 33 (13%) | .02 |
| Latino | 1 (2%) | 6 (6%) | .42 | 20 (9%) | 17 (7%) | .30 |
| White | 28 (55%) | 55 (59%) | .62 | 113 (52%) | 145 (57%) | .32 |
| Other | 4 (8%) | 6 (6%) | .74 | 3 (1%) | 4 (2%) | 1 |
| Hours elapsed since last seen normal (IQR) | N/A | N/A | - | 13 (5–21) | 8 (2–18) | <.01 |
| Hours elapsed ≥ 1 | - | - | - | 181 (93%) | 193 (81%) | <.01 |
| Hours elapsed ≥ 6 | - | - | - | 144 (74%) | 131 (55%) | <.01 |
| Hours elapsed ≥ 12 | - | - | - | 102 (53%) | 95 (40%) | .01 |
| Hours elapsed ≥ 15 | - | - | - | 88 (56%) | 69 (44%) | <.01 |
| Symptom | ||||||
| Chest pain | 6 (12%) | 9 (10%) | .70 | 4 (2%) | 11 (4%) | .19 |
| Shortness of breath | 12 (24%) | 9 (10%) | .03 | 12 (6%) | 9 (4%) | .29 |
| Vomiting/nausea | 3 (6%) | 9 (10%) | .54 | 4 (2%) | 0 (0%) | .05 |
| Fatigue | 3 (6%) | 6 (6%) | 1 | 12 (6%) | 16 (6%) | .74 |
| Cough | 3 (6%) | 1(1%) | .13 | 12 (6%) | 6 (2%) | .08 |
| Presenting EMS rhythm | ||||||
| Agonal/idioventricular | 2 (4%) | 3 (3%) | 1 | 0 (0%) | 2 (1%) | .50 |
| PEA | 15 (29%) | 3 (3%) | 0 | 5 (2%) | 3 (1%) | .48 |
| VT/VF | 5 (10%) | 43 (46%) | 0 | 2 (1%) | 6 (2%) | .30 |
| Asystole | 20 (39%) | 35 (37%) | .85 | 206 (95%) | 236 (93%) | .21 |
| History of: | ||||||
| AF/AFL | 6 (12%) | 5 (5%) | .20 | 22 (10%) | 25 (10%) | .89 |
| Excess alcohol use | 8 (16%) | 9 (10%) | .29 | 70 (32%) | 61 (24%) | .04 |
| Any cardiac disease† | 26 (51%) | 38 (41%) | .24 | 81 (38%) | 118 (46%) | .06 |
| Any neurological disease† | 12 (24%) | 11 (12%) | .07 | 56 (26%) | 52 (20%) | .15 |
| CHF | 5 (10%) | 9 (10%) | 1 | 22 (10%) | 43 (17%) | .04 |
| CKD (non-ESRD) | 5 (10%) | 6 (6%) | .52 | 26 (12%) | 32 (13%) | .87 |
| COPD | 4 (8%) | 4 (4%) | .45 | 32 (15%) | 34 (13%) | .64 |
| Stroke | 3 (6%) | 2 (2%) | .35 | 14 (6%) | 18 (7%) | .80 |
| Depression | 8 (15%) | 5 (5%) | .06 | 58 (27%) | 39 (15%) | <.01 |
| DM | 10 (20%) | 19 (20%) | .91 | 43 (20%) | 60 (24%) | .34 |
| Dyslipidemia | 13 (25%) | 31 (33%) | .33 | 46 (21%) | 94 (37%) | <.01 |
| HTN | 27 (53%) | 53 (57%) | .64 | 105 (41%) | 150 (59%) | .03 |
| Illicit drug use | 8 (16%) | 6 (6%) | .08 | 54 (25%) | 29 (11%) | <.01 |
| LVH | 13 (25%) | 12 (13%) | .06 | 38 (18%) | 58 (23%) | .17 |
| MI | 3 (6%) | 13 (14%) | .17 | 27 (13%) | 46 (18%) | .09 |
| Non-metastatic cancer | 5 (10%) | 10 (11%) | 1 | 20 (9%) | 35 (14%) | .13 |
| Other psychiatric disorder† | 6 (12%) | 4 (4%) | .17 | 32 (15%) | 30 (12%) | .33 |
| Seizure disorder | 4 (8%) | 2 (2%) | .19 | 24 (11%) | 15 (6%) | .04 |
| Syncope | 4 (8%) | 5 (5%) | .72 | 14 (6%) | 20 (8%) | .57 |
| Tobacco use | 21 (41%) | 23 (25%) | .04 | 92 (43%) | 107 (42%) | .89 |
| Medication Use†: | ||||||
| Antiarrhythmic | 3 (6%) | 4 (4%) | .70 | 19 (9%) | 22 (9%) | .95 |
| Anticoagulant | 13 (25%) | 28 (30%) | .56 | 62 (29%) | 92 (36%) | .09 |
| Beta blocker | 9 (18%) | 28 (30%) | .11 | 50 (23%) | 94 (37%) | <.01 |
| SSRI | 3 (6%) | 4 (4%) | .70 | 36 (17%) | 14 (5%) | <.01 |
| Thiazide | 8 (16%) | 18 (19%) | .58 | 48 (22%) | 52 (20%) | .63 |
| Electrocardiogram‡: | 29 (57%) | 34 (37%) | .02 | 110 (51%) | 131 (51%) | .92 |
| Heart block, 1st degree | 5 (10%) | 4 (4.3%) | .28/.05 | 7 (3%) | 16 (6%) | .13/.32 |
| Heart block, 3rd degree | 3 (6%) | 1 (1%) | .13/.03 | 2 (1%) | 10 (4%) | .04/.17 |
| IVCD | 2 (4%) | 2 (2%) | .61/.07 | 2 (1%) | 10 (4%) | .04/.16 |
| LAFB | 5 (8%) | 1 (1%) | .05/.03 | 7 (3%) | 4 (2%) | .36/.48 |
| LBBB | 0 (0%) | 4 (4%) | .30/.01 | 3 (1%) | 11 (4%) | .09/.21 |
| NSVT | 1 (2%) | 0 (0%) | .35/.03 | 3 (1%) | 1 (.4%) | .34/.53 |
| PVCs | 3 (6%) | 4 (4%) | .70/.07 | 9 (4%) | 13 (5%) | .63/.89 |
| RBBB | 4 (8%) | 3 (3%) | .24/.05 | 9 (4%) | 10 (4%) | .89/.98 |
| Echocardiogram‡: | 11 (22%) | 18 (19%) | .75 | 44 (20%) | 68 (27%) | .11 |
| Aortic stenosis | 1 (2%) | 0 (0%) | .35/.93 | 2 (1%) | 4 (2%) | .69/.26 |
| Mitral regurgitation | 4 (8%) | 3 (3%) | .24/.47 | 5 (2%) | 14 (5%) | .10/.14 |
| Mitral valve prolapse | 0 (0%) | 3 (4%) | .55/.70 | 3 (2%) | 2 (1%) | .67/.13 |
AF/AFL, atrial fibrillation/atrial flutter; CKD, chronic kidney disease; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; DM, diabetes mellitus; EMS, emergency medical system; ESRD, End-stage renal disease; HTN, hypertension; IQR, interquartile range; IVCD, interventricular conduction delay; LAFB, left anterior fascicular block; LBBB, left bundle branch block; LVH, left ventricular hypertrophy; MI, myocardial infarction; NSVT, non-sustained ventricular tachycardia; PEA, pulseless electrical activity; POST SCD, POstmortem Systematic InvesTigation of Sudden Cardiac Death; PVCs, premature ventricular contractions; RBBB, right bundle branch block; SAD, sudden arrhythmic death; SCD, sudden cardiac death; SSRI, selective serotonin reuptake inhibitor; VT/VF, ventricular tachycardia/ventricular fibrillation
Predictive Models for Witnessed Sudden Deaths
From an initial EMS set of 40 premortem variables, 4 were retained after the initial screening. The EMS LASSO model retained only presenting rhythm of ventricular tachycardia/ventricular fibrillation (VT/VF) and pulseless electrical activity (PEA) (Table 2) and achieved an AUROC of 0.75 (0.67–0.82). Probabilities of autopsy-defined SAD for the three possible conditions under this model (VT/VF, PEA, and neither) along with associated test characteristics are presented in Table 2. Notably, if only VT/VF cases (n=48) are classified as SAD, the sensitivity was 0.46 (0.36–0.57), specificity was 0.90 (0.79–0.97), and positive predictive value was 0.90 (0.77–0.97). If only PEA cases (n=18) are classified as non-SAD, the sensitivity was 0.97 (0.91–0.99), specificity was 0.29 (0.17–0.44), and negative predictive value was 0.83 (0.59–0.96).
Table 2.
EMS and comprehensive LASSO models for autopsy-defined SAD among 144 witnessed WHO-defined SCDs with predicted probabilities for selected scenarios
| Odds Ratio For Autopsy-Defined SAD | Selected WHO SCD Scenarios* | |||||||
|---|---|---|---|---|---|---|---|---|
| Premortem Variables | EMS Model | Comprehensive Model | ||||||
| PEA | 0.42 | 0.42 | + | + | - | - | - | |
| VT/VF | 1.91 | 1.92 | - | - | - | - | + | |
| Depression‡ | 0.92 | + | - | + | - | - | ||
| EMS model: Predicted probability of SAD‡ | 0.42 | 0.42 | 0.63 | 0.63 | 0.76 | |||
| Comprehensive model: Predicted probability of SAD | 0.40 | 0.42 | 0.61 | 0.63 | 0.76 | |||
| Comprehensive Model Test Characteristics Associated with Scenario as Probability Threshold† | Sensitivity | 100 (96–100) | 97 (91–99) | 96 (89–99) | 46 (36–57) | 0 (0–4) | ||
| Specificity | 0 (0–7) | 29 (17–44) | 39 (26–54) | 90 (77–97) | 100 (93–100) | |||
| PPV | 65 (56–72) | 71 (63–79) | 74 (65–82) | 90 (77–97) | - | |||
| NPV | - | 83 (59–96) | 83 (63–95) | 48 (38–58) | 35 (28–44) | |||
EMS, emergency medical system; LASSO, Least Absolute Selection and Shrinkage Operator; NPV, negative predictive value; PEA, pulseless electrical activity; PPV, positive predictive value; SAD, sudden arrhythmic death; SCD, sudden cardiac death; VT/VF, ventricular tachycardia/ventricular fibrillation; WHO, World Health Organization
From an initial comprehensive set of 114 premortem variables, 22 were retained after the initial screening. The comprehensive LASSO model retained VT/VF, PEA, and past medical history of depression (Table 2). The AUROC for this model was 0.78 (0.70–0.84); sensitivity, specificity, negative predictive value, and positive predictive value for different clinical scenarios are shown in Table 2. If only VT/VF cases are classified as SAD, or PEA as non-SAD, performance is the same as the EMS model.
VT/VF was not 100% specific for SAD: of the 48 VT/VF cases, 43 (90%) were determined to be SAD. Of the 5 non-SAD VT/VF cases, 3 were due to overdose (2 of which had history of depression) and 2 were due to precipitating aortic dissection. Of the 18 PEA cases, 15 (83%) were determined to be of non-SAD etiology. Among 13 cases with depression, 5 (38%) were SAD (4 of which had a presenting rhythm of VT/VF). Of 6 cases with depression but without VT/VF or PEA, 2 died of overdose, 1 of neurological cause, 1 of gastrointestinal cause, 1 of SAD, and 1 of other non-cardiac cause.
Predictive Models for Unwitnessed Sudden Deaths
From an initial EMS premortem variable set of 40 predictors, 24 were retained after the initial screening. The EMS model retained black race, male sex, age, and hours elapsed since last seen normal (Table 3). The AUROC of this model was 0.68 (0.64–0.73). From an initial comprehensive premortem variable set of 114 variables, 48 were retained after the initial screening. The comprehensive model included all variables in the EMS model plus beta blocker use, selective serotonin reuptake inhibitor use, QT-prolonging drug use, opiate use, dyslipidemia, and history of illicit drug use (Table 3). The AUROC of this model was 0.75 (0.71–0.79).
Table 3.
EMS and comprehensive LASSO models for autopsy-defined SAD among 471 unwitnessed WHO-defined SCDs
| Odds Ratio for Autopsy-defined SAD* | |||
|---|---|---|---|
| Premortem Variables | EMS | Comprehensive | |
| Demographics | Age <60 | Reference | Reference |
| Age 60–64 | 1.21 | 1.09 | |
| Age ≥ 65 | 1.49 | 1.17 | |
| Black | 0.95 | 0.98 | |
| Male | 1.41 | 1.22 | |
| Circumstances of death: Time since last seen normal | <1 hour | Ref | Ref |
| 1–6 hours | 0.76 | 0.82 | |
| 6–15 hours | 0.62 | 0.72 | |
| 15–24 hours | 0.52 | 0.61 | |
| Medications | Beta blocker | 1.25 | |
| SSRI | 0.63 | ||
| QT-Prolonging Drug | 0.76 | ||
| Medical History | Opiate Use | 0.76 | |
| Dyslipidemia | 1.27 | ||
| Illicit drug use | 0.81 | ||
| AUROC | 0.68 (0.64–0.73) | 0.75 (0.71–0.79) | |
AUROC, area under the receiver-operator curve; EMS, emergency medical system; LASSO, Least Absolute Selection and Shrinkage Operator; PEA, pulseless electrical activity; SAD, sudden arrhythmic death; SSRI, selective serotonin reuptake inhibitor; VT/VF, ventricular tachycardia/ventricular fibrillation
For both models, the odds ratios are presented in Table 3. Sensitivities and specificities associated with varying probability thresholds are presented in Figures 1 (EMS) and and22 (comprehensive). Probabilities of clinically relevant scenarios and associated test characteristics are also presented in these Figures. Notably, of 59 unwitnessed cases with < 1 hour elapsed since last seen normal, 76% were SAD. If only these 59 cases were classified as SAD, sensitivity was 0.18 (0.13–0.22) and specificity of 0.95 (0.90–0.97).
EMS model for Autopsy-defined SAD Among Unwitnessed WHO-Defined SCDs: sensitivity and specificity curves with predicted probabilities of selected scenarios. Sensitivity (black, blue) and specificity (orange, red) curves are displayed for a range of probability thresholds for classifying WHO-defined SCDs as autopsy-adjudicated SAD in the unwitnessed EMS model. Dashed lines highlight probability thresholds associated with clinically relevant scenarios (A-E) and corresponding test characteristics. CI, confidence interval; EMS, emergency medical system; SAD, sudden arrhythmic death; WHO, World Health Organization.
Comprehensive model for Autopsy-defined SAD Among Unwitnessed WHO-Defined SCDs: sensitivity and specificity curves with predicted probabilities of selected scenarios. Sensitivity (black, blue) and specificity (orange, red) curves for a range of probability thresholds for classifying WHO-defined SCDs as autopsy-adjudicated SAD in the unwitnessed EMS model. Dashed lines highlight probability thresholds associated with clinically relevant scenarios (A-E) and corresponding test characteristics. CI, confidence interval; EMS, emergency medical system; SAD, sudden arrhythmic death; SSRI, selective serotonin reuptake inhibitor; QTp, QT-prolonging; WHO, World Health Organization.
An Excel calculator for predicted probabilities of autopsy-defined SAD among presumed, WHO-defined SCDs is available at: https://ucsfhealthcardiology.ucsf.edu/sites/ucsfhealthcardiology.ucsf.edu/files/2018-08/SADCalculator.xlsx (Figure 3). These predicted probabilities are intended to be employed in the specific contexts outlined in the discussion section.
Demo of unwitnessed EMS calculator for probability of autopsy-defined SAD. This figure demonstrates the functionality of the calculator. For a given patient, an investigator will enter the status of all variables (green column) included in the applicable model (a calculator has been built for each of the four models produced – this figure demonstrates the unwitnessed EMS calculator). The calculator will then output the associated predicted probability of autopsy-defined SAD (red box). In addition, the calculator demonstrates the equation used to calculate the predicted probability such that it can be readily extrapolated to the entire cohort of interest. The full calculator also demonstrates the test characteristics (sensitivity and specificity) associated with different probability thresholds such that the investigator can select the threshold most appropriate for their intended purpose.
Discussion
We used the unique autopsied POST SCD study population to develop predictive models using combinations of premortem variables to identify autopsy-defined SADs among WHO-defined (presumed) SCDs. The presented models may be useful in identifying true SADs among SCDs defined by conventional criteria7–11 or improving consensus definitions for SCD.12 Separate models were generated for unwitnessed and witnessed cases using EMS and comprehensive sets of premortem data. In POST SCD, the premortem variables were chiefly used to determine whether out-of-hospital cardiac arrest deaths met criteria as WHO-defined SCDs, while autopsy results were the key determinant in the adjudication of SAD among these presumed SCDs. Our models predicted autopsy-defined SAD with AUROCs ranging from 0.68 to 0.78. The moderate performance of our models is not surprising given the array of non-cardiac and non-arrhythmic conditions we found on autopsy in POST SCD and affirms the importance of autopsy in determining the underlying cause of sudden deaths.4 Therefore, though it may not be possible to completely replace autopsy, our results suggest that our models may assist in the identification of SCDs that more closely approximate those confirmed by postmortem investigation. Our results also suggest that the WHO definition can be refined to better specify SAD by restricting witnessed SCD cases to VT/VF or non-PEA rhythms and unwitnessed cases to < 1 hour last seen normal.
A full autopsy, including vitreous chemistries, toxicology, and histology, is the gold standard for determining cause of death, but outside of the POST SCD study has rarely been used to define SAD. As such, recent risk models of SCD and SAD developed in large cohorts, intended to identify high-risk populations to better study and implement prevention strategies,8,9 are limited by an outcome definition that is largely unconfirmed, due to the very low autopsy rates typical for out-of-hospital sudden deaths13 and, as we demonstrated in POST SCD, heterogenous and problematic as a proxy for either SCD or SAD.14 For instance, the recent VEST trial was underpowered to show a difference in the primary outcome of SAD in defibrillator vest and control arms after acute MI, but this may be due to misclassification of SADs.15,16 Similarly, misclassification may limit studies of genetic determinants of SCD,16 where accurate phenotype is critical for precise genotype-phenotype associations.17
The witnessed models performed better than the unwitnessed counterparts, mainly because presenting rhythm is more meaningful in witnessed cases; unwitnessed WHO-defined SCDs were almost exclusively found in asystole, even though some likely started as VT/VF. This is reflected in the EMS witnessed model, where VT/VF emerged as the lone positive predictor and PEA as the lone negative predictor. Though the AUROC of the EMS model was below the conventional standard of 0.8,18 we found that presumed SCDs with VT/VF were almost always adjudicated as SAD on autopsy, while PEA cases were almost always adjudicated as non-SAD. As a result, high specificity and sensitivity can be achieved simply by excluding PEA cases and/or including VT/VF cases. Notably, while a presenting rhythm of VT/VF is almost universally regarded as synonymous with SAD, 10% of such cases were due to non-arrhythmic causes (overdoses and aortic dissections) identified by postmortem investigation.
These results may be used to refine current definitions for SCDs to better specify SAD. If we modified the WHO definition for witnessed cases of SCD to exclude those found in PEA, specificity would be improved to 0.29 (0.17–0.44), while maintaining sensitivity of 0.97 (0.91–0.99), i.e., this would result in a more specific definition without greatly impacting sensitivity. However, the definition of PEA in the EMS record should be carefully assessed as done in POST SCD, as we found a number of errors with the use of this term. Similarly, investigators with a goal of identifying cases of SAD with high confidence (such as for inclusion in genetic or molecular association studies) could narrow their focus to VT/VF cases, given that these rhythms had a positive predictive value of 0.9.
The comprehensive witnessed model added history of depression and increased the AUROC from 0.75 to 0.78. Given the improved performance, an investigator might use the predicted probability produced from the model to weight a cohort of WHO-defined SCDs to better approximate autopsy-adjudicated SAD (formulas to calculate predicted probabilities from each model is included in Excel calculator for ease of applying on a cohort level). Furthermore, as proposed for the EMS model and as applies to all models presented, a researcher could employ various probability thresholds to define autopsy-defined SAD that meet individual requirements for sensitivity and specificity.
The witnessed models lost the majority of its discriminatory power in the 78 (46%) witnessed cases that did not have a presenting rhythm of VT/VF or PEA. To investigate if any other variables could be useful in its absence, we attempted to build both an EMS and a comprehensive model for witnessed cases that did not consider presenting rhythm. However, the resulting models were null suggesting that there may not be a useful substitute for presenting rhythm in determining autopsy-adjudicated SADs among witnessed sudden deaths.
As might be expected, the unwitnessed models did not perform as well as their witnessed counterparts, likely due to the greater inherent heterogeneity of unwitnessed sudden deaths and the absence of a meaningful presenting rhythm. With respect to conventional SCD definitions, typified by the WHO definition, our model suggests that among unwitnessed sudden deaths, a reduction of the period last seen in normal health from 24 to 12 or even 1 hour would maximize specificity for autopsy-defined SAD. However, unlike excluding PEA for witnessed cases, this would come at a greater cost to sensitivity. These considerations can be balanced based upon objective: a highly sensitive SCD definition may be desired from a public health perspective to provide a liberal estimate of overall burden, while a separate, more specific definition for SAD may be the goal for implantable cardioverter-defibrillator trials or molecular studies.
Given the lower performance of the unwitnessed models, we would not recommend directly using the predicted probabilities to weight each case. However, from the ROC curves these models perform well at either extreme of the probability spectrum. Therefore, using probability cutoffs at these extremes could be used to create cohorts that are highly specific or maintain sensitivity with interval increases in specificity.
Study Limitations
Several limitations are worth addressing. First, our ability to validate our models is limited due to the lack of other unbiased cohorts of autopsy-adjudicated SAD, especially the lack of non-cardiac cases critical to evaluating the discriminatory performance of our models. Given the sample size of our cohort, cross-validation was not feasible. However, we employed LASSO, which is a well-established method using cross-validation of prediction error to optimize the bias-variance tradeoff.19,20 Moreover, we employed the one-standard error rule to reduce the risk of overfitting. Second, because predictive models do not account for confounding or mediation and their sole selection criteria is to maximize predictive performance, causal inferences should not be drawn from our results; nonetheless, they may generate hypotheses to test in future research. Many of the variables included in our models have been previously identified as risk factors for SAD: presenting rhythm of VT/VF, older age, male sex, beta blocker use, and dyslipidemia. Of the factors predicting lower probability of SAD, presenting rhythm of PEA, history of depression, selective serotonin receptor inhibitor use, opiate use, and history of illicit drug use are also biologically plausible risk factors for non-arrhythmic causes of death, which was chiefly occult overdose (14% of all presumed SCDs) in POST SCD. The other major negative predictor was increasing hours elapsed from last seen in normal health among unwitnessed cases, which may be explained by the fact that the timeline of “suddenness” is less certain and therefore arrhythmic cause less likely the longer a victim was unwitnessed.
Additionally, our models are based on data from a diverse metropolitan area that may not be entirely reflective of other populations. Risk score and prediction models are often generalized beyond the populations (e.g., Framingham) from which they were originally derived and this practice warrants careful consideration. The discriminatory power and generalizability of our models may be threatened if the underlying substrates of the non-SADs and SADs are different. Given the unique nature of the San Francisco POST SCD Study, the degree of potential mismatch in the distribution of autopsy-adjudicated etiologies of presumed SCD in other communities is unknown. In lieu of other comprehensive postmortem data, we are reassured that the predictors of non-SAD and SADs identified by our models are prevalent throughout the U.S. In regards to SADs, because the majority share a common substrate of atherosclerosis and ischemia/infarction, irrespective of population, we are confident that the positive predictors identified by our model are generalizable. In regards to predictors of non-SADs, occult overdose death was a surprising contributor and hence, our predictors of depression and illicit drug use may be less informative in areas with lower overdose rates. The degree to which this is the case is unknown without systematic toxicology to identify these causes among other cohorts of presumed SCDs. Notably, the available evidence (and current opioid epidemic) suggests that occult overdose is an important contributor to apparent SCD in other regions, including a recent Danish study in which more than half of all toxicologically investigated SCD victims had positive post-mortem toxicological findings.21 Moreover, the rate of overdose sudden deaths in San Francisco is similar to other urban communities such as King County (Seattle).22 Even if the opioid epidemic comes under control, history informs that other illicit drugs likely will take its place in the future, much as opioids took the place of barbiturates, benzodiazepines, and cocaine before it.23 Given that postmortem toxicology is not routine and costly, our models may represent a cost-effective means of identifying causes of death that have a high likelihood of not being true SAD.
Conclusions
Our results suggest that the conventional definition of SCD can be substantially improved to specify SAD by restricting witnessed cases to VT/VF or non-PEA rhythms and unwitnessed cases to <1 hour last seen normal, at a cost of sensitivity. As we demonstrated in POST SCD, conventional definitions for SCD have limited accuracy for autopsy-confirmed SAD. Because gold standard autopsy adjudication may not be widely practical, our models can be used to better approximate true SAD among existing cohorts of presumed SCDs without the associated prohibitive costs.
Supplementary Material
007171 - Supplemental Material
007171_graphic abstract
Acknowledgments:
The authors are grateful for the efforts of all personnel in the Office of the Chief Medical Examiner and SF EMS. Dr. Tseng contributed to study design and funding acquisition, data analysis, and article drafting. Dr. Salazar contributed to data analysis and article drafting. Drs. Olgin, Ursell, and Hart contributed to study design and article revision. Dr. Kim contributed to data acquisition and article revision. Mses. Bedigian and Probert contributed to data acquisition and presentation. Dr. Moffatt contributed to study design, data acquisition and analysis, and article revision. Dr. Vittinghoff contributed to the study design, data analysis, and article drafting.
Sources of Funding: This study was funded by NIH R01 HL102090 (National Heart, Lung, and Blood Institue to ZHT). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.
Footnotes
Disclosures: Dr. Tseng reports grants from NIH/NHLBI and CDC during the conduct of the study, and personal fees from Biotronik, outside the submitted work. Dr. Olgin reports grants from NIH/NIBIB/NHLBI and ZOLL, outside the submitted work. Dr. Kim reports grants from NIH/NCATS and NIH/NINDS during the conduct of the study, research grants from SanBio, and consulting fees as a DSMB member from Neuravi outside of the submitted work. Dr. Ursell reports personal fees from St. Jude Medical, Biotronik and Biosense Webster, outside the submitted work. Dr. Salazar, Ms. Bedigan, Ms. Probert, Dr. Moffatt, Dr. Hart, and Dr. Vittinghoff report no disclosures.
Facebook
Twitter
Google+