• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
J Am Coll Surg. Author manuscript; available in PMC Feb 1, 2011.
Published in final edited form as:
PMCID: PMC2853922

Lack of Emergency Medical Services Documentation Is Associated with Poor Patient Outcomes: A Validation of Audit Filters for Prehospital Trauma Care

Dann J Laudermilch, BS,* Melissa A Schiff, MD, MPH, Avery B Nathens, MD, MPH, PhD, FACS, and Matthew R Rosengart, MD, MPH, FACS*



Our previous Delphi Study identified several audit filters considered sensitive to deviations in prehospital trauma care and potentially useful in conducting performance improvement (PI), a process currently recommended by the American College of Surgeons Committee on Trauma (ACS-COT). This study validates two of those proposed audit filters.

Study Design

We studied 4744 trauma patients using the electronic records of the Central Region Trauma registry and Emergency Medical Services (EMS) patient logs for the period January 1, 2002 to December 31, 2004. We studied whether 1) request by on-scene Basis Life Support (BLS) for Advanced Life Support (ALS) assistance or 2) failure by EMS personnel to record basic patient physiology at the scene was associated with increased in-hospital mortality. We performed multivariate analyses, including a propensity score quintile approach, adjusting for differences in case mix and clustering by hospital.


Overall mortality was 6.1%. A total of 28.2% (n=1337) of EMS records were missing patient scene physiologic data. Multivariate analysis revealed that patients missing one or more measures of patient physiology at the scene had increased risk of death (adjusted OR 2.15; 95% CI 1.13–4.10). In 17.4% (n=402) of cases BLS requested ALS assistance. Patients for whom BLS requested ALS had a similar risk of death as patients for whom ALS was initially dispatched (OR 1.04; 95% CI 0.51 to 2.15).


Failure of EMS to document basic measures of scene physiology is associated with increased mortality. This deviation in care may serve as a sensitive audit filter for performing performance improvement. The need by BLS for ALS assistance was not associated with increased mortality.


To ensure that an organized system of trauma care evolves and improves over time, there must be a mechanism for continued system review to ensure that protocols are being followed, that sources of error are identified and addressed, and recurrences prevented. Current recommendations by the American College of Surgeons Committee on Trauma (ACS-COT) include a performance improvement (PI) process, which describes the continuous evaluation of the center and providers through structured review of the process of care and patient outcome.14 However, current center-oriented (i.e. patient morbidity and mortality) outcome performance measures are insensitive to regional system failures, as well as, to the effect of any corrective measure implemented for suboptimal performance. In an evaluation of pediatric trauma care using audit filters, Kissoon concluded that deviations from care persist, occurring in approximately 50% of cases, and that mortality was an inadequate indicator of the quality of the trauma system. Relying upon morbidity and mortality neglects those system failures that are not associated with such consequences. Furthermore, using the criterion of patient outcome to assess the effectiveness of any corrective strategy is neither sensitive nor specific; certainly there are circumstances in which death is not avoidable.5 Similar studies note that current trauma filters perform inconsistently; and hence current quality indictors lack scientific validation.6

The development of expectations or criteria from evidence-based guidelines presents an alternative for measuring the process of expected outcomes and consistency of care. Monitoring these process measures may improve the sensitivity to identify deviations in care that are associated with patient outcome. To identify criteria to evaluate prehospital trauma care, we conducted a Delphi study, involving 81 nationally recognized experts in trauma system development.7 From that study, consensus was achieved for 28 criteria considered sensitive to monitoring variations in prehospital care and of potential utility as audit filters. These proposed filters encompassed the main stages of prehospital trauma care, triage (n=8), EMS care (n=15), transportation time (n=2), and interfacility transfer (n=3).

Despite expert consensus as to the face validity of these filters, the association of each to deviations in patient care associated with untoward patient outcome is unknown. Their clinical utility, however, is founded upon the premise that as a surrogate for poor outcome, monitoring these filters improves PI, and corrective interventions in circumstances when they are violated improves patient care and outcome. Hence, we have begun to validate these filters to determine which might enhance the process of PI. Utilizing linked patient and EMS data, we were able to explore whether two of these criteria are sensitive to patient outcome: 1) request by on-scene Basis Life Support (BLS) for Advanced Life Support (ALS) assistance; and 2) failure by EMS personnel to record patient physiology at the scene. We hypothesized that 1) circumstances in which on-scene BLS requests ALS assistance; and 2) failure of EMS to record basic scene physiology would be associated with worse outcome and provide a sensitive means by which to assist PI. Validating these filters would ensure that identification and the subsequent institution of plans for remediation would make prehospital care PI feasible and efficient, and reduce patient morbidity, mortality, and the cost of care.


Study design

This is a retrospective cohort study to determine the content and criterion validity of two previously identified criteria considered sensitive in evaluating prehospital care by studying their association with patient mortality: 1) request by on-scene Basis Life Support (BLS) for Advanced Life Support (ALS) assistance; 2) failure by EMS personnel to record patient physiology at the scene.7 Our primary outcome was in-hospital death. This study was approved by the University of Washington and Washington State Department of Health Institutional Review Boards.

Study Subjects

We used data obtained from the Central Region Trauma Registry (King County) as part of the Washington State regional trauma registry, which was linked to King County Emergency Medical Service (EMS) records for the period January 1, 2002 to December 31, 2004. The final database included detailed EMS and hospital patient data for 5959 patients, who received care for an injury in King County, Washington during this period.

Patient characteristics

Patients older than 13 years of age requiring admission for trauma were identified through review of the trauma registry. These data were linked through patient identifier to EMS patient electronic database. Because of the lack of adequate documentation for and potential bias introduced by patients transferred between facilities, we restricted our analysis to those subjects directly admitted from the field, leaving a total of 4744 patients for evaluation.

Demographic data and comorbidities were abstracted from each database and compared, including age (categorized by APACHE II classification system (≤44, 45–54, 55–64, 65–74, ≥75), gender and comorbidities.8 Comorbidities were identified and u tilized to calculate the Charlson-Deyo score.9,10 The index is based on 17 indicators of pre-existing conditions, which are weighted and then totaled to give a single value. A value of 0 indicates that there are no serious pre-existing conditions.

We collected data regarding the mechanism of injury from each database and divided patients in the following groups: gunshot wound, knife wound, motor vehicle crash (including motorcycle and pedestrian versus vehicle), and other (including strangulation, sports injury, machinery injury, fall, electrical shock, explosion, drowning, child abuse, burn, blunt instrument, bicycle, fight without weapons, and animal caused injury). Severity of injury was defined by the abbreviated injury scale (AIS) score and injury severity score (ISS). We also classified the degree of physiological derangement as detailed below.


In our prior studythe need for ALS to rendezvous with BL S was considered an important audit filter to monitor prehospital care, and one we proposed to evaluate in this study.7 The EMS electronic logs divided patients into three groups by level and sequence of EMS care: 1) ALS initially dispatched and arrived at the scene, ALS; 2) BLS initially dispatched but upgraded to ALS without BLS request for ALS assistance, +ALS; 3) BLS initially dispatched and arrived at the scene, but upgraded to ALS after on-scene BLS request for ALS assistance, BLS+ALS.

EMS data

We abstracted patient physiology data recorded by EMS at the scene and en route, including heart rate (highest, lowest), systolic blood pressure (lowest), respiratory rate (highest, lowest), and scene Glasgow coma scale (GCS). Physiologic variables were categorized using the APACHE II classification: pulse (≤39, 40–54, 55–69, 70–109, 110–139, 140–179, and ≥180 beats per minute), systolic blood pressure (≤49, 50–69, 70–109, 110–129, 130–159, ≥160 mmHg), respiratory rate (≤5, 6–9, 10–11, 12–24, 25–34, 35–49, ≥50 breaths per minute), and GCS (<8, 8–12, >12).8

In our prior study the accurate and complete monitoring/recording of scene physiology by EMS personnel was considered an important audit filter to monitor prehospital care, and one we proposed to evaluate.7 For the purposes of evaluating whether failure by EMS to record physiology was associated with mortality, we additionally defined the completeness of the EMS physiology (heart rate, blood pressure, respiratory rate) record as complete or incomplete. Complete data was defined as a numerical (including zero) EMS entry for each of these 3 physiologic measures; incomplete data required that at least one measure be not recorded.

Hospital data

The hospital data included emergency department (ED) physiology: minimum systolic blood pressure, respiratory rate and heart rate as defined by APACHE II classification.8 We also abstracted data regarding ICU and hospital length of stay.

Statistical analyses

Univariate analyses of continuous and categorical variables were performed using Student’s t test and Pearson’s chi-square test. Significance was determined at p<0.05. We performed multivariate logistic regression to assess the association between BLS vs. ALS care and the risk of mortality utilizing generalized estimates equations to calculate crude and adjusted odds ratios (OR) with 95% confidence intervals (95% CI), accounting for the correlation within hospitals. Similar methodology was used to assess for the association between completeness of EMS records regarding patient physiology and mortality. We hypothesized that the risk of death associated with EMS care (ALS vs. BLS) may vary with severity of injury. Hence, we repeated this analysis incorporating interactions terms between level of EMS care and GCS (≤8, >8), ISS (≥15, >15), or the presence of shock at the scene (SBP<90 mmHg). A similar analysis was performed to assess the association between monitoring of scene physiology and mortality.

After all univariate analyses, each predictor and all significant potential confounders (maximum AIS, ISS, age, gender, mechanism of injury, maximum scene or ED pulse, minimum scene or ED respiratory rate, minimum scene or ED SBP, GCS, and year) were included in the full multivariate model. In addition to adjusting for significant covariates in multivariate analysis, residual confounding and selection effects for significant estimates were addressed using propensity scores.1113 To develop the propensity score for completeness of documentation of scene physiology by EMS personnel, we first performed multivariate logistic regression analysis of all factors that differed in the incomplete and complete EMS documentation groups. The final derivation model included 41 predictor variables and interaction terms. The C statistic for the propensity score derivation model was 77, indicating a strong ability to discriminate between patients with complete and incomplete documentation of scene physiology by EMS personnel. A propensity score for incomplete documentation was then calculated for each patient and ranged from 0.0208973 to 0.995554. These scores were classified into 5 strata each with a balanced distribution of the predictor covariates and the mean propensity score. A second multivariate logistic regression based on a propensity score quintile approach was performed. A summary estimate was calculated using the fixed and random effects methods of DerSimonian and Laird.14 All statistical analyses were conducted using Intercooled Stata version 10, Stata Software (College Station, Texas).


A total of 4744 patients were indentified, who met our criteria and comprised the cohort. The majority was male, Caucasian, with a mean age of 48.7 years (median 46 years) (Table 1). They were moderately injured with an ISS of 11.1, typically by a blunt mechanism; 5.3% (n=203) were in shock at the scene, and the overall mortality was 6.1% (n=290).

Table 1
Characteristics of the Entire Cohort

Level of EMS Response

Of the 4744 patients, 2314 patients had complete data regarding initial EMS level of care and were analyzed (Table 2). Patients for whom BLS requested ALS (BLS+ALS) were less likely to have been injured by a penetrating injury and more likely to be injured in a motor vehicle or motorcycle crash than patients initially cared for by ALS (ALS) or upgraded to ALS (+ALS). There was no significant difference in ISS or maximum AIS; however, BLS+ALS and ALS upgrade patients had a higher prevalence of shock at the scene (Table 2).

Table 2
Characteristics by Level of Emergency Medical Service Response

In univariate analysis, mortality was lower in BLS+ALS than ALS (6.0% vs. 10.7%, p=0.02) (Table 2). After adjusting for differences in case mix, the risk of death for BLS+ALS was similar to that of ALS (OR 1.04; 95% CI 0.51–2.15) (Table 3).

Table 3
Multivariate Analysis for Level of Emergency Medical Service Response

We hypothesized a priori that the relative risk of death may vary with the degree of physiologic perturbation and severity of injury, being higher for BLS+ALS than ALS in a severely injured and physiologically compromised cohort (e.g. GCS≤ 8, scene shock, ISS>25) (Table 3). However, as shown in Table 3, the risk of death with BLS+ALS by comparison to ALS did not significantly vary with the magnitude of injury or degree of physiological compromise (Table 3).

Documentation of Scene Physiology by EMS personnel

All 4744 patients had complete data and were included in the scene physiology data analysis. A total of 1337 (28.2%) patients had at least one missing scene physiologic parameter with the following distribution: scene heart rate, 877 (18.5%); scene blood pressure, 925 (19.5%); and scene respiratory rate, 1,195 (25.2%). A total of 788 patients were missing all three values, 84 were missing two, and 465 were missing one. Of the 465 patients missing just one variable, 25 were missing heart rate, 102 were missing blood pressure, and 338 were missing respiratory rate. Patients with missing EMS physiologic data were older, more likely to be female, and less likely to be injured by a penetrating mechanism or an crash involving a motor vehicle or motorcycle (Table 4) compared to those with complete data. Those with incomplete data also tended to be more severely injured as evidenced by higher maximum AIS and ISS and a lower scene GCS. Among those patients who died, 110 (37.9%) were missing respiratory rate values, 104 (35.9%) were missing systolic blood pressure values, and 87 (30%) were missing heart rate values.

Table 4
Characteristics by Documentation of Scene Physiology by Emergency Medical Services Personnel

In univariate analysis, patients with incomplete EMS physiologic data had a higher mortality (10.3% vs. 4.5%, p<0.001). After adjusting for differences in case mix, patients missing physiologic data were more likely to die than those for whom the EMS log was complete (OR 2.15; 95% CI 1.13–4.10) (Table 5). The stratified propensity multivariate model yielded similar results, demonstrating an increased risk of death in patients with incomplete EMS scene physiology log by contrast to those for whom EMS documentation was complete (OR 2.11; 95% CI, 1.04 to 4.26).

Table 5
Multivariate Analysis for Documentaton of Scene Physiology by Emergency Medical Service Personnel


Performance improvement (PI), the process of continuous evaluation of the trauma system, centers, and providers through structured review of the processes of care and patient outcome, is an essential component for ACSCOT verification.1,4 Though intended to ensure that organized trauma systems improve over time, the center orientation (i.e. patient morbidity and mortality) of current outcome performance measures, including those promulgated by the ACSCOT, is insensitive in identifying system failures. The reliance on morbidity and mortality neglects those system failures not associated with such consequences, and mortality may not always be avoidable.5 Risk adjustment is challenging in analyses of mortality, and several studies note that audit filters other than morbidity and mortality lack scientific validation, causing them to perform inconsistently.6,15 This characteristic appears particularly prominent in prehospital filters.6,15 Alternatively, focusing upon processes of care to monitor and assess the system may obviate many of these shortcomings. Hence, we extended our prior Delphi study that identified by consensus several criteria considered on face validity to be sensitive to inappropriate variation in prehospital care.7 Specifically, we investigated whethe r 1) request by on-scene Basis Life Support (BLS) for Advanced Life Support (ALS) assistance; and 2) failure by EMS personnel to monitor and record patient physiology at the scene represent deviations in care that are associated with mortality. If so, these criteria could provide sufficient sensitivity with which to audit the prehospital phase of patient care and facilitate PI.

Failure of EMS personnel to document basic measures of patient physiology (i.e. heart rate, systolic blood pressure, respiratory rate) at the scene was associated with a greater than twofold increased risk of mortality. This lack of documentation may stem from a greater acuity of illness and intensity of care necessitated by a more severely injured cohort. However, even after adjustment for severity of injury, ED physiology and a stratified propensity adjusted model, the point estimates remained unchanged. Additional factors or extenuating circumstances, including EMS training, EMS proficiency, leadership at the scene and en route, available manpower, and available resources may also explain our observations. We speculate that inadequate EMS documentation may more globally measure inappropriate prehospital care of the injured patient. If so, this simple filter might help to identify those EMS services in need of remediation, and thereby enhance the sensitivity and facilitate the process of PI.

Inadequate documentation has commonly been utilized as a measure of the process of patient care, despite a lack of scientific validation that it is associated with patient outcome.1517 The ACSCOT recommends review of cases in which the ambulance report is absent from the medical record of a patient transported by prehospital EMS.1 However, this filter has not been shown to be associated with patient outcome.18 More importantly, it does not specifically evaluate prehospital care, but rather, a hospital-based mechanism by which to transfer documentation.18 In fact, only one of 22 ACSCOT proposed filters measures the care delivered prior to hospital admission. The existing literature emphasizes the need for additional methods by which to monitor this phase of trauma care if we are to improve upon the trauma system as a whole.5,18,19 Our criterion focuses upon a simple measure, specific to prehospital care: EMS recording patient physiology. We have validated that documentation failure is associated with mortality. Our finding may enable an evidence-based recommendation for incorporating this prehospital quality measure into PI, with the potential to improve patient outcome. However, a corrective intervention focused upon merely ensuring proper documentation is unlikely to yield improved outcomes. Additional prospective study is needed to identify the aspect of the patient or the care rendered that this filter represents, and how it is associated with worse outcome. Only then can the validity of this filter be ensured and the necessary compliment of interventions identified to enable PI and improve patient outcome.

By contrast, the dispatch of an ‘insufficient’ level of EMS care was not associated with worse patient outcome. Specifically, the request for ALS assistance by BLS at the scene was not associated with increased mortality. Even within a more severely injured or physiologically perturbed cohort, BLS request for ALS upgrade was not associated with worse outcome. Several studies have been conducted to determine the optimal level of EMS care for the injured patient.2022 Despite these investigations, equipoise persists as to whether higher levels of EMS services (i.e. ALS) improve, harm, or fail to alter patient outcome.2022 These studies, too, are subject to the limitations of observational design. However, our study did not seek to determine the optimal level of EMS care, but rather, explore whether a circumstance in which EMS care was determined to be insufficient was associated with worse outcome. We speculate that residual confounding in severity of injury may have affected our analysis. Nonetheless, our data suggest that despite face validity, this filter possesses low criterion validity as a measure of patient care associated with worse outcome; hence, it is unlikely to facilitate PI of prehospital trauma care.

We recognize that our study has several limitations. Though our sample size was large, missing data of the level of EMS response limited our analysis and may have biased our results. Our ability to adjust for patient and injury characteristics was subject to the accuracy of chart documentation. Hence, if missing data were more prevalent among critically injured patients cared for by BLS than by ALS, our results would be biased toward the null. Our original Delphi study suggested “the need for ACLS or advanced medical management should be a criterion for dispatching ALS versus BLS to the scene.” We utilized circumstances in which BLS requested ALS as a proxy to cases requiring ACLS or advanced medical management.7 However, even within severely injured cohorts (GCS<9, ISS>25, shock) for whom we hypothesized ACLS and advanced medical management would be critical, BLS request for ALS assistance was not significantly associated with worse outcome. An analysis of other endpoints, such as allocation of resources (i.e. length of stay) might have been more sensitive. These secondary endpoints, however, are influenced by characteristics distinct from medical care (i.e. insurance and socioeconomic status) and for which we could not adjust. We were unable to determine the indication for requesting ALS support. We were unable to determine whether the lack of EMS documentation indicated a lack of monitoring scene physiology or a lack of recording the scene physiology. Though documentation might be postponed during the management of severely injured patients, even after extensive adjustment and a propensity score adjusted analysis, lack of EMS documentation was associated with an increased risk of death.

This study provides evidence that lack of EMS documentation of physiology is associated with worse outcome in the injured patient. Incorporating this simple quality measure specific to prehospital care may facilitate PI of the trauma system, and through appropriate corrective interventions, improve patient care.


We are grateful to Christopher Mack who performed the data acquisition and linkage and the King County EMS for providing the EMS data.

Funding Source: Supported by the NIH Roadmap Multidisciplinary Clinical Research Career Development Award Grant (1 KL2 RR024154-01) from the National Institutes of Health.


American College of Surgeons Committee on Trauma
Abbreviated Injury Scale
Advanced Life Support
Acute Physiology and Chronic Health Evaluation
Basic Life Support
Emergency Medical Services
Glasgow Coma Score
Injury Severity Score
Motorcycle Crash
Motor Vehicle Crash
Systolic Blood Pressure


Disclosure Information: Nothing to disclose.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.


1. American College of Surgeons, Committee on Trauma. Trauma Performance Improvement Reference Manual. Chicago, IL: American College of Surgeons; 2002.
2. West JG, Williams MJ, Trunkey DD, et al. Trauma systems. Current status future challenges. JAMA. 1988;259:3597–3600. [PubMed]
3. Bazzoli GJ, Madura KJ, Cooper GF, et al. Progress in the development of trauma systems in the United States. Results of a national survey. JAMA. 1995;273:395–401. [PubMed]
4. American College of Surgeons, Committee on Trauma. Resources for the Optimal Care of the Injured Patient: 1999. Chicago, IL: American College of Surgeons; 1999.
5. Kissoon N, Tepas JJ, Peterson RJ, et al. The evaluation of pediatric trauma care using audit filters. Pediatr Emerg Care. 1996;12:272–6. [PubMed]
6. Willis CD, Gabbe BJ, Cameron PA. Measuring quality in trauma care. Injury Int J Care Injured. 2007;38:527–537. [PubMed]
7. Rosengart MR, Nathens AB, Schiff MA. The identification of criteria to evaluate prehospital trauma care using the Delphi technique. J Trauma. 2007;62:708–713. [PubMed]
8. Knaus WA, Draper EA, Wagner DP, Zimmerman, et al. APACHE II:a severity of disease classification system. Critical Care Medicine. 1985;13:818–829. [PubMed]
9. Charlson ME, Pompei P, Ales KL, et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373–383. [PubMed]
10. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45:613–619. [PubMed]
11. D’Agostino RB., Jr Propensity score method for bias reduction in the comparison of a treatment group to a non-randomized control group. Statist Med. 1998;17:2265–2281. [PubMed]
12. Brookhart MA, Schneeweiss S, Rothman KJ, et al. Variable selection for propensity score models. Am J Epidemiol. 2006;163:1149–1156. [PMC free article] [PubMed]
13. Joffe MM, Rosenbaum PR. Invited commentary: propensity scores. Am J Epidemiol. 1999;150:327–333. [PubMed]
14. DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials. 1986;7:177–188. [PubMed]
15. Myers JB, Slovis CM, Eckstein M, et al. Evidence-based performance measures for emergency medical services systems: a model for expanded EMS benchmarking. Prehospital Emergency Care. 2008;12:141–151. [PubMed]
16. Dunford J, Domeier RM, Blackwell T, et al. Performance measurements in emergency medical services. Prehospital Emergency Care. 2002;6:92–98. [PubMed]
17. Joyce ST, Dutkowski KL, Hynes T. Efficacy of an ems quality improvement program in improving documentation and performance. Prehospital Emergency Care. 1997;1:140–144. [PubMed]
18. Copes WS, Staz CF, Konvonlinka CW, et al. American College of Surgeons audit filters: Associations with patient outcome and resource. J Trauma. 1995;38:432–438. [PubMed]
19. Bartolomeo SD, Valent F, Sanson G, et al. Are the ASCOT filters associated with outcome? Examining morbidity and mortality in a european setting. Injury Int J Care Injured. 2008;39:1001–1006. [PubMed]
20. Liberman M, Mulder D, Sampalis J. Advanced or basic life support for trauma: Meta-analysis and critical review of the literature. J Trauma. 2000;49:584–599. [PubMed]
21. Jacobs LM, Sinclair A, Beiser A, et al. Prehospital advanced life support: Benefits in trauma. J Trauma. 1984;24:8–13. [PubMed]
22. Eckstein M, Chan L, Schneir A, et al. Effect of prehospital of advanced life support on outcomes of major trauma patients. J Trauma. 2000;48:643–648. [PubMed]
PubReader format: click here to try


Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...


Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...