Send to

Choose Destination
Appl Clin Inform. 2018 Jul;9(3):714-724. doi: 10.1055/s-0038-1668093. Epub 2018 Sep 12.

A Heuristic Evaluation to Assess Use of After Visit Summaries for Supporting Continuity of Care.

Author information

Health Devices Department, ECRI Institute, Plymouth Meeting, Pennsylvania, United States.
Department of Psychology, Rowan University, Glassboro, New Jersey, United States.
Department of Biomedical and Health Informatics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States.
Division of General Pediatrics, Department of Pediatrics, University of Pennsylvania, Philadelphia, Pennsylvania, United States.
Department of Clinical Effectiveness and Quality Improvement, University of Pennsylvania Health System, Philadelphia, Pennsylvania, United States.
National Center for Human Factors in Healthcare, MedStar Health, Washington, District of Columbia, United States.
ECRI Institute Technology Assessment, Plymouth Meeting, Pennsylvania, United States.



Outpatient providers often do not receive discharge summaries from acute care providers prior to follow-up visits. These outpatient providers may use the after-visit summaries (AVS) that are given to patients to obtain clinical information. It is unclear how effectively AVS support care coordination between clinicians.


Goals for this effort include: (1) developing usability heuristics that may be applied both for assessment and to guide generation of medical documents in general, (2) conducting a heuristic evaluation to assess the use of AVS for communication between clinicians, and (3) providing recommendations for generating AVS that effectively support both patient/caregiver use and care coordination.


We created a 17-item heuristic evaluation instrument for assessing usability of medical documents. Eight experts used the instrument to assess each of four simulated AVS. The simulations were created using examples from two hospitals and two pediatric patient cases developed by the National Institute of Standards and Technology.


Experts identified 224 unique usability problems ranging in severity from mild to catastrophic. Content issues (e.g., missing medical history, marital status of a 2-year-old) were rated as most severe, but widespread formatting and structural problems (e.g., inconsistent indentation, fonts, and headings; confusing ordering of information) were so distracting that they significantly reduced readers' ability to efficiently use the documents. Overall, issues in the AVS from Hospital 2 were more severe than those in the AVS from Hospital 1.


The new instrument allowed for quick, inexpensive evaluations of AVS. Usability issues such as unnecessary information, poor organization, missing information, and inconsistent formatting make it hard for patients, caregivers, and clinicians to use the AVS. The heuristics in the new instrument may be used as guidance to adapt electronic health record systems so that they generate more useful and usable medical documents.

[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for Georg Thieme Verlag Stuttgart, New York Icon for PubMed Central
Loading ...
Support Center