Send to

Choose Destination
J Am Med Inform Assoc. 2016 Sep;23(5):859-65. doi: 10.1093/jamia/ocv211. Epub 2016 Mar 21.

Is the problem list in the eye of the beholder? An exploration of consistency across physicians.

Author information

Division of Hematology/Oncology, Department of Internal Medicine, University of Michigan Medical School, 3-219 Cancer Center, 1500 E Medical Center Dr, Ann Arbor, MI, 48109, USA.
Department of Biostatistics, School of Public Health, II, 1415 Washington Heights, University of Michigan, Ann Arbor, MI, 48109-2029, USA.
San Francisco VA Medical Center, Mental Health Service, 4150 Clement St, San Francisco CA, 94121, USA.
Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, MI, 48109, USA.



Quantify the variability of patients' problem lists - in terms of the number, type, and ordering of problems - across multiple physicians and assess physicians' criteria for organizing and ranking diagnoses.


In an experimental setting, 32 primary care physicians generated and ordered problem lists for three identical complex internal medicine cases expressed as detailed 2- to 4-page abstracts and subsequently expressed their criteria for ordering items in the list. We studied variability in problem list length. We modified a previously validated rank-based similarity measure, with range of zero to one, to quantify agreement between pairs of lists and calculate a single consensus problem list that maximizes agreement with each physician. Physicians' reasoning for the ordering of the problem lists was recorded.


Subjects' problem lists were highly variable. The median problem list length was 8 (range: 3-14) for Case A, 10 (range: 4-20) for Case B, and 7 (range: 3-13) for Case C. The median indices of agreement - taking into account the length, content, and order of lists - over all possible physician pairings was 0.479, 0.371, 0.509, for Cases A, B, and C, respectively. The median agreements between the physicians' lists and the consensus list for each case were 0.683, 0.581, and 0.697 (for Cases A, B, and C, respectively).Out of a possible 1488 pairings, 2 lists were identical. Physicians most frequently ranked problem list items based on their acuity and immediate threat to health.


The problem list is a physician's mental model of a patient's health status. These mental models were found to vary significantly between physicians, raising questions about whether problem lists created by individual physicians can serve their intended purpose to improve care coordination.


coordination of patient care; electronic health record; medical problem list; ranked lists; ranked-biased overlap

[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for Silverchair Information Systems Icon for PubMed Central
Loading ...
Support Center