Misdiagnosis of non-infectious conditions such as cellulitis is a common error and can result in unnecessary hospitalization and antibiotic use. We sought to prospectively determine the misdiagnosis rate of cellulitis among hospitalized patients and to determine if a visually-based computerized diagnostic decision support system (VCDDSS, also named VisualDx) could generate an improved differential diagnosis (DDx) for misdiagnosed patients. In two separate institutions, attending dermatologists or infectious disease specialists evaluated all consecutive patients hospitalized for "cellulitis" by the emergency department. Among 145 subjects enrolled, misdiagnosis occurred in 41 (28%) patients. The diagnosis most commonly mistaken as cellulitis was stasis dermatitis (37%). At one center, in cases that were misdiagnosed by the emergency department, the VCDDSS included the correct diagnosis in the DDx more frequently than the admitting team (18/28 cases (64%) compared to 4/28 cases (14%), p=0.0003). These results demonstrate the capability of this VCDDSS to assist primary care physicians with generating a more accurate DDx when confronted with patients presenting with possible skin infections. Misdiagnoses may result in a significant source of healthcare costs and misdiagnosis-related patient harm. Inclusion of decision support tools early in the diagnostic workflow may reduce misdiagnosis and result in more efficient healthcare management.