Send to

Choose Destination
Invest Radiol. 2019 Jan;54(1):7-15. doi: 10.1097/RLI.0000000000000503.

Deep Learning in Diagnosis of Maxillary Sinusitis Using Conventional Radiography.

Author information

Department of Radiology, Seoul National University Bundang Hospital, Seongnam.
Department of Radiology, Hallym University Sacred Heart Hospital, Anyang.
Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea.



The aim of this study was to compare the diagnostic performance of a deep learning algorithm with that of radiologists in diagnosing maxillary sinusitis on Waters' view radiographs.


Among 80,475 Waters' view radiographs, examined between May 2003 and February 2017, 9000 randomly selected cases were classified as normal or maxillary sinusitis based on radiographic findings and divided into training (n = 8000) and validation (n = 1000) sets to develop a deep learning algorithm. Two test sets composed of Waters' view radiographs with concurrent paranasal sinus computed tomography were labeled based on computed tomography findings: one with temporal separation (n = 140) and the other with geographic separation (n = 200) from the training set. Area under the receiver operating characteristics curve (AUC), sensitivity, and specificity of the algorithm and 5 radiologists were assessed. Interobserver agreement between the algorithm and majority decision of the radiologists was measured. The correlation coefficient between the predicted probability of the algorithm and average confidence level of the radiologists was determined.


The AUCs of the deep learning algorithm were 0.93 and 0.88 for the temporal and geographic external test sets, respectively. The AUCs of the radiologists were 0.83 to 0.89 for the temporal and 0.75 to 0.84 for the geographic external test sets. The deep learning algorithm showed statistically significantly higher AUC than radiologist in both test sets. In terms of sensitivity and specificity, the deep learning algorithm was comparable to the radiologists. A strong interobserver agreement was noted between the algorithm and radiologists (Cohen κ coefficient, 0.82). The correlation coefficient between the predicted probability of the algorithm and confidence level of radiologists was 0.89 and 0.84 for the 2 test sets, respectively.


The deep learning algorithm could diagnose maxillary sinusitis on Waters' view radiograph with superior AUC and comparable sensitivity and specificity to those of radiologists.

Supplemental Content

Full text links

Icon for Wolters Kluwer
Loading ...
Support Center