Affordable Access

deepdyve-link
Publisher Website

Evaluation of deep learning for COVID‐19 diagnosis: Impact of image dataset organization

Authors
  • Kim, Ga Young1
  • Kim, Jae Yong2
  • Kim, Chae Hyeon3
  • Kim, Sung Min1
  • 1 Dongguk University, Korea , (South Korea)
  • 2 JLK, Korea , (South Korea)
  • 3 Dongguk University‐Seoul, Korea , (South Korea)
Type
Published Article
Journal
Journal of Applied Clinical Medical Physics
Publisher
John Wiley and Sons Inc.
Publication Date
Jun 23, 2021
Volume
22
Issue
7
Pages
297–305
Identifiers
DOI: 10.1002/acm2.13320
PMID: 34159697
PMCID: PMC8292699
Source
PubMed Central
Keywords
Disciplines
  • Medical Imaging
License
Unknown
External links

Abstract

Introduction Coronavirus disease 2019 (COVID‐19) has spread all over the world showing high transmissibility. Many studies have proposed diverse diagnostic methods based on deep learning using chest X‐ray images focusing on performance improvement. In reviewing them, this study noticed that evaluation results might be influenced by dataset organization. Therefore, this study identified whether the high‐performance values can prove the clinical application potential. Methods This study selected chest X‐ray image databases which have been widely applied in previous studies. One database includes images for COVID‐19, while the others consist of normal and pneumonia images. Then, the COVID‐19 classification model was designed and trained on diverse database compositions and evaluated using confusion matrix‐based metrics. Also, each database was analyzed by graphical representation methods. Results The performance was significantly different according to dataset composition. Overall, higher performance was identified on the dataset organized with different databases for each class, compared with the dataset from same database. Also, there were significant differences in the image characteristics between different databases. Conclusions The experimental results indicate that model may be trained based on differences of the image characteristics between databases and not on lesion features. This shows that evaluation metrics can be influenced by dataset organization, and high metric values would not directly mean the potential for clinical application. These emphasize the importance of suitable dataset organization for applying COVID‐19 diagnosis methods to real clinical sites. Radiologists should sufficiently understand about this issue as actual user of these methods.

Report this publication

Statistics

Seen <100 times