A hybrid deep neural network-based automated diagnosis system using x-ray images and clinical findings

Abstract

Image-based computer-aided diagnosis systems are frequently utilized to detect vital disorders. These systems consist of methods based on machine learning and work on data obtained from imaging technologies such as x-rays, magnetic resonance imaging, and computed tomography. In addition to image data, clinical findings usually consist of text data that have a critical role in diagnosing diseases. In this study, an effective classification approach that can automatically detect diseases using a deep learning algorithm is proposed. A clinically usable deep machine-learning model is presented with an approach based on the hybrid use of image processing and text processing methods. Although image-processing techniques are used frequently in the literature by using images, text processing techniques are not often used in this field. The proposed method was evaluated on a dataset consisting of x-ray images of subjects with and without Covid-19 disease and their clinical notes. When the dataset was classified using image processing and text processing techniques accuracy values were obtained as 90.9% and 92.0% respectively. With the hybrid approach proposed in this study, the classification performance was increased and an accuracy value of 93.5% was obtained.