Deep Multi-Modal Fusion Model for Identification of Eight Different Particles in Urinary Sediment

Abstract

Urine sediment examination (USE) is an essential aspect in detecting urinary system diseases, and it is a prerequisite for diagnostic procedures. Urine images are complex, containing numerous particles, which makes a detailed analysis and interpretation challenging. It is crucial for both patients and medical professionals to conduct urine analysis automatically, quickly and inexpensively, without compromising reliability. In this paper, we present a deep multi-modal fusion system, commonly employed in artificial intelligence, capable of automatically distinguishing particles in urine sediment. To achieve this objective, we first created a new dataset comprising erythrocytes, leukocytes, yeast, epithelium, bacteria, crystals, cylinders, and other particles (such as sperm). The data were gathered from urinalysis requests made between July 2022 and September 2022 at the biochemistry laboratory of Fethi Sekin Medical Center Hospital. A dataset containing 8509 images was compiled using the Optika B293PLi microscope with trinocular brightfield. We propose a 5-step process for detecting particles in the dataset using a multi-modal fusion deep learning model: i) The obtained images were augmented by applying affine transformation. ii) To distinguish images, we opted for ResNet18 and ResNet50 models, which yielded high performance in medical data. iii) Feature vectors from both models were fused to generate more consistent, accurate, and useful particle features. iv) We employed ReliefF, Neighborhood Component Analysis (NCA), and Minimum-Redundancy Maximum-Relevancy (mRMR) feature selection methods, widely used to determine features that maximise particle discrimination success. v) In the final step, Support Vector Machine (SVM) was utilised to distinguish the particles. The results demonstrate that the highest accuracy value achieved is 98.54 % when employing the ReliefF algorithm. Contributions of the study include eliminating standardisation differences in manual microscopy, achieving high accuracy in particle discrimination, offering an artificial intelligence-based system applicable in laboratory environments, and providing the dataset as educational and practical material for biochemistry professionals.