Med-VQA: Performance Analysis of Question-Answering Systems on Medical Images

Abstract

This study explores multimodal fusion methods and deep learning architectures for visual question answering (VQA) systems in the healthcare domain. ResNet50, DenseNet, LSTM, and BiLSTM models were evaluated for image feature extraction and text-based question processing, along with their integration into architectures enhanced by attention mechanisms. The ResNet50+DenseNet+LSTM model achieved the highest accuracy of 74.90% on the VQA-RAD dataset. Models such as ResNet50+DenseNet+BiLSTM and ResNet50+DenseNet+BiLSTM+Attention demonstrated strong performance with an accuracy of 72.80%, whereas the ResNet50+DenseNet+LSTM+Attention model showed comparatively lower performance. The results highlight the effectiveness of deep learning-based models, particularly in contextual information extraction and multimodal data fusion. This study underscores the significance of leveraging deep learning architectures in VQA systems to generate accurate and context-aware responses, especially in critical domains like healthcare. © 2025 IEEE.