Arrhythmia detection in multi-channel ECG images: vision transformer and explainable approaches

Abstract

The electrocardiogram(ECG) signals are usually converted to spectrogram images for analysis, but this approach has significant limitations. Firstly, it loses about time information, and secondly, it lacks resolution. In our research, we introduce an approach to directly convert ECG signals into PNG format and systematically sequentially incorporate each lead data. This method offers a precise depiction by maintaining the intricate temporal and spatial attributes of the signals through the use of Vision Transformer (ViT) models. The study evaluated the arrhythmia detection capabilities of four ViT models (ViT-B/224, ViT-L/224, ViT-B/384, ViT-L/ 384) for classifying ECG images. Six different Class Activation Mapping (CAM) techniques (ScoreCAM, Eigen-CAM, EigenGradCAM, GradCAM++, XGradCAM and LayerCAM) were employed to enhance model interpretability. Our best-performing model, ViT-B/384, achieved 96.79% accuracy and an F1-score of 96.78%, outperforming recent state-of-the-art CNN-based approaches for arrhythmia detection, such as DenseNet (F1: 98.9%, binary tasks) and ResNet-based models (Acc: 95.8%) in multi-class scenarios, while providing improved interpretability. Comparative analysis shows that our method improves multi-class arrhythmia classification accuracy by up to 1-2% over prior ViT or CNN-based methods on similar datasets. These results demonstrate that integrating high-resolution ECG image transformation with ViT models enhances diagnostic precision and model transparency, representing a significant step toward trustworthy AI-based medical diagnostic tools.