Evaluating the Performance of PEFT and LoRA Approaches in Transformer-based Architecture for Handwritten Character Recognition

Abstract

Handwritten character recognition poses significant challenges due to the variability in writing styles, inconsistencies in character shapes, and noise in the data. In this study, it was conducted a comparative analysis of Parameter-Efficient Fine-Tuning (PEFT) and Low-Rank Adaptation (LoRA) approaches against a traditional Convolutional Neural Network (CNN) for handwritten character recognition. The approaches have been evaluated using the A-Z Handwritten Alphabets dataset based on accuracy, training loss, validation loss, and training time. The PEFT and LoRA approaches were fine-tuned using pre-trained transformers, while the CNN model was trained from scratch. Our results demonstrated that the LoRA approach of the Google/vit-base-patch16-224-in21k achieved the highest accuracy of 100% with the shortest training time of approximately 58 minutes. Conversely, the traditional CNN model, while competitive in accuracy (98.28%), required longer training times. The study highlights the efficiency and adaptability of PEFT and LoRA approaches in resource-constrained environments and contributes to the literature by offering a detailed performance comparison. These insights can guide the selection of appropriate model architectures for various handwritten character recognition applications. © 2024 IEEE.