Sentiment Analysis in Turkish Using Language Models: A Comparative Study

Abstract

Sentiment analysis is a natural language processing (NLP) task that aims to automatically identify positive, negative and neutral emotions in texts. Agglutinative languages such as Turkish pose challenges for sentiment analysis due to their complex morphological structure. Traditional methods are inadequate for detecting sentiment in texts. Language models (LMs), on the other hand, achieve successful results in sentiment analysis as well as in many other NLP tasks thanks to their ability to learn context and structural features of the language. In this study, XLM-RoBERTa, mBERT, BERTurk 32k, BERTurk 128k, ELECTRA Turkish Small and ELECTRA Turkish Base models were fine-tuned using the Turkish Sentiment Analysis – Version 1 (TRSAv1) dataset and the performances of the models were compared. The dataset consists of 150,000 texts containing user comments on e-commerce platforms. The classes have a balanced distribution for positive, negative and neutral classes. The fine-tuned models are evaluated using the test set with metrics such as accuracy, precision, recall and F1 score. The findings show that models customized for the Turkish language exhibit better performance in emotion detection compared to multilingual models. The BERTurk 32k model achieved strong results with an accuracy of 83.69% and an F1 score of 83.65%, while the BERTurk 128k model followed closely with an accuracy of 83.68% and an F1 score of 83.66%. On the other hand, the XLM-RoBERTa model, a multilingual model, delivered competitive performance with an accuracy of 83.27% and an F1 score of 83.22%.