A hybrid approach to facial expression classification using appearance-based filters

Abstract

Facial expressions are universally understood and often linked to specific emotional states. These expressions result from the movement of facial muscles and are typically characterized by changes in the eyes, eyebrows, mouth, and cheeks. Basic facial expressions include surprise, sadness, neutrality, happiness, disgust, anger, and fear. Various approaches have been developed for facial expression classification. In this study, a new dataset was created by merging two commonly used datasets, CK + and KDEF, resulting in a total of 2,101 samples. Initially, appearance-based filters such as Gabor, SIFT, HOG, and LBP were applied for image preprocessing. Feature extraction was then conducted using the EfficientNetB7 convolutional neural network. The extracted features were individually classified for each filter using Support Vector Machines (SVM) and Multi-Layer Perceptron (MLP) models. Additionally, a hybrid model was proposed, evaluating features extracted from each appearance-based filter independently, which were then classified using deep learning methods. Among all methods evaluated, the highest accuracy of 91.92% was achieved using the Gabor filter with the proposed hybrid model. This study suggests that different preprocessing strategies can influence classification performance within deep learning-based pipelines for improved facial expression recognition.