Iterated and Linear SVMs in Text Classification: An Ensemble Feature Engineering Model

Abstract

Term frequency-inverse document frequency (TF-IDF), a widely used method for feature representation in text classification, has limitations such as inconsistent results and high dimensionality. The main objective of this research is to develop an accurate feature engineering model for text classification. We propose a novel TF-IDF-based feature engineering architecture for legal text classification, which consists of 4 phases: (1) feature extraction using TF-IDF; (2) feature selection with iterative neighborhood component analysis, iterative Chi2, iterative ReliefF, and iterative minimum redundancy maximum relevance, generating four distinct feature vectors; (3) classification of the selected feature vectors using an iterative linear support vector machine, producing 40 prediction vectors (10 for each feature vector); and (4) final result selection using a greedy algorithm and these phases makes the architecture self-organizing.The dataset used in this study included 235 court decision documents (108 accepted and 127 rejected). These documents were collected from the LegalBank dataset of the European Court of Human Rights and were translated into Turkish by lawyers to enable the application of this model to Turkish courts. Our proposed model achieved a classification accuracy of 99.15% using tenfold cross-validation. The model operates with linear time complexity, making it efficient for law-related text classification. The ultimate goal of this research is to build a digital assistant for Turkish courts.