A hybrid CNN-ViT based framework for automatic traffic actions detection in smart cities

Abstract

It is crucial to automatically detect traffic accidents and hazardous situations in a timely and accurate manner. In this way, both individual security will be ensured and significant contributions will be made to economic efficiency and sustainable urban life. Millions of people die in traffic accidents every year. This situation also places an additional burden on health systems and will lead to many undesirable consequences. Early detection of events such as traffic density, accidents, and road closures accelerates emergency response processes, regulates traffic flow, and prevents secondary accidents. Therefore, artificial intelligence-supported automatic systems stand out as a key component of smart cities. This study aims to detect traffic accidents and traffic situations automatically. For this purpose, feature extraction was performed with five Convolutional Neural Network (CNN) and five Vision Transformer (ViT) based models. Then, the features obtained from these models were evaluated in different classifiers. The ViT model and the CNN model, which yielded the most successful results, served as the base for the proposed model. The features obtained from the best ViT model and CNN model were combined to bring together different features of the same image. Then, these features were classified into eight different categories using various classifiers. It was observed that the proposed model produced more successful results than the ten models whose preliminary results were obtained in the study. The accuracy value of the proposed model was 96.88%. This value is promising for future studies and plays a strategic role in terms of sustainability and enhancing the quality of life in smart cities.