Determining the Type of Document Read Using Eye Movement Properties by Hybrid CNN Method

Abstract

By examining the change of eye movements during reading, it is possible to determine the type of document read. Returning to the previous position (negative saccade), blink, fixation, and position are important indicators in determining the type of document being read. In this paper, a hybrid deep learning model is proposed to determine the type of document read. The MPIIDPEye dataset, which includes eye movement data of 10-minute comic, newspaper and text document readings from 20 participants, was used. First, the eye movements obtained over time were augmented by the non-linear interpolation technique. In order to process the data of each class with convolutional neural network, spectrogram images of the signals were created. Spectrogram images were given as input to Resnet architectures and the features in the Fc1000 layer were combined. Concatenated feature vectors were given as an input to feature selection algorithms. The most effective features in classification accuracy were determined and classified using the SVM algorithm. The classification was carried out for 3 different cases, and the highest accuracy of 98.41% was obtained for case-2, where Fixation, Position, and Blink properties were used.