Improved Cognitive Load Classification with Pupil-based Fixation and Saccade Detection Using Wearable Infrared Cameras

Abstract

Eye movement detection, separating the eye positions into distinct oculomotor events such as saccade and fixation, has been associated with cognitive load classification, referring to the process of estimating the mental effort involved with a certain task. However, there exist three questions remaining to be answered for wearable applications: (i) will algorithms originally developed for fixation and saccade detection from gaze positions give similar accuracy from pupil center positions, particularly when the head is not fixed?; (ii) how much improvement to the performance of cognitive load classification can be achieved by separating fixation and saccade?; and (iii) will the fixation- and saccade-related measure be affected by differing cognitive load processes from diverse task designs? Regarding the first research question, three representative saccade detection algorithms are applied to both pupil center positions and gaze positions collected with and without head movement, and their performance is evaluated against a stimulus-based ground truth under different measures. Results from a novel dataset recorded using wearable infrared cameras indicate that saccade/fixation detection using pupil center positions generally pro- vides better performance than using gaze positions with an 8.6% improvement in Cohen’s Kappa. Regarding the second and third research questions, statistical tests of several pupil-related measures extracted from all samples, fixation-only samples and saccade-only samples are evaluated for varied cognitive load levels, which indicate that pupil-related measures from fixation-only samples can be used as a substitute for those from all samples in distinguish- ing different levels of cognitive loads. From the statistical test results of several fixation- and saccade-related measures across two task types, the possibility for such measures to distinguish varied cognitive load levels, together with their trends among varied cognitive load levels are different under varied cognitive load processes. Furthermore, for the cognitive load classification systems trained with and without fixation- and saccade-related features, accuracy can be improved by 14.0%-23.4% for a random forest classifier across two different task types by including fixation and saccade-related features. In general, this thesis contributes to fixation and saccade based cognitive load classification research by demonstrating that pupil center positions can be used as an alternative to gaze positions for fixation and saccade detection in a wearable context, and moreover, fixation and saccade separation can improve the cognitive load classification performance.