Skip to main content Skip to main navigation

Publication

Implicit Search Intent Recognition using EEG and Eye Tracking: Novel Dataset and Cross-User Prediction

Mansi Sharma; Shuang Chen; Philipp Müller; Maurice Rekrut; Antonio Krüger
In: Proceedings of the 25th International Conference on Multimodal Interaction. ACM International Conference on Multimodal Interaction (ICMI), Pages 345-354, IEEE, 2023.

Abstract

For machines to efectively assist humans in challenging visual search tasks, they must diferentiate whether a human is simply glancing into a scene (navigational intent) or searching for a target object (informational intent). Previousresearch proposed combining electroencephalography (EEG) and eye-tracking measurements to recognize such search intents implicitly, i.e., without explicit user input. However, the applicability of these approaches to real-world scenarios sufers from two key limitations. First, previous work used fxed search times in the informational intent condition - a stark contrast to visualsearch, which naturally terminates when the target is found. Second, methods incorporating EEG measurements addressed prediction scenarios that require ground truth training data from the target user, which isimpractical in many use cases. We address these limitations by making the frst publicly available EEG and eye-tracking dataset for navigational vs. informational intent recognition, where the user determines search times. We present the frst method for cross-user prediction of search intents from EEG and eye-tracking recordings and reach 84.5% accuracy in leaveone-user-out evaluations - comparable to within-user prediction accuracy (85.5%) but ofering much greater fexibility

Projects

More links