Multi-modal Multi-scale Attention Guidance in Cyber-Physical Environments

Guillermo Reyes, Alexandra Katrin Alles

In: 26th International Conference on Intelligent User Interfaces. International Conference on Intelligent User Interfaces (IUI-2021) April 13-17 College Station/Virtual TX United States Pages 356-365 ISBN 9781450380171 Association for Computing Machinery (ACM) 4/2021.


This work proposes a new method for guiding a user’s attention towards objects of interest in a cyber-physical environment (CPE). CPEs are environments that contain several computing systems that interact with each other and with the physical world. These environments contain several sensors (cameras, eye trackers, etc.) and output devices (lamps, screens, speakers, etc.). These devices can be used to first track the user’s position, orientation, and focus of attention to then find the most suitable output device to guide the user’s attention towards a target object. We argue that the most suitable device in this context is the one that attracts attention closest to the target and is salient enough to capture the user’s attention. The method is implemented as a function which estimates the ”closeness” and ”salience” of each visual and auditive output device in the environment. Some parameters of this method are then evaluated through a user study in the context of a virtual reality supermarket. The results show that multi-modal guidance can lead to better guiding performance. However, this depends on the set parameters.


Weitere Links

reyes_and_alles_iui21_attention_guidance.pdf (pdf, 2 MB )

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz