Skip to main content Skip to main navigation


VRySmart: A Framework for Embedding Smart Devices in Virtual Reality

Akhmajon Makhsadov; Donald Degraen; André Zenner; Felix Kosmalla; Kamila Mushkina; Antonio Krüger
In: CHI Conference on Human Factors in Computing Systems Extended Abstracts. ACM International Conference on Human Factors in Computing Systems (CHI-2022), New York, NY, USA, CHI EA '22, ISBN 9781450391566, Association for Computing Machinery, 2022.


As immersive virtual experiences find their way into our living room entertainment, they are becoming part of our daily technological consumption. However, state-of-the-art virtual reality (VR) remains disconnected from other digital devices in our environment, such as smartphones or tablets. As context switches between acting in the virtual environment and resolving external notifications negatively influence immersion, we look towards integrating smart devices into virtual experiences. To this aim, we present the VRySmart framework. Through either optical marker tracking or simultaneous localization and mapping (SLAM), embedded smart devices can be used as VR controllers with different levels of integration while their content is incorporated into the virtual context to support the plausibility of the illusion. To investigate user impressions, we conducted a study (N = 10) where participants used a smartphone in four different virtual scenarios. Participants positively assessed smart device usage in VR. We conclude by framing implications for future work.

Weitere Links