Mixing realities for sketch retrieval in Virtual Reality

Daniele Giunchi, Stuart James, Donald Degraen, Anthony J Steed

In: The 17th International Conference on Virtual-Reality Continuum and Its Applications in Industry. International Conference on Virtual-Reality Continuum and Its Applications in Industry (VRCAI-2019) Association for Computing Machinery Brisbane, QLD, Australia 2019.


Users within a Virtual Environment often need support designing the environment around them with the need to find relevant content while remaining immersed. We focus on the familiar sketch-based interaction to support the process of content placing and specifically investigate how interactions from a tablet or desktop translate into the virtual environment. To understand sketching interaction within a virtual environment, we compare different methods of sketch interaction, i.e., 3D mid-air sketching, 2D sketching on a virtual tablet, 2D sketching on a fixed virtual whiteboard, and 2D sketching on a real tablet. The user remains immersed within the environment and queries a database containing detailed 3D models and replace them into the virtual environment. Our results show that 3D mid-air sketching is considered to be a more intuitive method to search a collection of models; while the addition of physical devices creates confusion due to the complications of their inclusion within a virtual environment. While we pose our work as a retrieval problem for 3D models of chairs, our results are extendable to other sketching tasks for virtual environments.

3359997.3365751.pdf (pdf, 1 MB )

Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence