Towards Medical Cyber-Physical Systems: Multimodal Augmented Reality for Doctors and Knowledge Discovery about Patients

Daniel Sonntag; Sonja Zillner; Christian Husodo Schulz; Takumi Toyama; Markus Weber

In: Aaron Marcus (Hrsg.). Design, User Experience, and Usability. User Experience in Novel Technological Environments. Pages 401-410, Springer, Berlin - Heidelberg, 2013.


In the medical domain, which becomes more and more digital, every improvement in efficiency and effectiveness really counts. Doctors must be able to retrieve data easily and provide their input in the most convenient way. With new technologies towards medical cyber-physical systems, such as networked head-mounted displays (HMDs) and eye trackers, new interaction opportunities arise. With our medical demo in the context of a cancer screening programme, we are combining active speech based input, passive/active eye tracker user input, and HMD output (all devices are on-body and hands-free) in a convenient way for both the patient and the doctor.

2013_Vision-Based_Location-Awareness_in_Augmented_Reality_Applications.pdf (pdf, 420 KB )

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz