How to support natural language dialogue and multimedia presentations on mobile devices by new graphical interfaces is our main objective. Consider the following optimized human-computer dialogue:

 

 

 

 

In a mobile question answering scenario, different interaction situations and different interaction circumstances can be encountered, compared to desktop computer interactions with similar functionality. Fortunately, not only challenges but also opportunities and great potentials are apparent. Question answering in mobile domains means, for first, lack of computational power, and for second, a small screen where only short messages and single multimedia items can be presented. But since mobile devices are real physical, hand-held, and portable mobile tools, interesting interaction metaphors including recorded speech, pointing gestures on screen, and the use of device buttons can be investigated.  

 

 

Perceptual feedback allows the user to understand the current processing state of the dialogue system. On mobile devices, system (re)activity can be expressed by lively metaphors for processing states: (1) Listening/Idle State, (2) Recording, (3) Query Processing (status bar), (4) exploring the results.

 

In this gallery, we present the interaction design and implementation for haptic-based perceptual feedback.