Table of ContentsSmartKom: Fusion and Fission of Speech, Gestures, and Facial Expressions SmartKom: Merging Various User Interface Paradigms Symbolic and Subsymbolic Fusion of Multiple Modes Outline of the Talk SmartKom: A Highly Portable Multimodal Dialogue System SmartKom: Intuitive Multimodal Interaction SmartKom`s SDDP Interaction Metaphor Multimodal Input and Output in the SmartKom System Symbiotic Interaction with a Life-like Character Multimodal Input and Output in SmartKom Fusion and Fission of Multiple Modalities SmartKom‘s Data Collection of Multimodal Dialogs Personalized Interaction with WebTVs via SmartKom (DFKI with Sony, Philips, Siemens) Using Facial Expression Recognition for Affective Personalization Recognizing Affect: A Negative Facial Expression of the User The SmartKom Demonstrator System Combination of Speech and Gesture in SmartKom Multimodal Input and Output in SmartKom Getting Driving and Walking Directions via SmartKom Getting Driving and Walking Directions via SmartKom SmartKom: Multimodal Dialogues with a Hybrid Navigation System Salient Characteristics of SmartKom The High-Level Control Flow of SmartKom SmartKom’s Multimodal Dialogue Back-Bone Unification of Scored Hypothesis Graphs for Modality Fusion in SmartKom SmartKom‘s Computational Mechanisms for Modality Fusion and Fission The Overlay Operation Versus the Unification Operation Overlay Operations Using the Discourse Model An Example of the Overlay Operation Smartkom‘s Three-Tiered Discourse Model The High-Level Control Flow of SmartKom Smartakus is a Self-Animated Interface Agent Some Complex Behavioural Patterns of the Interaction Agent Smartakus M3L Representation of the Multimodal Discourse Context M3L Specification of a Presentation Task SmartKom‘s Presentation Planner SmartKom‘s Use of Semantic Web Technology Conclusions First International Conference on Perceptive & Multimodal User Interfaces (PMUI’03) |
Author:Wolfgang Wahlster
E-Mail: wahlster@dfki.de Homepage: http://www.dfki.de/~wahlster |