Table of ContentsDialog-based Human-Computer Interaction by Coordinated Analysis and Generation of Multiple Modalities From Spoken Dialogue to Multimodal Dialogue Merging Various User Interface Paradigms Multimodal Input and Output in the SmartKom System Multimodal Interaction with a Life-like Character SmartKom: Multimodal Dialogs with a Life-like Character Outline of the Talk SmartKom: Intuitive Multimodal Interaction Salient Characteristics of SmartKom SmartKom: A Transportable Interface Agent SmartKom-Home on a Portable Webpad Smartkom-Mobile SmartKom`s SDDP Interaction Metaphor Some Complex Behavioural Patterns of the Interaction Agent Smartakus Visual Support for SDDP The Perspective of the User Decomposition of Behavioural Schemata: Phases of Gestures Some Complex Behavioural Patterns of the Interaction Agent Smartakus Multimodal Input and Output in SmartKom Modality-Specific Representation Languages as an Intermediate Representation before Media Fusion SmartKom‘s Data Collection of Multimodal Dialogs Mobile Presentation Unit for SmartKom-Public The Architecture of the SmartKom Agent (cf. Maybury/Wahlster 1998) The High-Level Control Flow of SmartKom Multimodal Access to Telephony Applications via SmartKom SmartKom: Towards Multimodal and Mobile Dialogue Systems for Indoor and Outdoor Navigation Getting Driving and Walking Directions via SmartKom Getting Driving and Walking Directions via SmartKom Classification of Facial Expressions (U. Erlangen) Recognizing Neutral Facial Expressions Recognizing Affect: An Angry Facial Expression of the User Multimodal Interaction with Consumer Electronic via SmartKom Combination of Speech and Gesture in SmartKom Multimodal Input and Output in SmartKom Three Levels of Mark-up Languages for the Web M3L Integrates Three Language Families Mapping Typed Feature Structures onto XML-based M3L Terms M3L Representation of the Multimodal Discourse Context M3L Representation of the Word Lattice Produced by the Speech Recognizer for “There [?] I would like to get a reservation.“ Gesture Recognition and Gesture Analysis “There [?] I would like to get a reservation.“ Language Analysis and Media Fusion: Turn8: “There [?] I would like to get a reservation.“ Result of the Action Planner: Presentation Tasks and Presentation Results Output Synchronization: Speech, Gesture, Graphics, Animation Language Generation SmartKom uses a Combination of Concept-to-Speech and Text-to-Speech Technologies Conclusions |
Author:Wolfgang Wahlster
E-Mail: wahlster@dfki.de Homepage: www.dfki.de/~wahlster |