ITable of ContentsSmartKom: Multimodal Dialogs with Mobile Web Users Merging Various User Interface Paradigms Code, Media and Modalities SmartKom: Intuitive Multimodal Interaction SmartKom: A Transportable and Transmutable Interface Agent The Architecture of the SmartKom Agent (cf. Maybury/Wahlster 1998) SmartKom-Mobile: A Handheld Communication Assistant SmartKom-Public: A Multimodal Communication Booth SmartKom-Home/Office: Versatile Agent-based Interface Integration of Speech and Gesture XTRA: Interpretation of pointing gestures (eXpert TRAnslator, Wahlster et al. 1986) Multimodal Input and Output in the SmartKom System Unification-based Media Fusion Unification-based Media Fusion PPT-Folie Augmented Reality: Combining Speech, Gestures and Graphics for Mobile Web Access Augmented Reality: Combining Speech, Gestures and Graphics for Mobile Web Access Augmented Reality: Combining Speech, Gestures and Graphics for Mobile Web Access Multimodal Input and Output in SmartKom Modality-Specific Representation Languages as an Intermediate Representation before Media Fusion The SmartKom Control GUI SmartKom‘s Data Collection of Multimodal Dialogs ANVIL: Multi-Track Annotation of Video and Language Annotation Tool for Multimodal Interaction Mobile Presentation Unit for SmartKom-Public Combination of Speech and Gesture in SmartKom Three Levels of Mark-up Languages for the Web M3L Integrates Three Language Families M3L Representation of the Multimodal Discourse Context M3L Representation of the Word Lattice Produced by the Speech Recognizer for “There [?] I would like to get a reservation.“ Gesture Recognition and Gesture Analysis “There [?] I would like to get a reservation.“ Language Analysis and Media Fusion: Turn8: “There [?] I would like to get a reservation.“ Result of the Action Planner: Presentation Tasks and Presentation Results Input into the Language Generator Language Generation Output Synchronization: Speech, Gesture, Graphics, Animation |
Author:Wolfgang Wahlster
E-Mail: wahlster@dfki.de Homepage: http://www.dfki.de/~wahlster |