SmartKom: Multimodal Communication with a Life-like Character

click here to start

From Spoken Dialogue to Multimodal Dialogue

Merging Various User Interface Paradigms

Multimodal Interaction with a Life-like Character

SmartKom: Multimodal Dialogs with a Life-like Character

SmartKom: Intuitive Multimodal Interaction

Salient Characteristics of SmartKom

SmartKom: A Transportable and Transmutable Interface Agent

SmartKom Home Office on a Portable Webpad

Smartkom-Mobile

SmartKom`s SDDP Interaction Metaphor

Visual Support for SDDP

The Perspective of the User

Decomposition of Behavioural Schemata: Phases of Gestures

Some Complex Behavioural Patterns of the Interaction Agent Smartakus

Multimodal Input and Output in SmartKom

SmartKom‘s Data Collection of Multimodal Dialogs

Mobile Presentation Unit for SmartKom-Public

The SmartKom Control GUI

Three Levels of Mark-up Languages for the Web

M3L Representation of the Multimodal Discourse Context

M3L Representation of the Word Lattice Produced by the Speech Recognizer for “There [pointing gesture] I would like to get a reservation.“

Gesture Recognition and Gesture Analysis “There [pointing gesture] I would like to get a reservation.“

Language Analysis and Media Fusion: Turn8: “There [pointing gesture] I would like to get a reservation.“

Result of the Action Planner: Presentation Tasks and Presentation Results

Output Synchronization: Speech, Gesture, Graphics, Animation

SmartKom uses a Combination of Concept-to-Speech and Text-to-Speech Technologies

Classification of Facial Expressions

The GUI of the Second SmartKom Prototype

Author:Wolfgang Wahlster

E-Mail: wahlster@dfki.de

Homepage: www.dfki.de/~wahlster