Mobile Multimodal Dialogue Systems


Click here to start


Mobile Multimodal Dialogue Systems

Multimodal Interfaces to 3G Mobile Services

Intelligent Interaction with Mobile Internet Services

The DFKI operates the first German Test Site for Innovative 3G/4G Broadband Mobile Technologies

Innovative 3G/4G Mobile Applications are designed and tested at DFKI in Saarbrücken

CityShopper‘s Added-Value Mobile Service

Spatial Alarm by CityShopper: A Location-based Active Shopping List based on UMTS (Cooperation with Telekom)

AllergyChecker: Mobile Bar Code Analyzer and Broadband Personalized Allergy Checker (cooperation with Mineway)

Mobile AllergyChecker: Real-Time Video-based Product Identification and Personalized Allergy Check

The Need for Mobile Multimodal Dialogue Systems

Outline of the Talk

Spoken Dialogues with the Car Navigation System

Augmented Reality: Combining Speech, Gestures and Graphics for Mobile Access to an Edutainment System

From Spoken Dialogue to Multimodal Dialogue

Merging Various User Interface Paradigms

The Fusion of Multimodal Input

The Fusion of Multimodal Input

Burning Research Issues in Multimodal Dialogue Systems

The SmartKom Consortium

SmartKom‘s Major Scientific Goals

SmartKom Provides Full Symmetric Multimodality

SmartKom Covers the Full Spectrum of Multimodal Discourse Phenomena

SmartKom`s SDDP Interaction Metaphor

Multimodal Interaction with a Life-like Character

SmartKom Understands Complex Encircling Gestures

Multimodal Input and Output in SmartKom Fusion and Fission of Multiple Modalities

SmartKom’s Multimodal Input and Output Devices (Kiosk Version)

SmartKom: A Flexible and Adaptive Shell for Multimodal Dialogues

Mobile Multimodal Dialogue with a Personalized Infotainment Assistant Running on a Tablet PC

Using Facial Expression Recognition for Affective Personalization

The High-Level Control Flow of SmartKom

Unification of Scored Hypothesis Graphs for Modality Fusion in SmartKom

M3L Representation of an Intention Lattice Fragment

A “Web of Meaning“ has more Personalization Potential than a “Web of Links“

Mapping Web Content Onto a Variety of Structures and Layouts

Spinning the Semantic Web

SmartKom‘s Computational Mechanisms for Modality Fusion and Fission

Fusing Symbolic and Statistical Information in SmartKom

A Fragment of a Presentation Goal, as specified in M3L

A Dynamically Generated Multimodal Presentation based on a Presentation Goal

An Excerpt from SmartKom’s Three-Tiered Multimodal Discourse Model

Seamless Navigation Services

Reference Resolution is based on a Symbolic Representation of the Smart Graphics Output

Generating Maps, Animations and Information Displays on the Fly

Synchronization of Map Update and Character Behaviour

Getting Driving and Walking Directions via SmartKom

SmartKom: Multimodal Dialogues with a Hybrid Navigation System

Adaptive Perceptual Feedback on the System State

Spoken Navigation Dialogues with SmartKom

Tailoring Mobile Presentations to the Age of the User by Unintrusive User Modeling (Müller/Wasinger 02)

Salient Characteristics of SmartKom

The Economic and Scientific Impact of SmartKom

An Example of Technology Transfer: The Virtual Mouse

SmartKom‘s Impact on Software Tools and Resources for Research on Multimodality

MISS: Multilingual Information Systems and Services

AC Systems: Augmented Cognition Lifelong Recording of Personal Sensor Logs and Multifaceted Journals

Fifth International Conference on Multimodal Interfaces (ICMI-PUI’03)


Author:Wolfgang Wahlster



Download Presentation Source