SmartKom: Modality Fusion for a Mobile Companion based on Semantic Web Technologies

Table of Contents

SmartKom: Modality Fusion for a Mobile Companion based on Semantic Web Technologies

Intelligent Interaction with Mobile Internet Services

SmartKom: A Highly Portable Multimodal Dialogue System

A Demonstration of SmartKom’s MultimodalInterface for the Federal President of Germany Dr. Rau

SmartKom`s SDDP Interaction Metaphor

SmartKom‘s Use of Semantic Web Technology

SmartKom: Intuitive Multimodal Interaction

Outline of the Talk

Mapping Web Content Onto a Variety of Structures and Layouts

The Markup Language Layer Model of SmartKom

SmartKom: Merging Various User Interface Paradigms

Multimodal Input and Output in SmartKom Fusion and Fission of Multiple Modalities

Symbolic and Subsymbolic Fusion of Multiple Modes

Personalized Interaction with WebTVs via SmartKom (DFKI with Sony, Philips, Siemens)

Using Facial Expression Recognition for Affective Personalization

The SmartKom Demonstrator System

Unification of Scored Hypothesis Graphs for Modality Fusion in SmartKom

SmartKom‘s Computational Mechanisms for Modality Fusion and Fission

The Role of the Semantic Web Language M3L

OIL2XSD: Using XSLT Stylesheets to Convert an OIL Ontology to an XML Schema

Using Ontologies to Extract Information from the Web

M3L as a Meaning Representation Language for the User‘s Input

Exploiting Ontological Knowledge to Understand and Answer the User‘s Queries

SmartKom’s Multimodal Dialogue Back-Bone

SmartKom's Three Tiered Discourse Model

SmartKom’s Domain Model based on M3L

Overlay Operations Using the Discourse Model

The Overlay Operation Versus the Unification Operation

Example for Overlay

Domain Model: A Type Hierarchy of TFS

Unification Simulation

Overlay Simulation

"Formal" Definition Overlay

Domain Models with Multiple Inheritance

Overlay - Scoring

Example: Enrichment and Validation

Example: Enrichment and Validation

Example: Enrichment and Validation

Animation of Scoring Parameters

The High-Level Control Flow of SmartKom

M3L Specification of a Presentation Task

SmartKom‘s Presentation Planner

Salient Characteristics of SmartKom

Conclusions

First International Conference on Perceptive & Multimodal User Interfaces (PMUI’03)

Spinning the Semantic Web

SmartKom Web Link

E-Mail: wahlster@dfki.de

Homepage: http://www.dfki.de/~wahlster

Download Presentation Source