SmartKom - Adaptive and Flexible Multimodal Access to Multiple Applications

Norbert Reithinger; Jan Alexandersson; Tilman Becker; Anselm Blocher; Ralf Engel; Markus Löckelt; Jochen Müller; Norbert Pfleger; Peter Poller; Michael Streit; Valentin Tschernomas

In: Proceedings of Fifth International Conference on Multimodal Interfaces. International Conference on Multimodal Interfaces (ICMI-2003), Fifth, November 5-7, Vancouver, British Columbia, Canada, ACM, 11/2003.


The development of an intelligent user interface that supports multimodal access to multiple applications is a challenging task. In this paper we present a generic multimodal interface system where the user interacts with an anthropomorphic personalized interface agent using speech and natural gestures. The knowledge-based and uniform approach of SmartKom enables us to realize a comprehensive system that understands imprecise, ambiguous, or incomplete multimodal input and generates coordinated, cohesive, and coherent multimodal presentations for three scenarios, currently addressing more than 50 different functionalities of 14 applications. We demonstrate the main ideas in a walk through the main processing steps from modality fusion to modality fission.


Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence