Realizing Multimodal Behavior

Michael Kipp; Alexis Heloir; Marc Schröder; Patrick Gebhard

In: Proceedings of the 10th International Conference on Intelligent Virtual Agents. International Conference on Intelligent Virtual Agents (IVA-2010), September 20-22, Philadelphia, Pennsylvania, USA, Pages 57-63, Springer, 2010.


Generating coordinated multimodal behavior for an embod- ied agent (speech, gesture, facial expression. . . ) is challenging. It requires a high degree of animation control, in particular when reactive behaviors are required. We suggest to distinguish realization planning, where ges- ture and speech are processed symbolically using the behavior markup language (BML), and presentation which is controlled by a lower level animation language (EMBRScript). Reactive behaviors can bypass plan- ning and directly control presentation. In this paper, we show how to define a behavior lexicon, how this lexicon relates to BML and how to resolve timing using formal constraint solvers. We conclude by demonstrating how to integrate reactive emotional behaviors.


kipp_etal2010.pdf (pdf, 1 MB )

Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence