Skip to main content Skip to main navigation

Publication

A Multimodal Listener Behaviour Driven by Audio Input

Etienne de Sevin; Elisabetta Bevacqua; Sathish Chandra Pammi; Catherine Pelachaud; Marc Schröder; Björn Schuller
In: Proceedings of the AAMAS 2010 Workshop on Interacting with ECAs as Virtual Characters. AAMAS Workshop on Interacting with ECAs as Virtual Characters (AAMAS-10), located at AAMAS '10, May 10-11, Toronto, ON, Canada, 2010.

Abstract

Our aim is to build a platform allowing a user to chat with virtual agent. The agent displays audio-visual backchannels as a response to the user's verbal and nonverbal behaviours. Our system takes as inputs the audio-visual signals of the user and outputs synchronously the audio-visual behaviours of the agent. In this paper, we describe the SEMAINE architecture and the data flow that goes from inputs (audio and video) to outputs (voice synthesizer and virtual characters), going through analysers and interpreters. We focus, more particularly, on the multimodal behaviour of the listener model driven by audio input.

Projekte