MultiMediate: Multi-modal Group Behaviour Analysis for Artificial Mediation

Philipp Müller; Michael Dietz; Dominik Schiller; Dominike Thomas; Guanhua Zhang; Patrick Gebhard; Elisabeth André; Andreas Bulling

In: Proceedings of the 29th ACM International Conference on Multimedia. ACM International Conference on Multimedia (ACM MM-2021), Pages 4878-4882, ACM, 2021.


Artificial mediators are promising to support human group conversations but at present their abilities are limited by insufficient progress in group behaviour analysis. The MultiMediate challenge addresses, for the first time, two fundamental group behaviour analysis tasks in well-defined conditions: eye contact detection and next speaker prediction. For training and evaluation, MultiMediate makes use of the MPIIGroup Interaction dataset consisting of 22 three- to four-person discussions as well as of an unpublished test set of six additional discussions. This paper describes the MultiMediate challenge and presents the challenge dataset including novel fine-grained speaking annotations that were collected for the purpose of MultiMediate. Furthermore, we present baseline approaches and ablation studies for both challenge tasks.


mueller21_mm.pdf (pdf, 1 MB )

Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence