Accessing a Large Multimodal Corpus Using an Automatic Content Linking Device

Andrei Popescu-Belis, Jean Carletta, Jonathan Kilgour, Peter Poller

In: Michael Kipp , J.-C. Martin , P. Paggio , D. Heylen (Hrsg.). Multimodal Corpora: From Models of Natural Interaction to Systems and Applications. Tools for Browsing, Coding and Exchanging Data in Multimodal Corpora Seiten 189-206 Lecture Notes on Aritificial Intelligence (LNAI) 5509 ISBN 3-642-04792-0 Springer Berlin Heidelberg 2009.


As multimodal data becomes easier to record and store, the question arises as to what practical use can be made of archived corpora, and in particular what tools allowing efficient access to it can be built. We use the AMI Meeting Corpus as a case study to build an automatic content linking device, i.e. a system for real-time data retrieval. The corpus provides not only the data repository, but is used also to simulate ongoing meetings for development and testing of the device. The main features of the corpus are briefly described, followed by an outline of data preparation steps prior to indexing, and of the methods for building queries from ongoing meeting discussions, retrieving elements from the corpus and accessing the results. A series of user studies based on prototypes of the content linking device have confirmed the relevance of the concept, and methods for task-based evaluation are under development.

Weitere Links

Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence