DFKI-LT - Overview of the CLEF 2006 Multilingual Question Answering Track

Bernardo Magnini, Danilo Giampiccolo, Pamela Forner, Christelle Ayache, Valentin Jijkoun, Petya Osenova, Anselmo Peņas, Paulo Rocha, Bogdan Sacaleanu, Richard Sutcliffe
Overview of the CLEF 2006 Multilingual Question Answering Track
in: Carol Peters, Paul Clough, Frederic C. Gey, Jussi Karlgren, Bernardo Magnini, Douglas W. Oard, Maarten de Rijke, Max Stempfhuber (eds.):
1 Evaluation of Multilingual and Multi-modal Information Retrieval (Lecture Notes in Computer Science) volume Volume 4730/2007, Pages 223-256, Springer Berlin / Heidelberg, 2007
 
Having being proposed for the fourth time, the QA at CLEF track has confirmed a still raising interest from the research community, recording a constant increase both in the number of participants and submissions. In 2006, two pilot tasks, WiQA and AVE, were proposed beside the main tasks, representing two promising experiments for the future of QA.Also in the main task some significant innovations were introduced, namely list questions and requiring text snippet(s) to support the exact answers. Although this had an impact on the work load of the organizers both to prepare the question sets and especially to evaluate the submitted runs, it had no significant influence on the performance of the systems, which registered a higher Best accuracy than in the previous campaign, both in monolingual and bilingual tasks. In this paper the preparation of the test set and the evaluation process are described, together with a detailed presentation of the results for each of the languages. The pilot tasks WiQA and AVE will be presented in dedicated articles.
 
Files: BibTeX, OverviewCLEF2006.pdf