DFKI-LT at QAST 2007: Adapting QA Components to Mine Answers in Speech Transcripts

Günter Neumann, Rui Wang

In: A. Nardi , C. Peters (Hrsg.). Working Notes for the Cross Language Evaluation Forum (CLEF-2007), September 19-21, Budapest, Hungary. Cross Language Evaluation Forum (CLEF) Online-Proceedings 9/2007.


The paper describes QAst-v1 a robust question answering system for answering factoid questions in manual and automatic transcriptions of speech. Our system is an adaptation of our text–based crosslingual open–domain QA system that we used for the Clef main tasks. In particular we assume that good answer candidates to factoid questions are named entities which are type–compatible with the expected answer type of the question. The main features of QAst-v1 are: use of preemptive off-line annotation of speech transcripts with sentence boundaries, chunk structures and named entities (NEs); construction of a fulltext search index using words and all found NEs; use of robust Wh-analysis component to determine shallow dependency structures, recognition of NEs, and expected answer type (EAT); use of EAT–driven retrieval of sentences and answer candidates; use of redundancy as an indicator of good answer candidates. The main focus of our effort was on the technical realization of a first QAST research prototype making use of as many of our existing QA components as possible. The results of evaluating the system’s performance by QAST 2007 were as follows: for subtask T1 (Question-Answering in manual transcriptions of lectures) we achieved an overall accuracy (ACC) of 15% and a mean reciprocal rank (MRR) of 0.17; for subtask T2 (Question-Answering in automatic transcriptions of lectures) we obtained 9% (ACC) and 0.09 (MRR).

Weitere Links

Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence