Publication
DFKI-LT at QAST 2007: Adapting QA Components to Mine Answers in Speech Transcripts
Günter Neumann; Rui Wang
In: A. Nardi; C. Peters (Hrsg.). Working Notes for the Cross Language Evaluation Forum (CLEF-2007), September 19-21, Budapest, Hungary. Cross Language Evaluation Forum (CLEF), Online-Proceedings, 9/2007.
Abstract
The paper describes QAst-v1 a robust question answering system for answering factoid
questions in manual and automatic transcriptions of speech. Our system is an
adaptation of our text–based crosslingual open–domain QA system that we used for
the Clef main tasks. In particular we assume that good answer candidates to factoid
questions are named entities which are type–compatible with the expected answer type
of the question. The main features of QAst-v1 are: use of preemptive off-line annotation
of speech transcripts with sentence boundaries, chunk structures and named
entities (NEs); construction of a fulltext search index using words and all found NEs;
use of robust Wh-analysis component to determine shallow dependency structures,
recognition of NEs, and expected answer type (EAT); use of EAT–driven retrieval of
sentences and answer candidates; use of redundancy as an indicator of good answer
candidates. The main focus of our effort was on the technical realization of a first
QAST research prototype making use of as many of our existing QA components as
possible. The results of evaluating the system’s performance by QAST 2007 were as
follows: for subtask T1 (Question-Answering in manual transcriptions of lectures) we
achieved an overall accuracy (ACC) of 15% and a mean reciprocal rank (MRR) of
0.17; for subtask T2 (Question-Answering in automatic transcriptions of lectures) we
obtained 9% (ACC) and 0.09 (MRR).