DFKI-LT - Redundancy Localization for the Conversationalization of Unstructured Responses

Sebastian Krause, Mikhail Kozhevnikov, Eric Malmi, Daniele Pighin
Redundancy Localization for the Conversationalization of Unstructured Responses
1 Proceedings of the 18th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Saarbrücken, Germany, Association for Computational Linguistics, 2017
Conversational agents offer users a natural- language interface to accomplish tasks, entertain themselves, or access information. Informational dialogue is particularly challenging in that the agent has to hold a conversation on an open topic, and to achieve a reasonable coverage it generally needs to digest and present unstructured information from textual sources. Making responses based on such sources sound natural and fit appropriately into the conversation con- text is a topic of ongoing research, one of the key issues of which is preventing the agent’s responses from sounding repetitive. Targeting this issue, we propose a new task, known as redundancy localization, which aims to pinpoint semantic overlap between text passages. To help address it systematically, we formalize the task, prepare a public dataset with fine-grained redundancy labels, and propose a model utilizing a weak training signal defined over the results of a passage-retrieval system on web texts. The proposed model demonstrates superior performance compared to a state-of-the-art entailment model and yields encouraging results when applied to a real-world dialogue.
Files: BibTeX, sigdial2017-redundancy-localization.pdf