Project

CORA4NLP

Co(n)textual Reasoning and Adaptation for Natural Language Processing

Co(n)textual Reasoning and Adaptation for Natural Language Processing

  • Duration:

Language is implicit - it omits information. Filling this information gap requires contextual inference, background- and commonsense knowledge, and reasoning over situational context. Language also evolves, i.e., it specializes and changes over time. For example, many different languages and domains exist, new domains arise, and both evolve constantly. Thus, language understanding also requires continuous and efficient adaptation to new languages and domains, and transfer to, and between, both. Current language understanding technology, however, focuses on high resource languages and domains, uses little to no context, and assumes static data, task, and target distributions.

The research in Cora4NLP aims to address these challenges. It builds on the expertise and results of the predecessor project DEEPLEE and is carried out jointly between the language technology research departments in Berlin and Saarbrücken. Specifically, our goal is to develop natural language understanding methods that enable:

  • reasoning over broader co- and contexts;
  • efficient adaptation to novel and/or low resource contexts;
  • continuous adaptation to, and generalization over, evolving contexts.

To achieve this, we pursue the following research directions:

  • memory- and language model-augmented few- and zero-shot learning;
  • self- and weakly-supervised pre-training for low-resource domains and long-tail classes;
  • multi-lingual, intra- and inter-document, and dialogue context representations;
  • integration of structured domain knowledge, background- and commonsense knowledge;
  • continual learning for open-domain and supervised tasks multi-hop contextual reasoning.

The resulting methods will be applied in the context of various natural language understanding tasks, such as information extraction, question answering, machine translation, and dialogue.

Publications about the project

Stalin Varanasi, Saadullah Amin, Günter Neumann

In: The 2021 Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing (EMNLP-2021) Findings of EMNLP November 7-11 Punta Cana Dominican Republic The Association for Computational Linguistics 209 N. Eighth Street Stroudsburg, PA 18360 USA 11/2021.

To the publication
Jörg Steffen, Josef van Genabith

In: Heike Adel , Shuming Shi (editor). Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Conference on Empirical Methods in Natural Language Processing (EMNLP-2021) November 7-11 Punta Cana Dominican Republic Pages 28-34 Association for Computational Linguistics 11/2021.

To the publication
Sharmila Upadhyaya, Siyu Tao, Natalia Skachkova, Tatiana Anikina, Cennet Oguz, Ivana Kruijff-Korbayová

In: Sopan Khosla , Ramesh Manuvinakurike , Vincent Ng , Massimo Poesio , Michael Strube , Carolyn Rosé (editor). Proceedings of the CODI-CRAC 2021 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue. Workshop on Computational Models of Reference, Anaphora and Coreference (CRAC-2021) located at EMNLP 2021 November 10-11 Punta Cana Dominican Republic Pages 63-70 Association for Computational Linguistics 11/2021.

To the publication

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz