Language is implicit - it omits information. Filling this information gap requires contextual inference, background- and commonsense knowledge, and reasoning over situational context. Language also evolves, i.e., it specializes and changes over time. For example, many different languages and domains exist, new domains arise, and both evolve constantly. Thus, language understanding also requires continuous and efficient adaptation to new languages and domains, and transfer to, and between, both. Current language understanding technology, however, focuses on high resource languages and domains, uses little to no context, and assumes static data, task, and target distributions.
The research in Cora4NLP aims to address these challenges. It builds on the expertise and results of the predecessor project DEEPLEE and is carried out jointly between the language technology research departments in Berlin and Saarbrücken. Specifically, our goal is to develop natural language understanding methods that enable:
- reasoning over broader co- and contexts;
- efficient adaptation to novel and/or low resource contexts;
- continuous adaptation to, and generalization over, evolving contexts.
To achieve this, we pursue the following research directions:
- memory- and language model-augmented few- and zero-shot learning;
- self- and weakly-supervised pre-training for low-resource domains and long-tail classes;
- multi-lingual, intra- and inter-document, and dialogue context representations;
- integration of structured domain knowledge, background- and commonsense knowledge;
- continual learning for open-domain and supervised tasks multi-hop contextual reasoning.
The resulting methods will be applied in the context of various natural language understanding tasks, such as information extraction, question answering, machine translation, and dialogue.
