Project

IMPRESS

Improving Embeddings with Semantic Knowledge

Improving Embeddings with Semantic Knowledge

  • Duration:
  • Application fields
    Other

Virtually all NLP systems nowadays use vector representations of words, a.k.a. word embeddings. Similarly, the processing of language combined with vision or other sensory modalities employs multimodal embeddings. While embeddings do embody some form of semantic relatedness, the exact nature of the latter remains unclear. This loss of precise semantic information can affect downstream tasks.

The goals of IMPRESS are to investigate the integration of semantic and common sense knowledge into linguistic and multimodal embeddings and the impact on selected downstream tasks. IMPRESS will also develop open source software and lexical resources, focusing on video activity recognition as a practical testbed. Furthermore, while there is a growing body of NLP research on languages other than English, most research on multimodal embeddings is still done on English. IMPRESS will consider a multilingual extension of the developed methods to handle French, German and English.

Partners

  1. DFKI 2. INRIA

Publications about the project

Sharmila Upadhyaya, Siyu Tao, Natalia Skachkova, Tatiana Anikina, Cennet Oguz, Ivana Kruijff-Korbayová

In: Sopan Khosla , Ramesh Manuvinakurike , Vincent Ng , Massimo Poesio , Michael Strube , Carolyn Rosé (editor). Proceedings of the CODI-CRAC 2021 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue. Workshop on Computational Models of Reference, Anaphora and Coreference (CRAC-2021) located at EMNLP 2021 November 10-11 Punta Cana Dominican Republic Pages 63-70 Association for Computational Linguistics 11/2021.

To the publication
Tatiana Anikina, Cennet Oguz, Natalia Skachkova, Ivana Kruijff-Korbayová

In: Sopan Khosla , Ramesh Manuvinakurike , Vincent Ng , Massimo Poesio , Michael Strube , Carolyn Rosé (editor). Proceedings of the CODI-CRAC 2021 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue. Workshop on Computational Models of Reference, Anaphora and Coreference (CRAC) CODI-CRAC 2021 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue located at EMNLP November 10-11 Punta Cana Dominican Republic Pages 32-42 Association for Computational Linguistics 11/2021.

To the publication
Christian Theobalt, Cennet Oguz, Noshaba Cheema, Anindita Ghosh, Philipp Slusallek

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). International Conference on Computer Vision (ICCV-2021) October 11-17 Virtual Pages 1396-1406 IEEE 10/2021.

To the publication

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz