Skip to main content Skip to main navigation

Publikation

Neural Machine Translation Methods for Translating Text to Sign Language Glosses

Dele Zhu; Vera Czehmann; Eleftherios Avramidis
In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. Annual Meeting of the Association for Computational Linguistics (ACL-2023), 61st, July 9-14, Toronto, ON, Canada, Association for Computational Linguistics, 7/2023.

Zusammenfassung

State-of-the-art techniques common to low resource Machine Translation (MT) are applied to improve MT of spoken language text to Sign Language (SL) glosses. In our experiments, we improve the performance of the transformer-based models via (1)~data augmentation, (2)~semi-supervised Neural Machine Translation (NMT), (3)~transfer learning and (4)~multilingual NMT. The proposed methods are implemented progressively on two German SL corpora containing gloss annotations. Multilingual NMT combined with data augmentation appear to be the most successful setting, yielding statistically significant improvements as measured by three automatic metrics (up to over 6 points BLEU), and confirmed via human evaluation. Our best setting outperforms all previous work that report on the same test-set and is also confirmed on a corpus of the American Sign Language (ASL).

Projekte

Weitere Links