Skip to main content Skip to main navigation


Experimental Machine Translation of the Swiss German Sign Language via 3D Augmentation of Body Keypoints

Lorenz Hufe; Eleftherios Avramidis
In: Proceedings of the Seventh Conference on Machine Translation. Conference on Machine Translation (WMT-22), December 7-8, Abu Dhabi, United Arab Emirates, Pages 983-988, Association for Computational Linguistics, 12/2022.


This paper describes the participation of DFKI-SLT at the Sign Language Translation Task of the Seventh Conference of Machine Translation (WMT22). The system focuses on the translation direction from the Swiss German Sign Language (DSGS) to written German. The original videos of the sign language were analyzed with computer vision models to provide 3D body keypoints. A deep-learning sequence-to-sequence model is trained on a parallel corpus of these body keypoints aligned to written German sentences. Geometric data augmentation occurs during the training process. The body keypoints are augmented by artificial rotation in the three dimensional space. The 3D-transformation is calculated with different angles on every batch of the training process.


Weitere Links