Few-shot keypose detection for learning of psychomotor skills

Benjamin Paaßen; Tobias Baumgartner; Mai Geisen; Nina Riedl; Milos Kravcik

In: Khaleel Asyraaf Mat Sanusi; Bibeg Limbu; Jan Schneider; Daniele Di Mitri; Roland Klemke (Hrsg.). Proceedings of the Second International Workshop on Multimodal Immersive Learning Systems (MILeS 2022). International Workshop on Multimodal Immersive Learning Systems (MILeS-2022), located at 17th European Conference on Technology Enhanced Learning (EC-TEL 2022), September 12-16, Toulouse, France, Pages 22-27, Vol. 3247, CEUR Workshop Proceedings, Aachen, 10/2022.


Some psychomotor tasks require students to perform a specific sequence of poses and motions. A natural teaching scheme for such tasks is to contrast a student’s execution to a teacher demonstration. However, this requires strategies to match the teacher demonstration of each motion to the student’s attempts and to identify differences between demonstration and attempt. In this paper, we investigate methods to automatically detect student attempts for poses with only a single correct teacher demonstration. We investigate relevance learning, prototype networks, and attention mechanisms to achieve a robust few-shot approach which generalizes across students. In an experiment with one teacher and 27 students performing a sequence of motion elements from the field of fitness and dance, we show that prototype networks combined with an attention mechanism performs best.


Weitere Links

MILeS_2022_paper_4794.pdf (pdf, 661 KB )

Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence