Skip to main content Skip to main navigation


AI-Based Open-Source Gesture Retargeting to a Humanoid Teaching Robot

Hae Seon Yun; Heiko Hübert; Volha Taliaronak; Ralf Mayet; Murat Kirtay; Verena V Hafner; Niels Pinkwart
In: Maria Mercedes Rodrigo; Noburu Matsuda; Alexandra I. Cristea; Vania Dimitrova. Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners’ and Doctoral Consortium. Pages 276-279, ISBN 978-3-031-11647-6, Springer International Publishing, 7/2022.


Gestures and speech modalities play potent roles in social learning, especially in educational settings. Enabling artificial learning companions (i.e., humanoid robots) to perform human-like gestures and speech will facilitate interactive social learning in classrooms. In this paper, we present the implementation of human-generated gestures and speech on the Pepper robot to build a robotic teacher. To this end, we transferred a human teacher gesture to a humanoid robot using a web and a kinect cameras and applied a video-based markerless motion capture technology and an observation-based motion mirroring method. To evaluate the retargeting methods, we presented different types of a humanoid robotic teacher to six teachers and collect their impressions on the practical usage of a robotic teacher in the classroom. Our results show that the presented AI-based open-source gesture retargeting technology was found attractive, as it gives the teachers an agency to design and employ the Pepper robot in their classes. Future work entails the evaluation of our solution to the stakeholders (i.e. teachers) for its usability.