LSTM-Based Early Recognition of Motion Patterns

Markus Weber; Christopher Schölzel; Marcus Liwicki; Seiichi Uchida; Didier Stricker

In: Proceedings of the 22nd International Conference on Pattern Recognition. International Conference on Pattern Recognition (ICPR-22), 22nd, August 24-28, Stockholm, Sweden, 2014.


In this paper a method for early recognition of motion templates is presented. We define early recognition as an algorithm to provide recognition results before a motion sequence is completed. In our experiments we apply Long Short- Term Memory (LSTM) and optimize the training for the task of recognizing the motion template as early as possible. The evaluation has shown that the recognition accuracy for a frameby- frame classification the LSTM achieves a recognition accuracy of 88% if no training data of the person him/herself is included, and 92% if the training data also contains motion sequences of the person. Furthermore, the average earliness - the number of time steps it takes before the LSTM correctly classifies a motion pattern - is around 24.77 time steps, which is less than a second with the used tracking technology, i.e., the Microsoft Kinect.

Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence