Enabling the Discovery of Manual Processes using a Multi-modal Activity Recognition Approach

Adrian Rebmann, Andreas Emrich, Peter Fettke

In: Chiara Di Francescomarino , Dijkman Remco , Zdun Uwe (Hrsg.). Business Process Management Workshops. International Workshop on Business Processes Meet the Internet-of-Things (BP-Meet-IoT-2019) befindet sich 17th Int. Conference on Business Process Management September 2-2 Vienna Austria Seiten 130-141 LNBIP 362 Springer International Publishing Cham 1/2020.


The analysis of business processes using process mining re- quires structured log data. Regarding manual activities, this data can be generated from sensor data acquired from the Internet of Things. The main objective of this paper is the development and evaluation of an approach which recognizes and logs manually performed activities, enabling the application of established process discovery methods. A system was implemented which uses a body area network, image data of the process environment and feedback from the executing workers in case of uncer- tainties during detection. Both feedback and image data are acquired and processed during process execution. In a case study in a laboratory environment, the system was evaluated using an example process. The implemented approach shows that the inclusion of image data of the environment and user feedback in ambiguous situations during recognition generate log data which well represent actual process behavior.

Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence