Multimodal Task Assignment and Introspection in Distributed Agricultural Harvesting Processes

Daniel Porta, Zeynep Tuncer, Michael Wirth, Michael Hellenschmidt

In: Proceedings of the Eighth International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies. International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies (UBICOMM-14) Pages 227-232 ISBN 978-1-61208-353-7 IARIA 8/2014.


Harvesting processes are in fact industrial manufacturing processes that follow a tight schedule. Unexpected incidents can disturb a harvest and require a replanning of the process in order to avoid severe financial losses. When a new plan has been found, it must be communicated to the affected process participants, i.e., drivers of agricultural machinery. This paper presents a cloud-based system for orchestrating and coordinating a fleet of agricultural machinery and their drivers during an ongoing harvest in case of an unexpected incident. A management dashboard allows the real-time replanning of a harvesting process and sends updated instructions to each affected driver's mobile device. The paper focuses on the communication between the contractor and a driver in the field as well as the interaction of the driver with his mobile device. It is explained how the system accomplishes a fast, traceable, and safe communication with the drivers that may suffer from bad network conditions and a high cognitive load. In order to understand the details of his new tasks, a driver can examine them in a multimodal dialogue including speech with the system. This is beneficial in a driving situation. By interacting with the mobile client, the system also deduces if the driver correctly understood his new instructions and can intervene if not.


Weitere Links

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz