Help Me to Help You: How to Learn Intentions, Actions and Plans

Harmish Khambhaita, Geert-Jan Kruijff, Matei Mancas, Mario Gianni, Panagiotis Papadakis, Fiora Pirri, Matia Pizzoli

In: 2011 AAAI Spring Symposium Series. AAAI Spring Symposium (AAAI SSS-11) Help Me Help You: Bridging the Gaps in Human-Agent Collaboration March 21-23 Palo Alto CA United States Seiten 30-35 AAAI 2011.


The collaboration between a human and a robot is here understood as a learning process mediated by the instructor prompt behaviours and the apprentice collecting information from them to learn a plan. The instructor wears the Gaze Machine, a wearable device gathering and conveying visual and audio input from the instructor while executing a task. The robot, on the other hand, is eager to learn both the best sequence of actions, their timing and how they interlace. The cross relation among actions is specified both in terms of time intervals for their execution, and in terms of location in space to cope with the instruction interaction with people and objects in the scene. We outline this process: how to transform the rich information delivered by the Gaze Machine into a plan. Specifically, how to obtain a map of the instructor positions and his gaze position, via visual slam and gaze fixations; further, how to obtain an action map from the running commentaries and the topological maps and, finally, how to obtain a temporal net of the relevant actions that have been extracted. The learned structure is then managed by the flexible time paradigm of flexible planning in the Situation Calculus for execution monitoring and plan generation.


Weitere Links

Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence