Publication
Situated Production and Understanding of Verbal References to Entities in Large-Scale Space
Hendrik Zender
Saarbrücken Dissertations in Computational Linguistics and Language Technology, Vol. 36, ISBN 978-3-933218-35-3, German Research Center for Artificial Intelligence and Saarland University, Saarbrücken, Germany, 12/2011.
Abstract
The work presented in this thesis addresses the fundamental questions of how machines,
such as robots and other autonomous agents, can acquire a mental representation of their environment
that allows them to (a) act and navigate in it, and (b) communicate about it with humans in natural
language.
We specifically investigate representations of structured environments that cannot be apprehended as
a perceptual whole (i.e., large-scale space). This comprises, for instance, indoor domestic
environments, or building ensembles. By that, the presented work goes beyond situated natural
language interaction about an agent's immediate surroundings (i.e., small-scale space), such
as table-tops or single room spaces.
Situated communication about entities – that is, things, places, properties, and
events – in large-scale space requires the interlocutors to draw attention to entities that are not
currently observable, and to comprehend which remote places and things are being talked about.
We furthermore show how such representations that can be used for spoken interaction with human
users also endow autonomous agents with skills for context-aware planning and execution of actions
in structured environments that are made by and for humans. To this end, the presented spatial
models have been implemented and deployed in integrated systems for intelligent mobile robots.
We then present an approach for natural language generation and understanding that makes use of the
acquired spatial models. It allows an agent to successfully generate and resolve natural language
expressions that refer to entities in large-scale space. The approach is backed by observations from
an empirical spoken language production experiment. The thesis concludes with a discussion of
ongoing work to transfer the models made for intelligent mobile robots to autonomous virtual agents
that act in an online virtual 3D world.
Projekte
CogX - Cognitive Systems that self-understand and self-extend},
CoSy - Cognitive Systems for Cognitive Assistants
CoSy - Cognitive Systems for Cognitive Assistants