Publication
Exploration in Relational Worlds
Tobias Lang; Marc Toussaint; Kristian Kersting
In: José L. Balcázar; Francesco Bonchi; Aristides Gionis; Michèle Sebag (Hrsg.). Machine Learning and Knowledge Discovery in Databases, European Conference. European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD-2010), September 20-24, Barcelona, Spain, Pages 178-194, Lecture Notes in Computer Science, Vol. 6322, Springer, 2010.
Abstract
One of the key problems in model-based reinforcement learning is balancing exploration and exploitation. Another is learning and acting in large relational domains, in which there is a varying number of objects and relations between them. We provide one of the first solutions to exploring large relational Markov decision processes by developing relational extensions of the concepts of the Explicit Explore or Exploit (E 3) algorithm. A key insight is that the inherent generalization of learnt knowledge in the relational representation has profound implications also on the exploration strategy: what in a propositional setting would be considered a novel situation and worth exploration may in the relational setting be an instance of a well-known context in which exploitation is promising. Our experimental evaluation shows the effectiveness and benefit of relational exploration over several propositional benchmark approaches on noisy 3D simulated robot manipulation problems.