Publication
OCALM: Object-Centric Assessment with Language Models
Timo Kaufmann; Jannis Blüml; Antonia Wüst; Quentin Delfosse; Kristian Kersting; Eyke Hüllermeier
In: Computing Research Repository eprint Journal (CoRR), Vol. abs/2406.16748, Pages 1-29, arXiv, 2024.
Abstract
Properly defining a reward signal to efficiently train a reinforcement learning (RL)
agent is a challenging task. Designing balanced objective functions from which a
desired behavior can emerge requires expert knowledge, especially for complex en-
vironments. Learning rewards from human feedback or using large language models
(LLMs) to directly provide rewards are promising alternatives, allowing non-experts
to specify goals for the agent. However, black-box reward models make it difficult to
debug the reward. In this work, we propose Object-Centric Assessment with Lan-
guage Models (OCALM) to derive inherently interpretable reward functions for RL
agents from natural language task descriptions. OCALM uses the extensive world-
knowledge of LLMs while leveraging the object-centric nature common to many
environments to derive reward functions focused on relational concepts, providing
RL agents with the ability to derive policies from task descriptions.