Can Finite State Automata, Numeric Mood Parameters and Reactive Behaviours Come Alive? Claus B. Madsen Paolo Pirjanian Erik Granum Project: I3-ESE PUPPET Laboratory of Image Analysis Aalborg University Contact address: Claus B. Madsen Laboratory of Image Analysis Fr. Bajers Vej 7D1 DK-9220 Aalborg East Denmark cbm@vision.auc.dk Introduction ------------ As a part of ongoing projects aimed at creating virtual, interactive theaters, we are engaged in the development of life-like, computer controlled characters. We aim at gaining insight into the design of computational models for autonomous agents, capable of partaking in believable and engaging interaction with humans (and other agents). In this paper we describe the design of a specific autonomous agent, focusing on the control of the agent's innate behaviours. Specifically how the notion of time plus perceived stimuli affect the internal state of the agent, which in turn affect agent behaviour. The agent, Bouncy, is implemented as a 3D animated character. Bouncy has been demonstrated to hundreds of people, and though conceptually simple, he has been wholeheartedly accepted by people in all age groups. In this paper we point out some important limitations in the current design, and us these for outlining the directions in which we are currently extending the agent model. Scenario and interaction ------------------------ Bouncy is a 3D animated character with behaviours similar to those of a dog. He moves around by bouncing up and down (hence the name Bouncy). He is able to express various emotions (happy, sad, etc.) by actuating his mouth, eyes, tail, and by the intensity with which he bounces. Interaction with Bouncy is facilitated by several interface modalities. Bouncy is displayed to the user on a graphical display (screen, optionally in 3D using shutter glasses). Bouncy also produces a "bounce sound" while bouncing. Conversely, the user can communicate himself to Bouncy through a microphone and a data glove. The actions of the user are perceived by Bouncy in the following manner: |---------------------------------------------------------| | User's behaviour || | |-------------------------|| Bouncy's perception of user | | glove input | mic input || | |---------------------------------------------------------| | none | shouting || "Master is calling me" | | pointing | shouting || "Master is scolding me" | | open hand | talking || "Master is comforting me" | |---------------------------------------------------------| Super-ordinate behaviour control -------------------------------- Bouncy has three main behaviours, one of which is further divided into three sub-behaviours: PLAY: Causes Bouncy to wander about and play SLEEP: Causes Bouncy to cease current activity, lie down and sleep INTERACT: Causes Bouncy to go to and/or follow user plus engage in interaction. Sub-behaviours: - tease: pay attention to user but do not approach - please: approach user, bounce, wag tail and smile - have-the-blues: stop bouncing, break eye-contact, look sad These behaviours constitute the states in a Finite State Automaton, (FSA), in which state-transitions occur based on symbols generated from Bouncy personality model. The personality model is based on 3 numerical parameters: excitedness, mood and sleepiness. Each parameter ranges from -1 to +1, and all are functions of time (e.g., Bouncy gets more and more sleepy when playing) and perceived user stimuli (e.g., excitedness rises when user is speaking, mood decreases when user is speaking and pointing). Using relatively simple logic expressions symbols are generated from the values of these parameters. E.g., when excitedness and mood are low, and sleepiness is high, the symbol "tired" is generated, forcing the FSA to go to the "sleep" state, which in turn causes the agent to cease any current activity, lie down, close the eyes and breathe slowly. Fusion of behaviours -------------------- The described behaviours affect the agent's "physical" degrees of freedom, in a manner where they do not interfere with each other. E.g., Bouncy can smile both while bouncing and at rest. Yet there are situations where a compromise has to be made between what two or more behaviours dictate. E.g. when trying to approach the user (user's avatar), but the direct path is obstructed by an obstacle. To solve this problem we have designed a formalism (Multiple Objective Actions Selection) for simultaneously satisfying the demands of several parallel behaviours requiring control of the agent's degrees of freedom. Thus, Bouncy can act in an environment with obstacles. Furthermore, we have experimented with flocks of Bouncys, where one Bouncy acts as leader dog, and the others merely follow. Because they all are independent autonomous agents, follow-dogs may be forced to take alternative paths in order to avoid obstacles, but they will always attempt to catch up with the leader. "Experimental results" ---------------------- Bouncy has been implemented in the SGI/Irix Performer graphics library, and over several months it has been demonstrated to hundreds of people, computer scientists, psychologists, dramaturgists, and "laymen" of all ages. Somewhat to our surprise audience reactions to Bouncy have been overwhelmingly positive, stressing the fact that an autonomous agent does not have to be complex in order to be engaging. As long as the agent is capable of spanning the thematic "universe" suggested to the spectator by the appearance of the agent and its surroundings. One major observation from demonstrating Bouncy to people is that interacting with Bouncy is not nearly as interesting, as it is watching somebody interacting with Bouncy. Thus, the fascinating aspect is not Bouncy per se, but the combined performance delivered by a person and Bouncy. This is in good accordance with our super-ordinate research goal, namely creating characters for various types of "performance", e.g., for allowing children to stage plays and tell stories. This is the aim of the I3-ESE project PUPPET. Conclusions and future research ------------------------------- Though intrigued by the results we feel we have more or less stretched the purely reactive formalism behind Bouncy to the limit. Since the agent is purely reactive an interaction with Bouncy is characterized by improvisation,- the user does something, Bouncy reacts, the user reacts to this, etc. Based on the experiences with Bouncy we have identified several areas where additional design is required. For one, a formalism is needed for adding an ability to plan behaviour in the pursuit of goals. Secondly, the concept of memory must be addressed, providing the agent with an ability to remember objects. In this context we are especially interested in the problem of attention selection (e.g., how does the agent select which object or which other characters to focus his attention on). Thirdly, more insight is required into how multiple agents can be coordinated to partake in longer, more conceptual activities. A simple example is the game of tag, which only is meaningful if the participating agents are synchronized in the perception of the state of the game, so that they can take turns in trying to catch the others. We are currently investigating these matters and will discuss them at the workshop. References to more detailed description of work ----------------------------------------------- The following to short technical reports contain further detail on the behaviour control of Bouncy. The first paper focuses on the personality model of Bouncy, the second paper focuses on how to integrate obstacle avoidance with Bouncy's other motion behaviours. http://www.vision.auc.dk/~cbm/psfiles/bouncy1.ps.gz http://www.vision.auc.dk/~cbm/psfiles/bouncy2.ps.gz Demo ---- If the paper is accepted for presentation at the workshop we will provide video demonstrations of Bouncy and the latest results from our other autonomous agent research.