Interactive Communicative Humanoid Agent for Movie Selection

Jacob Buck

Intelligent Multimedia    &  Depart of Artificial Intelligence
Aalborg University           Edinburgh University
Denmark                      Scotland

jacobb@dai.ed.ac.uk


With pay-per-view TV and video-on-demand systems that are on the rise,
it will be possible to select which movie/show to watch at any
time. With fixed programme schedules it is very convenient just to sit
back and watch. The only interaction requiring any special choice is
_when_ to use the remote control and _when_ to stop zapping to _what_
a specific program. Expanding the choice with selection from a broad
variety of programs or movies suddenly requires knowledge of what you
want to watch and when. It is not always easy to know these
things. Often you just have rough idea of what to watch, but you would
like some assistance.

An "intelligent" interactive agent can be used to help the user search
and select what to watch. The agent can then in discourse help with
suggestions and guidance through a database of movie information. This
database can contain both objective facts (e.g. list of movies an
actor appeared in) and subjective information (e.g. critics and lists
of similar movies).

The motivation for this project is that face-to-face interfaces may
improve the usability of some types of applications. This is mainly
because face-to-face interaction is generally effortless and effective
[1]. This type of HCI may even fill the gap between people who use
computers often and those who have know computer skills. For a long
time speech technology has been employed, but I think we need to get
even closer to how humans communicate.

Equipping the agent with communicative skills, both verbal and
non-verbal, is a complex task combining multiple modalities. In
addition the agents task knowledge (the knowledge it is designed to
mediate) conversation depends on speech, gaze, facial gestures and
gesticulation [2]. As any HCI system feedback is important. With an
embodied agent speech and gestures are useful and natural in
conversational feedback. For instance, gaze behaviour can support the
information structure and turn taking behaviour [3]. The non-verbal
feedback can be content-related, emotional or based on the process of
conversation. Though the emotional feedback can make the agent more
believable [4], the process based feedback is more important
[5]. Having faces and facial expressions is likeable, engaging [6] and
it enables the agent to imitate face-to-face communication. Today's
computer graphics and speech technology enables the agent to appear as
3D human-like talking head, though the underlying processing is still
at it's childhood.

The application under development will enable you and a movie agent
(as a 3D human-like talking head) to have conversations about movie
features (e.g. telling which movies where two specified actors appears
in) with the aim for you to select a specific movie. In this domain,
the agent during discourse can present pictures, movie clips, audio
clips and trailers back. Eventually a specific movie can be selected
and displayed. Until I get a full digital movie library, the agent
will just give reference to the videotape in my personal video library
containing the selected movie.

References

[1] Thorisson, K.R., "Layered Modular Action Control for Communicative
Humanoids", Computer Animation '97, Geneva, Switzerland June 5-6,
1997.

[2] Thorisson, K.R., "Dialog control in social interface agents",
InterCHI Adjunct Proceedings '93, conference on Human Factors in
Computing Systems, Amsterdam (p139-140), April 24-29, 1993.

[3] Torres, E.O., Cassell, J., Prevost, S., "Modeling Gase Behaviour
as a Function of Discourse Structure", First International Workshop on
Human-Computer Conversations. Bellagio, Italy, 1997.

[4] Loyall, A.B., Bates, J., "Personality-Rich Believable Agents That
Use Language", (Technical Report CMU-CS-94-136): Pittsburgh, PA:
School of Computer Science, Carnegie Mellon University, 1995.

[5] Cassell, J. and Thórisson, K.R. (to appear), "The Power of a Nod
and a Glance: Envelope vs. Emotional Feedback in Animated
Conversational Agents.", Journal of Applied Artificial Intelligence.

[6] Koda, T., Maes, P., "Agents with Faces: The Effects of
Personification of Agents", Proceedings of HCI'96, The British HCI
Group, London, UK (p98-103), August 20-23, 1996.