Dealing with the Lurking Lutheran view on Interfaces: Evaluation of the Agneta and Frida system

Kristina Höök, Marie Sjölinder, Anna-Lena Ereback, Per Persson

SICS, Box 1263, 164 29 Kista, Sweden

http://www.sics.se/humle/projects/persona/web/index.html

{kia, marie, annalena, perp}@sics.se

 

Agneta and Frida

Navigation in information spaces is a cognitively demanding activity, which sometimes make us feel frustrated or anxious. We get lost, we do not find what we wanted, we are exposed to technical problems where files are not found, we wander in circles, and we sometimes get so frustrated so that we give up. Furthermore, there are large individual differences (partly due to our spatial ability (Dahlbäck et al., 1996)) that point at huge differences in how well different groups of users are at navigating information spaces. It seems obvious that we need to find new approaches to design.

We are exploring a couple of ideas where we rely much more on inducing a sense of relaxed relationship and social aspects to the navigation in the information space. One such idea is the AGNETA & FRIDA system (Höök et al, 1998) – turn to Persson (this workshop) for a description of the system. In summary, Agneta and Frida are two animated females (mother and daughter), sitting in their living-room chairs, watching the user’s browser (more or less like watching television). In contrast to usual agents Agneta and Frida are not serious and polite guides that are part of the interface, but often distance themselves from computer culture in general and its male dominance in particular. Humour and irony are crucial elements here.

Studying anthropomorphic interfaces

A "normal" usability study of Agneta & Frida would not address any of the factors why such a system is interesting to develop. One reason for the vehement critique of interface agents is that it is rooted in a preconcieved notion of what human activities that computer interfaces are intended to support. The design of direct manipulation interfaces was motivated by the usage of computers as fast and efficient tools to support users in task-oriented behaviour. But today, computer interfaces have a much wider range of applications: they are used to support users learning about novel domains, controlling real-time processes, communicating with other people, and simply for pure entertainment. This has already lead to novel design cultures, which in many cases violate the basic principles of interface design.

Instead of measuring efficiency of Agneta&Frida, it is the experience of them, whether it is a delightful, motivating, or arousing experience, which matters. The move to interface agent interaction is thus largely motivated by a change in the usage of interfaces, that requires a new, multi-dimensional approach to usability evaluation that take effects on learning, motivation, arousal, etc. into full account. Synthetic characters in the interface enables several other reactions in their users: they may induce empathy, they make users attribute intelligence and in general anthropomorphic qualities to the system (Reeves and Nass, 1998), they may allow for a more narrative, subjective, interactive experience (Murray, 1997), they may induce both negative and positive affective responses in the users (Picard, 1997), etc.

Previous studies

When anthropomorphic interfaces have been studied it has foremost been with the aim to see which expectations they raise in their users. (The general conclusion seem to be that the more "natural" the interface, the higher expectations on intelligence in the system (Sproull et al, 1996, Brennan and Ohaeri, 1994, King and Ohaya, 1995, Koda and Maes, 1996, Reeves and Nass, 1997).) Another question we may want to ask is what is improved by an anthropomorphic interface? Is there such a thing as a "persona effect"? That is, if everything else is kept the same, and the only addition we make is a character in the interface – what will it add? This was studied by van Muelken et al. (1998). They compared the PPP Persona system in a version with the character presenting information to one where everything was similar except that they had removed the character (but the pointing was still present) (van Muelken et al., 1998). The study did not reveal any effects on the objective measurements taken of how much was remembered afterwards or whether the explanation was understood, but it revealed a positive effect on the subjective estimating on whether the explanation was difficult or not. Subjects experienced the explanation as simpler with the PPP Persona character than without.

There are of course numerous variables that are not controlled in such a study, like for example softer aspects of the characteristics of the character. In a study by Wright et al. (1998) a plain textual explanation of a medicine was compared to one with the same text but with an animated draggon illustrating the different threats to the blood system. Here a negative effect on how much was remembered afterwards appeared, so the draggon disturbed subjects, rather than aid them.

These conflicting results (PPP Persona and the draggon studies) points to the need of better understanding how animated or synthetic characters must be designed to not draw attention from the content or main tasks of users. As pointed out by Andrew Stern (Hayes-Roth et al. 1998) (designer of the Catz and Dogz system) the artistic design and practical understanding of the creating of synthetic characters is crucial in determining the success of a system. Chris Elliott (1998) cites Lester et al. (1997):

"Lester gives the examples of, on the one hand, a humorous, lifelike, joke-cracking, character that ultimately impedes problem solving through his distracting presence; and on the other, a dull assistant that always operates appropriately but yet fails to engage the student. When communications from an agent must be coordinated to be both engaging and purposeful issues in timing, and the multi-layering of actions arise."

But synthetic characters can be used in many ways and their purpose may not only be to enhance a presentation task. In a study by Elliott, (1997), it was investigated to which extent a computer-generated face with spoken output (and music) could express recognisable emotions. The generated face was compared to a human actor. Both were given 12 versions of one sentence to be pronounced with different emotions. The computer was given the emotion category and the text, and it automatically selected the face, music, and spoken inflection appropriate to that category. Face morphing, speech generation, and music retrieval and synthesis were all done in real time. Actual music selection was up to the program, based on pre-existing categories.

Overall, subjects did significantly better at correctly matching videotapes of computer-generated presentations with the intended emotion scenarios (70%) than they did with videotapes of a human actor attempting to convey the same scenarios.

In summary, the studies described above tells us something about the compelling still difficult nature of anthropomorphic interfaces, but they also tell us that designing these characters will not be an easy task. It is furthermore not clear exactly how to measure the effects of these characters. What is a successful system? Is it one that impedes learning but is perceived as fun and subjectively easy to grasp? What is more important and when? There is a lurking Lutheran view here that we need to come to grips with: why must computer systems first of all be useful and only secondly delightful or fun?

Studying Agneta & Frida

We are currently in the middle of a user study on the AGNETA & FRIDA system where we aim to explore some of the more "soft" features of navigation. Our study will not be fixed on how well users perform the tasks that they face in terms of time spent, number of errors made, or any of the "old" usability measures. We want to know whether Agneta & Frida:

The problem is how to measure these aspects (and whether measurements in some objective meaning are of any real use?). The following data has been collected for 20 subjects using Agneta & Frida:

Early results

Early results from our study indicate that:

Some subjects experienced the characters as a distracting feature, which had a negative effect on concentration and attention in the browsing situation. These subjects also reported lack of concentration in understanding the information in the pages and that the characters’ comments caused a lack of interest in the information in the pages. On the other hand some subjects felt less restless in the browsing situation with the company of the characters. These subjects also reported that they had noticed information in the pages that they otherwise (without the characters) wouldn’t have noticed.

In the case of memory support, there where subjects that reported that both the content of a page and the way back to it where better memorised due to the characters’ comments.

Some of the subjects talked about the characters in an anthromorphic way which is showed in subject comments as that the characters made them felt less lonely or that the characters where enervating beings in the information search situation. Other subject comments concerned the characters behaviour or feelings, for instance "it is fun that the characters are far too self-assured" or "the characters thought it was dull when…". Several subjects also wanted to be able to answer the characters or get even with them when their comments had been nasty. In the questionnaire, subjects indicate that the characters are "humanlike" to a large degree.

Lutheran views

The comments made by subjects show that they expect systems like Agneta & Frida to make help them search for better, that it should be a tool through which tasks are efficiently performed. Some subjects claim that despite the fact that Agneta & Frida would disturb them in a pure search situation, they would still use them. Other subjects claim that they would be too disturbed in the long run, or that they do not believe that Agneta & Frida adds anything important to their interaction. In both cases, it reveals how strong the prevailing tool-view of interface is.

Acknowledgement

The work reported here was done as part of the EEC-funded PERSONA project.

REFERENCES

Benyon, D. and Höök, K. (1997). Navigation in Information spaces: supporting the individual, Paper presented at INTERACT ‘97, Australia.

Dahlbäck, N., Höök, K. and Sjölinder, M. (1996). Spatial Cognition in the Mind and in the World - the Case of Hypermedia Navigation, The Eighteenth Annual Meeting of the Cognitive Science Society, University of California, San Diego.

Elliot Clark and Brzeinski, Jacek (1998) Autonomous Agents as Synthetic Characters, Communications of the ACM, Summer 1998 issue.

Hayes-Roth, Gene Barbara, Ball, Christine Lisetti, Rosalind W. Picard, Andrew Stern (1998) Panel Session: Affect and Emotion in the User Interface at the Intellignet User Interface (IUI) conference, San Fransisco.

King, W.J., and Ohya, J. (1995) The representation of agents: a study of phenomena in virtual environments, Proc. of the 4th IEEE International Workshop on Robot and Human Communication, RO-MAN’95, Tokyo, Japan.

Koda, T. and Maes, P. (1996) Agents with Faces: The Effects of Personification of Agents, Proceedings of HCI'96, London, U.K.

Lester, J., Converse, S., Stone, B., Kahler, S., and Barlow, T. (1997) Animated Pedagogical Agents and Problem-Solving Effectiveness: A Large-Scale Empirical Evaluation. In Proceedings of the Eighth World Conference on Artificial Intelligence in Education, Amsterdam, the Netherlands. IOS Press, pages 23-30.

Maglio P. and Matlock T. (1998). Metaphors We Surf the Web By, Paper presented at Workshop on PERSONALISED AND SOCIAL NAVIGATION IN INFORMATION SPACE, Stockholm

Murray, J. (1997). Hamlet on the Holodeck: The Future of Narrative in Cyberspace, Cambridge: MIT Press.

Picard, Rosalind. (1997). Affective Computing. MIT Press.

Reeves, Byron and Nass, Clifford (1996) The Media Equation: How People Treat Computers, Television, and New media Like Real People and Places, Cambridge University Press.

Sproull, Lee, Subramani, M., Kiesler, S., Walker, J. H., and Waters, K. (1996) When the Interface is a Face, Human-Computer Interaction, 1996, Vol. 11, pp. 97 – 124.

Patricia Wright, Steve Belt & Ann Lickorish (1998) Animation, the fun factor and memory, at the Computers and Fun workshop in York, organised by Andrew Monk on behalf of the British HCI group.