next up previous contents New Search Help FLUIDS_Home DFKI_Home
Next: Intelligent User Interfaces - Latest Up: Intelligent User Interfaces Previous: Intelligent User Interfaces


Intelligent User Interfaces - New Achievements in Research

 The currently dominant interaction metaphor of direct manipulation requires the user to initiate all tasks explicitly and to monitor all events. The use of techniques from the field of autonomous agents provides a new complementary style of human-computer interaction, where the computer becomes an intelligent, active and personalized collaborator. Interface agents are computer programs that employ Artificial Intelligence methods to provide active assistance to a user of a particular computer application. The metaphor used is that of a personal assistant who is collaborating with the user in the same work environment. The assistant becomes gradually more effective as it learns the user's interests, habits and preferences. Several prototype agents have been built using this technique.

Separator_Line

Antropomorphizing software agents constitutes another important aspect of the evolution towards highly personalized interfaces. The core idea of the emerging Persona technique is that the system personifies itself audio-visually, e.g. by means of an animated character. The motivation for this sort of peripheral personalization is that an animated interface agent adds expressive power to a system's communicative skills. Research prototypes of intelligent user interfaces that achieve a lively and appealing behaviour of the animated agent are now available.

Separator_Line

(New Sep. 96) Although significant results have been gained from building the first generation of intelligent multimedia presentation systems, until recently there have been no promising attempts to assemble the different lines of expertise into a common framework. The lack of a well established methodological basis makes the industrial development of intellimedia systems an expensive, high risk undertaking.

The technical working group on ``Graphics and Knowledge Engineering'' within the Computer Graphics Network of ERCIM , the European Research Consortium for Informatics and Mathematics, has made significant progress towards the development of a standard reference model for intelligent multimedia presentation systems. A first draft of a generic reference model has been devised to serve as a basis for discussions (see Ruggieri et al.). Furthermore, a workshop has been held during ECAI-96, the 12th European Conference on Artificial Intelligence, to bring together leading researchers in the field of intelligent multimedia presentation systems. The high quality of the contributions and the results obtained during the workshop form a promising basis for further improvement of the preliminary version of the reference model.

Separator_Line

(New Feb. 97) Dalal et al. [1996] are concerned with the automatic coordination of temporal media, such as speech and animation, in a knowledge-based multimedia presentation system. They propose a multi-stage negotiation process in which media-specific generators interact with a temporal constraint solver serving as coordination component. The approach is realized and further tested within a multimedia authoring tool for a healthcare application. This particular system generates multimedia briefings that describe the postoperative status of a patient undergoing cardiac bypass surgery. Intelligent multimedia presentation techniques are exploited to allow automatic generation of briefings tailored for different kinds of caregivers.

The solution proposed by Dalal et al. bears many similarities to the approach followed within the PPP intelligent multimedia presentation system. The details of temporal coordination in PPP are described in André and Rist [1996].

Separator_Line

(New Feb. 97) The work by Faraday and Sutcliffe [1996] addresses the evaluation and successful design of multimedia presentations. The paper reports on empirical studies concerning attention to and comprehending multimedia presentations. The results of investigating a commercially produced CD-ROM show that design problems may lead to difficulties in comprehension. Attention can easily be overloaded, resulting in presentation elements being missed. Thus attentional design should take into account the need to order shifts of attention according to the content being shown. The authors suggest that comprehension may be improved by ensuring that the elements of the presentation could be formed together in unified propositions. This requires careful coordination and tight cross-references between the different media.

With respect to intelligent multimedia interfaces, the empirical work started here could provide better guidelines as they are necessary for integrating design advice into knowledge-based multimedia presentation systems.

Separator_Line

(New Mar. 97) Schlimmer and Wells [1996] report on a study related to intelligent user interfaces which aimed at a detailed evaluation of the benefits of three different input techniques for entering information into a computer. Experiments were conducted to compare handwriting recognition, adaptive menus, and predictive fillin in the context of adding a person's name and address to an electronic organizer. The results of these tests show that handwriting recognition is slower than typing on an on-screen, soft keyboard, while adaptive menus and predictive fillin can be twice as fast. The paper also shows that all three techniques work well together and presents some principles for applying them to other information collection domains.

Separator_Line

(New Mar. 97) Latest advances in the field of multimedia-based user interfaces are beeing addressed in recent articles by Blattner and Glinert [1996]; MacIntyre and Feiner [1996]; Waibel et al. [1996]. The paper by Blattner and Glinert surveys recent research related to the general problem of designing integrated multimodal interfaces, i.e. systems that include novel interaction techniques like touch, gestures, voice, and 3D sound. A major focus is on approaches which aim to provide a generic platform in support of multimodal interaction.

Waibel et al. present an overview of their pioneering work on multimodal human-computer interfaces. These research activities target at a new generation of user interfaces which will involve all available human communication modalities for human-machine interaction including speech, gesture and pointing, eye-gaze, lip motion and facial expression, handwriting, face recognition, face tracking, and sound localization.

The paper of MacIntyre and Feiner provides another outlook on future multimedia interfaces. The authors predict that virtual environments on the one hand and ubiquitous computing on the other hand will play major roles in the next generation of multimedia user interfaces. The article reviews the state-of-the-art in these two emerging areas of user interface research and explores where these new paradigms are heading.

Separator_Line

(New Apr. 97) The proceedings of IUI-97 present recent outstanding research results from the whole field of intelligent user interfaces. The excellent contributions discuss novel approaches and provide new results especially from the following subareas:

So-called interface agents constitute one of the hot topics in the field. The superiority of agent-based approaches versus direct manipulation interfaces is, however, an open question among practitioners and researchers in human-computer interaction. The papers by Maes [1997] and Shneiderman [1997] present these controversial positions and provide some insight into the debate.

Separator_Line

(New July 97) Also the area of software development itself offers a high potential for the application of intelligent user interface technology. Advanced interfaces can aid programmers to better understand algorithms and computer programs not only during software production but also for training purposes. The field of Software Visualization (see e.g. Stasko et al. [1998]) explicitly addresses the use of typography, graphics, sound, and animation techniques to present program code, data, and control flow. In making programming a multimedia experience, software visualization equips programmers and computer science researchers with intelligent interfaces that simplify routine tasks and allow the user to concentrate on more challenging problems.

The article by Baecker et al. [1997] is specifically concerned with software visualization for debugging and discusses various approaches for representing programs. The different techniques which have been investigated include algorithm animation, typographic source-code presentation, and interactive auralization. Debugging as a particularly challenging form of problem solving requires effective representation and presentation aids and the results of the study indicate that the described techniques offer a high potential for future programming environments.

Separator_Line

(New Aug. 97) The session on ``Intelligent Support'' of the CHI-97 conference on Human Factors in Computing Systems provided two interesting contributions from the field of intelligent user interfaces. The papers by Lieberman and Barrett et al. present recent research results which are both related to practical applications in the context of accessing information on the World Wide Web.
Lieberman [1997] focusses on employing agent technology within intelligent user interfaces. A system which aims to provide intelligent assistance for Web browsing is presented as a sample application for a novel approach to agent-based interfaces. An important feature of this kind of intelligent interface is the use of software agents that are both interface agents and autonomous agents. Such a system does not only mediate between the interface and the user but is also able to operate the interface independently of and concurrently with the human user in order to provide an even higher level of assistance.
A similar application scenario is considered in Barrett et al. [1997]. The paper describes WBI (Web Browser Intelligence), a system which offers an enhancement of conventional browser software by automatically personalizing WWW access using agent technology. WBI relies on a multi-agent system to observe user actions, proactively offer assistance, modify web documents, and perform new functions. As the system can annotate hyperlinks with network speed information, record pages viewed for later access, and provide shortcut links for common paths it equips a user with an interface that provides a more personalized experience of World Wide Web access.

Separator_Line

(New Feb. 98) Shih and Davis [1997] introduce an interactive multimedia presentation development system that enables a designer to construct an adaptive presentation as a CD-ROM title. IMMPS, the Intelligent Multimedia Presentation System, follows a knowledge-based approach and provides a developer with a presentation specification language and an environment for the specification and learning of audience characteristics. Furthermore, an object-oriented multimedia resource and presentation database management system is integrated into IMMPS and facilitates full or partial reuse of resources and presentations. A rule-based format is used to describe audience background information and to specify knowledge inheritance relations between the different presentation windows. The underlying inference engine ensures that common information and knowledge is inherited and shared by presentation windows. During actual presentation time, the inference system can be exploited to learn from the users responses in order to dynamically adjust the presentation according to the user's background and knowledge. IMMPS support for personalization makes the environment especially attractive for intelligent tutoring. The designer can use the system to create a presentation that is customized on the fly based on the changing user information.

Separator_Line

(New March 98) Flexible access to large amounts of multimedia information offers a very interesting application area for intelligent user interface technology. The emerging field called intelligent multimedia information retrieval lies at the intersection of artificial intelligence, information retrieval, human-computer interaction, and multimedia computing. Research and development in this area aims at powerful systems which enable users to create, process, summarize, present, interact with, and organize information within and across different media such as text, speech, graphics, imagery, and video. These novel systems go beyond traditional hypermedia and hypertext environments to analyze and generate media, and support intelligent interaction with or via multiple media.
The contributions in Maybury [1997] document recent advances regarding intelligent multimedia information retrieval and provide a good overview off current research activities in this new field.

Separator_Line

(New May 98) A new anthology edited by Maybury and Wahlster [1998] provides an excellent overview of the current state-of-the-art in the field of Intelligent User Interfaces. The book represents a collection of the classic and contemporary readings in the field and includes both fundamental research and applied innovations in all key areas of Intelligent User Interfaces research and development. Topics covered include multimedia input analysis, multimedia presentation design, automated graphic design, automated layout, user and discourse modeling, model-based interfaces, agent interfaces, and evaluation aspects. Each paper concludes with a reflection by the original author on what worked, what did not, and where opportunities remain, as well as commentary on subsequent research and advances since the publication of their work, including important developments and key follow-up publications by the author and others.

Separator_Line

(New Aug. 98) Anthropomorphic interface agents add expressive power to the communicative capabilities of an interactive software system. Consequently, the use of lifelike characters as animated interface agents has developed into one of the salient topics within the field of intelligent user interfaces. The article Elliott and Brzezinski [1998] provides a recent review of the state-of-the-art in this area. The authors present various approaches towards autonomous agent systems that embody their intelligence at least partially through the projection of a believable, engaging, synthetic persona. The topics covered in particular include aspects like synthetic personality and representations of emotion, character societies, intelligent automated tutors, as well as projecting users into virtual worlds using avatars.

Separator_Line

(New Sep. 98) A recent collection by Lee [1998], which has been published as an electronic book, presents recent results from the field of intelligent multimedia systems. The book offers a nice combination of papers that tackle technical problems in improving the responsiveness and adaptiveness of systems, and contributions which are directed more towards design and evaluation of the presentations and interactions that users will need. The specific topics being addressed include multimodality and natural language, theoretical and formal approaches to multimodality, system design based on the intentions and expectations of the user, methodologies for multimodal system design, as well as cognitive approaches and system evaluation.

Separator_Line

(New Oct. 98) Two new technical reports from the American Association for Artificial Intelligence document latest results from specific sub-areas within the general field of intelligent user interfaces. These reports contain the papers from two recent workshops that have taken place in the context of the 1998 AAAI conference . The first report AAAI [1998a] focuses on automatic recommender systems which employ collaborative information filtering techniques to provide assistance in making choices. The various contributions explore current issues in this application area like techniques for generating recommendations and learning user profiles, identification of different types of recommendations, visualization of recommendation spaces, personalized versus non-personalized recommendations, social filtering, and commercial exploitation of the technology.

The second report AAAI [1998b] is concerned with intelligent multimedia systems and concentrates on computer representations for multimodal human-machine communication. The different workshop contributions discuss both facets of human-computer interaction: On the input side, machine understanding of human communication in multiple modalities and on the output side natural communication with humans using multiple media. The papers contained in the report cover a broad research spectrum and provide a good overview concerning the state-of-the-art in developing and using representations that facilitate multi-modal interaction.


next up previous contents New Search Help FLUIDS_Home DFKI_Home
Next: Intelligent User Interfaces - Latest Up: Intelligent User Interfaces Previous: Intelligent User Interfaces


Gerd Herzog
Last update: Fri Feb 26 13:17:30 MET 1999


Send comments to herzog@acm.org