Epilogue on the ECAI'98 workshop on

Combining AI and Graphics for the Interface of the Future

Apart from immediate discussions after the presentations, each session ended with a discussion round in which we tried to identify topics and issues of general interest. The outcome of these rounds provided the basis for the three working groups which were formed at the end of the workshop. The members of each working group were asked to recall and formulate questions and directions for further research in the area. This exercise created an atmosphere for more informal and personal discussions and led to interesting questions which are presented in the following.

Group I "Symbolical descriptions of graphics"

Identified issues:

• Is it possible to calculate appropriate and sufficient representations at all?
• How "complete" should a representation be? Which aspects can be omitted and if, which reduction/filtering methods are then needed?
• How to cope with non-material concepts such as holes, gaps, in between spaces etc., and metagraphical objects such as encirclements, annotation arrows and the like?
• For picture retrieval tasks it might be necessary to describe differences between an "ideal candidate" and a "best match". But what shall we compare with each other? - graphical elements or symbolic descriptions associated with pictures? How can we measure and express similarities/differences?
• Having a logical structure of a picture is oftentimes useful - but what should it look like? Can we carry over linguistic concepts (say structuring principles such as rhetorical and intentional structuring) in a straightforward way?
• Do we need a compositional "(description) language for pictures" or is it better to treat pictures as holistic units?
• How to describe the pragmatic dimension of a certain picture style?
• Visual highlighting is often used to set the focus in a graphics while animations are a common means to indicate a focus shift in a graphical presentation. But there is still the question of how to describe the attentional structure of static graphics and animations?
• How to define a valid versus an invalid graphical discourse?
• What are appropriate methods for creating (symbolic) representations for pictures. What can be deduced automatically? If it is done by a human, what kind of tools would facilitate the task (e.g. use classification and identification heuristics in a hierarchical approach and rely on clarification dialogues to resolve ambiguities).

Group II "Interaction with graphical representations"

Identified issues:

• Transparency: Even though clickable graphics are getting more and more popular it is often not clear which interaction possibilities are offered by a graphics. So what are appropriate means to make such possibilities transparent to the user? How to make them intuitive and easy to use? But also, how to make transparent in which way the interaction is constrained?
• Which role play graphical representations in interaction? Many applications rely on graphical elements as an easy device for referring to objects (likewise material or abstract) of an underlying domain. In this case, the user may interact with domain objects through their graphical representations. However, there are other forms of interaction with graphics. Consider for example navigation in a 3D environment where the depicted scene objects provide the frame of reference for viewpoint manipulations. Also, a user may employ other modalities (e.g. language) to interact with graphical elements or indirectly with the domain concepts behind them. Also graphical interaction methods and styles has been addressed in various projects, a common theory of graphical interaction has not yet evolved. Among other things, such a theory should explain observable differences in the way users interact with graphical representations:
• in relation to the choice of input and output devices, and
• in the absence and presence of other modalities
• How can we measure the effectiveness of a graphical representation for interaction.

Group III "Computer supported and automated graphics design"

In order to support the creation of a graphics by a computer or even strive for a fully automated generation process it is necessary to identify design constraints and to formulate them in a machine-interpretable way. Constraints on both contents and style of graphics are first of all derived from the purpose the graphics has to serve. E.g., a map to help its user in location and navigation tasks. The formulation of appropriate constraints usually requires expert knowledge from professional design areas, though some constraints may reflect common-sense knowledge. However, there are various questions which has to be addressed in more detail:

• what constraints are relevant for which type of graphics?
• how to relate communication goals with design constraints? For example, which constraints must be fulfilled for a graphics that deserves the attribute "good looking"?
• how to formulate constraints: positive or negative, (constructive, exclusive)
• how to classify constraints with regard to computational effort
• which of the constraints must be fulfilled (hard constraints), which are less important (soft constraints)?
• when and how to apply constraints at which granularity?
• how to cope with interfering design constraints?

As to the operationalization of a design process one has to consider that there are usually multiple design solutions which satisfy the required constraints. Issues to addressed here are:

• how do we rank different design solutions?
• how to perform an evaluation? What can be anticipated and what remains to be done during design time?