Research department/group

Intelligent User Interfaces

The Intelligent User Interfaces department develops the basics for multi-modal human-computer interaction and personalized dialogue systems integrating speech, gestures, and facial expressions with physical interaction. In the process, user, task and domain models are used to design dialog patterns that are as natural and robust as possible so as to be understandable, even in group discussions or loud environments. By integrating virtual characters even emotions and socially interactive behavior can be achieved as an output. A major focus are mobile user interfaces to location-based and context sensitive services especially for applications in automobiles and shopping situations as well as at theme parks and museums. In addition to the intuitive access to the Internet of Services and Things in the context of the semantic web, the department also studies barrier-free access to instrumented environments and networked worlds for seniors and the physically challenged. Our research results are also applied to innovative security solutions and self-defending networks. Significant for this research department is the interdisciplinary method of operation especially the cooperation of computer scientists with computational linguists and cognitive psychologists as well as the empirical and ergonomic evaluation of spoken dialog systems and multimodal user interfaces.

The research programme THESEUS of the Federal Ministry of Economics and Technology aims at developing a new internet based knowledge infrastructure in order to better deploy and make use of the information contained in the internet. In the context of innovative basic technologies, this research department is responsible for the work package “situation-aware dialogue shell” in which multimodal interfaces for dialogue interaction between man and machine are being realised. Thus, users can formulate their questions intuitively and refine them during the dialogue with the system. Within the use case TEXO researchers develop an infrastructure for new web-based cross-company services that rely on Service Oriented Architecture (SOA), which allows for a flexible and semantically oriented procurement between service providers and service users. In this context the research department creates a mobile, multimodal interface for the access to Business Webs.

The Innovation Alliance Digital Product Memory, being part of the promotion programme IKT2020, has done research on the semantic interoperability of different product memories and on ensuring ubiquitous, multimodal access for their users conducting the project SemProM – Digital Product Memory with partners from the fields of research, trade, logistics, the pharmaceutical industry and automobile industry - lead by our research department. Work on the development of open standards is actively continued in the W3C Object Memory Modeling Incubator Group . SemProM has developed the basics for the next generation of mobile, embedded and function-based elements for the semantic internet communication between everyday objects. Thereby, capacities of such “intelligent” products go far beyond the mere identification function of current RFID labelling. They are able to evaluate data of different embedded sensors (e.g. temperature, lights, humidity, speed, acceleration, position) and to register all relevant product and process data at the same time. In terms of the “Internet of Things” this information can be actively exchanged with their environment, users and others products.

Another project in the Digital Product Memory research association is the Alliance Digital Product Flow (Allianz Digitaler Warenfluss or ADiWa) . ADiWa aims at making real-world information proactively available in complex and dynamic business processes through the Internet of Things. Currently the Internet of Things mainly uses real objects for the sole purpose of data capture, such as products tagged with intelligent labels. In the future, these objects could be further integrated into business processes. Furthermore, mechanisms will be explored to help select, design, guide, and even develop business processes from scratch. The software solutions developed could, for instance, help to improve maintenance strategies for plants and machinery or prove the authenticity of products and components.

The joint project RES-COM (resource conservation by context-avtivated machine-to-machine communication) will play an important role towards resource-conserving production and service offerings in the context of recommendations presented by the German Industry-Science Research Alliance concerning the project Industry 4.0 for the future. The RES-COM approach is based on a whole new technology which is composed of active digital product memories combined with software service agents enhanced with embedded sensors and actuators: that is recording resource consumption by decentralized active product memories, evaluating and communicating for coordination in order to autonomously optimize the underlying processes. RES-COM addresses these requirements by networking between local autonomous systems and centralized structures according to the vision of an Internet of Resource Efficiency.

The aim of the Peer Energy Cloud consortium is the development of innovative recording and forecasting methods for the development of load curves to optimize the load flow, as well as the establishment of a virtual marketplace for power trading within the so-called "Micro Grids." This marketplace allows the balancing of energy demand and supply within the local grid, which in turn reduces stress on the nationwide grid. This bottom-up approach helps to increase the overall network stability and reduces the need for costly upgrades to the nationwide grid infrastructure. To date, the actual electric power consumption per consumer at a certain point in time has not been at all predictable to electricity generators. Standard load profiles are too imprecise. The load curve is hard to optimize. Thus, the national grid always has to be oriented towards the possibility of maximum loads, with back-up power stations ready as needed for overloads. Our research department represents the DFKI in the Peer Energy Cloud consortium; other partners are the Karlsruhe Institute of Technology (KIT), security and public safety solutions organization AGT Germany, business integration and managed file transfer solutions and cloud computing provider SEEBURGER AG, and the public utility company Saarlouis.

Together with the German retail chain Globus SB-Warenhaus Holding the research department operates a new research laboratory Innovative Retail Laboratory – IRL . There innovative solutions in trade and logistics are investigated, developed and tried which clients are to benefit from especially when shopping.

The aim of the project CARMINA - sponsored by the Federal Ministry of Education and Research - is to lay down the basic principles for a new generation of context-sensitive multimodal interfaces of automotive applications which will support the following kinds of interactions: interaction of driver and passengers with the car and mobile internet services, multi-party interaction between passengers, which means between the driver, the co-driver, and passengers sitting in the back seat, and finally, interaction of driver and passengers with the environment outside the car. The project develops a task-oriented dialogue system which supports information access on basis of new opportunities of Car2X communication and provides interaction based on speech and tangible interfaces. Furthermore, multi-party aspects will be incorporated in order to support a seamless role-based access to devices and services. However, this requires that some serious problems must be solved, namely the handling of overlapping speech and speaker identification.

The research department complements its activities in the automotive sector by coordinating the European research project GetHomeSafe . The project aims to develop a system for safe information access (search, navigation, points of interest) and communication (texting) while driving as well as basic functionalities such as music management, address book, phone management search facilities and even social media update. The project will develop solutions involving multimodal dialogue, where speech, a key modality, will be handled by a hybrid (local and remote) speech recognition module. Further project partners are IBM, Nuance, Daimler and KTH.

The research department is coordinating the DFKI competence center Ambient Assisted Living - CCAAL , which is a virtual organization within the DFKI coordinating and conducting research and development in the area of Ambient Assisted Living - AAL. Therein, the department participates in a number of research activities in the context of intuitive interaction with domestic appliances for everybody based on industry standards. The aim is to provide an easier access to domestic appliances for people with mild cognitive disorders and elderly people, by applying a new standard for user interfaces, i.e. the “Universal Remote Console” (URC) (see also the EU project i2home ). The intelligent environment Smart Kitchen is operated for demonstration and evaluation purposes. Furthermore, the research department is involved in the BMBF project SmartSenior -Longer independence for senior citizens by the provision of reliable technology for locating users in the context of all applications and the development of web interaction free of language barriers with special focus given to the needs of the target group.

In the past the research department has been intensively involved in the field of multimodal interaction systems among others through the SmartWeb project, which was a lead innovation of the Federal Ministry of Education and Research. In this context the following tasks were taken over: project management, system integration and the realisation of a multimodal dialogue component. SmartWeb allowed for an intuitive access to the Semantic Web and its services by means of intelligent user interfaces in a mobile multimodal dialogue.

A Video Archive contains videos about various prototypes developed in past and current projects of our research department.

Head: Prof. Dr. Wolfgang Wahlster
Deputy Head: Dr. Dietmar Dengler


German Research Center for Artificial Intelligence, DFKI GmbH
Stuhlsatzenhausweg 3
Building D 3 2
D-66123 Saarbruecken

Phone: +49 681 85775 5252
Fax: +49 681 85775 5341