A Multimodal Dialogue Mashup for Medical Image Semantics

Daniel Sonntag; Manuel Möller

In: Proceedings of the International Conference on Intelligent User Interfaces. International Conference on Intelligent User Interfaces (IUI-2010), February 7-10, Hong Kong, China, 2/2010.


This paper presents a multimodal dialogue mashup where different users are involved in the use of different user interfaces for the annotation and retrieval of medical images. Our solution is a mashup that integrates a multimodal interface for speech-based annotation of medical images and dialogue-based image retrieval with a semantic image annotation tool for manual annotations on a desktop computer. A remote RDF repository connects the annotation and querying task into a common framework and serves as the semantic backend system for the advanced multimodal dialogue a radiologist can use.


Weitere Links


Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence