R A D S P E E C H
A SEMANTIC SPEECH DIALOGUE SYSTEM FOR RADIOLOGISTS                              
 GERMAN HIGH TECH AWARD 2011!  
                                                                      
German High Tech Award 2011
Youtube Video    IUI-2011 Flyer    DFKI-Flyer
  
Radspeech is part of the THESEUS-MEDICO project which is funded by the Federal Ministry of Economics and Technology.

Bundesministerium für Wirtschaft und Technologie          RadSpeech - Speech Dialogue in Biomedicine             RadSpeech Example Dialogue Sequence  

Mobile Interface Screenshots
Images with courtesy of Siemens AG / University Hospital Erlangen.


Motivation


With traditional user interfaces, users may browse or explore visualised patient data, but little to no help is given when it comes to the interpretation of what is being displayed. Semantic annotations should provide the necessary image information, and a semantic dialogue system should be used to ask questions about the image annotations while engaging the clinician in a natural speech dialogue. Our motivation in developing RadSpeech is the design and implementation of a multimodal dialogue system for the radiologist. Dialogue-based semantic image retrieval should provide the basis for help in clinical decision support and computer aided diagnosis.
iPad ODP Architecture

Main Objectives

With RadSpeech, we aim to build the next generation of intelligent, scalable, and user-friendly semantic search interfaces for the medical imaging domain, based on semantic technologies. Ontology-based knowledge representation is used not only for the image contents, but also for the complex natural language understanding and dialogue management process. With the incorporation of higher level knowledge represented in ontologies, different semantic views of the same medical images (such as structural, functional, and disease aspects) can be explicitly stated and integrated.

A radiologist’s daily task in the SIEMENS patient image finding stations as installed at the University Hospital Erlangen defines the application scenario. We strive for more efficiency during the medical finding process and for more structured finding reports including semantic image annotations.

Project Plan

Project Duration: June 2010 - June 2012
(As of June 2012, RadSpeech' s development will be carried on in THESEUS Data Intelligence)

Principal Investigator: Dr. Daniel Sonntag, DFKI
Industrial Partners: Siemens Healthcare, Siemens Technology
Clinical Partner: University Hospital Erlangen

THESEUS MEDICO RadSpeech

Associated researchers and developers:
Christian Reuschling (DFKI), Christian Husodo Schulz (DFKI), Michael Sintek (DFKI), Daniel Porta (DFKI), Dr. Manuel Möller (DFKI), Maha Ahmed Baker (DFKI), Jochen Setz (DFKI), Milos Kalab (DFKI), Colette Weihrauch (DFKI), Luis Galárraga (DFKI), Martin Kramer (Siemens)

Industrial and clinical advisors:
Dr. Sascha Seifert (Siemens),  Prof. Dr. Alexander Cavallaro (University Hospital Erlangen)Dr. Matthias Hammon (University Hospital Erlangen)

News

(November 2012) Follow-up project ERmed, Kick-off! ERmed

(June 2012)  Interview with Daniel Sonntag in eHealth.com Journal (Issue June 2012, DE)

(June  2012) We exhibit at the eHealth Conference 2012!
- Collaborative iPad scenario
- Press articles: THESEUS News 
(DE), eHeathServer.de (DE)    

(May  2012) We exhibit at Deutscher Röntgenkongress 2012!

Deutsche Röntgengesellschaft
BMWi press release of the MEDICO event (DE)

(March 2012) We exhibit at CeBIT 2012!  DFKI Newletter, 1/2012 (EN DE)Cebit 2012
- (1) iPad / iPhone Collaborative Speech Dialogue Interaction!
- (2) Interactive Paper and Smartphones for Mammography!

Please visit the Cebit THESEUS / EIT talks  on Friday, March 9


(February 2012) We have been selected to (re)present DFKI's  IT solutions at the THESEUS final congress to Dr. Philipp Rösler, German Minister for Economic Affairs (DE, FR)

(January 2012)
- Radspeech Article in HealthTechWire (DE)
- Radspeech added to Nuance's medical speech documentation news (DE)

(December 2011) We organise a THESEUS medical imaging event
(DE)

(November 2011) Radspeech wins the German High Tech Champions (GHTC) Award! (EN)  award Research in Germany  

(September 2011) Deutsches Ärzteblatt (German Medical Journal) 2011; 108(38)
(DE)
"Radiologiearbeitsplatz am iPad" / "Radiology workstation for the iPad"

(June 2011)  THESEUS Newsletter: Radiology workstations of the future / "Radiologenarbeitsplatz der Zukunft" (DE)

(June 2011) We exhibit at Deutscher Röntgenkongress 2011!  
Deutsche Röntengesellschaft(THESEUS Newsletter DE)
- (1) iPad Multitouch Interaction and Semantic Search!
- (2) iPad Speech Dialogue Interaction!


(Mai 2011) We exhibit at "Night of Science" in Berlin 2011!  
Night of Science(THESEUS Newsletter DE DE)


(March 2011) We exhibit at CeBIT 2011!  BMWi Announcement (DE)

CeBIT 2011
 (DFKI Newsletter DE EN) 
- (1) iPad Multitouch Interaction and Semantic Search!
- (2) iPad Speech Dialogue Interaction!
- (3) Interactive Paper! 

(January 2011) We extended the speech dialogue interaction scenario by a new input modality: interactive paper. In order to improve reporting practices, we allow the radiologist to write structured
reports with a special pen on normal paper. "Interactive Paper for Radiology Findings" was presented
at IUI 2011 (the premier conference for reporting on the study of user interfaces with intelligent devices). The following picture shows (Left) Daniel Sonntag, DFKI; (Right) Daniel Bobrow, PARC at
the IUI demo event. (Flyer)    

IUI 2011 Interactive Paper Demo    


(September 2010) After the last scenario update of innovative radiology applications (cf. the
IUI 2010 demo), we plan to extend the desktop-based dialogue interaction to mobile speech interaction with medical image contents. The THESEUS CTC-WP4 dialogue shell should provide architecture to include mobile speech-based interfaces into the complex medical finding workflow in order to implement “RadSpeech on Mobile Devices.”
RadSpeech on Mobile Devices
Dr. Daniel Sonntag at the THESEUS Innovation Center Berlin with the Radspeech
and THESEUS-Medico prototypes.


Some introductory publications

- RadSpeech, a mobile dialogue system for radiologists, IUI 2012
- Theseus Usecase MEDICO (in German, pp. 97), acatech 2011
- Mobile Facetted Browsing LODD Applications for Supporting Clinicians, OM 2011
- A Discourse and Dialogue Infrastructure for Industrial Dissemination, IWSDS 2010
- Design and Implementation of a Semantic Dialogue System for Radiologists, Nova Science 2010

© Daniel Sonntag, DFKI