Robust Translation of Spontaneous Speech: A Multi-Engine Approach

Mobile Speech-to-Speech Translation of Spontaneous Dialogs

Mobile Speech-to-Speech Translation of Spontaneous Dialogs

Robust Realtime Translation with Verbmobil

Outline

Telephone-based Dialog Translation

Verbmobil: The First Speech-Only Dialog Translation System

Verbmobil is a Multilingual System

Verbmobil Partner

Three Levels of Language Processing

Challenges for Language Engineering

Verbmobil II: Three Domains of Discourse

Context-Sensitive Speech-to-Speech Translation

The Control Panel of Verbmobil

Verbmobil‘s Massive Data Collection Effort

Extracting Statistical Properties from Large Corpora

Multilinguality

Language Identification (LID)

From a Multi-Agent Architecture to a Multi-Blackboard Architecture

Multi-Blackboard/Multi-Engine Architecture

A Multi-Blackboard Architecture for the Combination of Results from Deep and Shallow Processing Modules

VIT (Verbmobil Interface Terms) as a Multi-Stratal Representation Language

VIT for ‘He is coming at the beginning of August‘

Information between Layers Linked Together Using Constant Symbols

The Use of Underspecified Representations

Verbmobil is the First Dialog Translation System that Uses Prosodic Information Systematically at All Processing Stages

Using Syntactic-Prosodic Boundaries to Speed-Up the Parsing Process

Integrating Shallow and Deep Analysis Components in a Multi-Engine Approach

Automatic Understanding and Correction of Speech Repairs in Spontaneous Telephone Dialogs

The Understanding of Spontaneous Speech Repairs

VHG: A Packed Chart Representation of Partial Semantic Representations

Semantic Correction of Recognition Errors

Robust Dialog Semantics: Deep Processing of Shallow Structures

Robust Dialog Semantics: Combining and Completing Partial Representations

Competing Strategies for Robust Speech Translation

Architecture of the Semantic Transfer Module

Lexical Disambiguation On-Demand

Three English Translations of the German Word “Termin” Found in the Verbmobil Corpus

Entries in the Transfer Lexicon: German ? English (Simplified)

Using Context and World Knowledge for Semantic Transfer

Integrating Deep and Shallow Processing: Combining Results from Concurrent Translation Threads

A Context-Free Approach to the Selection of the Best Translation Result

Learning the Normalizing Factors Alpha and Beta from an Annotated Corpus

Integrating a Deep HPSG-based Analysis with Probabilistic Dialog Act Recognition for Semantic Transfer

The Dialog Act Hierarchy used for Planning, Prediction, Translation and Generation

Learning of Probabilistic Plan Operators from Annotated Corpora

Automatic Generation of Multilingual Summaries of Telephone Conversations

Dialog Summary

Microplanning: Create Syntactic Building Blocks

Speeding Up the Language Generation Process by the Compilation of the HPSG Grammar to an LTAG Generation Grammar

Corpus-based Speech Synthesis

Verbmobil: Long-Term, Large-Scale Funding and Its Impact

Verbmobil: Long-Term, Large-Scale Funding and Its Impact

Distribution of Sentence Length in Large-Scale Evaluation

Evaluation Results

Results of End-to-End Evaluation Based on Dialog Task Completion for 31 Trials

Checklist for Final Verbmobil System

Results of the Verbmobil Project have been used in 20 Spin-Off Products by the Industrial Partners DaimlerChrysler, Philips and Siemens

Linguatronic : Spoken Dialogs with a Mercedes-Benz

Spoken Dialogs about Schedules

Successful Technology Transfer: 8 High-Tec Spin-Off Companies in the Area of Language Technology have been founded by Verbmobil Researchers

Verbmobil was the Key Resource for the Education and Training of Researchers and Engineers Needed to Build Up Language Industry in Germany

From Spoken Dialog to Multimodal Dialog

Merging Various User Interface Paradigms

SmartKom: Intuitive Multimodal Interaction

SmartKom-Mobile: A Handheld Communication Assistant

SmartKom: Multimodal Dialogs with a Life-like Character

Verbmobil is a Very Large Dialog System

Lessons Learned from Verbmobil

Conclusions and Take-Home Messages

Conclusions and Take-Home Messages

Conclusions and Take-Home Messages

Conclusions and Take-Home Messages

Conclusions and Take-Home Messages

Open Problems:

Further Reading

The Verbmobil Book