From Speech Recognition Towards Speech Understanding

Outline

Signal-Symbol-Signal Transformations in Spoken Dialog Systems

Three Levels of Language Processing

Challenges for Language Engineering

Telephone-based Dialog Translation

Context-Sensitive Speech-to-Speech Translation

Dialog Translation 1

Dialog Translation 2

Dialog Translation 3

Verbmobil II: Three Domains of Discourse

Data Collection with Mulitiple Input Devices

Extracting Statistical Properties from Large Corpora

Verbmobil Partner

The Control Panel of Verbmobil

From a Multi-Agent Architecture to a Multi-Blackboard Architecture

Multi-Blackboard/Multi-Agent Architecture

A Multi-Blackboard Architecture for the Combination of Results from Deep and Shallow Processing Modules

Integrating Shallow and Deep Analysis Components in a Multi-Blackboard Architecture

VHG: A Packed Chart Representation of Partial Semantic Representations

Robust Dialog Semantics: Deep Processing of Shallow Structures

Semantic Correction of Recognition Errors

Robust Dialog Semantics: Combining and Completing Partial Representations

The Understanding of Spontaneous Speech Repairs

Automatic Understanding and Correction of Speech Repairs in Spontaneous Telephone Dialogs

Integrating a Deep HPSG-based Analysis with Probabilistic Dialog Act Recognition for Semantic Transfer

The Dialog Act Hierarchy used for Planning, Prediction, Translation and Generation

Combining Statistical and Symbolic Processing for Dialog Processing

Statistical Dialog Act Recognition

Learning of Probabilistic Plan Operators from Annotated Corpora

Automatic Generation of Multilingual Protocols of Telephone Conversations

Automatic Generation of Minutes

Dialog Protocol

Spoken Clarification Dialogs between the User and the Verbmobil System

Competing Strategies for Robust Speech Translation

Architecture of the Semantic Transfer Module

Extensions of Discourse Representation Theory

Three English Translations of the German Word "Termin" Found in the Verbmobil Corpus

Entries in the Transfer Lexicon: German to English (Simplified)

Context-Sensitive Translation Exploiting a Discourse Model

The Use of Underspecified Representations

The Control Panel of Verbmobil

Integrating Deep and Shallow Processing: Combining Results from Concurrent Translation Threads

A Context-Free Approach to the Selection of the Best Translation Result

Learning the Normalizing Factors Alpha and Beta from an Annotated Corpus

Example of a Linear Inequation Used for Offline Learning

The Context-Sensitive Selection of the Best Translation

Verbmobil: Long-Term, Large-Scale Funding and Its Impact

SmartKom: Intuitive Multimodal Interaction

The Architecture of the SmartKom Agent (cf. Maybury/Wahlster 1998)

SmartKom: A Transportable and Transmutable Interface Agent

SmartKom-Public: A Multimodal Communication Booth

SmartKom-Mobile: A Handheld Communication Assistant

SmartKom-Home/Office: A Versatile Agent-based Interface

Speech-based Interaction with an Organizer on a WAP Phone (Voice In - WML out)

Conclusion

Conclusion

Conclusion