Publication

A new metric for the evaluation of dialog act classification.

Stephan Lesch, Thomas Kleinbauer, Jan Alexandersson

In: In Proceedings of the Nineth Workshop on the semantics and pragmatics of dialogue. Workshop on the Semantics and Pragmatics of Dialogue (EDILOG) Nancy 6/2005.

Abstract

The standard evaluation metrics for dialog act classifiers are based on the boolean outcome of the exact classification. For multidimensional tag sets, such as the ICSI-MRDA tag set, this is stricter than necessary, since the miss- classification might be partial and this can be good enough for the application in which the classifier is embedded. We propose a new forgiving metric and show some preliminary results. Some future work is sketched.

Projekte

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz