A new metric for the evaluation of dialog act classification.

Stephan Lesch; Thomas Kleinbauer; Jan Alexandersson

In: In Proceedings of the Nineth Workshop on the semantics and pragmatics of dialogue. Workshop on the Semantics and Pragmatics of Dialogue (EDILOG), Nancy, 6/2005.


The standard evaluation metrics for dialog act classifiers are based on the boolean outcome of the exact classification. For multidimensional tag sets, such as the ICSI-MRDA tag set, this is stricter than necessary, since the miss- classification might be partial and this can be good enough for the application in which the classifier is embedded. We propose a new forgiving metric and show some preliminary results. Some future work is sketched.


Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence