Proceedings-Artikel
A new metric for the evaluation of dialog act classification
Stephan Lesch; Thomas Kleinbauer; Jan Alexandersson
In: In Proceedings of the Nineth Workshop on the semantics and pragmatics of dialogue. Nineth Workshop on the semantics and pragmatics of dialogue (DIALOR), Nancy, n/a, 6/2005.
Abstract
The standard evaluation metrics for dialog act classifiers are based on the boolean outcome of the exact classification. For multidimensional tag sets, such as the ICSI-MRDA tag set, this is stricter than necessary, since the miss- classification might be partial and this can be good enough for the application in which the classifier is embedded. We propose a new forgiving metric and show some preliminary results. Some future work is sketched.