Publication
Class error rates for evaluation of machine translation output
Maja Popovic
In: Proceedings of the Seventh Workshop on Statistical Machine Translation. Workshop on Statistical Machine Translation (WMT-12), 7th, located at NAACL, June 7-8, Montreal, QC, Canada, Pages 71-75, Association for Computational Linguistics, 6/2012.
Abstract
We investigate the use of error classification results for
automatic evaluation of machine translation output. Five basic
error classes are taken into account: morphological errors,
syntactic (reordering) errors, missing words, extra words and
lexical errors. In addition, linear combinations of these
categories are investigated. Correlations between the class error
rates and human judgments are calculated on the data of the third,
fourth, fifth and sixth shared tasks of the Statistical Machine
Translation Workshop. Machine translation outputs in five
different European languages are used: English, Spanish, French,
German and Czech. The results show that the following combinations
are the most promising: the sum of all class error rates, the
weighted sum optimised for translation into English and the
weighted sum optimised for translation from English.