TerrorCat: a Translation Error Categorization-based MT Quality Metric

Mark Fishel, Rico Sennrich, Maja Popovic, Ondrej Bojar

In: Proceedings of the Seventh Workshop on Statistical Machine Translation. Workshop on Statistical Machine Translation (WMT-12) located at NAACL June 7-8 Montreal QC Canada Pages 64-70 Association for Computational Linguistics 6/2012.


We present TerrorCat, a submission to the WMT 12 metric sharedtask. TerrorCat uses frequencies of automatically obtained translation error categories as base for pairwise comparison of translation hypotheses, whichis in turn used to generate a score for every translation. The metric shows high overall correlation with human judgements on the system level and more modest results on the level of individual sentences.


terrorcat.pdf (pdf, 126 KB )

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz