TerrorCat: a Translation Error Categorization-based MT Quality MetricMark Fishel; Rico Sennrich; Maja Popovic; Ondrej Bojar
In: Proceedings of the Seventh Workshop on Statistical Machine Translation. Workshop on Statistical Machine Translation (WMT-12), located at NAACL, June 7-8, Montreal, QC, Canada, Pages 64-70, Association for Computational Linguistics, 6/2012.
We present TerrorCat, a submission to the WMT 12 metric sharedtask. TerrorCat uses frequencies of automatically obtained translation error categories as base for pairwise comparison of translation hypotheses, whichis in turn used to generate a score for every translation. The metric shows high overall correlation with human judgements on the system level and more modest results on the level of individual sentences.