DFKI-LT - TerrorCat: a Translation Error Categorization-based MT Quality Metric

Mark Fishel, Rico Sennrich, Maja Popovic, Ondrej Bojar
TerrorCat: a Translation Error Categorization-based MT Quality Metric
Proceedings of the Seventh Workshop on Statistical Machine Translation, Pages 64-70, Montreal, QC, Canada, Association for Computational Linguistics, 6/2012
 
We present TerrorCat, a submission to the WMT 12 metric sharedtask. TerrorCat uses frequencies of automatically obtained translation error categories as base for pairwise comparison of translation hypotheses, whichis in turn used to generate a score for every translation. The metric shows high overall correlation with human judgements on the system level and more modest results on the level of individual sentences.
 
Files: BibTeX, terrorcat.pdf