Recent trends in recognizing textual entailment


Computational Lingustics
Advanced Bachelor Programme / Master Programme
Winter Semester 2012



Speakers are assigned as well as some reviewers. Some topics still need reviewers. 

IMPORTANT: Registration deadline for the seminar is 21th January 2013. You then have to make a decision concerning the credit points 4 (no written report) or 7 (written report). It will not be possible to change later, so be careful to make sure to submit a written report if you decided to go for 7 CP!


General Information

Moderator: Günter Neumann, Rui Wang


Recognizing Textual Entailment (RTE) is a concrete task based on a relationship between two plain texts, Text (T) and Hypothesis (H). If the meaning of H can be inferred from the meaning of T, we say that T entails H. This task can be viewed as a binary classification task or as a probabilistic function mapping the pair T-H to a value between zero (not entailed) and one (fully entailed). Generally, the application of RTE falls into one of the following two categories: building a semantic model with the ability to perform inferences or improving the current NLP applications. From a linguistic perspective, several possible types of textual entailment exist: syntactic entailment, like “I’m eating at home” entails “I’m eating”; semantic entailment, like “He loves her” entails “He likes her”; implicature, like “He beat Mary” entails “Mary is injured”.
In this seminar will discuss recent research proposal and methods covering a broad spectrum ranging from corpus construction perspective, to linguistics and feature extraction perspective, and to Knowledge base and rule-based perspective.

Seminar Language: English

Available Certificate Modalities:

Placement in Study Programme:



Session Number

Date (D.M.Y)






Organizational meeting





Rui Wang



Manual analysis

Danielle Robinson

Mariona Coll Ardanuy



Corpus construction

Andreas Farner

Evi Kiagia/Susanne Fertmann



Lexical methods

Mariona Coll Ardanuy

David Przybilla/Susanne Fertmann



Syntactic methods

David Przybella

Noushin Fadaei



Semantic methods

Susanne Fertmann

Nicholas Whittier



Logic methods

Yudong Zhou

Andreas Farner



Natural logic

Nicholas Whittier

Noushin Fadaei



DIRT rules

Evi Kiagia

Tobias Backes



Entailment rules

Tobias Backes

Noushin Fadaei/Alexander Chernov



Directionality and Entailment graph

Noushin Fadaei

Andreas Farner



RTE and QA

Alexander Chernov

Moinuddin Mushirul Haque/Yudong Zhou




Moinuddin Mushirul Haque

Danielle Robinson

Please click on the session number to jump to the corresponding references. If available, the topics of the presentations will be linked to the slides of the presentations.




Session 2: General References

Dagan, I., Dolan, B., Magnini, B., & Roth, D. 2009. Recognizing textual entailment: Rational, evaluation and approaches. Natural Language Engineering, 15(4), i–xvii. Editorial of the special issue on Textual Entailment.

I. Androutsopoulos and P. Malakasiotis. 2010. A survey of paraphrasing and textual entailment methods. Journal of Artificial Intelligence Research, 38:135–187.


Session 3: Manual Analysis

Konstantina Garoufi. 2007. Towards a better understanding of applied textual entailment: Annotation and evaluation of the RTE-2 dataset. Master’s thesis, Saarland University.

P. Clark, W. R. Murray, J. Thompson, P. Harrison, J. Hobbs, C. Fellbaum. 2007. On the Role of Lexical and World Knowledge in RTE3. In Proceedings of the ACL-PASCAL Workshop of Textual Entailment and Paraphrasing.

Shachar Mirkin, Ido Dagan, and Sebastian Padó. 2010. Assessing the role of discourse references in entailment inference. In Proceedings of ACL 2010, Uppsala, Sweden.

Mark Sammons, Vinod Vydiswaran, and Dan Roth. 2010. Ask not what textual entailment can do for you... In Proceedings of ACL 2010, Uppsala, Sweden.


Session 4: Corpus Construction

John Burger and Lisa Ferro. 2005. Generating an entailment corpus from news headlines. In Proceedings of the ACL Workshop on Empirical Modeling of Semantic Equivalence and Entailment, pages 49–54, Ann Arbor, Michigan, USA.

Rui Wang and Caroline Sporleder. 2010. Constructing a textual semantic relation corpus using a discourse treebank. In Proceedings of the seventh international conference on Language Resources and Evaluation (LREC), Valletta, Malta.

Matteo Negri and Yashar Mehdad. 2010. Creating a Bi-lingual Entailment Corpus through Translations with Mechanical Turk: $100 for a 10-day Rush. In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk, pages 212–216. Los Angeles, California.


Session 5: Lexical Methods

Timothy Chklovski and Patrick Pantel. 2004. Verbocean: Mining the web for fine-grained semantic verb relations. In Proceedings of EMNLP, Barcelona, Spain.

O. Glickman, I. Dagan and M. Koppel. 2005. A Probabilistic Classification Approach for Lexical Textual Entailment, Twentieth National Conference on Artificial Intelligence (AAAI-05).

Rowan Nairn, Cleo Condoravdi, and Lauri Karttunen. 2006. Computing relative polarity for textual inference. In Proceedings of ICoS-5 (Inference in Computational Semantics), Buxton, UK, 2006.

Y. Mehdad and B. Magnini. A word overlap baseline for the recognizing textual entailment task. Online, 2009.

Shachar Mirkin, Ido Dagan, and Eyal Shnarch. 2009. Evaluating the inferential utility of lexical-semantic resources. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics (EACL-09), pages 558–566, Athens, Greece.


Session 6: Syntactic Methods

F. M. Zanzotto and A. Moschitti. 2006. Automatic learning of textual entailments with cross-pair similarities. COLING-ACL 2006.

Rui Wang and Guenter Neumann. 2007. Recognizing textual entailment using a subsequence kernel method. In Proceedings of AAAI, pages 937–942.

Zanzotto, F. M., Pennacchiotti, M., and Moschitti, A. 2009. A machine-learning approach to textual entailment recognition. Natural Language Engineering, 15(4), 551–582.


Session 7: Semantic Methods

Aljoscha Burchardt, Nils Reiter, Stefan Thater, and Anette Frank. 2007. A semantic approach to textual entailment: System evaluation and task analysis. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, Prague, Czech Republic.

Rui Wang and Yi Zhang. 2009. Recognizing textual relatedness with predicate-argument structures. In Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP 2009), Singapore, Singapore.

Yashar Mehdad, Alessandro Moschitti, and Fabio Massiomo Zanzotto. 2010. Syntactic/Semantic Structures for Textual Entailment Recognition. In Proceedings of Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 1020–1028, Los Angeles, California, June 2010.


Session 8:  Logic Methods

M. Tatu and D. Moldovan. 2005. A Semantic Approach to Recognizing Textual Entailment. HLT-EMNLP 2005.

M. Tatu and D. Moldovan. 2006. A Logic-based Semantic Approach to Recognizing Textual Entailment. COLING-ACL 2006.

J. Bos, K. Markert. 2005. Recognising Textual Entailment with Logical Inference. Proceedings of the 2005 Conference on Empirical Methods in Natural Language Processing (EMNLP 2005), pp. 628–635.

J. Bos, K. Markert. 2006. Recognising textual entailment with robust logical inference. In: Glickman, O. et al (ed):  Springer Lecture Note Series.


Session 9: Natural Logic

Bill MacCartney and Christopher D. Manning. 2007. Natural logic for textual inference. In Proceedings of the ACL Workshop on Textual Entailment and Paraphrasing, Prague.

MacCartney, B., Galley, M., & Manning, C. 2008. A phrase-based alignment model for natural language inference. In Proc. of the Conf. on EMNLP, pp. 802–811, Honolulu, Hawaii.

MacCartney, B., & Manning, C. 2009. An extended model of natural logic. In Proceedings of the 8th International Conference on Computational Semantics, pp. 140–156, Tilburg, The Netherlands.


Session 10: DIRT Rules

Lin, D., & Pantel, P. 2001. Discovery of inference rules for question answering. Natural Language Engineering, 7, 343–360.

Erwin Marsi, Emiel Krahmer, and Wauter Bosma. 2007. Dependency-based paraphrasing for recognizing textual entailment. In Proceedings of the Workshop on Textual Entailment and Paraphrasing, pages 83–88, Prague.

Georgiana Dinu and Rui Wang. 2009. Inference rules and their application to recognizing textual entailment. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics (EACL-09), pages 211–219, Athens, Greece.


Session 11: Entailment Rules

Idan Szpektor, Eyal Shnarch, and Ido Dagan. 2007. Instance-based evaluation of entailment rule acquisition. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 456–463, Prague, Czech Republic.

Idan Szpektor and Ido Dagan. 2008. Learning entailment rules for unary templates. In Proceedings of COLING 2008, pages 849–856, Manchester, UK.

Roni Ben Aharon, Idan Szpektor, and Ido Dagan. 2010. Generating entailment rules from FrameNet. In Proceedings of ACL 2010 Conference Short Papers, pages 241–246, Uppsala, Sweden.


Session 12: Directionality and Entailment Graph

R. Bhagat, P. Pantel, and E. Hovy. 2007. Ledir: An unsupervised algorithm for learning directionality of inference rules. In Proceedings of EMNLP-CoNLL.

Lili Kotlerman, Ido Dagan, Idan Szpektor, and Maayan Zhitomirsky-Geffet. 2009. Directional distributional similarity for lexical expansion. In Proceedings of ACL.

Jonathan Berant, Ido Dagan, and Jacob Goldberger. 2010. Global learning of focused entailment graphs. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1220–1229, Uppsala, Sweden.


Session 13:  RTE and Related Tasks

S. Harabagiu and A. Hickl. 2006. Methods for Using Textual Entailment in Open-Domain Question Answering. COLING-ACL 2006.

Marie-Catherine de Marneffe, Anna N. Rafferty, and Christopher D. Manning. 2008. Finding contradictions in text. In Proceedings of ACL-08: HLT, pages 1039–1047, Columbus, Ohio.

Anselmo Peñas, Álvaro Rodrigo, and Felisa Verdejo. 2008. Overview of the Answer Validation Exercise 2007. In Advances in Multilingual and Multimodal Information Retrieval. Lecture Notes In Computer Science, Vol. 5152. pages 237-248.

Álvaro Rodrigo, Anselmo Peñas, and Felisa Verdejo. 2008. Overview of the answer validation exercise 2008. In Proceedings of the 9th Cross-language evaluation forum conference on Evaluating systems for multilingual and multimodal information access (CLEF'08). pages 296-313.

Sebastian Padó, Daniel Cer, Michel Galley, Daniel Jurafsky, and Christopher D. Manning. 2009. Measuring machine translation quality as semantic equivalence: A metric based on entailment features. Machine Translation, 23(2–3):181–193.

Michael Heilman and Noah A. Smith. 2010. Tree edit models for recognizing textual entailments, paraphrases, and answers to questions. In Proceedings of NAACL-HLT, pages 1011–1019.

Óscar Ferrández, Christian Spurk, Milen Kouylekov, Iustin Dornescu, Sergio Ferrández, Matteo Negri, Rubén Izquierdo, David Tomás, Constantin Orasan, Günter Neumann, Bernardo Magnini and Jose Luis Vicedo. 2011. The QALL-ME Framework: A specifiable-domain multilingual Question Answering architecture. Journal of Web Semantics: Science, Services and Agents on the World Wide Web.

Deniz Yuret, Aydın Han, and Zehra Turgut. 2012. SemEval-2010 Task 12: Parser Evaluation using Textual Entailments. Proceedings of the 5th International Workshop on Semantic Evaluation, pages 51–56, Uppsala, Sweden.



Written Report

Students enrolled in the Master's programme can choose to submit a written report (see available certificate modalities). Submission of a report for B.Sc. students is mandatory. The length of the written report is restricted to eight pages, disregarding bibliographical sources. For this purpose, the linked conference-style template should be used (available for Latex and MS Word). The submission deadline is April the 1st, 2013, 23:59. The written report should have the the style of conference proceedings. We expect you to digest the material related to your topic and perform further research. In your report, you should add value to the available information by comparing, criticizing, and highlighting plus points. We want to encourage you to think and develop your own opinion, and will disapprove of copy-pasting. If you have questions on the written report, we will be happy to help you.

You can turn in your report in electronic or print form. Electronic copies should be submitted via e-mail to the following addresses: and