Publication
Human and LLM-based Assessment of Teaching Acts in Expert-led Explanatory Dialogues
Aliki Anagnostopoulou; Nils Feldhus; Yi-Sheng Hsu; Milad Alshomary; Henning Wachsmuth; Daniel Sonntag
In: Michael Strube; Chloe Braud; Christian Hardmeier; Junyi Jessy Li; Sharid Loaiciga; Amir Zeldes; Chuyuan Li (Hrsg.). Proceedings of the 6th Workshop on Computational Approaches to Discourse, Context and Document-Level Inferences (CODI 2025). Workshop on Computational Approaches to Discourse, Context and Document-Level Inferences (CODI-2025), Suzhou, China, Pages 166-181, ISBN 979-8-89176-343-2, Association for Computational Linguistics, 2025.
Abstract
Understanding the strategies that make expert-led explanations effective is a core challenge in didactics and a key goal for explainable AI. To study this computationally, we introduce ReWIRED, a large corpus of explanatory dialogues annotated by education experts with fine-grained, span-level teaching acts across five levels of explainee knowledge. We use this resource to assess the capabilities of modern language models, finding that while few-shot LLMs struggle to label these acts, fine-tuning is a highly effective methodology. Moving beyond structural annotation, we propose and validate a suite of didactic quality metrics. We demonstrate that a prompt-based evaluation using an LLM as a ``judge'' is required to capture how the functional quality of an explanation aligns with the learner's expertise -- a nuance missed by simpler static metrics. Together, our dataset, modeling insights, and evaluation framework provide a comprehensive methodology to bridge pedagogical principles with computational discourse analysis.