Publication
Feedback on feedback: student’s perceptions for feedback from teachers and few-shot LLMs
Sylvio Rüdian; Julia Podelo; Jakub Ku¸ílek; Niels Pinkwart
In: Proceedings of the 15th International Learning Analytics and Knowledge Conference. International Conference on Learning Analytics & Knowledge (LAK), Pages 82-92, ISBN 979-8-4007-0701-8/25/03, 3/2025.
Abstract
Large language models (LLMs) can be a valuable resource for generating texts and performing various instruction-based tasks. In this paper, we explored the use of LLMs, particularly for generating feedback for students in higher education. More precisely, we conducted an experiment to examine students’ perceptions regarding LLM-generated feedback. This has the overall aim of assisting teachers in the feedback creation process. First, we examine the different student perceptions regarding the feedback that students got without being aware of whether it was created by their teacher or an LLM. Our results reveal that the feedback source has not impacted how it was perceived by the students, except in cases where repetitive content has been generated, which is a known limitation of LLMs. Second, students have been asked to identify whether the
feedback comes from an LLM or the teacher. The results demonstrate, that students were unable to identify the feedback source. A small subset of indicators has been identified, that clearly revealed from whom the feedback comes from. Third, student perceptions are analyzed while knowing that feedback has been auto-generated.
This examination indicates that generated feedback is likely to be met with resistance. It contradicts the findings of the first examination. This emphasizes the need of a teacher-in-the-loop approach when employing auto-generated feedback in higher education.