The goal of the GeAR project is to implement augmented reality (AR) intelligent user interfaces for evaluating learning behaviour in educational settings. In GeAR, we focus on the the deployment and testing of AR technologies (e.g., acceptance, utility, and usability). We aim at developing multimodal-multisensor interfaces with eye tracking and digital pens as input, and AR-based multimodal presentation as output. GeAR (2019-2021) is a BMBF project based on interaction technology developed in Kognit and Interakt.

Research Questions

  • To identify interface design principles that most effectively support AR scenarios for education in schools (or in cases of mild cognitive impairment, see Kognit and Interakt) and automatic multimodal feedback in AR.
  • At the computational level, it will be important to investigate approaches to designing systems that recognize and model students’ cognitive and affective states.
  • At the interface level, it will be important to devise design principles that can inform the development of innovative multimodal-multisensor interfaces for a variety of student populations, education contexts, and learning environments.



Daniel Sonntag (DFKI, supervisor)
Michael Barz (DFKI, PhD student)
Alexander Prange (DFKI, PhD student)
Maximilian Biwersi (DFKI, data management and visualization)
Mira Niemann (DFKI, cognitive assessments)
Carolin Grieser (DFKI, digital pen analysis)
Divyam Saran (DFKI, multimodal human-computer interaction)