Towards a Multimodal Multisensory Cognitive Assessment Framework

Mira Niemann, Alexander Prange, Daniel Sonntag

In: Proceedings of the 30th IEEE International Symposium on Computer-Based Medical System. IEEE International Symposium on Computer-Based Medical Systems (CBMS-2018) 30th June 18-21 Karlstad Sweden IEEE 2018.


Traditionally, neurocognitive testing is done using pen and paper, which is both expensive and time consuming and often leads to a biased outcome. In this paper, we present an approach towards selecting and digitizing existing cognitive tests and supporting the assessment of cognitive impairments through automated evaluation of different input modalities recorded during the assessments. Our multimodal multisensory framework currently records and analyzes handwriting input captured using a digital pen and electrodermal activity captured by the BITalino sensor board. Using artificial intelligence methods, we aim at analyzing the multisensory data in order to support objective assessments of cognitive impairments. In this work, we describe the current state of our framework and outline future research objectives.

Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence