Immergensim: EU project CORTEX² brings XR into crisis operations
| Press release | Learning & Education | Virtual & Augmented Reality | Augmented Vision | Kaiserslautern
Mentor Maud remains calm. The AI-supported advisor looks the team firmly in the digital eyes. Outside, in the virtual crisis zone, the water is rising – closely followed by a landslide. The responders are new to the job – but they are prepared. Not through live field exercises, but through an immersive trial run that builds decision-making under pressure and ensures coordinated response when it counts. Inside the simulation, Mentor Maud watches over the team. She supports their choices, provides resources, answers questions, and helps build the instincts they’ll need when real emergencies happen.
What sounds like the future is already a reality: researchers at the German Research Center for Artificial Intelligence (DFKI) have created a flexible infrastructure under the CORTEX² project that enables humanitarian responders to conduct virtual crisis training. The immersive training system Immergensim was developed within the XRisis sub-project by XR Ireland in close collaboration with Action Contre la Faim (ACF), building directly on CORTEX²’s technological foundation.
From pandemic to platform
"The idea for CORTEX² was born at DFKI in the middle of the COVID-19 pandemic. While many work processes were successfully shifted to the home office, a serious shortcoming became apparent: there was a lack of suitable digital tools for tasks that required physical presence, visual collaboration or physical-gestural interaction."
Traditional video conferencing had reached its limits here. There was a lack of solutions that transmitted more than just images and sound – namely presence, interaction and real-time coordination in three-dimensional space. This is why DFKI joined forces with European partners to develop CORTEX²: a modular platform for cooperative extended reality that combines real-time communication, AI integration and immersive learning spaces.
Our practical application example demonstrates how technical support can also be realised intuitively and precisely from a distance:
CORTEX² - Infrastructure for immersive cohesion
At its core, CORTEX² offers an architecture that is designed for interoperability and customisability. This means that CORTEX² not only provides an application, but also a platform-capable infrastructure – open for adaptation, expansion and cross-sector use. This includes a real-time communication system, an avatarisation system for maintaining digital identity, virtual conversation partners based on AI, an automated evaluation tool for conversations and real-time modelling of real objects. An IoT server also enables physical devices to be connected to virtual scenarios.
© CORTEX²To do this, the researchers built on the Rainbow product from Alcatel Lucent Enterprise, a commercial tool for video conferencing comparable to applications such as Teams and Zoom. Various functions of the Rainbow core were then transferred into a functional framework for XR applications. This in turn provides a wide range of AI applications from a modular system of functions and tools in order to fulfil the individual needs of users:
- Rainbow CPaaS (Alcatel-Lucent): secure voice and video connections in XR rooms
- VCAA (DFKI): Video Compression Alternative Appearance - an avatarisation system for preserving presence and identity
- CoVA (CEA): Conversational agents - AI-based virtual dialogue partners
- Summarisation Agent (Linagora): Automatic call summaries for follow-up & analysis
- CORTEX IoT Server: Management and integration of IoT devices in extended reality (XR) applications
3D reconstruction: Real-time 3D modelling of real objects in virtual space
All these and other services are provided by CORTEX², taking into account data protection, scalability and technical resilience – and above all with a view to real needs.
Immergensim: training innovation in action
With XRisis, the CORTEX² consortium has created a project that specifically addresses the serious gaps in the training of humanitarian aid workers. Traditional approaches are often limited by high costs, restricted accessibility, and a lack of realism – especially for local actors with fewer resources. To overcome these bottlenecks, XR Ireland and Action Contre la Faim developed Immergensim: an immersive simulation platform featuring low-threshold, interactive training formats that run on desktop computers or VR headsets, recreating real-life emergencies in a virtual environment.
© XR IrelandXR on humanitarian missions
Immergensim’s training concept is structured into three tightly interlinked modules: an individualized introduction guided by the AI mentor avatar “Maud”; a scenario-based team decision-making simulation in a virtual crisis coordination center; and, finally, the implementation of emergency measures in simulated field conditions with real-time AI support. These modular training units – from orientation to shared situational understanding and tactical execution – are designed to flexibly adapt to diverse humanitarian missions. Interactive “injects” and dynamic decision paths make the experience practical, realistic, and measurable.
The system is powered by key CORTEX² technologies. Integrating the Rainbow communication service into the Unity environment enables voice calls, bubble meetings, and collaboration between physical and virtual participants in a shared XR space. The hardware-intensive VCAA component was replaced with a lightweight web-based avatar emulator built using MediaPipe, WebRTC, and React – allowing it to run smoothly even on modest humanitarian hardware. In addition, the system integrates CoVA and the Summarisation Agent to structure and analyze feedback discussions, debriefings, and after-action reviews. This architecture makes Immergensim a robust simulation-based training solution – not only innovative, but operable even in resource-constrained settings such as NGOs and field offices.
Impact, awards and prospects
Field feedback confirms Immergensim’s strong potential: around half of the participants gave the highest possible rating. Participants particularly valued the soft skills training (avg. 4.2 out of 5) and collaboration in the virtual crisis coordination environment (avg. 3.6 out of 5). Even on laptops without a headset, the immersive experience proved effective. One Action Contre la Faim team member remarked, “The use of VR and AI avatars significantly improves the training value – even on a laptop.” The experience gained is now being used to refine and expand the platform, making it more versatile and applicable across new domains.
© XR IrelandExternal recognition underlines the project’s impact. In 2025, Immergensim was awarded the Unity for Humanity Grant, affirming both its technological and social relevance. With this support, the team is developing a low-code SimEx Builder to enable NGOs worldwide to design their own immersive training simulations – without internal development resources but with full access to CORTEX²’s capabilities.
The system’s potential extends far beyond humanitarian operations. What works in disaster relief can be adapted for other sectors: safety training in industry and utilities, triage drills in healthcare, story-based learning in education, and scenario planning for smart cities.
The project partners XR Ireland and Action Contre la Faim have identified three key takeaways:
"Truly effective innovation arises when technology partners not only know the reality of the user organisations but also shape it together with them. Success does not require the perfect solution from the outset, but rather bold prototypes that mature step by step. And finally, the best ideas often come from directions that were not initially considered."
Contact:
Dr.-Ing. Alain Pagani
Head of CORTEX² and Deputy Head of Augmented Reality, DFKI
- Alain.Pagani@dfki.de
- Phone: +49 631 20575 3530
Press contact:
Further information:
- XR Ireland
Homepage of XR Ireland
- Winner of Unity for humanity award 2025
One of 29 CORTEX² sub-projects wins: XRisis uses XR technology for humanitarian aid - and receives a Unity for Humanity Grant 2025.