A deeper dive into the inner workings of LLMs
The Foundations of Systemic AI research department focuses on how to design AI systems that are powerful, reliable, comprehensible and secure. At NeuRIPS 2025, the team of Professor Kristian Kersting and Dr Patrick Schramowski will present various topics from their ongoing research, including sparse autoencoders. The researchers have developed methods for measuring and enhancing mechanistic interpretability, i.e. reverse engineering AI models, as well as controlling large language models (LLMs). These methods aim to make the internal processes of LLMs easier to understand and influence.
AI systems that learn through interactions
Researchers from the Department of Interactive Machine Learning, led by Professor Daniel Sonntag, will present their latest research at this year's renowned AI conference. Their work centres on developing fundamental methods for intelligent algorithms and user interfaces that facilitate machine learning through direct human interaction, thereby making the training of AI systems easier. At NeurIPS, the researchers will present, amongst other things, ExgraMed, a method for the optimised use of vision-language models through extended context graph alignment.
LD3M: an innovative framework for dataset distillation with latent generative priors
Researchers from Prof. Andreas Dengel's ‘Smart Data & Knowledge Services’ research group are also attending NeurIPS. The group's focus is on extracting useful information from large and diverse data sets. To this end, they are developing new methods that are both model-based and data-driven. These methods are designed to facilitate better decision-making, provide new insights, analyse behaviour, identify inconsistencies, make predictions, and optimise processes. At NeurIPS, the researchers will present Latent Dataset Distillation with Diffusion Models (LD3M): an innovative framework for dataset distillation with latent generative priors, which enables improved gradient flow from diffusion models during the distillation process.
Further information on DFKI's participation in NeurIPS 2025:
- Paper: xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memories; Maurice Kraus, Felix Divo, Devendra Singh Dhami, Kristian Kersting
- Paper: Object-Centric Concept-Bottlenecks; David Steinmann, Wolfgang Stammer, Antonia Wüst, Kristian Kersting
- Paper: Measuring and Guiding Monosemanticity; Ruben Härle, Felix Friedrich, Manuel Brack, Björn Deiseroth, Stephan Waeldchen, Patrick Schramowski, Kristian Kersting
- Paper: When Causal Dynamics Matter: Adapting Causal Strategies through Meta-Aware Interventions; Moritz Willig, Tim Woydt, Devendra Singh Dhami, Kristian Kersting
- Paper: EmoNet-Face: An Expert-Annotated Benchmark for Synthetic Emotion Recognition; Christoph Schuhmann, Robert Kaczmarczyk, Gollam Rabby, Maurice Kraus, Felix Friedrich, Huu Nguyen, Kalyan Sai Krishna, Kourosh Nadi, Kristian Kersting, Sören Auer
- Paper: Exploring Neural Granger Causality with xLSTMs: Unveiling Temporal Dependencies in Complex Data; Harsh Poonia, Felix Divo, Kristian Kersting, Devendra Singh Dhami
- Paper: How Many Tokens Do 3D Point Cloud Transformer Architectures Really Need?; Tuan Tran Anh, Duy M. H. Nguyen, Hoai-Chau Tran, Michael Barz, Khoa D Doan, Roger Wattenhofer, Vien Ngo, Mathias Niepert, Daniel Sonntag, Paul Swoboda
- Paper: EXGRA-MED: Extended Context Graph Alignment for Medical Vision-Language Models; Duy M. H. Nguyen, Nghiem Diep, Trung Nguyen, Hoang-Bao Le, Tai Nguyen, Anh-Tien Nguyen, TrungTin Nguyen, Nhat Ho, Pengtao Xie, Roger Wattenhofer, Daniel Sonntag, James Zou, Mathias Niepert
- Paper: Mitigating Reward Over-optimization in Direct Alignment Algorithms with Importance Sampling; Nguyen Phuc, Ngoc-Hieu Nguyen, Duy M. H. Nguyen, Anji Liu, An Mai, Thanh Binh Nguyen, Daniel Sonntag, Khoa D Doan
- Workshop Paper: AuditCopilot: Leveraging LLMs for Fraud Detection in Double-Entry Bookkeeping; Md Abdul Kadir, Sai Suresh Macharla Vasu, Sidharth S. Nair, Daniel Sonntag
- Paper: Unlocking Dataset Distillation with Diffusion Models; Brian Bernhard Moser, Federico Raue, Sebastian Palacio, Stanislav Frolov, Andreas Dengel