Explainable AI and Neural Networks

Explainable AI and Neural Networks

Despite astonishing progress in the field of Machine Learning (ML), the robustness of high-performance models, especially the ones based on Deep Learning technologies, has been lower than initially predicted. These networks do not generalize as expected, remaining vulnerable to small adversarial perturbations (also known as adversarial attacks). Such shortcomings pose a critical obstacle to implement Deep Learning models for safety-critical scenarios such as autonomous driving, medical imaging, and credit rating.

Moreover, the gap between good performance and robustness also demonstrates the severe lack of explainability for modern AI approaches: Despite good performance, even experts cannot reliably explain model predictions.

Hence, the goals of this project are threefold:

  • Investigate methods of explainability and interpretability for existing AI approaches (focusing on Deep Neural Networks).
  • Develop novel architectures and training schemes that are more interpretable by design.
  • Analyze the trade-offs between explainability, robustness, and performance.


Bundesministerium für Bildung und Forschung


Bundesministerium für Bildung und Forschung

Share project:

Contact Person
Sebastian Palacio, M.Sc.


Publications about the project

Stanislav Frolov, Shailza Jolly, Jörn Hees, Andreas Dengel

In: Proceedings of the Second Workshop on Beyond Vision and LANguage: inTEgrating Real-world kNowledge (LANTERN). International Conference on Computational Linguistics (COLING-2020) 28th COLING December 13 Online-Conference Association for Computational Linguistics Barcelona, Spain 12/2020.

To the publication
Adriano Lucieri, Muhammad Naseer Bajwa, Andreas Dengel, Sheraz Ahmed

In: Proceedings of the 27th International Conference on Neural Information Processing (ICONIP2020). International Conference on Neural Information Processing (ICONIP-2020) November 18-22 Bangkok Thailand LNCS Springer 11/2020.

To the publication

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz