Publication
Spectral Occlusion - Attribution Beyond Spatial Relevance Heatmaps
Fabian Schmeisser; Adriano Lucieri; Andreas Dengel; Sheraz Ahmed
In: World Conference on Explainable Artificial Intelligence. xAI: World Conference on Explainable Artificial Intelligence (xAI-2025), July 9-11, Istanbul, Turkey, Springer Nature Switzerland, Cham, Switzerland, 2025.
Abstract
Real-world computer vision tasks in high-stakes domains like medicine often go beyond mere object localization. Accurate diagnoses often require the detection and combination of complex factors present in the input. Explanation is therefore particularly challenging, but also necessary due to the high stakes involved. Most existing explanation methods fall short in isolating distributed and overlapping features such as colors and textures. This paper introduces Spectral Occlusion (S-Occ), a method designed to address this limitation, providing multiple additional perspectives to the explanation of complex decisions. Beyond the conventional highlighting of spatial regions, S-Occ makes use of spectral manipulation to indicate dispersed image features such as colors and textures. Different visualizations offer an additional, nuanced insight into the model’s decision-making, resulting in a more holistic representation of the contributing factors. This can help to facilitate the explainability of complex systems. The method is evaluated quantitatively and qualitatively on real-world skin lesion analysis. S-Occ outperforms established methods by an average of 0.38 in explanation Sensitivity, demonstrating its ability to complement spatial attribution methods by facilitating the highlighting of non-trivial, decision-relevant factors. The method’s potential impact spans various high-stakes domains, with particular relevance in medical fields like dermatology and ophthalmology, where nuanced insights are imperative for trustworthy decision-making.