Publication
Hierarchical Modeling with Neurodynamical Agglomerative Analysis
Michael Marino; Georg Schröter; Gunther Heidemann; Joachim Hertzberg
In: Proceedings of the International Conference on Artificial Neural Networks. International Conference on Artificial Neural Networks (ICANN-2020), September 15-18, Pages 180-191, Springer, 10/2020.
Abstract
We propose a new analysis technique for neural networks, Neurodynamical Agglomerative Analysis (NAA), an analysis pipeline de- signed to compare class representations within a given neural network model. The proposed pipeline results in a hierarchy of class relationships implied by the network representation, i.e. a semantic hierarchy analo- gous to a human-made ontological view of the relevant classes. We use networks pretrained on the ImageNet benchmark dataset to infer seman- tic hierarchies and show the similarity to human-made semantic hierar- chies by comparing them with the WordNet ontology. Further, we show using MNIST training experiments that class relationships extracted us- ing NAA appear to be invariant to random weight initializations, tending toward equivalent class relationships across network initializations in suf- ficiently parameterized networks.