The Focus-Aspect-Value Model for Explainable Prediction of Subjective Visual Interpretation

Tushar Karayil, Philipp Blandfort, Jörn Hees, Andreas Dengel

In: Proceedings of the 2019 on International Conference on Multimedia Retrieval. ACM International Conference on Multimedia Retrieval (ICMR-2019) June 10-13 Ottawa Ontario Canada Seiten 16-24 ISBN 978-1-4503-6765-3 ACM 2019.


Subjective visual interpretation is a challenging yet important topic in computer vision. Many approaches reduce this problem to the prediction of adjective-or attribute-labels from images. However, most of these do not take attribute semantics into account, or only process the image in a holistic manner. Furthermore, there is alack of relevant datasets with fine-grained subjective labels. In this paper, we propose the Focus-Aspect-Value (FAV) model to structure the process of capturing subjectivity in image processing, and introduce a novel dataset following this way of modeling. We run experiments on this dataset to compare several deep learning methods and find that incorporating context information based on tensor multiplication outperforms the default way of information fusion (concatenation).


Weitere Links

Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence