Conditional GANs for Image Captioning with Sentiments

Publication

Tushar Karayil, Federico Raue, Jörn Hees, Dengel

In: 2019 International Conference on Artificial Neural Networks. International Conference on Artificial Neural Networks (ICANN-2019) 28th International Conference on Artificial Neural Networks. September 17-19 Munich Germany Springer 2019.

Abstract

The area of automatic image captioning has witnessed much progress recently. However, generating captions with sentiment, which is a common dimension in human generated captions, still remains a challenge. This work presents a generative approach that combines sentiment (positive/negative) and variation for caption generation. The presented approach consists of a Generative Adversarial Network which takes as input, an image and a binary vector indicating the sentiment of the caption to be generated. We evaluate our model quantitatively on the state-of-the-art image caption dataset and qualitatively using a crowd-sourcing platform. Our results, along with human evaluation prove that we competitively succeed in the task of creating variations and sentiment in image captions.

Projekte

DeFuseNN - Deep Fusion für Neuronale Netze

Conditional GANs for Image Captioning with Sentiments

Abstract

Projekte

Follow us on:

Editor

Newsletter

Locations