Scene Labeling with LSTM Recurrent Neural Networks

Publikation

Wonmin Byeon; Thomas Breuel; Federico Raue; Marcus Liwicki

In: International Conference on Computer Vision and Pattern Recognition. International Conference on Computer Vision and Pattern Recognition (CVPR-2015), June 7-12, Boston, USA, IEEE, 2015.

Zusammenfassung

This paper addresses the problem of pixel-level segmentation and classification of scene images with an entirely learning-based approach using Long Short Term Memory (LSTM) recurrent neural networks, which are commonly used for sequence classification. We investigate two-dimensional (2D) LSTM networks for natural scene images taking into account the complex spatial dependencies of labels. Prior methods generally have required separate classification and image segmentation stages and/or pre- and post-processing. In our approach, classification, segmentation, and context integration are all carried out by 2D LSTM networks, allowing texture and spatial model parameters to be learned within a single model. The networks efficiently capture local and global contextual information over raw RGB values and adapt well for complex scene images. Our approach, which has a much lower computational complexity than prior methods, achieved state-of-the-art performance over the Stanford Background and the SIFT Flow datasets. In fact, if no pre- or post-processing is applied, LSTM networks outperform other state-of-the-art approaches. Hence, only with a single-core Central Processing Unit (CPU), the running time of our approach is equivalent or better than the compared state-of-the-art approaches which use a Graphics Processing Unit (GPU). Finally, our networks' ability to visualize feature maps from each layer supports the hypothesis that LSTM networks are overall suited for image processing tasks.

Weitere Links

https://ieeexplore.ieee.org/document/7298977/

Scene Labeling with LSTM Recurrent Neural Networks

Zusammenfassung

Weitere Links

Folgen Sie uns auf:

Herausgeber

Newsletter

Standorte