Dynamic Vocabularies for Web-based Concept Detection by Trend Discovery

Damian Borth; Adrian Ulges; Thomas Breuel

In: Proceedings of the International Conference on Multimedia. ACM International Conference on Multimedia (ACM MM-2012), October 29 - November 2, Nara, Japan, ACM, 10/2012.


We present a novel approach towards automatic vocabulary selection for video concept detection. Our key idea is to expand concept vocabularies with trending topics that we mine automatically on other media like Wikipedia or Twitter. We evaluate several strategies for extending concept detection to auto-detect these topics in new videos, either by linking them to a static concept vocabulary, by a visual learning of trends on the fly, or by an expansion of the vocabulary. Our study on 6,800 YouTube clips and the top 23 target trends (covering a timespan of 6 months) demonstrates that a direct visual classification of trends (by a “live” learning on trend videos) outperforms an inference from static vocabu- laries. However, further improvements can be achieved by a combination of both approaches.

msp076-borth.pdf (pdf, 1 MB )

Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence