Disclosure of video material on the basis of sub-titles

In film and video productions budget-saving reuse of visual material plays an important role. Cataloguing and indexing of film and video recordings as a pre-condition for this is, however, an extremely time and money consuming process: it must be done manually and requires the expertise of specialised archivists. Systems which can understand and interpret moving images or that have the unrestricted ability to recognise and comprehend spoken language do not as yet exist.

Via one form of information it is already possible to gain access to the contents of a film with extant technology: namely by use of sub-titles. These are produced either during translation of foreign-language films or they are offered (in the original language) as a comprehension-aid for people with hearing difficulties. This connection of text and image sequences offers the key to the completely new approach to indexing of visual material which is being developed by Pop-Eye. This method combines recognition of sub-titles with techniques of language processing and partial translation. As a result a system is being created that can automatically extract complete scripts from the sub-titles of film or video sequences and then generate multi-lingual indices of the picture sequences - this will also allow access in various languages.

Funded by:European Union
Project Manager:Hans Uszkoreit (Hans.Uszkoreit@dfki.de)
Contact:Hans Uszkoreit (Hans.Uszkoreit@dfki.de)
Duration: 01.01.1997 - 31.12.1998
Partners:NetherlandsTROS, Hilversum (coordinator),
GermanySWF, Baden-Baden,
NetherlandsUniversity of Twente,
GermanyUniversität Tübingen,
NetherlandsVDA Media