Dynamic Retrieval, Analysis & Semantic Metadata Management

Automatic indexing and retrieval of digital data poses major challenges. The main problem arises from the ever increasing mass of digital media and the lack of efficient methods for indexing and retrieval of such data based on their semantic content rather than keywords. The Dynamic REtrieval Analysis and semantic metadata Management (DREAM) project aims at paving the way towards semi-automatic acquisition of knowledge from visual content to support intelligent indexing and retrieval of digital media.  This project was undertaken in collaboration with Partners from the UK Film Industry, including Double Negative, The Foundry and FilmLight. Double Negative was the test domain Partner who provided the test materials and user requirements and evaluated the system prototype.

One of the main challenges for the users in the film (post) production sector is the storage and management of huge repositories of multimedia data, in particular, video files, and, having to search through distributed repositories to find a particular video shot.

The DREAM project aims at addressing these challenges by proposing a knowledge-assisted intelligent visual information indexing and retrieval system. The main challenge in this research work was to architect an indexing, retrieval and query support framework. The proposed framework exploits content, context and search-purpose knowledge as well as any other domain related knowledge in order to ensure robust and efficient semantic-based multimedia object labelling, indexing and retrieval.  The framework is underpinned by a network of scalable ontologies, which grows alongside ongoing incremental annotation of video content.  The DREAM Demonstrator has been evaluated through real-life deployment in the film post-production phase to support the process of storage, indexing and retrieval of large data sets of special effects video clips as an exemplar application domain. The performance and usability evaluation results in this film post-production domain proves that the DREAM framework helps in resolving existing indexing and retrieval problems of video clips.

Project Details

Publications

• Badii, A., Lallah, C., Kolomiyets, O, Zhu, M. & Crouch, M. 2008, “KAIFIA: Knowledge Assisted Intelligent Framework for Information Access.” Scaling Topic Maps, Lecture Notes in Computer Science, vol. 4999/2008, p. 226–236.
• Badii, A., Lallah, C., Kolomiyets, O, Zhu, M. & Crouch, M. 2008, “Semi-automatic annotation and retrieval of visual content using the topic map technology,” in 1st WSEAS International Conference on Visualization, Imaging and Simulation (VIS08), Bucharest, Romania, p. 77–82.
• Badii, A., Lallah, C., Zhu, M., Crouch, M. 2009, “The DREAM Framework: Using a Network of Scalable Ontologies for Intelligent Indexing and Retrieval of Visual Content,” in Web Intelligence and Intelligent Agent Technologies, 2009. (WI-IAT09), IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology, Milano, Italy, pp. 551-554.
• Badii, A., Lallah, C., Zhu, M., Crouch, M. 2009, “Semi-automatic knowledge extraction, representation and context-sensitive intelligent retrieval of video content using collateral context modelling with scalable ontological networks.” Signal Processing: Image Communication, vol. 24, no. 9, pp. 759-773.
• Badii, A., Meng, Z, Lallah, C. & Crouch, M. 2009, “Semantic-driven context-aware visual information indexing and retrieval: Applied in the film post-production domain,” in Computational Intelligence for Visual Intelligence, 2009 (CIVI09), IEEE Workshop on Computational Intelligence for Visual Intelligence, Nashville, TN, US, pp. 44-51.
• Badii, A., Lallah, C., Zhu, M. & Crouch, M. 2011, “Using a Network of Scalable Ontologies for Intelligent Indexing and Retrieval of Visual Content.” Information Retrieval and Mining in Distributed Environments, Studies in Computational Intelligence, vol. 324/2011, p. 233–248.