Project Description

Large Analytics Library and Scalable Concept Ontology for Multimedia Research Our goal is to make available standard data sets, organized into concepts as well as tools for analyzing video and building applications. The starting point is the concept list from LSCOM, the Large-Scale Concept Ontology for Multimedia. This the expanded version of the LSCOM concepts list has been used as the seed to collect video from youtube based on these concept names as queries. The results are described on this web site, in a variety of different views.

The video analysis community has long attempted to bridge the gap from low-level feature extraction to semantic understanding and retrieval. One important barrier impeding progress is the lack of infrastructure needed to construct useful semantic concept ontologies, building modules to extract features from the video, interpreting the semantics of what the video contains, and evaluating the tasks against benchmark truth data. To solve this fundamental problem, this project will create a shared community resource around large video collections, extracted features, video segmentation tools, scalable semantic concept lexicons with annotations, ontologies relating the concepts to each other, tools for annotation, learned models and complete software modules for automatically describing the video through concepts, and finally a benchmark set of user queries for video retrieval evaluation. The resource will allow researchers to build their own image/video classifiers, test new low-level features, expand the concept ontology, and explore higher level search services, etc., without having to redevelop several person year's worth of infrastructure. Using this tool suite and reference implementation, researchers can quickly customize concept ontologies and classifiers for diverse subdomains.

The contribution of the proposed work lies in the development of a large number of critical research resources for digital video analysis and searching. The modular architecture of the proposed resources provides great flexibility in adding new ontologies and testing new analytics components developed by other researchers in different domains. The use of large diverse standardized video datasets and well-defined benchmark procedures ensures a rigorous process to assess scientific progress. The results will facilitate rapid exploration of new ideas and solutions, contributing to advancements of major societal interest, such as next-generation media search and security.


Work shown on this web site is supported in part by NSF Grant No. 0751185

Contact Alex Hauptmann for more information.

Designed by Anlu Wang, copyright 2007