INTERNATIONAL ORGANISATION FOR STANDARDISATION
ORGANISATION INTERNATIONALE DE NORMALISATION
ISO/IEC JTC1/SC29/WG11
CODING OF MOVING PICTURES AND AUDIO
ISO/IEC JTC1/SC29/WG11
MPEG2001/N4528
Pattaya, TH
3 7 December 2001
|
Title: |
MPEG-7 Multimedia Description Schemes - Frequently Asked Questions (FAQ) ver. 4.0 |
|
Source: |
MPEG Multimedia Description Schemes (MDS) Group |
|
Authors: |
|
|
Status: |
Approved |
This document provides answers to Frequently Asked Questions (FAQs) related to MPEG Multimedia Description Schemes. It is subject to continuous update and improvements. Any comments or suggestions should be forwarded to the MDS reflector (mpeg-7-ds@advent.ee.columbia.edu ) or directly to the FAQ editor (jrsmith@watson.ibm.com)
The following questions and answers were developed by the Multimedia Description Scheme (MDS) Group. The revision history is given as follows:
· MPEG-7 MDS FAQ ver. 4: 58th meeting, Pattaya, TH, December, 2001.
· MPEG-7 MDS FAQ ver. 3: 57th meeting, Sydney, AU, July, 2001.
· MPEG-7 MDS FAQ ver. 2: 51st meeting, Noordwijkerhout, NL, March, 2000.
· MPEG-7 MDS FAQ ver. 1: 50th MPEG meeting, Maui, HI, December, 1999.
The Multimedia Description Schemes (MDS) Group is a working group of MPEG formally established in July, 1999 to develop Multimedia Description Schemes as Part 5 of the MPEG-7 Multimedia Content Description Interface standard. The MDS group is currently working on MPEG-7 and MPEG-21 standards.
Multimedia Description Schemes refer to specific MPEG-7 metadata structures that are used to describe and annotate multimedia data. The Multimedia Description Schemes are categorized as pertaining to visual, audio, or generic description. In general, the Multimedia Description Schemes can contain MPEG-7 Descriptors and other Description Schemes (DSs) and can be extended for domain-specific applications.
Multimedia Description Schemes provide a way to describe in XML the important concepts related to multimedia content. The Multimedia Description Schemes facilitate the searching, indexing and filtering of multimedia data. Many of the multimedia concepts are defined in the MPEG-7 Principal Concepts List. The Description Schemes (DSs) are designed to describe both the generic aspects of multimedia content management and the specific features of multimedia content.
The DSs provide a way to specify immutable metadata related to the creation, production, usage and management of the multimedia data. For example, these DSs can be used to describe the title and author of a multimedia program. The multimedia content-specific DSs provide a way to specify the content directly at a number of levels including signal structure, features, models and semantics, as illustrated in Figure 1. Other multimedia specific DSs are designed to allow efficient navigation and access of the multimedia data.
Figure 1. Multimedia Description Schemes describe the multimedia content at a number of levels including signal structure, features, models, and semantics.
Some examples of the different types of Multimedia Description Schemes are given as follows:
§ Semantics the semantics-based DSs provide a way to describe what is depicted in the multimedia content from the real world, such as objects, people, and events.
§ Structure the structure-based DSs provide a way to describe the structure of the multimedia data, such as the regions, segments, shots, and spatial and temporal relationships. Some of the structure-based DSs contain MPEG-7 Descriptors that describe perceptual features of the multimedia data, such as color, texture, shape, motion, melody and timbre.
§ Models the model-based DSs describe feature classes and their relationships with semantic information. The model-based DSs also form classifiers and models that relate the structural and semantic aspects of the multimedia content.
§ Navigation and Access Some of the DSs are designed specifically to address navigation, access and browsing by providing structured summaries and management of different variations of the multimedia programs.
Description Schemes (DSs) are defined using the MPEG-7 Description Definition Language (DDL). DDL is based on XML-schema. DDL allows the creation and extension of DSs.
Multimedia Description Schemes provide a standard set of data types and description structures needed for generating XML descriptions of multimedia content. The descriptions are human-readable and can be searched, transmitted and filtered in applications that deal with multimedia data.
Many Description Schemes (DSs) are computed automatically by processing the multimedia data directly, such as by using feature extraction algorithms or content analysis methods. Other descriptions are generated manually by human annotation or by processing external metadata related to the multimedia material.
The full set of Multimedia Description Schemes, which are related to each other through linking, form a data model or schema for multimedia content description. At both the conceptual- and implementation-levels, the data model is object-oriented and employs modeling principals such as generalization, association, aggregation and spatio-temporal composition. The instantiation of the Description Schemes (DSs) results in description data that is in XML format.
MPEG-7 Description Schemes (DSs) are defined using the MPEG-7 Description Definition Language (DDL). Multimedia Description Schemes are unique in their modeling of multimedia content directly at a number of levels including multimedia signal structure, features, models and semantics, as well as describing immutable metadata related to multimedia content management.
Description Schemes (DSs) can be used for searching, filtering and browsing multimedia material. Since they provide a more complex data model for content description than Descriptors, they are particularly useful in producing rich descriptions of the multimedia content that involve a number of Descriptors and DSs.
The MPEG Description Definition Language (DDL) can be used to define, extend and generate new Description Schemes (DSs) as shown in the following picture. By using the object-oriented aspects of the DS model, it is possible to extend the DSs to develop customized, application specific DSs that are outside of the MPEG-7 standard.
Although there is no formally stated distinction, generally, MPEG-7 Descriptors address the low-level perceptual features of multimedia data and are meant to be extracted automatically. In many cases Descriptors correspond to the perceptual attributes of multimedia data and its regions and segments, such as to describe the color of a region.
Furthermore, the Descriptor data is typically not human-readable since the Descriptor provides a compact binary description of a feature of the multimedia data. On the other hand, the Description Schemes (DSs) are defined using DDL and produce human-readable XML descriptions.
§ AHG Ad Hoc Group
§ CD Committee Draft
§ CE Core Experiment
§ D Descriptor
§ DDL Description Definition Language
§ DS Description Scheme
§ FDIS Final Draft International Standard
§ IS International Standard
§ MDS Multimedia Description Scheme
§ MPEG Moving Picture Experts Group
§ MPEG-7 MPEG standard for a Multimedia Content Description Interface
§ WD Working Draft
§ XML Extensible Markup Language
§ XM Experimentatal Model
There are a number of documents available at the MPEG Home Page at http://mpeg.cselt.it/ including:
§ MPEG-7 Context and Objectives and Technical Roadmap
§ MPEG-7 Principal Concepts List
The Multimedia Description Schemes (MDS) Group is made up of about 40-50 people that attend the MPEG meetings and >200 people that participate in the various MDS Group Ad Hoc Group mail reflectors. The members represent a significant diversity of interests including content providers, software companies, hardware manufacturers, research organizations, and academic institutions.
The MPEG-7 Multimedia Description Schemes (MDS) specification advanced to the Final Draft International Standard (FDIS) stage in July, 2001. The MPEG-7 International Standard is currently under production by ISO. Proposals are currently being made for new tools for MPEG-7 MDS version 2.
First of all, it is not too late to propose new Multimedia Description Schemes. Work on MPEG-7 MDs version 2 is underway.
The standardization process requires that an initial proposal be made to MPEG for the specific Description Scheme (DS). The presentation made to the MPEG Multimedia Description Schemes (MDS) Group should identify sufficient reason and evidence for needing the DS. The proposal should also provide justification of the specific structure of the DS and its relationship with other DS.
The subsequent review and evaluation of the proposal by the MDS Group may result in the establishment of a Core Experiment (CE) in order to better study the DS.
If the DS successfully progresses to the MPEG-7 Experimental Model (XM) or the MPEG-7 Working Draft (WD) stages, software needs to be provided by the proponents of the DS to allow its implementation.
The Multimedia Description Schemes are validated using one of two procedures depending on the type of Description Scheme (DS). First, the specific DSs related to content management, creation, production and usage go through a harmonization process in which the important concepts for description are identified and verified in other multimedia data standards and existing practices.
All other remaining DSs go though an experimentation and verification process in which multiple companies implement the DSs in the context of multimedia applications and demonstrate the usefulness and effectiveness of the DSs.
In both cases, ultimately, it is the consensus of the Multimedia Description Schemes (MDS) Group that determines the sufficiency of the harmonization, validation, and promotion of the DSs to the MPEG-7 Working Draft (WD).
A
Multimedia Description Scheme Core Experiment (CE) is a particular kind of
experiment in which competing proposals for Description Schemes (DSs) are
compared. The situation may arise
when a new proposal is made with the intention of revising, replacing, or
removing a DS that is in the MPEG-7 Multimedia Description Schemes (MDS) Working
Draft (WD). A CE represents a
head-to-head competition in which the alternative proposals are evaluated based
on the comparative advantages.
The Final Draft International Standard (FDIS) of the MPEG-7 Multimedia Description Schemes includes many Description Schemes (DSs), which have been validated in the work of the Multimedia Description Schemes (MDS) Group since 1999. A number of additional DSs are currently being examined in the process of core experimentation for possible inclusion in MPEG-7 MDS version 2.0.
The
FDIS of the MPEG-7 MDS was completed in October 2001.
If you are a member of MPEG, you may join the Multimedia Description Schemes (MDS) Group by joining its meetings which are held at the MPEG meetings. You may also participate in one of the MDS Ad Hoc Groups (AHGs), subscribe to one of the MDS reflectors, or participate in a Core Experiments (CEs).
If you have any further questions, you may send email to the MDS Group reflector at mpeg-7-ds@advent.ee.columbia.edu
If you are not yet a member of MPEG, please see find information about joining at http://mpeg.cselt.it/how_to_join.htm.