INTERNATIONAL ORGANISATION FOR STANDARDISATION
ORGANISATION INTERNATIONALE DE NORMALISATION
ISO/IEC JTC1/SC29/WG11
CODING OF MOVING PICTURES AND AUDIO

ISO/IEC JTC1/SC29/WG11 N
November 2001

MPEG: achievements and current work

1. Introduction 

MPEG is a working group of ISO, the International Organisation for Standardisation. Its formal name is ISO/IEC JTC 1/SC 29/WG 11. The title is: Coding of moving pictures and audio. The are of work assigned to it is: Development of international standards for compression, decompression, processing, and coded representation of moving pictures, audio, and their combination, in order to satisfy a wide variety of applications. The full MPEG terms of reference can be found here.

MPEG held its first meeting in Ottawa, CA on 88/05/10-12. Until October 2000 MPEG has met as a group 54 times. The full list of meeting dates and venues can be found here.

2. MPEG-1 (ISO/IEC 11172)

The first MPEG work item was: Coding of moving pictures and associated audio for digital storage media at up to about 1.5 Mbit/s. In practice this meant a standard for efficient storage and retrieval of audio and video on compact disc. Parts 1 (Systems), part 2 (Video) and part 3 (Audio) of the standard were approved at the November 1992 meeting in London. The Systems part provide multiplexing and synchronisation support to elementary Audio and Video streams. The Video part provides efficient encoding of non-interlaced pictures with roughly VHS quality at 1,15 Mbit/s. The Audio part provides encoding of stereo audio with transparency (i.e. subjective quality similar to the original streo) at 384, 256 and 192  kbit/s per Layer II, II and III respectively. A more comprehensive description of the standard can be found here.

Part 4 of the standard "Conformance Testing" provides methods and reference bitstreams that can be used to assess conformance of a bitstream or of a decoder, was approved one year later. Part 5 of the standard "Reference Software" was approved in 1994. The latter contains the C-code implementation of a Systems multiplexer/demultiplexer and of encoders and decoders for Audio and Video.

This is the table of MPEG-1 parts:

MPEG-1 has been and is being used by many industries in a variety of products, services and applications and has triggered the start of a number of others.

There are many versions of a full MPEG-1 audiovisual players that can be used in a software environment. These utilise all 3 parts of Audio standard with Audio typically in Layer II. Many software packages exists that are capable of encoding audio and video in MPEG-1 and editing the resulting files.

The Video CD is a full application of MPEG-1 that is typically used to encode movies on 2 CDs. In excess of 60 million hardware Video CD decoders have been sold worldwide. Software Video CD decoders are also avilable from multiple sources.

MPEG-1 Audio Layer III, also known as MP3, has been implmented in manifold ways. Many software packages exist to rip a track from a CD Audio and compress it in MP3. This has given rise to innovative ways of consuming music, such as the ability to create compilations to one's liking that can then downloaded to light non-mechanical MP3 players. With the arrival of MP3 the music world has been changed without recognition.

MPEG-2 (ISO/IEC 13818)

The Porto meeting in July 1990 was the first to address the MPEG-2 standard called "Generic coding of Moving Pictures and Associated Audio" and the Singapore meeting in November 1994 was the one that approved the first 3 parts: Systems, Video and Audio. Conformance (part 5) was approved one year later and Reference Software in 1996. The Systems part, in its "Transport Stream" version, provides support for efficient transmission over error-prone delivery systems, while the "Program Stream" version, similar to MPEG-1 Systems, is more useful for digital storage media. The Video part provides support for efficient coding of interlaced pictures ad different spatial resolution. The Audio part provides support to encoding of multichannel audio in such a way that an MPEG-1. A more comprehensive description of the standard can be found here.

MPEG-2 has more parts than MPEG-1. Part 6 "Digital Storage Media Command and Control' or DSM-CC provide protocols for session set up across different networks and for remote control of a server containing MPEG-2 content. Part 7 "Advanced Audio Coding" or AAC provides a new multichannel audio coding that is not backward compatible with MPEG-1 Audio. Part 8 was intended to support video coding when samples are represented with an accuracy of more than 8 bits, but its development was discontinued when the interest of the industry that had requested it did not materialise. Part 9 "Real Time Interface" provides a standard interface between an MPEG-2 Transport Stream and a decoder.

Parts 1, 2 and 3 (this last one sometimes replaced with a proprietary solution) are used in the some 50 million digital television set top boxes and 10 million Digital Versatile Discs (DVD). Some MPEG-2 encoders are very costly professional equipment and some are very inexpensive PC board that are sold with video editing software.

Several examples of DSM-CC is widely used in set top boxes for satellite and cable. This part of the standard is also at the basis of provision of other set top box functionalities by other standards bodies and industry consortia. AAC has been adopted by Japan for a national digital television standard and by several manufacturers of secure digital music.

Over the years several amendments, i.e. extensions, of the standard have been developed. One of the most important is the 4:2:2 profile that extends the use of MPEG-2 into the television studio.

A number of patents are thought to be relevant for implementing the MPEG-2 standard. As MPEG is prevented by ISO rules to deal with patent issues, there is at least one organisation known that handles the licensing of MPEG-2 Systems and Video and another of MPEG-2 Audio.

MPEG-4 (ISO/IEC 14496)

Work on the MPEG-4 standard "Coding of audio-visual objects" began in July 1993 in New York, NY and the first set of standards (so-called version 1) was approved at the Atlantic City, NJ meeting in October 1998. A major extension of the standard (so-called version 2) was approved at the Maui, HI meeting in December 1999.

The first 6 parts of the standard correspond roughly to those of MPEG-2. The title of the first 5 is the same as MPEG-2, the title of the 6th is Delivery Multimedia Integration Framework. There are, however, a number of significant differences of content.

MPEG-4 enables the coding of individual objects. This means that the video information needs not be of rectangular shape as MPEG-1 and MPEG-2 Video assume. The same applies for audio, which provides all tools to encode speech and audio ad different rates and with different functionalities, including an extension of AAC. The systems part, therefore, contains, in addition to the traditional parts of MPEG-1 and MPEG-2 Systems, also the "composition function. Further, since a composition object can be also of synthetic nature, MPEG-4 Systems also contains standard technology to represent time-varying synthetic 3D information. A framework to deal with management and protection of rights arising from individual objects is also provided by MPEG-4 Systems. Finally a file format has been standardised. Part 5 is a complete software implementation of both encoders and decoders. Compared with the reference software of MPEG-1 and MPEG-2 whose value is purely informative, the MPEG-4 Reference Software has the same normative value as the textual parts of the standard. The software may also be used for commercial products and the copyright of the software is licensed at no cost by ISO/IEC for products conforming to the standard. Part 6 has similar functions as DSM-CC and provides an abstraction from the underlying delivery mechanism. A more comprehensive description of the standard can be found here.

So far the industry - both formal and informal - has enthusiastically taken over MPEG-4 Video. MPEG-4 Video has been selected by several industry fora setting standards for next generation mobile communication and is being utilised to develop solutions for video on demand and related applications.

An important MPEG-2 amendment has been developed to support the carriage of MPEG-4 objects on MPEG-2 Transport Streams. This will enable rich multimedia applications in the television domain.

Parts 1, 2, 3 and 6 of the version 1 and 2 portions of the standard have been integrated in what is now called the 2000 edition. Part 5 will be integrated as 2001 edition. Relative to that a number of extensions are being worked on as detailed in the table below.

The full list of MPEG-4 parts is

A brief explanation of some of the main items follows:

• XMT is a textual format that encompasses (parts of) other XML format such as SMIL and Web3D and that can be compiled to the binary MPEG-4 format (BIFS)

• Multiuser will allow the use of MPEG-4 in multiuser environment

• Animation framework will support rich 3D environment

• IPMP (Intellectual Property Management and Protection) will provide an extension to the current  IPMP Framework that will  to provide interoperability at the user level.

• Studio Profile extends the Video algorithm to bitrates in the range of 1 Gbit/s for studio applications

• FGS (Fime Granularity Scalability) extends the Video algorithm to support scalability with a high number of levels

MPEG-4 is a standard that can be used in manifold application areas, each with their own agendas and practices concerning the business models of how technology is actually deployed. As again a number of patents are thought to be relevant for implementing the MPEG-4 standard and as MPEG is prevented by ISO rules to deal with patent issues, the MPEG-4 Industry Forum has been established as a not-for-profit organisation external to MPEG with the goal of promoting the adoption of the standard and in particular creating the conditions for organisations dealing with patent licensing to be set up.

MPEG-7 (ISO/IEC 15938)

Work on MPEG-7 "Multimedia Content Description Interface" standard started at the April 1997 meeting in Bristol. MPEG-7 is an audio-visual information representation that is different from the previous MPEG standards in the sense that what is represented is not the information itself but the information about the information. MPEG-7 is a 8-part standard:

The technical content of the standard is as follows:

1. Systems provides the architectural framework of the standard, the carriage of MPEG-7 content and the binarisation of MPEG-7 content

2. Description Definition Language allows to create descriptors and description schemes

3. Visual provides standard descriptors and description schemes that are purely visual

4. Audio provides standard descriptors and description schemes that are purely audio

5. Multimedia Description Schemes provides standard descriptors and description schemes that are neither visual nor audio

6. Reference software has the same normative value as the MPEG-4 reference software and may be used for products at the same conditions

7. Conformance is the means to test an implementation or data for conformity.

8. Part 8 describes how feature extraction can be implmented.

Work aimed at producing a Version 2 of MPEG-7 will be done in the areas of Systems, Visual, Audio and Multimedia Description Schemes.

MPEG-21 (ISO/IEC 18034)

Work on MPEG-21 "Multimedia Framework" standard started at the May-June 2000 meeting in Geneva. MPEG-21 seeks to describe a multimedia framework and sets out a vision for the future of an environment where delivery and use of all content types by different categories of users in multiple application domains will be possible.

MPEG-21 assumes that there are Users (anybody in the value network) and Digital Items (assembly of content) on which Users execute Actions that generate other Digital Items that can become object of Transactions. In order to make this possible a number of technologies are needed that fall under the following categories

1. Digital Item Declaration
2. Digital Item Identification and Description
3. Intellectual Property Management and Protection
4. Terminals and Networks
5. Digital Item Management and Usage
6. Digital Item Representation
7. Event Reporting

The current table of MPEG-21 standards is

Miscellanea

MPEG is currently engaged in several other related projects:

1. 3DAV: coding of audio and video information from a 3D space

2. Scalable Video Coding: coding of video with high scalability features and high compression