Can MPEG cope with new media technologies? 

Leonardo Chiariglione,
Convenor, ISO/IEC JTC1/SC29/WG11 (MPEG)


This paper scans the almost 20 year of MPEG life to discover the movements of the industry, as represented by the standards produced, and analyses the current status.. 

1. Introduction

Formally the Moving Pictures Experts Group (MPEG) is Working Group 11 (WG 11) Coding of moving pic­tures and audio” of Subcommittee 29 (SC 29) “Coding of audio, picture, multimedia and hypermedia inform­ation" of the Joint Technical Committee 1 (JTC 1) “Inf­ormation Technologies” of the International Organ­isation for Standardisation (ISO) and the International Electrotechnical Commission (IEC).

Working Groups are typically established by a Sub­committee to deal with some specific technical issue. Once the issue has been resolved working groups are typically disbanded.

This has not been the case of MPEG. Born in 1988 as an experts group of the then WG 8 of SC 2 MPEG was driven by a vision whose main points can be summarised as below: 

 2. The bread and butter

2.1. MPEG-1

This standard was developed in the early days when it was not really clear what the impact of digital would be. Indeed MPEG-1 meant having the technology to handle audio and video bits for the first time with the magic word of “interactivity”.

In those days anything that implied digital audio and video had to confront with issues of hardware – i.e. in­tegrated circuit (IC) – implementation. So all decisions were weighed against possible impacts on IC design. 

 The most serious problem faced by MPEG-1 was one of the actual use of digital technologies based on the standard. The Digital Compact Cassette (DCC), meant to be a user of MPEG-1 Audio Layer I was a flop, the Digital Audio Broadcasting (DAB) standard, meant to be a user of MPEG-1 Audio Layer II, is not really flying even now and Copact Disc Interactive (CD-i), meant to be a user of the complete set of MPEG-1 technologies, is long dead.

Still MPEG-1 is well alive and kicking: Video CD uses the complete set of MPEG-1 technologies and MP3 uses MPEG-1 Audio Layer III. Both need no introduction. 

2.2. MPEG-2

Once it was proved that the machine could work it was possible to make true the long-held dream of television made digital. What MPEG added was the definition of a truly universal digital video format (audio, as it well known, is not part of the truly universal digital television format).

The development of MPEG-2 Video was obsessed by the issues of hardware implementation. In the early 1990s designing an MPEG-1 Video chip was becoming possible, but an MPEG-2 chip was another story. Hardware problems existed not only for the signal processing part but also for the Random Access Memory (RAM) that was required to store decoded pictures. A special profile (Simple) was designed to avoid use of too much RAM, although with some picture quality loss.

The issue of patents came to the fore with all its importance. The Consumer Electronics (CE) industry had its ways to handle the issue of patents in its standards: the IPR of a successful device belonged to the company that had originally conceived it, but with MPEG quite a few companies held IPR on MPEG-2 Video. The first patent pool after many years was established to handle the over 100 patents that the patent pool administrator has found to be relevant to implementing the MPEG-2 standard. The royalty model followed the established CE practice: a fixed amount per device and Digital Versatile Disc (DVD) disc carrying MPEG-2 Video encoded content. 

2.3. MPEG-4

MPEG-4 was conceived as a unified multimedia coding standard. Indeed MPEG-4 Video has been designed to cover all bitrates and all video formats and MPEG-4 Audio is capable of handling music and speech.

MPEG-4, however, is also capable of handling synthetic content, such as 2D and 3D graphics, human faces and bodies, synthetic music etc. Moreover it has a pow­erful technology to handle the composition of natural and synthetic information.

MPEG-4 has been extremely prolific in Visual extens­ions and profiles. The basic Advanced Audio Coding (AAC) technology inherited and extended in MPEG-4 has undergone many extensions.

MPEG-4 did not develop a transport mechanism for MPEG content but only a generic interface to transport. It did develop, however, a Real Time Protocol (RPT) payload – and after untold vicissitudes got it adopted by the Internet Engineering Task Force (IETF) – for gen­eric content.

MPEG-4 has been an innovator in several respects. The first is the fact that MPEG-4 has provided the first ex­ample to manage Intellectual Property Rights (IPR) in a complete fashion through its Intellectual Property Man­agement and Protection (IPMP).

The second form of innovation was brought by MPEG-4 as the convergence point of the traditional hardware-based and the new software-based audio-visual industry. This prompted the transformation of the reference software from a nice-to-have add on to the formal status of standard on par with the textual form of the standard. The development process of the reference software was also codified – independently – along the lines of the Open Source Software tradition.

Another innovation was made by MPEG-4 in patent licensing. While MPEG-2 Video has a licensing model that is based on physical objects, be they set top boxes (STB) or DVD discs, MPEG-4 Visual also has a licens­ing form that charges the use of content per unit time. 

3. New land

3.1. MPEG-7

The MPEG-1/-2/-4 standards have each played a role in shaping the digital audio and video industry as we know it today. MPEG-7 has the potential to reshape the industry in even more substantial ways.

The natural observation – a tautology today when we use a search engine to find information on the web – is that content is important but more so is the description of the content so that it can be found and organised.

The idea of MPEG-7 was triggered by the realisation that as much as a standard form of audio and video coding had created horizontal markets of content, a standard form of audio and video descriptions would create horizontal markets of descriptions.

The needed standard technology has all been develop­ed. There are visual descriptions, audio descriptions and multimedia descriptions. It is also possible to def­ine Multimedia Description Schemes (MDS) using the MPEG-7 Description Definition Language (DDL).

Today the MPEG-7 vision has still some way to go. The main reason is that digital technologies have rem­oved value from “free” content. Premium content, for which value is still high, is offered as proprietary pack­age using Digital Rights Management (DRM) technol­ogies.

We are not exactly in a horizontal market and this will remain so for as long as there will be no interoperable DRM (iDRM) solution broadly adopted by the industry. iDRM is the only way for content to retain its value thereby providing an incentive for companies to add more features to their offers. 

3.2. MPEG-21

As much as MPEG-7 is not an audio and video coding standard, MPEG-21 is also not a “signal processing” standard. Its need was triggered by the realisation that compression and description are two important comp­onents of “Multimedia” but to enable a complete multi­media market where content can be traded at the atomic level there are more disparate technologies to integrate.

To achieve this goal a number of new technologies are required: 

4. New horizons

4.1. MPEG-A

Some 19 years after MPEG began its existence the sit­uation today is that there is a lot of multimedia technol­ogies out there both from MPEG and from other sour­ces, but making successful multimedia applications is as hard as ever – just too many choices.

On the other hand MPEG has an impressive repository of multimedia tools and has the know how to define the appropriate combination of technologies that enable an application.

This is the rationale for the MPEG-A (A as Applicat­ion) standard for which the following has been done or is under way

 4.2. MPEG-E

The MPEG Multimedia Middleware (M3W) is another non-coding and non-signal processing but very much multimedia oriented standard.

The standard will be eventually made up of the follow­ing parts 

  1. Architecture

  2. Multimedia API

  3. Component Model

  4. Resource and Quality Management

  5. Component Download

  6. Fault Management

  7. System Integrity Management

 The M3W API is intended to be a generic API suitable for use in multiple products with different capabilities that have significant overlap in the functionality they require from the platform, but there are also differences. 

5. The situation today

While MPEG is running towards the end of its second decade it is possible to make a few assessments regar­ding its field of work. 

6. Conclusions

In its life span close to 2 decades MPEG has played a key role in moving the incumbent (20 years ago) audio and video industry to the digital shore and creating the new converged digital audio and video industry that we see around us.

Many things have changed in these years and the indus­try is far from reaching a stable configuration. As the state of technology continues to progress the need for standards will continue.

MPEG will continue its role of anticipating the stan­dardisation needs of the industry by continuing to stick to its original vision, relying on the considerable tech­nology assets developed so far and designing new standards in a less structured world.