Summary of MPEG technologies
The scope of activity of the Moving Picture Coding Experts Group (MPEG) covers standardisation of all technologies that are required for interoperable multimedia. A rough classification of areas is
|
1. |
|
|
|
|
|
1. |
|
|
||
| 1. |
What it does: |
|||
| Provides technology to code non-interlaced video | ||||
| What it is for: | ||||
| Good for up to SIF resolution video | ||||
| Where: ISO/IEC 11172-1, -2, -3 | ||||
|
2. |
What it does: |
|||
| Provides technology to code both non-interlaced and interlaced video | ||||
| What it is for: | ||||
| Good for high quality, up to HD video. Particularly suitable for TV broadcast and DVD storage. | ||||
| Where: ISO/IEC 13818-2 | ||||
|
3. |
What it does: |
|||
| Provides technology to code both non-interlaced and interlaced video over a broad range of bitrates and resolutions | ||||
| What it is for: | ||||
| Internet and mobile video (Simple Profile), video on demand (Advanced Simple Profile) and studio applications (Studio Profile) | ||||
| Where: ISO/IEC 14496-2 | ||||
|
4. |
What it does: |
|||
| Provides technology to encode visual content for interactive multimedia, supporting non-rectangular video objects, mesh-based deformations and sprites, face and body animation | ||||
| What it is for: | ||||
| Web and 3D navigation applications | ||||
| Where: ISO/IEC 14496-2 | ||||
|
5. |
What it does: |
|||
| Provides technology to code both non-interlaced and interlaced video over a broad range of bitrates and resolutions with a coding efficiency approximately double compared to MPEG-2 and MPEG-4 Visual | ||||
| What it is for: | ||||
| Practically all non-scalable digital video applications | ||||
| Where: ISO/IEC 14496-10 | ||||
|
6. |
What it does: |
|||
|
Provides a scalable video stream with a compression efficiency comparable to state-of-the-art non-scalable codecs over large range of data rates. Supports various combinations of spatial, temporal and SNR scalability, base layer compatibility with AVC. |
||||
| What it is for: | ||||
| Internet video, wireless LAN video, mobile wireless video for conversational, VOD, and live broadcasting, multi-channel content production and distribution, surveillance-and-storage applications, and layered contents protection | ||||
| Where: ISO/IEC 14496-10 | ||||
|
7. |
What it does: |
|||
| This is an exploration activity designed to collect evidence on the potential advantages offered by wavelet transforms for future video coding standards. | ||||
| What it is for: | ||||
| Same as scalable video | ||||
| Where: TBD | ||||
|
2. |
1. |
|
||
|
1. |
What it does: |
|||
| Provides efficient representation of the signals of an array of video camera shooting the same scene. | ||||
| What it is for: | ||||
|
||||
| Where:TBD | ||||
|
3. |
|
|
||
| 1. |
What it does: |
|||
Provides technology to represent stereo sound in a quality
scalable manner
|
||||
| What it is for: | ||||
| Potentially all digital sound and television applications can be covered by this technology | ||||
| Where: ISO/IEC 11172-3 | ||||
|
2. |
What it does: |
|||
Provides technology to represent
|
||||
| What it is for: | ||||
| Multichannel audio applications | ||||
| Where: ISO/IEC 13818-3 | ||||
| 3. |
What it does: |
|||
| Provides technology to represent stereo and multichannel sound with transparency achieved at 128 kbit/s (stereo) and 320 kbit/s (5.1 multichannel). AAC comes in two versions: MPEG-2 and MPEG-4, the latter being an extension of the technology with more coding tools | ||||
| What it is for: | ||||
| Potentially all digital sound and television applications can be covered by this technology | ||||
| Where: ISO/IEC 13818-3 and ISO/IEC 14496-3 | ||||
|
4. |
What it does: Represent an audio or speech signal by means of 4 objects (sinusoids, noise, transients and stereo image) that are parameterized and subsequently efficiently coded into a bitstream format |
|||
| What it is for: Low bit-rate audio and speech applications demanding medium to good quality. The parametric coding scheme inherently provides for high quality tempo and pitch scalability for no additional complexity in the decoder. The parametric stereo object can also be combined with HE-AAC providing a significant advantage in complexity. This particular combination is referred to as the HE-AAC v2 profile and has been an enabling technology for music download services. | ||||
| Where: | ||||
| 5. |
What it does: Provides technology to recreate high frequencies that have been discarded in the encoding process by the underlying audio coder it is used in combination with. Provides full audio bandwidth for all bitrates. |
|||
| What it is for: All applications in the bitrate range where the full bandwidth cannot be sufficiently well coded by the underlying audio coder. | ||||
| Where: ISO/IEC 13818-3 and ISO/IEC 14496-3 | ||||
|
6. |
What it does: |
|||
| Provides lossless compression of audio waveforms. Supports both consumer (e.g. 48 kHz/16 bit) and professional (e.g. 192 kHz / 24 bit) formats. | ||||
| What it is for: | ||||
| A significant application is archiving of professional studio recordings. | ||||
| Where: ISO/IEC 14496-3/Amd.? | ||||
|
7. |
What it does: |
|||
| Provides fine-grain scalable compression of audio waveforms. The architecture consists of a core lossy audio codec plus an enhancement quantizer. Scalability is from the rate of the core lossy codec through a rate that achieves lossless reconstruction. | ||||
| What it is for: | ||||
| Where: ISO/IEC 14496-3/Amd.? | ||||
| 8. | What it does: | |||
| What it does: Provides lossless compression of 1-bit, highly oversampled audio representations | ||||
| What it is for: | ||||
| Archiving of high quality professional studio recordings in a 1-bit oversampled audio format. This includes the Direct Stream Digital (DSD) format as applied on the widely used Super Audio CD (SACD) medium. | ||||
| Where: ISO/IEC 14496-3/Amd.? | ||||
|
|
9. |
MPEG Surround |
What it does: |
|
| Provides a multi-channel signal which approximates the original multi-channel material as closely as possible in a perceptual sense, and which exploits the human perception of a sound stage. Coding consists of e.g. 2 audio channels encoded with e.g. an MPEG audio codec plus “spatial cue” side information. | ||||
| What it is for: | ||||
| Facilitates transmission of 5.1 channels of compressed audio over channels that currently support only 2 channels of compressed audio, such as satellite or terrestrial DAB transmission channels. | ||||
| Where: ISO/IEC 14496-3/Amd.? | ||||
|
10. |
What it does: | |||
An exploration activity designed to seek technology that
has
|
||||
| What it is for: | ||||
| This is expected to have two broad application areas: Applications in which transmission is a mix of speech and speech plus music or only music. Applications in which network-based scalability is required, for example audio streaming over a local or wide-area network. | ||||
| Where:TBD | ||||
|
4. |
|
|
||
|
|
1. |
Texture coding | What it does: | |
| What it is for: | ||||
| Where: ISO/IEC 14496-2 | ||||
|
|
2. |
2D mesh coding | What it does: | |
| Provides coded representation of 2D Meshes with Implicit Structure | ||||
| What it is for: | ||||
| Where: | ||||
|
5. |
|
|
||
|
|
1. |
Face and Body Animation |
What it does: |
|
|
Provides coded representation of parameters that can define, calibrate and animate synthetic faces and bodies |
||||
|
What it is for: |
||||
|
Avatar animation for low bitrate communication |
||||
|
Where: ISO/IEC 14496-2 |
||||
|
|
2. |
3D mesh coding |
What it does: |
|
|
Provides 40~50:1 compression for 3D mesh (IndexedFaceSet) with the following functionalities
|
||||
|
What it is for: |
||||
|
3D applications requiring compression of 3D mesh model that can be represented by IndexedFaceSet. |
||||
|
Where: ISO/IEC 14496-2 |
||||
|
|
3. |
AFX |
What it does: |
|
|
Provides technologies for
|
||||
|
What it is for: |
||||
|
3D applications requiring content compression and low bitrate communication |
||||
|
Where: ISO/IEC 14496-16 |
||||
|
6. |
Synthetic audio coding | |||
|
|
1. |
Structured audio | What it does: | |
Provides two languages
|
||||
| What it is for: | ||||
| Where: ISO/IEC 14496-2 | ||||
| 7. | Text coding | |||
|
|
1. |
Streaming text format | What it does: | |
| Provides specification of concatenation of text access units, format of text streams and text access units, signaling and decoding of text streams | ||||
| What it is for: | ||||
| Where: ISO/IEC 14496-17 | ||||
|
8. |
|
Font coding | ||
|
|
1. |
Font compression and streaming | What it does: | |
Provides technologies for
|
||||
| What it is for: Faithful encoding and reproduction of text in multimedia content | ||||
| Where: ISO/IEC 14496-18 | ||||
|
|
2. |
Open Font Format | What it does: | |
| Conversion of the OpenType font format specification from the external normatively referenced specification to an ISO/IEC standard. | ||||
| What it is for: | ||||
| Where: ISO/IEC 14496-22 | ||||
|
2. |
|
|
||
|
|
1. |
|
||
|
|
1. |
Binary Format for Scenes | What it does: | |
| Provides a compact representation of scene description information. The scene can be animated by means of BIFS Anim and BIFS Update | ||||
| What it is for: | ||||
| Where: | ||||
|
|
2. |
What it does: |
||
| Provides a scene representation targeting a trade-off between expressivity, compression efficiency, decoding and rendering efficiency, and memory footprint. | ||||
| What it is for: | ||||
| For constrained environment (e.g. mobile devices) | ||||
| Where: ISO/IEC 14496-20 | ||||
|
|
3. |
Symbolic Music Representation | What it does: | |
| It provides a logical structure that relates and synchronizes symbolic music elements with the audio-visual events represented and rendered using MPEG technology. | ||||
| What it is for: | ||||
SMR can
|
||||
| Where: TBD | ||||
|
3. |
|
|
||
|
|
1. |
|
||
|
|
1. |
What it does: | ||
| What it is for: | ||||
| Where: ISO/IEC 15938-2 | ||||
|
2. |
|
|||
|
|
1. |
What it does: | ||
Specifies the following descriptors of visual information
|
||||
| What it is for: Search and retrieval applications, automatic and semi-automatic indexing by image/video based features | ||||
| Where: ISO/IEC 15938-3 | ||||
|
|
2. |
What it does: | ||
Specifies the following descriptor
|
||||
| What it is for: As under 1. above with increased semantic expressiveness. | ||||
| Where: ISO/IEC 15938-3 | ||||
|
2. |
|
|||
|
|
1. |
Low level descriptions |
What it does: | |
Specifies the following descriptors
|
||||
| What it is for: MPEG-7 low level descriptors are useful for semantically low level search and retrieval applications as music identification, music similarity or genre recognition | ||||
| Where: ISO/IEC 15938-4 | ||||
|
|
2. |
High level descriptions |
What it does: | |
Specifies the following descriptors
|
||||
| What it is for: Allow a search and retrieval of semantically higher expressiveness. With the help of this descriptor search applications manage to have access to melody, rhythmic information or spoken content analysis | ||||
| Where: ISO/IEC 15938-4 | ||||
|
3. |
|
|||
|
|
1. |
What it does: | ||
| What it is for: | ||||
| Where: ISO/IEC 15938-5 | ||||
| 4. |
|
|
||
|
1. |
|
|||
|
|
1. |
What it does: | ||
| What it is for: | ||||
| Where: ISO/IEC 11172-1 | ||||
|
|
2. |
What it does: | ||
| What it is for: | ||||
| Where: ISO/IEC 13818-1 | ||||
|
|
3. |
What it does: | ||
| What it is for: | ||||
| Where: ISO/IEC 14496-1 | ||||
|
2. |
|
|||
|
|
1. |
What it does: | ||
| What it is for: | ||||
| Where: ISO/IEC 13818-6 | ||||
|
|
2. |
What it does: | ||
| What it is for: | ||||
| Where: ISO/IEC 14496-6 | ||||
|
5. |
||||
|
1. |
|
|||
|
|
1. |
What it does: Provides means to identify the copyright of MPEG-2 coded content | ||
| What it is for: Intellectual Property Management and Protection | ||||
| Where: ISO/IEC 13818-2 | ||||
|
|
2. |
What it does: | ||
MPEG-4 OCI descriptors convey descriptive information about
audio-visual objects, e.g.
|
||||
| What it is for: | ||||
| Where: | ||||
|
|
3. |
What it does: | ||
Specifies
Note that in the list above Digital Item includes also its parts. |
||||
| What it is for: | ||||
| Where: ISO/IEC 21000-3 | ||||
|
2. |
|
|||
|
|
1. |
What it does: | ||
| Specifies the syntax and semantics of the MPEG-21 Rights Expression Language | ||||
| What it is for: | ||||
| Where: ISO/IEC 21000-5 | ||||
|
|
2. |
What it does: | ||
|
||||
| What it is for: | ||||
| Where: ISO/IEC 21000-6 | ||||
|
3. |
|
|||
|
3. |
1. |
What it does: | ||
| Documents best practice in the evaluation of tools for persistent association | ||||
| What it is for: | ||||
It allows to conduct such evaluations using a common
methodology. This gives confidence to those relying on the
results that they are:
|
||||
| Where: ISO/IEC 21000-11 | ||||
|
3. |
|
|||
|
|
1. |
What it does: | ||
| What it is for: | ||||
| Where: ISO/IEC 13818-11 | ||||
|
|
2. |
What it does: | ||
| What it is for: | ||||
|
|
3. |
What it does: | ||
Provides:
|
||||
| What it is for: | ||||
| IPMP of Digital Item resources | ||||
| Where: ISO/IEC 21000-4 | ||||
|
6. |
|
|||
|
1. |
|
|||
|
|
1. |
What it does: | ||
| What it is for: | ||||
| Where: ISO/IEC 21000-2 | ||||
|
|
2. |
What it does: | ||
| Specifies the syntax and semantics of tools that may be used to process Digital Items. The tools provide a normative set of tools that specify the processing of a Digital Item in a predefined manner | ||||
| What it is for: | ||||
| Where: ISO/IEC 21000-10 | ||||
|
|
3. |
What it does: | ||
| Provides mechanisms to express Event Report Requests and Event Reports | ||||
| What it is for: | ||||
| iAllow Users to monitor Events relating to Digital Items and/or Peers | ||||
| Where: ISO/IEC 21000-15 | ||||
|
|
4. |
What it does: | ||
| What it is for: | ||||
| Where: http://standards.iso.org/ittf/PubliclyAvailableStandards/MPEG-21_schema_files | ||||
|
2. |
|
|||
|
|
1. |
What it does: | ||
| Specifies the syntax and semantics of tools that may be used to assist the adaptation of Digital Items, i.e., the Digital Item Declaration and resources referenced by the declaration. | ||||
| What it is for: | ||||
| The tools can be used to satisfy transmission, storage and consumption constraints, as well as Quality of Service management by the various Users. | ||||
| Where: ISO/IEC 21000-7 | ||||
|
|
2. |
What it does: | ||
| Provides a framework and schemes for addressing fragments of resources with an MPEG MIME media type. The domains addressable include spatial, spatiotemporal, temporal, physical format, and logical models. | ||||
| What it is for: | ||||
| Where: ISO/IEC 21000-17 | ||||
|
7. |
|
|
|
|
|
1. |
|
|||
|
|
1. |
What it does: | ||
| What it is for: | ||||
| Where: ISO/IEC 11172-1 and ISO/IEC 13818-1 | ||||
|
|
2. |
What it does: | ||
| What it is for: | ||||
| Where: ISO/IEC 13818-1 | ||||
|
|
3. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
2. |
|
|||
|
|
1. |
What it does: | ||
| Provides a File Format to contain timed media information for a presentation in a flexible, extensible format. This presentation may be ‘local’ to the system containing the presentation, or may be via a network or other stream delivery mechanism. | ||||
| What it is for: | ||||
| It facilitates interchange, management, editing, and presentation of media | ||||
| Where: ISO/IEC 14496-12 | ||||
|
|
2. |
What it does: | ||
| Provides an instance of the ISO Base Media File format capable of supporting highly dynamic MPEG-4 (BIFS) presentations | ||||
| What it is for: | ||||
| It facilitates interchange, management, editing, and presentation of MPEG-4 (BIFS) presentations | ||||
| Where: ISO/IEC 14496-14 | ||||
|
|
3. |
What it does: | ||
| Defines the storage of AVC video in all file formats in the ISO base media file format family | ||||
| What it is for: | ||||
| Provides a common interchange format for AVC video by using a uniform file storage in a powerful container format | ||||
| Where: ISO/IEC 14496-15 | ||||
|
|
4. |
What it does: | ||
| Specifies how a Digital Item Declaration (DID) and some or all of its referenced content can be placed in a single ‘content package’ file. | ||||
| What it is for: | ||||
| Enables the interchange, editing, and ‘playback’ of MPEG-21 Digital Items. | ||||
| Where: ISO/IEC 21000-9 | ||||
|
8. |
|
|
|
|
|
|
1. |
|
|
|
|
|
1. |
What it does: | ||
| What it is for: | ||||
| Where: ISO/IEC 11172-1 | ||||
|
|
2. |
What it does: | ||
| What it is for: | ||||
| Where: ISO/IEC 13818-1 | ||||
|
|
3. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
4. |
What it does: | ||
| What it is for: | ||||
| Where: ISO/IEC 15938-1 | ||||
|
|
2. |
|
|
|
|
|
1. |
What it does: | ||
Provides the following APIs
|
||||
| What it is for: | ||||
| Where: ISO/IEC 14496-11 | ||||
|
|
2. |
What it does: | ||
| What it does: Provides a lightweight API for 3D entertainment applications that integrates Java, graphics and AV subsystems | ||||
| What it is for: | ||||
| What it is for: By using this API with MPEG-J, existing Java applications can be enhanced with MPEG's systems and audio-visual streams | ||||
| Where: ISO/IEC 14496-21 | ||||
|
|
3. |
What it does: | ||
Provides a set of APIs to allow multimedia applications to:
|
||||
| What it is for: | ||||
|
||||
| Where:TBD | ||||
|
9. |
|
|
|
|
| 1. | Players | |||
|
|
1. |
What it does: | ||
| Specification of a format comprising MP3 audio, song title and artist meta data and JPEG album art using the components of MPEG-4 audio and file format , MPEG-7 meta data constructs and JPEG image coding | ||||
| What it is for: | ||||
| can be used to construct PC-based or portable music players. The standard has sufficient flexibility to construct libraries of individual songs or collections of songs (e.g. Compact Disk albums). The metadata associated with each song permits sorting, search and play list generation. MPEG Music Player MAF | ||||
| Where:ISO/IEC 23000-2 | ||||
|
|
2. |
What it does: | ||
| Specification of a format comprising images and metadata | ||||
| What it is for: | ||||
| Provides a convenient format to store images with metadata to facilitate search | ||||
| Where:TBD | ||||
|
10. |
|
|
|
|
|
|
1. |
|
|
|
|
|
|
1. |
What it does: | |
Provides a generic binary format to
|
||||
| What it is for: | ||||
| It allows both to precisely manage bandwidth consumption and reduce application latency. | ||||
| Where:TBD ISO/IEC 23001-1 | ||||
|
|
2. |
|
|
|
|
|
1. |
What it does: | ||
| Provides specification for the IDCT accuracy. | ||||
| What it is for: | ||||
| Provides a standard that can be referenced in lieu of IEEE 1180 which has been withdrawn | ||||
| Where: ISO/IEC 11172-6 | ||||
|
|
|
2. |
What it does: |
|
|
||||
| What it is for: | ||||
| The approximation can become the reference implementation of the 8x8 IDCT because it satisfies the conformance requirements for the MPEG-1, MPEG-2 and MPEG-4 part 2 video coding standards. | ||||
| Where: TBD | ||||
|
|
3. |
What it does: | ||
|
||||
| What it is for: | ||||
Provides
|
||||
| Where: TBD | ||||
|
11. |
|
|
|
|
| 1. | Reference software | |||
|
|
1. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
2. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
3. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
4. |
What it does: Provides
|
||
| What it is for: To enable widespread use of the MPEG-7 standard through reference implementations of tools, bitstreams, interfaces and informative technologies for extracting descriptions | ||||
| Where: ISO/IEC 15938-6 | ||||
|
|
5. |
What it does: | ||
| Provides Reference Software for MPEG-21 DID, DIA, REL, RDD, DIP and File Format: software for individual parts, integrated software, utility software for individual parts and integrated | ||||
| What it is for: | ||||
| Where: ISO/IEC 21000-8 | ||||
|
|
6. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
| 2. | Reference hardware description | |||
|
|
1. |
What it does: provides
|
||
What it is for: to promote
|
||||
| Where: ISO/IED 14496-9 | ||||
|
12. |
|
|
|
|
|
|
1. |
|
|
|
|
|
1. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
2. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
3. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
2. |
|
|
|
|
|
1. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
2. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
3. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
4. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
3. |
|
|
|
|
|
1. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
2. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
3. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
4. |
|
|
|
|
|
1. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
2. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
3. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
5. |
|
|
|
|
|
1. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
2. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
3. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
4. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
|
5. |
|
|
|
|
|
1. |
What it does: | ||
| What it is for: | ||||
| Where: | ||||
|
13. |
|
|
|
|
|
|
1. |
|
|
|
|
|
1. |
|
|
|
|
|
1. |
|
|
|
|
|
1. |
|
|
|
|
|
1. |
|
|
|
|
|
1. |
|
|