The new MPEG-4 standard provides a suite of functionalities under one standard: streaming multimedia content, good compression, and user interactivity. This paper provides an introduction to the use and internal mechanisms of these functions.

An MPEG-4 system can be conceptually decomposed into several components, as shown in Figure 1. Starting at the bottom of the figure, we can identify the following layers:

• Deliver (or Transport) layer: The delivery layer is media unaware and delivery aware. MPEG-4 does not define any specific deliver layer. Rather, MPEG-4 media can be transported on existing transport layers such as RTP, TCP, MPEG-2 Transport Streams, H-323 or ATM.
• Sync or elementary stream layer: This component of the system is in charge of the synchronization and buffering of compressed media. The sync layer deals with elementary streams. Elementary streams are a key notion in MPEG-4. A complete MPEG-4 presentation transports each medium/object in a different elementary stream. Elementary streams are composed of access units (e.g., a video object frame), packetized into Sync Layer (SL) packets. Some media may be transported in several elementary streams, for instance if scalability is involved. This layer is media-unaware and delivery-unaware, and talks to the Deliver layer through something called the DMIF application interface (DAI) [7]. The DAI is only a non-normative conceptual abstraction of the deliver layer and, in addition to the usual session set up and stream control functions, also enables setting the quality of service requirements for each stream. The DAI is network independent.
• Media or Compression layer: This is the component of the system performing the decoding of the media: audio, video, etc. Media are extracted from the Sync Layer through the elementary stream interface. Note that BIFS itself also has to go through the compression layer and then be decoded. Another special MPEG-4 medium type is the Object Descriptor (OD). An OD is a structure similar to a URL, containing pointers to elementary streams. Typically, however, these pointers are not to remote hosts, but to elementary streams that have already been received by the client. ODs also contain additional information such as Quality of Service parameters. This layer is media aware, but delivery unaware. The BIFS content is delivered in an elementary stream and is shown in the figure as the "scene description information."

MPEG-4 is a large and complex standard because it addresses a huge problem space: the creation and efficient delivery of compelling interactive audiovisual content across a variety of networks. MPEG-4, however, defines profiles in order to best match specific application areas with subsets of its specifications. The same approach has been used very effectively with MPEG-2. Not all applications need the entire set of MPEG-4 tools, and it would be a burden to force terminals to fully implement features that they do not require. MPEG-4 thus defines a limited number of subsets of Visual, Audio and Systems tools, to which specific applications can conform. Within each profile, levels may be defined to match the varying complexity of different terminals.

Within the context of this toolkit approach, BIFS can also be seen as a separate tool. While full MPEG-4 implementations would get the maximum benefit from the standard in terms of features and capabilities, it is certainly possible for a given application to comply with certain Scene Graph profiles and levels without using other parts of the MPEG-4 specification. The details of the Scene Graph profiles are beyond the scope of this article, but in broad terms they consist of an audio profile, a simple 2D profile, a complete 2D profile, and a complete profile.

An MPEG-4 scene is constructed as a direct acyclic graph of nodes. The following types of nodes exist:

• Grouping nodes construct the scene structure;
• Children nodes are the children of grouping nodes that represent the multimedia objects in the scene.
• Bindable children nodes are a specific type of children nodes that represent the node types for which only one instance of the node type can be active at a time in the scene (a typical example of that is the Viewpoint for a 3D scene. A 3D scene may contain multiple viewpoints or "cameras," but only one can be active at a time).
• Interpolator nodes are another subtype of children nodes that represent interpolation data to perform key frame animation. These nodes generate a sequence of values as a function of time or other input parameters.
• Sensor nodes sense the user and environment changes for authoring interactive scenes.

These nodes are summarized in Section 3.9, below.

BIFS and VRML scenes are both composed of a collection of nodes arranged in a hierarchical tree. Each node represents, groups, or transforms an object in the scene and consists of a list of fields that define the particular behavior of the node. For example, a Sphere node has a radius field that specifies the size of the sphere. MPEG-4 has roughly 100 nodes with 20 basic field types for representing the basic field data types: boolean, integer, floating point, 2- and 3-dimensional vectors, time, normal vectors, rotations, colors, URLs, strings, images, and other more arcane data types such as scripts. Table 1 shows the list of the more common MPEG-4 types.

The node fields are labeled as being of type field, eventIn , eventOut , or exposedField . The field label is used for values that are set only when instantiating the node. Fields that can receive incoming events have the eventIn label, whereas fields that emit events are labeled as eventOut. Finally, some fields may set values but also receive or emit events, in which case they are labeled as exposedField. An additional important feature of the node and field structure is that node fields receive default values. When instantiated, the default values are assumed if not specified explicitly in the bit stream.

Node fields sometimes represent one value, as in the radius of the Sphere node, or many values, as in a list of vertices that define a polygon. Thus, each basic data type can be represented as a single-value type, denoted by type names that begin with an ‘S’, or as a multiple-value type, denoted by type names that begin with an ‘M’. For example, the floating point radius of the Sphere node has type SFFloat, where as the collection of points with 3-dimensions in a Coordinate node that can be used to define a polygon has the type MFVec3f. Nodes are defined using a semantic declaration that contains the node name, followed by a list of the node’s fields. The list specifies each field’s label, type, name and default value. For example, the Viewpoint node has the following semantic declaration.

}

Many nodes have fields that hold other nodes – this is what gives the scene its tree structure. Whereas VRML has only two node types, SFNode and MFNode, MPEG-4 has a rigidly typed collection of nodes that specifies exactly which nodes can be contained within which other nodes. A node may have more than one type, however, when it can be contained as a child node in different contexts.

For example, the Shape node is used to include a geometric shape in the scene. It has a geometry field that holds any node with type SFGeometryNode, of which there are 18, and an appearance field that holds nodes of type SFAppearanceNode, of which there is only one. An example is shown in Figure 2. In this example, the geometry field holds a Cube node which is of size 1 in each direction; the appearance field of Shape node holds an Appearance node, which in turn holds a Material node that defines the color associated with the Cube (in this case red).

}

The example in Figure 2 gives a textual description of a portion of a scene that is based on VRML. MPEG-4, however, only specifies a binary description of a scene. The translation from a textual to a binary description is almost direct. As an example, consider the simple scene portion in Figure 3, and the following explanation of its BIFS representation.

}

The BIFS representation of the scene in Figure 3 would consist of the following:

1. a header that contains some global information about the encoding;
2. a binary value representing the Transform node;
3. a bit specifying that the fields of the Transform node will be specified by their index, rather than in an exhaustive list;
4. the index for the ‘translation’ field;
5. a binary encoding of the SFVec3f value 1 0 0 (since there is no quantization defined here, this encoding consists of three 32-bit values; during decoding, the decoder knows the type of the field it is reading and thus knows how many bits to read and how to interpret them);
6. the index of the children field of the Transform node;
7. the binary representation of the Shape node, which is:
1. a binary value for the Shape node;
2. a bit specifying that all of the fields of the Shape node and their values will be listed sequentially rather than by index/value pairs;
3. a binary representation for the Cube node which is:
1. a binary value for the Cube node;
2. a bit specifying that the fields of the Cube will be specified by index;
3. the index of the ‘size’ field;
4. a binary encoding of the SFVec3f value 1 1 1;
5. a bit specifying that no more fields for the Cube node will be sent;
4. a binary value for the Appearance node, followed by its encoding, omitted here;
8. a bit terminating the list of fields for the Transform node.

VRML and BIFS both have a mechanism for reusing nodes. For example, once a wheel is defined as a collection of geometric nodes collected inside a Group node, it is possible to reuse the wheel elsewhere in the scene, rather than copying it explicitly wherever it is to appear. In VRML, this is done using the DEF and USE keywords. A node is given a DEF name, and it is inserted into the scene graph wherever the same name appears after a USE statement. In BIFS, a bit is used to determine if the node has an ID, and when this bit is set, it is followed by a certain number of bits that hold an integer ID value. The number of bits used in the scene to specify IDs is given in a header that is sent at the beginning of the scene (in fact, contained in the decoder configuration information within the Object Descriptor through which the BIFS stream was accessed).

MPEG-4 is designed to be used in broadcast applications and in interactive and one-on-one communication applications. To fit this requirement, an important concept developed within MPEG-4 BIFS is that the application itself can be seen as a temporal stream. This means that the presentation, or the scene itself, has a temporal dimension. On the web, the model used for multimedia presentations is that a scene description (for instance an HTML page or a VRML scene) is downloaded once, and then played locally. In the MPEG-4 model, a BIFS presentation, which describes the scene itself, is delivered over time. The basic model is that an initial scene is loaded and can then receive further updates. In fact, the initial scene loading itself is considered an update. The concept of a scene in MPEG-4, therefore, encapsulates the elementary stream(s) that convey it over time.

The mechanism with which BIFS information is provided to the receiver over time comprises the BIFS-Command protocol (also known as BIFS-Update), and the elementary stream that carries it is thus called BIFS-Command stream. BIFS-Command conveys commands for the replacement of a scene, addition or deletion of nodes, modification of fields, etc. For example, a "ReplaceScene" command becomes the entry (or random access) point for a BIFS stream, exactly in the same way as an Intra frame serves as a random access point for video. A BIFS-Command stream can be read from the Web as any other scene, potentially containing only one "ReplaceScene" command, but it can also be broadcast as a "push" stream, or even exchanged in a communications or collaborative application.

BIFS commands come in four main functionalities: scene replacement, node/field/route insertion, node/value/route deletion, and node/field/value/route replacement. The commands enable the following operations:

• Replacing the entire current scene with a new one. When a BIFS Replace Scene command is received, the whole context is reset, and a new scene graph corresponding to the new BIFS scene is constructed.
• Insertion command: This command comes in three subtypes:
• Node insertion: Nodes can be inserted into the children field of grouping nodes. This command allows the insertion of a node at the beginning, end, or indexed position in the list of children nodes of an already existing node.
• Indexed Field insertion: with this update command, a generic field is inserted into a specified position in a multiple value field.
• ROUTE insertion: This can be used to enable user interaction or other dynamic functionality in the scene by linking event source and sink fields in the scene.
• Deletion command: This command also comes in three subtypes, analogous to the insertion command:
• Node deletion: Simply deletes the identified node. Note that it is possible to delete a node that does not have an ID (i.e., it has not been DEF-ed) by using the IndexedValue deletion command (see below).
• IndexedValue deletion: This command allows the deletion of a specified entry in a multiple value field.
• ROUTE deletion: Simply deletes a ROUTE.
• Replacement command: This command also comes in the standard three flavors, plus one more for replacing single value field values:
• Node replacement: Replaces an existing node in the scene with a new one.
• Field replacement: Changes the value of a field within a node. This command can be used, for example, to change a color, a position, the vertices of a mesh, or to switch an object on and off.
• IndexedValue replacement: This command is very similar to the field replacement command, except that the field referred to is a multiple valued one, and hence the command also includes indexing information in order to identify the particular value that should be replaced.
• ROUTE replacement: Replaces an existing ROUTE with a new one.

In the previous section, we described how it is possible to trigger a key frame animation by downloading interpolator data in the scene along with an appropriate TimeSensor. This can be used, for example, to modify the fields of a Transform node in order to move its children around. An alternative method is to use BIFS-Command to update the fields of a Transform node. These methods are well suited for small animations. However, when the source of animation is a live one, or when better compression is sought, MPEG-4 provides an alternative way to stream animation to the scene with the BIFS-Anim tool. BIFS-Anim enables optimal compression of the animation of all parameters of a scene: 2D and 3D positions, rotations, normals, colors, scalar values, etc.

The BIFS-Anim framework works as follows:

• The BIFS scene is loaded with objects that have been DEF’ed, i.e., assigned a unique node ID.
• An Animation mask is loaded, containing the list of nodes and fields to be animated. When multiple fields are animated, it is possible to select the field to be animated.
• The animation stream itself is streamed, containing animation frames in time-stamped access units.

The field values in the stream can consist of initial values (Intra-frames), used to set or reset the field values, and difference values (Predictive-frames), used to successively modify the previous field value. The Intra- and Predictive-frames are further arithmetically coded to give a highly compressed representation of the animation values.

MPEG-4 Systems supports scripting in the same way that VRML does. However, whereas VRML allows both inlined JavaScript (recently standardized as ECMAScript) as well as URLs pointing to externally available Java class files, MPEG-4 currently only allows JavaScript to be used. Scripts in MPEG-4 have three primary uses:

• Scripts can manipulate the scene directly, for example by routing output values of the script to other parts of the scene.
• Scripts can store state information in local variables. For example, if the last mouse click position is needed, it can be routed to a script that will hold the value until it is required (perhaps by another script).
• Scripts can do type conversion. For example, if only the red component of a color is needed for some operation, a script can extract this value and convert it from a component of an SFColor to an SFFloat.

As mentioned earlier, nodes and ROUTEs in BIFS can receive unique identifiers or IDs. In MPEG-4 terminals, the following scoping rules apply for the ID name spaces:

• A BIFS-Command stream shares the name space across the whole stream.
• An Inlined BIFS-Command stream (i.e., a BIFS-Command stream accessed through an Inline node) opens a new name space.
• Multiple BIFS-Command streams referenced by the same Object Descriptor structure share the same name space.

The BIFS Node are grouped in semantic categories.

In Version 1, MPEG-4 provides the ability to considerably enhance the basic VRML 2.0 sound model. In Version 2, MPEG-4 will go further and provide new sound features:

2. Enhanced physical sound rendering model, so that more natural sound source and sound environment modeling is made possible in BIFS. The functionality of the new sound source includes modeling of air absorption and more natural distance dependent attenuation, as well as sound source directivity modeling. In Version 2, BIFS also takes into account the response of the environment, so that the sound can be rendered to correspond to the visual parts of the scene. For example, polygons representing walls cause sound to be attenuated.
3. An enhanced sound -rendering model, based on geometry-independent perceptual parameters. These perceptual parameters provide high level interfaces to control the "aural aspect" of the sound rendering. Parameters such as the source presence, heaviness, envelope, brilliance and warmth can be controlled.

MPEG-J is a set of Application Programming Interfaces that allow Java code to communicate with an MPEG-4 player engine. By combining MPEG-4 media with safe executable code, content creators may embed complex control and data processing mechanisms with their media data to intelligently manage the operation of the audio-visual session.

MPEG-J defines the following interfaces:

• Scene graph API (similar to the VRML EAI), that enables scene control.
• Network API, to control the up- or down-loading of content.
• Decoder API, to control media decoders.
• Devices API, to control various input devices.
• System capability APIs, in order to be able to assess the status of the system in terms of memory, CPU load, etc.

In this section we show a complete example of a complex 2D/3D scene and its corresponding scene graph.

The example illustrates some of the 2D and 3D mixing capabilities of MPEG-4 through the concept of Layers as well as the corresponding scene graph. Layers can be used to determine a region of the screen in which a scene is drawn.

In the example, shown in Figures 9 and 10, a 2D scene and 3D scene are displayed simultaneously. The 3D scene is displayed with two different viewpoints, using 3Dlayer-1 and 3Dlayer-2. The 2D scene is overlaid on top of the 3D scenes. The 2D scene is also displayed as a map in the 3D scene (in 3Dobj-3).

This scene could represent a 3D scene seen from 2 different viewpoints and a 2D MAP of the world displayed both facing the viewer and in the 3D scene.

OrderedGroup{ children [

DEF 2Dlayer-1 Layer2D { children [

DEF 2DScene-1 Transform2D { children[

DEF 2DObj-1 Transform2D { children [

DEF 2Dobj-3 …

DEF 2Dobj-4…..

]}

DEF 2Dobj-2 Shape{….}

]}

]}

Transform2D { # Transforms to translate and scale the layer

children [

DEF 3DLayer-1 Layer3D {

viewpoint DEF v1 Viewpoint {…}

children [

DEF 3Dscene-1 Transform2D { children [

DEF 3DObj-1 Transform { children [

DEF 3Dobj-3 Transform{ children USE 2Dscene-1}

DEF 3Dobj-4…..

]}

DEF 3Dobj-2 Shape {……}

]}

]

}

]

}

Transform2D {

# Another set of transforms to translate and scale the layer3D-2

children [

DEF 3Dlayer-2 Layer3D {

Viewpoint DEF v2 Viewpoint {…….}

children [

DEF 3Dscene-2 USE 3DScene-1

]

}

]

}

]}

BIFS takes advantage of context dependency in order to compress the scene efficiently at the node and the field level. Whereas VRML treats all nodes as being of type SFNode or MFNode, BIFS makes use of the fact that only certain nodes can appear as children of other nodes. This allows the binary specification of the children nodes to be more efficient.

BIFS introduces the concept of a Node Data Type (NDT). There are some 30 different NDTs, and each node belongs to one or more of them. The NDT table consists of a list of nodes for each NDT. In each node data type, the node receives a fixed-binary-length local ID value that corresponds to its position in the NDT table. For example, the Shape node can appear in both a 3D context and a 2D context. It thus has both the SF3DNode and SF2DNode node data type (in fact, as with all nodes, it also has the global SFWorldNode NDT). In a 3D context, the Shape node can appear in the children field of a Transform node – this field has the type MF3Dnode and it accepts nodes of type SF3DNode. In a 2D context, the Shape node can appear in the children field of a Transform2D node – this field has type MF2DNode. The binary ID code for a Shape node thus depends on the context in which it appears. When it is an SF3DNode, it has a 6-bit binary ID value of 100100 (decimal value 36), because it occupies position number 36 in the list of 48 total SF3DNodes. In the 2D case, its ID is represented by 5-bits with value 10111. There are only 30 different SF2DNodes, requiring only 5 bits to specify them all.

This node representation is quite efficient. It doesn’t make sense to apply entropy-coding techniques based on the probabilistic distribution of nodes in scenes – such data does not currently exist. Moreover, as the ID value 0 is reserved as an escape value for future extensions, and because a fractional bit arises from the fact that the number nodes in each category is not a power of 2, the coding would be slightly sub-optimal regardless.

At the field level, BIFS introduces types that do not exist in VRML. These exist only for coding purposes and are cast into some of the basic field types (see 3.2) at runtime:

• The SFURL type is used for representing URLs. VRML uses a string to encode URLs, but BIFS requires a finer resolution and allows URLs to be encoded either as strings or as references to stream IDs (object descriptors).
• The SFScript type is used to encode scripts. VRML uses URLs to hold scripts, which are typically either externally accessible Java class files or in-lined JavaScript (or, more precisely, ECMAScript). In BIFS, only JavaScript is supported and it is represented using a binary format. The SFScript type allows either an MPEG-4 URL to be encoded (which allows full compatibility with VRML) or a binary script representation.
• The SFBuffer type is used to encode buffers in the MPEG-4 Conditional node. This special field holds a binary representation of a BIFS-Update command that can be triggered using the ROUTE mechanism.

Each node is associated with a Node Coding Table (NCT). Each node’s NCT holds information related to how the fields of the node are coded. The NCT specifies the type of values that each field can hold. Thus, for example, the NCT for the Transform2D node specifies that its children field has type MF2DNode and thus can hold a list of SF2DNode nodes. The NCT also specifies the other field types that can occur, for example, SFFloat, SFInt32, etc.

The NCT also specifies an index for each field according to one of four usage categories. These categories are:

• DEF – used for defining field values when a node is transmitted. This corresponds to the Field and ExposedField modes, since these are the only modes that have values that can be specified.
• IN – used for data that can be modified using BIFS updates or ROUTEs. This corresponds to the EventIn and ExposedField modes.
• OUT – is used for the EventOut and ExposedField modes. That is, fields that can be used as input values for ROUTEs.
• DYN – is a subset of the IN category, used for BIFS-Animation, discussed in Section 4.4.

By creating these 4 categories, a BIFS scene always references fields with the minimal number of bits needed. For example, the IndexedFaceSet node is used to hold a collection of polygons that form a 3D object. It has 13 fields that can be defined when it is created, and thus each field requires 4 bits when it is indexed. However, this node only has 4 fields that can output values to a ROUTE, so that only two bits are needed to specify which field is being used when this node is routed from. The IndexedFaceSet node has 8 fields that can accept ROUTEd values or be updated using BIFS-Command, and thus these protocols need only 3 bits to specify the modified field. The Node Coding Table for the Fog node is shown in Figure 11.

The NCT also defines a quantization type for each field that holds numerical values. This is the topic of the next section.

Quantizing BIFS scenes is achieved using the QuantizationParameter node. This node affects the quantization of field values for all nodes that appear hierarchically after or beneath it (i.e., its siblings and children). Using a node to convey the quantization parameters makes use of the already existing BIFS framework to encode the parameters efficiently and takes advantage of specific reuse mechanisms (VRML DEF/USE) in order to reuse locally existing parameters. This mechanism also allows content creators to fine-tune the compression of their content when it has special redundant characteristics.

Quantizing BIFS data efficiently is complex because no clear statistics can be derived from the source data. The problem is to find the best trade off between the declaration cost of the local quantization parameters using the QuantizationParameter node and the gain brought by this quantization. To overcome this difficulty, every numerical field in each node is classified into one of 14 quantization categories (see Table 2). The idea is that each category, for example "Position 3D", will contain data in a similar range of values, and that a minimal number of categories enable a minimal number of parameters addressing the maximum number of parameters. A value-range and number of bits for a linear quantizer for each quantization type is specified in the QuantizationParameter node. When a field is to be encoded, its quantization type is looked up in the NCT for the field’s node. The values of the field are then coded using the number of bits and value-range specified for that quantization type in the QuantizationParameter node.

Figure 12 shows a scene portion in which a QuantizationParameter node affects the quantization of the node that follows it. In this example, the original VRML text has unlimited precision, but, because the keyValue field of the PositionInterpolator node has "position3D" quantization type in the NCT, the binary BIFS representation of this field’s values would use only 4 bits per number. For example, the last coordinate of the last keyValue would be stored using binary value 1111, since it is the maximum value in the range specified in the QuantizationParameter node. The key values would be encoded using 8 bits, because this is the default behavior of the QuantizationParameter node for the quantization type of this field. Since the QuantizationParameter node affects all the nodes that follow it, if there were any other nodes with fields in the position3D category and with values outside the range given in the QuantizationParameter node, those values would be clipped.

The QuantizationParameter node has several other features. It has a useEfficientFloat field that holds a boolean value. When this value is true and when floating point values are not quantized in one of the quantization categories described above, they will be quantized using MPEG-4 specified "efficientFloat" coding that has less resolution than standard floating point representations and which requires fewer bits per number, especially for 0 values. The details of this coding are beyond the scope of this chapter, but the idea is that a small number of bits is used to specify how many bits are used for the exponent and mantissa of the floating point value. Values with small exponents and easily specifiable mantissas can then be stored using a smaller number of bits overall. However, the coding is limited to a 15 bit mantissa and a 7 bit exponent.

The QuantizationParameter node has another boolean field, isLocal, that specifies if the quantization parameters provided in the node should apply only to the next node in the scene graph. This design allows the declaration of quantization parameters to be factored and applied to the maximum amount of data.

The BIFS-Command protocol represents its four top-level functionalities using 2 bits:

• Replace the whole scene with this new scene;
• Insertion command;
• Deletion command;
• Replacement command.

The insertion, deletion, and replacement commands can insert, delete, and replace nodes, indexed values of MF fields, and ROUTEs. The replacement command can also replace a field value, bringing the total number of replacement comments to four, which makes that command completely efficient in its use of the 2 bits.

Figure 13 shows a textual representation of a BIFS update command that can be applied to the scene in Figure 4. After this command is sent to the MPEG-4 client, the scene would look almost identical, except with the new recent news headline replacing the previous field in the Text node.

The (simplified) binary representation of this update would consist of the following:

1. Two bits (with binary representation 10) indicating that the update is a replacement command.
2. Two bits (with binary representation 00) specifying that the update is replacing a Node (as opposed to an MFField element or a ROUTE).
3. A number of bits specifying the node ID of the node that is to be replaced. The number of bits is specified in a header that is sent at the top of the scene. In the example, the node ID is the string "MyHeadline," and this must be mapped uniquely to an integer in the BIFS representation.
4. A Binary representation of the Text Node that contains the new node. The specification of the node type is made by sending an index into the NDT table. In this case, the SFWorldNode NDT which contains all of the MPEG-4 nodes is used. We discuss this further below.

The effect achieved in the example of Figure 13 can also be produced by just replacing the value of the string field of the Text node. In this case the binary representation of the replacement command would consist of the following:

1. Two bits (with binary representation 10) indicating that the update is a replacement command.
2. Two bits (with binary representation 01) specifying that the update is replacing a field
3. A number of bits specifying the node ID of the node containing the field to be replaced.
4. Two bits (with binary representation 00) specifying that the string field is to be replaced. There are four replaceable (IN) fields in the Text node, and string is the first. Note that in order to know this the decoder must first read the node id, look up the node type of the node with this id, and use the Node Coding Tables to determine how many IN fields this node has.

1. A Binary representation of string field. Since the decoder knows which field is to be replaced, it can look in the Node Coding Table to determine the field type and hence know what type of data to expect.

}

The BIFS Animation (or BIFS-Anim) protocol uses an arithmetic coder applied to a sequence of difference values that are computed from a sequence of animation parameter values. The binary animation data consists of an Animation Mask that contains a collection of Elementary Masks, one for each node that is animated. This is followed by a stream of Animation Frames that contain the animation data. Only updateable nodes (i.e., nodes with IDs) can be animated, because the Elementary Masks use these IDs to specify which nodes are animated. Each Elementary Mask also specifies which of the animated nodes’ fields are animated and initial quantization parameters for the animation. Finally, when the animated fields are multiple value fields, the Elementary Mask specifies the indices of the elements that are to be animated.

Only fields that have a DYN index defined in the Node Coding Tables can be animated. These dynamic fields may be of the numerical data types:

• SFInt32/MFInt32
• SFFloat/MFFloat
• SFRotation/MFRotation
• SFColor/MFColor
• SFVec2f/MFVec2f
• SFVec3f/MFVec3f

The Animation Frames specify the values of the animated fields. An Animation Frame can contain new values for a subset of the animated fields at a specified time. That is, all the fields or a selection of fields can be modified at each specified time. The field values can be sent in Intra (the absolute value is sent) and Predictive modes (the difference between the current and previous values is sent).

In Intra-mode, the field values are quantized (without any entropy coding) using the quantization parameters defined for the animated field in the associated Elementary Mask. In Predictive-Mode, the difference with the previous sample is computed and then entropy coded. Figure 14 summarizes the decoding steps.

The quantization is similar to the one used in the BIFS compression of field values. In BIFS Animation, as for BIFS scene compression, the notion of quantization category is used. The following animation categories are defined:

• Position 3D
• Positions 2D
• Color
• Angle
• Float
• BoundFloat
• Normals
• Rotation
• Size 3D
• Size 2D
• Integer

However, the animation categories correspond more to data types rather than a semantic grouping, since quantization parameters are not shared but declared individually for each field to be animated. Each category has a specific syntax for declaring its quantization parameters: Min and Max values, number of bits, in Intra and Predictive modes. Computing the difference of quantized values is direct except for normalized vectors such as rotations and normals. Rotations are coded using quaternions, and a specific prediction mechanism enables coding of rotations and normals in a predictive mode, just as with the other data types. The entropy coding uses an adaptive arithmetic encoder, which avoids the issue of unknown data statistics.

Finally, the BIFS-Anim elementary stream will contain animation frames, containing:

• The AnimFrameHeader, containing some specific information to enable the choice of nodes animated in the frame (the Transfrom or Coordinate node animation could be switched off at any frame), a number of skipped frames, timing information, etc….
• The AnimationFrameData, with:
• The I or P value of the Transform rotation;
• The I or P value of the Transform translation;
• The I or P value of the Coordinate node point selected positions.

The authors would like to thank Shout Interactive Inc. (www.shoutinteractive.com) for providing the visual material that appears in Figures 16 and 17.

Julien Signès graduated from Ecole Polytechnique, Paris, 1992 and from Ecole Nationale Superieure des Telecommunications in 1994. For three years, he worked as a member of the research staff in the multimedia division of CNET, the France Telecom research center, in Rennes, France. During that period, he worked on fractal still image compression, and was then deeply involved in the MPEG-4 and VRML standardization process, as one of the main contributors to the MPEG-4 scene description language, BIFS. Currently, he works at ThinkOne, the San Francisco based R&D unit of France Telecom and Deutsche Telekom, where he leads a project on mixed media client server services based on Java, VRML and MPEG-4.

Yuval Fisher has extracted various degrees from universities, most recently a Ph.D. in Mathematics from Cornell University. He's written and edited several books, and published a number of articles, mostly in the area of Fractal Image Encoding. Most recently, he has actively participated in the MPEG-4 standardization process. He is currently a visiting scholar at the institute for nonlinear science at the University of California, San Diego.

Alexandros Eleftheriadis

Alexandros Eleftheriadis was born in Athens, Greece, in 1967, and received his Ph.D. in Electrical Engineering from Columbia University in 1995. Since 1995 he has been an Assistant Professor at Columbia, where he is leading a research team working on media representation with emphasis on multimedia software, video signal processing and compression, video communication systems and the mathematical fundamentals of compression. Dr. Eleftheriadis is a member of the ANSI NCITS L3.1 Committee and the ISO-IEC JTC1/SC29/WG11 (MPEG) Group, where he serves as the Editor of the MPEG-4 Systems specification. His awards include an NSF CAREER Award.