Distributed Multimedia Database Technologies Supported by MPEG-7 and MPEG-21

2017-07-07 02:10:07

< Day Day Up >

3.4 Digital Item Adaptation

The DIA (ISO/IEC 21000-7) comprises descriptions of DIA architecture and descriptions specifying all factors that may influence and that may support the adaptation process. Specifying the adaptation of a digital item (DIA) is of importance, as the access to any multimedia resource from any type of terminal or network (often referred to as universal multimedia access) in an interoperable way is desired these days. ^[9], ^[10]

There are many environmental factors that may influence the transparent access to the multimedia resource, including terminal, network, delivery, user, and natural environment factors. MPEG-21 regroups these factors and proposes a huge set of descriptors, the so-called usage environment descriptions, to specify these factors and their behavior.

In addition, MPEG-21 DIA introduces an adaptation architecture that is illustrated in Exhibit 3.5. As shown in this architecture, an input DI is adapted in the DI Adaptation Engine, containing a resource adaptation engine and a description adaptation engine. This separation is necessary, as the nature of the resource and the description associated are different and, therefore, require different adaptation processes. These engines together produce the modified DI. To achieve the goal of adaptation, it is essential to use a description of how the media resource adaptation may be performed. Therefore, DIA tools, which support the adaptation process, are the second input to the DI Adaptation Engine. A detailed overview of the DIA architecture is given in Section 3.4.1.

Exhibit 3.5: Digital item adaptation architecture.

The descriptors and tools proposed in MPEG-21 DIA are specified in the DIA Schema, which is detailed in Section 3.4.2. Finally, Section 3.4.3 describes the relationship between DIA and the other parts of MPEG-21.

3.4.1 Overview of the DIA Architecture

The DIA architecture is broken into three main areas:

Usage environment descriptions

DI resource adaptation tools

DI declaration adaptation tools

The usage environment descriptions include descriptions for user, terminal capabilities, network characteristics, user preferences, and natural environment characteristics.

Terminal capabilities include hardware properties, such as processor speed and memory capacity, and display properties, such as screen resolution and device profiles.

Network characteristics specify delay characteristics, such as end-to-end delay, one-way delay, or delay variation, and bandwidth characteristics, such as amount of available bandwidth or bandwidth variation.

User preferences include filtering and search preferences, quality of service preferences, and demographic information, such as gender and age.

Natural environment characteristics include location information, such as Global Positioning System coordinates, the type of location (e.g., indoor and outdoor), and the illumination properties affecting a user or a terminal.

The DI resource adaptation tools comprise descriptions and algorithms that target the adaptation of resources in a DI.

Bitstream syntax description (BSD) is the main descriptive element in this group. A BSD describes the syntax (called bistream) of a binary media resource. It does not, in general, detail each bit of the bitstream, but rather relies on logical units (so called syntactical units) of a bitstream. For a video, this is typically the frame level. A digital item resource adaptation engine can then transform the bitstream and the corresponding BSD using editing-style operations such as data truncation and simple modifications. This transformation can, for instance, be performed under control of an Extensible Stylesheet Language: Transformations style sheet. It is desirable that BSDs be independent of specific media resource formats. This enables adaptation of a binary resource to take place at adaptation nodes, for instance, network gateways, that are not aware of the specific resource representation format.

Terminal and network quality of service descriptors and algorithms are the second category of descriptive elements within the DI resource adaptation tools. They describe the relationship between constraints (e.g., maintaining a certain bit rate), feasible adaptation operations satisfying these constraints (e.g., dropping of video B-frames), and associated qualities (e.g., peak signal-to-noise ratio [PSNR]). The descriptors proposed provide means to make trade-offs among these parameters so that an adaptation strategy can be formulated.

Metadata adaptability descriptors are the third category of descriptors. They enable the filtering and scaling of XML instances with reduced complexity.

DID adaptation tools contain descriptions to specify session mobility, DID configuration preferences, and DIA description messages:

Session mobility descriptions specify the configuration state information that pertains to the consumption of a DI on one device when it is transferred to a second device. This enables the DI to be consumed on the second device in an adapted way.

DID configuration preferences provide mechanisms to configure the choices within a DID according to a user's intention and preference. This enables operations to be performed on elements within the DID, such as sorting and deleting.

3.4.2 DIA Schema

The DIA Schema assumes that a schema wrapper is provided that identifies the XML Schema namespace: urn:mpeg:mpeg21:dia:schema:2003. Two root elements for DIA descriptions are defined:

DIA, if a complete description is used. A complete description is self-contained (see also the definition of a complete description in MPEG-7, Chapter 2, Section 2.5.3).

DIADescriptionUnit, if a single DIA descriptor is used. A DIADescriptionUnit can be used to represent partial information from a complete description; for example, the description of only the display preferences of the user.

Exhibit 3.6 shows the top-level types that may be used in DIA descriptions (only the DIA root for complete descriptions is shown in Exhibit 3.6). Information and examples of use for the top-level types follow.

Exhibit 3.6: Top-level types in the digital item adaptation schema.

UsageEnvironmentType is the top-level type for describing a usage environment description. Elements of this type describe the user preferences, terminal, network, and natural environment characteristics.

Example: The static and dynamic capabilities of a sample network are described below. The static capabilities of the network are specified by using the capability element and the attributes maxCapacity and minGuaranteed (bandwidth in bps). The dynamics of the network are described by a condition and its subelement, utilization. Utilization contains means for specifying the current delay and error characteristics of the network.

BSDType is the top-level type for the BSDs. These descriptions are not meant to replace the original binary format, but to act as an additional descriptive layer. Thus, they may be embedded as descriptors of a component in a DI. BSDs are employed for a more efficient adaptation process. This is because of the more compact representation of the BSD compared with the described bitstream (up to 98 percent of the bitstream size). ^[11]

Example: Exhibit 3.7 shows an extract of the BSD for the Coast Guard video from the MPEG-4 reference streams. We use for this example our standardized gBSD (generic BSD) schema, ^[12] which is independent of specific media resource formats. The example shows the use of the basic gBSD elements. The header defines the classification alias with its Uniform Resource Name. It is required to identify the meaning of the syntactical labels. The header also contains the default addressing values, which include the location of the bitstream and the mode used for addressing (absolute vs. relative). Subsequently, the description of the bitstream syntax follows. gBSDUnits are used to represent the frames in the bitstream. We have to distinguish between I-, P-, and B-frames; thus, the gBSDUnits are labeled with syntacticalLabel ":MV4:I_VOP" (for I-Frames), ":MV4:P_VOP" (for P-Frames), and ":MV4:B_VOP" (for B-Frames).

Exhibit 3.7: Example of bitstream syntax description.

In addition (not shown in Exhibit 3.7), gBSDunits may be marked to support the adaptation process with hints. For instance, a B-VOP may be labeled with "important," so that it is saved from removal in adaptation. Further (also not shown in the simple example), gBSDUnits may be grouped to form other gBSDUnits; for instance, frames to form a group of Video Object Plans (VOPs).

AdaptationQoSType is the top-level type that specifies the relationship between constraints, feasible adaptation operations satisfying these constraints, and associated utilities (qualities). Therefore, AdaptationQoS descriptions let an adaptation engine know what adaptation operations are feasible for satisfying the given constraints and the quality resulting from each adaptation.

Example: The following fragment gives a simple AdaptationQoS example for the constraint "bandwidth." Two constraint points are specified, one for 1359 kbps (kilobits per second) and one for 1071 kbps. For a better readability, these constraint points are highlighted in the XML fragment below. For each of these points, an adaptation operator is declared to meet the constraints. In the first case, we have to perform a Coefficient Dropping (specified by the use of the element FrameCoeffDropping with the attributes as given below), and the second operator is B-Frame Dropping (here the attribute coeffDroppingRatio is set to 0.0 and the attribute numberOfBFrame to 1). The UtilityValue element specifies the quality of the adapted video in the metric given in attribute "name" of the element utility. The metric used in the example is the PSNR in decibels.

MetadataAdaptationHintType is the top-level type for describing adaptation hint information pertaining to metadata within a DI. This hint information is a set of syntactical elements with prior knowledge about the metadata that is useful for reducing the complexity of the description adaptation process.

Example: The following example shows the use of the MetadataAdaptationHint, including information about the metadata of a video resource. InstanceFileSize expresses the actual file size of metadata for a video resource, and TotalNumOfElements expresses the number of elements that are included in the metadata. The following hint informs the adaptation engine that there are 10 video segments in the metadata and that each of the video segments does not contain any temporal or spatial sub-segments (elements MaxTemporalDepth and MaxSpatialDepth have the value 1). A description adaptation engine, which has to react to a resource constraint, may compute from this information the number of video segments to be removed from the description.

The remaining top-level types belong to the group of DI declaration adaptation tools, and they address the complete adaptation chain:

SessionMobilityType is the top-level type that specifies a mechanism to preserve a user's current state (which is defined by the state of selection elements) of interaction with a DI. Examples of use include providing a mechanism to transfer a user's session between different terminals.

DIDConfigurationPreferencesType is the top-level type that defines descriptions to express the user's intention of a DID choice configuration preferences.

3.4.3 Relation between DIA and Other Parts of MPEG-21

The fundamental unit of distribution and transaction in the MPEG-21 multimedia framework is the DI. Although the existing parts of ISO/IEC 21000 deal with different aspects of the DI, together they facilitate the complete multimedia framework. It is therefore crucial to understand the relationships between MPEG-21 DIA and the other parts to be able to achieve an interoperable framework. Exhibit 3.8 gives an overview of the DIA with respect to concepts defined in related MPEG-21 parts.

Exhibit 3.8: Digital item adaptation detailed with respect to the other MPEG-21 parts.

The input to the DIA Engine is a Content DI (CDI). The prefix Content is added to this DI to indicate that a resource is referenced and that descriptions for this resource are available and subject to adaptation. DIA tools may be embedded into the CDI using Part 2 of ISO/IEC 21000, the DID. For instance, the CDI may incorporate a BSD for the resource bitstream. The resource adaptation engine relies on the BSD to perform the adaptation. It transforms the BSD, for example, with an Extensible Stylesheet Language: Transformations style sheet, and then generates back an adapted bitstream. Another example of a DIA description that is useful for embedding into the CDI is an AdaptationQoS description offering a set of adaptation operations for different resource constraint points.

The type of the DI is indicated using descriptions provided by ISO/IEC 21000-3 and 5, the DII, and the REL. For instance, a particular user owing the rights of the CDI might define, in an adaptation specific license, the permissible types of adaptations that are allowed; for example, the bit rate should not be <1500 kbps.

DIA descriptions can also be carried in a Context DI (XDI). A XDI carries only descriptions to support the adaptation process and contains no resources. As shown in Exhibit 3.8, the XDI is used as input to the DIA Engine. The DIA descriptions in an XDI drive the adaptation and can provide parameters for both adaptation engines (resource and description).

These descriptions are responsible for steering the adaptation process. For instance, a new network capacity may be specified by using the capability element of the network. This new situation is incorporated into an XDI and put into the DIA Engine. The DIA Engine analyzes the content of the XDI and decides to parse an available AdaptationQoS description (e.g., from the CDI) to obtain an appropriate adaptation operator, which is then applied to the resource.

The REL expressions of the XDI can be used for protection of user-sensitive information that may be conveyed by the DIA descriptions.

The DIA engine operates on the input CDI according to the DIA descriptions carried in the XDI, as well as the DIA descriptions that are carried in the CDI itself. The DID Identifiers, REL elements, DIA elements, as well as the resources themselves are all subject to change after being processed by the DIA engine and will be contained in the adapted CDI, which is the output of the DIA Engine.

^[9]Panis, G., Hutter, A., Heuer, J., Hellwagner, H., Kosch, H., Timmerer, C., Devillers, S., and Amielh, M., Binary multimedia resource adaptation using XML bitstream syntax description, Image Communication, 18(8), 721–747, 2003.

^[10]Vetro, A. and Devillers, S., Position Paper: Delivery Context in MPEG-21, W3C Delivery Context Workshop, March 2002, http://www.w3.org/2002/02/DIWS/agenda.html.

^[11]Panis, G., Hutter, A., Heuer, J., Hellwagner, H., Kosch, H., Timmerer, C., Devillers, S., and Amielh, M., Binary multimedia resource adaptation using XML bitstream syntax description, Image Communication, 18(8), 721–747, 2003.

^[12]Hill, K. and Bormans, J., Overview of the MPEG-21 standard. ISO/IECJTC1/SC29/WG11 N4041 (Shanghai Meeting), October 2002, http://www.chiariglione.org/mpeg/.

< Day Day Up >

3.4 Digital Item Adaptation

3.4.1 Overview of the DIA Architecture

3.4.2 DIA Schema

3.4.3 Relation between DIA and Other Parts of MPEG-21

Категории