The ISO/MPEG standardization process

Requirements Call for proposals Evaluation Core experiments Draft specification National bodies agree

Some major MPEG-4 requirements

Initial goal – utmost compression efficiency In 1994 – refocussing on ____________-based fu

nctionalities – _______-based representation of audio-visual data – ________-side composition of AV objects – Allow user ___________ with AV objects – Allow downloading and streaming – Error __________ bitstream syntax – Provide ___________ coding – …

The structure of the MPEG-4 Standard (1/4)

Part 1 : Systems (IS 14496-1:2001 – w3850) Part 2 : Visual (IS 14496-2:1999 – w305

6) Part 3 : Audio (IS 14496-3:2001 – w379

8) Part 4 : Conformance (IS 14496-4:2001 – w402

4) Part 5 : Reference Software (IS 14496-5:2001 – w4025) Part 6 : Delivery Multimedia Integration Framework (DMIF)

(IS 14496-6:2000 – w3713)

Part 7 : Optimized Visual Reference Software (PDTR 14496-7 – w405

7) Part 8 : Carriage of MPEG-4 contents over IP networks

(CD 14496-8 – w4081)

The structure of the MPEG-4 Standard (2/4)

Part 2 : Visual – Still texture coding – Arbitrary _________ video coding – Synthetic coding tools (Meshes, Face & Body) – Fine Grain Scalability – …

Part 3 : Audio – General audio coding (transform coder) – Speech coding (CELP, Parametric) – Text-to-speech interface – _________ audio coding – …

Part 4 : Conformance – Conformance can only be tested for a _________ & __________ combination

Per Profile & Level Test bit streams Additional test information

The structure of the MPEG-4 Standard (3/4)

Part 5 : Reference Software – Contains all _____________ elements of Part 1 – 3 (decoders) – “Informative” part contains encoders & tools

Part 6 : DMIF - ________ Multimedia Integration Framework – Mainly the abstraction of the Delivery (Transport) Layer – The MPEG-4 world ends (begins :-) at the ____(DMIF Applicatio

n Interface) – Actual delivery adaptations mostly _________ MPEG (-4)

(MPEG-4 over IP; MPEG-4 over MPEG-2) Part 7 : Optimized Visual Reference Software

– Non-standard tools to document state of the art Part 8 : Carriage of MPEG-4 contents over IP networks

– Umbrella for specifications done by _____ (e.g. RTP payloads, SDP extensions)

The structure of the MPEG-4 Standard (4/4)

Part 1 : Systems Audiovisual Scene

Coded Representation 14496-1

Natural and Synthetic Audio Information Coded Representation 14496-2

Management & Synchronization of Audiovisual Information

14496-1

Natural and Synthetic Visual Information Coded Representation 14496-2

An example of MPEG4 scene

Object-based compression and delivery

Comp-ress

Comp-ress

Comp-ress

Comp-ress

DE LIVERY

INTERFACE

(DAI)

Comp-ressScene Des.

decomp-ress

decomp-ress

decomp-ress

decomp-ress

DE LIVERY

INTERFACE

(DAI)

decomp-ress

COMPOSITOR

Linking elementary streams to the scene

Linking streams into the scene

ObjectdescriptorID

ObjectdescriptorID

Linking streams into the scene

ObjectdescriptorID

ObjectdescriptorID

Linking streams into the scene

ObjectdescriptorID

ObjectdescriptorID

ES_Descriptor { ES_ID_4 ....... }

ES_Descriptor { ES_ID_5 ....... }

ObjectDescriptor { OD_ID_1 List of { Elementary.....

ObjectDescriptor { OD_ID_2 List of { Elementary Stream Descriptors } }

Linking streams into the scene

ObjectdescriptorID

ObjectdescriptorID

ES_Descriptor { ES_ID_1 ....... }

ES_Descriptor { ES_ID_2 ....... }

ObjectDescriptor { OD_ID_1 List of { Elementary Stream Descriptors } }

ObjectDescriptor { OD_ID_2 List of { Elementary Stream ……….

ES_Descriptor { ES_ID_3 ....... }

Linking streams into the scene

ObjectdescriptorID

ObjectdescriptorID

ObjectDescriptor { OD_ID_1 List of { elementary stream descriptor}

ObjectDescriptor { OD_ID_2 List of { elementary stream descriptor }}

• URLs in ESD point to remote stream• All description local

ES_Descriptor { URL ....... }

ES_Descriptor { URL ....... }

Linking streams into the scene

An object descriptor containsES descriptors pointing to:

– _________ coded content streams – Alternate _____ content streams – Object content __________ – ____ information

ES descriptors have subdescriptors for:– ________ configuration– _______ layer configuration– ____________ information– Future / private extensions

Allows the terminal to select suitable streams

“stream type” “stream header”

Configures the flexible SL syntax

Mainly for heteroge- neous networks

Describing scalable content

Describing alternate content versions

Decoder configuration info in older standards

Cfg = configuration information (“stream headers”)

Decoder configuration information in MPEG-4

The Initial Object Descriptor

Derived from the generic object descriptor – Contains additional elements to signal ________ and ______

Profile and level indications are the default way of content selection

– The terminal reads the P&L indications and knows whether it has capability to process the presentation

Profiles are signaled in multiple separate dimensions – ______________ – Graphics – Object descriptors – Audio – Visual

The “first” object descriptor for an MPEG-4 presentation is always an initial object descriptor

Transport of object descriptors

Object descriptors are encapsulated in OD _____________s – ObjectDescriptorUpdate / ObjectDescriptorRemove – ES_DescriptorUpdate / ES_DescriptorRemove

OD commands are conveyed in their own object descriptor _________ in a synchronized manner with time stamps

– Objects / streams may be announced during a presentation There may be __________ OD & scene description streams

– A partitioning of a large scene becomes possible Name scopes for identifiers (OD_ID, ES_ID) are defined

– Resource management for sub scenes can be distributed A propose “resource management”

– If the location of streams is changed, only the ___s need modification.Not the scene description

Initial OD pointing to scene and OD stream

Initial OD pointing to a scalable scene

Ancilliary streams

IPMP streams – Information for Intellectual Property Management and Protection – Structured in (time stamped) messages. – Content is defined by ____________ IPMP systems – Complemented by IPMP ___________s

OCI (Object Content Information) streams – Meta data for an object. – Structured descriptors conveyed in (time stamped) _________s – Content author, date, keywords, description, language, ... – Some OCI descriptors may be directly in __s or ___s

ES_Descriptors pointing to such streams may be attached to any _________ descriptor

– This scopes the _____ or ___ stream. – An IPMP stream attached to the object descriptor stream is valid for _

__ streams

Adding an OCI stream to an Audio stream

Adding OCI descriptors to Audio streams

Linking streams to the scene - including “upstreams”

ObjectdescriptorID

ObjectDescriptor { OD_ID_2 ....... ES downstream ES upstream}