read.pudn.comread.pudn.com/downloads75/doc/279385/15938/15938-5.doc · web viewinternational...

INTERNATIONAL ORGANIZATION FOR STANDARDIZATIONORGANISATION INTERNATIONALE NORMALISATION

ISO/IEC JTC 1/SC 29/WG 11CODING OF MOVING PICTURES AND AUDIO

ISO/IEC JTC 1/SC 29/WG 11/M6156

July 2000, Beijing

Source: MDSTitle: MPEG-7 Multimedia Description Schemes WD (Version 3.1)Status: DraftEditors: Peter van Beek, Ana B. Benitez, Joerg Heuer, Jose Martinez, Philippe Salembier,

John Smith, Toby Walker

ISO/IEC TC JTC1/SC 29 N xxxxDate: 2000-05-3

ISO/IEC TC JTC1/SC 29/WG 11

Secretariat:

Information Technology — Multimedia Content Description Interface — Part 5: Multimedia Description Schemes

Élément introductif — Élément central — Partie 5 : Titre de la partie

Warning

This document is not an ISO International Standard. It is distributed for review and comment. It is subject to change without notice and may not be referred to as an International Standard.

Recipients of this document are invited to submit, with their comments, notification of any relevant patent rights of which they are aware and to provide supporting documentation.

Document type: International StandardDocument subtype: Document stage: (20) PreparatoryDocument language: E

/tt/file_convert/5e7a22db6194c70b7a789fab/document.doc STD Version 1.0

Copyright notice

This ISO document is a working draft or committee draft and is copyright-protected by ISO. While the reproduction of working drafts or committee drafts in any form for use by participants in the ISO standards development process is permitted without prior permission from ISO, neither this document nor any extract from it may be reproduced, stored or transmitted in any form for any other purpose without prior written permission from ISO.

Requests for permission to reproduce this document for the purpose of selling it should be addressed as shown below or to ISO’s member body in the country of the requester:

[Indicate :the full addresstelephone numberfax numbertelex numberand electronic mail address

as appropriate, of the Copyright Manager of the ISO member body responsible for the secretariat of the TC or SC within the framework of which the draft has been prepared]

Reproduction for sales purposes may be subject to royalty payments or a licensing agreement.

Violators may be prosecuted.

IV © ISO/IEC 2000 – All rights reserved

Contents

List of Figures............................................................................................................................................... viii1 Scope................................................................................................................................................... 11.1 Organization of the document...........................................................................................................11.2 Overview of MDSs............................................................................................................................... 12 Normative references.........................................................................................................................33 Terms, definitions, symbols, abbreviated terms..............................................................................44 Datatypes and basic structures.........................................................................................................54.1 Integer Datatypes................................................................................................................................ 54.1.1 Datatype Syntax.................................................................................................................................... 54.1.2 Datatype Semantics.............................................................................................................................. 74.2 Vectors and matrices..........................................................................................................................74.2.1 Vector.................................................................................................................................................... 74.2.2 Matrix.................................................................................................................................................... 94.2.3 Diagonal Matrix................................................................................................................................... 114.3 Probability Datatypes....................................................................................................................... 124.4 Histograms........................................................................................................................................ 124.4.1 Histogram D........................................................................................................................................ 124.5 Quantizers.......................................................................................................................................... 134.5.1 Uniform Quantization D....................................................................................................................... 13

5 Link to the media and Localization..................................................................................................155.1 References to Ds and DSs................................................................................................................155.1.1 Reference D........................................................................................................................................ 155.1.2 ReferenceToSegment D..................................................................................................................... 165.1.3 ReferenceToProgram D...................................................................................................................... 165.2 Unique Identifier................................................................................................................................ 175.2.1 Uidentifier D........................................................................................................................................ 175.3 Time elements................................................................................................................................... 185.3.1 TimePoint D........................................................................................................................................ 185.3.2 Duration D........................................................................................................................................... 205.3.3 IncrDuration D..................................................................................................................................... 215.3.4 RelTimePoint D................................................................................................................................... 215.3.5 RelIncrTimePoint D............................................................................................................................. 225.3.6 Time DS.............................................................................................................................................. 225.3.7 MediaTimePoint D............................................................................................................................... 235.3.8 MediaDuration D................................................................................................................................. 245.3.9 MediaIncrDuration D........................................................................................................................... 255.3.10 MediaRelTimePoint D......................................................................................................................... 255.3.11 MediaRelIncrTimePoint D................................................................................................................... 265.3.12 MediaTime DS.................................................................................................................................... 265.4 Media locators................................................................................................................................... 275.4.1 MediaURL D........................................................................................................................................ 275.4.2 MediaLocator DS................................................................................................................................ 275.4.3 VideoSegmentLocator DS................................................................................................................... 285.4.4 ImageLocator DS................................................................................................................................ 295.4.5 AudioSegmentLocator DS................................................................................................................... 295.4.6 SoundLocator DS................................................................................................................................ 29

6 Basic elements.................................................................................................................................. 316.1 Textual description........................................................................................................................... 316.1.1 Language attribute.............................................................................................................................. 31

V

6.1.2 Language Datatype............................................................................................................................. 326.1.3 ControlledTerm D................................................................................................................................ 326.1.4 Textual Description Datatype..............................................................................................................336.1.5 StructuredAnnotation DS....................................................................................................................336.2 Description of persons.....................................................................................................................356.2.1 Person DS........................................................................................................................................... 356.2.2 Person Name Datatype.......................................................................................................................356.2.3 Individual DS....................................................................................................................................... 376.2.4 PersonGroup DS................................................................................................................................. 386.2.5 Organization DS.................................................................................................................................. 386.3 Description of places........................................................................................................................396.3.1 Place DS............................................................................................................................................. 396.4 Description of importance or priority..............................................................................................406.5 Entity-relationship graph..................................................................................................................406.5.1 EntityRelationshipGraph DS................................................................................................................416.6 Description of Time Series...............................................................................................................466.7 Scalable Series.................................................................................................................................. 467 Description of the media.................................................................................................................. 477.1.1 MediaIdentification DS........................................................................................................................ 477.1.2 MediaFormat DS................................................................................................................................. 487.1.3 MediaCoding DS................................................................................................................................. 497.1.4 MediaInstance DS............................................................................................................................... 507.1.5 MediaProfile DS.................................................................................................................................. 517.1.6 MediaInformation DS..........................................................................................................................52

8 Description of the content creation & production..........................................................................538.1.1 Creation DS......................................................................................................................................... 538.1.2 Classification DS................................................................................................................................. 558.1.3 RelatedMaterial DS............................................................................................................................. 568.1.4 Creation MetaInformation DS..............................................................................................................57

9 Description of the content usage....................................................................................................599.1.1 Rights DS............................................................................................................................................ 599.1.2 UsageRecord DS................................................................................................................................ 599.1.3 Financial DS........................................................................................................................................ 609.1.4 UsageMetaInformation DS..................................................................................................................61

10 Description of the structural aspects of the content.....................................................................6310.1 Segment............................................................................................................................................. 6310.1.1 Segment DS........................................................................................................................................ 6310.1.2 VideoSegment DS............................................................................................................................... 6710.1.3 StillRegion DS..................................................................................................................................... 6910.1.4 MovingRegion DS............................................................................................................................... 7110.1.5 VideoText DS...................................................................................................................................... 7210.1.6 AudioSegment DS............................................................................................................................... 7310.2 Segment Features............................................................................................................................. 7410.2.1 MediaTimeMask DS............................................................................................................................ 7410.2.2 Mosaic DS........................................................................................................................................... 7410.2.3 MatchingHint DS................................................................................................................................. 7810.2.4 PointOfView DS.................................................................................................................................. 7910.3 Segment-relation graph....................................................................................................................7910.3.1 SegmentRelationshipGraph DS..........................................................................................................7910.3.2 Type of relationships........................................................................................................................... 81

11 Description of the conceptual aspects of the content...................................................................8311.1.1 Affective DS........................................................................................................................................ 83

12 Content navigation and access.......................................................................................................8412.1 Summarization.................................................................................................................................. 8412.1.1 Summarization DS.............................................................................................................................. 8412.1.2 Summary DS....................................................................................................................................... 8512.1.3 HierarchicalSummary DS.................................................................................................................... 85

VI © ISO/IEC 2000 – All rights reserved

12.1.4 HighlightLevel DS................................................................................................................................ 8812.1.5 HighlightSegment DS.......................................................................................................................... 8912.1.6 SequentialSummary DS...................................................................................................................... 9012.1.7 FrameProperty DS.............................................................................................................................. 9112.1.8 SoundProperty DS.............................................................................................................................. 9212.1.9 TextProperty DS.................................................................................................................................. 9212.2 Partitions and decompositions........................................................................................................9212.2.1 View DS.............................................................................................................................................. 9212.2.2 Space View DS................................................................................................................................... 9412.2.3 Frequency View DS............................................................................................................................ 9512.2.4 SpaceFrequencyView DS................................................................................................................... 9512.2.5 ResolutionView DS............................................................................................................................. 9612.2.6 SpaceResolutionView DS................................................................................................................... 9712.2.7 Filter DS.............................................................................................................................................. 9812.2.8 1D/2D-Filter DS................................................................................................................................... 9812.2.9 ViewSet DS....................................................................................................................................... 10012.2.10 SpaceTree DS................................................................................................................................... 10112.2.11 FrequencyTree DS............................................................................................................................ 10112.2.12 SpaceFrequencyGraph DS...............................................................................................................10212.2.13 VideoViewGraph DS......................................................................................................................... 10312.2.14 MultiResolutionPyramid DS...............................................................................................................10412.3 Description of variation of the content..........................................................................................10412.3.1 Variation DS...................................................................................................................................... 104

13 Organization of the content............................................................................................................10713.1 Collections ...................................................................................................................................... 10713.1.1 Collection Structure DS..................................................................................................................... 10713.2 Models.............................................................................................................................................. 11413.2.1 Model DS.......................................................................................................................................... 11413.3 Probability Models.......................................................................................................................... 11513.3.1 ProbabilityModel DS.......................................................................................................................... 11513.3.2 Gaussian DS..................................................................................................................................... 11713.4 Analytic Model................................................................................................................................. 11913.4.1 AnalyticModel DS.............................................................................................................................. 11913.4.2 Cluster DS......................................................................................................................................... 11913.4.3 Examples DS.................................................................................................................................... 12013.4.4 ProbabilityModelClass DS................................................................................................................. 12113.5 Classifiers........................................................................................................................................ 12113.5.1 Classifier DS..................................................................................................................................... 12113.5.2 ClusterSet DS................................................................................................................................... 12213.5.3 ExamplesSet DS............................................................................................................................... 12313.5.4 ProbabilityModelClassifier DS...........................................................................................................123

14 User Interaction............................................................................................................................... 12514.1 User Preferences............................................................................................................................. 12514.1.1 UserPreference DS........................................................................................................................... 12514.1.2 UserIdentifier DS............................................................................................................................... 12514.1.3 PreferenceType DS.......................................................................................................................... 12614.1.4 UsagePreferences DS...................................................................................................................... 12714.1.5 BrowsingPreferences DS.................................................................................................................. 12714.1.6 SummaryPreferences DS.................................................................................................................. 12814.1.7 FilteringAndSearchPreferences DS..................................................................................................12914.1.8 ClassificationPreferences DS............................................................................................................13014.1.9 CreationPreferences DS................................................................................................................... 13214.1.10 SourcePreferences DS..................................................................................................................... 133

15 Bibliography.................................................................................................................................... 13516 Annex 1: Schema definition...........................................................................................................136

© ISO/IEC 2000 – All rights reserved VII

List of Figures

Figure 1: Overview of the MDSs................................................................................................................................. 1

Figure 2: Multiple entities in an EntityNode...............................................................................................................46

Figure 3: Examples of segments: a) and b) segments composed of one single connected component; c) and d) segments composed of three connected components.......................................................................................66

Figure 4: Examples of SegmentDecomposition: a) and b) SegmentDecompositions without gap nor ovelap; c) and d) SegmentDecompositions with gap or ovelap.................................................................................................67

Figure 5: Illustrations of possible deformations of region a) by variation of single parameter values; for affine, perspective and quadratic model: b) translation a1; c) scale a3; d) shear a4; quadratic model deformations e) a9; f) a10; g) a7; perspective model deformations: h) a7; i) a8....................................................................................78

Figure 6: Spatial reference system in relation of frame #0 and mosaic....................................................................78

Figure 7: Temporal reference system in relation to frame #0....................................................................................78

VIII © ISO/IEC 2000 – All rights reserved

Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC participate in the development of International Standards through technical committees established by the respective organization to deal with particular fields of technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the work.

International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 3.

In the field of information technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1. Draft International Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as an International Standard requires approval by at least 75 % of the national bodies casting a vote.

Attention is drawn to the possibility that some of the elements of this part of ISO/IEC 15938 may be the subject of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.

International Standard ISO/IEC 15938-5 was prepared by Joint Technical Committee ISO/IEC JTC JTC1, , Subcommittee SC 29, .

ISO/IEC 15938 consists of the following parts, under the general title Information Technology — Multimedia Content Description Interface:

Part 1:Systems

Part 2: Description Definition Language

Part 3: Visual

Part 4: Audio

Part 5: Multimedia Description Schemes

Part 6: Reference Software

© ISO/IEC 2000 – All rights reserved IX

Introduction

The MPEG-7 standard also known as "Multimedia Content Description Interface" aims at providing standardized core technologies allowing description of audiovisual data content in multimedia environments. This is a challenging task given the broad spectrum of requirements and targeted multimedia applications, and the broad number of audiovisual features of importance in such context.

In order to achieve this broad goal, MPEG-7 will standardize:

Descriptors (D): representations of Features, that define the syntax and the semantics of each feature representation,

Description Schemes (DS), that specify the structure and semantics of the relationships between their components, which may be both Ds and DSs,

A Description Definition Language (DDL), to allow the creation of new DSs and, possibly, Ds and to allows the extension and modification of existing DSs,

System tools, to support multiplexing of description, synchronization issues, transmission mechanisms, file format, etc.

The standard will be subdivided into seven parts:

1. Systems: Architecture of the standard, tools that are needed to prepare MPEG-7 Descriptions for efficient transport and storage, and to allow synchronization between content and descriptions. Also tools related to managing and protecting intellectual property.

2. Description Definition Language: Language for defining new DSs and perhaps eventually also for new Ds, binary representation of DDL expressions.

3. Visual: Visual elements (Ds and DSs).

4. Audio: Audio elements (Ds and DSs).

5. Multimedia Description Schemes: Elements (Ds and DSs) that are generic, i.e. neither purely visual nor purely audio.

6. Reference Software: Software implementation of relevant parts of the MPEG-7 Standard.

7. Conformance: Guidelines and procedures for testing conformance of MPEG-7 implementations.

This document contains the elements of the Multimedia Description Schemes (MDSs) part of the standard that are currently under consideration (part 5). This document defines the MDS Working Draft (WD). It addresses only normative aspects of the MDSs. However, non-normative aspects such as examples, description creation or usage can be found in the eXperimentation Model (MDS XM) document. In order to make easier the reading, a set of hyperlinks between the MDS XM and the MDS WD documents have been created. These links assume that both documents are in the same directory with their official names: MDS XM file: M6155.doc, MDS WD file: M6156.doc.

Current MDS XM document: Version 3.1 M6155 (Beijing, July 2000)

Current MDS WD document: Version 3.1 M6155 (Beijing, July 2000)

The syntax of the descriptors and DSs is defined using the following DDL WD:

X © ISO/IEC 2000 – All rights reserved

DDL WD document: Version 3.0 N3391 (Geneva, May 2000)

The Syntax defined in this document assumes the following Schema Wrapper

<schema xmlns="http://www.w3.org/1999/XMLSchema" xmlns:mds="http://www.example.com/???" targetNamespace="http://www.example.com/???"

elementFormDefault="unqualified" attributeFormDefault="unqualified">

© ISO/IEC 2000 – All rights reserved XI

http://www.example.com/???

WORKING DRAFT

Information Technology — Multimedia Content Description Interface — Part 5: Multimedia Description Schemes

1 Scope

1.1 Organization of the document

This document describes the MDS elements under consideration of part 5 of the MPEG-7 standard. In the sequel, each element is described by two sections:

Syntax: Normative DDL specification of the Ds or DSs.

Semantic: Normative definition of the semantic of all the components.

Non-normative aspects can be found in the MDS XM document.

1.2 Overview of MDSs

The elements, Ds and DSs, described in this document are mainly structured on the basis of the functionality they provide. An overview of the structure is described in Figure 1.

Figure 1: Overview of the MDSsAt the lower level, basic elements can be found. They deal with basic datatypes, mathematical structures, linking and media localization tools as well as basic DSs, that are found as elementary components of more complex DSs. Based on this lower level, content description & management elements can be defined. These elements describe the content from several viewpoints. Currently five viewpoints are defined: Creation & Production, Media, Usage,

© ISO/IEC 2000 – All rights reserved 1

WORKING DRAFT

Structural aspects and Conceptual aspects. The three first elements address primarily information related to the management of the content (content management) whereas the two last ones are mainly devoted to the description of perceivable information (content description). The following table defines more precisely the functionality of each set of elements:

Set of Elements Functionality

Creation & Production Meta information describing the creation and production of the content: typical features include title, creator, classification, purpose of the creation, etc. This information is most of the time author generated since it cannot be extracted from the content.

Usage Meta information related to the usage of the content: typical features involve rights holders, access right, publication, and financial information. This information may very likely be subject to change during the lifetime of the AV content.

Media Description of the storage media: typical features include the storage format, the encoding of the AV content, elements for the identification of the media. Note that several instances of storage media for the same AV content can be described.

Structural aspects Description of the AV content from the viewpoint of its structure: the description is structured around segments that represent physical spatial, temporal or spatio-temporal components of the AV content. Each segment may be described by signal-based features (color, texture, shape, motion, audio features) and some elementary semantic information.

Conceptual aspects Description of the AV content from the viewpoint of its conceptual notions. (Note that currently this part of the MDS is still under Core Experiment and no elements are included in the XM or WD).

The five sets of elements are presented here as separate entities. As will be seen in the sequel, they are interrelated and may be partially included in each other. For example, Media, Usage or Creation & Production elements can be attached to individual segments involved in the structural description of the content. Depending on the application, some areas of the content description will have to be emphasized and other may be minimized or discarded.

Beside the direct description of the content provided by the five sets of elements described in the previous table, tools are also defined for navigation and access: Browsing is supported by the summary elements and information about possible variations of the content is also given. Variations of the AV content can replace the original, if necessary, to adapt different multimedia presentations to the capabilities of the client terminals, network conditions or user preferences. Another set of tools (Content organization) addresses the organization of the content by classification, by the definition of collections and by modeling. Finally, the last set of tools specified in User Interaction describes user’s preferences pertaining to consumption of multimedia material.


2 Normative references

The following ITU-T Recommendations and International Standards contain provisions, which, through reference in this text, constitute provisions of ISO/IEC 15938. At the time of publication, the editions indicated were valid. All Recommendations and Standards are subject to revision, and parties to agreements based on ISO/IEC 15938 are encouraged to investigate the possibility of applying the most recent editions of the standards indicated below. Members of ISO and IEC maintain registers of currently valid International Standards. The Telecommunication Standardization Bureau maintains a list of currently valid ITU-T Recommendations.

ISO 8601: Data elements and interchange formats -- Information interchange -- Representation of dates and times.

ISO 639: Code for the representation of names of languages.

ISO 3166-1: Codes for the representation of names of countries and their subdivisions -- Part 1: Country codes

ISO 3166-2: Codes for the representation of names of countries and their subdivisions -- Part 2: Country subdivision code.


3 Terms, definitions, symbols, abbreviated terms

For the purposes of this International Standard, the terms and definitions given in the following apply:

AV: Audio Visual

CIF: Common Intermediate Format

D: Descriptor

DCT: Discrete Cosine Transform

DDL: Description Definition Language

DS: Descripton Scheme

IANA: Internet Assigned Numbers Authority

JPEG: Joint Photographic Experts Group

MDS: Multimedia Description Scheme

MPEG: Moving Picture Experts Group

MP3: MPEG1/2 layer 3 (audio conding)

QCIF: Quarter Common Intermediate Format

SMPTE: Society of Motion Picture and Television Engineers

TZD: Time Zone

URI: Uniform Resource Identifier (IETF Standard is RFC 2396)

URL: Uniform Resource Locator (IETF Standard is RFC 2396)

XML: Extensible Markup Language

4 © ISO/IEC 2000 – All rights reserved

4 Datatypes and basic structures

Editor's Note: The basic datatypes (in the XML Schema sense) defined by MPEG-7 should be probably be moved from the MDS specification to the DDL specification.

4.1 Integer Datatypes

These datatypes constrain an integer value to lie within a range that can be represented within a fixed number of bits.

4.1.1 Datatype Syntax

<simpleType name="unsigned1" base="nonNegativeInteger">

<minInclusive value="0"/><maxInclusive value="1"/>

</simpleType>

<simpleType name="unsigned2" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="3"/>

</simpleType>


</simpleType>


</simpleType>


</simpleType>


</simpleType>


</simpleType>

<simpleType name="unsigned8" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="255"/>

</simpleType>

</simpleType>


</simpleType>


</simpleType>


</simpleType>


</simpleType>


</simpleType>


</simpleType>

<simpleType name="unsigned16" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="65535"/>

</simpleType>


</simpleType>

<simpleType name="unsigned32" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="4294967295"/>

</simpleType>

<simpleType name="unsigned64" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="18446744073709551615"/>

</simpleType>

4.1.2 Datatype Semantics

Name Definition

Unsigned1 A datatype representing a 1-bit unsigned integer value.

unsigned2 A datatype representing a 2-bit unsigned integer value.


















4.2 Vectors and matrices

Details in this section may change, depending on whether MPEG-7 adopts specific extensions for representing matrix and vector types.

These datatypes represent arbitrary-sized vectors or matrices of integer or real numbers.

4.2.1 Vector

A vector datatype represents a one-dimensional array of either integer or real values.


4.2.1.1 Descriptor Syntax



<simpleType name="listOfFloat" base="float" derivedBy="list"/><complexType name="FloatVector" base="mds:listOfFloat" derivedBy="extension">

<attribute name="Size" type="nonNegativeInteger" use="optional"/></complexType>

<simpleType name="listOfDouble" base="double" derivedBy="list"/><complexType name="DoubleVector" base="mds:listOfDouble" derivedBy="extension">


<simpleType name="listOfDecimal" base="decimal" derivedBy="list"/><complexType name="DecimalVector" base="mds:listOfDecimal" derivedBy="extension">


<simpleType name="listOfInteger" base="integer" derivedBy="list"/><complexType name="IntegerVector" base="mds:listOfInteger" derivedBy="extension">


4.2.1.2 Descriptor Semantics

Semantics of the IntegerVector type:

Name Definition

IntegerVector An n-dimensional vector of integer values.

Size The size of the vector. This value must be zero or greater and is optional. If missing, the size of the vector is determined by the number of element present.

Semantics of the FloatVector Type:

Name Definition

FloatVector An n-dimensional vector of float values.

Size The size of the vector. This value must be zero or greater and is optional. If

missing, the size of the vector is determined by the number of element present.

Semantics of the DoubleVector Type:

Name Definition

DoubleVector An n-dimensional vector of double values.


Semantics of the DecimalVector Type:

Name Definition

DecimalVector An n-dimensional vector of decimal values. These values can be of arbitrary precision.


Non-normative issues (creation, use and examples) are described in the MDS XM document.

4.2.2 Matrix

Details in this section may change, depending on whether MPEG-7 adopts specific extensions for representing matrix and vector types.

A matrix represents a two-dimensional matrix made up of row and columns of real or integer numbers.




<complexType name="FloatMatrix" base="mds:listOfFloat" derivedBy="extension">

<attribute name="Size" use="required"><simpleType base="positiveInteger" derivedBy="list"/>

</attribute></complexType>

<complexType name="DoubleMatrix" base="mds:listOfDouble" derivedBy="extension">



<complexType name="DecimalMatrix" base="mds:listOfDecimal" derivedBy="extension">



<complexType name="IntegerMatrix" base="mds:listOfInteger" derivedBy="extension">



Editor's Note: Should the size attribute be optional?


In order to represent n-dimensional matrices the elements of the matrices are written as a linear list of elements in row-major order. That is, the elements are listed with the final subscript varying most first, the second to last subscript varying second, and so on. For example, for an mxn matrix this means that the elements would appear in the order: (1,1), (1,2) …. (1, n), (2, 1),….., (2, m), …. (m, 1), (m, 2), …, (m,n).

Semantics of the FloatMatrix datatype:

Name Definition

FloatMatrix An n-dimensional matrix of float values.

Size The size of matrix specified in list form. The number of elements in the list indicates the dimension of the matrix. Each element in the list, which must be greater than zero, specifies the size of one dimension of the matrix.

Semantics of the DoubleMatrix datatype:

Name Definition

DoubleMatrix An n-dimensional matrix of double values.

Semantics of the DecimalMatrix datatype:

Name Definition

DecimalMatrix An n-dimensional matrix of decimal values.


Semantics of the IntegerMatrix datatype:

Name Definition

IntegerMatrix An b-dimensional matrix of integer values.



4.2.3 Diagonal Matrix

A diagonal matrix is a square matrix whose non-diagonal elements are all zero. Thus, such a matrix can be represented using a vector containing only its diagonal components.




<complexType name="FloatDiagonalMatrix" base="mds:FloatVector"/>

<complexType name="DoubleDiagonalMatrix" base="mds:DoubleVector"/>

<complexType name="DecimalDiagonalMatrix" base="mds:DecimalVector"/>

<complexType name="IntegerDiagonalMatrix" base="mds:IntegerVector"/>


Semantics of the FloatDiagonalMatrix D:

Name Definition

FloatDiagonalMatrix A diagonal matrix of values with float datatype.

Semantics of the DoubleDiagonalMatrix D:

Name Definition

DoubleDiagonalMatrix A diagonal matrix of values with double datatype.

Semantics of the DecimalDiagonalMatrix D:

Name Definition

DecimalDiagonalMatrix A diagonal matrix of values with decimal datatype.

Semantics of the IntegerDiagonalMatrix D:

Name Definition

IntegerDiagonalMatrix A diagonal matrix of values with integer datatype.


4.3 Probability Datatypes

These datatypes are described in the MDS XM document.

4.4 Histograms

4.4.1 Histogram D

This descriptor captures the distribution of the value for a feature using a division of the feature values into specific bins.


<complexType name="Histogram">

<element name="HistogramValues" type="mds:IntegerVector"/><attribute name="HistogramNormFactor" type="integer" use="required"/>

</complexType>

Name Definition

Histogram A representation of a feature’s values as a histogram.

HistogramNormFactor The normalization factor of the histogram, which identifies the possible range of histogram values.

HistogramValues The bins in the histogram represented as an integer vector. The size depends on the specific feature that is represented. The sum of histogram values over all bins is equal to the HistogramNormFactor.

The histogram values are normalized to make the descriptor independent of the target item (such as image size). The histogram values are all multiplied by the HistogramNormFactor, which makes it possible to encode the histogram using integers.


4.5 Quantizers

Currently, this descriptor is not being used anywhere in the MPEG-7 XM or WD. The Video group color quantitizor descriptor does not use this form. Therefore, this section has been left unmodified.

Recommend that it be removed.

4.5.1 Uniform Quantization D

This descriptor specifies the uniform quantization of a feature space.


<complexType name="UniformQuantizer">

<element name="BinNumber" type="integer" minOccurs="0" maxOccurs="unbounded"/><attribute name="SpaceDimension" type="nonNegativeInteger" use="required"/>

</complexType>


Name Definition

UniformQuantizer A representation of a uniform quantization of a feature space.

SpaceDimension The dimension of the feature space being quantitized.

Name Definition

BinNumber The number of bins into which a dimension is divided. One such value exists for each dimension of the feature space.



5 Link to the media and Localization

5.1 References to Ds and DSs

5.1.1 Reference D

This descriptor is a general tool for referencing part of the description (either a D or a DS instantiation).






<complexType name="Reference" content="empty"><attribute name="href" type="uriReference" use="optional"/><attribute name="idref" type="IDREF" use="optional"/>

</complexType>


Name Definition

Reference Reference to an element of a description.

href If an element is referenced not contained in the current description href specifies the location of the description that contains the referenced element and the ID of this element. Location of the description that contains the referenced element. If href is not instantiated, the location is assumed to be the one of the current description.

idref Identifier of a referenced element that is contained within the same description the reference is located in.Identifier of a referenced element that is contained within the description defined by href.

Editor's Note: The Referencing mechanisms (such as ID/IDREF) used within MPEG-7 is still an open issue of the DDL .

5.1.2 ReferenceToSegment D

This descriptor is a general tool for referencing a Segment.


If "href" refers to another description, it is not certain that a parser can actually check the type of element "idref" is referring to.The current XML Schema WD requires a complex type derived by restriction to repeat all the declarations it inherited from its base. This becomes tedious and hard to manage as the inheritance hierarchy grows deeper. Is it possible to specify only those declarations that are further constrained in the derivedtype? Nevertheless, repeating a deeply nested structure might be the only practical way to specify restrictions to the structure without causing ambiguity. -->

<complexType name="ReferenceToSegment" base="mds:Reference" derivedBy="restriction">

<attribute name="href" type="uriReference" use="optional"/><attribute name="idref" type="IDREF" refType="mds:Segment" use="optional"/>

</complexType>


Name Definition

ReferenceToSegment Reference to the description of a segment of AV content. Shall refer to the id attribute of a Segment.

idref Identifier of a referenced segment that is contained within the description the reference is located indefined by href. This descriptor is derived from the Reference D and adds the constraint that the referenced element should be a segment.

Editor's Note: The Referencing mechanisms (such as ID/IDREF) used within MPEG-7 is still an open issue of the DDL. In this case no type constrain can be specified for the href mechanism. Also the scope of the refType element with respect to inheritance is not clearified.


5.1.3 ReferenceToProgram D

This descriptor is a general tool for referencing a Program.

It will be a DDL extension on top of XML Schema. If "href" refers to another description, it is not certain that a parser can actually check the type of element "idref" is referring to. The current XML Schema WD requires a complex type derived by restriction to repeat all the declarations it inherited from its base. This becomes tedious and hard to manage as the inheritance hierarchy grows deeper. Is it possible to specify only those declarations that are further constrained in the derived type? Nevertheless, repeating a deeply nested structure might be the only practical way to specify restrictions to the structure without causing ambiguity. -->

<complexType name="ReferenceToProgram" base="mds:Reference" derivedBy="restriction">

<attribute name="href" type="uriReference" use="optional"/><attribute name="idref" type="IDREF" refType="MediaInstance" use="optional"/>

</complexType>


Name Definition

ReferenceToProgram Reference to the description of an instance of AV content. Shall refer to the id attribute of a MediaInstance.

idref Identifier of a referenced program that is contained within the description defined by hrefthe reference is located in. This descriptor is derived from the Reference D and adds the constraint that the referenced element should be a MediaInstance.

Editor's Note: The Referencing mechanisms (such as ID/IDREF) used within MPEG-7 is still an open issue of the DDL. In this case no type constrain can be specified for the href mechanism. Also the scope of the refType element with respect to inheritance is not clearified.


5.2 Unique Identifier

5.2.1 Uidentifier D

This descriptor allows the identification of the AV content under description. The identifier may be used to identify the content as a unique work being described (e.g., ISAN) or to identify its instances (e.g., SMPTE copy number).




<complexType name="UIdentifier"><element name="IdValue" type="string"/><attribute name="IdOrganization" type="string"/><attribute name="IdName" type="string"/>

</complexType>

Name Definition

Uidentifier Type allowing the unique identification of AV content.

IdValue String specifying the value of the identification.

IdOrganization The organization that defines the identifier or the registration authority (e.g., ISO, SMPTE).

IdName The name of the identifier used (e.g., ISAN, ISWC, UMID, UPID).

Editor's Note: Identifiers can be used within Uidentifier D if a textual format of the used id exists. A default mapping between binary and textual format is not specified.


5.3 Time elements

The temporal descriptors are based on a restricted set of lexical expressions of the ISO8601 standard. Instead of specifying fractions of a second by an arbitrary number of decimals these are specified by counting prespecified fractions of a second. This approach is widely used within AV media for instance in MPEG-4 video. The lexical Time expressions can represent time both as used in media streams or real world time, e.g. production time.

The time specification is done either by using a Gregorian Date and day time or by counting time units relative to a time base. The time units are specified using an arbitrary number of full days plus a part of a day defined by hours, minutes, etc. The MediaTime DS as described in section 5.3.12 specifies a structure using these descriptors to specify a mediatime segment. The Time DS specification differs from the MediaTime DS specification by an additional description of the Time Zone (TZD) according to UTC.

As described before there are two ways to specify time: a description of time using the common definition of date and time or a specification of time using arbitrarily defined time units. The common description of time can be used to describe an absolute (start) time point (Media/TimePoint D) or a time point relative to another time point (Media/RelTimePoint D) and a duration of e.g. a segment (Media/Duration D). The specification using time units can be applied to describe a duration (Media/IncrDuration D) or a time point relative to another time point (Media/RelIncrTimePoint D).

5.3.1 TimePoint D

This Time Descriptor specifies a time point according to the Gregorian dates and day time and the TZD. The format is based on the ISO 8601 norm. To reduce converting problems only a subset of the ISO 8601 formats is used.




<simpleType name="FractionalTime" base="string" derivedBy="restriction"> <pattern value="-?(\d+(\-\d{2}(\-\d{2})?)?)? (T\d{2}(:\d{2}(:\d{2}(\:d+(\.d{2})?)?)?)?)? ((\-|\+)\d{2}\:d{2})?(F\d+)?"/>

</simpleType>

<simpleType name="TimePoint" base="mds:FractionalTime" derivedBy="restriction"> <pattern value="-?(\d+(\-\d{2}(\-\d{2})?)?)? (T\d{2}(:\d{2}(:\d{2}(\:d+)?)?)?)? (\-|\+\d{2}:d{2})?(F\d+)?"/></simpleType>

5.3.1.2 TimePoint D Semantic

Semantic of the basic dataypes:

Name Definition

FractionalTime A simpletype representing an instance in time using a lexical representation of Gregorian date (YYYY-MM-DD), day time and a fraction specification (Thh:mm:ss:nnn.ff) including the time zone according to UTC (hh:mm) and the specification of the number of fractions of one second (FNNN):

YYYY-MM-DDThh:mm:ss:nnn.ffhh:mmFNNN

While the date representation is a subset of the ISO8601 formats the Time representation is specified according to the SMPTE timestamp formats.

The following lexicals are used for digits of the corresponding date/time elements: Y (Year), M (Month), D (Day), h (hour), m (minute), s (second), n/f (decimal numbers of fractions), N (number of fractions of one second). Also delimiters for the time specification (T) and the number of fractions of one second are used (F).

Semantic of the TimePoint D:

Name Definition

TimePoint The TimePoint D is derived from the FractionalTime type by restricting the fractions to integer numbers instead of decimals.

Beside the specification of Gregorian Date, Daytime counting the number (nnn) of fractions (FNNN) of a second is possible to allow a higher precision than one second. If counting fractions is used, beside the counted number of fractions (nnn) also the number of fractions building up one second (FNNN) has to be specified. Thus the FNNN defines the value range of the counted number of fractions (nnn) value. So the value range of ‘nnn’ is limited from 0 to FNNN-1. If FNNNis not specified the FNNN of the first ancestor (usually the root node) in the description that has a TimePoint D or Duration D included in the same description scheme is used. If no time zone is specified or can not be inherited as in the case of FNNN a time zone of 00:00 is assumed.

Note: Attribute data inheritance used here is a pending issue of the DDL WD. For the time being this functionality is described as a semantic of the descriptor.

It is not required to instantiate all components of the Gregorian date and day time in order to be flexible in scope and precision.



5.3.2 Duration D

This Descriptor specifies the duration of a time period according to days and day time. The format is based on the ISO 8601 norm. To reduce converting problems only a subset of the ISO 8601 formats is used. Fractions of a second are specified according to the TimePoint D.

5.3.2.1 Duration D Syntax



<simpleType name="FractionalDuration" base="string" derivedBy="restriction"><pattern value="-?P(\d+D)?(T(\d+H)?(\d+M)?(\d+S)?(\d+N)?(\d{2}f)?)?(\-|\+\d{2}:d{2}Z)?(\d+F)?"/>

</simpleType>

<simpleType name="Duration" base="mds:FractionalDuration" derivedBy="restriction"><pattern value="-?P(\d+D)?(T(\d+H)?(\d+M)?(\d+S)?(\d+N)?)? (\-|\+\d{2}:d{2}Z)?(\d+F)?"/>

</simpleType>

5.3.2.2 Duration D Semantic

Semantic of the basic dataypes:

Name Definition

FractionalDuration A simpletype representing a duration in time using a lexical representation of days (nD), time duration and a fraction specification (TnHnMnSnNnf) including the difference in the time zone according to UTC (hh:mm) and the specification of the number of fractions of one second (nF):

PnDTnHnMnSnNnfhh:mmZnF

While the day representation is a subset of the ISO8601 formats the Time duration representation is specified according to the SMPTE timestamp formats.

In the used lexical expression n specifies a non negative number. The numbers are followed by a separator specifying the semantic of the number: D (days), H (hours), M (minutes), S (seconds), N (number of fractions), f (for a decimal expression of fractions), F (number of fractions of one second). Also a delimiter for the time specification (T) is used.

Editor's Note: For a validity checking as it can be done for date and time datatypes in XML Schema the datatypes FractionalTime and FractionalDuration should be defined as built in datatypes of the DDL.

Name Definition

Duration The Duration D is derived from the FractionalDuration type by restricting the fractions to integer numbers instead of decimals.

The duration for instance of a time segment can be specified using a number of days and time. The specification of fractions of a second (nN) is the same as in the case of TimePoint. The scope of Duration is limited to days to avoid ambiguities depending on the date of the start time point if for instances describing the duration of a segment with “one month".

Note also that the Duration can specify a difference in the TZD specification that indicates the difference in the time zone from the starting point to the end point (for instance using a mobile camera). This is important to map the times to local times.


5.3.3 IncrDuration D

To enable a simplified and efficient description of segment duration using a periodical time specification (e.g. periodic samples along the timeline) the IncrDuration D specifies a duration of such a segment by counting time units. Such a time unit can e.g. be a time increment between successive frames with respect to the world time when the sequence was recorded. The duration is then specified by the number of these time units.

5.3.3.1 IncrDuration D Syntax



<complexType name="IncrDuration" base="integer" derivedBy="extension"><attribute name="timeunit" type="mds:Duration" use="optional"/>

</complexType>

5.3.3.2 IncrDuration D Semantic

Name Definition

IncrDuration The IncrDuration D is derived from the simple type integer by adding an attribute specifying the timeunit (of type Duration) counted by the integer value.

The IncrDurationD specifies a duration by counting ‘timeunits’. The semantics specified for the number of fractions of one second (FNNN) in the TimePoint D specification are also valid for the time unit specification.


5.3.4 RelTimePoint D

This Time Descriptor specifies a time point relating to a time base using a number of days and time. The format is based on the ISO 8601 norm. The specification is similar to the one used for DurationD.

5.3.4.1 RelTimePoint D Syntax



<complexType name="RelTimePoint" base="mds:Duration" derivedBy="extension"><attribute name="TimeBase" type="uriReference" use="optional"/>

</complexType>

5.3.4.2 RelTime D Semantic

Name Definition

RelTimePoint The RelTimePoint D is derived from the Duration type by adding the specification of the TimeBase from which a time point is specified relatively by using an expression of type Duration.

The RelTimePoint D is a specification of an offset according to a time base. The time base is referenced using an URI. If the link to a TimeBase is not specified a link to the start time of the first ancestor (usually the root node) in the description is assumed.


5.3.5 RelIncrTimePoint D

This Time Descriptor specifies a time point relative to a time base counting time units as already specified for the IncrDuration D. If for instance an addressing of a frame by counting frames is needed RelIncrTime D can be used referencing to the starting time stamp of the shot or the whole video as a time base.

5.3.5.1 RelIncrTimePoint D Syntax

<complexType name="RelIncrTimePoint" base="mds:IncrDuration" derivedBy="extension">

<attribute name="TimeBase" type="uriReference" use="optional"/></complexType>

5.3.5.2 RelIncrTime D Semantic

Name Definition

RelIncrTimePoint The RelIncrTimePoint D is derived from the IncrDuration type by adding the specification of the TimeBase from which a time point is specified relatively by using an expression of type IncrDuration.

The semantics specified for the TimeBase attribute in the RelTimePoint D and for the ‘timeunit’ attribute in the IncrDuration D specification are also valid here.


5.3.6 Time DS

For the specification of time segments the Time DS is composed of two elements, the (start) time point and the duration. If only a time point has to be specified, the duration can be omitted.

5.3.6.1 TimeDS Syntax

<complexType name="Time"><sequence>

<choice><element name="TimePoint" type="mds:TimePoint"/><element name="RelTimePoint" type="mds:RelTimePoint"/><element name="RelIncrTimePoint" type="mds:RelIncrTimePoint"/>

</choice><choice minOccurs="0">

<element name="Duration" type="mds:Duration"/><element name="IncrDuration" type="mds:IncrDuration"/>

</choice></sequence>

</complexType>

5.3.6.2 Time DS Semantic

Name Definition

Time A description scheme to specify time points or time segments using the following descriptors for time specification

TimePoint A Descriptor specifying a time point using Gregorian date and day time

RelTimePoint A Descriptor specifying a time point relative to a time base using a number of days and day time

RelIncrTimePoint A Descriptor specifying a time point relative to a time base counting time units

Duration A Descriptor specifying the duration of a time period according to days and day time

IncrDuration A Descriptor specifying the duration of a time period counting time units

The Time DS uses two different ways to describe a time point or period:

Using Gregorian date and time (TimePoint D, RelTimePoint D, Duration D).

By counting time units specified in the attributes (RelIncrTimePoint D, IncrDuration D). In the case of video segments for example, these time units may correspond to the frame rate.

This enables to describe at what time the sequence is taken using Gregorian date, time and also to specify the time according to samples in time such as video frames.


5.3.7 MediaTimePoint D

This Media Time Descriptor specifies a time point according to the Gregorian dates and day time of media time stamps. The format is based on the ISO 8601 norm. To reduce converting problems only a subset of the ISO 8601 formats is used. Fractions of a second are specified according to the TimePoint D.


<simpleType name="MediaTimePoint" base="mds:FractionalTime" derivedBy="restriction"> <pattern value="-?(\d+(\-\d{2}(\-\d{2})?)?)? (T\d{2}(:\d{2}(:\d{2}(\:d+)?)?)?)?(F\d+)?"/></simpleType>

5.3.7.2 MediaTimePoint D Semantic

Semantic of the MediaTimePoint D:

Name Definition

MediaTimePoint A Descriptor specifying a time stamp of the media using Gregorian date and day time without specifying the TZD (see also TimePoint D)

The specification of FNNN in the case of the MediaTimePoint D corresponds to that of the TimePoint D.


5.3.8 MediaDuration D

This Descriptor specifies the duration of a time period according to days and day time of the media time stamps. The format is based on the ISO 8601 norm. To reduce converting problems only a subset of the ISO 8601 formats is used. Fractions of a second are specified according to the TimePoint D.

5.3.8.1 MediaDuration D Syntax

<simpleType name="MediaDuration" base="mds:FractionalDuration"

derivedBy="restriction"> <pattern value="-?P(\d+D)?(T(\d+H)?(\d+M)?(\d+S)?(\d+N)?)?(\d+F)?"/></simpleType>

5.3.8.2 MediaDuration D Semantic

Name Definition

MediaDuration A Descriptor specifying the duration of a time period according to days and day time of the media time stamps without specifying a difference in the TZD (see also Duration D)

5.3.9 MediaIncrDuration D

To enable a simplified and efficient description of a media segment duration using a periodical time specification (e.g. periodic samples along the timeline) the MediaIncrDuration D specifies a duration of such a segment by counting time units. Such a time unit can e.g. be the time increment of the timestamps of successive frames in a video stream. The duration is then specified by the number of these time units.

5.3.9.1 MediaIncrDuration D Syntax

<complexType name="MediaIncrDuration" base="integer" derivedBy="extension">

<attribute name="timeunit" type="mds:MediaDuration" use="optional"/></complexType>

5.3.9.2 MediaIncrDuration D Semantic

Name Definition

MediaIncrDuration A Descriptor specifying the duration of a time period with respect to the media time counting time units (see also the IncrDuration D)

The MediaIncrDurationD specifies a duration by counting ‘timeunits’. The semantics specified for the number of fractions of one second (FNNN) in the MediaTimePoint D specification are also valid for the time unit specification.


5.3.10 MediaRelTimePoint D

This Media Time Descriptor specifies a time point relating to a time base using a number of days and time according to media time stamps. The format is based on the ISO 8601 norm. The specification is similar to the one used for MediaDurationD.

5.3.10.1 MediaRelTimePoint D Syntax



<complexType name="MediaRelTimePoint" base="mds:MediaDuration" derivedBy="extension"><attribute name="TimeBase" type="uriReference" use="optional"/>

</complexType>

5.3.10.2 MediaRelTimePoint D Semantic

Name Definition

MediaRelTimePoint A Descriptor specifying a media time point relative to a time base without specifying a difference of the TZD (see also the RelTime D)

The MediaRelTimePoint D is a specification of an offset according to a time base. The time base is referenced using an URI. If the link to a TimeBase is not specified a link to the start time of the first ancestor (usually the root node) in the description is assumed. The semantics specified for the nnFraction in the MediaTimePoint D specification are also valid for the time unit specification.


5.3.11 MediaRelIncrTimePoint D

This Media Time Descriptor specifies a time point relative to a time base counting time units as already specified for the MediaIncrDuration D. If for instance an addressing of a frame by counting frames is needed MediaRelIncrTimePoint D can be used referencing to the starting time stamp of the shot or the whole video as a time base.

5.3.11.1 MediaRelIncrTimePoint D Syntax

<complexType name="MediaRelIncrTimePoint" base="mds:MediaIncrDuration"

derivedBy="extension"><attribute name="TimeBase" type="uriReference" use="optional"/>

</complexType>

5.3.11.2 MediaRelIncrTimePoint D Semantic

Name Definition

MediaRelIncrTimePoint A Descriptor specifying a media time point relative to a time base counting time units

The semantics specified for the MediaTimeBase attribute in the MediaRelTimePoint D and for the ‘timeunit’ attribute in the MediaIncrDuration D specification are also valid here.


5.3.12 MediaTime DS

For the specification of time segments according to the time stamps of the media the MediaTime DS is composed of two elements, the (start) time point and the duration. If only a time point has to be specified, the duration can be omitted.

5.3.12.1 MediaTimeDS Syntax



<complexType name="MediaTime"><sequence>

<choice><element name="MediaTimePoint" type="mds:MediaTimePoint"/><element name="MediaRelTimePoint" type="mds:MediaRelTimePoint"/><element name="MediaRelIncrTimePoint"

type="mds:MediaRelIncrTimePoint"/></choice><choice minOccurs="0">

<element name="MediaDuration" type="mds:MediaDuration"/><element name="MediaIncrDuration" type="mds:MediaIncrDuration"/>

</choice></sequence>

</complexType>

5.3.12.2 MediaTime DS Semantic

Name Definition

MediaTime A description scheme to specify media time points or media time segments using the following descriptors for time specification

MediaTimePoint A Descriptor specifying a media time point using Gregorian date and day time

MediaRelTimePoint A Descriptor specifying a media time point using a number of days and day time

MediaRelIncrTimepoint A Descriptor specifying a media time point relative to a time base counting time units

MediaDuration A Descriptor specifying the duration of a media time period according to days and day time

MediaIncrDuration A Descriptor specifying the duration of a media time period counting time units

The MediaTime DS uses two different ways to describe a time point or period:

Using Gregorian date and time (MediaTimePoint D, MediaRelTimePoint D, MediaDuration D).

By counting time units specified in the attributes (MediaRelIncrTimePoint D, MediaIncrDuration D). In the case of video segments for example, these time units may correspond to the frame rate.

This enables to specify the media time according to the format used for the media time stamps.


5.4 Media locators

Media locators are used to specify the “location" of AV content.

5.4.1 MediaURL D

The MediaURL uses an URI to locate the AV content. This descriptor is defined in the XM document and has not been promoted to WD.

5.4.2 MediaLocator DS

The MediaLocator DS is used to specify the “location" of a particular image, audio or video segment by referencing the media data. There are four types of MediaLocators: the VideoSegmentLocator, the AudioSegmentLocator, the ImageLocator, and the SoundLocator.


5.4.2.1 MediaLocator DS Syntax



<complexType name="MediaLocator"> <choice> <sequence>

<element name="MediaURL" type="mds:MediaURL" minOccurs="0"/> <element name="MediaTime" type="mds:MediaTime" minOccurs="0"/>

</sequence> <element name="MediaTime" type="mds:MediaTime"/> </choice>

</complexType>

5.4.2.2 MediaLocator DS Semantic

Name Definition

MediaLocator A description scheme to specify the location of media segments by referencing the media data.

MediaTime A description scheme to specify time points or segments using the time information of the AV content

MediaURL A Descriptor specifying the location of AV content using an URI

The MediaLocator D can be composed of a MediaURL for locating a media element. Additionally the MediaTime can be used to locate media chunks within this media element addressed by the MediaUrl. A short form of the latter format can use the MediaTime as a single element within the MediaLocator D if a global MediaLocator D (usually defined within the MediaLocator of the root segment) using a MediaURL is defined.


5.4.3 VideoSegmentLocator DS

5.4.3.1 VideoSegmentLocator DS Syntax



<complexType name="VideoSegmentLocator" base="mds:MediaLocator" derivedBy="extension"/>

5.4.3.2 VideoSegmentLocator DS Semantic

Name Definition

VideoSegmentLocator A media locator for video segments

5.4.4 ImageLocator DS

5.4.4.1 ImageLocator DS Syntax



<complexType name="ImageLocator" base="mds:MediaLocator" derivedBy="extension"/>

5.4.4.2 ImageLocator DS Semantic

Name Definition

ImageLocator A media locator for images. These can be a single frame using only the MediaTime specification or in combination with an URL. Also images for instance coded within a file can be addressed by specifying only an URL in the ImageLocator.. Or it can be a file then only an URL is used within the ImageLocator.


5.4.5 AudioSegmentLocator DS

5.4.5.1 AudioSegmentLocator DS Syntax



<complexType name="AudioSegmentLocator" base="mds:MediaLocator" derivedBy="extension"/>

5.4.5.2 AudioSegmentLocator DS Semantic

Name Definition

AudioSegmentLocator A media locator for audio segments


5.4.6 SoundLocator DS

The SoundLocator DS is used to locate a specific type of audio segments which by itself describe scenes or events by sound effects.

5.4.6.1 SoundLocator DS Syntax

<complexType name="SoundLocator" base="mds:AudioSegmentLocator" derivedBy="extension"/>

5.4.6.2 SoundLocator DS Semantic

Name Definition

SoundLocator A media locator for sound effects

6 Basic elements

6.1 Textual description

6.1.1 Language attribute

There are two situations in which specification of language is required in MPEG-7: (1) to describe the language of the description information itself or (2) to describe the language of the content. In the first case the language is a property of the MPEG-7 description—i.e. the metadata—and in the second case the language information is itself metadata. Depending on which case applies, MPEG-7 provides two solutions. In case (1), a special attribute (xml:lang) is used to specify the language used to write the description. In case (2), the Language descriptor is used. Note that the content itself may be textual.

The XML Language attribute must be used for all Descriptors where the language needs to be identified (e.g., Annotation or Title). In the case of controlled vocabularies and thesauri (if they are closed), the list can be mapped to any language, although a default one may be used for the specification.

The XML lang attribute

Any description which needs to identify the language in which a description in written must use the xml:lang attribute for this purpose. To declare that this attribute may be attached to a description element, the following form must be used to define the attribute:

<attribute ref="xml:lang"/>

6.1.1.1 Attribute Syntax





6.1.1.2 Attribute Semantics

Name Definition

xml:lang The XML language attribute.

The possible value attributes of the language code are as defined in the basic types of MPEG-7 DDL. The language specified with the xml:lang attribute implicitly applies to all attributes, elements, and content contained within the description where the attribute is specified, unless overridden with an xml:lang attribute on another element within that description.

Do we need to further restrict the value of the language to allow only ISO 639 language identifiers? Do we want to support the full IANA language identifiers?

6.1.2 Language Datatype

The Language type describes the language of the content. MPEG-7 uses the “language" datatype defined in the XML Schema specification to represent natural language identifiers.

6.1.2.1 Datatype Syntax





6.1.2.2 Descriptor Datatype Semantics

This descriptor is currently in a Core Experiment and is likely to change from the syntax specified. Until the Core Experiment completes, this section is being untouched.

The Language identifier contained within the language code shall be the primary method of designating the language of the content. The capability to append a subcode indicating a country is an optional, secondary method for further specifying the language.


6.1.3 ControlledTerm D

The ControlledTerm descriptor allows the specification of a value for a textual field and its relation to a classification scheme (thesaurus or controlled vocabulary). The textual field is not always controlled, as the attributes for defining the classification scheme are optional; that is, an arbitrary string may also be used.




<complexType name="ControlledTerm" base="string" derivedBy="extension"><attribute name="CSName" type="string" use="optional"/><attribute name="CSTermId" type="string" use="optional"/><attribute name="CSLocation" type="uri" use="optional"/>

</complexType>


Name Definition

ControlledTerm Textual field descriptor (string) allowing to be related to a controlled vocabulary.

CSName The name of the registration authority or of the classification scheme used to obtain the term value. It may include the version of the classification scheme.

CSTermId The identifier that uniquely identifies the value (string) set in the descriptor within the classification scheme. It is a label which format depends on the classification

Name Definition

scheme.

CSLocation The reference to the location of the whole thesaurus or classification scheme.


6.1.4 Textual Description Datatype

The Textual Description datatype provides an abstract base type from which textual descriptions are derived. An optional attribute, xml:lang, may be used to specify the language of the textual description contained within this element and elements derived from it.




<complexType name="TextualDescription" base="string" derivedBy="extension"><attribute ref="xml:lang"/>

</complexType>


Name Definition

TextualDescription

xml:lang The language in which the textual description is written. This attribute need not be specified. If no value is specified for this attribute, and the element occurs within the scope of an element that specifies a value, the value of is taken to be the value of the closest (innermost) such element. See the definition of the language attribute in section Language attribute 6.1.1.

6.1.5 StructuredAnnotation DS

The StructuredAnnotation DS (or 6W+1 DS) contains the description tools simple and structured textual description of events, people, animals, objects, places, actions, purposes, and times..

6.1.5.1 Description Scheme Syntax



<element name="TextAnnotation" type="mds:TextualDescription"/>

<element name="StructuredAnnotation" type="mds:StructuredAnnotation"/><complexType name="StructuredAnnotation">

<element name="Who" type="mds:ControlledTerm" minOccurs="0"/>

<element name="WhatObject" type="mds:ControlledTerm" minOccurs="0"/><element name="WhatAction" type="mds:ControlledTerm" minOccurs="0"/><element name="Where" type="mds:ControlledTerm" minOccurs="0"/><element name="When" type="mds:ControlledTerm" minOccurs="0"/><element name="Why" type="mds:ControlledTerm" minOccurs="0"/><element name="TextAnnotation" type="mds:TextualDescription" minOccurs="0"/><attribute name="id" type="ID"/><attribute ref="xml:lang"/>

</complexType>

Should the maxOccurs of each element above be unbounded?

6.1.5.2 Description Scheme Semantics

Name Definition

TextAnnotation Free textual annotation.

StructuredAnnotation Textual free annotation and description of people, animals, objects, actions, places, time, and/or purpose.

Who Textual description of people and animals. May be from a thesaurus or a controlled vocabulary.

WhatObject Textual description of objects. May be from a thesaurus or a controlled vocabulary.

WhatAction Textual description of actions. May be from a thesaurus or a controlled vocabulary.

Where Textual description of places. May be from a thesaurus or a controlled vocabulary.

When Textual description of time. May be from a thesaurus or a controlled vocabulary.

Why Textual description of purpose. May be from a thesaurus or a controlled vocabulary.

Annotation Textual free annotation and description of people, animals, objects, actions, places, time, and/or purpose.

id Identifier for an instantiation of the StructuredAnnotation DS.

Ambiguity of the natural language representation is one of the inherent drawbacks of free text annotation. However, the Annotation DS provides a structured format which, when used in conjunction with with associated thesaurus or controlled vocabulary lists, provides a simple but powerful annotation tool. Nevertheless, free text can always be used in any of the 6W fields.



6.2 Description of persons

This section contains the description schemes and datatypes concerned with the description of people. In MPEG-7, people refer to

6.2.1 Person DS

The Person DS is an abstract DS intended for the description of persons (e.g., actor, director, character, dubbing actor), organizations (e.g., a company) and groups of persons (e.g. a musical ensemble). The Person DS, and the description schemes that derive from it can be used to describe real, historical, or fictional persons. This description scheme can be used for describing both person depicted inside the content itself—historical, fictional, or real—and person in the real world—the actor, users, and others who relate in some fashion to the content What distinguishes these two uses of the Person DS is the context: the description scheme containing the Person DS.




<complexType name="Person" abstract="true"><attribute name="id" type="ID"/>

</complexType><element name="Person" type="mds:Person"/>


The Person DS is an instance of an Individual DS, PersonGroup DS or Organization DS.

Name Definition

Person DS describing a person. The person can be an individual, a quasi-person or an organization.

Id Identification of the instantiation of the person description

The semantic should be updated to reflect the syntax


6.2.2 Person Name Datatype

The Person Name datatype describes the name of a person, whether historical, existing, or fictional. While the conventions for naming people vary widely across cultures and languages, the Person Name datatype provides a simple model for the structure of names. However it does not require that this structure be used.

The model for names is as follows. First, it is assumed that everyone is named by some given name. Secondly, a name may contain a family name, which is shared by the members of a relevant social group, identified according to the social conventions of that culture. In many cultures this is a family name—but it some cultures it may be a place name used to identify the place where a person comes from or lives in. In some cultures, no such name exists and there is only a given name. Finally, a title indicates a part of a name conferred upon a person as result of obtaining a certain social status, such as “Sir" for knighthood or “Ph.D." for a degree.

The elements of a name may also be broken down into name components. However, this is entirely optional. While decomposition of a name is culture- and language-bound, each component will usually correspond to one word in the language of the name. Furthermore, as attributes of each name component, one can associate an abbreviation and an initial. The abbreviation is a variant form of the name—such as a nickname or pet name like “Johnny" for “Jonathon". The initial is a shorter string use to denote a name component. For example, an initial can be the first letter of a name in English. Both of these attributes are optional. It is permissible to represent only the abbreviated or initials of a name component if the full form is unknown.

The order in which components of a name appear in the description is significant and should adhere to the order used for writing names in the language in which they are described.

A name is not assumed to be unique identifier for a person, and the description is not in a canonical form, so there may be several different ways to describe the same name using the Person Name datatype.

6.2.2.1 Datatype Syntax



<complexType name="NameComponent" base="mds:TextualDescription" derivedBy="extension">

<attribute name="initial" type="string"/><attribute name="abbrev" type="string"/>

</complexType>

<complexType name="PersonName"><element name="GivenName" type="mds:NameComponent" maxOccurs="unbounded"/><element name="FamilyName" type="mds:NameComponent"

minOccurs="0" maxOccurs="unbounded"/><element name="PersonTitle" type="mds:NameComponent"

minOccurs="0" maxOccurs="unbounded"/><attribute ref="xml:lang"/>

</complexType>

6.2.2.2 Datatype Semantics

Name Definition

NameComponent A component of a name.

initial An initial form of the name component. Initials are a truncated form of the description of the name.

Abbrev An alternative shorter form for a name component.

PersonName The description of the name of a person.

FamilyName The family name(s) of an individual. This is optional.

GivenName The given name(s) of a person. At least one given name must be specified in a name description.

Title The titles that are part of the name of an individual. This part of a name is optional.

xml:lang The language in which the name is described.

Name Definition

While the conventions for naming people vary widely across cultures and languages, the Person Name datatype provides a simple model for the structure of names. However it does not require that this structure be used.

The model for names is as follows. First, it is assumed that everyone is named by some given name. Secondly, a name may contain a family name, which is shared by the members of a relevant social group, identified according to the social conventions of that culture. In many cultures this is a family name—but it some cultures it may be a place name used to identify the place where a person comes from or lives in. In some cultures, no such name exists and there is only a given name. Finally, a title indicates a part of a name conferred upon a person as result of obtaining a certain social status, such as “Sir" for knighthood or “Ph.D." for a degree.

The elements of a name may also be broken down into name components. However, this is entirely optional. While decomposition of a name is culture- and language-bound, each component will usually correspond to one word in the language of the name. Furthermore, as attributes of each name component, one can associate an abbreviation and an initial. The abbreviation is a variant form of the name—such as a nickname or pet name like “Johnny" for “Jonathon". The initial is a shorter string use to denote a name component. For example, an initial can be the first letter of a name in English. Both of these attributes are optional. It is permissible to represent only the abbreviated or initials of a name component if the full form is unknown.

The order in which components of a name appear in the description is significant and should adhere to the order used for writing names in the language in which they are described.

A name is not assumed to be unique identifier for a person, and the description is not in a canonical form, so there may be several different ways to describe the same name using the Person Name datatype.


6.2.3 Individual DS

The Individual DS contains the description tools (Ds and DSs) intended for a description of persons, whether real historical or fictional. For example, this DS can be used to describe an actor, a character, or producer.




<complexType name="Individual" base="mds:Person" derivedBy="extension"><element name="Name" type="mds:PersonName" maxOccurs="unbounded"/>

</complexType><element name="Individual" type="mds:Individual" equivClass="Person"/>


Name Definition

Individual DS describing an individual. .

Name The names associated with this person. Multiple names are allowed. The interpretation of different name is not defined in MPEG-7.

6.2.4 PersonGroup DS

The PersonGroup DS contains the description tools (Ds and DSs) intended for a description of groups of people.




<complexType name="PersonGroup"><element name="GroupName" type="mds:TextualDescription"

maxOccurs="unbounded"/> <choice minOccurs="1" maxOccurs="unbounded">

<element name="Person" type="Person" maxOccurs="unbounded"/><element refname="ReferenceToPerson"/>

</choice></complexType><element name="PersonGroup" type="mds:PersonGroup" equivClass="Person"/>

Need to define a ReferenceToPerson to implement this DS. However, currently no such descriptor is defined. Will the ReferencetoXXX approach continue? Many of these are missing.


Name Definition

PersonGroup DS describing a group of persons.

GroupName The name of the group. A group may have multiple names.

Person A description of one member of the group. Each group can be composed of one or persons, which can include other person groups, recursively.

ReferenceToPerson Reference to a description of a person who is a member of the group. Members of a group may be identified either by reference to an existing person description or by describing the person directly inside the Person Group DS.

The Semantic should be described


6.2.5 Organization DS

The Organization DS contains the description tools (Ds and DSs) intended for a description of organizations (e.g., a company).

<complexType name="Organization" base="mds:Person" derivedBy="extension"><element name="Name" type="mds:TextualDescription"/><element name="Contact" type="mds:Individual"

minOccurs="0" maxOccurs="unbounded"/><element name="Address" type="mds:Place" minOccurs="0" maxOccurs="unbounded"/>

</complexType><element name="Organization" type="mds:Organization" equivClass="Person"/>


Name Definition

Organization DS describing an organization.

Name The name of the organization.

Contact The description of the individual person acting as a contact for the organization.

Address The address of the organization.


6.3 Description of places

6.3.1 Place DS

The Place DS contains the description tools (Ds and DSs) intended for a description of locations. As with the Person DS, the Place DS uses the same description for describing existing, historical, and fictional places. The same DS can be used to describe locations depicted in the content and locations in the real-world, such as where the content was filmed.




<complexType name="GPSCoordinates" base="string" derivedBy="extension"><element name="GPSsystem" type="mds:ControlledTerm"/>

</complexType>

<complexType name="PostalAddress" base="mds:TextualDescription"><element name="AddressLine" type="string" maxOccurs="unbounded"/>

</complexType>

<complexType name="Place"><element name="PlaceName" type="mds:TextualDescription"

maxOccurs="unbounded"/><element name="PlaceRole" type="mds:ControlledTerm" minOccurs="0"/>

<element name="Planet" type="mds:ControlledTerm" minOccurs="0"/><element name="GPSCoordinates" type="mds:GPSCoordinates" minOccurs="0"/><element name="Country" type="mds:ISO3166-01CountryCode" minOccurs="0"/><element name="Region" type="mds:ISO3166-2RegionCode" minOccurs="0"/><element name="PostingIdentifier" type="mds:TextualDescription"

minOccurs="0"/><element name="AdministrativeUnit" type="string" minOccurs="0"/><element name="PostalAddress" type="mds:PostalAddress" minOccurs="0"/><element name="InternalCoordinates" type="string" minOccurs="0"/><attribute name="id" type="ID" use="optional"/>

</complexType>


Name Definition

Place DS describing a location.

id Identification of the instance of the place description.

PlaceName The name of the place. Multiplicity is allowed to support multiple languages.

PlaceRole The role of the place: real –location, address- or fictional -setting- (e.g., Shooting location, Represented location, Postal Address, General Locator).

Planet The name of the planet. Default value is "Earth".

GPSCoordinates The coordinates of the place in a GPS system specified by an attribute.

Country The country code using ISO 3166-1.

Region The region code using ISO 3166-2.

PostingIdentifier The ZIP or Postal code.

AdministrativeUnit The name of the administrative unit to which the place belongs—e.g. a city, a town, a village, or even the USS Enterprise.

PostalAddress The postal address including street (or square) and house number or name.

InternalCoordinates Text used to identify location within unit with specified postal address (e.g., Flat 4, the drawing room, etc).


6.4 Description of importance or priority

The Weight DS is a primitive DS for describing with weights the importance or priority of the elements in a content description. This DS is currently included in the MDS XM document.

6.5 Entity-relationship graph

The Entity-Relationship Graph DS provides the tools for representing graphs of relationships among entities (MPEG-7 description elements which have an unique identifier—e.g. segments) and/or entity sets. Although


hierarchical structures such as trees are adequate for efficient access and retrieval, some relationships can not be expressed using such structures. The EntityRelationshipGraph DS (abbreviated as ER Graph) is defined to add flexibility in describing more general relationships among entities.

An Entity-Relation Graph is made up of two kinds of nodes: Entity Nodes and Relationship Nodes. An Entity Node represents an entity or set of entities participating as one argument—i.e. vertex—of a relation. A Relationship Node describes which entity nodes participate in a relation in the graph. In order to express the graph more compactly, an entity node may specify the relationships originating from the node and a relationship node can specify the relationships among its arguments (vertices). Relations in the ER Graph are n-ary relationship, but are not limited to binary relationships. The number of arguments of a relation is called its degree. In other words, the degree of a Relationship Node is equivalent to the arity of the relation it represents.

Clarify meaning and check that the use of the terms Relation vs. Relationship is consistent.

6.5.1 EntityRelationshipGraph DS


<complexType name="EntityNode">

<sequence maxOccurs="unbounded"><choice minOccurs="0" maxOccurs="unbounded">

<element name="EntityReference" type="mds:Reference"/><element name="EntityNode" type="mds:EntityNode"/>

</choice><element name="RelationshipNode" type="mds:RelationshipNode"

minOccurs="0" maxOccurs="unbounded"/></sequence><attribute name="id" type="ID"/>

</complexType>

<complexType name="Relationship">

<attribute name="id" type="ID" use="optional"/><attribute name="type" type="string" use="optional"/><attribute name="name" type="string" use="required"/><attribute name="degree" type="integer"/><attribute name="symmetric" type="boolean" use="optional"/><attribute name="property" use="optional">

<simpleType base="string"><enumeration value="AscendingOrder"/><enumeration value="DescendingOrder"/><enumeration value="PartialOrder"/><enumeration value="Equivalence"/>

</simpleType></attribute>

</complexType>

<complexType name="RelationshipNode">

<element name="Relationship" type="mds:Relationship"/><choice maxOccurs="unbounded">

<element name="RelationshipNode" type="mds:RelationshipNode"/><element name="EntityNode" type="mds:EntityNode"/>

</choice></complexType>

<complexType name="EntityRelationshipGraph">

<choice maxOccurs="unbounded"><element name="RelationshipNode" type="mds:RelationshipNode"/><element name="EntityNode" type="mds:EntityNode"/>

</choice><attribute name="id" type="ID" use="optional"/><attribute name="type" type="string" use="optional"/><attribute name="name" type="string" use="optional"/>

</complexType>


Semantic of the EntityNode DS.

Name Definition

EntityNode The entity node specifies the set of references to entities that participate in one argument (vertex) of a relationship. In a description, an entity node can also contain an arbitrary number of relationship nodes and other entity nodes to allow the compact definition of relationships between entities.

id Identifier of an instantiation of the EntityNode DS.

Reference Reference to an element that participate in a relationship.

RelationshipNode Relationship that applies to the EntityNode.

Semantic of the Relationship DS.

Name Definition

DirectionType The kind of relationship direction. It specifies two possible values for the relationship direction: "unidirectional" and "bi-directional". Bi-directional relationships are symmetric (if aRb, then bRa); unidirectional relationships are not symmetric.

PropertyType The kind of relationship property. It specifies two possible values for the relationship property: "AscendingOrder" and "DescendingOrder". When a relationship describes an order of entities based on a specific criterion, the order criterion will be included in the attribute name of the Relationship DS. The attribute property (of type PropertyType) of the Relationship DS will specify that the relationship is an order in ascending or descending form.

Relationship Definition of a relationship

id Identifier for an instance of the Relationship DS

Name Definition

type String that specifies the type of information described by the relationship (e.g. "temporal", “spatial", and “semantic"). For a possible taxonomy for segment relationship types see section 10.3.2 on the XM document..

name String that describes the semantics of the relationship (e.g. "before"). This attribute is mandatory.

level String which specifies the degree of generality of the information described by the relationship (e.g. "generic" and "specific").

degree Integer which specifies the degree of a relationship (i.e. the number of arguments). For example, the degree will equal 2 for a binary relationship. The degree of a relationship specifies the number of vertices of a relationship, i.e. the number of entity nodes participating in the relationship. By default, in a description, the degree of a relationship is determined by the number of entity nodes occurring in the relationship node.

symmetric Attribute that specifies the symmetry of a relationship. A relation is symmetric is the relationship is closed under all permutations of its arguments. For a binary relationship this means that for a symmetric relation aRb iff bRa.

property Attribute that specifies the properties of the relation. It has the following possible values: "AscendingOrder", "DescendingOrder", “PartialOrder", and “Equivalence". The “AscendingOrder" and “DescendingOrder" identifies a relationship that totally orders the entities based on some criterion. “PartialOrder" and “Equivalence" identify relations that are partial orders or equivalence relations. See below for a more formal definition of these properties.

Semantic of the RelationshipNode DS.

Name Definition

RelationshipNode Specification of a relationship of any degree (e.g. unary, binary or n-ary) and cardinality (e.g. 1:N) that applies to the set of entities referenceed within the given entity nodes. In a description, a relationship node can also contain an arbitrary number of entity nodes and other relationship nodes to allow the compact definition of relationships among entities.

Relationship Definition of the type of relationship

Semantic of the EntityRelationshipGraph DS.

Name Definition

EntityRelationshipGraph

Graph of relationship nodes and entity nodes.

A relationship node among entity nodes can be specified by instantiating first the RelationshipNode DS or the EntityNode DS. If a relationship node contains two entity nodes, the relationship applies from the first entity node to the second. If an entity node contains a relationship node that contains another entity node, the relationship applies for the first entity node to the second entity


Name Definition

node. A relationship node can include more entity nodes than the relationship's specified degree. In this case, the relationship applies to every contiguous set of number-degree entity nodes.

id Identifier of an instance of the EntityRelationshipGraph DS.

type String which specifies the type of the entity-relationship graph (e.g. "temporal").

name String which specifies the semantics of the entity-relationship graph. Specific types of graphs can be assigned a name.

6.5.1.2.1 Identifying Relations

Within the context of a relationship graph, a relation is considered to be uniquely identified by the following sets of attributes:

1. The name

2. The degree.

3. The type.

Because the type is optionally, if unspecified it is given the special value “null" when defining the identity of a relation.

This means that the following relationships are all distinct:

<Relation name="r" degree=1/><Relation name="r" degree=2/><Relation name="r" type="type1" degree=2/><Relation name="r" type="type2" degree=2/>

6.5.1.2.2 Relation Properties and Ordering

The property attribute of the Relation allows one to express different properties of the relationship, which may be used in systems that make inferences based on ER graphs. A relation can satisfy one of four properties: ascending order, descending order, partial order, and equivalence. These properties are only applicable to binary relations. Specifying these properties for non-binary relations is ill-defined.

A binary relation R is defined as follows: Let M be a set, and let certain ordered pairs (a, b) of elements of M be called labeled. If (a, b) is a labeled pair, we say that a is related to b by the binary relation R and write aRb. The following elementary properties are defined for a relation R:

1. Reflexivity: R is reflexive iff aRa for every a in M.

2. Symmetry: R is symmetric iff aRb implies bRa.

3. Transitivity: R is transitive iff aRb and bRc implies aRc.

4. Anti-symmetry. R is anti-symmetric iff aRb and bRa together imply a=b.

A binary relation R is said to be a partial ordering, and the set M itself is said to be partially ordered, if


1. R is reflexive.

2. R is transitive.

3. R is anti-symmetric.

A total order “<<" on a set M is defined as a relation that satisfies the following properties:

1. The order “<<" is a partial order.

2. Given any x and y in S, exactly one of the following three possibilities is true:

If a total ordering can be defined on a set M, then M is said to be ordered.

For a relationship graph the set M is defined to be the set of all entities reference in the graph—i.e. the set union of all entities referenced by entity nodes in the graph.

Need more work on formal definitions of relationships graphs. Are implied relationships considered? Must the relationship graph be completely expressed? Etc.

6.5.1.2.3 Rules for Instantiating the EntityRelationshipGraph DSs

The relationships among entities in a relationship graph can be listed by either the relationships (RelationshipNode DS) or the entities (EntityNode DS). The two examples below describe the same graph: “A has a relationship R with B":

<RelationshipNode> <Relationship name="R"/> <EntityNode id="A"/> <EntityNode id="B"/></RelationshipNode>

If the same relationship applies to many entities, it is more efficient to instantiate the graph by the relationship

<EntityNode id="A"> <RelationshipNode> <Relationship name="R"/> <EntityNode id="B"/> </RelationshipNode></EntityNode>

On the other hand, if we want to describe several relationships for a single entity, we list the entity first.

When an Entity Node includes a Relationship Node, the entity or set of entities defined by Entity Node have a Relationship to other entities as described by the contained Relationship Node—the example on the right above. The referenced entities within an Entity Node can be further grouped into Entity Node to recursively describe relationships of these. When a Relationship Node includes one or more Entity Nodes and the Relationship Node is itself included itself in an Entity Node, it means that the entity or set of entities defined by the parent Entity Node are related to the ones of the child Entity Node by the relationship specified by Relationship. When a Relationship Node includes two or more Entity Nodes and the Relationship Node is not included within an Entity Node, it means that the Relationship applies to between the entity or set of entities in the child Entity Nodes.

Relationship Nodes can also include other Relationship Nodes to represent graphs. This is the equivalent of including Entity Nodes insider other Entity Nodes. The Entity Nodes participating in a Relationship Node can be included recursively within a Relationship Node to represent other relationships among them.

If a relationship applies from one entity node to another, it applies from each referenced entity of the first entity node to each entity of the second entity node, but not among the entities of the same entity nodes. As an example, the following entity-relationship graph (a 1:2 binary relationship) is represented graphically on the right side. A N:M


relationship among entities would be specified by including N entities in the first entity node and M in the second one. In this example, the relationshionships expressed are aRb and aRc.

<EntityNode> <Reference id="A"> <RelationshipNode> <Relationship name="R"/> <EntityNode> <Reference id="B"> <Reference id="C"> </EntityNode> </RelationshipNode></EntityNode>

Figure 2: Multiple entities in an EntityNode

Relationships with a higher degree than two can be specified by grouping more than two EntityNode DSs in a RelationshipNode DS. By default, the degree of a relationship is equal to the number of entity nodes related by the relationship. As an example, assuming the relationship graph of Figure 2 involves a 1:1:1 ternary relationship, it could be represented by the following expression:

<RelationNode><Relationship name="R" degree="3"><EntityNode id="A"/><EntityNode id="B"/><EntityNode id="C"/>

</RelationNode>

A binary RelationNode DS (degree = 2) can also group more than two EntityNode DSs. In this case, the relationship applies to pairs of contiguous entity nodes. The order is the order in which the entities occur. As an example, the following relationship R:

<RelationNode><Relationship name="R" degree="2"><EntityNode id="A"/><EntityNode id="B"/><EntityNode id="C"/>

</RelationNode>

specifies that A is in relation R with B and that B is also in relation R with C: aRb and bRc.


6.6 Description of Time Series

The TemporalInterpolation D is a primitive D for describing a temporal interpolation by connected polynomials. This can be used to approximate variable values that change with time—such as an object position in a video. This element is currently included in the MDS XM document.

6.7 Scalable Series

Scalable series are datatypes for series of values (scalars or vectors). They allow the series to be scaled (downsampled) in a well-defined fashion. These elements are currently included in the MDS XM document.


7 Description of the media

7.1.1 MediaIdentification DS

The Media Identification DS contains description tools (Ds and DSs) that are specific to the identification of the master media (the instances are described by the Media Instance DS). The master media is the original copy from where the different available instances are produced.




<complexType name="MediaIdentification"><element name="Identifier" type="mds:UIdentifier" minOccurs="0"/><element name="AudioDomain" type="mds:ControlledTerm"

minOccurs="0" maxOccurs="unbounded"/><element name="VisualDomain" type="mds:ControlledTerm"

minOccurs="0" maxOccurs="unbounded"/><element name="ImageDomain" type="mds:ControlledTerm"

minOccurs="0" maxOccurs="unbounded"/><attribute name="id" type="ID"/>

</complexType>


Name Definition

MediaIdentification DS for the identification of the master media (the instances are described by the Media Instance DS).

id Identification of the instance of the MediaIdentification description

Identifier The identifier that uniquely identifies the particular and unique AV content (e.g., ISO’s ISAN). The Identifier may be the same for each Media instance, but differs in some of the other attributes. It also may be possible that each Media instance has a different Identifier.

AudioDomain The domain informs about the source, acquisition and/or use of an audio (e.g., “synthetic", “natural", “music", “speech", “mixed", “multi-track").

VisualDomain The domain informs about the source, acquisition and/or use of a video (e.g., “synthetic", “natural", “photographic", “graphic", “mixed", “chart", “visualization", “scanned document", “scientific", “medical", “aerial", “remote-sensing").

ImageDomain The domain informs about the source, acquisition and/or use of an image (e.g., “b/w photograph", “b/w graphic", “gray photograph", “gray graphic", “color photograph", “complex color graphic", “simple color graphic").

7.1.2 MediaFormat DS

The Media Format DS contains description tools (Ds and DSs) that are specific to the storage format of the media.




<complexType name="AspectRatio"> <attribute name="Height" type="float"/> <attribute name="Width" type="float"/> </complexType>

<complexType name="MediaFormat"><element name="FileFormat" type="mds:ControlledTerm"/>

<element name="AspectRatio" type="mds:AspectRatio" minOccurs="0"/><element name="System" type="mds:ControlledTerm" minOccurs="0"/>

<element name="SamplingStructure" type="mds:ControlledTerm" minOccurs="0"/><element name="Medium" type="mds:ControlledTerm" minOccurs="0"/><element name="Color" type="mds:ControlledTerm" minOccurs="0"/><element name="Sound" type="mds:ControlledTerm" minOccurs="0"/><element name="FileSize" type="nonNegativeInteger" minOccurs="0"/><element name="Length" type="mds:TimePointDuration" minOccurs="0"/><element name="AudioChannels" type="nonNegativeInteger" minOccurs="0"/>

<element name="AudioCodingFormat" type="mds:ControlledTerm" minOccurs="0"/><element name="AudioLanguage" type="language" minOccurs="0"/><attribute name="id" type="ID"/>

</complexType>


Editor's Note: The semantic of the AspectRatio complexType should be defined.

Name Definition

MediaFormat Description of the storage format of the media

id Identification of the instance of the media format description

FileFormat The file format or MIME type of the AV content instance.

AspectRatio The aspect ratio of images and frames (e.g., 2.35:1 theatrical aspect ratio).

System The analogue video system of the AV content (e.g., PAL, NTSC).

SamplingStructure The sampling structure used for the coding of the video, including scanning format (e.g., progressive, interlaced, CIF, QCIF).

Medium The physical storage medium on which the AV content is stored (e.g., tape, CD, DVD).

Color The color domain of the AV content (e.g., color, b/w, colored).

Sound The sound domain of the AV content (e.g., no sound, stereo, mono, dual).

Name Definition

FileSize The size, in bytes, of the file where the AV content is stored.

Length The duration of the AV content.

AudioChannels The number of audio channels in the AV content.

AudioLanguage The language used in the audio of the AV content.

AudioCodingFormat The coding format of the audio component of the AV content (e.g., AC-3, DTS).


7.1.3 MediaCoding DS

The Media Coding DS contains description tools (Ds and DSs) that are specific to the coding parameters of the media.




<complexType name="AspectRatio"> <attribute name="Height" type="nonNegativeInteger"/> <attribute name="Width" type="nonNegativeInteger"/> </complexType>

<complexType name="MediaCoding"><element name="Resolution" type="nonNegativeInteger" minOccurs="0"/><element name="FrameWidth" type="nonNegativeInteger" minOccurs="0"/><element name="FrameHeight" type="nonNegativeInteger" minOccurs="0"/>

<element name="AspectRatio" type="mds:AspectRatio" minOccurs="0"/><element name="FrameRate" type="float" minOccurs="0"/><element name="AudioSamplingRate" type="float" minOccurs="0"/><element name="CompressionFormat" type="mds:ControlledTerm" minOccurs="0"/><element name="QuantizerCoarseness" type="nonNegativeInteger" minOccurs="0"/>

<element name="SamplingStructure" type="mds:ControlledTerm" minOccurs="0"/><element name="PredictionMode" type="mds:ControlledTerm" minOccurs="0"/><element name="BitRate" type="float" minOccurs="0"/><element name="Bandwidth" type="float" minOccurs="0"/><attribute name="id" type="ID"/>

</complexType>


Name Definition

MediaCoding DS describing the coding parameters of the media

id Identification of the instance of the media coding.

Resolution The resolution of images and frames in dpi (dots per inch).

Name Definition

FrameWidth The width of images and frames in pixels.

FrameHeight The height of images and frames in pixels.

FrameRate Frame rate in Hz.

AudioSamplingRate The sampling rate for audio in Hz.

CompressionFormat The compression standard used in the coding of the AV content (e.g., MPEG-1, MP3, JPEG).

QuantizerCoarseness The sample accuracy in bits per sample.

PredictionMode The algorithm type used in the predicitive coding of the AV content (e.g., motion compensation mode, intra-frame).

BitRate The bit rate in bit/s required for the synchronous transmission of the coded AV content.

Bandwidth The bandwidth range in Hz covered by the coded AV content. It dependents on the acquisition filters and/or transcoding applied to the AV content.


7.1.4 MediaInstance DS

The Media Instance DS contains the description tools (Ds and DSs) that identify and locate the material instances.




<element name="MediaName" type="string"/><element name="InstanceLocator" type="mds:InstanceLocator"/>

<complexType name="InstanceLocator"><choice>

<element ref="MediaURL"/><element ref="MediaName"/>


<complexType name="MediaInstance"><element name="Identifier" type="mds:UIdentifier"/><element ref="InstanceLocator"/><attribute name="id" type="ID"/>

</complexType>

Name Definition

MediaInstance DS describing the identification and the localization of the AV content instances

id Identification of the instance of the MediaInstance description.

Identifier The identifier of the instance or copy (e.g., SMPTE UMID, instance number).

MediaName A unique reference to the location of the AV content instance not available on-line (e.g., tapes stored in the analogue archive of a broadcaster). It may be a unique identifier or a free text description.

MediaURL The URL type location of the AV content instance available in digital format on-line on the Internet or any Intranet.

InstanceLocator The location of the AV content instance. It is either a URL or a string for AV content not available on-line.


7.1.5 MediaProfile DS

The Media Profile contains the different description tools that allow to describe one profile of the media being described. The profile concept refers to the different variations that can be produced from a master media depending of the values chosen for the coding, storage format, etc. For each profile there can be one or more instances of the master media profile.




<complexType name="MediaProfile"><element name="MediaFormat" type="mds:MediaFormat"/><element name="MediaCoding" type="mds:MediaCoding"

minOccurs="0" maxOccurs="unbounded"/><element name="MediaInstance" type="mds:MediaInstance"

minOccurs="0" maxOccurs="unbounded"/><attribute name="id" type="ID"/><attribute name="master" type="boolean" use="default" value="false"/>

</complexType>


Name Definition

MediaProfile DS describing one profile of the media being described.

id Identification of the instance of the MediaProfile description.

master Attribute allowing the identification of the profile of the master of the media.

Name Definition

MediaFormat Description of the storage format of the master media profile

MediaCoding Description of the coding parameters of the master media profile

MediaInstance Identification and the localization of the master media profile


7.1.6 MediaInformation DS

The Media information DS contains description tools (Ds and DSs) that are specific to the storage media.




<element name="MediaInformation" type="mds:MediaInformation"/><complexType name="MediaInformation">

<element name="MediaIdentification" type="mds:MediaIdentification"/><element name="MediaProfile" type="mds:MediaProfile" maxOccurs="unbounded"/><attribute name="id" type="ID"/>

</complexType>


Name Definition

MediaInformation The MediaInformation DS contains one or more MediaProfile DSs. Each MediaInformation DS is related to one master, and its versions, containing one recording of a reality or a unique creation. If a concert may have been recorded in audio and in audio-visual media, this will imply two different masters of a recorded reality, and therefore will have two different description (being different media), although some descriptions may be identical (e.g., classification description, semantic description of the event). Afterwards each media may be available in different formats and qualities, yielding to the different Media Profiles.

id Identification of the instance of the MediaInformation description.

MediaIdentification Identification of the reality.

MediaProfile DS describing one profile of the content being described.

8 Description of the content creation & production

8.1.1 Creation DS

The Creation DS contains the description tools (Ds and DSs) related to the creation of the content, including places, dates, actions, materials, staff (technical and artistic) and organizations involved.




<simpleType name="TitleType" base="string"><enumeration value="original"/><enumeration value="alternative"/><enumeration value="popular"/>

</simpleType>

<element name="TitleText" type="mds:TextualDescription"/><element name="Title" type="mds:Title"/>

<complexType name="Title"><element ref="TitleText" minOccurs="0" maxOccurs="1"/><element name="TitleImage" type="mds:ImageLocator"

minOccurs="0" maxOccurs="1"/><element name="TitleVideo" type="mds:VideoSegmentLocator"

minOccurs="0" maxOccurs="1"/><element name="TitleAudio" type="mds:AudioSegmentLocator"

minOccurs="0" maxOccurs="1"/><attribute name="type" type="mds:TitleType"/>

</complexType>

<complexType name="Creator" base="mds:Person" derivedBy="extension"><element name="role" type="mds:ControlledTerm"/>

</complexType>

<element name="CreationMaterial"><complexType>

<element name="device_instrument" type="mds:ControlledTerm"/><element name="device_settings" type="mds:ControlledTerm"/>

</complexType></element>

<element name="Creation" type="mds:Creation"/><complexType name="Creation">

<element ref="Creation" minOccurs="0" maxOccurs="unbounded"/><element ref="Title" minOccurs="0" maxOccurs="unbounded"/><element name="CreationDescription" type="mds:StructuredAnnotation"

minOccurs="0" maxOccurs="unbounded"/><element name="Creator" type="mds:Creator"

minOccurs="0" maxOccurs="unbounded"/><element name="CreationLocation" type="mds:Place"

minOccurs="0" maxOccurs="unbounded"/><element name="CreationDate" type="mds:TimePoint"

minOccurs="0" maxOccurs="unbounded"/>

<element ref="CreationMaterial" minOccurs="0" maxOccurs="unbounded"/><attribute name="id" type="ID"/>

</complexType>


The semantic of the entities involved in the Creation DS is defined as follows:

Name Definition

Creation The Creation DS describes the creation of the content, including places, dates, actions, materials, staff (technical and artistic) and organizations involved. The self inclusion of the Creation DS allows to describe creation at several levels of detail.

Title The title(s) of the AV content. Multimodal titles (of multiple types) may be defined: text, image, video, and audio.

CreationDescription The description of the AV content making use of the 6W descriptors and a free text descriptor in the Annotation DS. This annotation is aimed to provide a normalized way to access to a simple description of the AV content.

Creator The creator(s) of the AV content. Using the Person DS it allows to describe individuals, organizations, groups, etc. Involved in the creation. The role D allows to qualify the role played in the creation.

CreationLocation The place where the content was created.

CreationDate The date when the content was created.

CreationMaterial The devices and instruments used for the creation of the content (e.g., types of device, lens, films, instruments, settings, etc.).

The Title DS semantic is given by:

Name Definition

Title The title of the AV content, which may contain a textual, an image, an audio and a video title. The latter are audiovisual identifiers of the content. The type of the title (e.g. original, popular, opus number) is specified using an attribute.

TitleText The (textual) title of the AV content. The language in which the title is given is specified using an attribute.

TitleImage The title of the AV content in image form (e.g. thumbnail).

TitleAudio The title of the AV content in audio form (e.g. jingle).

TitleVideo The title of the AV content in video form (e.g. video clip).



8.1.2 Classification DS

The Classification DS contains the description tools (Ds and DSs) that allow classifying the material.


<complexType name=”ParentalGuidance”> <element name="Country" type="ISO3166-1CountryCode" minOccurs="0" maxOccurs="unbounded"/>

<element name="ParentalRatingScheme" type="mds:ControlledTerm" minOccurs="0" maxOccurs="1"/> <element name="ParentalRatingValue" type="mds:ControlledTerm" minOccurs="0" maxOccurs="1"/> <element name="MinimumAge" type="nonNegativeInteger" minOccurs="0" maxOccurs="1"/>

</complexType>

<complexType name="Classification"><element ref="Country" minOccurs="0" maxOccurs="unbounded"/><element name="Language" type="language" minOccurs="0" maxOccurs="unbounded"/><element name="Genre" type="mds:ControlledTerm"

minOccurs="0" maxOccurs="unbounded"/><element name="PackagedType" type="mds:ControlledTerm"

minOccurs="0" maxOccurs="unbounded"/><element name="Purpose" type="mds:ControlledTerm"

minOccurs="0" maxOccurs="unbounded"/> <element name=”AgeClassification” type=”mds:ControlledTerm” minOccurs=”0” maxOccurs=”unbounded”/>

<element name="AgeClassificationParentalGuidance" type="mds:ControlledTermParentalGuidance" minOccurs="0" maxOccurs="unbounded"/>

<element name="MarketClassification" type="mds:ControlledTerm" minOccurs="0"/><attribute name="id" type="ID"/>

</complexType>


Semantics of the ParentalGuidance DS

Name Definition

ParentalGuidance The parental guidance classification, using a classification scheme (e.g., MPA, ICRA, PICS) and a rating value, of the AV content.

Country The country code using ISO 3166-1 to indicate the country for which the parental guidance description is specified.

ParentalRatingScheme The classification scheme used in determining the rating value (e.g., MPA, ICRA, PICS)

ParentalRatingValue The rating value according to the rating scheme (e.g., PG-13 according to MPA)

MinimunAge The minimum recommended age in years of the end user.

Semantic of the Classification DS

Name Definition

Classification Classification of the AV content.

Country The country code using ISO 3166-1 from where the AV content comes. It may be different than the location where it was created.

Language The language of the AV content.

Genre The genre that applies to the subject of the AV content. It can specify styles for a specific genre (e.g, Contemporary Classical Music).

PackagedType The packaged type that applies to the subject of the AV content. The packaged type classifies AV content from a point of view of types of program, without considering genre classification.

Purpose The original or principal purpose for which the AV content was created (e.g. broadcasting, interactive show, meteorology studies, surveillance).

AgeClassification The targeted age range, using a classification scheme, of the AV content.

ParentalGuidance The parental guidance classification, using a classification scheme (e.g., MPA, ICRA, PICS) and rating, of the AV content.

MarketClassification The targeted market , using a classification scheme, of the AV content.

Editor’s Notes: Should PackagedType be renamed Subject? Are AgeClassification and ParentalGuidance not redundant?Non-normative issues (creation, use and examples) are described in the MDS XM document.8.1.3 RelatedMaterial DS

The Related Material DS contains the description tools (Ds and DSs) related to additional information about the AV content available in other materials.




<complexType name="RelatedMaterial"><element name="MediaType" type="mds:ControlledTerm"/><element name="MediaLocator" type="mds:MediaLocator" minOccurs="0"/><element ref="MediaInformation" minOccurs="0"/><element ref="CreationMetaInformation" minOccurs="0"/><element ref="UsageMetaInformation" minOccurs="0"/><attribute name="id" type="ID"/><attribute name="Master" type="boolean" use="default" value="true"/>

</complexType>

Name Definition

RelatedMaterial Description of the materials containing additional information about the AV content.

Master Boolean attribute that allows to identify if the referenced related material is the master.

MediaType The media type of the referenced related material (e.g., Web page, printed book, AV)

MediaLocator The locator of the referenced related material.

MediaInformation The Media Information description of the referenced related material.

CreationMetaInformation The Creation Meta Information description of the referenced related material.

UsageMetaInformation The Usage Meta Information description of the referenced related material.


8.1.4 Creation MetaInformation DS

The Creation Meta information DS contains the description tools (Ds and Dss) that carry author-generated information about the generation/production process of an AV program or an image that cannot usually be extracted from the content itself (the actors creating the content can be extracted from the content). This information is related to the material but it is not explicitly depicted in the actual content.

The Creation Meta Information DS contains:

Information about the creation not perceived in the material (e.g., the author of the script, the director, the character description, the target audience, the rating, etc.) and information about the creation perceived in the material (e.g., the actors in the video, the players in a concert, ), and

Classification related information (target audience, style, genre, rating, etc.).

Note that this DS may be attached to any Segment of the structure DS. Indeed, a complete visual description may contain segments that are annotated with more details and these segments may be produced and used independently and/or in different AV materials and segments.

The description of actors within the Creation DS may be linked with the description of the content in terms of characters (to be defined within the Semantic DS?), using a Cast (Creator-Character Link) DS.




<element name="CreationMetaInformation" type="mds:CreationMetaInformation"/><complexType name="CreationMetaInformation">

<element ref="Creation" minOccurs="0"/><element name="Classification" type="mds:Classification" minOccurs="0"/>

<element name="RelatedMaterial" type="mds:RelatedMaterial" minOccurs="0"/><attribute name="id" type="ID"/>

</complexType>


Name Definition

CreationMetaInformation Description of author-generated information about the generation and production process of the AV content.

ID Identification of the instance of the creation meta information

Creation By whom, when and where the AV content was created.

Classification User oriented and service oriented classification of the AV content

RelatedMaterial Additional information about the AV content.



9 Description of the content usage

9.1.1 Rights DS

The Rights DS contains the description tools (Ds and DSs) related to the right holders of the annotated content (IPR) and the Access Rights.




<complexType name="Rights" content="mixed"><element name="RightsId" type="mds:UIdentifier"

minOccurs="0" maxOccurs="unbounded"/><attribute name="id" type="ID"/>

</complexType>


Name Definition

Rights Description of a link to the right holders and to the access rights information. It contains a unique reference to an identifier under management by an external authority. It covers the IPMP requirement regarding not to provide explicitly Rights Owners identification, but an ID (such as specified in the MPEG-4 IP Identification Data Set) that will enable current Rights Owner information about content to be accessed from appropriate rights management databases.

ID Identification of the instance of the rights DS

RightsId The identifier that references the IPR of the AV content under management by an external IPMP system.


9.1.2 UsageRecord DS

The UsageRecord DS contains the description tools (Ds and DSs) related to the use (broadcasting, on demand delivery, CD sales, etc.) of the content, that is its life.




<complexType name="UsageRecord"><element name="Type" type="mds:ControlledTerm" minOccurs="0"/><element name="Channel" type="mds:ControlledTerm" minOccurs="0"/><element name="Place" type="mds:Place" minOccurs="0"/><element name="Date" type="mds:TimePoint" minOccurs="0"/>

<element name="Audience" type="nonNegativeInteger" minOccurs="0" maxOccurs="1"/>

<element name="Distributor" type="mds:Person" minOccurs="0"/><element name="Financial" type="mds:Financial" minOccurs="0"/><element name="Rights" type="mds:Rights" minOccurs="0"/><attribute name="id" type="ID"/>

</complexType>


Name Definition

UsageRecord Description of the use (e.g., broadcasting, on demand delivery, CD sales) of the AV content

ID Identification of the UsageRecord DS instance.

Type The type (or medium) of use of the AV content (e.g., broadcasting, CD, DVD).

Channel The channel of use of the AV content. It may refer to a broadcast channel, a music company, etc.

Place The place of use of the AV content. It refers to the place from where the AV content is distributed.

Date The date of use of the AV content. In the case of a persistent medium (e.g., DVD) the date of publication.

Audience The number of users received by the AV content in the concrete use described in the UsageRecord DS instance.

Distributor The distributor of the AV content in the concrete use described in the UsageRecord DS instance. Usually it will be an organization.

Financial The financial information related to the concrete use described in the UsageRecord DS instance.

Rights The additional rights information related to the concrete use described in the UsageRecord DS instance.


9.1.3 Financial DS

The Financial DS contains information related to the costs generated and income produced by AV content.




<element name="InternationalPrice"><complexType content="empty">

<attribute name="currency" type="mds:ISO4217CurrencyCode"/><attribute name="value" type="decimal"/>


<complexType name="Cost"><element name="CostType" type="mds:ControlledTerm"/><element ref="InternationalPrice"/><attribute name="id" type="ID"/>

</complexType>

<complexType name="Income"><element name="IncomeType" type="mds:ControlledTerm"/><element ref="InternationalPrice"/><attribute name="id" type="ID"/>

</complexType>

<complexType name="Financial"><element name="Cost" type="mds:Cost" minOccurs="0" maxOccurs="unbounded"/><element name="Income" type="mds:Income" minOccurs="0" maxOccurs="unbounded"/><attribute name="id" type="ID"/>

</complexType>


Name Definition

Financial Description of the cost and income generated by the AV content.

ID Identification of the Financial DS instance.

CostType The type of a partial cost associated to the AV content.

IncomeType The type of a partial income associated to the AV content.

InternationalPrice The currency (using ISO 4217) and value of a partial cost or income associated to the AV content.


9.1.4 UsageMetaInformation DS

The Usage Meta information DS contains the description tools (Ds and DSs) that carry information about the usage process of an AV program or an image. The Usage Meta Information DS contains:

Information about the rights for using the material

Information about the ways and means to provide service over the material (e.g., edition, emission, etc.) and the results of the service provision (e.g., audience)

Financial information about the financial results of the production (in the Financial DS within the UsageMetaInformation DS) and of the publication (in the Financial DS within each Publication DS) of the material. .


<element name="UsageMetaInformation" type="mds:UsageMetaInformation"/><complexType name="UsageMetaInformation">

<element name="Rights" type="mds:Rights" minOccurs="0"/><element name="UsageRecord" type="mds:UsageRecord"

minOccurs="0" maxOccurs="unbounded"/><element name="Financial" type="mds:Financial" minOccurs="0"/><attribute name="id" type="ID"/>

</complexType>


Name Definition

UsageMetaInformation Description of the usage process of the AV Content. This information is related to the material but it is not explicitly linked to the actual content, and what is more important, many components may vary over time.

ID Identification of the UsageMetaInformation DS instance.

Rights Who owns the AV content, and how the AV can be used.

UsageRecord Where, when, how, by whom the AV content was used.

Financial How much the creation of the AV content was, how much the publication of the AV content is making. Part of it (income) will vary with time.

10 Description of the structural aspects of the content

The physical and logical aspects of AV content is described by segment DSs segment features, and SegmentRelationshipGraph DSs. The segment DSs may be used to form segment trees to define the structure of the AV content, i.e. a table of contents. The segment features describe features of segments. The SegmentRelationshipGraph DS is used to describe temporal, spatial, and spatio-temporal relationships, among others, between segments that are not described by the tree structures.

10.1 Segment

10.1.1 Segment DS

A segment represents a section of an AV content item. Its role is to define the common properties of the subclasses AudioSegment DS, StillRegion DS, MovingRegion DS and VideoSegment DS. Therefore, it may have both spatial and temporal properties. A temporal segment may be a set of samples in an audio sequence, represented by an AudioSegment DS, or a set of frames in a video sequence, represented by a VideoSegment DS. A spatial segment may be a region in an image or a frame in a video sequence, represented by a StillRegion DS. Finally, a spatio-temporal segment may correspond to a moving region in a video sequence, represented by a MovingRegion DS. The Segment DS is abstract and cannot be instantiated on its own: it is used to define the common properties of its subclasses.

Editor’s Note: The specification of an AudioVisualSegment DS is being discussed.Editor's Note: The four types of segments should include MediaLocators.




<simpleType name="DecompositionDataType" base="string">

<enumeration value="spatial"/><enumeration value="temporal"/><enumeration value="spatio-temporal"/><enumeration value="MediaSource"/>

</simpleType>

<complexType name="SegmentDecomposition">

<element ref="Segment" minOccurs="1" maxOccurs="unbounded"/><attribute name="DecompositionType" type="mds:DecompositionDataType"

use="required"/><attribute name="Overlap" type="boolean" use="default" value="false"/><attribute name="Gap" type="boolean" use="default" value="false"/>

</complexType><element name="Segment" type="mds:Segment"/>

<complexType name="Segment" abstract="true">

<element name="MediaInformation" type="mds:MediaInformation" minOccurs="0" maxOccurs="1"/>

<element name="CreationMetaInformation" type="mds:CreationMetaInformation" minOccurs="0" maxOccurs="1"/>

<element name="UsageMetaInformation" type="mds:UsageMetaInformation" minOccurs="0" maxOccurs="1"/>

<element name="StructuredAnnotation" type="mds:StructuredAnnotation" minOccurs="0" maxOccurs="unbounded"/>

<element name="MatchingHint" type="mds:MatchingHint" minOccurs="0" maxOccurs="unbounded"/>

<element name="PointOfView" type="mds:PointOfView" minOccurs="0" maxOccurs="unbounded"/>

<element name="SegmentDecomposition" type="mds:SegmentDecomposition" minOccurs="0" maxOccurs="unbounded"/>

<attribute name="id" type="ID" use="required"/><attribute name="href" type="uriReference" use="optional"/><attribute name="idref" type="IDREF" refType="Segment" use="optional"/>

</complexType>


Semantic of the SegmentDecomposition DS

Name Definition

SegmentDecomposition Decomposition of a segment into one or more segments.

DecompositionDataType Datatype defining the kind of segment decomposition. The possible kinds of segment decomposition are spatial, temporal, spatio-temporal, and media source.

DecompositionType Attribute that specifies the decomposition type of a segment.

Overlap Boolean that specifies if the segments resulting from a segment decomposition overlap in time or space. This attribute value is “false" by default.

Gap Boolean that specifies if the segments resulting from a segment decomposition leave gaps in time or space. This attribute value is “false" by default.

Segment Set of (Sub-)segments that form the decomposition

Semantic of the Segment DS

Name Definition

Segment Abstract structure that represents a fragment or section of the AV content. For example, a segment could be a region in an image or a moving region in a video sequence.

A segment can be decomposed into other segments through the SegmentDecomposition DS.

id Identifier of a segment description.

href Reference to description where referenced segment is contained.

idref Identifier of a referenced segment that is contained within the description defined by href. The referenced element is constraint to be a Segment type. href and idref attributes are used to represent the equivalence of segment's descriptions. It avoids repeating the description of the same segment appearing in two different segment trees.


Name Definition

MediaInformation Media information related to the segment and its descendants.

CreationMetaInformation Creation meta information related to the segment and its descendants.

UsageMetaInformation Usage meta information related to the segment and its descendants.

SegmentDecomposition Decomposition of the segment into sub-segments.

MatchingHint Relative importance of descriptors and/or components of descriptors in the segment.

PointOfView Relative importance of the segment given a specific point of view or importance criteria.

Annotation Textual annotation and description of people, animals, objects, actions, places, time, and/or purpose which are instantiated in the segment.

10.1.1.2.1 Abstract property

The Segment DS is abstract in the meaning of Object Oriented (OO) languages such as Java or C++. This means that it cannot be instantiated as such but only as one of its specialized subclass (in the OO sense). The role of the Segment DS is to define the common properties of its subclasses. The list of specialized subclasses of the Segment DS is as follows:

The VideoSegment DS

The StillRegion DS

The MovingRegion DS

The AudioSegment DS

10.1.1.2.2 Segment Connectivity

A segment is not necessarily connected, but may be composed of several non-connected sub-segments. Connectivity refers here to both spatial and temporal domains. Attributes such as "SpatialConnectivity" and DS such as MediaTimeMask, are used for the subclasses of the Segment DS to define the temporal and spatial connectivity in the following sense:

A temporal segment is said to be temporally connected if it is a sequence of continuous video frames or audio samples.

A spatial segment is said spatially connected if it is a group of connected pixels.

A spatio-temporal segment is said spatially and temporally connected if the temporal segment where it is instantiated is temporally connected and if each one of its temporal instantiation in a frame is spatially connected.

Figure 3 illustrates several examples of temporal or spatial segments and their connectivity. Figure 3.a) and b) illustrate a temporal and a spatial segment composed of a single connected component. Figure 3.c) and d) illustrate a temporal and a spatial segment composed of three connected components. Note that, in the latter case, the descriptors and DSs attached to the segment are global to the union of the connected components building the segment. At this level, it is not possible to describe individually the connected components of the segment. If


connected components have to be described individually, then the segment has to be decomposed (see section 10.1.1.2.3) into various sub-segments corresponding to its individual connected components.

Figure 3: Examples of segments: a) and b) segments composed of one single connected component; c) and d) segments composed of three connected components

It is known that these attributes and the features of specialized segments are not relevant in all cases. For example, describing an audio segment as spatially connected is irrelevant. Furthermore, some descriptors will be irrelevant for non-connected segments. For example, defining the contour of a non connected segment is irrelevant.

10.1.1.2.3 Decomposition of segments (SegmentDecomposition)

The Segment DS is recursive i.e. it may be subdivided into sub-segments, and thus may form a hierarchy (tree). The resulting segment tree is used to define the media source, the temporal and / or spatial structure of the AV content. For example, a video program may be temporally segmented into scenes, shots, and sub-shots; a table of contents may thus be generated based on this structure. Similar strategies can be used for spatial and spatio-temporal segments. A segment may also be decomposed into various media sources such as various audio tracks or various viewpoints from several cameras. The hierarchical decomposition is useful to design efficient search strategies (global search to local search). It also allows the description to be scalable: A segment may be described by its direct set of descriptors and DSs, but it may also be described by the union of the descriptors and DSs that are related to its sub-segments.

Note that a segment may be subdivided into sub-segments of different types, e.g. a video segment may be decomposed in moving regions that are themselves decomposed in static regions. As it is done in a spatio-temporal space, the decomposition has to be described by a set of attributes defining the type of sub-division: temporal, spatial or spatio-temporal. Moreover, the spatial and temporal subdivisions may leave gaps and overlaps between the sub-segments, as explained below. The four possibilities are the following:

No gaps, no overlaps (partition in the mathematical sense)

No gaps, overlaps


Gaps, no overlaps

Gaps, overlaps

Several examples of decompositions are described for temporal segments in Figure 4. Figure 4.a) and b) describe two examples of decompositions without gaps and overlaps. In both cases the union of the children correspond exactly to the temporal extension of the father, even if the father is itself no connected (Figure 4.b). Figure 4.c) shows an example of decomposition with gap but no overlap. Finally, Figure 4.d) illustrates a more complex case where the father is composed of two connected components and its decomposition creates three children: the first one is itself composed of two connected components, the two remaining children are composed of a single connected component. Tthe decomposition allows gap and overlap.

Figure 4: Examples of SegmentDecomposition: a) and b) SegmentDecompositions without gap nor ovelap; c) and d) SegmentDecompositions with gap or ovelap.

Note that, in any case, the decomposition implies that the union of the spatio-temporal space defined by the children segments is included in the spatio-temporal space defined by their ancestor segment (Children are contained in their ancestors).

Note that this decomposition can be viewed as a specific relationship between segments. It could have been defined as a particular case of general relationships. However, because of its important role in the structural DSs, it is declared here as a specific entity.


10.1.2 VideoSegment DS

A VideoSegment DS is a specific Segment DS. It inherits all the properties of the Segment DS (attributes, decomposition, descriptors and DSs). The VideoSegment DS describes a set of frames belonging to a video sequence. A single frame extracted from a video sequence is considered as a video segment. The frames may be contiguous in time or not. This is defined by the TimeMask DS.


<element name="VideoSegment" type="mds:VideoSegment" equivClass="Segment"/><complexType name="VideoSegment" base="mds:Segment" derivedBy="extension">

<element name="MediaTime" type="mds:MediaTime" minOccurs="1" maxOccurs="1"/><element ref="MediaTimeMask" minOccurs="0" maxOccurs="1"/><element ref="ColorSpace" minOccurs="0" maxOccurs="1"/><element ref="ColorQuantization" minOccurs="0" maxOccurs="1"/><element ref="GofGopColorHistogram" minOccurs="0" maxOccurs="1"/><element ref="ColorLayout" minOccurs="0" maxOccurs="1"/><element ref="CameraMotion" minOccurs="0" maxOccurs="1"/><element ref="MotionActivity" minOccurs="0" maxOccurs="1"/><element ref="Mosaic" minOccurs="0" maxOccurs="1"/><attribute name="idref" type="IDREF" refType="VideoSegment" use="optional"/>

</complexType>

Editor’s Note: min/maxOccursPar are not currently supported by the MPEG-7 DDL.

Editor’s Note: The occurrence of all the visual descriptions should be 0..* to allow descriptor values with different parameters (e.g. color histograms for different color spaces and color quantizations).10.1.2.2 Description Scheme Semantics

The VideoSegment inherits all the properties from the abstract segment DS. In particular, it includes all the attributes, Ds and DSs defined in the Segment DS with the same syntax and semantic.

Name Definition

VideoSegment Set of frames from a video sequence. A VideoSegment can be of duration one frame (single frame belonging to a video sequence). Motion information can be associated with a VideoSegment even if it is of temporal duration one frame. The frames do not need to be continuous in time (see the MediaTimeMask DS).

idref Identifier of a referenced VideoSegment that is contained within the description defined by href (see Segment DS). The referenced element is constraint to be a VideoSegment type.

MediaTime Start MediaTime of the video segment and its duration. If the video segment is non-connected, the duration should be equal to the duration of the smallest connected video segment that includes the unconnected video segment. This DS is mandatory.

MediaTimeMask If absent, the segment is composed of a single connected component and refers to the entire interval of frames defined by MediaTime. If present, the segment refers to a collection of shorter non-overlapping subintervals in time. These subintervals are defined in MediaTimeMask.

ColorSpace Description of the color space used for the color Ds and DSs of the video segment (see the Visual part of the standard)

ColorQuantization Description of the color quantization used for the color Ds and DSs of the video segment (see the Visual part of the standard)

GofGopColorHistogram Description of the color histogram for a group of frames (see the Visual part of the

Name Definition

standard)

ColorLayout Description of the color layout of the region (see the Visual part of the standard)

CameraMotion Description of the camera work in the video segment (see the Visual part of the standard)

MotionActivity Description of the motion activity within the video segment (see the Visual part of the standard)

Mosaic Description of mosaic of a video shot. A mosaic is constructed by aligning and warping the frames of a shot upon each other in a single reference system giving a panoramic view of the whole shot in one single image.


10.1.3 StillRegion DS

A StillRegion DS is a specific Segment DS. It inherits all the properties of the Segment DS (attributes, decomposition, descriptors and DSs). The StillRegion DS describes a spatial area (e.g. set of pixels) belonging to a still image or a single frame of a video sequence. The still images can be natural images or synthetic images. A still image is a particular case of StillRegion DS. The spatial area may be connected or not. This is defined by the SpatialConnectivity attribute.

Editor's note: The way to describe the SpatialConnectivity should be revised in view of the changes made for the VideoSegment and AudioSegment.

A possible solution can be based on the RegionLocator (for Still Region DS) as shown below. Localization for regions is expensive; therefore, redundant descriptions should be avoided (segment decompositions).

<complexType name="MediaSpatiallMask"><element ref="RegionLocator" minOccursPar="NumberOfSpatialRegions"

maxOccursPar="NumberOfSpatialRegions"/><attribute name="NumberOfSpatialRegions" type="positiveInteger" use="required"/>

</complexType>




<element name="StillRegion" type="mds:StillRegion" equivClass="Segment"/><complexType name="StillRegion" base="mds:Segment" derivedBy="extension">

<element ref="ColorSpace" minOccurs="0" maxOccurs="1"/><element ref="ColorQuantization" minOccurs="0" maxOccurs="1"/><element ref="DominantColor" minOccurs="0" maxOccurs="1"/><element ref="ColorHistogram" minOccurs="0" maxOccurs="1"/><element ref="BoundingBox" minOccurs="0" maxOccurs="1"/><element ref="RegionShape" minOccurs="0" maxOccurs="1"/><element ref="ContourShape" minOccurs="0" maxOccurs="1"/><element ref="ColorStructureHistogram" minOccurs="0" maxOccurs="1"/><element ref="ColorLayout" minOccurs="0" maxOccurs="1"/>

<element ref="CompactColor" minOccurs="0" maxOccurs="1"/><element ref="HomogeneousTexture" minOccurs="0" maxOccurs="1"/><element ref="TextureBrowsing" minOccurs="0" maxOccurs="1"/><element ref="EdgeHistogram" minOccurs="0" maxOccurs="1"/><attribute name="SpatialConnectivity" type="boolean" use="required"/><attribute name="idref" type="IDREF" refType="StillRegion" use="optional"/>

</complexType>

Editor’s Note: The occurrence of all the visual descriptions should be 0..* to allow descriptor values with different parameters (e.g. color histograms for different color spaces and color quantizations).

Editor’s Note: The StillRegion DS could contain a MediaTimePoint DS.


The StillRegion inherits all the properties from the abstract segment DS. In particular, it includes all the attributes, Ds and DSs defined in the Segment DS with the same syntax and semantic.

Name Definition

StillRegion Set of pixels from an image or a frame in a video sequence. Note however, that no motion information can be used to describe a still region. Still images can be natural images or synthetic images. A still image is a particular case of still region. The pixels do not need to be connected (see the SpatialConnectivity attribute).

SpatialConnectivity Boolean which specifies if a still region is connected in space, i.e. connected pixels.

idref Identifier of a referenced StillRegion that is contained within the description defined by href (see Segment DS). The referenced element is constraint to be a StillRegion type.

ColorSpace Description of the color space used for the color Ds and DSs of the still region (see the Visual part of the standard).

ColorQuantization Description of the color quantization used for the color Ds and DSs of the still region (see the Visual part of the standard).

DominantColor Description of the Dominant color of the region (see the Visual part of the standard).

ColorHistogram Description of the color histogram of the region (see the Visual part of the standard).

BoundingBox Description of shape of the region (see the Visual part of the standard)

RegionShape Description of the region shape (see the Visual part of the standard)

ContourShape Description of the region shape (see the Visual part of the standard)

ColorStructureHistogram Description of the color structure histogram of the region (see the Visual part of the standard)


CompactColor Description of the compact color of the region (see the Visual part of the standard)

Name Definition

HomogeneousTexture Description of the homogeneous texture of the region (see the Visual part of the standard)

TextureBrowsing Description of the texture browsing of the region (see the Visual part of the standard)

EdgeHistogram Description of the edge histogram of the region (see the Visual part of the standard)


10.1.4 MovingRegion DS

A MovingRegion DS is a specific Segment DS. It inherits all the properties of the Segment DS (attributes, decomposition, descriptors and DSs). The MovingRegion DS describes a spatio-temporal area belonging to a video sequence. The spatio-temporal area may be connected in several ways. This is defined by the SpatialConnectivity attribute and by the MediaTimeMask DS in time.

Editor's Note: The way to describe the SpatialConnectivity and the TemporalConnectivity should be revised in view of the changes made for the VideoSegment and AudioSegment.

A possible solution can be based on the SpatioTemporalLocator (for Moving Region DS). The SpatioTemporalLocatorDS can currently describe several non-connected regions. Then, SpatioTemporalLocatorDS can have the role of a MediaSpatioTemporalMask DS. Localization for regions is expensive; therefore, redundant descriptions should be avoided (segment decompositions).




<element name="MovingRegion" type="mds:MovingRegion" equivClass="Segment"/><complexType name="MovingRegion" base="mds:Segment" derivedBy="extension">

<element ref="MediaTime" minOccurs="1" maxOccurs="1"/><element name="MediaTime" type="mds:MediaTime" minOccurs="1" maxOccurs="1"/><element ref="ColorSpace" minOccurs="0" maxOccurs="1"/><element ref="ColorQuantization" minOccurs="0" maxOccurs="1"/><element ref="GofGopColorHistogram" minOccurs="0" maxOccurs="1"/><element ref="ColorLayout" minOccurs="0" maxOccurs="1"/><element ref="MotionTrajectory" minOccurs="0" maxOccurs="1"/><element ref="ParametricMotion" minOccurs="0" maxOccurs="1"/><element ref="SpatioTemporalLocator" minOccurs="0" maxOccurs="1"/><attribute name="SpatialConnectivity" type="boolean" use="required"/><attribute name="idref" type="IDREF" refType="MovingRegion" use="optional"/>

</complexType>

Editor’s Note: The occurrence of all the visual descriptions should be 0..* to allow descriptor values with different parameters (e.g. color histograms for different color spaces and color quantizations).


The MovingRegion inherits all the properties from the abstract segment DS. In particular, it includes all the attributes, Ds and DSs defined in the Segment DS with the same syntax and semantic.

Name Definition

MovingRegion Set of pixels from a set of frames in a video sequence. The pixels do not need to be connected temporally (i.e. in continuous frames) or spatially (i.e. connected pixels).

SpatialConnectivity Boolean that specifies if a still region is connected in space, i.e. connected pixels. A MovingRegion is said spatially connected if each one of its temporal instantiation in a frame is spatially connected.

idref Identifier of a referenced MovingRegion that is contained within the description defined by href (see Segment DS). The referenced element is constraint to be a MovingRegion type.

MediaTime Start MediaTime of the moving region and its duration. If the moving region is non-connected, the duration should be equal to the duration of the smallest connected moving region that includes the unconnected moving region. This DS is mandatory in the MovingRegion DS.

MediaTimeMask If absent, the moving region is composed of a single connected temporal component and refers to the entire temporal interval defined by MediaTime. If present, the moving region refers to a collection of shorter non-overlapping subintervals in time. These subintervals are defined in MediaTimeMask.

ColorSpace Description of the color space used for the color Ds and DSs of the moving region (see the Visual part of the standard)

ColorQuantization Description of the color quantization used for the color Ds and DSs of the moving region (see the Visual part of the standard)

GofGopColorHistogram Description of the color histogram for a group of frames (see the Visual part of the standard)


MotionTrajectory Description of the motion trajectory of the region (see the Visual part of the standard)

ParametricMotion Description of the motion of the moving region using parametric models (see the Visual part of the standard)

SpatioTemporalLocator Description of the spatio-temporal shape of the region (see the Visual part of the standard)


10.1.5 VideoText DS

A VideoText DS is a specific MovingRegion DS. It inherits all the properties of the MovingRegion DS (attributes, decomposition, descriptors and DSs). The VideoText DS describes a text region on a still image or a set of video frames. The text region is described by the superimposed characters, syntactic attributes of the text such as


language, font size, font style, and other information such as its time, color, motion, and spatial location through its derivation from the MovingRegion DS. This DS is defined in the MDS XM document and has not been promoted to WD.

10.1.6 AudioSegment DS

An AudioSegment DS is a specific Segment DS devoted to audio information. It inherits all the properties of the Segment DS (attributes, decomposition, descriptors and DSs). The AudioSegment DS describes a temporal segment: a group of samples from an audio program. The audio segment may be a composed of a single interval or not. This is defined by the TimeMask DS.




<element name="AudioSegment" type="mds:AudioSegment" equivClass="Segment"/><complexType name="AudioSegment" base="mds:Segment" derivedBy="extension">

<element name="MediaTime" type="mds:MediaTime" minOccurs="1" maxOccurs="1"/><element ref="MediaTimeMask" minOccurs="0" maxOccurs="1"/><element name="AudioDescriptor" type="audio:AudioDescriptorType"

minOccurs="0" maxOccurs="unbounded"/><attribute name="idref" type="IDREF" refType="mds:AudioSegment"

use="optional"/></complexType>


The AudioSegment inherits all the properties from the abstract Segment DS. In particular, it includes all the attributes, Ds and DSs defined in the Segment DS with the same syntax and semantic.

Name Definition

AudioSegment A DS that describes audio material that falls within a time interval. The material may be limited to subintervals that do not need to be contiguous. Descriptors attached to the segment refer to this material.

idref Identifier of a referenced AudioSegment that is contained within the description defined by href (see Segment DS). The referenced element is constraint to be a AudioSegment type.

MediaTime Start time of the audio segment and its MediaDuration. This DS is mandatory.

MediaTimeMask If absent, the segment refers to the entire interval of continuous audio data defined by MediaTime. If present, the segment refers to a collection of shorter non-overlapping subintervals. These subintervals are defined in MediaTimeMask.

MediaTime

(within MediaTimeMask)

Start time and duration of each subinterval of a non-connected segment. Subintervals do not overlap, and all are included within the limits of the segment. There may be a gap at beginning and/or end.

AudioDescriptor One or more audio descriptors (see Audio WD document).

Name Definition


10.2 Segment Features

Segment features are descriptors and description schemes representing features of segments. Segment features are the MediaTimeMask for temporal segments, i.e. VideoSegment DS, MovingRegion DS, and AudioSegment DS; the Mosaic DS for VideoSegment DSs; and the MatchingHint DS and the PointOfView DS for the Segment DS. The visual features of VideoSegment DS, StillRegion DS, and MovingRegion DS are defined in the Video XM and WD documents. The audio features of AudioSegment DS are defined in the Audio WD document.

10.2.1 MediaTimeMask DS

The MediaTimeMask DS defines a collection of non-overlapping connected subintervals in time.




<element name="MediaTimeMask" type="mds:MediaTimeMask"/><complexType name="MediaTimeMask">

<element ref="MediaTime" minOccursPar="NumberOfIntervals" maxOccursPar="NumberOfIntervals"/>

<attribute name="NumberOfIntervals" type="positiveInteger" use="required"/></complexType>


Name Definition

MediaTimeMask The MediaTimeMask defines the collection of non-overlapping connected subintervals in time.

NumberOfIntervals Number of subinterval included in the non-connected segment.

Time Start time and duration of each subinterval of a non-connected segment. Subintervals should not overlap, and should be included within the limits of the segment.


10.2.2 Mosaic DS

The Mosaic DS describes a mosaic of a set of video frames. A mosaic of a video shot is constructed by aligning and warping the frames of the shot upon each other in a single reference system giving a panoramic view of the whole shot in one single image. The Mosaic DS consists of a still region plus a set of warping parameters.

<element name="Mosaic" type="mds:Mosaic"/><complexType name="Mosaic">

<element ref="StillRegion" minOccurs="0" maxOccurs="1"/><element name="MosaicWarpingParameters" type="mds:MosaicWarpingParameters"

minOccurs="0" maxOccurs="1"/><attribute name="id" type="ID" use="required"/>

</complexType>



<complexType name="MosaicWarpingParameters"><sequence minOccursPar="NoOfMotionParameterSets"

maxOccursPar="NoOfMotionParameterSets"><element ref="MediaTime"/><element name="MotionParameters" type="float"

minOccursPar="ModelType" maxOccursPar="ModelType"/></sequence><attribute name="ModelType" type="positiveInteger" use="required"/><attribute name="SourceSequenceWidth" type="positiveInteger"

use="required"/><attribute name="SourceSequenceHeight" type="positiveInteger"

use="required"/><attribute name="Xoffset" type="float" use="required"/><attribute name="Yoffset" type="float" use="required"/><attribute name="Xorigin" type="float" use="required"/><attribute name="Yorigin" type="float" use="required"/><attribute name="NoOfMotionParameterSets" type="positiveInteger"

use="required"/></complexType>

Editor’s Note: min/maxOccursPar is not supported by MPEG-7 DDL v 2.0.Editor’s Note: Only a media time point needs to be specified; however, MediaTime is used.Editor’s Note: The Syntax of the Mosaic DS needs to be updated based on related Video Group’s work.


Semantics of Mosaic DS

Name Definition

Mosaic Description of mosaic of a video shot. A mosaic is constructed by aligning and warping the frames of a shot upon each other in a single reference system giving a panoramic view of the whole shot in one single image.

Id Identification of the instance of the Mosaic description.

StillRegion The Mosaic itself is a still image that can be further described by a

Name Definition

StillRegion DS. (see definition of StillRegion DS).

MosaicWarping Parameters Warping parameters and reference information used to construct the mosaic from a video sequence (see definition of MosaicWarpingParameters DS).

Semantics of MosaicWarpingParameters DS

Name Definition

MosaicWarping Parameters Warping parameters and reference information used to construct the mosaic from a video sequence.

ModelType Integer number out of 2, 4, 6, 8, 12, that specifies the number of motion parameters the applied motion model consists of and with this it specifies the motion model.

SourceSequenceWidth X dimension of the constructing video sequence (pixel units).

SourceSequenceHeight Y dimension of the constructing video sequence (pixel units).

Xoffset X (pixel-) coordinate in mosaic, the topleft pixel of the first frame of the constructing video sequence is located.

Yoffset Y (pixel-) coordinate in mosaic, the topleft pixel of the first frame of the constructing video sequence is located.

Xorigin X (pixel-) coordinate of the first frame, the origin of the spatial reference coordinate system is located.

Yorigin Y (pixel-) coordinate of the first frame, the origin of the spatial reference coordinate system is located.

NoOfMotionParameter Sets Total number of motion parameter sets (frames) used to construct the mosaic.

MediaTime Time point (frame) in relation to first frame, a set of motion parameters refers to.

MotionParameters Motion parameter values.

The MosaicWarpingParameters DS describes the connection between the mosaic and the video sequence it is composed of. It contains all information that is necessary to reconstruct an output sequence from the mosaic. ModelType is an integer number out of 2, 4, 6, 8, 12, that specifies the number of parameters the applied motion model consists of and with this it specifies the motion model. These motion models are already included in the Visual XM as part of the ParametricMotion D:

Translational model: vx(x, y) = a1

vy (x, y) = a2

Rotation/Scaling model: vx(x, y) = a1 + a3 x +a4 y


vy (x, y) = a2 - a4 x + a3 y

Affine model: vx(x, y) = a1 + a3 x +a4 y

vy (x, y) = a2 + a5 x +a6 y

Planar perspective model: vx(x, y) = (a1 + a3 x +a4 y) / (1 + a7 x +a8 y)

vy(x, y) = (a2 + a5 x +a6 y) / (1 + a7 x +a8 y)

Quadratic model: vx(x, y) = a1 + a3 x +a4 y +a7 xy + a9 x2 + a10 y2

vy (x, y) = a2 + a5 x +a6 y +a8 xy + a11 x2 + a12 y2

where vx(x, y) and vx(x, y) represent the x and y displacement components of the pixel of coordinates (x, y). The possible deformations of the different model are illustrated in Figure 5. This flexibility allows the construction of many different types of mosaics suitable for different video sequences and applications.

-150

-100

-50

0

50

100

150

-150 -100 -50 0 50 100 150

x

y

a)

-150

-100

-50

0

50

100

150

-150 -100 -50 0 50 100 150

x

y

-150

-100

-50

0

50

100

150

-150 -100 -50 0 50 100 150

x

y

-150

-100

-50

0

50

100

150

-150 -100 -50 0 50 100 150

x

y

b) c) d)

-150

-100

-50

0

50

100

150

-150 -100 -50 0 50 100 150

x

y

-150

-100

-50

0

50

100

150

-150 -100 -50 0 50 100 150

x

y

-150

-100

-50

0

50

100

150

-150 -100 -50 0 50 100 150

x

y

e) f) g)

-150

-100

-50

0

50

100

150

-150 -100 -50 0 50 100 150

x

y

-150

-100

-50

0

50

100

150

-150 -100 -50 0 50 100 150

x

y

h) i)


Figure 5: Illustrations of possible deformations of region a) by variation of single parameter values; for affine, perspective and quadratic model: b) translation a1; c) scale a3; d) shear a4; quadratic model deformations e) a9; f) a10; g) a7; perspective model deformations: h) a7; i) a8.The fixed spatial reference system is defined in terms of the relation between the mosaic and the first frame of the constructing video sequence. All values are specified in 'pixel' units for the time being. As illustrated in Figure 6, Xoffset and Yoffset specify the pixel coordinates of the mosaic, the top left pixel of the first frame is located. The origin of the coordinate system the motion parameters refer to is further shifted as specified by Xorigin and Yorigin. This enables full flexibility for positioning the spatial reference system. Finally, the dimensions of the constructing video sequence are specified by SourceSequenceWidth and SourceSequenceHeight allowing the reconstruction of a corresponding output sequence.

Xoffset

XoriginYorigin

SourceSequenceHeight

SourceSequenceWidth

Yoffset

M osaic

Fram e #0

x

y

Figure 6: Spatial reference system in relation of frame #0 and mosaic.A mosaic captures image information over a period of time. Therefore also temporal reference information has to be provided. The time frame (a shot) of the video sequence the mosaic belongs to is assumed to be provided by a higher order DS. As illustrated in Figure 7, sets (vectors) of motion parameters (MotionParameters) are spread over this period, the total number is specified by the value NoOfMotionParameterSets. Each set of motion parameters carries own temporal information in Time DS, that specifies the frame it is related to. This provides high flexibility for temporal spacing of the frames used for mosaicing. It is possible to use for instance every frame, every fifth or even an irregular spacing.

shot(tim e fram e provided by higher order DS)t0

t1 t2 tN

tend

m otion parametersets

N = no of M P sets

tim e stamps. . . .

Figure 7: Temporal reference system in relation to frame #0.Non-normative issues (creation, use and examples) are described in the MDS XM document.

10.2.3 MatchingHint DS

The MatchingHint DS describes the relative importance of low-level descriptors and/or components of low-level descriptors. This DS is defined in the MDS XM document and has not been promoted to WD.


10.2.4 PointOfView DS

The PointOfView DS describes the relative importance of a segment given a specific viewpoint. The viewpoint is the criteria for the relative importance. This DS is defined in the MDS XM document and has not been promoted to WD.

10.3 Segment-relation graph

10.3.1 SegmentRelationshipGraph DS

The SegmentRelationshipGraph DS can represent a graph of segments and relationships among the segments. Although hierarchical structures such as trees provided by the segment DSs are adequate for efficient access and retrieval, some relationships can not be expressed using such structures. The SegmentRelationGraph DS adds flexibility in describing more general relationships among segments.




<complexType name="SegmentNode"><choice minOccurs="0" maxOccurs="unbounded">

<element name="ReferenceToSegment" type="mds:ReferenceToSegment"/><element name="SegmentNode" type="mds:SegmentNode"/>

</choice><element name="SegmentRelationshipNode" type="mds:SegmentRelationshipNode"

minOccurs="0" maxOccurs="unbounded"/><attribute name="id" type="ID" use="optional"/>

</complexType>

<complexType name="SegmentRelationship" base="mds:Relationship" derivedBy="extension"/>

<complexType name="SegmentRelationshipNode"><element name="SegmentRelationship" type="mds:SegmentRelationship"

minOccurs="1" maxOccurs="1"/><choice minOccurs="0" maxOccurs="unbounded">

<element name="SegmentRelationshipNode" type="mds:SegmentRelationshipNode"/>

<element name="SegmentNode" type="mds:SegmentNode"/></choice>

</complexType>

<element name="SegmentRelationshipGraph" type="mds:SegmentRelationshipGraph"/><complexType name="SegmentRelationshipGraph">

<choice minOccurs="1" maxOccurs="unbounded"><element name="SegmentRelationshipNode"

type="mds:SegmentRelationshipNode"/><element name="SegmentNode" type="mds:SegmentNode"/>

</choice><attribute name="id" type="ID" use="optional"/><attribute name="type" type="string" use="optional"/><attribute name="name" type="string" use="optional"/>

</complexType>


Semantic of the SegmentNode DS.

Name Definition

SegmentNode The segment node includes the set of references to segments that participate in one vertex of a relationship. The number of vertices of a relationship is given by the relationship's degree. In a description, a segment node can also contain an arbitrary number of segment relationship nodes and other segment nodes to allow the definition of efficient and nested graphs of relationships among segments.

Id Identifier of an instance of the SegmentNode DS.

ReferenceToSegment Reference to segments that participate in the relationship.

SegmentNode SegmentNode grouping the set of references to segments (or SegmentNode) that participate in the relationship.

SegmentRelationshipNode Relationship between the segments.

Semantic of the SegmentRelationship DS.

Name Definition

SegmentRelationship Definition of a segment relationship. It is a sub DS of the Relationship DS. Section 10.3.2 defines the list of normative SegmentRelationship.

Semantic of the SegmentRelationshipNode DS.

Name Definition

SegmentRelationshipNode Relationship of any degree (e.g. binary) and cardinality (e.g. 1:N) among the set of segments. In a description, a segment relationship node contains one segment relationship and can also contain an arbitrary number of segment nodes and other segment relationship nodes to allow the definition of efficient and nested graphs of relationships among segments.

SegmentRelationship Definition of a segment relationship. It is a sub DS of the Relationship DS. Section 10.3.2 defines the list of normative SegmentRelationship.

SegmentNode SegmentNode grouping the set of references to segments (or SegmentNode) that are involved in the segment relationship node.

SegmentRelationshipNode Node that groups the segment relationship nodes involved in the segment relationship node.

Semantic of the SegmentRelationshipGraph DS.

Name Definition

SegmentRelationshipGraph Graph of segment relationship nodes and segment nodes. A segment relationship among segment nodes can be specified by instantiating first the SegmentRelationshipNode DS or the SegmentNode DS. If a segment


Name Definition

relationship node contains two segment nodes, the segment relationship applies from the first segment node to the second. If a segment node contains a segment relationship node that contains another segment node, the segment relationship applies for the first segment node to the second segment node. A segment relationship node can include more segment nodes than the relationship's specified degree. In this case, the segment relationship applies to every continuous set of number-degree segment nodes.

Id Identifier of an instance of the SegmentRelationshipGraph DS.

Type String which specifies the type of the segment-relationship graph (e.g. "temporal"). For a possible taxonomy for segment relationship types see section 8.2.2.2.

Name String which specifies the semantics of the segment-relationship graph. Specific types of graphs can be assigned a name.

SegmentRelationshipNode Node defining the set of segment relationships involved in the graph.

SegmentNode Node defining the set of segment involved in the relation graph.

Editor's Note: The possibility of defining segments inside the segment relationship graph could be investigated.

10.3.2 Type of relationships

Some segment relationships are normalized.

10.3.2.1 Normative segment relationships

The table below includes the normalized segment relationships. The first 7 rows correspond to well-known Allen's 13 temporal interval relationships.

Relationships Inverse Relationships

Meaning Type

Before After AAA BBB Temporal, Directional

Meet Does not Meet AAABBB Temporal, Directional

Overlaps Does not Overlap AAA

BBBBB

Temporal, Topological

During Not During AAA

BBBBBBB


Starts Does not Start AAA Temporal, Directional


Relationships Inverse Relationships

Meaning Type

BBBBB

Finishes Does not finish AAA

BBBBB

Temporal, Directional

Equal AAA

BBB


Editor’s Note: Work with the Video Group is needed to normalize spatial (e.g. Left) relationships.



11 Description of the conceptual aspects of the content

The corresponding elements are still under validation and core experiments.

Most elements describing the conceptual aspects of content are still under validation and core experiments. The Affective DS describes audiences’ affective information of segments.

11.1.1 Affective DS

The Affective DS describes the audiences’ affective information of segments by assigning a score to each segment. The resulting representation of score along the story timeline includes the audiences’ mood/emotion change during their content viewing as well as a story shape, i.e. how a story is developing along the story timeline. This DS is defined in the MDS XM document and has not been promoted to WD.


12 Content navigation and access

This section contains the syntax and semantics of description schemes that support the specification of:

audio-visual summaries and abstracts for efficient browsing and navigation of AV content;

views or partitions of the AV data in the space or frequency domain;

the association or relation between different variations of AV material.

12.1 Summarization

This section specifies description schemes that facilitate discovery, browsing, navigation, visualization and sonification of AV content. The Summarization DS enables fast and effective browsing and navigation of AV content by providing immediate access to a set of audio-visual summaries or abstracts. Each audio-visual summary or abstract is used to convey the essential information about the AV content.

Each DS contains links to the AV content and its segments and frames, which can be rendered by a presentation engine to show an audiovisual summary or abstract. The Summarization DS enables the specification of multiple summaries of the same content, at different level of detail, without the need for generating or storing multiple versions of the content.

12.1.1 Summarization DS

The Summarization DS is used to specify a set of summaries to enable rapid browsing, navigation, visualization and sonification of AV content. Each summary is an audio-visual abstract of the content.




<element name="Summarization" type="mds:Summarization"/><complexType name="Summarization">

<element name="Summary" type="mds:Summary" maxOccurs="unbounded"/></complexType>


Name Definition

Summarization Specifies a set of Summary elements. Each Summary is associated with an audio-visual program or program segment.

Summary See section 12.1.2.

12.1.2 Summary DS

The Summary DS is used to specify an audio-visual abstract of AV content. The role of the Summary DS is to convey information about the AV content that is essential for rapid browsing and navigation.




<complexType name="Summary" abstract="true"> <element name="RefLocator" type="mds:MediaLocator"

minOccurs="0"/><element name="ReferenceToSegment" type="mds:ReferenceToSegment"

minOccurs="0"/><element name="ReferenceToProgram" type="mds:ReferenceToProgram"

minOccurs="0"/><attribute name="name" type="ID"/>

</complexType>


Name Definition

Summary An abstract DS from which the following description schemes are derived:

HierarchicalSummary DS

SequentialSummary DS

name Identifies an instantiation of the Summary DS.

RefLocator Specifies location of the original (reference) AV content.

ReferenceToSegment Reference to the description of a segment of the AV content. Shall refer to the id attribute of a Segment.

ReferenceToProgram Reference to the description of an instance of the AV content. Shall refer to the id attribute of a MediaInstance.


12.1.3 HierarchicalSummary DS

The HierarchicalSummary DS is used to specify a group of audio-visual summaries, possibly ordered hierarchically. The audio-visual summaries in this group represent alternative views of the AV content. A basic, yet functional, instance of a HierarchicalSummary DS may contain a single summary without any hierarchy. The HierarchicalSummary DS may organize summaries into a succession of levels, each describing the audio-visual content at a particular level of detail. In general, levels closer to the root of the hierarchy provide coarse summaries and levels further away from the root provide more detailed summaries. Elements in the hierarchy are specified by the HighlightLevel DS. Each element in a hierarchy has one parent, except the root element. Each element in a hierarchy may have zero or more children.

<simpleType name="SummaryType" base="string"><enumeration value="keyFrames"/><enumeration value="keyVideoClips"/><enumeration value="keyAudioClips"/><enumeration value="keyEvents"/><enumeration value="unconstrained"/>

</simpleType>

<simpleType name="SummaryTypeList" base="mds:SummaryType" derivedBy="list"/>

<simpleType name="HierarchyType" base="string"><enumeration value="independent"/><enumeration value="dependent"/>

</simpleType>

<complexType name="SummaryThemeList"><element name="SummaryTheme" maxOccurs="unbounded">

<complexType base="mds:TextualDescription" derivedBy="extension"><attribute name="id" type="ID"/>


</complexType>

<complexType name="HierarchicalSummary" base="mds:Summary" derivedBy="extension"><element name="SummaryThemeList" type="mds:SummaryThemeList" minOccurs="0"/><element name="HighlightLevel" type="mds:HighlightLevel"

maxOccurs="unbounded"/><attribute name="summaryTypeList" type="mds:SummaryTypeList"/><attribute name="hierarchyType" type="mds:HierarchyType" use="optional"/>

</complexType>

Editor's Note: SummaryType and SummaryTypeList can probably be combined into one simpleType.


Name Definition

SummaryTheme Specifies an event or theme in textual form, in terms of which a video can be summarized.

id Identifies an instantiation of a SummaryTheme element.

SummaryThemeList Defines a list of SummaryTheme elements.

Name Definition

HierarchicalSummary Specifies a summary which contains hierarchically ordered audio-visual segments. A HierarchicalSummary is the root element of the hierarchical summary. A HierarchicalSummary element contains HighlightLevel DS

elements.

SummaryType (as datatype) Constrains the values of summaryType to the following types, with associated semantics:

keyFrames – A keyFrames summary contains key-frames, possibly ordered hierarchically. A keyFrames summary may contain multiple summaries, each on a different level of detail. A key-frame can be a specific frame from a video segment, or an image that is not in the video, possibly a synthetic image (pre-composed from multiple images).

keyVideoClips – A keyVideoClips summary contains key-videoclips, possibly ordered hierarchically. Such video clips form a video highlight of the program of a particular duration. A keyVideoClips summary may contain multiple summaries, each with different duration. A key-videoclip can be a video segment from the content, or from related media.

keyAudioClips – A keyAudioClips summary contains key-audioclips, possibly ordered hierarchically. Such audio clips form an audio highlight of the program of a particular duration. A keyAudioClips summary may contain multiple summaries, each with different duration. A key-audioclip can be an audio segment from the content, or from related media.

keyEvents – A keyEvents summary contains videoclips, possibly ordered hierarchically, as well as textual descriptions of associated events or themes. A keyEvents summary is a collection of videoclips referring to particular key-events. A keyEvents summary may contain multiple summaries, each associated with a certain event or theme. Events or themes may be described textually by key-words.

unconstrained – The unconstrained type is reserved for future use.

SummaryTypeList (as datatype)

A datatype for a list of SummaryType elements.

summaryTypeList (in HierarchicalSummary)

Specifies the types of summary elements included in the HierarchicalSummary.

hierarchyType (in HierarchicalSummary)

Specifies the type of the hierarchy with respect to the parent-child relationships between elements at different levels of the hierarchy. This attribute may be used to eliminate unnecessary duplication of information in a hierarchy of elements.

hierarchyType (as datatype) Constrains the values of hierarchyType to the following types, with associated semantics:

independent – The information in an HighlightLevel DS completely specifies a particular summary, without reference to the information in its parent element. Information in the parent element shall not be re-used in the child element.

dependent – The information in an HighlightLevel DS adds to, or refines, the information in its parent element. Information in the parent element shall be re-used in the child element.

HighlightLevel Contains audiovisual segments. See section 12.1.4.



12.1.4 HighlightLevel DS

The HighlightLevel DS is used to specify a summary at a particular level of detail by referring to a sequence of audio-visual segments (such as video clips or audio clips) or images (such as key-frames). A HighlightLevel at a particular level in the hierarchy may correspond to, for example, a highlight with particular time duration or a particular set of events. A HighlightLevel element may contain children HighlightLevel elements to construct summaries with more detail.




<simpleType name="FidelityType" base="float"><minInclusive value="0.0"/><maxInclusive value="1.0"/>

</simpleType>

<complexType name="HighlightLevel"><element name="HighlightSegment" type="mds:HighlightSegment"

maxOccurs="unbounded"/><element name="HighlightLevel" type="mds:HighlightLevel"

minOccurs="0" maxOccurs="unbounded"/><attribute name="name" type="string" use="required"/><attribute name="level" type="integer" use="optional"/>

<attribute name="duration" type="mds:timeMediaDuration" use="optional"/> <attribute name="numberOfKeyFrames" type="nonNegativeInteger" use="optional"/><attribute name="fidelity" type="mds:FidelityType" use="optional"/>

<attribute name="themeIds" type="IDREFS" use="optional"/></complexType>


Name Definition

HighlightLevel Specifies a single summary or part of a summary. Contains a set of audio-visual segments which form a summary. A HighlightLevel element may contain HighlightLevel elements as its children.

name Identifies this element by name.

level Indicates the level of this element in a hierarchy.

duration Indicates the total duration of the HighlightSegments contained in this element.

numberOfKeyFrames Indicates the total number of key-frames contained in the HighlightLevel summary.

fidelity Indicates how well the information in the HighlightLevel element is represented by the information in its parent HighlightLevel element, relative to the other children of the parent HighlightLevel element.

FidelityType Constrains the values of fidelity to be real values ranging (inclusively) between 0.0 and 1.0, where values closer to 1.0 correspond to better representations of this element by the associated

parent element.

themeIds Indicates theme (event) instances common to all children HighlightLevel and HighlightSegment elements. Shall refer to valid id attributeof SummaryTheme elements.

HighlightSegment See section 12.1.5.


12.1.5 HighlightSegment DS

The HighlightSegment DS is used to specify an audio-visual segment of AV content. A HighlightSegment DS may be used to refer to a videoclip, an audioclip, key-frames and key-sounds.




<complexType name="HighlightSegment"><element name="VideoSegmentLocator" type="mds:VideoSegmentLocator"

minOccurs="0"/><element name="AudioSegmentLocator" type="mds:AudioSegmentLocator"

minOccurs="0"/><element name="ImageLocator" type="mds:ImageLocator"

minOccurs="0" maxOccurs="unbounded"/><element name="SoundLocator" type="mds:SoundLocator"

minOccurs="0" maxOccurs="unbounded"/><attribute name="name" type="string" use="optional"/><attribute name="themeIds" type="IDREFS" use="optional"/>

</complexType>


Name Definition

HighlightSegment Specifies an audio-visual segment. May contain a video segment, an audio segment, images or sounds.

name Identifies the segment by name.

themeIds Indicates event instances associated with the HighlightSegment, by referring to valid id attributes of SummaryTheme elements.

VideoSegmentLocator Specifies a video segment. See section 5.4.2.

AudioSegmentLocator Specifies an audio segment. See section 5.4.2.

ImageLocator Specifies a key-frame. See section 5.4.2.

SoundLocator Specifies a key-sound. See section 5.4.2.

12.1.6 SequentialSummary DS

The SequentialSummary DS is used to specify a single audio-visual summary, which may contain a sequence of images or video frames, possibly synchronized with audio, composing a slide-show or audio-visual skim. The summary may also contain a sequence of audio clips. The images or video frames and the audio clips that are part of the summary may be stored separately from the original AV content, to allow fast playback. Video frames may be stored individually or as part of a composite. Alternatively, the SequentialSummary may refer to the original frames in the AV program. Alternatively, the SequentialSummary may refer to the original frames in the AV program. Analogously, audio clips may be store individually, as part of a composite, or refer to the original audio clips in the AV program.




<complexType name="SequentialSummary" base="mds:Summary" derivedBy="extension"><element name="VideoSegmentLocator" type="mds:VideoSegmentLocator"

minOccurs="0"/><element name="AudioSegmentLocator" type="mds:AudioSegmentLocator"

minOccurs="0"/><element name="FrameProperty" type="mds:FrameProperty" minOccurs="0"

maxOccurs="unbounded"/> <element name="SoundProperty" type="mds:SoundProperty" minOccurs="0" maxOccurs="unbounded"/>

<element name="TextProperty" type="mds:TextProperty" minOccurs="0" maxOccurs="unbounded"/>

</complexType>


Name Definition

SequentialSummary Specifies a summary which contains sequentially ordered images or video frames, possibly synchronized with audio. Each image or video frame may be described by certain properties relevant to the summary.

VideoSegmentLocator Specifies a video segment, containing a composite of the images or video frames. See section 5.4.2.

AudioSegmentLocator Specifies an audio segment, containing a composite of audioclips. See section 5.4.2.

FrameProperty Specifies temporal and spatial properties of an individual image or video frame. See section 12.1.7.

SoundProperty Specifies temporal properties of an individual audio clip. See section 12.1.8.

TextProperty Specifies temporal properties of textual information associated with the original AV content and/or the audio-visual summary. See section 12.1.9.

12.1.7 FrameProperty DS

The FrameProperty DS is used to specify certain properties associated with an image or video frame in a slide show or audio-visual skim. These properties may specify: the location of the image or video frame, possibly by referring to the original AV content; the relative amount of scene-activity associated with the video frame; a region-of-interest in the image or video frame.




<simpleType name="FrameActivity" base="float"><minInclusive value="0.0"/><maxInclusive value="1.0"/>

</simpleType>

<complexType name="FrameProperty"><choice>

<element name="RefTime" type="mds:MediaTime" minOccurs="0"/> <element name="RefLocator" type="mds:MediaLocator" minOccurs="0"/>

</choice><element name="SyncTime" type="mds:MediaTime" minOccurs="0"/><element name="ImageLocator" type="mds:ImageLocator" minOccurs="0"/><element name="FrameActivity" type="mds:FrameActivity" minOccurs="0"/>

<element name="Region" minOccurs="0"> <complexType> <attribute name="px" type="nonNegativeInteger"/> <attribute name="py" type="nonNegativeInteger"/> <attribute name="sx" type="nonNegativeInteger"/> <attribute name="sy" type="nonNegativeInteger"/> </complexType> </element>

</complexType>


Name Definition

FrameProperty Specifies temporal and spatial properties associated with an image or video frame.

RefTime Specifies the frame-number or time-stamp of the video frame in the original AV program.

RefLocator Specifies the location of the original AV program, in addition to the frame-number or time-stamp of the video frame in the original AV program. This is used if multiple AV programs are referenced in a summary.

SyncTime Specifies the time-stamp or duration of the video frame in the audio-visual summary. It is assumed that the time stamps in the summary, that is, SyncTime elements, do not overlap in time.

ImageLocator Locates an image or video frame that is part of the video skim. May contain a URL to a separate file; otherwise, locates an image in the composite video of the

parent SequentialSummary. See section 5.4.2.

FrameActivity Specifies the relative amount of scene-activity between the previous video frame and the current frame. It can be used for scalable keyframe extraction, and smart fast forward.

Region Specifies a region-of-interest in a video frame from the original AV program. If an ImageLocator element is used in combination with a Region element, the image referred to by the ImageLocator corresponds to this region-of-interest.

px The x-coordinate of the top-left point of a region, where the origin is at the top-left of the image.

py The y-coordinate of the top-left point of a region, where the origin is at the top-left of the image.

sx The width of a region in pixels.

sy The height of a region in pixels.

Editor’s Note: the Region Descriptor may be unified with / replaced by BoundingBox descriptor.


12.1.8 SoundProperty DS

The SoundProperty DS is used to specify certain properties associated with an audio clip or the components of an audio slide show. This DS is defined in the MDS XM document and has not been promoted to WD.

12.1.9 TextProperty DS

The TextProperty DS is used to specify certain properties of textual information associated with an audio-visual summary (slide show, audio slide show, audio-visual slide show, and audio-visual skim). This DS is defined in the MDS XM document and has not been promoted to WD.

12.2 Partitions and decompositions

This section describes space- and frequency-views, which provide a way to specify partitions of the audio-visual data in the space or frequency domain.

12.2.1 View DS

The View DS is an abstract DS that specifies a view of AV data. The View DS provides a base class for other specific types of views such as Space Views, Frequency Views, Resolution Views, Space Resolution Views, and Space Frequency Views. The different types of Views rely on a Partition DS to specify the parameters of the partition in multi-dimensional space and/or frequency. The view itself can be divided into regions if they belong to different partitions in the multi-dimensional space and/or frequency.

12.2.1.1 View Syntax

<complexType name="View" abstract="true">

<element name="ViewData" type="mds:MediaLocator" minOccurs="0"/><element name="SourceData" type="mds:MediaLocator" minOccurs="0"/>

</complexType>

<simpleType name="PartitionUnits" base="string">

<enumeration value="fraction"/><enumeration value="samples"/>

</simpleType>

<element name="Partition" type="mds:Partition"/><complexType name="Partition">

<element name="Start" type="mds:FloatVector"/><choice>

<element name="End" type="mds:FloatVector"/><element name="Extent" type="mds:FloatVector"/>

</choice><attribute name="dimensions" type="integer"/><attribute name="units" type="mds:PartitionUnits" use="default"

value="fraction"/></complexType>

Editor's Note: The relation between the Partition DS and the GridLayout D of the visual WD has to be studied.

12.2.1.2 View Semantics

Name Definition

View View specifies an abstract view of data

Viewdata Viewdata specifies the MediaLocator of the View data

Sourcedata Sourcedata specifies the MediaLocator of the Source data from which the View data has been derived via space and or frequency partitioning

Name Definition

PartitionUnits PartitionUnits specifies the units used in describing the partition: values include “fraction" and “samples"

Fraction “fraction" indicates that the partition is specified by values between 0.0 and 1.0 that describe the fraction along each dimension of the source data.

Samples “samples" indicates that the partition is specified by values that describe the number of samples along each dimension of the source data.

Name Definition

Partition Partition specifies a multi-dimensional rectangle

Name Definition

dimensions Dimensions specifies the number of dimensions of the rectangle

units

Start Start specifies the vector or multi-dimensional start point of the partition rectangle. The elements of the vector are given in PartitionUnits along each dimension of the source data that the start position corresponds to.

End End specifies the vector or multi-dimensional end point of the partition rectangle. The elements of the vector are given in PartitionUnits along each dimension of the source data that the end position corresponds to.

Extent Extent specifies the vector or multi-dimensional extent of the partition rectangle. The elements of the vector are given in PartitionUnits that specify the coverage of the dimensions of the source data.

Editor's note: definition of semantics of 'units' is missing.

12.2.2 Space View DS

A Space View specifies a multi-dimensional spatial view of data, which corresponds to a partition in multi-dimensional space.

12.2.2.1 SpaceView Syntax



<element name="SpaceView" type="mds:SpaceView"/><complexType name="SpaceView" base="mds:View" derivedBy="extension">

<choice minOccurs="0"><element name="SpacePartition" type="mds:Partition"/><element name="SpaceRegion" type="mds:RegionLocator"/>


12.2.2.2 SpaceView Semantics

Name Definition

SpaceView SpaceView specifies a multi-dimensional spatial view of data.

SpacePartition SpacePartition specifies the multi-dimensional rectangle that defines the partition in multi-dimensional space.

SpaceRegion SpaceRegion specifies a multi-dimensional Region of the view as a partition in the multidimensional space.

Editor’s Note: the RegionLocator DS will be specified in the visual XM.

12.2.3 Frequency View DS

A FrequencyView specifies a multi-dimensional view of data in frequency, which corresponds to a partition in the multi-dimensional frequency plane.

12.2.3.1 FrequencyView Syntax

<element name="FrequencyView" type="mds:FrequencyView"/><complexType name="FrequencyView" base="mds:View" derivedBy="extension">

<element name="FrequencyPartition" type="mds:Partition" minOccurs="0"/><element name="Filter" type="mds:Filter" minOccurs="0"/>

</complexType>

12.2.3.2 FrequencyView Semantics

Name Definition

FrequencyView FrequencyView specifies a multi-dimensional view of data in the frequency plane

FrequencyPartition FrequencyPartition specifies the multi-dimensional rectangle that defines the partition in the multi-dimensional frequency plane

Filter Filter specifies the filter that characterizes the partition in the multi-dimensional frequency plane in terms of passband and stopband characteristics

12.2.4 SpaceFrequencyView DS

A Space Frequency View specifies a multi-dimensional view of data simultaneously in space and frequency, which corresponds to partitions in the multi-dimensional space and frequency planes.

12.2.4.1 SpaceFrequencyView Syntax



<element name="SpaceFrequencyView" type="mds:SpaceFrequencyView"/><complexType name="SpaceFrequencyView" base="mds:View" derivedBy="extension">

<element name="FrequencyPartition" type="mds:Partition" minOccurs="0"/><element name="Filter" type="mds:Filter" minOccurs="0"/><choice minOccurs="0">

<element name="SpacePartition" type="mds:Partition"/><element name="SpaceRegion" type="mds:RegionLocator"/>

12.2.4.2 SpaceFrequencyView Semantics

Name Definition

SpaceFrequencyView SpaceFrequencyView specifies a multi-dimensional view of data in the space and frequency planes

SpacePartition SpacePartition specifies the multi-dimensional rectangle that defines the partition in multi-dimensional space


SpaceRegion SpaceRegion specifies a multi-dimensional Region of the view as a partition in the multidimensional space

Filter Filter specifies the filter that characterizes the partition in the multi-dimensional frequency plane in terms of passband and stopband characteristics

12.2.5 Resolution View DS

A ResolutionView specifies a multi-dimensional view of data, which corresponds to a low-frequency region from the frequency plane.

12.2.5.1 ResolutionView Syntax

<element name="ResolutionView" type="mds:ResolutionView"/><complexType name="ResolutionView" base="mds:View" derivedBy="extension">

<element name="ResolutionOfView" type="mds:FloatVector" minOccurs="0"/><element name="FrequencyPartition" type="mds:Partition" minOccurs="0"/><element name="Filter" type="mds:Filter" minOccurs="0"/>

</complexType>

12.2.5.2 ResolutionView Semantics

Name Definition

ResolutionView ResolutionView specifies a multi-dimensional low-resolution or low-frequency view of data

Resolution Resolution specifies the vector that defines the resolution along each dimension of the data, where each element of the vector is a real number between 0.0 and 1.0. A value of 1.0 specifies full resolution. A value of 0.5 specifies one-half resolution.


Filter Filter specifies the low-pass filter that characterizes the partition in the multi-dimensional frequency plane in terms of passband and stopband characteristics

12.2.6 SpaceResolutionView DS

A SpaceResolution View specifies a multi-dimensional view of data, which corresponds to a low-frequency region from the frequency plane.

12.2.6.1 SpaceResolutionView Syntax



<element name="SpaceResolutionView" type="mds:SpaceResolutionView"/><complexType name="SpaceResolutionView" base="mds:View" derivedBy="extension">

<choice minOccurs="0"><element name="SpacePartition" type="mds:Partition"/><element name="SpaceRegion" type="mds:RegionLocator"/>

</choice><element name="SpaceResolutionOfView" type="mds:FloatVector" minOccurs="0"/><element name="FrequencyPartition" type="mds:Partition" minOccurs="0"/><element name="Filter" type="mds:Filter" minOccurs="0"/>

</complexType>

12.2.6.2 SpaceResolutionView Semantics

Name Definition

SpaceResolutionView Specifies a spatial view of data

SpacePartition SpacePartition specifies the multi-dimensional rectangle that defines the partition in multi-dimensional space


SpaceRegion SpaceRegion specifies a multi-dimensional Region of the view as a partition in the multidimensional space

Resolution Resolution specifies the vector that defines the resolution along each dimension of the data, where each element of the vector is a real number between 0.0 and 1.0. A value of 1.0 specifies full resolution. A value of 0.5 specifies one-half resolution.

Filter Filter specifies the low-pass filter that characterizes the partition in the multi-dimensional frequency plane in terms of passband and stopband characteristics

12.2.7 Filter DS

The Filter DS specifies a multi-dimensional filter to characterize the loss in information of a signal to which this filter is applied to.

12.2.7.1 Filter Syntax

<element name="Filter" type="mds:Filter"/><complexType name="Filter">

<choice><element ref="1DFilter" maxOccurs="unbounded"/><element ref="2DFilter"/>

</choice><attribute name="dimensions" type="integer"/>

</complexType>

12.2.7.2 Filter Semantics

Name Definition

Filter Specifies a spatial view of data

dimensions Dimensions specifies the number of dimensions the filter is applied to

1Dfilter The separable higher dimensional filter is specified by using 1D-Filter. The filter applies to the dimensions in the order they are specified.

2Dfilter 2DFilter is a specification of a non-separable 2D Filter

12.2.8 1D/2D-Filter DS

The 1D/2D Filter DS specifies a one-dimensional or two-dimensional filter (which has to be non-separable) to characterize the loss in information of a signal to which this filter is applied.

12.2.8.1 1D/2D-Filter Syntax

<element name="1DFilter" type="mds:1DFilter"/><complexType name="1DFilter" base="mds:FloatVector" derivedBy="extension">

<attribute name="leadin" type="integer" use="optional"/><attribute name="padType" type="mds:PadType" use="default"

value="data/const"/><attribute name="dimension" type="integer" use="optional"/>

</complexType>

<element name="2DFilter" type="mds:2DFilter"/><complexType name="2DFilter" base="mds:FloatMatrix" derivedBy="extension">

<attribute name="leadin1" type="integer" use="optional"/><attribute name="leadin2" type="integer" use="optional"/><attribute name="padType1" type="mds:PadType" use="default"

value="data/const"/><attribute name="padType2" type="mds:PadType" use="default"

value="data/const"/></complexType>

<simpleType name="PadType" base="string">

<enumeration value="zero"/><enumeration value="const"/>

<enumeration value="cyclic"/><enumeration value="mirror"/><enumeration value="anti-symmetric"/><enumeration value="data/zero"/><enumeration value="data/const"/><enumeration value="data/cyclic"/><enumeration value="data/mirror"/><enumeration value="data/anti-symmetric"/>

</simpleType>

12.2.8.2 1D/2D-Filter Semantics

Name Definition

1Dfilter Specifies the coefficients of a 1D filter

leadin Number of coefficients composing the ‘lead in’ of the filter

padType Specifies the type of padding the filtered data

dimension The dimension of the data the filter is applied to

Name Definition

2Dfilter Specifies the coefficients of a 2D filter

leadin1/leadin2 Number of coefficients in the ‘lead in’ of the filter for both dimensions

padType1/padType2 Specifies the type of padding of the filtered data for both dimensions

Name Definition

PadType This datatype specifies how the signal is extended for the filtering: with zeros, by continuing with the last value, by cyclic repeating the signal, by the mirror signal, by anti-symmetric extension or the already mentioned methods only if a source signal is not available

zero Padding with zeros

const Padding with constant value

cyclic Padding using periodic extension

mirror Padding using mirror extension

anti-symmetric Padding using anti-symmetric mirror extension

data/zero Padding with zeros

data/const Padding with constant value

data/cyclic Padding using periodic extension


Name Definition

data/mirror Padding using mirror extension

data/anti-symmetric Padding using anti-symmetric mirror extension

12.2.9 ViewSet DS

A View Set specifies a set of views. The View Set is complete in the case that it completely covers the space and frequency planes and is incomplete when it does not. The View Set is nonredundant in the case that the views do not overlap in the space and frequency planes and redundant when the views do overlap.

12.2.9.1 ViewSet Syntax



<element name="ViewSet" type="mds:ViewSet"/><complexType name="ViewSet">

<element name="ViewElement" type="mds:View" maxOccurs="unbounded"/><attribute name="complete" type="boolean"/><attribute name="nonredundant" type="boolean"/>

</complexType>

12.2.9.2 ViewSet Semantics

Name Definition

ViewSet ViewSet specifies a set of views

complete Complete specifies that the set of views completely covers the space and frequency planes.

nonredundant Nonredunant specifies that the set of views nonredundantly covers the space and frequency planes.

ViewElement Element specifies the view or multiple views in the set

12.2.10 SpaceTree DS

Space Tree specifies a tree of Space Views.

12.2.10.1 SpaceTree Syntax

<element name="SpaceTree" type="mds:SpaceTree"/><complexType name="SpaceTree">

<element name="Index" type="mds:IntegerVector" minOccurs="0"/><element name="ViewElement" type="mds:SpaceView" minOccurs="0"/><element name="Child" type="mds:SpaceTree"

minOccurs="0" maxOccurs="unbounded"/><attribute name="branching" type="integer" use="optional"/>

</complexType>

12.2.10.2 SpaceTree Semantics

Name Definition

SpaceTree SpaceTree specifies a tree of views

branching Branching specifies the branching factor of each parent node of the SpaceTree into a number children. Branching factor specifies the number of children.

Index Index specifies the index value of the node in the SpaceTree that gives as elements the path from the root node to the node

ViewElement ViewElement specifies the SpaceView at the node

Child Child specifies the SpaceTrees that are descendents of the node

12.2.11 FrequencyTree DS

View is an abstract DS that specifies a view of data.

Editor's Note: Definition should be updated/corrected.

12.2.11.1 FrequencyTree Syntax

<element name="FrequencyTree" type="mds:FrequencyTree"/><complexType name="FrequencyTree">

<element name="Index" type="mds:IntegerVector" minOccurs="0"/><element name="ViewElement" type="mds:FrequencyView" minOccurs="0"/><element name="Child" type="mds:FrequencyTree" minOccurs="0"

maxOccurs="unbounded"/><attribute name="branching" type="integer" use="optional"/>

</complexType>

12.2.11.2 FrequencyTree Semantics

Name Definition

FrequencyTree FrequencyTree specifies a tree of views

Name Definition

branching Branching specifies the branching factor of each parent node of the FrequencyTree into a number children. Branching factor specifies the number of children.

Index Index specifies the index value of the node in the FrequencyTree that gives as elements the path from the root node to the node

ViewElement ViewElement specifies the FrequencyView at the node

Child Child specifies the FrequencyTree that are descendents of the node

12.2.12 SpaceFrequencyGraph DS

A Space Frequency Graph specifies a decomposition in space and frequency.

12.2.12.1 SpaceFrequencyGraph Syntax

<element name="SpaceFrequencyGraph" type="mds:SpaceFrequencyGraph"/><complexType name="SpaceFrequencyGraph">

<element name="Index" type="mds:IntegerVector" minOccurs="0"/><element name="ViewElement" type="mds:SpaceFrequencyView" minOccurs="0"/><element name="SpaceChild" type="mds:SpaceFrequencyGraph"

minOccurs="0" maxOccurs="unbounded"/><element name="FrequencyChild" type="mds:SpaceFrequencyGraph"

minOccurs="0" maxOccurs="unbounded"/><attribute name="branching" type="integer"/>

</complexType>

12.2.12.2 SpaceFrequencyGraph Semantics

Name Definition

SpaceFrequencyGraph SpaceFrequencyGraph specifies a tree of views

branching Branching specifies the branching factor of each parent node of the SpaceFrequencyGraph into a number children. Branching factor specifies the number of children along each dimension.

Index Index specifies the index value of the node in the SpaceFrequencyGraph that gives as elements the path from the root node to the node

ViewElement ViewElement specifies the SpaceFrequencyGraph at the node

SpaceChild SpaceChild specifies the SpaceFrequencyGraphs along the spatial dimension that

Name Definition

are descendents of the node

FrequencyChild FrequencyChild specifies the SpaceFrequencyGraphs along the frequency dimension that are descendents of the node

12.2.13 VideoViewGraph DS

A Video View Graph specifies a decomposition of video in spatial- and temporal-frequency.

12.2.13.1 VideoViewGraph Syntax



<element name="VideoViewGraph" type="mds:VideoViewGraph"/><complexType name="VideoViewGraph">

<element name="Index" type="mds:IntegerVector" minOccurs="0"/><element name="ViewElement" type="mds:FrequencyView" minOccurs="0"/><element name="SpaceChild" type="mds:VideoViewGraph"

minOccurs="0" maxOccurs="unbounded"/><element name="Timechild" type="mds:VideoViewGraph"

minOccurs="0" maxOccurs="unbounded"/><attribute name="branching" type="integer"/>

</complexType>

12.2.13.2 VideoViewGraph Semantics

Name Definition

VideoViewGraph VideoViewGraph specifies a graph of views

branching Branching specifies the branching factor of each parent node of the VideoViewGraph into a number children. Branching factor specifies the number of children along each dimension.

Index Index specifies the index value of the node in the VideoViewGraph that gives as elements the path from the root node to the node

ViewElement ViewElement specifies the VideoViewGraph at the node

SpaceChild SpaceChild specifies the VideoViewGraphs along the spatial dimension that are descendents of the node

TimeChild TimeChild specifies the VideoViewGraphs along the frequency dimension that are descendents of the node

12.2.14 MultiResolutionPyramid DS

Multiresolution Pyramid specifies a hierarchy of views of data.

12.2.14.1 MultiResolutionPyramid Syntax

<element name="MultiResolutionPyramid" type="mds:MultiResolutionPyramid"/><complexType name="MultiResolutionPyramid">

<element name="ViewElement" type="mds:ResolutionView" minOccurs="0"/><element name="Child" type="mds:MultiResolutionPyramid"

minOccurs="0" maxOccurs="unbounded"/><attribute name="Level" type="integer"/>

</complexType>

12.2.14.2 MultiResolutionPyramid Semantics

Name Definition

MultiResolutionPyramid Multiresolution pyramid specifies a hierarchy of views

Level Level specifies the current level in the hierarchy

ViewElement ViewElement specifies the ResolutionView

Child Child specifies the MultiresolutionPyramid child

12.3 Description of variation of the content

12.3.1 Variation DS

The Variations DS is used to specify variations of audio-visual data. The variations may be, in general, generated in a number of different ways, or reflect revisions of the original data. The quality of the variation compared to the original is given by a variation fidelity value. The type of variation is indicated by a variation type attribute. The different types of variations are described as follows:

Translation – translation involves the conversion from one modality (image, video, text, audio, synthetic model) to another. Examples of translation include text-to-speech (TTS) conversion, speech-to-text (speech recognition), video-to-image (video mosaicing), image-to-text (embedded caption recognition), and 3-D model rendering.

Summary – summarization involves the reduction of information detail. Examples of summaries include those defined in the Summary DS.

Scaling – scaling involves operations of data transcoding, manipulation and compression that result in reduction in size, quality and data rate. Examples of scaling include image, video, and audio transcoding, image size reduction, video frame dropping, color conversion and DCT coefficient scaling.

Extract – extraction involves the extraction of information from the input program. Examples of extraction include key-frame extraction from video, audio-band and voice extraction from audio, paragraph and key-term extraction from text, region, segment, object, and event extraction from audio and video.

Abstract – abstract refers to an overview of the input program in which the salient points are presented.

Substitute – substitution indicates that one program can be used to substitute for another. Examples of substitution include a text passage that replaces a photographic image when a photographic image cannot be handled by a terminal device, or an audio track that replaces a chart in a presentation.

Revision – revision indicates that the audio-visual program has been revised in some way, such as through editing or post-processing, to produce the variation.




<simpleType name="VariationType" base="string"><enumeration value="Translation"/><enumeration value="Summary"/><enumeration value="Extract"/><enumeration value="Abstract"/><enumeration value="Scaling"/><enumeration value="Substitute"/><enumeration value="Revision"/>

</simpleType>

<simpleType name="VariationFidelity" base="float"><minInclusive value="0.0"/><maxInclusive value="1.0"/>

</simpleType>

<complexType name="Variation"> <element name="SourceMedia" type="mds:MediaLocator" minOccurs="0" />

<element name="VariationMedia" type="mds:MediaLocator" minOccurs="0" /> <attribute name="Type" type="mds:VariationType" use="optional"/> <attribute name="VariationFidelity" type="mds:VariationFidelity" use="optional" />

</complexType>

<element name="Variations" type="mds:Variations" /><complexType name="Variations">

<element name="Source" type="mds:MediaLocator" minOccurs="0" /> <element name="Variation" type="mds:Variation" maxOccurs="unbounded" />

</complexType>


Name Definition

VariationType Specifies the different types of variations of the audio-visual data.

VariationFidelity Specifies the value of the fidelity of the variation of the audio-visual data with respect to the original data.

SourceMedia Specifies the media location of the source data – original source for the variation

VariationMedia Specifies the media location of the variation data.

Variation Specifies a single variation of audio-visual data in terms of the media locator for the source media and variation media, a variation type and fidelity value.

Variations Specifies a set of variations of a source media.

13 Organization of the content

13.1 Collections and classification schemes

13.1.1 Collection Structure DS

The Collection Structure DS provides the tools to describe collections of multimedia documents and collections of elements within the description of a single multimedia item (e.g. collection of segments). The Collection Structure DS can group segments, events, and/or objects from multiple descriptions in collection clusters, specify common properties to all the elements in a collection cluster, and describe statistics of attribute values of the elements in a collection cluster (e.g. color histogram for a set of segments). Relationships among collection clusters can also be represented.

13.1.1.1 Collection Structure Syntax

<element name="CollectionStructure" type="mds:CollectionStructure"/><complexType name="CollectionStructure">

<element name="CollectionCluster" type="mds:CollectionCluster" minOccurs="0" maxOccurs="unbounded"/>

<element name="ClusterRelationshipNode" type="mds:ClusterRelationshipNode" minOccurs="0" maxOccurs="unbounded"/>

<attribute name="id" type="ID" use="required"/></complexType>

<complexType name="CollectionCluster">

<element ref="ProbabilityModelClass" minOccurs="0" maxOccurs="unbounded"/><element name="StructuredAnnotation" type="mds:StructuredAnnotation"

minOccurs="0" maxOccurs="unbounded"/><element name="ClusterAttribute" type="mds:ClusterAttribute"

minOccurs="0" maxOccurs="unbounded"/><element name="ClusterRelationshipNode" type="mds:ClusterRelationshipNode"

minOccurs="0" maxOccurs="unbounded"/><attribute name="id" type="ID" use="required"/>

</complexType>

<complexType name="ClusterAttribute" abstract="true">

<attribute name="type" type="string" use="optional"/></complexType><complexType name="StructureClusterAttribute" base="mds:ClusterAttribute"

derivedBy="extension" abstract="true"/> <complexType name="ClusterStatistics" base="mds:StructureClusterAttribute" derivedBy="extension">

<element name="Mean" type="mds:FloatVector" minOccurs="0" maxOccurs="1"/><element name="Max" type="mds:FloatVector" minOccurs="0" maxOccurs="1"/><element name="Min" type="mds:FloatVector" minOccurs="0" maxOccurs="1"/>

<element name="Median" type="mds:FloatVector" minOccurs="0" maxOccurs="1"/><element name="Variance" type="mds:FloatVector" minOccurs="0" maxOccurs="1"/><attribute name="DescriptorName" type="string" use="required"/>

</complexType>

<complexType name="NumberElements" type="integer" base="mds:StructureClusterAttribute" derivedBy="extension"/> 

<complexType name="NumberElements" base="mds:StructureClusterAttribute" derivedBy="extension"> <attribute name="value" type="positiveInteger" use="required"/>

</complextype>

<complexType name="MediaClusterAttribute" base="mds:ClusterAttribute" derivedBy="extension" abstract="true"/>

<complexType name="MetaClusterAttribute" base="mds:ClusterAttribute" derivedBy="extension" abstract="true"/>

<complexType name="ClusterCreation" base="mds:MetaClusterAttribute" derivedBy="extension">

<element name="Method" type="mds:Method" minOccurs="0" maxOccurs="1"/><element name="CreationDate" type="mds:TimePoint" minOccurs="0"

maxOccurs="unbounded"/><element name="Creator" type="mds:Creator" minOccurs="0"

maxOccurs="unbounded"/></complexType>

<complexType name="Method" base="mds:MetaClusterAttribute" derivedBy="extension"><element name="Classifier" type="mds:Classifier" minOccurs="0" maxOccurs="1"/><attribute name="mode" use="required">

<simpleType base="string"><enumeration value="Automatic"/><enumeration value="Manual"/>

</simpleType></attribute>

</complexType>

<complexType name="RepresentativeIcons" base="mds:MetaClusterAttribute" derivedBy="extension">

<element ref="MediaLocator" minOccurs="1" maxOccurs="unbounded"/></complexType>

<complexType name="ClusterNode">

<choice minOccurs="0" maxOccurs="unbounded"><element ref="ReferenceToSegment"/><element ref="ReferenceToCollectionCluster"/><element ref="ReferenceToObject"/><element ref="ReferenceToEvent"/><element ref="ReferenceToGenericAV"/><element ref="GenericAV"/><element name="CollectionCluster" type="mds:CollectionCluster"/><element name="ClusterNode" type="mds:ClusterNode"/>

</choice><element name="ClusterRelationshipNode" type="mds:ClusterRelationshipNode"

minOccurs="0" maxOccurs="unbounded"/><attribute name="id" type="ID" use="required"/>

</complexType>

<complexType name="ClusterRelationshipNode">

<element name="ClusterRelationship" type="mds:ClusterRelationship" minOccurs="1" maxOccurs="1"/>

<choice minOccurs="1" maxOccurs="unbounded"><element name="CollectionCluster"

type="mds:CollectionCluster"/><element name="ClusterNode" type="mds:ClusterNode"/><element name="ClusterRelationshipNode"

type="mds:ClusterRelationshipNode"/></choice>

</complexType>

<complexType name="ClusterRelationship" base="mds:Relationship"

derivedBy="extension"/><complexType name="StructureClusterRelationship" base="mds:ClusterRelationship" derivedBy="extension"/><complexType name="ClusterDecomposition" base="mds:StructureClusterRelationship"

derivedBy="extension"><attribute name="type" use="fixed" value="Structure Topological"/><attribute name="name" use="fixed" value="ClusterDecomposition"/><attribute name="degree" use="fixed" value="2"/><attribute name="DecompositionType" type="string" use="optional"/><attribute name="overlaps" type="boolean" use="optional"/><attribute name="gaps" type="boolean" use="optional"/>

</complexType>

<complexType name="ClusterUnion" base="mds:StructureClusterRelationship" derivedBy="extension">

<attribute name="type" use="fixed" value="Structure Topological"/><attribute name="name" use="fixed" value="ClusterUnion"/><attribute name="degree" use="fixed" value="2"/>

</complexType>

<complexType name="ClusterIntersection" base="mds:StructureClusterRelationship" derivedBy="extension">

<attribute name="type" use="fixed" value="Structure Topological"/><attribute name="name" use="fixed" value="ClusterIntersection"/><attribute name="degree" use="fixed" value="2"/>

</complexType>

<complexType name="SemanticClusterRelationship" base="mds:ClusterRelationship" derivedBy="extension"/>

<complexType name="MediaClusterRelationship" base="mds:ClusterRelationship" derivedBy="extension"/>

<complexType name="MetaClusterRelationship" base="mds:ClusterRelationship" derivedBy="extension"/>

13.1.1.2 Collection Structure Semantics

CollectionStructure DS:

Name Definition

CollectionStructure Graph of collection clusters, cluster nodes, and cluster relationship nodes.

The rules for using the Cluster Node DS and Cluster Relationship Node DS are equivalent to those specified for the Entity Node DS and Relationship Node DS in the Entity Relationship Graph DS

Id Identifier of an instance of the Collection Structure DS.

CollectionCluster Structure that represents a group of elements in a multimedia collection. These elements can be video programs, video segments, moving regions, and images, among others.

ClusterRelationshipNode Relationship that applies among two or more cluster nodes, collection clusters, and other description elements (e.g. segment and objects).

Cluster DS:

Name Definition

CollectionCluster Structure that represents a group of elements in a multimedia collection. These elements can be video programs, video segments, moving regions, and images, among others. For example, a collection cluster could be a set of news video programs broadcasted by a TV station.

A collection cluster can be decomposed into other collection clusters. This and relationships among cluster can be described through the Cluster Relationship Node DS.

Id Identifier of an instance of the Collection Cluster DS.

ProbabilityModelClass Specifies a class of descriptors in terms of a probability model for the elements in a collection cluster.

Annotation Textual annotation and description of people, animals, objects, actions, places, time, and/or purpose which are instantiated in the collection cluster.

ClusterAttribute Attributes related to the cluster. Different types of cluster attributes are defined: structure, semantic, media, and meta.

ClusterRelationshipNode Relationship that applies between the root collection cluster to other cluster nodes, collection clusters, and other description elements (e.g. segment and objects).

ClusterAttribute DS:


Name Definition

ClusterAttribute Attributes related to the cluster. Different types of cluster attributes are defined: structure, semantic, media, and meta.

Type String that specifies the type of information described by the cluster attributes. Possible values of this attribute are structure, semantic, media, and meta. Other classification schemes are also possible.

StructureclusterAttribute:

Name Definition

StructureclusterAttribute Attribute that describes statistical and other structural information of a collection cluster.

ClusterStatistics Structure cluster attribute that describes statistical information about a collection cluster in terms of descriptor values for the elements in the collection cluster.

For example, if a collection cluster were composed of a set of images, cluster statistics could be used to describe statistics of the images in the cluster based on color histogram.

DescriptorName String that specifies the name of the descriptor for which the statistics are provided in the Cluster Statistics DS.

Mean Mean value of the descriptor specified by Descriptor Name for the elements of the collection cluster. The current type of the Mead DS is a vector of reals.

Max Maximum value of the descriptor specified by Descriptor Name for the elements of the collection cluster. The current type of the Mead DS is a vector of reals.

Min Minimum value of the descriptor specified by Descriptor Name for the elements of the collection cluster. The current type of the Mead DS is a vector of reals.

Median Median value of the descriptor specified by Descriptor Name for the elements of the collection cluster. The current type of the Mead DS is a vector of reals.

Variance Variance value of the descriptor specified by Descriptor Name for the elements of the collection cluster. The current type of the Mead DS is a vector of reals.

NumberElements Structure cluster attribute that specifies the number of elements in the cluster. Their value is an integer.

Editor’s Note: ClusterStatistics and NumberElements could be defined in the Model DS.

MetaClusterAttribute:


Name Definition

MetaClusterAttribute Author generated information for a collection cluster.

ClusterCreation Meta cluster attribute that describes different aspects of the creation of the collection cluster.

Method Information about the method used to create the cluster. Among others, it specifies if the collection cluster was generated automatically or manually.

Mode Mode of generation of a collection cluster. This attribute is of Mode Type type.

ModeType Datatype that specifies the different values of the creation mode of a collection cluster. Its possible values are “Manual" and “Automatic".

Classifier Classifier used to generate the cluster (see Model DS section 13.5.1).

RepresentativeIcons Meta cluster attribute that includes media locator references to representative icons of a collection cluster.

MediaLocator Locator of referenced material.

Name Definition

MediaClusterAttribute Media related information of a collection cluster.

ClusterNode DS:

Name Definition

ClusterNode The cluster node includes the set of references to collection clusters, cluster nodes, and other description elements that participate in one vertex of a cluster relationship.

In a description, a cluster node can also contain an arbitrary number of cluster relationship nodes, collection clusters, and other cluster nodes to allow the definition of efficient and nested graphs of relationships among cluster and other elements.

Id Identifier of an instantiation of the Cluster Node DS.

ReferenceToSegment Reference to a segment element that participates in a cluster relationship.

ReferenceToObject Reference to an object element that participates in a cluster relationship. This element is TBD in the MPEG-7 MDS XM and WD.

ReferenceToEvent Reference to an event element that participates in a cluster relationship. This element is TBD in the MPEG-7 MDS XM and WD.

ReferenceToGenericAV Reference to a GenericAV element that participates in a cluster relationship. This element is TBD in the MPEG-7 MDS XM and WD.

GenericAV Generic AV element that participates in a cluster relationship. This element is


Name Definition

TBD in the MPEG-7 MDS XM and WD.

ReferenceToCollectionCluster

Reference to a collection cluster element that participates in a cluster relationship.

CollectionCluster Collection cluster element that participates in a cluster relationship.

ClusterNode Cluster node that participates in a cluster relationship.

ClusterRelationshipNode Specification of a cluster relationship.

Editor’s Note: The Generic AV element is TBD in the MPEG-7 MDS XM and WD. It represents the description of a multimedia item (e.g. an image or a video clip). The definition of ReferenceToProgram D is different in the MDS XM/WD. Only if ReferenceToProgram D is a reference to the description of a program, ReferenceToGenericAV and ReferenceToProgram are the same.

ClusterRelationshipNode DS:

Name Definition

ClusterRelationshipNode Specification of a relationship of any degree (e.g. binary) and cardinality (e.g. 1:N) that applies between a set of cluster nodes, clusters, segments, objects, events, and Generic AVs (maybe through references).

In a description, as for the Entity Relationship Node DS, a cluster relationship node can also contain an arbitrary number of cluster nodes and other cluster relationship nodes to allow the definition of efficient and nested graphs of cluster relationships among cluster and other elements.

ClusterRelationship Definition of the type of cluster relationship.

Semantics of the ClusterRelationship DS and refined DSs.

Name Definition

ClusterRelationship Defines a relationship among clusters and other elements. It is derived from the Relationship DS in the MPEG-7 MDS XM.

StructureClusterRelationship

Cluster relationship that specifies statistical or other structural associations among cluster and other elements.

ClusterDecomposition Structure cluster relationship that specifies the decomposition of a collection clusters among other cluster or elements (e.g. segments). For this relationship type, type is fixed to “Structure Topological"; name to “ClusterDecomposition", and degree to “2". DecompositionType, overlap, and gap describe the cluster decomposition.

DecompositionType String that describes the criteria for decomposition of a collection cluster. Possible values are “Semantic".


Name Definition

Overlaps Boolean that specifies if there are overlaps in the decomposition of a collection cluster.

Gaps Boolean that specifies if there are gaps in the decomposition of a collection cluster.

ClusterUnion Structure cluster relationship that describes the union of two or more collection clusters. For this relationship type, type is fixed to “Structure Topological"; name to “ClusterUnion", and degree to “2".

ClusterIntersection Structure cluster relationship that describes the intersection of two or more collection clusters. For this relationship type, type is fixed to “Structure Topological"; name to “ClusterIntersection", and degree to “2".

SemanticClusterRelationship

Structure cluster relationship that specifies the decomposition of a collection clusters among other cluster or elements (e.g. segments). For this relationship type, type is fixed to “Structure Topological"; name to “ClusterDecomposition", and degree to “2". DecompositionType, overlap, and gap describe the cluster decomposition.

MediaClusterRelationship Structure relationship that describes a media relationship among clusters and other elements.

MetaClusterRelationship Structure relationship that describes a meta relationship among clusters and other elements.


13.2 Models

13.2.1 Model DS

The Model DS is used for analysis and classification of audio-visual data. It is an abstract DS.




<complexType name="Model" abstract="true"/>


Name Definition

Model Abstract DS from which the ProbabilityModel and the AnalyticalModel DSs are derived.

13.3 Probability Models

13.3.1 ProbabilityModel DS

The Probability Model DS is used to specify statistical functions and probabilistic structures. The Probability Model DS can be used for representing samples of audio-visual data and classes of descriptors using statistical approximation.


<complexType name="ProbabilityModel" base="mds:Model" derivedBy="extension"

abstract="true"/>

<element name="DiscreteModel" type="mds:DiscreteModel"/><complexType name="DiscreteModel" base="mds:ProbabilityModel"

derivedBy="extension"><element name="vector" type="mds:FloatVector"/>

</complexType>

<element name="MixtureModel" type="mds:MixtureModel"/><complexType name="MixtureModel" base="mds:ProbabilityModel"

derivedBy="extension"><element name="mixtures" type="mds:FloatVector"/><sequence minOccurs="1" maxOccurs="unbounded">

<element name="Gaussian" type="mds:Gaussian"/></sequence>

</complexType>

<element name="State" type="mds:State"/><complexType name="State" base="mds:AnalyticModel"

derivedBy="extension"><element name="ProbabilityModel"/>

</complexType>

<complexType name="StateTransitionModel" base="mds:ProbabilityModel"

derivedBy="extension"><element name="initial" type="mds:FloatVector"/><element name="transitions" type="mds:FloatMatrix"/><sequence minOccurs="1" maxOccurs="unbounded">

<element name="State" type="mds:State"/></sequence>

</complexType>


Name Definition

Model Abstract DS from which the ProbabilityModel DS is derived.

ProbabilityModel Abstract DS from which the Gaussian DS is derived (in the future more models will be added).

Discrete Model:

Name Definition

DiscreteModel DiscreteModel extends the Probability Model by defining a discrete probability distribution

Vector Vector specifies the discrete probability distribution of the DiscreteModel

Mixture Model:

Name Definition

MixtureModel MixtureModel extends the Probability Model by defining a mixture (weighted sum) of individual Gaussian probability models

Vector Vector specifies the weighting of the mixture of Gaussian probability models

Gaussian Gaussian specifies the sequence of Gaussian (can be GeneralizedGaussian or HighOrderGaussian) probability models the contribute to the mixture. The order in the sequence corresponds to the elements of the weighting vector.

State:

Name Definition

State State extends Analytic Model and allows the definition of a state with a class label and probability model

ProbabilityModel ProbabilityModel specifies the probability model for the state, which allows the state to have a confidence

StateTransitionModel:

Name Definition

StateTransitionModel StateTransitionModel specifies the state-transition model in terms of set of states, transitions between them and initial probabilities of the states. Consider that there are N states.


Name Definition

initial Initial is an N-dimensional vector that specifies the initial probability of each of the N states. There is a correspondence between the N elements of the initial vector and the N states.

transitions Transitions is an N x N matrix the specifies the probability of transition between the N states.

State State is the length N vector of states.


13.3.2 Gaussian DS

The Gaussian DS is used to specify a multi-dimensional probability distribution in terms of low-order Gaussian approximation. The Gaussian DS includes statistical measures of multi-dimensional centroid and variance represented by either a variance vector or covariance matrix.




<element name="Gaussian" type="mds:Gaussian"/><complexType name="Gaussian" base="mds:ProbabilityModel" derivedBy="extension">

<element name="Mean" type="mds:FloatVector"/><choice minOccurs="0">

<element name="Variance" type="mds:FloatVector"/><element name="Covariance" type="mds:FloatMatrix"/>


<element name="GeneralizedGaussian" type="mds:GeneralizedGaussian"/><complexType name="GeneralizedGaussian" base="mds:Gaussian"

derivedBy="extension"><attribute name="shape" type="float"/>

</complexType>

<element name="HigherOrderGaussian" type="mds:HigherOrderGaussian"/><complexType name="HigherOrderGaussian"

base="mds:GeneralizedGaussian" derivedBy="extension"><sequence minOccurs="0" maxOccurs="unbounded">

<element name="moment" type="mds:FloatVector"/></sequence>

</complexType>


Gaussian:

Name Definition

ProbabilityModel Abstract DS from which the Gaussian DS is derived.

Mean A multi-dimensional vector which gives the multi-dimensional mean or centroid of the Gaussian.

Variance A multi-dimensional vector which gives the multi-dimensional variance of the Gaussian. The Gaussian DS allows choice of multi-dimensional variance vector or covariance matrix. The Variance is optional in which case the Gaussian is represented only by its mean.

Covariance A matrix which gives the covariance of the Gaussian. The Gaussian DS allows choice of multi-dimensional variance vector or covariance matrix. The Covariance is optional in which case the Gaussian is represented only by its mean.

Gaussian Specification of a multi-dimensional probability distribution in terms of low-order Gaussian approximation. The Gaussian DS includes statistical measures of multi-dimensional centroid and variance represented by either a variance vector or covariance matrix.

Generalized Gausssian:

Name Definition

GeneralizedGaussian GeneralizedGaussian extends the Gaussian by adding a shape parameter

Shape Shape gives the shape of the Generalized Gaussian distribution, i.e., Gaussian has shape=2

Higher Order Gaussian:

Name Definition

HigherOrderGaussian HigherOrderGuassian extends the Gaussian by adding optional higher order moments

Moment Moments in the form of vectors give the higher-order moments of the HigherOrderGaussian


13.4 Analytic Model

The Analytic model DS is used to specify a sample set of audio-visual data, class of descriptors, group of classes, or cluster of descriptors. The Analytic model DS gives a semantic label represented by a string, which is used by


the derived DSs to give a semantic label to each model. The Analytic model DS also optionally indicates the confidence in which the semantic label is assigned to the model.

13.4.1 AnalyticModel DS




<complexType name="AnalyticModel" base="mds:Model" derivedBy="extension" abstract="true">

<attribute name="SemanticLabel" type="string" use="optional"/><attribute name="Confidence" type="float" use="optional"/>

</complexType>


Name Definition

ModelDS Abstract DS from which the AnalyticModel DS is derived.

SemanticLabel A semantic label for the cluster given as a string. The semantic label is optional.

Confidence Gives an optional confidence value, which indicates the confidence in which the semantic label is assigned to the class.

AnalyticModel The Analytic model DS is used to specify a sample set of audio-visual data, class of descriptors, group of classes, or cluster of descriptors. It is an abstract class from which the Cluster, the Example, the ProbabilityModelClass DSs are derived.


13.4.2 Cluster DS

The Cluster DS is used to specify a group of audio-visual data. In general, the elements of the cluster can be audio samples, images, regions, segments, video programs, and so forth. The Cluster DS allows the assignment of a semantic label to the cluster.


<element name="Cluster" type="mds:Cluster"/><complexType name="Cluster" base="mds:AnalyticModel" derivedBy="extension">

<element name="MediaLocator" type="mds:MediaLocator" maxOccurs="unbounded"/><attribute name="NumElements" type="integer" use="optional"/>

</complexType>

Name Definition

AnalyticModel Abstract DS from which the Cluster DS is derived.

NumElements The number of elements in the cluster given as an integer.

MediaLocator A list of MediaLocators that give the audio-visual data items that are members of the cluster.

Cluster DS derived from the AnalyticModel DS. The Cluster DS is used to specify a group of audio-visual data and allows the assignment of a semantic label to the cluster.


13.4.3 Examples DS

The Examples DS describes a group of descriptors. The Examples DS allows the assignment of a semantic label to the group.




<element name="Examples" type="mds:Examples"/><complexType name="Examples" base="mds:AnalyticModel" derivedBy="extension">

<element name="Descriptor" type="mds:Descriptor" maxOccurs="unbounded"/><attribute name="NumElements" type="integer" use="optional"/><attribute name="DescriptorName" type="string" use="optional"/>

</complexType>

Editor's note: The type descriptor is not defined. 13.4.3.2 Description Scheme Semantics

Name Definition

AnalyticModel Abstract DS from which the Examples DS is derived.

NumElements The number of elements (Descriptors) in the group given as an integer.

DescriptorName The name of the Descriptor class.

Descriptor A list of Descriptors that are members of the group of examples.

Examples The Examples DS describes a group of descriptors. The Examples DS allows the assignment of a semantic label to the group.

13.4.4 ProbabilityModelClass DS

The ProbabilityModelClass DS is used to specify a class of descriptors in terms of a probability model. The confidence value that is associated with the probability model class, which indicates the confidence of the assignment of the semantic label, may be based on the sample size for the probability model.




<element name="ProbabilityModelClass" type="mds:ProbabilityModelClass"/><complexType name="ProbabilityModelClass" base="mds:AnalyticModel"

derivedBy="extension"><element name="ProbabilityModel" type="mds:ProbabilityModel"/><attribute name="DescriptorName" type="string" use="optional"/>

</complexType>


Name Definition

AnalyticModel Abstract DS from which the ProbabilityModelClass DS is derived.

DescriptorName The name of the Descriptor class.

ProbabilityModel A Probability Model that gives a statistical approximation of the Descriptors in the class.

ProbabilityModelClass The ProbabilityModelClass DS is used to specify a class of descriptors in terms of a probability model. The confidence value that is associated with the probability model class, which indicates the confidence of the assignment of the semantic label, may be based on the sample size for the probability model.


13.5 Classifiers

13.5.1 Classifier DS

The Classifier DS provides a way to describe the different types of classifiers that are used to assign semantic labels to audio-visual data.




<complexType name="Classifier" base="mds:Model" derivedBy="extension" abstract="true">

<attribute name="Confidence" type="float" use="optional"/></complexType>

Name Definition

Model Abstract DS from which the Classifier DS is derived.

Confidence Gives the overall confidence of the classifier in assigning semantic labels. The confidence attribute is optional.

Classifier The Classifier DS describes different types of classifiers that are used to assign semantic labels to audio-visual data. It is an abstract DS from which the ClusterSet, the ExampleSet, the probabilityModelClassifier DSs are derived.


13.5.2 ClusterSet DS

The ClusterSet DS describes a set of clusters that each represent a different semantic concept. This allows the ClusterSet DS to be used for purposes of classification whereby audio-visual data is assigned semantic labels based on the cluster sets.




<element name="ClusterSet" type="mds:ClusterSet"/><complexType name="ClusterSet" base="mds:Classifier" derivedBy="extension">

<element name="Cluster" type="mds:Cluster" maxOccurs="unbounded"/><attribute name="NumElements" type="integer" use="optional"/>

</complexType>


Name Definition

Classifier Abstract DS from which the ClusterSet DS is derived.

NumElements The number of Cluster sets in the classifier

ClusterSet The list of Cluster sets, where each ClusterSet gives a semantic label to its class.

ClusterSet The ClusterSet DS describes a set of clusters that each represent a different semantic concept. This allows the ClusterSet DS to be used for purposes of classification whereby audio-visual data is assigned semantic labels based on the cluster sets.

13.5.3 ExamplesSet DS

The ExamplesSet DS describes a set of example groups where each example group represents a different semantic concept. This allows the ExamplesSet DS to be used for purposes of classification whereby based on descriptor values, audio-visual data is assigned semantic labels.




<element name="ExamplesSet" type="mds:ExamplesSet"/><complexType name="ExamplesSet" base="mds:Classifier" derivedBy="extension">

<element name="Examples" type="mds:Examples" maxOccurs="unbounded"/><attribute name="NumElements" type="integer" use="optional"/>

</complexType>


Name Definition

Classifier Abstract DS from which the ExamplesSet DS is derived.

NumElements The number of Example sets in the classifier

Examples The list of Examples DSs where each ExamplesSet gives a semantic label to its class.

ExamplesSet The ExamplesSet DS describes a set of example groups where each example group represents a different semantic concept. This allows the ExamplesSet DS to be used for purposes of classification whereby based on descriptor values, audio-visual data is assigned semantic labels.


13.5.4 ProbabilityModelClassifier DS

The Probability Model Classifier DS describes a set of classes in terms of probability models for representing different semantic concepts. This allows the Probability Model Classifier DS to be used for classifying and assigning semantic labels to audio-visual data based on the classes.




<element name="ProbabilityModelClassifier" type="mds:ProbabilityModelClassifier"/>

<complexType name="ProbabilityModelClassifier" base="mds:Classifier" derivedBy="extension">

<element name="ProbabilityModelClass" type="mds:ProbabilityModelClass" maxOccurs="unbounded"/>

<attribute name="NumElements" type="integer"/>

</complexType>


Name Definition

Classifier Abstract DS from which the ProbabilityModelClassifer DS is derived.

NumElements The number of Probability Model Classes in the classifier

ProbabilityModelClass The list of Probability Model Classes that gives statistical approximations of the classes. Each ProbabilityModelClass gives a semantic label to its class.

ProbabilityModelClassifier The ProbabilityModelClassifier DS describes a set of classes in terms of probability models for representing different semantic concepts. This allows the Probability Model Classifier DS to be used for classifying and assigning semantic labels to audio-visual data based on the classes



14 User Interaction

14.1 User Preferences

The UserPreference DS is used to describe user’s preferences pertaining to consumption of multimedia material. User preference descriptions can be correlated with media descriptions to find and consume desired content. Correspondence between user preference and media descriptions facilitate accurate and efficient personalization of content access and content consumption.

The UserPreference DS enables users to specify their preferences for certain types of content and preferences for ways of browsing the content. The DS enables users to specify preferences that depend on their context in terms of time and place. The DS allows users to specify the relative importance of their preferences with respect to each other. The DS enables users to indicate whether their preferences or parts of their preferences should be kept private or not. The DS also enables users to indicate whether the automatic update of their usage preferences description, e.g., by an agent, should be permitted or not.

14.1.1 UserPreference DS

The UserPreference DS is used to describe user’s preferences pertaining to consumption of multimedia material. User preference descriptions can be correlated with media descriptions to find and consume desired content. Correspondence between user preference and media descriptions facilitate accurate and efficient personalization of content access and content consumption.




<element name="UserPreference" type="mds:UserPreference"/><complexType name="UserPreference">

<element name="UserIdentifier" type="mds:UserIdentifier" minOccurs="1" maxOccurs="1"/>

<element name="UsagePreferences" type="mds:UsagePreferences" minOccurs="0" maxOccurs="1"/>

</complexType>


Name Definition

UserIdentifier Identifies a particular preference description of a user

UsagePreferences A container DS that contains:

FilteringAndSearchPreferences DS

BrowsingPreferences DS

14.1.2 UserIdentifier DS

The UserIdentifier DS is used to identify a particular description of user preferences. A single person can use multiple identifiers, each of which identifies a different set of user preferences.

<complexType name="UserIdentifier"><attribute name="protection" type="boolean" use="default" value="true"/><attribute name="userName" type="string" use="default" value="anonymous"/>

</complexType>


Name Definition

UserIdentifier Identifies a particular preference description of a user.

protection Describes user’s desire to keep the identifier description private.

userName Name associated with the user, or name given to a particular set of user preferences.

14.1.3 PreferenceType DS

The PreferenceType DS is used to specify a combination of time and/or place that can be associated with a particular set of user preferences. The PreferenceType DS is used to specify a dependency of user preferences on time and location.




<complexType name="PreferenceType"><element name="Place" type="mds:Place" minOccurs="0" maxOccurs="1"/><element name="Time" type="mds:Time" minOccurs="0" maxOccurs="1"/>

</complexType>


Name Definition

PreferenceType Specifies dependency of a set of user preferences on time and place.

Place Place associated with user preferences. See Section 6.3.1.

Time Time associated with user preferences. See Section 5.3.

14.1.4 UsagePreferences DS

The UsagePreferences DS is used to specify user’s preferences pertaining to filtering, searching and browsing of audio-visual content.




<element name="UsagePreferences" type="mds:UsagePreferences"/><complexType name="UsagePreferences">

<element name="FilteringAndSearchPreferences" type="mds:FilteringAndSearchPreferences" minOccurs="0" maxOccurs="unbounded"/>

<element name="BrowsingPreferences" type="mds:BrowsingPreferences" minOccurs="0" maxOccurs="unbounded"/>

<attribute name="allowAutomaticUpdate" type="boolean" use="default" value="true"/>

</complexType>


Name Definition

UsagePreferences A container DS that contains:

FilteringAndSearchPreferences DS

BrowsingPreferences DS

BrowsingPreferences Describes user’s preferences for AV content browsing. See Section 14.1.5.

FilteringAndSearchPreferences Describes user’s preferences for filtering of AV content or searching for preferred AV content. See Section 14.1.7.

allowAutomaticUpdate Indicates whether the user permits automatic update of the usage preferences description, e.g., by an agent.

14.1.5 BrowsingPreferences DS

The BrowsingPreferences DS is used to specify user preferences pertaining to navigation of and access to media.




<element name="BrowsingPreferences" type="mds:BrowsingPreferences"/><complexType name="BrowsingPreferences">

<element name="SummaryPreferences" type="mds:SummaryPreferences"

minOccurs="0" maxOccurs="unbounded"/><element name="BrowsingPreferenceType" type="mds:PreferenceType"

minOccurs="0" maxOccurs="unbounded"/><attribute name="protection" type="boolean" use="default" value="true"/><attribute name="preferenceValue" type="integer" use="optional"/>

</complexType>


Name Definition

BrowsingPreferences Describes user’s preferences for AV content browsing.

SummaryPreferences Describes user’s preferences relating to AV media summaries.

BrowsingPreferenceType Identifies the usage condition(s) for a particular browsing preference description. See Section 14.1.3.

protection Describes user’s desire to keep browsing preferences private.

preferenceValue Describes the relative priority or weight assigned to a particular browsing preference description, in case multiple browsing preference descriptions are present.

14.1.6 SummaryPreferences DS

The SummaryPreferences DS is used to specify user’s preferences for visualization and sonification of particular AV content.




<complexType name="SummaryPreferences"><element name="SummaryTypePreference" minOccurs="0" maxOccurs="unbounded">

<complexType base="mds:SummaryType" derivedBy="extension"> <attribute name="preferenceValue" type="integer" use="optional"/> </complexType>

</element><element name="PreferredSummaryTheme" minOccurs="0" maxOccurs="unbounded">

<complexType base="string" derivedBy="extension"> <attribute ref="xml:lang"/> <attribute name="preferenceValue" type="integer" use="optional"/> </complexType>

</element><element name="SummaryDuration" type="mds:timeMediaDuration"

minOccurs="0" maxOccurs="1"/><element name="MinSummaryDuration" type="mds:MediatimeDuration"

minOccurs="0" maxOccurs="1"/><element name="MaxSummaryDuration" type="mds:MediatimeDuration"

minOccurs="0" maxOccurs="1"/><element name="NumOfKeyframes" type="positiveInteger"

minOccurs="0" maxOccurs="1"/>

<element name="MinNumOfKeyframes" type="positiveInteger" minOccurs="0" maxOccurs="1"/>

<element name="MaxNumOfKeyframes" type="positiveInteger" minOccurs="0" maxOccurs="1"/>

<attribute name="preferenceValue" type="integer" use="optional"/></complexType>


Name Definition

SummaryPreferences Describes user’s preferences relating to media summaries and their visualization and sonification.

SummaryTypePreference Describes the type of the preferred summary and corresponds to the value of SummaryType in a media summary description. A preferenceValue attribute may be attached to each instantiation of SummaryTypePreference, to indicate the relative priority or weight of each element.

PreferredSummaryTheme Describes the preferred summary names or names/themes for media segments where the names/themes are included in summary and segment media descriptions. A preferenceValue attribute may be attached to each instantiation of PreferredSummaryTheme, to indicate the relative priority or weight of each element.

NumOfKeyframes Describes the preferred number of keyframes and corresponds to the value of the NumOfKeyframes in a media summary description.

MinNumOfKeyframes Describes the preferred minimum number of keyframes in a video summary.

MaxNumOfKeyframes Describes the preferred maximum number of keyframes in a video summary.

SummaryDuration Describes the preferred duration for a highlight summary of media and corresponds to the value of Duration in a media summary description .

MinSummaryDuration Describes the preferred minimum duration for a highlight video summary.

MaxSummaryDuration Describes the preferred maximum duration for a highlight video summary.

preferenceValue Describes the relative priority or weight assigned to a particular SummaryPreferences description, in case multiple preference descriptions are present.

14.1.7 FilteringAndSearchPreferences DS

The FilteringAndSearchPreferences DS specifies users’ filtering or searching preferences in terms of creation and classification aspects of the AV content.

<element name="FilteringAndSearchPreferences" type="mds:FilteringAndSearchPreferences"/>

<complexType name="FilteringAndSearchPreferences"><element name="ClassificationPreferences" type="mds:ClassificationPreferences"

minOccurs="0" maxOccurs="unbounded"/><element name="CreationPreferences" type="mds:CreationPreferences"

minOccurs="0" maxOccurs="unbounded"/><element name="SourcePreferences" type="mds:SourcePreferences"

minOccurs="0" maxOccurs="unbounded"/><element name="FilteringAndSearchPreferenceType" type="mds:PreferenceType"

minOccurs="0" maxOccurs="unbounded"/><attribute name="protection" type="boolean" use="default" value="true"/><attribute name="preferenceValue" type="integer" use="optional"/>

</complexType>


Name Definition

FilteringAndSearchPreferences Describes user’s preferences for filtering of AV content or searching for preferred AV content.

ClassificationPreferences Describes the user’s preference related to media classification descriptions.

CreationPreferences Describes the user’s preference related to media creation descriptions.

SourcePreferences Describes the user’s preference for a particular source of media.

FilteringAndSearchPreferenceType Identifies the usage condition(s) for a particular filtering and search preference description. See Section 14.1.3.

protection Describes user’s desire to keep filtering and search preferences private.

preferenceValue Describes the relative priority or weight assigned to a particular filtering and search preference description, in case multiple filtering and search preference descriptions are present.

14.1.8 ClassificationPreferences DS

The ClassificationPreferences DS is used to specify user preferences related to classification of the content, e.g., preferred genre, preferred country of origin or preferred language.


Editor's Note: Are country, language defined as a global datatype?PvB: Tentatively corrected below (is Language correct like this?)

<complexType name="ClassificationPreferences"> <element name="Country" minOccurs="0" maxOccurs="unbounded">

<complexType base="mds:ISO3166-1CountryCodeCountry" derivedBy="extension"> <attribute name="preferenceValue" type="integer" use="optional"/> </complexType> </element> <element name="Language" minOccurs="0" maxOccurs="unbounded"> <complexType base="xsd:languagemds:Language" derivedBy="extension"> <attribute name="preferenceValue" type="integer" use="optional"/> </complexType> </element> <element name="Genre" minOccurs="0" maxOccurs="unbounded"> <complexType base="mds:ControlledTerm" derivedBy="extension"> <attribute name="preferenceValue" type="integer" use="optional"/> </complexType> </element> <element name="PackagedType" minOccurs="0" maxOccurs="unbounded"> <complexType base="mds:ControlledTerm" derivedBy="extension"> <attribute name="preferenceValue" type="integer" use="optional"/> </complexType> </element> <attribute name="id" type="ID"/> <attribute name="preferenceValue" type="integer" use="optional"/>

</complexType>


Name Definition

ClassificationPreferences Specifies user preferences related to the class of the content.

Country Describes user’s preference for country of origin of the content. See Section Error: Reference source not found. A preferenceValue attribute may be attached to each element to indicate its relative priority with respect to other Country elements.

Language Describes user’s preference for language of origin of the content. See Section Error: Reference source not found. A preferenceValue attribute may be attached to each element to indicate its relative priority with respect to other Language elements.

Genre Describes user’s preference for the genre of the content. See Section 8.1.2. A preferenceValue attribute may be attached to each element to indicate its relative priority with respect to other Genre elements.

PackagedType Describes user’s preference for the packaged type that applies to the subject of the AV content. The packaged type classifies AV content from a point of view of types of program, without considering genre classification. See Section 8.1.2. A preferenceValue attribute may be attached to each element to indicate its relative priority with respect to other PackagedType elements.

preferenceValue Describes the relative priority or weight assigned to a particular filtering and search preference description, in case multiple preference descriptions are present.


14.1.9 CreationPreferences DS

The CreationPreferences DS is used to specify user preferences related to creation of the content, e.g., favorite titles and favorite actors.




<complexType name="CreationPreferences"> <element name="Title" minOccurs="0" maxOccurs="unbounded"> <complexType base="mds:Title" derivedBy="extension"> <attribute name="preferenceValue" type="integer" use="optional"/> </complexType> </element> <element name="Creator" minOccurs="0" maxOccurs="unbounded">

<complexType base="mds:Creator" derivedBy="extension"> <attribute name="preferenceValue" type="integer" use="optional"/> </complexType> </element> <element name="Location" minOccurs="0" maxOccurs="unbounded"> <complexType base="mds:Place" derivedBy="extension"> <attribute name="preferenceValue" type="integer" use="optional"/> </complexType> </element> <attribute name="id" type="ID"/> <attribute name="preferenceValue" type="integer" use="optional"/>

</complexType>


Name Definition

CreationPreferences Specifies user preferences related to the creation of the content.

Title Describes user’s preference for the title of the content. See Section 8.1.1. A preferenceValue attribute may be attached to each element to indicate its relative priority with respect to other Title elements.

Creator Describes user’s preference for the creator of the content (e.g., director or actor). See Section 8.1.1. A preferenceValue attribute may be attached to each element to indicate its relative priority with respect to other Creator elements.

Location Describes user’s preference for the Location where the content is created. See Section 8.1.1. A preferenceValue attribute may be attached to each element to indicate its relative priority with respect to other Location elements.

preferenceValue Describes the relative priority or weight assigned to a preference description, in case multiple preference descriptions are present.

14.1.10 SourcePreferences DS

The SourcePreferences DS is used to specify preferences for the source of the media, such as its medium.

<complexType name="SourcePreferences"> <element name="PublicationType" minOccurs="0" maxOccurs="unbounded">

<complexType base="mds:ControlledTerm" derivedBy="extension"> <attribute name="preferenceValue" type="integer" use="optional"/> </complexType> </element> <element name="PublicationChannel" minOccurs="0" maxOccurs="unbounded"> <complexType base="mds:ControlledTerm" derivedBy="extension"> <attribute name="preferenceValue" type="integer" use="optional"/> </complexType> </element> <element name="PublicationPlace" minOccurs="0" maxOccurs="unbounded"> <complexType base="mds:Place" derivedBy="extension"> <attribute name="preferenceValue" type="integer" use="optional"/> </complexType> </element> <element name="PublicationDate" minOccurs="0" maxOccurs="unbounded"> <complexType base="mds:TimePoint" derivedBy="extension"> <attribute name="preferenceValue" type="integer" use="optional"/> </complexType> </element> <element name="Publisher" minOccurs="0" maxOccurs="unbounded"> <complexType base="mds:Person" derivedBy="extension"> <attribute name="preferenceValue" type="integer" use="optional"/> </complexType>

</element> <attribute name="id" type="ID"/> <attribute name="preferenceValue" type="integer" use="optional"/>

</complexType>


Name Definition

SourcePreferences Specifies preferred source of the media.

PublicationType Describes user’s preference for publication medium of content, e.g., terrestrial broadcast, CD, etc. See Section 9. A preferenceValue attribute may be attached to each element to indicate its relative priority with respect to other PublicationType elements.

PublicationChannel Describes user’s preference for a particular channel referring to, e.g., a broadcast channel. See Section 9. A preferenceValue attribute may be attached to each element to indicate its relative priority with respect to other PublicationChannel elements.

PublicationPlace Describes user’s preference for the place where content is distributed from. See Section 9. A preferenceValue attribute may be attached to each element to indicate its relative priority with respect to other PublicationPlace elements.

PublicationDate Describes user’s preference for time and date of the content distribution, e.g., time and date of the broadcast. See Section 9. A preferenceValue attribute may be

attached to each element to indicate its relative priority with respect to other PublicationDate elements.

Publisher Describes user's preference for a publisher or distributor of the content, e.g. a broadcaster or other organization.

preferenceValue Describes the relative priority or weight assigned to a particular preference description, in case multiple preference descriptions are present.


15 Bibliography

[1] J. Meng, Y. Juan and S.-F. Chang, Scene Change Detection in a MPEG Compressed Video Sequence, Proceedings, IS&T/SPIE's Symposium on Electronic Imaging: Science & Techno logy (EI'95) – Digital Video Compression: Algorithms and Technologies, San Jose, February 1995.


Closing the schema:

</schema>

16 Annex 1: Schema definition

In this section, the syntax defined in the current document is extracted so that it can be easily used for XM developments and processed by validating parsers.

<schema xmlns="http://www.w3.org/1999/XMLSchema"xmlns:mds="http://www.example.com/???" targetNamespace="http://www.example.com/???"elementFormDefault="unqualified"attributeFormDefault="unqualified"><simpleType name="unsigned1" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="1"/></simpleType><simpleType name="unsigned2" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="3"/></simpleType><simpleType name="unsigned3" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="7"/></simpleType><simpleType name="unsigned4" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="15"/></simpleType><simpleType name="unsigned5" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="31"/></simpleType><simpleType name="unsigned6" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="63"/></simpleType><simpleType name="unsigned7" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="127"/></simpleType><simpleType name="unsigned8" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="255"/></simpleType><simpleType name="unsigned9" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="511"/></simpleType><simpleType name="unsigned10" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="1023"/>

</simpleType><simpleType name="unsigned11" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="2047"/></simpleType><simpleType name="unsigned12" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="4095"/></simpleType><simpleType name="unsigned13" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="8191"/></simpleType><simpleType name="unsigned14" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="16384"/></simpleType><simpleType name="unsigned15" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="32767"/></simpleType><simpleType name="unsigned16" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="65535"/></simpleType><simpleType name="unsigned24" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="16777215"/></simpleType><simpleType name="unsigned32" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="4294967295"/></simpleType><simpleType name="unsigned64" base="nonNegativeInteger"><minInclusive value="0"/><maxInclusive value="18446744073709551615"/></simpleType><simpleType name="listOfFloat" base="float" derivedBy="list"/><complexType name="FloatVector" base="mds:listOfFloat" derivedBy="extension"><attribute name="Size" type="nonNegativeInteger" use="optional"/></complexType><simpleType name="listOfDouble" base="double" derivedBy="list"/><complexType name="DoubleVector" base="mds:listOfDouble" derivedBy="extension"><attribute name="Size" type="nonNegativeInteger" use="optional"/></complexType>

<simpleType name="listOfDecimal" base="decimal" derivedBy="list"/><complexType name="DecimalVector" base="mds:listOfDecimal" derivedBy="extension"><attribute name="Size" type="nonNegativeInteger" use="optional"/></complexType><simpleType name="listOfInteger" base="integer" derivedBy="list"/><complexType name="IntegerVector" base="mds:listOfInteger" derivedBy="extension"><attribute name="Size" type="nonNegativeInteger" use="optional"/></complexType><complexType name="FloatMatrix" base="mds:listOfFloat" derivedBy="extension"><attribute name="Size" use="required"><simpleType base="positiveInteger" derivedBy="list"/></attribute></complexType><complexType name="DoubleMatrix" base="mds:listOfDouble" derivedBy="extension"><attribute name="Size" use="required"><simpleType base="positiveInteger" derivedBy="list"/></attribute></complexType><complexType name="DecimalMatrix" base="mds:listOfDecimal" derivedBy="extension"><attribute name="Size" use="required"><simpleType base="positiveInteger" derivedBy="list"/></attribute></complexType><complexType name="IntegerMatrix" base="mds:listOfInteger" derivedBy="extension"><attribute name="Size" use="required"><simpleType base="positiveInteger" derivedBy="list"/></attribute></complexType><complexType name="FloatDiagonalMatrix" base="mds:FloatVector"/><complexType name="DoubleDiagonalMatrix" base="mds:DoubleVector"/><complexType name="DecimalDiagonalMatrix" base="mds:DecimalVector"/><complexType name="IntegerDiagonalMatrix" base="mds:IntegerVector"/>

<complexType name="Histogram"><element name="HistogramValues" type="mds:IntegerVector"/><attribute name="HistogramNormFactor" type="integer" use="required"/></complexType><complexType name="UniformQuantizer"><element name="BinNumber" type="integer" minOccurs="0" maxOccurs="unbounded"/><attribute name="SpaceDimension" type="nonNegativeInteger" use="required"/></complexType><complexType name="Reference" content="empty"><attribute name="href" type="uriReference" use="optional"/><attribute name="idref" type="IDREF" use="optional"/></complexType><complexType name="ReferenceToSegment" base="mds:Reference"derivedBy="restriction"><attribute name="href" type="uriReference" use="optional"/><attribute name="idref" type="IDREF" refType="mds:Segment" use="optional"/></complexType><complexType name="ReferenceToProgram" base="mds:Reference"derivedBy="restriction"><attribute name="href" type="uriReference" use="optional"/><attribute name="idref" type="IDREF" refType="MediaInstance" use="optional"/></complexType><complexType name="UIdentifier"><element name="IdValue" type="string"/><attribute name="IdOrganization" type="string"/><attribute name="IdName" type="string"/></complexType><simpleType name="FractionalTime" base="string" derivedBy="restriction"><pattern value="-?(\d+(\-\d{2}(\-\d{2})?)?)?(T\d{2}(:\d{2}(:\d{2}(:d+(\.d{2})?)?)?)?)?((\-|\+)\d{2}:d{2})?(F\d+)?"/></simpleType>

<simpleType name="TimePoint" base="mds:FractionalTime" derivedBy="restriction"><pattern value="-?(\d+(\-\d{2}(\-\d{2})?)?)?(T\d{2}(:\d{2}(:\d{2}(:d+)?)?)?)?(\-|\+\d{2}:d{2})?(F\d+)?"/></simpleType><simpleType name="FractionalDuration" base="string" derivedBy="restriction"><pattern value="-?P(\d+D)?(T(\d+H)?(\d+M)?(\d+S)?(\d+N)?(\d{2}f)?)?(\-|\+\d{2}:d{2}Z)?(\d+F)?"/></simpleType><simpleType name="Duration" base="mds:FractionalDuration" derivedBy="restriction"><pattern value="-?P(\d+D)?(T(\d+H)?(\d+M)?(\d+S)?(\d+N)?)?(\-|\+\d{2}:d{2}Z)?(\d+F)?"/></simpleType><complexType name="IncrDuration" base="integer" derivedBy="extension"><attribute name="timeunit" type="mds:Duration" use="optional"/></complexType><complexType name="RelTimePoint" base="mds:Duration" derivedBy="extension"><attribute name="TimeBase" type="uriReference" use="optional"/></complexType><complexType name="RelIncrTimePoint" base="mds:IncrDuration" derivedBy="extension"><attribute name="TimeBase" type="uriReference" use="optional"/></complexType><complexType name="Time"><sequence><choice><element name="TimePoint" type="mds:TimePoint"/><element name="RelTimePoint" type="mds:RelTimePoint"/><element name="RelIncrTimePoint" type="mds:RelIncrTimePoint"/></choice><choice minOccurs="0"><element name="Duration" type="mds:Duration"/><element name="IncrDuration" type="mds:IncrDuration"/></choice></sequence></complexType><simpleType name="MediaTimePoint" base="mds:FractionalTime" derivedBy="restriction"><pattern value="-?(\d+(\-\d{2}(\-\d{2})?)?)?(T\d{2}(:\d{2}(:\d{2}(:d+)?)?)?)?(F\d+)?"/></simpleType><simpleType name="MediaDuration" base="mds:FractionalDuration"

derivedBy="restriction"><pattern value="-?P(\d+D)?(T(\d+H)?(\d+M)?(\d+S)?(\d+N)?)?(\d+F)?"/></simpleType><complexType name="MediaIncrDuration" base="integer" derivedBy="extension"><attribute name="timeunit" type="mds:MediaDuration" use="optional"/></complexType><complexType name="MediaRelTimePoint" base="mds:MediaDuration" derivedBy="extension"><attribute name="TimeBase" type="uriReference" use="optional"/></complexType><complexType name="MediaRelIncrTimePoint" base="mds:MediaIncrDuration"derivedBy="extension"><attribute name="TimeBase" type="uriReference" use="optional"/></complexType><complexType name="MediaTime"><sequence><choice><element name="MediaTimePoint" type="mds:MediaTimePoint"/><element name="MediaRelTimePoint" type="mds:MediaRelTimePoint"/><element name="MediaRelIncrTimePoint"type="mds:MediaRelIncrTimePoint"/></choice><choice minOccurs="0"><element name="MediaDuration" type="mds:MediaDuration"/><element name="MediaIncrDuration" type="mds:MediaIncrDuration"/></choice></sequence></complexType><complexType name="MediaLocator"><element name="MediaURL" type="mds:MediaURL" minOccurs="0"/><element name="MediaTime" type="mds:MediaTime" minOccurs="0"/></complexType><complexType name="VideoSegmentLocator" base="mds:MediaLocator"derivedBy="extension"/><complexType name="ImageLocator" base="mds:MediaLocator" derivedBy="extension"/><complexType name="AudioSegmentLocator" base="mds:MediaLocator"derivedBy="extension"/>

<complexType name="SoundLocator" base="mds:AudioSegmentLocator"derivedBy="extension"/><attribute ref="xml:lang"/><complexType name="ControlledTerm" base="string" derivedBy="extension"><attribute name="CSName" type="string" use="optional"/><attribute name="CSTermId" type="string" use="optional"/><attribute name="CSLocation" type="uri" use="optional"/></complexType><complexType name="TextualDescription" base="string" derivedBy="extension"><attribute ref="xml:lang"/></complexType><element name="TextAnnotation" type="mds:TextualDescription"/><element name="StructuredAnnotation" type="mds:StructuredAnnotation"/><complexType name="StructuredAnnotation"><element name="Who" type="mds:ControlledTerm" minOccurs="0"/><element name="WhatObject" type="mds:ControlledTerm" minOccurs="0"/><element name="WhatAction" type="mds:ControlledTerm" minOccurs="0"/><element name="Where" type="mds:ControlledTerm" minOccurs="0"/><element name="When" type="mds:ControlledTerm" minOccurs="0"/><element name="Why" type="mds:ControlledTerm" minOccurs="0"/><element name="TextAnnotation" type="mds:TextualDescription" minOccurs="0"/><attribute name="id" type="ID"/><attribute ref="xml:lang"/></complexType><complexType name="Person" abstract="true"><attribute name="id" type="ID"/></complexType><element name="Person" type="mds:Person"/><complexType name="NameComponent" base="mds:TextualDescription"derivedBy="extension"><attribute name="initial" type="string"/><attribute name="abbrev" type="string"/></complexType><complexType name="PersonName">

<complexType name="EntityNode"><sequence maxOccurs="unbounded"><choice minOccurs="0" maxOccurs="unbounded"><element name="EntityReference" type="mds:Reference"/><element name="EntityNode" type="mds:EntityNode"/></choice><element name="RelationshipNode" type="mds:RelationshipNode"minOccurs="0" maxOccurs="unbounded"/></sequence><attribute name="id" type="ID"/></complexType><complexType name="Relationship"><attribute name="id" type="ID" use="optional"/><attribute name="type" type="string" use="optional"/><attribute name="name" type="string" use="required"/><attribute name="degree" type="integer"/><attribute name="symmetric" type="boolean" use="optional"/><attribute name="property" use="optional"><simpleType base="string"><enumeration value="AscendingOrder"/><enumeration value="DescendingOrder"/><enumeration value="PartialOrder"/><enumeration value="Equivalence"/></simpleType></attribute></complexType><complexType name="RelationshipNode"><element name="Relationship" type="mds:Relationship"/><choice maxOccurs="unbounded"><element name="RelationshipNode" type="mds:RelationshipNode"/><element name="EntityNode" type="mds:EntityNode"/></choice></complexType><complexType name="EntityRelationshipGraph"><choice maxOccurs="unbounded"><element name="RelationshipNode" type="mds:RelationshipNode"/><element name="EntityNode" type="mds:EntityNode"/></choice><attribute name="id" type="ID" use="optional"/><attribute name="type" type="string" use="optional"/><attribute name="name" type="string" use="optional"/></complexType><complexType name="MediaIdentification"><element name="Identifier" type="mds:UIdentifier" minOccurs="0"/><element name="AudioDomain" type="mds:ControlledTerm"minOccurs="0" maxOccurs="unbounded"/><element name="VisualDomain" type="mds:ControlledTerm"

minOccurs="0" maxOccurs="unbounded"/><element name="ImageDomain" type="mds:ControlledTerm"minOccurs="0" maxOccurs="unbounded"/><attribute name="id" type="ID"/></complexType><complexType name="AspectRatio"><attribute name="Height" type="float"/><attribute name="Width" type="float"/></complexType><complexType name="MediaFormat"><element name="FileFormat" type="mds:ControlledTerm"/><element name="AspectRatio" type="mds:AspectRatio" minOccurs="0"/><element name="System" type="mds:ControlledTerm" minOccurs="0"/><element name="SamplingStructure" type="mds:ControlledTerm" minOccurs="0"/><element name="Medium" type="mds:ControlledTerm" minOccurs="0"/><element name="Color" type="mds:ControlledTerm" minOccurs="0"/><element name="Sound" type="mds:ControlledTerm" minOccurs="0"/><element name="FileSize" type="nonNegativeInteger" minOccurs="0"/><element name="Length" type="mds:Duration" minOccurs="0"/><element name="AudioChannels" type="nonNegativeInteger" minOccurs="0"/><element name="AudioCodingFormat" type="mds:ControlledTerm" minOccurs="0"/><element name="AudioLanguage" type="language" minOccurs="0"/><attribute name="id" type="ID"/></complexType><complexType name="MediaCoding"><element name="Resolution" type="nonNegativeInteger" minOccurs="0"/><element name="FrameWidth" type="nonNegativeInteger" minOccurs="0"/><element name="FrameHeight" type="nonNegativeInteger" minOccurs="0"/><element name="FrameRate" type="float" minOccurs="0"/><element name="AudioSamplingRate" type="float" minOccurs="0"/><element name="CompressionFormat" type="mds:ControlledTerm" minOccurs="0"/><element name="QuantizerCoarseness" type="nonNegativeInteger" minOccurs="0"/><element name="PredictionMode" type="mds:ControlledTerm" minOccurs="0"/><element name="BitRate" type="float" minOccurs="0"/><element name="Bandwidth" type="float" minOccurs="0"/><attribute name="id" type="ID"/></complexType><element name="MediaName" type="string"/><element name="InstanceLocator" type="mds:InstanceLocator"/><complexType name="InstanceLocator"><choice><element ref="MediaURL"/><element ref="MediaName"/></choice></complexType><complexType name="MediaInstance"><element name="Identifier" type="mds:UIdentifier"/><element ref="InstanceLocator"/><attribute name="id" type="ID"/></complexType>

<complexType name="MediaProfile"><element name="MediaFormat" type="mds:MediaFormat"/><element name="MediaCoding" type="mds:MediaCoding"minOccurs="0" maxOccurs="unbounded"/><element name="MediaInstance" type="mds:MediaInstance"minOccurs="0" maxOccurs="unbounded"/><attribute name="id" type="ID"/><attribute name="master" type="boolean" use="default" value="false"/></complexType><element name="MediaInformation" type="mds:MediaInformation"/><complexType name="MediaInformation"><element name="MediaIdentification" type="mds:MediaIdentification"/><element name="MediaProfile" type="mds:MediaProfile" maxOccurs="unbounded"/><attribute name="id" type="ID"/></complexType><simpleType name="TitleType" base="string"><enumeration value="original"/><enumeration value="alternative"/><enumeration value="popular"/></simpleType><element name="TitleText" type="mds:TextualDescription"/><element name="Title" type="mds:Title"/><complexType name="Title"><element ref="TitleText" minOccurs="0" maxOccurs="1"/><element name="TitleImage" type="mds:ImageLocator"minOccurs="0" maxOccurs="1"/><element name="TitleVideo" type="mds:VideoSegmentLocator"minOccurs="0" maxOccurs="1"/><element name="TitleAudio" type="mds:AudioSegmentLocator"minOccurs="0" maxOccurs="1"/><attribute name="type" type="mds:TitleType"/></complexType><complexType name="Creator" base="mds:Person" derivedBy="extension"><element name="role" type="mds:ControlledTerm"/></complexType><element name="CreationMaterial"><complexType><element name="device_instrument" type="mds:ControlledTerm"/><element name="device_settings" type="mds:ControlledTerm"/></complexType></element><element name="Creation" type="mds:Creation"/><complexType name="Creation"><element ref="Creation" minOccurs="0" maxOccurs="unbounded"/><element ref="Title" minOccurs="0" maxOccurs="unbounded"/><element name="CreationDescription" type="mds:StructuredAnnotation"minOccurs="0" maxOccurs="unbounded"/><element name="Creator" type="mds:Creator"minOccurs="0" maxOccurs="unbounded"/><element name="CreationLocation" type="mds:Place"minOccurs="0" maxOccurs="unbounded"/><element name="CreationDate" type="mds:TimePoint"minOccurs="0" maxOccurs="unbounded"/><element ref="CreationMaterial" minOccurs="0" maxOccurs="unbounded"/>

<complexType name="UsageRecord"><element name="Type" type="mds:ControlledTerm" minOccurs="0"/><element name="Channel" type="mds:ControlledTerm" minOccurs="0"/><element name="Place" type="mds:Place" minOccurs="0"/><element name="Date" type="mds:TimePoint" minOccurs="0"/><element name="Audience" type="nonNegativeInteger"minOccurs="0" maxOccurs="1"/><element name="Distributor" type="mds:Person" minOccurs="0"/><element name="Financial" type="mds:Financial" minOccurs="0"/><element name="Rights" type="mds:Rights" minOccurs="0"/><attribute name="id" type="ID"/></complexType><element name="InternationalPrice"><complexType content="empty"><attribute name="currency" type="mds:ISO4217CurrencyCode"/><attribute name="value" type="decimal"/></complexType></element><complexType name="Cost"><element name="CostType" type="mds:ControlledTerm"/><element ref="InternationalPrice"/><attribute name="id" type="ID"/></complexType><complexType name="Income"><element name="IncomeType" type="mds:ControlledTerm"/><element ref="InternationalPrice"/><attribute name="id" type="ID"/></complexType><complexType name="Financial"><element name="Cost" type="mds:Cost" minOccurs="0" maxOccurs="unbounded"/><element name="Income" type="mds:Income" minOccurs="0" maxOccurs="unbounded"/><attribute name="id" type="ID"/></complexType><element name="UsageMetaInformation" type="mds:UsageMetaInformation"/><complexType name="UsageMetaInformation"><element name="Rights" type="mds:Rights" minOccurs="0"/><element name="UsageRecord" type="mds:UsageRecord"minOccurs="0" maxOccurs="unbounded"/><element name="Financial" type="mds:Financial" minOccurs="0"/><attribute name="id" type="ID"/></complexType><simpleType name="DecompositionDataType" base="string"><enumeration value="spatial"/><enumeration value="temporal"/><enumeration value="spatio-temporal"/><enumeration value="MediaSource"/></simpleType><complexType name="SegmentDecomposition"><element ref="Segment" minOccurs="1" maxOccurs="unbounded"/>

<element name="MotionParameters" type="float"minOccursPar="ModelType" maxOccursPar="ModelType"/></sequence><attribute name="ModelType" type="positiveInteger" use="required"/><attribute name="SourceSequenceWidth" type="positiveInteger"use="required"/><attribute name="SourceSequenceHeight" type="positiveInteger"use="required"/><attribute name="Xoffset" type="float" use="required"/><attribute name="Yoffset" type="float" use="required"/><attribute name="Xorigin" type="float" use="required"/><attribute name="Yorigin" type="float" use="required"/><attribute name="NoOfMotionParameterSets" type="positiveInteger"use="required"/></complexType><complexType name="SegmentNode"><choice minOccurs="0" maxOccurs="unbounded"><element name="ReferenceToSegment" type="mds:ReferenceToSegment"/><element name="SegmentNode" type="mds:SegmentNode"/></choice><element name="SegmentRelationshipNode" type="mds:SegmentRelationshipNode"minOccurs="0" maxOccurs="unbounded"/><attribute name="id" type="ID" use="optional"/></complexType><complexType name="SegmentRelationship" base="mds:Relationship"derivedBy="extension"/><complexType name="SegmentRelationshipNode"><element name="SegmentRelationship" type="mds:SegmentRelationship"minOccurs="1" maxOccurs="1"/><choice minOccurs="0" maxOccurs="unbounded"><element name="SegmentRelationshipNode"type="mds:SegmentRelationshipNode"/><element name="SegmentNode" type="mds:SegmentNode"/></choice></complexType><element name="SegmentRelationshipGraph" type="mds:SegmentRelationshipGraph"/><complexType name="SegmentRelationshipGraph"><choice minOccurs="1" maxOccurs="unbounded"><element name="SegmentRelationshipNode"type="mds:SegmentRelationshipNode"/><element name="SegmentNode" type="mds:SegmentNode"/></choice><attribute name="id" type="ID" use="optional"/><attribute name="type" type="string" use="optional"/><attribute name="name" type="string" use="optional"/></complexType><element name="Summarization" type="mds:Summarization"/><complexType name="Summarization"><element name="Summary" type="mds:Summary" maxOccurs="unbounded"/></complexType><complexType name="Summary" abstract="true"><element name="RefLocator" type="mds:MediaLocator"

minOccurs="0"/><element name="ReferenceToSegment" type="mds:ReferenceToSegment"minOccurs="0"/><element name="ReferenceToProgram" type="mds:ReferenceToProgram"minOccurs="0"/><attribute name="name" type="ID"/></complexType><simpleType name="SummaryType" base="string"><enumeration value="keyFrames"/><enumeration value="keyVideoClips"/><enumeration value="keyAudioClips"/><enumeration value="keyEvents"/><enumeration value="unconstrained"/></simpleType><simpleType name="SummaryTypeList" base="mds:SummaryType" derivedBy="list"/><simpleType name="HierarchyType" base="string"><enumeration value="independent"/><enumeration value="dependent"/></simpleType><complexType name="SummaryThemeList"><element name="SummaryTheme" maxOccurs="unbounded"><complexType base="mds:TextualDescription" derivedBy="extension"><attribute name="id" type="ID"/></complexType></element></complexType><complexType name="HierarchicalSummary" base="mds:Summary" derivedBy="extension"><element name="SummaryThemeList" type="mds:SummaryThemeList" minOccurs="0"/><element name="HighlightLevel" type="mds:HighlightLevel"maxOccurs="unbounded"/><attribute name="summaryTypeList" type="mds:SummaryTypeList"/><attribute name="hierarchyType" type="mds:HierarchyType" use="optional"/></complexType><simpleType name="FidelityType" base="float"><minInclusive value="0.0"/><maxInclusive value="1.0"/></simpleType><complexType name="HighlightLevel"><element name="HighlightSegment" type="mds:HighlightSegment"maxOccurs="unbounded"/><element name="HighlightLevel" type="mds:HighlightLevel"minOccurs="0" maxOccurs="unbounded"/><attribute name="name" type="string" use="required"/><attribute name="level" type="integer" use="optional"/><attribute name="duration" type="mds:MediaDuration" use="optional"/><attribute name="numberOfKeyFrames" type="nonNegativeInteger" use="optional"/><attribute name="fidelity" type="mds:FidelityType" use="optional"/><attribute name="themeIds" type="IDREFS" use="optional"/></complexType><complexType name="HighlightSegment"><element name="VideoSegmentLocator" type="mds:VideoSegmentLocator"minOccurs="0"/>

</simpleType><element name="Partition" type="mds:Partition"/><complexType name="Partition"><element name="Start" type="mds:FloatVector"/><choice><element name="End" type="mds:FloatVector"/><element name="Extent" type="mds:FloatVector"/></choice><attribute name="dimensions" type="integer"/><attribute name="units" type="mds:PartitionUnits" use="default"value="fraction"/></complexType><element name="SpaceView" type="mds:SpaceView"/><complexType name="SpaceView" base="mds:View" derivedBy="extension"><choice minOccurs="0"><element name="SpacePartition" type="mds:Partition"/><element name="SpaceRegion" type="mds:RegionLocator"/></choice></complexType><element name="FrequencyView" type="mds:FrequencyView"/><complexType name="FrequencyView" base="mds:View" derivedBy="extension"><element name="FrequencyPartition" type="mds:Partition" minOccurs="0"/><element name="Filter" type="mds:Filter" minOccurs="0"/></complexType><element name="SpaceFrequencyView" type="mds:SpaceFrequencyView"/><complexType name="SpaceFrequencyView" base="mds:View" derivedBy="extension"><element name="FrequencyPartition" type="mds:Partition" minOccurs="0"/><element name="Filter" type="mds:Filter" minOccurs="0"/><choice minOccurs="0"><element name="SpacePartition" type="mds:Partition"/><element name="SpaceRegion" type="mds:RegionLocator"/></choice></complexType><element name="ResolutionView" type="mds:ResolutionView"/><complexType name="ResolutionView" base="mds:View" derivedBy="extension"><element name="ResolutionOfView" type="mds:FloatVector" minOccurs="0"/><element name="FrequencyPartition" type="mds:Partition" minOccurs="0"/><element name="Filter" type="mds:Filter" minOccurs="0"/></complexType><element name="SpaceResolutionView" type="mds:SpaceResolutionView"/><complexType name="SpaceResolutionView" base="mds:View" derivedBy="extension"><choice minOccurs="0"><element name="SpacePartition" type="mds:Partition"/><element name="SpaceRegion" type="mds:RegionLocator"/></choice><element name="SpaceResolutionOfView" type="mds:FloatVector" minOccurs="0"/>

<element name="ClusterRelationshipNode"type="mds:ClusterRelationshipNode"/></choice></complexType><complexType name="ClusterRelationship" base="mds:Relationship"derivedBy="extension"/><complexType name="StructureClusterRelationship" base="mds:ClusterRelationship"derivedBy="extension"/><complexType name="ClusterDecomposition" base="mds:StructureClusterRelationship"derivedBy="extension"><attribute name="type" use="fixed" value="Structure Topological"/><attribute name="name" use="fixed" value="ClusterDecomposition"/><attribute name="degree" use="fixed" value="2"/><attribute name="DecompositionType" type="string" use="optional"/><attribute name="overlaps" type="boolean" use="optional"/><attribute name="gaps" type="boolean" use="optional"/></complexType><complexType name="ClusterUnion" base="mds:StructureClusterRelationship"derivedBy="extension"><attribute name="type" use="fixed" value="Structure Topological"/><attribute name="name" use="fixed" value="ClusterUnion"/><attribute name="degree" use="fixed" value="2"/></complexType><complexType name="ClusterIntersection" base="mds:StructureClusterRelationship"derivedBy="extension"><attribute name="type" use="fixed" value="Structure Topological"/><attribute name="name" use="fixed" value="ClusterIntersection"/><attribute name="degree" use="fixed" value="2"/></complexType><complexType name="SemanticClusterRelationship" base="mds:ClusterRelationship"derivedBy="extension"/><complexType name="MediaClusterRelationship" base="mds:ClusterRelationship"derivedBy="extension"/><complexType name="MetaClusterRelationship" base="mds:ClusterRelationship"derivedBy="extension"/><complexType name="Model" abstract="true"/><complexType name="ProbabilityModel" base="mds:Model" derivedBy="extension"abstract="true"/><element name="DiscreteModel" type="mds:DiscreteModel"/><complexType name="DiscreteModel" base="mds:ProbabilityModel"derivedBy="extension"><element name="vector" type="mds:FloatVector"/></complexType><element name="MixtureModel" type="mds:MixtureModel"/><complexType name="MixtureModel" base="mds:ProbabilityModel"

derivedBy="extension"><element name="mixtures" type="mds:FloatVector"/><sequence minOccurs="1" maxOccurs="unbounded"><element name="Gaussian" type="mds:Gaussian"/></sequence></complexType><element name="State" type="mds:State"/><complexType name="State" base="mds:AnalyticModel"derivedBy="extension"><element name="ProbabilityModel"/></complexType><complexType name="StateTransitionModel" base="mds:ProbabilityModel"derivedBy="extension"><element name="initial" type="mds:FloatVector"/><element name="transitions" type="mds:FloatMatrix"/><sequence minOccurs="1" maxOccurs="unbounded"><element name="State" type="mds:State"/></sequence></complexType><element name="Gaussian" type="mds:Gaussian"/><complexType name="Gaussian" base="mds:ProbabilityModel" derivedBy="extension"><element name="Mean" type="mds:FloatVector"/><choice minOccurs="0"><element name="Variance" type="mds:FloatVector"/><element name="Covariance" type="mds:FloatMatrix"/></choice></complexType><element name="GeneralizedGaussian" type="mds:GeneralizedGaussian"/><complexType name="GeneralizedGaussian" base="mds:Gaussian"derivedBy="extension"><attribute name="shape" type="float"/></complexType><element name="HigherOrderGaussian" type="mds:HigherOrderGaussian"/><complexType name="HigherOrderGaussian"base="mds:GeneralizedGaussian" derivedBy="extension"><sequence minOccurs="0" maxOccurs="unbounded"><element name="moment" type="mds:FloatVector"/></sequence></complexType>

</complexType></element><element name="PreferredSummaryTheme" minOccurs="0" maxOccurs="unbounded"><complexType base="string" derivedBy="extension"><attribute ref="xml:lang"/><attribute name="preferenceValue" type="integer" use="optional"/></complexType></element><element name="SummaryDuration" type="mds:MediaDuration"minOccurs="0" maxOccurs="1"/><element name="MinSummaryDuration" type="mds:MediaDuration"minOccurs="0" maxOccurs="1"/><element name="MaxSummaryDuration" type="mds:MediaDuration"minOccurs="0" maxOccurs="1"/><element name="NumOfKeyframes" type="positiveInteger"minOccurs="0" maxOccurs="1"/><element name="MinNumOfKeyframes" type="positiveInteger"minOccurs="0" maxOccurs="1"/><element name="MaxNumOfKeyframes" type="positiveInteger"minOccurs="0" maxOccurs="1"/><attribute name="preferenceValue" type="integer" use="optional"/></complexType><element name="FilteringAndSearchPreferences"type="mds:FilteringAndSearchPreferences"/><complexType name="FilteringAndSearchPreferences"><element name="ClassificationPreferences" type="mds:ClassificationPreferences"minOccurs="0" maxOccurs="unbounded"/><element name="CreationPreferences" type="mds:CreationPreferences"minOccurs="0" maxOccurs="unbounded"/><element name="SourcePreferences" type="mds:SourcePreferences"minOccurs="0" maxOccurs="unbounded"/><element name="FilteringAndSearchPreferenceType" type="mds:PreferenceType"minOccurs="0" maxOccurs="unbounded"/><attribute name="protection" type="boolean" use="default" value="true"/><attribute name="preferenceValue" type="integer" use="optional"/></complexType><complexType name="ClassificationPreferences"><element name="Country" minOccurs="0" maxOccurs="unbounded"><complexType base="mds:ISO316601CountryCode" derivedBy="extension"><attribute name="preferenceValue" type="integer" use="optional"/></complexType></element><element name="Language" minOccurs="0" maxOccurs="unbounded"><complexType base="xsd:language" derivedBy="extension"><attribute name="preferenceValue" type="integer" use="optional"/></complexType></element><element name="Genre" minOccurs="0" maxOccurs="unbounded"><complexType base="mds:ControlledTerm" derivedBy="extension"><attribute name="preferenceValue" type="integer" use="optional"/></complexType></element><element name="PackagedType" minOccurs="0" maxOccurs="unbounded"><complexType base="mds:ControlledTerm" derivedBy="extension"><attribute name="preferenceValue" type="integer" use="optional"/>

</complexType></element><attribute name="preferenceValue" type="integer" use="optional"/></complexType><complexType name="CreationPreferences"><element name="Title" minOccurs="0" maxOccurs="unbounded"><complexType base="mds:Title" derivedBy="extension"><attribute name="preferenceValue" type="integer" use="optional"/></complexType></element><element name="Creator" minOccurs="0" maxOccurs="unbounded"><complexType base="mds:Creator" derivedBy="extension"><attribute name="preferenceValue" type="integer" use="optional"/></complexType></element><element name="Location" minOccurs="0" maxOccurs="unbounded"><complexType base="mds:Place" derivedBy="extension"><attribute name="preferenceValue" type="integer" use="optional"/></complexType></element><attribute name="preferenceValue" type="integer" use="optional"/></complexType><complexType name="SourcePreferences"><element name="PublicationType" minOccurs="0" maxOccurs="unbounded"><complexType base="mds:ControlledTerm" derivedBy="extension"><attribute name="preferenceValue" type="integer" use="optional"/></complexType></element><element name="PublicationChannel" minOccurs="0" maxOccurs="unbounded"><complexType base="mds:ControlledTerm" derivedBy="extension"><attribute name="preferenceValue" type="integer" use="optional"/></complexType></element><element name="PublicationPlace" minOccurs="0" maxOccurs="unbounded"><complexType base="mds:Place" derivedBy="extension"><attribute name="preferenceValue" type="integer" use="optional"/></complexType></element><element name="PublicationDate" minOccurs="0" maxOccurs="unbounded"><complexType base="mds:TimePoint" derivedBy="extension"><attribute name="preferenceValue" type="integer" use="optional"/></complexType></element><element name="Publisher" minOccurs="0" maxOccurs="unbounded"><complexType base="mds:Person" derivedBy="extension"><attribute name="preferenceValue" type="integer" use="optional"/></complexType></element><attribute name="preferenceValue" type="integer" use="optional"/></complexType></schema>

read.pudn.comread.pudn.com/downloads75/doc/279385/15938/15938-5.doc · web viewinternational...

Documents