international organisation for … · web viewthe number of objects in the originally proposed...

INTERNATIONAL ORGANISATION FOR STANDARDISATIONORGANISATION INTERNATIONALE DE NORMALISATION

ISO/IEC JTC 1/SC 29/WG 11CODING OF MOVING PICTURES AND AUDIO

ISO/IEC JTC 1/SC 29/WG 11 N3045October 1999

Source: Leonardo Chiariglione – ConvenorTitle: Report of 50th WG 11 meetingStatus:

1 OpeningThe 50th MPEG meeting was held in Maui, HI on 99/12/06-10 at the kind invitation of ANSI, the US National Body.

1.1 Opening AddressDr. Alan Bell, Director, Digital Media Strategy and Standards, IBM, representing the sponsoring companies, gave an opening address highlighting the use that his company made of some of the standards produced by MPEG in the multimedia area.

1.2 Welcoming remarksParticipants were welcomed by Mr. Peter Schirling, HoD, representing the hosting country.

1.3 Logistical remarks Ms. Julie Higgins briefed the delegates on the logistics of the meeting.

2 Roll call of participants The list of participants is given in annex 1.

3 Approval of agendaThe approved agenda is given in annex 2

4 Allocation of contributionsAnnex 3 gives the list of contributions submitted at the meeting.

5 Communications from Convenor There were no communications.

6 Report of previous meetingThe report of the Vancouver meeting was approved.

7 Processing of NB Position PapersNational Body contributions were received from Germany, France, Japan, Korea, The Netherlands, Singapore, UK and USA. These were processed and responses provided.

8 MPEG Phase 2

8.1 MPEG-2 partsNo activity

8.2 Verification of MPEG-2No activity

8.3 AmendmentsThe following documents related to MPEG-2 amendments were producedText of 13818-1 FDAM 7 N3050Rationale for 13818-2:2000 PDAM-1 N3061Text of 13818-2:2000 PDAM-1 N3062DoC on 13818-6 FPDAM 2 N3051Text of 13818-6 FDAM 2 N3052

8.4 CorrigendaNo activity

8.5 WorkplanThis was approved

9 MPEG Phase 4

9.1 Version 19.1.1 Final Draft International Standard

All 6 parts having been released there was no activity

9.1.2 Verification Tests

No activity

9.1.3 Quality of service

No activity

9.1.4 Conformance Testing

FDIS level was achieved at this meeting.

9.1.5 Corrigenda

No activity

9.2 Version 29.2.1 Patent statements

All patents statements were collected

9.2.2 Requirements

The activity was concluded

9.2.3 Tools


9.2.4 Verification Models


9.2.5 Amendment

2

Version 2 parts 1, 2, 3, 6 were achieved FDAM status at this meetingDoC on 14496-1 FPDAM 1 N3053Text of 14496-1 FDAM 1 N3054DoC on 14496-2 FPDAM 1 N3055Text of 14496-2 FDAM 1 N3056DoC on 14496-3 FPDAM 1 N3057Text of 14496-3 FDAM 1 N3058DoC on 14496-6 FPDAM 1 N3059Text of 14496-6 FDAM 1 N3060

9.2.6 Conformance Testing

PDAM was released at this meetingText of 14496-4 PDAM 1 N3065

9.2.7 Reference Software

No activity

9.2.8 Verification tests

The Audio verification tests were approved.

9.3 Other versionsVersion 3 achieved the first status

Rationale for 14496-1 PDAM 2 N3203Text of 14496-1 PDAM 2 N3204Rationale for 14496-2 PDAM 2 N3170Text of 14496-2 PDAM 2 N3146


10 MPEG Phase 7

10.1 RequirementsA new version of the requirements was produced

10.2 Tools10.2.1 Systems

The first working draft was produced

10.2.2 Description Definition Language


10.2.3 Visual


10.2.4 Audio


10.2.5 Description Schemes


10.3 Experimentation Model

3



11 Overall WG11 workplanThis was approved

12 ExplorationsThe MPEG-21 NP proposal was approved.

13 Liaison mattersInput documents were reviewed and liaison statements issued

14 Administrative matters

14.1 Schedule of future MPEG meetingsThis was approved

14.2 Promotion of MPEGThe press release and a new version of FAQs was approved

15 Organisation of this meeting

15.1 Tasks for subgroupsRequirements Review FAQ

Overview of the MPEG-4 standard MPEG-4 Version 2 profiles MPEG-4 Requirements (IPMP) Overview of the MPEG-7 standard MPEG-7 Requirements (IPMP) Subdivision of MPEG-7 work item Description of MPEG-7 process Multimedia Framework Press release

Systems Review FAQ DSM-CC Amendment 1 Contribution to Overview of the MPEG-4 standard Contribution to MPEG-4 v. 1 conformance MPEG-4 Version 2 FDIS (Systems & DMIF) MPEG-4 Version 2 patent statements OK? MPEG-4 version 3 (Systems & DMIF) Contribution to Overview of the MPEG-7 standard MPEG-7 WD Contribution to Press release

MDS Review FAQ Contribution to Overview of the MPEG-7 standard MPEG-7 WD Contribution to Press release

Video Progressive/interlace signalling

4

Encoder optimisation Contribution to Overview of the MPEG-4 standard Contribution to MPEG-4 v. 1 conformance MPEG-4 Version 2 FDIS MPEG-4 Version 2 patent statements OK? MPEG-4 version 3? Call for testing of video coding algorithms Contribution to Overview of the MPEG-7 standard MPEG-7 WD Contribution to Press release

Audio Review FAQ Contribution to Overview of the MPEG-4 standard Contribution to MPEG-4 v. 1 conformance MPEG-4 Version 2 FDIS MPEG-4 Version 2 patent statements OK? Call for testing of audio coding algorithms Contribution to Overview of the MPEG-7 standard MPEG-7 WD Contribution to Press release

SNHC Review FAQ Contribution to Overview of the MPEG-4 standard Contribution to MPEG-4 v. 1 conformance MPEG-4 Version 2 FDIS MPEG-4 Version 2 patent statements OK? MPEG-4 version 3? Contribution to Press release

ISG Contribution to Overview of the MPEG-7 standard MPEG-7 XM Contribution to Press release

Conformance Testing MPEG-4 v. 1 conformance FDIS MPEG-4 v. 2 conformance

Reference Software MPEG-4 v. 2 reference software CD Liaison JPEG

ITU-R W3C W3D

HoD MPEG-4 V. 2 patent statements Publication of part 5 on ISO web site ISO certificates MPEG-7 software policy Use of MPEG logo Conformance to MPEG standards Meeting schedule Support for Friday plenary Preparation for SC 29 meeting

15.2 Finalization of meeting allocationMeeting rooms were allocated

15.3 Joint meetingsThe following joint meetings were held

5

Group 1 Group 2 What Where WhenMDS ISG SW integration in XM MDS Tue 9-10SNHC Systems Ver. 2 integration - Surface

nodeSystems Tue 14-16

SNHC Requirements SNHC profiles Requirements Mon 17-18Systems MDS Video DDL, DS MDS Tue 14-15ISG Systems,Req IPMP Requirements Tue 16-17Audio MDS M5434 MDS Tue 16:30Reference SW

Systems Tue 8:30-9

Conformance Systems Wed 11:30-13MM Framework

Requirements Tue all dayThu pm

16 Planning of future activitiesThe following ad hoc groups were establishedNo. Title

3163 AHG for Study on potential new Progressive Profile and Levels3086 AHG on Audio part of MPEG-4 Version 1 FDIS and Version 2 FPDAM Conformance3143 AHG on complexity evaluations of video tools3137 AHG on Conceptual Modelling 3102 AHG on conformance in MPEG-4 video3108 AHG on core experiments for Color/Texture descriptors in MPEG-7 3109 AHG on core experiments for Shape/Motion descriptors in MPEG-73136 AHG on DDL integration of MPEG-7 Ds and DSs3110 AHG on editing the document of the MPEG-7 Visual part of XM and WD3105 AHG on editing the documents of the MPEG-4 Visual FDAM and the MPEG-4 video

verification model 3133 AHG on Editing the Multimedia Description Schemes XM and WD3149 AHG on FBA3103 AHG on Fine Granularity Scalability in MPEG-4 video3213 AHG on IM13107 AHG on MPEG Video FAQ3085 AHG on MPEG-4 Audio Reference Software Editing3087 AHG on MPEG-4 Audio Version 2 FDAM Editing 3210 AHG on MPEG-4 BIFS3209 AHG on MPEG-4 Conformance3214 AHG on MPEG-4 Content on MPEG-2 Systems and IP Network3211 AHG on MPEG-4 Textual Format3169 AHG on MPEG-7 Applications Study3088 AHG on MPEG-7 Audio Core Experiments 3218 AHG on MPEG-7 DDL3135 AHG on MPEG-7 DS Validation and Core Experiments 3166 AHG on MPEG-7 Intellectual Property Management & Protection3217 AHG on MPEG-7 Linking3134 AHG on MPEG-7 Media/Meta DSs and Harmonization with Other Schemes3138 AHG on MPEG-7 Semantic information representation

6

No. Title3216 AHG on MPEG-7 Systems3168 AHG on MPEG-7 Verification Experiments3212 AHG on MPEG-J3167 AHG on Multimedia Framework3215 AHG on multi-user applications3111 AHG on organizing the software integration of MPEG-7 Visual part of XM tools3148 AHG on SNHC conformance3101 AHG on software integration and verification in MPEG-4 video3150 AHG on Study of 3D Animation Coding3172 AHG on Study of ISO/IEC 14496-2/PDAM 23106 AHG on study of the publication of Technical Report on non-normative aspects of

MPEG-4 Video.3165 AHG on Study on standard Digital Watermarking technology3164 AHG on Study on Standard IPMP Systems3104 AHG on the Studio Profile in MPEG-4 video3100 AHG on the study of MPEG-2 Video production processes for supplemental information 3139 AHG on User Preferences 3144 AHG on XM Development3153 Applications, Operational Environments and Requirements for new Progressive Profiles

and Levels3072 Audio Contribution to MPEG-7 WD3084 Audio FAQ Update3070 Call for Information on the operation of the MPEG Web and FTP site3157 Call for Proposals for an MPEG-4 Textual Format3098 Call for testing of video coding algorithms3140 Case study of tools for potential inclusion in a technical report on non-normative tools in

MPEG-43122 Core Experiment on Collection Structure DS3131 Core Experiment on Measuring Entropy for Descriptors and Description Schemes3130 Core Experiment on Media Transcoding Hint DS3125 Core Experiment on Mosaic DS3127 Core Experiment on Region Localization3124 Core Experiment on Space and Frequency View DS3129 Core Experiment on the Media Title DS 3119 Core Experiment on the Probability Model DSs3123 Core Experiment on the Semantic DS3121 Core Experiment on the Summary DS3120 Core Experiment on the Weight DS 3128 Core Experiment on User Preferences3092 Description of Changes from ISO/IEC 14496-2/ FPDAM1 to ISO/IEC 14496-2/FDAM1 3099 Description of Core Experiments for MPEG-7 Shape/Motion descriptors3090 Description of Core Experiments for MPEG-7 Colour/Texture descriptors3096 Description of Core Experiments in FGS3194 Disposition of comments on ISO/IEC 14498-1/FPDAM 13196 Disposition of comments on ISO/IEC 14498-6/FPDAM 13202 DMIF Roadmap3051 DoC of ISO/IEC 13818-6/FPDAM 2

7

No. Title3053 DoC of ISO/IEC 14496-1/FPDAM 13055 DoC of ISO/IEC 14496-2/FPDAM 13057 DoC of ISO/IEC 14496-3/FPDAM 13059 DoC of ISO/IEC 14496-6/FPDAM 13066 DoC of ISO/IEC FCD 14496-43197 Draft of MPEG-J Reference Explanatory Report3174 Establishing Liaison with OCLC3141 Experimental conditions for evaluating encoder motion estimation algorithms.3193 FDAM 1 of 14496-1 (MPEG-4 Systems Version 2)3195 FDAM 1 of 14496-6 (MPEG-4 DMIF Version 2)3162 First Ideas on defining a Multimedia Framework, V0.23078 Framework for MPEG-7 Audio Descriptors3208 Frequently Asked Questions on Systems Revision 12.03049 Guide to WG 11 Meeting Hosts3046 Guidelines for Electronic Distribution of MPEG and WG 11 Documents3076 Information on Use of MPEG-4 Low-overhead Audio Transport Multiplex (LATM)3089 Justification for MPEG-2 Video Elementary Stream supplemental information3182 Liaison Statement to ITU-T SG16 Regarding Video Coding Activities and IDCT

Specification3184 Liaison Statement to RIAA3175 Liaison to ATSC on Data Broadcasting3176 Liaison to ATSC on ES Supplementary Information3180 Liaison to DVB3181 Liaison to DVB3177 Liaison to DVD Consortium3178 Liaison to EBU3183 Liaison to IEC TC 1003220 Liaison to International Olympic Committee3173 Liaison to ITU-R JWP 10-11Q3185 Liaison to ITU-T SG16/Q.11, 14 on MPEG-4/Audio3179 Liaison to ProMPEG3186 Liaison to Web3D on Convergence with MPEG-43044 List of AHGs Established at the 50th Meeting in Maui, Hawaii(US)3042 List of Documents from Maui, Hawaii(US) Meeting3117 Mathematical Models for Description Schemes Concepts3048 Meeting Notice of the 51st Meeting in Noordwijkerhout, The Netherlands3151 Meeting Report on WG 1 and WG 11 collaboration on Motion JPEG20003207 MPEG Systems Contribution to the joint WG1/WG11 activity on Motion JPEG 20003155 MPEG-4 New Profiles under Consideration3156 MPEG-4 Overview3154 MPEG-4 Requirements Document, V.133198 MPEG-4 Systems VM 9.0 3064 MPEG-4 Version 2 Profile Support and Conformance Commitments3093 MPEG-4 Video VM 15.0 3132 MPEG-7 Application Experiments3077 MPEG-7 Audio Core Experiment Methodology3201 MPEG-7 DDL WD 1.0

8

No. Title3159 MPEG-7 Development Process3160 MPEG-7 Intellectual Property Management and Protection, V.0.23113 MPEG-7 Multimedia Description Schemes WD3112 MPEG-7 Multimedia Description Schemes XM3158 MPEG-7 Overview v.13116 MPEG-7 Principal Concept List 3115 MPEG-7 Software integration policy & Workplan for DS integration3199 MPEG-7 Systems WD 0.13068 MPEG-7 visual part of XM 4.03118 Multimedia Description Schemes FAQs3047 Press Release of the 50th Meeting3073 Proposed Corrections to 14496-3, MPEG-4 Version 1 Audio 3045 Report of the 50th Meeting in Maui, Hawaii(US)3075 Report on MPEG-4 Version 2 Audio Verification Test3170 Request for Amendment 2 to ISO/IEC 14496-23203 Request for Amendment for 14496-1 Amendment 23206 Request for Amendment for 14496-1 Amendment 33061 Request for New Amendment: ISO/IEC 13818-2:2000/Amd 13200 Request for NP: MPEG-213063 Request for Subdivision: ISO/IEC 15938 (MPEG-7)3205 Requirements for Multi-users Worlds3189 Requirements for the Registration Authority for MPEG-4 IPMP3043 Resolutions of the Maui, Hawaii(US) Meeting3187 Response to NB comments3142 Schedule for the software implementation of XM visual tools.3147 SNHC Work Plan on Conformance3071 Status and Workplan of MPEG-4 Version 2 Audio Reference Software3079 Status of MPEG-7 Audio Core Experiments3191 Status of the Systems Version 1&2 Software Implementation 3188 Study of Corrigendum for 14496-13190 Study of Corrigendum for 14496-63091 Study of ISO/IEC 14496-2/DCOR13145 Study of ISO/IEC 14496-5/FDIS (Video part)3219 Study text of ITU-T Rec. H.222.0 | ISO/IEC 13818-1:20003171 Study text of ITU-T Rec. H.262 | ISO/IEC 13818-2:20003114 Supporting information for MPEG-7 Description Schemes3152 Systems Conformance Bitstream Production Workplan3192 Systems Software Implementation Work plan 3204 Text for ISO/IEC 14496-1 Amendment 2 PDAM3050 Text of ISO/IEC 13818-1/FDAM 73062 Text of ISO/IEC 13818-2:2000/PDAM 13052 Text of ISO/IEC 13818-6/FDAM 23054 Text of ISO/IEC 14496-1/FDAM 13095 Text of ISO/IEC 14496-2 FGS Amendment WD 3.03097 Text of ISO/IEC 14496-2 MPEG-4 Video FGS VM v 3.03094 Text of ISO/IEC 14496-2 Studio Profile Amendment WD 3.03056 Text of ISO/IEC 14496-2/FDAM 1

9

No. Title3146 Text of ISO/IEC 14496-2/PDAM 23058 Text of ISO/IEC 14496-3/FDAM 13065 Text of ISO/IEC 14496-4/PDAM 13060 Text of ISO/IEC 14496-6/FDAM 13067 Text of ISO/IEC FDIS 14496-43126 Validation Experiment on Segment Hint DS3161 Verifying MPEG-7 using real-world applications3069 Working Draft of MPEG-7 visual descriptors3074 Workplan for MPEG-4 Audio Conformance3083 Workplan for MPEG-7 Music Genre Core Experiment3082 Workplan for MPEG-7 Sound Effects Core Experiment3080 Workplan for MPEG-7 Speech Recognition Core Experiment3081 Workplan for MPEG-7 Timbre Similarity Core Experiment

17 Resolutions of this meeting These were approved

18 A.O.BThere were none

19 ClosingThe meeting was closed at 21:50 with thanks to the hosting organisation.

10

Annex 1Attendance list

COUN. L. NAME F. NAME COMPANYAU Barrass Stephen CSIRO AU Frater Michael University of New South WalesAU Leditschke Matthew Motorola Australian Research CentreAU Pickering Mark University of New South WalesAU Srinivasan Uma CSIROAU Wan Ernest CISRAAU Ye Lei Motorola AustraliaBE LaFruit Gaulthier IMECBE Lindsay Adam Starlab NVBE Saraceno Caterina Starlab NV/SACA Haighton Peter Telexis CorporationCN Au Oscar Hong Kong UniversityCN Feng Wu MicrosoftCN Gao Wen Inst. of Comp. Tech. Chinese Academy of SciencesCN Li Shipeng Microsoft Research ChinaFI Castagno Roberto Nokia Research CenterFI Huopaniemi Jyri Nokia Research CenterFI Vaananen Riitta Helsinki University of TechnologyFI Väänänen Mauri Nokia Research CenterFI Varsa Viktor Nokia Research CenterFR Curet Dominique France TelecomFR Devillers Sylvain Philips LaboratoriesFR Didier Mary FIAPFFR Dufour Cecile PhilipsFR Dufourd Jean-Claude ENSTFR Guez Marc SCPPFR Guillotel Philippe Thomson Multimedia R&D FR Herera Perfecto IRCAMFR Herrmann Laurent Philips LEPFR Hoogvorst Philippe ENSTFR Labelle Lilian Canon Research CentreFR Le Guyader Alain France TelecomFR Morlet Jean-Noel France TelecomFR Mory Benoit Laboratories d’ Electronique PhilipsFR Nicholson Didier Thomson-CSF CommunicationsFR Peeters Geoffroy IRCAMFR Pretaux Francoise INTFR Seyrat Claude University of ParisFR Signes Julien France TelecomFR Thienot Cedric University of ParisFR Yon Dominique CISACDE Avaro Olivier Deutsche Telekom DE Bauer Sven Robert Bosch GmbHDE Brandenburg Karlheinz Fraunhofer IIS

11

COUN. L. NAME F. NAME COMPANYDE Diepold Klaus International Digital TechnologiesDE Feige Frank Deutsche Telekom BerkomDE Grill Bernhard Fraunhofer IISDE Heising Guido Heinrich Hertz InstituteDE Hermann Stephen Technical University Of MunichDE Herpel Carsten Deutsche Thomson-Brandt GmbHDE Herre Jürgen Fraunhofer IISDE Heuer Joerg Siemens AGDE Homm Daniel Fraunhofer IIS DE Hutter Andreas Siemens AGDE Kohler Joachim GMD- IMKDE Mlasko Torsten Robert Bosch GmbHDE Ohm Jens-Rainer Heinrich Hertz InstituteDE Purnhagen Heiko University of HannoverDE Sebestyen Istvan SiemensDE Smolic Aljoscha Heinrich Hertz InstituteDE Sperschneider Ralph FhG-IISDE Spille Jens Deutsche Thomson-Brandt GmbHIL Cohen Noam Vsoft, Ltd.IL Gill Aharon Zoran Microelectronics Ltd.IL Lifshitz Zvi Optibase Ltd.IL Peterfreund Natan Harmonic LightwavesIT Chiariglione Leonardo CSELTIT DePetris Gianluca CSELTIT Flaiani Roberto Aethra TelecomunicazioniIT Franceschini Guido CSELTIT Gandini Marco CSELTIT Riccardo Leonardi University of BresciaJP Aoki Terumasa Tokyo UniversityJP Asai Kohtaro Mitsubishi ElectricJP Azami Tomohiro JVCJP Bossen Frank SonyJP Day Neil Ricoh Company, Ltd.JP Emura Koichi Matsushita Electric IndustrialJP Fujita Takehiro Hitachi, LtdJP Gonno Yoshihisa IB LabsJP Hasida Koiti Information Science DivisionJP Hirose Narumi IPSJ/ITSCJJP Hori_ Osamu ToshibaJP Hotani Sanae NTT DoCoMoJP Inoue Akia Sony CorporationJP Inoue Hiroshi Canon Inc.JP Ishii Katsuki Canon Inc.JP Ito Hiroshi Mitsubishi Electric JP Ito_ Norio Sharp CorporationJP Iwadare Masahiro NECJP Iwasaki Masajiro Ricoh Company, Ltd.

12

COUN. L. NAME F. NAME COMPANYJP Kaneko Itaru ASCII CorporationJP Kaneko Toshimitsu Toshiba CorporationJP Kawahara Toshiro NTT DoCoMoJP Kazui Kimihiko FujitsuJP Kishigami Junichi NTTJP Kogure_ Takuyo Matsushita Electric Industrial JP Kouya Tochikubo Toshiba CorporationJP Koyama Hitoshi NEC JP Kuhn_ Peter Sony JPJP Kunieda Takayuki Ricoh Company, Ltd.JP Masaru Sugano KDD R&D Laboratories, Inc.JP Miki Toshio NTT DoCoMoJP Miyamoto Yoshihiro NECJP Morimatsu Eishi Fujitsu LaboratoriesJP Moriya_ Takehiro NTTJP Murakami Tokumichi Mitsubishi Electric JP Nakashiba Takahiko Tokyo UniversityJP Nakata Mayumi University of TokyoJP Nakaya_ Yuichiro Hitachi, LtdJP Nishiguchi Masayuki Sony CorporationJP Nishikawa Hirofumi Mitsubishi Electric JP Nomura Toshiyuki NEC JP Sakamoto Hideaki NTTJP Sasaki Mikio Desno CorporationJP Sekiguchi Shun-ichi Mitsubishi Electric JP Senoh Takanori Matsushita Electric Industrial JP Shibata Masahiro NHKJP Shibata Yoshiaki Sony CorporationJP Shigeru Fukunaga Oki Electric Industry, Co. Ltd.JP Shiiyama Hirotaka Canon Inc.JP Shishikui Yoshiaki NHKJP Sugihara Yoshinori Electronic Industries Association of JPJP Sugiura Maki TAOJP Sunohara Osamu Sony CorporationJP Suzuki Teruhiko Sony CorporationJP Suzuki Yoshinori Hitachi, LtdJP Tahara Katsumi SonyJP Takahashi Katsunao TAOJP Takamura Seishi NTTJP Takashima Yoichi NTTJP Takeda Junichi Canon Inc.JP Toguri Yasuhiro Sony CorporationJP Tokumo Yasuaki Sharp CorporationJP Tomioka Yoshiaki Sharp CorporationJP Ueda Shigeru Sharp CorporationJP Ueda Yoshihiro OKI Electric Industry, Co. Ltd.JP Walker Toby Sony

13

COUN. L. NAME F. NAME COMPANYJP Watanabe_ Hiroshi NTTJP Watanabe_ Shuichi Sharp CorporationJP Yagasaki Yoichi Sony CorporationJP Yamada Akio NEC JP Yasuda Hiroshi University of TokyoJP Yasuyuki Ooki Hitachi, LtdKR Han Mahnjin Samsung AITKR Jang Euee S. Samsung AITKR Kim Do Kyon Samsung AITKR Kim Heon Jun LG Corporate Institute of TechnologyKR Kim Jin Woon ETRIKR Kim Kyu Heon ETRIKR Kim Munchurl ETRIKR Kim Sang-Wook Samsung KR Kim Sungham ETRIKR Kim Sungham ETRIKR Kim Whoi Yul Hanyang UniversityKR Kim YeonBae Samsung AITKR Lee Han-Kyu ETRIKR Lee Jin-Soo LGCITKR Lee Yung-Lyul Samsung ElectronicsKR Lim Young Kwon ETRIKR Oh Seoung Jun Kwangwoon UniversityKR Park Soo-Jun ETRIKR Park Sung Hee Samsung AITKR Shin Hyundoo SamsungKR Shin Jae-Seob Samsung AITKR Sim Dong Gyu HyundaiKR Son Se-Hoon Samsung AITKR Sull Sanghoon KR UniversityKR Won CheeSun Dongguk UniversityKR Yang Jin Young ETRIKR Yoon Kyoungro LG Corporate Institute of TechnologyNO Pedersen Lena Telenor R & DSG Lim Chong Soon Panasonic SG LaboratoriesSG Lin Weisi Institute of MicroelectronicsSG Ma Kai-Kuang Nanyang Technological UniversitySG Powell Gerard Infineon Technologies Asia/PacificSG Tan Thiow-Keng Panasonic SG LaboratoriesSG Wang Lei Panasonic SG LaboratoriesSG Yamada Yasuhiro JVCES Ferreras Antonio Telefonica, I+DES Martinez Jose Universidad Politecnica de MadridES Salembier Philippe Universitat Politecnica de CatalunyaSE Christopoulos Charilaos Ericsson Radio SystemsSE Cognell Anna Telia ResearchSE Lindquist Morgan Ericsson Research

14

COUN. L. NAME F. NAME COMPANYCH Abdeljaoued Yousri EPFLCH Buttar Alistair MotorolaCH Capin Tolga EPFLCH Ebrahimi Touradj EPFLCH Erne Markus Swiss Federal Institute of TechnologyCH Ledermann Daniel SwisscomCH Mattavelli Marco EPFLCH Schroeter Philippe SwisscomCH Vergnenegre Fabrice EPFLNL Depovere Geert Philips ResearchNL Funken Ralf Philips Consumer ElectronicsNL Gelissen Jean Philips ResearchNL Koenen Rob KPN ResearchNL Mooij Wim MindportNL Mulder Peter NOBNL Van der Meer Jan Philips ElectronicsNL Van Der Schaar Mihaela Philips ResearchNL Velthausz Daan Telematica InstituutUK Bangham James University of East AngliaUK Bober Miroslaw Mitsubishi ElectricUK Cieplinski Leszek Mitsubishi ElectricUK Cox Michael Mirador Techniques, Ltd.UK Garner Phil Canon Research Centre Europe, Ltd.UK Grant Catherine Nine TilesUK Hill Keith MCOS LTDUK Koudoumakis Panos CRLUK Lacy Jack InterTrustUK Rump Niels InterTrustUS Abdel-Mottaleb Mohamed Philips ElectronicsUS Addington Tim Scientific AtlantaUS Apostolopoulos John HP LabsUS Belknap William IBMUS Benitez Ana Columbia UniversityUS Birkmaier Craig P-CubeUS Burns Ron Hughes ElectronicsUS Buxton Mark Intel CorporationUS Casey Michael Mitsubishi Electronic ResearchUS Chang Wo NISTUS Chau Kwok NJR CorporationUS Chen Xuemin General Instrument US Chen Yen-Kuang Intel CorporationUS Cherry Guy Tektronix, Inc.US Chiang Tihao NCTUUS Cornog Katie Avid Technology, Inc.US Crinon Regis Sharp LabsUS Divakaran Ajay Mitsubishi Electric ITAUS Dolson Mark E-mu Systems

15

COUN. L. NAME F. NAME COMPANYUS Eleftheriadis Alexandros Columbia UniversityUS Fang Jianjun Motorola, Inc.US Fleming Jim RIAAUS Foshee Scott AdobeUS Goldman Christopher busybox.comUS Goldman_ Matthew DiviComUS Gove Bob Micron TechnologyUS Gray Patricia Intel CorporationUS Haskell Barry AT&T Labs US Hoddie Peter Apple ComputerUS Horne Caspar Mediamatics, Inc.US Huang Si Jun Scientific AtlantaUS Izzat Izzat Thomson Consumer ElectronicsUS Jabbi Amandeep Micron TechnologyUS Jeannin Sylvie Philips ResearchUS Jiang Hong Intel CorporationUS Jorgensen Corinne University at Buffalo, State University of New YorkUS Kar_ Mukta Cable Television LaboratoriesUS Kharrati Hamid Doctor Design Inc.US Kim Michelle IBMUS Kim Youngmoo MIT Media LabUS Kneib Kristine KNK-MPEG.comUS Kolarov Krasimir Interval Research CorporationUS Lee Ming OracleUS Li Weiping Optivision, Inc.US Li Xiaoming 3Dfx Interactive, Inc.US Liang Kyminh Sorenson Vision Inc.US Lin Tao NeoMagic CorporationUS Lu Jian Apple Computer US Luthra Ajay General InstrumentUS Manjunath B.S. University of California at Santa BarbaraUS Maurer Thomas Eyematic InterfacesUS McVeigh Jeff Intel CorporationUS Mead Don Digital Electronics Cinema, Inc.US Mehrotra Rajiv Eastman KodakUS Melnikov Gerry Motorola, Inc.US Moccagatta Iole Rockwell Science CenterUS Morgan Oliver Avid Technology, Inc.US Narasimhan Sam General InstrumentUS Neven Hartmut Eyematic InterfacesUS Ostermann Joern ATTUS Owens James Diamond Back SystemsUS Peach Allan DemoGraFXUS Petajan Eric Lucent TechnologiesUS Poncelon Dulce IBMUS Quackenbush Schuyler AT & TUS Rafey Rick Web 3D Consortium

16

COUN. L. NAME F. NAME COMPANYUS Rehm Eric singingfish.comUS Rising Hawley Sony ElectronicsUS Rose Steve Video Network, Inc.US Rountree Janet SAICUS Rubinfeld Mike NISTUS Rynderman_ Michel Avid Technology, Inc.US Schirling Peter IBMUS Schmidt Robert AT&TUS Severson Bill Diamond Back SystemsUS Sezan Ibrahim Sharp Labs of AmericaUS Singer David Apple ComputerUS Singhal Manish Motorola, Inc.US Smith John IBM Watson Research CenterUS Sodagar Iraj Sarnoff CorporationUS Sullivan Gary Microsoft US Sun Huifang MitsubishiUS Swaminathan Viswanathan Sun MicrosystemsUS Tabatabai Ali Media Soft Labs-Sony ElectronicsUS Taubin Gabriel IBMUS Tekalp Murat University of RochesterUS Tescher Andrew Lockheed-MartinUS Thom David Mitsubishi Electronics America US Urban Stephen Delta Information SystemsUS van Beek Peter Sharp Labs of AmericaUS Westerink Peter IBMUS White Tom MMAUS Winder Simon MicrosoftUS Yang Chao-Kung Hughes ElectronicsUS Zamora Javier XBIND, INC.

17

Annex 2Agenda

1. Opening1.1 Opening Address 1.2 Welcoming remarks 1.3 Logistical remarks 2. Roll call of participants 3. Approval of agenda4. Allocation of contributions 5. Communications from Convenor 6. Report of previous meeting 7. Processing of NB Position Papers 8 MPEG Phase 28.1 MPEG-2 parts8.2 Verification of MPEG-28.3 Amendments8.3.1 Video8.3.2 DSM-CC8.4 Corrigenda8.5 Workplan9. MPEG Phase 49.1 Version 19.1.1 Final Draft International Standard9.1.1.1 Reference software 9.1.2 Verification Tests9.1.2.1 Systems9.1.2.2 Video9.1.2.3 Audio9.1.3 Quality of service9.1.4 Conformance Testing9.1.4.1 System9.1.4.2 Visual9.1.4.3 Audio9.1.4.4 DMIF9.2 Version 29.2.1 Patent statements9.2.2 Requirements9.2.3 Tools9.2.3.1 DMIF9.2.3.2 Systems9.2.3.3 Natural Visual9.2.3.4 Synthetic Visual9.2.3.5 Natural Audio

18

9.2.3.6 Synthetic Audio9.2.4 Verification Models9.2.4.1 System9.2.4.2 Video9.2.4.3 Audio9.2.4.4 SNHC9.2.3 Conformance Testing9.2.3.1 System9.2.3.2 Visual9.2.3.3 Audio9.2.3.4 DMIF9.2.4 Amendment9.2.4.1 Systems9.2.4.2 Visual9.2.4.3 Audio9.2.4.4 Reference Software9.2.4.5 DMIF9.2.5 Verification tests9.2.5.1 Systems9.2.5.2 Video9.2.5.3 Audio9.3 Other versions9.4 Workplan10. MPEG Phase 710.1 Requirements10.2 Tools10.2.1 Description Definition Language10.2.2 Description Schemes10.2.3 Descriptors10.2.3.1 Visual10.2.3.2 Audio10.3 Systems10.4 Experimentation Model10.4 Workplan11. Overall WG11 workplan 12. Explorations13. Liaison matters14. Administrative matters 14.1 Schedule of future MPEG meetings 14.2 Promotion of MPEG15. Organisation of this meeting 15.1 Tasks for subgroups15.2 Finalisation of meeting allocation15.3 Joint meetings16. Planning of future activities

19

17. Resolutions of this meeting 18. A.O.B19. Closing

20

Annex 3Documents submitted

No. Source Title5283 Pete Schirling Document Register for 50th Meeting in Maui,

Hawaii (US)5284 Olivier Avaro, Regis Crinon,

Guido Franceschini, Carsten Herpel, Young-Kwon Lim, Niels Rump, David Singer, Vishy Swaminathan

Systems October 1999 (Melbourne) Meeting Report

5285 Niels Rump, Hiroshi Inoue, Yen-Kuang "Y.-K." Chen

Report of the MPEG-4 Ad-hoc Group

5286 Niels Rump, Keith Hill Report on the MPEG-7 IPMP Ad-hoc Group5287 AHG on MPEG-4 Audio V2 Study

on FPDAM EditingStudy on MPEG-4 Version 2 Audio FPDAM

5288 Osamu Sunohara, Yoichi Yagasaki The reference software of MPEG-4 Studio Profile Amendment WD2.0

5289 SC 29 Secretariat Summary of Voting: ISO/IEC 13818-6/FPDAM 25290 SC 29 Secretariat Summary of Voting: ISO/IEC 14496-1/FPDAM 15291 SC 29 Secretariat Summary of Voting: ISO/IEC 14496-2/FPDAM 15292 SC 29 Secretariat Summary of Voting: ISO/IEC 14496-3/FPDAM 15293 SC 29 Secretariat Summary of Voting: ISO/IEC 14496-6/FPDAM 15294 SC 29 Secretariat Summary of Voting: ISO/IEC FCD 14496-45295 ISO/IEC ITTFUS Summary of Voting: ISO/IEC FDIS 14496-15296 ISO/IEC ITTFUS Summary of Voting: ISO/IEC FDIS 14496-35297 SC 32US NP on Technical Report on the Procedures for

Achieving Content Consistency in ISO/IEC 11179 Metadata Registries (SC29N3310)

5298 SC 32US NP on MetaData Query Service - An Object Technology Extension to the ISO/IEC 11179 Specification (SC29N3311)

5299 IEC TC 100US IEC TC 100: CD 61998: Model and Framework for Standardization in Multimedia Equipment and Systems (SC29N3312)

5300 IEC TC 100US IEC TC 100: IEC CD PT61997: Guidelines for the User Interface in Multimedia equipment for general purpose use (SC29N3313)

5301 Audio Conformance AHG Study on the audio part of MPEG-4 Version 1 Conformance

5302 ITU-T SG 16 Q.11 and Q.14 Liaison Statement from ITU-T SG 16 Q.11 and Q.14 on a Generic Capability Parameter Addition for H.245 MPEG-4 Audio (SC29N3321)

5303 Ioana M. Martin, Gabriel Taubin Release Notes Update for 3D Mesh Coding Integration into The Pact 3D Player

5304 Peter Westerink, Aizaz Akhtar, Patrice Bazerque, Michelle Kim

IM1 Implementation of the Advanced Synchronization Model

21

5305 The SC 29 Secretariat Request for rationale of the proposed subdivision of MPEG-7 work item

5306 Ralph Sperschneider FhG Comments on Study on MPEG-4 Audio Version 2 FPDAM

5307 Michelle Kim, Peter Westerink, William Belknap

Advanced Synchronization Model (FlexTime Model)

5308 Mikio Sasaki, Katsushi Asami A Proposal for In-Vehicle Profiles5309 Mikio Sasaki, Katsushi Asami Interactive Agent using In-Vehicle Profiles5310 Mikio Sasaki Business Model of Interactive Media

Communication5311 Visual FDAM Editors Preliminary Version of ISO/IEC 14496-2 FDAM15312 Jane Hunter,, Robert Douglass Report of the Ad Hoc Group on the MPEG-7 DDL5313 Gauthier Lafruit Tutorial example of MPEG-4 backchannel5314 Panos Kudumakis for the UK

National BodyUK National Body position on 'advanced IPMP descriptors' and on standardising IPMP systems

5315 Niels Rump, Dominique Yon Input on the Registration Authority for MPEG-4 IPMP

5316 Stephan Herrmann Ad-hoc Group Report: AHG on organizing the software integration of MPEG-7 Visual XM Tools

5317 Stephan Herrmann Report of AHG on XM Development5318 Daniel Ledermann, Denis Schlauss,

Erich Dellenbach, Peter L³thi, John Riordan

Subjective evaluation of low bitrate video codecs

5319 Michelle Kim, Patrice Bazerque, Peter Westerink, Steve Wood

Augmenting the TemporalForm node for Intra-Object Synchronization

5320 Sunil Hadap, Laurent Moccozet, Tom Molet, Igor Pandzic

Body Animation Tables for Hand Deformations

5321 Alexis M. Tourapis, Oscar C. Au, Ming L. Liou, Guobin Shen

Hardware/software complexity issues between two fast motion estimation algorithms: the Advanced Diamond Zonal Search with Embedded Radar (ADZS-ER) and Diamond S

5322 L.Wang,, T.K.Tan A new proposal for homogeneous texture descriptor5323 A.P. Tan,, Takonori Senoh MPEG-4 Platform Verification Bitstream

Development Project5324 C.S. Lim,, T.K. Tan FGS Residual Representation and Prediction5325 T.K. Tan Report of the Ad-hoc group on conformance in

MPEG-4 video5326 IL National Body Comments on ISO/IEC 14496-1/FPDAM15327 Zvi Lifshitz Im1Player _ A Bitstream Compliance Verification

Tool5328 Junichi Kishigami, Shigeru Ueda,

Susumu Fujiwara, Hiroshi Yasuda A proposal to MPEG21, "the Big Picture" (Part 1) --- Purpose of the Content ID and a Comment to the MPEG P

5329 Youichi Takashima, Shinsuke Honjo, Junichi Kishigami, Hiroshi Yasuda

A proposal to MPEG21, "the Big Picture" (Part 2) --- Technologies for CID ---

5330 Junichi Kishigami, Hirokazu A proposal to MPEG21, "the Big Picture" (Part 3)

22

Shimizu, Hiroshi Yasuda --- Requirements on Content ID from Digital Archives' Viewpoint ---

5331 Ralph Sperschneider Workplan for MPEG-4 Audio Version 2 Verification Test

5332 Ralph Sperschneider AHG Report on MPEG-4 Version 2 Audio Verification Tests

5333 Patricia Gray, Jeff McVeigh Recommended modifications to W2930 MPEG-2 working draft amendment

5334 Ioana M. Martin, Gabriel Taubin Software Updates of PACT 3D Player with Support for Hierarchical 3D Meshes

5335 I. Sezan and K. Yoon (On behalf of AHG on User Descriptions)

User Descriptions in MPEG-7

5336 I. Sezan and K. Yoon (Chairmen) Report of AHG on User Descriptions in MPEG-75337 Jean-Claude Dufourd WD 4.1 of ISO/IEC 14496-4 Amendment 15338 Jean-Claude Dufourd Report of AHG on MPEG-4 Systems Conformance5339 Marc Brelot, Jean-Claude Dufourd New BIFS Sensors for V3: refinement and

architecture5340 Marc Brelot, Jean-Claude Dufourd Ideas for other applications and BIFS sensors for

V35341 Jean-Claude Dufourd, Souhila

Boughoufalah, Frederic Bouilhaguet

Update of ENST/"MPEG-4 PC" tools for content creation

5342 Philippe Hoogvorst, Jean-Claude Dufourd

Request for inclusion of JPEG in visual profiles

5343 John R. Smith on behalf of MPEG-7 Conceptual Modeling AHG

MPEG-7 Description Schemes Principle Concept List (V0.4)

5344 John R. Smith, Hawley K. Rising III, Uma Srinivasan

Report of the Ad-hoc Group on a Generic MPEG-7 DS Conceptual Model

5345 Santhana Krishnamachari, Mohamed Abdel-Mottaleb

Results of core experiment binary color histograms (CT6)

5346 Rosa Ruiloba, Philippe Joly, Cedric Thienot, Claude Seyrat

An Analytical Description Scheme for Video Editing Work

5347 Ron Burns, Chao Yang, David Shu Custom Quantizer for Grayscale Shape Coding with Low Latency Sprites

5348 John R. Smith, Uma Srinivasan, Clement Leung

Results of Validation Experiment for MPEG-7 Model-DS on Visual Data

5349 Gabriel Taubin 3D Mesh Coding Reference Software5350 Gabriel Taubin, Mahnjin Han,

Frank BossenReport of AHG on SNHC Conformance

5351 Jeff McVeigh Report of AHG on the study of MPEG-2 Video production processes for supplemental information

5352 Manish Singhal, Jian-Jun Fang, Gerry Melnikov

Update of level definition for Scene Graph Profiles

5353 Sanae Hotani, Toshiro Kawahara, Norio Nakamura

The level of EP-tool for MPEG-4 audio Version2 profile

5354 Sanae Hotani, Toshiro Kawahara, Norio Nakamura

The conformance methods of EP-tool for MPEG-4 audio version2

23

5355 Masahiro Shibata The Second Report of the Verification Experiment on MPEG-7 DS from the Viewpoint of Video Editing and Production

5356 Akio Yamada, Eiji Kasutani, Mutsumi Ohta

Results on core experiment of compact color layout descriptor

5357 German National Body Comments on MPEG-7 Binary Code and Copyright Issues

5358 German National Body Additional Comments to Study of 14496-2/DCOR15359 German National Body Comments on the Assessment of Watermarking

Technology for MPEG5360 Takanori Senoh, Takuyo Kogure Where a common IPMP system is needed5361 The National Body of Belgium ISO/IEC 14496-1 (Systems) FPDAM-1 ballot

comment of BNB5362 The National Body of Belgium ISO/IEC 14496-2 (Visual) FPDAM-1 ballot

comment of BNB5363 Schuyler Quackenbush, Ralph

Sperschneider, Frank FeigeDraft Report on MPEG-4 Version 2 Audio Verification Test

5364 Andrew Perkis, Joe Chicharo, Sigmund Jensen

Report on "Validation Experiments for Universal Multimedia Access (UMA)", Part 2

5365 Ajay Divakaran, Anthony Vetro, Kadir Peker, Huifang Sun

Report on Validation Experiment on Ordered Relation Graphs

5366 Ajay Divakaran, Huifang Sun, Cheol-Soo Park, Haekwang Kim, Ganesh Rattinassababady, V.V. Vinod, D. Manoranjan, Xinding Sun, B.S. Manjunath, Hyundoo Shin

Integration of the Motion Activity Descriptor in the AV DS

5367 Ajay Divakaran, Ana Belen Benitez

Properties of Relations in Entity-Relation Graphs

5368 Web3D Consortium via the SC 29 Secretariat

Liaison from Web3D on VRML/MPEG-4 Convergence (SC 29 N 3346)


Liaison from Web3D on MPEG-4 Technical Review (SC 29 N 3347)


Liaison from Web3D on MPEG-4 IPR Overview (SC 29 N 3348)

5371 Web3D Confortimum via the SC 29 Secretariat

Liaison from Web3D on Proposed X3D SufraceNode for Consideration in Aplication Texture Discussion (BIFS) (SC 29 N 3349)

5372 Ahmet Ekin, Rajiv Mehrotra, A. Murat Tekalp

Video EMU Retrieval Using Parametric Object Motion

5373 TK Tan Proposed Additions to Study of DCOR1 of 14496-2.

5374 Masajiro Iwasaki, Neil Day Core Experiment Results for Spatial Intensity Descriptor (CT4)

5375 I. Sezan and K. Yoon (Editors) (On Behalf of User DS AhG)

Proposed Validation Experiments on User Preferences

5376 Takayuki Kunieda,, Neil Day,, Yuki Wakita

Validation experiment work plan from the viewpoint of office applications

24

5377 Yuki Wakita,, Takayuki Kunieda,, Neil Day

Introduction of Multimedia Document Description Tool and Time Cost for Semantic Annotation

5378 Neil Day, Takayuki Kunieda The Need for an MPEG-7 Applications Evaluation Group

5379 Neil Day, Takayuki Kunieda, Yuki Wakita, Nozomu Takahashi, Masajiro Iwasaki

First Outline of MPEG-7 Vertical Applications Description Tools

5380 Philippe Salembier (on behalf of the DS AHG)

MPEG-7 Generic AV DS (V0.8)

5381 P. Salembier, S. Quackenbush, C. Saraceno, S. Jeannin, T. Walker

AHG Report on MPEG-7 Generic AV DS Development

5382 The National Body of JP (JNB), written by Katsumi Tahara

Comments on working draft of Content Data Extension (N2930)

5383 Akio Yamada, Takehiro Fujita, Katsunao Takahashi

Extended Query set for Sketch-based and Video-segment-based Retrieval Experiments

5384 Jose M. Martinez Content DS5385 Jose M. Martinez (on behalf of the

MedMet AHG)Report of the AHG on Media and Meta DSs and Harmonization with other Schemes

5386 Mike Cox, Jose M. Martinez, Peter Mulder

First attempt on MPEG-7 harmonisation with SMPTE

5387 Jose M. Martinez (on behalf of the MedMet AHG)

Current status and open issues of the MedMet DSs

5388 Jens-Rainer Ohm, Karsten Mueller, Miroslaw Bober

Integration of 2D and 3D description

5389 Jens-Rainer Ohm, Detlef Zier Dataset and Queries for MPEG-7 Core Experiment CT4

5390 Jens-Rainer Ohm, Bela Makai, Detlef Zier

Results of MPEG-7 Core Experiment CT1

5391 Jens-Rainer Ohm, Ferry Bunjamin, Detlef Zier

Results of MPEG-7 Core Experiment CT4

5392 Jens-Rainer Ohm, Ferry Bunjamin Results of MPEG-7 Core Experiment CT55393 Jens-Rainer Ohm Report of Adhoc Group on Fine Granularity

Scalability5394 Aljoscha Smolic, Michael

Hoeynck, Jens-Rainer OhmResults of MPEG-7 Core Experiment on Parametric Object Motion

5395 Guido Franceschini Comments on MP45396 IL National Body Comments on ISO/IEC 14496-1/DCOR 15397 Torsten Mlasko, Takehiro Moriya Report of the Ad-hoc Group on MPEG-4 Audio

Version 1 and Version 2 Conformance5398 Caterina Saraceno Supporting information for the Generic AV

Description Schemes5399 Mark Pickering, Adrian Whichello,

Michael Frater, John ArnoldA Proposal for an Automatic Face Extraction Algorithm

5400 KyuWon Lee, Woong-Shik You, and Jinwoong Kim

Quantitative analysis for the motion trajectory descriptors in MPEG-7

5401 The National Body of JP JNB comments on ISO/IEC 14496-1 (Systems) FPDAM

25

5402 The National Body of JP JNB comments on ISO/IEC 14496-1 (Systems) DCOR

5403 Yoshiaki Shishikui, Hiroyuki Imaizumi, Shinichi Sakaida

Comments for Studio Profile Amendment WD2.0

5404 Yoshiaki Shishikui, Shinichi Sakaida, Hiroyuki Imaizumi

Revised draft description and experimental results of MPEG-4 Core Studio Profile

5405 The National Body of JP JNB comments on the study of ISO/IEC 13818-1/FDAM7

5406 Sang-Wook Kim AHG Report on MPEG-4 Audio version 2 Study on FPDAM Editing

5407 The National Body of JP(JNB) JNB comments on ISO/IEC 14496-2(Visual) DCOR1

5408 Silvia Pfeiffer Obvious Audio Ds / DSs5409 DDL Group MPEG-7 Description Definition Language

Document V 35410 KNB On the operation of the MPEG document FTP site

in IBM(N2903)5411 Jinwoong Kim Report of the ad-hoc group on Object-based content

creation for MPEG-75412 Osamu Hori, Toshimitsu Kaneko,

Koichi MasukuraCross Verification Results of Spatial Localized Motion Activity D

5413 Osamu Hori, Toshimitsu Kaneko Report of Spatio-temporal Region DS Core/Validation Experiment - Experimental Conditions and Test Data

5414 Osamu Hori, Toshimitsu Kaneko Results of Spatio-temporal Region DS Core/Validation Experiment

5415 Osamu Hori, Toshimitsu Kaneko, Koji Yamamoto

Results of Validation Experiments on Summarization DS

5416 N.Ito, E.Morimatsu Report on the Ad Hoc group on Software Integration and Verification in MPEG-4 Video

5417 Joerg Heuer, Andre Kaup, Ulrich Eckhardt, Longin Latecki, Rolf Lakaemper

Region Localisation in MPEG-7 with good coding and retrieval performance

5418 IL National Body Comments on ISO/IEC 14496-6/DCOR15419 Paul Porskamp,, Mettina Veenstra,,

Daan VelthauszResults of the Digital Patient Record Validation Experiment for MPEG-7

5420 Ralf Funken Information on the Philips System standards & Licensing website

5421 Andreas Engelsberg CT5 Cross-Check: Wavelet-based Descriptor5422 Geoffroy Peeters, Perfecto Herrera,

Xavier AmatriainAudio CE on Instruments Description (Timbre Similarity)

5423 German National Body Comments on MPEG-4 as Coder for VoIP Applications

5424 Cidric Thiinot, Claude Seyrat, Pascal Faudemay, Philippe Joly

Status of the MPEG-7 parser

5425 G. De Petris, S. Dal Lago, M. Balestri, A. Varesio

Contribution to MPEG-21: An Integration Attempt of MPEG-4 IPMP, OPIMA and SDMI

26

5426 G. De Petris, S. Dal Lago, M. Balestri, A. Varesio

Contribution to MPEG-21: An Integration Attempt of MPEG-4 IPMP, OPIMA and SDMI

5427 Torsten Mlasko, Ertugrul Akyuez Proposed modifications to error resilient syntax of CELP objects

5428 Guido Franceschini OCR fields in ESD5429 Joerg Heuer Report of Ad Hoc Group on MPEG-7 linking5430 Ana B. Benitez, Shih-Fu Chang,

John R. Smith, Chung-Sheng LiWork Plan for Validating the Collection Structure DS

5431 Ana B. Benitez, Neil Day, Sylvain Devillers, Jin-Soo Lee

Report of the AHG on MPEG-7 DS Validation and Core Experiments

5432 Ana B. Benitez, Di Zhong, Shih-Fu Chang, Caterina Saraceno, Markus Kangas, B. J. Brown

Report on the Validation Experiment on the Syntactic DS, the Semantic DS, and the Syntactic/Semantic Link DS

5433 Ana B. Benitez, Paul Bocheck, Shih-Fu Chang, John R. Smith, Rick Han, Chung-Sheng Li, Joe Chicharo, Andrew Perkis, Sanghoon Sull

Work Plan for Validating the Media Transcoding Hint DS for Universal Multimedia Access

5434 Masaru Sugano, Yasuyuki Nakajima

Consideration on MPEG-7 Summary DS for Audio Contents


Improvement on the Summary DS for Collection of Summaries


Consideration on Media Synchronization for Audiovisual Summaries

5437 John R. Smith, Sanghoon Sull, Charilaos Christopoulos, Yousri Abdeljaoued, Ana B. Benitez

Report on Validation Experiments for Universal Multimedia Access (UMA) - Part 1

5438 John R. Smith, Chung-Sheng Li, Sanghoon Sull, Ana B. Benitez

Work Plan for Validating the Segment Hint DS for Universal Multimedia Access (UMA)

5439 John R. Smith, Chung-Sheng Li, Ana B. Benitez

Work Plan for Validation of Space and Frequency View Description Schemes

5440 Sanghoon Sull, Jung-Rim Kim, Yunam Kim

Report on Core/Validation Experiments of Weight DS


Report on Validation Experiments of Summary DS


Improved notion of the fidelity for efficient browsing

5443 A. de Cheveigné Scale Tree Update5444 A. de Cheveigné, G. Peeters A Set of Scalable Audio Descriptors5445 Zvi Lifshitz IM1 Core code + authoring tools version 4.05446 HASIDA Koiti MPEG-7 Basic Ontology and

Linguistic/Data/Sematic Links5447 Itaru Kaneko Necessity of basic regularity of IPMP, as summary

of past discussions5448 Itaru Kaneko IPMP-delay for transcoding of MPEG-4.5449 B. S. Manjunath, P. Wu, S.

Newsam, B. Sumengen, H. D. ShinTexture descriptor core experiments: browsing component

27

5450 Zvi Lifshitz BIFS/OD Encoder version 4.05451 Dulce Ponceleon, Nevenka

DimitrovaExtended SummaryDS

5452 A. G. Tescher USNB Contribution: Commercial software issues5453 Prabhudev Irappa Hosur , Kai-

Kuang MaReport on Performance of Fast Motion Estimation using Motion Vector Field Adaptive Search Technique (MVFAST)

5454 Soo-Jun Park, Seong-Joon Yoo, Chee Sun Won, Dong Kwon Park

Core Experiments on MPEG-7 spatial image intensity distribution descriptors (CT4)

5455 Heon Jun Kim Report of CT1: HMMD Color Space and Histogram Quantization

5456 Heon Jun Kim CT3 Cross Check: Generalized Histogram5457 Heon Jun Kim Studies of Confidence Measure in Dominant Color

Descriptor5458 Kyoungro Yoon, Sung Bae Jun Validation Experiment Result Report on Summary

DS5459 Kyoungro Yoon, Sung Bae Jun,

Jung-Min SongValidation Experiment Result Report on Ordered Relation

5460 Jin-Soo Lee Report of Validation/Core Experiment on Weight DS

5461 Young-Kwon LIM, Carsten Herpel, Michelle Kim, Zvi Lifshitz

Report of AHG on MPEG-4 backchannel and ESM

5462 Young-Kwon Lim, Gauthier Lafruit

Draft text of the informativ annex on MPEG-4 upstream walkthrough

5463 The National Body of JP Comments on FCD ISO/IEC 14496-45464 Shun-ichi Sekiguchi, Yoshimi Isu,

Hirofumi Nishikawa and Kohtaro Asai

Experimental Result on Weight DS

5465 Shuichi Watanabe, Yasuaki Tokumo

Spatial Localized Motion Activity for MPEG-7

5466 Shuichi Watanabe, Yasuaki Tokumo

Experimental Methods for Spatial Localized Motion Activity D

5467 Yasuaki Tokumo, Shuichi Watanabe

Results of Experiments for Spatial Localized Motion Activity D


Cross-Verification Results of Spatio-temporal Region DS

5469 Yoichi Yagasaki Ad hoc Group Report on the Studio Profile in MPEG-4 Video

5470 The National Body of JP JNB Comments on MPEG-4/Audio Transport Stream Format

5471 Masahiro Iwadare, Toshiyuki Nomura

Withdrawal of support for an MPEG-4 Version 2 Audio Profile

5472 Whoi-Yul Kim, Yong-Sung Kim A new region-based shape descriptor: The ART (Angular Radial Transform) Descriptor

5473 Whoi-Yul Kim, Yong-Sung Kim Cross-verification results of Modified Zernike Moment Descriptor

5474 Eric Rehm Related Material DS

28

5475 Eric Rehm Thesauri and the Annotation DS5476 Weisi LIN, Chengyu XIONG Verification Report of Performance Evaluation of

Fast Motion Estimation Algorithm: MVFAST5477 S. Fukunaga, Y. Nakaya, S.H. Son,

T. NagumoMPEG-4 Video Verification Model version 14.2

5478 Eric REhm System vs. Component Format/Coding in the Media Profile DS

5479 A. Basso, S. Varakliotis FEC performances in Streaming MPEG-4 over IP5480 The National Body of JP(JNB) JNB Comments on Contributing Binary Modules to

the MPEG-7 XM5481 Yoshiaki Shibata, Yasushi

Takahashi, Mikio Kamada, Ei-ichi Osawa, Hitoshi Kimura, Masayoshi Miura

Weight DS Extension for Affect-based Content Characterization

5482 The National Body of SG A Request for Fast Motion Estimation5483 The National Body of JP JNB comments on ISO/IEC 14496-2 FPDAM1

(MPEG-4 Visual Version 2 FPDAM)5484 The National Body of JP Proposal on editing procedure of ISO/IEC 14496-

1/2/6 FDAM15485 Yuichiro Nakaya, Yoshihiro

Kikuchi, Yoshihiro Miyamoto, Eishi Morimatsu

Request for clarification of VBV specifications

5486 KNB Supporting statement on the release of source code for the non-normative tools related to the normative part of MPEG-7

5487 Hae-Kwang Kim, Jong-Deuk Kim, Nam-Kyu Kim, Dong-Gyu Sim

Mini-Core Experiment Results - Modified Zernike Moments Shape Descriptor

5488 Han-kyu Lee,Munchurl Kim,Jinwook Kim,Nam-Kyu Kim,Hae-Kwang Kim

Considerations on Structure of Synthetic Model DS for Describing Human Body Object

5489 Hong Jiang, Gregory M. Thayer Using Frequency Weighting in FGS Bit-plane Coding for Natural Video

5490 Yong Man Ro, Ki-Won Yoo, Munchurl Kim, Jinwoong Kim, B. S. Manjunath, Hyundoo Shin, Dong-Gyu Sim, Hae-Kwang Kim,, Jens-Rainer Ohm, Ferry Bunjamin

A unified texture descriptor

5491 Munchurl Kim, Jinwoong Kim and Yong Man Ro

Cross-check results on scale/rotation-invariant texture descriptor proposed by Hyundai Electronics

5492 Munchurl Kim, Jae-Gon Kim, Hyun Sung Chang, and Jinwoong Kim

Results of the Validatioin Experiments on Summary DS

5493 Munchurl Kim, Jae-Gon Kim, Hyun Sung Chang, and Jinwoong Kim

An efficient Hierarchical Summary DS for event based summarization

5494 E.Hartley (For UK National Body) UK National Body Position on Source Code for Normative and Non Normative Tools

5495 E.Hartley (For UK National Body) UK National Body Position on Document

29

Management Technologies5496 E.Hartley (For UK NAtional Body) UK National Body Position on Provision of Web

and FTP Services5497 Yoshinori Suzuki, Seishi

Takamura, Yuichiro Nakaya Request for revision of Profiles under Consideration Document

5498 Yong Man Ro, Ki-Won Yoo, Munchurl Kim, and Jinwoong Kim

Cross-check results on PBC based texture description proposed by UCSB/Samsung

5499 Yong Man Ro, Ki-Won Yoo, Munchurl Kim, and Jinwoong Kim

Preliminary experimental results on boundary shape effects in homogeneous texture description

5500 Woon Kee Min, Young-tae Kim, Yong Man Ro, Munchurl Kim, and Jinwoong Kim

Effective Color Indexing using a Multi-quantization Technique

5501 Ho Young Lee, Young-tae Kim, Yeong Ho Ha, Munchurl Kim, and Jinwoong Kim

Color constancy consideration for illumination-invariant color description

5502 Dong-Gyu Sim, Hae-Kwang Kim Translation, rotation, and scale invariant texture descriptor of Zernike moments on DFT image

5503 Masahiro Serizawa, Toshiyuki Nomura, Masahiro Iwadare

A Draft Conformance Testing of ER CELP for MPEG-4 Version 2 Audio

5504 Y. Abdeljaoued, T. Ebrahimi, Ch. Christopoulos, Aljoscha Smolic

Validation Experiment for MosaicDS

5505 Takehiro Moriya, Takeshi Mori, Naoki Iwakami, Akio Jin

Proposal of Conformance Test Procedure for the ER-TwinVQ Object Type


Results of Similarity based Retrieval by using Spatial Localized Motion Activity D

5507 Shaojie Zhang and Kai-Kuang Ma A New Proposal for Shape Description and Matching

5508 P. N. Garner, J. P. A. Charlesworth, S. Srinivasan

Current Status of Spoken Content DS

5509 Takehiro Moriya, Takeshi Mori, Naoki Iwakami, Akio Jin

Proposal of Editorial Corrections of 14496-3 FPDAM 1 (ER-TwinVQ Object Part)

5510 Ju Han and Kai-Kuang Ma A Novel Color Histogram Representation for Color Images

5511 Takehiro Moriya Supporting Statement of MPEG-4 Version 2 Audio Profile

5512 Jae-Seob Shin, Norio Ito, Yoichi Yagasaki

Consideration on combining CSP and FGS - Response to the Resolution 2.1.7 of the Melbourne Resolution Document (N2903)

5513 Dae-Sung Cho, Jae-Seob Shin Definition of conformance bitstreams for scalable shape for still texture

5514 Sung-Hee Park, Yeon-Bae Kim A description of conformance testing for ER Fine Granule Audio

5515 Samsung Electronics Co., Ltd Support statement for the Natural Audio Profile5516 Sanghoon Sull, Sangwook Oh,

Seong Soo ChunMechanism for Checking Link Validity in General AV DS

5517 Eishi Morimatsu DRC related commnets on ISO/IEC 14496-2/FPDAM1

30

5518 Tolga K. Capin, Fabrice Vergnenegre

Release Notes Update for Body Animation Integration into the im1 3D Player

5519 Jose M. Martinez Overview of MPEG-7 description tools: Descriptors and Description Schemes (V0.3)

5520 Do-Kyoon Kim, Mun-Sup Song, S.Y. Jung, Mahnjin Han, Euee S. Jang

Analysis of existing tools for animated 3D model coding

5521 Sungjin Kim, Mahnjin Han, Mun-Sup Song

3DMC Reference Software

5522 Do-Kyoon Kim, Sungjin Kim, Mahnjin Han, Mun-Sup Song, Euee S. Jang

Proposal for upstreamming with ServerCommand node

5523 Mahnjin Han, Gabriel Taubin, Frank Bossen

The conformance methods for SNHC 3DMC

5524 Zvi Lifshitz IM1 Software Platform AHG Report5525 Yuji Maeda, Masayuki Nishiguchi Revisions of V2 text for ER HVXC object5526 Jean H.A. Gelissen (editor), Ralf

Funken, Jan van der MeerDNB Comments to the Amendment 7 to ISO/IEC 13818 and the Audio parts of MPEG-2 and MPEG-4.

5527 Heiko Purnhagen, Bodo Teichmann

Report of AHG on MPEG-4 Audio Version 2 Reference Software editing

5528 Tihao Chiang, Huifang Sun Report of Ad Hoc Group on Encoder Optimization5529 Heiko Purnhagen, Bernd Edler,

Nikolaus MeineStudy of HILN in 14496-3 FPDAM1 and proposed revisions

5530 Heiko Purnhagen, Bernd Edler, Nikolaus Meine

Information on HILN conformance testing

5531 Heiko Purnhagen, Bernd Edler, Nikolaus Meine

Study on HILN subjective quality

5532 Heiko Purnhagen, Silvia Pfeiffer MPEG Audio FAQ Update5533 Jan Bormans, Gauthier Lafruit Contribution to MPEG-21: Advanced QoS

scenarios5534 Miroslaw Z. Bober Strategies for Region Localisation DS5535 Miroslaw Z. Bober Report on the AHG on Shape5536 Keith Hill Report of the AHG on Multimedia Framework5537 Miroslaw Z. Bober, Jim Atkinson Investigation of the Robustness of the Contour-

Based Shape Descriptor to Perspective Transformation

5538 Jim Atkinson, Miroslaw Z. Bober Analysis of the Performance of the Modified Zernike Moments for Complex Shapes

5539 Leszek Cieplinski Further Results on the Extension of the Dominant Colours Descriptor

5540 Joachim Koehler Evaluation of the MPEG-7 speech DS5541 Kevin Smith, Dan Popescu, Chris

GunnProposal for a packed editing command for multi-valued fields

5542 Euee S. Jang, Mikael Bourges-Sevenier

AHG report on Study of 3D animation coding

5543 Euee S. Jang, Yuichiro Nakaya, AHG report on MPEG-4 Video VM and Visual

31

Visual Editors FDAM editing5544 Takehiro Fujita, Takafumi

MiyatakeMini-Core Experiment Results on Run-length D of Compact Color Layout (CT2)

5545 Shigeru Fukunaga Demonstration of Realtime Video Transmission System

5546 Osamu Sunohara, Yoichi Yagasaki The draft of MPEG-4 Studio Profile Amendment Working Draft 3.0

5547 Osamu Sunohara, Yoichi Yagasaki, Hiroyuki Imaizumi, Shinichi Sakaida, Yoshiaki Shishikui

The report of bitstream exchange for MPEG-4 Simple Studio Prifle

5548 Osamu Sunohara, Yoichi Yagasaki The proposal of VLC method for MPEG-4 Studio Profile

5549 Osamu Sunohara, Yoichi Yagasaki Study of Normative DCT operation for MPEG-4 Studio Profile

5550 Marco Mattavelli Implementation and synchronization considerations on IPMP and Audio BIFS decoders embedded in the MPEG-4 System Layer

5551 Yingwei Chen; Bertrand Thebault Selective Enhancement for Fine Granular Scalable Video

5552 Mihaela van der Schaar, Hayder Radha, Yingwei Chen

An all FGS solution for hybrid Temporal-SNR scalability

5553 Leszek Cieplinski Cross-Verification of Perceptual Browsing Component of UCSB/Samsung Texture Descriptor

5554 Roberto Castagno, Serkan Kiranyaz, Florin Lohan, Irek Defée

MPEG-4 over IP: preliminary results using MPEG/IETF AVT payload and DAI/RTSP mapping

5555 Peter Westerink, Michelle Kim, Steve Wood

Incomplete specifications in the Systems FDIS

5557 Kris Huber, Kyminh Liang, Chien-Min Huang

Influence of decode/shift/encode cycles on image quality

5558 Tolga K. Capin, Eric Petajan Support on FBA Simple, 3D Hybrid, and Version 2 Complete Graphics Profiles

5556 Peter Westerink, Michelle kim, Ed Snible

Some minor comments on the Systems Conformance Scene Dump format

5559 Sylvain Devillers, Jean-Christophe Broudin

Shape and shape deformation descriptors based on complex Fourier descriptors

5560 V. Vinod, B. S. Manjunath AHG Report on Color/Texture Core experiments5561 Andreas Hutter, Ralf Buschmann,

Fernando Garcia, Robert Kutka, J³rgen Pandel

Improved bit-plane coding for FGS video coding

5562 Adam Lindsay (on behalf of the MPEG-7 Audio AHG)

Report of AHG on MPEG-7 Audio Matters

5563 B. S. Manjunath, B. Sumengen, S. Newsam, H. D. Shin

CT-5 texture core experiments results

5564 French NB Support for SVP profile with AC/DC prediction5565 French NB Watermarking standardisation considerations5566 Yousri Abdeljaoued, Touradj A proposal for the modification of the Sequential

32

Ebrahimi, Charilaos Christopoulos Summary DS5567 French NB Towards a normative ESI ?5568 Xuemin Chen,Ajay Luthra,

Weiping Li, Minhua ZhouCoding-Tool Configuration Packet for MPEG-4 Video

5569 Kilian Jacob, Andre Kaup, Andreas Hutter, Sven Bauer, Ulrich Benzler

Status of Conformance for the ACE Profile and Comments on Video Buffering Verifier

5570 Kilian Jacob, Paulo Nunes Proposed Editorial Change for ISO/IEC 14496-2, Annex D

5571 Bernhard Grill, Masahiro Iwadare Report of the ahg on MPEG-4 Version 2 Audio Technical Matters

5572 P. van Beek, D. Messing, I. Sezan Results of CE CT3 on Spatial Information Embedding Color Descriptors

5573 Eric Petajan, Tolga Capin FBA adhoc Report5574 P. van Beek, D. Messing, I. Sezan Cross-check results of CE CT1 on Color Space and

Histogram Quantization5575 P. van Beek, I. Sezan Results of the Validation Experiment on Summary

Description Schemes5576 P. van Beek (on behalf of VE

participants)Workplan and Status of the Validation Experiment on Summary Description Schemes

5577 Sylvie Jeannin Report of the AHG on editing the MPEG-7 Visual XM

5578 Sylvie Jeannin MPEG-7 Visual part of eXperimentation Model Version 3.1

5579 R.L.Schmidt, Atul Puri Specification of an Advanced BIFS node for an Application Window

5580 A. Murat Tekalp, Rajiv Mehrotra, Masato Kurokawa

Semantics-Driven Multimedia Description

5581 Andrea Basso AT&T Labs Activities on RTSP/MPEG4 mapping in IETF MMUSIC

5582 Weiping Li, Ajay Luthra, Xuemin Chen, Fan Ling, Hayder Radha, Yingwei Chen, Mihaela van der Schaar, Anthony Vetro, Huifang Sun, Caspar Horne, Iraj Sodagar, Shigeru Fukunaga, Zhixiong Wu, Hong Jiang, Chao-Kung Yang, M

5583 Shipeng Li, Feng Wu, Ya-Qin Zhang

Study of a new approach to improve FGS video coding efficiency

5584 S. R. Quackenbush Using the mp4 library to create AAC audio bitstreams

5585 S. R. Quackenbush Core experiment proposal for a description scheme for music program creation

5586 D.Curet, M.Veillard Comments on Draft amendment 7 to ISO/IEC 13818-1

5587 D.Curet, F.Cerisier, K.Renout SL layer & ESI interface clarifications5588 D.Curet, C.Daguet Possible improvements to the ESI interface5589 Weiping Li Frequency Weighting for FGS

33

5590 Javier Zamora, Joe Caldarola Update on the new DAI syntax proposal5591 Javier Zamora, Joe Caldarola Proposal for a DMIF Roadmap5592 Titus Zaharia, Marius Preda,

Francoise Prêteux3D Shape Descriptors: Results and Performance Evaluation

5593 Alejandro Jaimes, Corinne Jorgensen, Ana B. Benitez, Shih-Fu Chang

Experiments for Multiple Level Classification of Visual Descriptors

5594 Titus Zaharia, Marius Preda, Frangoise Prjteux

Motion Content Set: New Contributions

5595 Titus Zaharia, Marius Preda, Frangoise Prjteux

Similarity Measures for Motion-based Retrieval

5596 Mauri Vaananen, Juha Ojanpera A description of conformance testing for ER AAC LTP

5597 Riitta Vaananen, Jyri Huopaniemi Proposals on MPEG-4 Version 2 Advanced AudioBIFS Conformance and Level Definitions

5598 Riitta Vaananen Study on Systems FPDAM (BIFS part)5599 Zvi Lifshitz Review of FlexTime implementation5600 Viswanathan Swaminathan MPEG-J AHG Report5601 Viswanathan Swaminathan, Gerard

FernandoEdit List for MPEG-J subpart of Systems DAM-1

5602 Kadir A. Peker, A. Aydin Alatan, Ali N. Akansu

Comparison of Motion Activity Descriptors Based on Activity Ground-Truth

5603 A. Aydin Alatan, Ajay Divakaran A Refinement on Probability Model DS5604 Michael Casey Sound Effects Core Experiment5605 Lorenzo Rossi, Nicola Adami,

Alessandro Bugatti and Riccardo Leonardi

Validation Experiment on the Ordering Key DS and an Unified Syntax for the Weight DS

5606 Riccardo Leonardi and Lorenzo Rossi

An Elementary Building Block for the Generic AV DS: the Node DS

5607 Gary Sullivan Liaison Statement: Status of ITU Video Coding Work and IDCT Coordination Issue

5608 Michelle Kim, William Belknap, Peter Schirling, Peter Westerink

MPEG-4 Over Multiple Transports/Protocols

5609 Hae-Kwang Kim, Jong-Deuk Kim Cross-verification results of ART Shape Descriptor5610 Ali J. Tabatabai, Hawley K. Rising

IIIA Proposed Structure for the KeyFrame DS

5611 Hawley K. Rising III, Ali J. Tabatabai

A Useful Generalization of the Syntactic/Semantic Division and its Application to Collections

5612 Hawley K. Rising III A Proposal for Using Entropy to Measure the Quality of Descriptors and Description Schemes

5613 Hawley K. Rising III, Ali J. Tabatabai

A Proposal for a Core Experiment on Entropy for Descriptors and Description Schemes

5614 Hawley K. Rising III An update of the Mathematical model for Description Schemes

5615 Adam Lindsay On the Use of Time-Series Style Descriptors5616 Gerard Fernando Input Draft Text for MPEG-J Explanatory Report5617 James Macnicol, Michael Frater, Results for Fine Granularity Scalability

34

John Arnold5618 Koichi Emura, Toshihiko

MunetsuguResults of Validation Experiment on Weight DS(DS4: Pointofview DS)

5619 Olivier Avaro et al. Report of the AHG on MPEG-7 Systems5620 Olivier Avaro et al. Report of the AHG on Multi-users Applications5621 Olivier Avaro et al. Requirements for an MPEG-4 Authoring Format5622 Olivier Avaro et al. Issues on MPEG-4 Profile and Level Indication5623 Olivier Avaro et al. Requirements on MPEG-4 Multi-users Applications5624 Riccardo Leonardi Report of AHG on MPEG-7 Semantic DS5625 Carsten Herpel, Jan van der Meer Report of AHG on MPEG-4 Content on MPEG-2

Systems and on the Internet5626 Yuwen He, Shiqiang Yang,

Yuzhuo ZhongImproved Method for Global Motion Estimation Used in Sprite Coding

5627 IEC TC 100 Liaison from IEC TC 100 on Collaboration with WG 11 (SC 29 N 3370)

5628 Weiping Li, Fan Ling, and Xuemin Chen

Preliminary Results on Dynamic Testing of FGS

5629 Weiping Li Verification Results on Progressive FGS5630 NB of Finland Late Comments on ISO/IEC 14496-1/FPDAM 1

(SC 29 N 3371)5631 NB of Finland Late Comments on ISO/IEC 14496-3/FPDAM 1

(SC 29 N 3372)5632 NB of Finland Late Comments on ISO/IEC FCD 14496-4 (SC 29

N 3373)5633 Weiping Li Improvement of Selective Enhancement Syntax for

FGS

35

Annex 4Requirements Group Report

Source: Rob Koenen

MPEG-2 / MPEG-4The AHG on the new (MPEG-2/progressive) profiles has produced very little result. It is still unclear what proponents want to achieve exactly; the discussion is confusing because requirements and solutions are all discussed together. Therefor, proponents were asked to will write a document that explains what they want to do: Applications, Operational Environments and Requirements for new Progressive Profiles and Levels. This document was approved.The next step will be to see whether existing tools (either MPEG-2 or MPEG-4 or MPEG-4 over MPEG-2) solve the problem - or why they don't solve the problem.

There is the question of whether to use MPEG-2 or MPEG-4 to meet the requirements. There are two sides to this question: a technology side and the aspect of education/image (MPEG-2 is well established and people are wary of using MPEG-4). For instance: people have problems with the multiple time bases that MPEG-4 supports, but are unware of the fact that there is a very simple way of limiting the number of time bases used in the system, even down to one..

Once it's clear what we are trying to make possible, we can assess whether MPEG-2 or MPEG-4 give is the right tools. Perhaps it is necessary to explain how the power of MPEG-4 can be restricted. Perhaps it is also an issue of using the MPEG-4 over MPEG-2 tools.

MPEG-4For all newly adopted profiles, conformance commitments were given by companies in MPEG. This was a necessary condition for the profile to be adopted in the standard. The complete list is N3064.

The profiles that were adopted were removed from the Profiles under Consideration document. Only profiles not yet adopted stay in there (N3155)

Material for a new version of the Overview was collected; the new version will be issued after an editing period.

Visual Profiles Version 2

5291 SC 29 Secretariat - Summary of Voting: ISO/IEC 14496-2/FPDAM 15483 JNB - JNB comments on ISO/IEC 14496-2 FPDAM1 (MPEG-4 Visual V.2 FPDAM)

The following requests with respect to V.2 profiles were made by NB’s: Add Advanced Core (US, KR, J) - accepted Add Advanced Scalable Texture (US) - accepted Add level to ARTS (J) - accepted Modify some level definitions, because they are wrong. (J) - accepted

The considerations were: ARTS, Advanced Core: enough support exists; the profiles are desired by companies that

want to implement it.

36

Advanced Scalable Texture - JP has withdrawn request for Advanced Scalable Texture, because 1) it is a still-picture

only profile and 2) there is the desire to limit the number of profiles. Members of JNB still consider it a good extension of the V.1 still texture profile, however.

- A number of companies supporting the profile now appear not to have concrete product plans.

- Other companies in the list do have these plans, and would value the conformance and interoperability point.

- Comparison with JPEG(2000): JPEG is not object-based - JPEG 2000 will not be object-based either - AST is object-based. No other standard solution available.

- The last to arguments led the group to accept the profile.

5497 Y. Suzuki et. al. - Request for revision of Profiles under Consideration Document :The problem is that text in the PuC document is inconsistent with the standard. This will be solved when the text about ACE is removed from PuC document.The decision was taken to remove all profiles that are incorporated in the FDAM from PuC document.

5512 Jae-Seob Shin, Norio Ito, Yoichi Yagasaki - Consideration on combining CSP and FGS Response to the Resolution 2.1.7 of the Melbourne Resolution Document (N2903)Products already underway, so the contribution concludes that delaying the Core Scalable is undesirable. The Requirements group decided accordingly.

5342 Ph. Hoogvorst, J.-C. Dufourd - Request for inclusion of JPEG in visual profiles The request is to include JPEG object types in Visual Profiles, notably Main and Core. In principle it is possible to include JPEG object types in Visual Profiles. (The same applies to MPEG-2 and MPEG-1 object types). However, Main and Core are already defined, and cannot be changed. Requests like this can be discussed in the context of new profiles.

SNHC-related Profiles

In the joint meeting with SNHC it became clear that:1. FBA is ready to go to FDIS, but no formal request was made by NB’s.2. For 3D Mesh no formal requests exist either, but a more serious issue is that there are

problems with the specification of the profile (e.g., there is no distinction between object types and profiles).

Issue 1) was solved with a late comment by the Swiss NB. Issue 2) could not be solved that easily. To still have the 3D Mesh profiles as soon as possible, it was decided to issue a PDAM at this meeting, and with very tight time management, the profiles can go to FDAM in July.

FGS

5582 Weiping Li et. al. - FGS ProfilesThe document addresses convergence of 4 FGS profiles in PuC document into 2 newly proposed profiles. It has the following characteristics: More than one object possible now. (but still not as many as in other profiles at comparable

bitrate!) Complexity AC prediction is too high. Coding gain only for intra at lower bitrate - not the

bitrates we are looking at. The contributors don’t want AC prediction in this profile. It is, however, undesirable that the profile cannot decode Simple and the advanced version cannot decode Simple and Core. Discussions will continue.

37

Interlace starts to make sense at L3. A proposal is made to exclude it in L1 and L2 in this special case. This would go against the principle of not excluding tools by defining levels.

In a joint meeting with Video, it becomes clear that FGS not quite ready yet for going to PDAM. Temporal scalability is not yet stable. The Video group doesn’t see problems with AC-DC – but is ‘beyond scope of Video Group’. A request has been made to ISG to make a study on the issue.During the discussions, it turns out that some people want to standardise the base layer for these profiles already now. This undocumented request, put forward late in the week, could not be accommodated.

Studio Profiles

5549 O. Sunohara, et. al.- Study of Normative DCT operation for MPEG-4 Studio Profile Sony is still studying the lossless area. There is no good technology yet. This is not needed in all the proposed Studio Profiles

???? Don Mead (Digital Electronic Cinema Inc.) Digital Cinema Spec.The goal is to enable ‘celluloidless’ cinema. Don presented a number of requirements and some anecdotes. He seeks lossless compression (1:5, 1:10?) and also 1:100 for distribution – and further depending on the device (even up to mobile pda’s and such.)He promised to share with MPEG the result of work done at Paramount. It was further agreed to further work on the issue in the context of the HiQ AHG.

5557 K. Huber, et. al. – Influence of decode/shift/encode cycles on image quality the contribution states that JPEG LS is better than MPEG-4 for lossless coding. Some evidence was given (with a few surprising results). Also these results will be studied in the HiQ AHG.

Audio ProfilesIn the audio area, 4 new profiles were under consideration, and they were all adopted in the FDIS:1. High Quality Audio Profile2. Low Delay Audio Profile3. Natural Audio Profile4. Mobile Communications Audio Profile (tentative name)As a departure from the strict Profile/Level policy, the addition of error resilience tools is dealt with in the Level definitions. This was done to keep the amount of new profiles from doubling. It was agreed to find a new name for the mobile communications profile, but at the end of the meeting no satisfactory name was on the table yet. (Mobile Audio Internetworking Profile wasn’t really any better than the original proposal.)The number of objects in the originally proposed Level definitions were replaced by PCU and RCU definitions, to allow more flexibility in constructing the scene.

Systems Profiles

V.2 Systems Profiles

It was decided not to adopt a number of Profiles under consideration for which there exist no deployment plans. This means that no Graphics and Scene Graph Profiles are included in the V.2 FDIS, and that we wait until there are mature deployment plans. MPEG-J Profiles were adopted, and named ‘Personal’ and ‘Main’.

38

5352 M. Singhal et. al. - Update of level definition for Scene Graph Profiles The document highlights a problem in the Level definition when other pixel depths than 8 are used. The definition is updated in such a way that the amount of pixels to be processed now depends on the depth. You can choose for greater depths in your scene, but then the requirement on the amount of pixels to be processed will go down. This will ensure roughly equal complexity constraints under all conditions.

Beyond V.2

5622 Olivier Avaro et al. - Issues on MPEG-4 Profile and Level Indication The issue on signalling scalable Profiles and the streams belonging to these Profiles was in principle resolved with the contribution by Avaro et. al. It will now be possible to send sets of streams constituting scenes according to multiple profiles, e.g. a base profile and a superset. It can then also be signalled which streams belong to which profile, where a base set of streams will belong to both profiles.

5621 Olivier Avaro et al. – Requirements for an MPEG-4 Authoring Format It was agreed to issue a Call for a textual format, which will be developed jointly with the web3D consortium. (N3157)

5623 Olivier Avaro et al. – Requirements on MPEG-4 Multi-users Applications The multi-user requirements document was agreed. There is consensus that the application area is important to MPEG-4, and it was good to hear several that companies are commited to work on it. An AHG was established. Extracting the MPEG-4 specific requirements from the generic multi-user world requirements still needs to be done.

5339 M. Brelot, J.-C. Dufourd – New BIFS Sensors for V3: refinement and architecture5340 M. Brelot, J.-C. Dufourd - Ideas for other applications and BIFS sensors for V3 It was agreed that some of the new sensors are useful. It was also concluded that not all the functionality should fall within MPEG, and MPEG does not hold the required expertise to address all the proposed sensors and parameters. Further study is needed to clarify this.

IPMPDuring the work of the AHG it became clear that the extended IPMP descriptors are not necessary, but more explanatory text in the standard would be helpful. The second issue is that of the standardized IPMP Systems.

Watermarking

5314 Panos Kudumakis for the UK National Body - UK National Body position on 'advanced IPMP descriptors' and on standardising IPMP systems 5359 German National Body – Comments on the Assessment of Watermarking Technology for MPEG 5565 French NB – Watermarking standardisation considerations Several NB’s had commented on the issue of standard watermarking.. The comments are address below.There was agreement that one technology cannot fit all application domains. It was noted that perhaps it is possible to do something for certain business models. The question was raised whether MPEG can do better than private schemes, and whether schemes would survive the publication inherent to standardization. Most people believed that publishable techniques exist.

39

It was noted that there are more reasons for doing watermarking, only some of them related to security. Also there would allegedly be a risk of litigation – if the technology fails MPEG could be sued. The argument does not hold in the same way that MPEG cannot be sued for failing STB’s. It was noted that watermarking goes across standards bodies – but so do more MPEG topics. If mpeg tackles the problem at all, the first action should be to collect requirements for applications and business models, and to address the issue of evaluation methodology. Robustness is a big question in many environments (There are experts who claim to be able to destroy any watermark fast – ‘a matter of minutes’)

The adopted proposal was to initiate a study, with the following topics: What problem is solved with watermarking?

What is the role in MM? In different application domains? In what environments do we have which requirements?

What technologies exist, and how well do they perform? What are the evaluation methodologies? What is the impact in (audio/image) quality? What is the capacity of a watermark relative to the picture/audio ‘size’? How robust is the watermark?

Standardizing IPMP Systems

m5360 T .Senoh, T .Kogure - Where a common IPMP system is needed m5635 H. Ohtsubo et.al. – Simple and Minimum interoperability among All IPMP Systems m5447 I. Kaneko – Necessity of basic regularity of IPMP, as summary of past discussions

These contributions go a step further than the watermarking question, as it requests the standardization of complete IPMP systems. The contribution represents the point of view of the MPEG-4 platform from JP.“In several environments, you will get multi-function devices. If these devices are to access different services, then standardization of IPMP systems is needed.”

Conclusion of the contributors: MPEG should specify common IPMP systems

It is remarked that it is very difficult to 1) design good systems and 2) get the involved people – the industry – to agree on and use such a system. That’s why MPEG decided not to go that road, and OPIMA had a difficult task at hand. “Application experts should do it, not MPEG.” Other people think that while it is perhaps difficult, it may not be impossible.

Another important question: the problem is very real from a user perspective, but it is unclear if ONE single IPMP system would be the right or the only possible solution? We should look at the problem from MPEG-21 perspective – which starts with the end user.

It was decided to again start a study, this time only focussed on requirements, not on solutions. Perhaps in the end it will turn out to be an MPEG-21 issue, but it was decided to start the discussions in the context of MPEG-4, and to take as a focus the requirements in 1. Mobile applications2. Set-top applications3. Desktop applications An AHG was created to address the issue (N3164).

40

MPEG-7

Systems, (including IPMP)IPMP for MPEG-7. A break out group has worked on requirements for IPMP in MPEG-7. Currently, the group addresses requirements for descriptors and DS’s, protection of descriptions may follow later. The work was done following the conceptual modeling principles. An AHG was established (N3006) that will have a meeting on 3 and 4 Februari in Paris, with people from the rights communities.

RequirementsAre there clusters of applications? Can we map the MPEG-7 application space?Do these clusters need to interoperate.

Approach: Take Application Document Are there any important applications missing? add those Assign weight to the applications (importance to MPEG-7) Draft requirements per application (starting with the most important ones) Do a clustering exercise

MPEG-7 Requirements AHG doing this. Who will draft the mandate? Who will chair the AHG? People involved: Miroslaw, Yamada-san, someone from Philips (Sylvie will see)

Will start application verification activity. (Neil’s contributions)

MPEG-7 Development ProcessThe document with the MPEG-7 development process was finalized after a last review by all the groups and decisions about the copyright statement/disclaimer and the software policy. Rather than repeating those here, the reader is referred to N2999 and the resolutions from the Maui meeting.

OverviewA very good result of the Maui meeting was the first presentable version of the MPEG-7 Overview, edited by Jose Martinez. All the groups have contributed to this very important document. Right now it serves as a document for internal MPEG reference, so that all the groups know what the others are doing. After a next iteration, it should be possible to release it publicly (which has been the intention for a number of meetings, but now it should really be possible).

Conceptual Modeling The conceptual modeling activity is catching on: more people use it, and it pays off. The group has chosen Extended Entity Relation modeling using UML over other modeling techniques (such as object-based). The modeling of spatiotemporal relationships still needs to better understood.

Application experimentsm5379 N. Day et.al. – First Outline of MPEG-7 Vertical Applications Description Toolsm5378 N. Day, T. Kunieda - The Need for an MPEG-7 Applications Evaluation Group m5377 Y. Wakita – Introduction of Multimedia Document Description Tool and Time Cost for Semantic Annotation m5376 T. Kunieda et. al. – Validation experiment work plan from the viewpoint of office applications

41

After the contributions were heard, it was decided to go with (and strengthen) the application experiments. These experiments try to solve a ‘real world’ problem with MPEG-7 technology. The AHG is N3168.

5308 Mikio Sasaki, Katsushi Asami - A Proposal for In-Vehicle Profiles 5309 Mikio Sasaki, Katsushi Asami - Interactive Agent using In-Vehicle Profiles 5310 Mikio Sasaki – Business Model of Interactive Media CommunicationThese interesting contributions describe application of MPEG-7 in a mobile environment, with the aim of prooviding the user with appropriate information. The discussion revealed that while some elements were definitely MPEG-7, other might not belong to the standard. It was suggested that the proponents join an application experiment, to see in how well MPEG-7 matches the problem, and to assess what is missing.

Another piece of application work was initiated to help the Video group in making choices for visual descriptors. The idea here is to describe applications, try to cluster them based on their requirements, and to see whether subsets of MPEG-7 applications might exist that would have different requirements for Descriptors. (AHG N3169)

MPEG-21

New work item / Acceptance criteriaA new work item on what was previously called ‘Multimedia Framework’ was proposed and submitted to SC29 for review and after that to JTC1, the committee which will issue the ballot. In the course of this reviewing, the name was changed to Digital Audiovisual Framework. The request for the new work item and the acceptance criteria are in N3200.

Discussions and contributions5536 Keith Hill - Report of the AHG on Multimedia Framework

Keith explained the status of this work using MP3 as an example. The main point was that users are now starting to be the ones who create the market, not manufacturers anymore.What this means is that we should take more regard of consumer requirements. There is, in the audiovisual field, a huge range of choice – but also a substantial lack of interoperability.

5 infra areas identified: rights management security delivery agents financial

A number of questions were raised: Where does MPEG-7 fit in? metadata, AUV-related agents infrastructure What is the meaning of interoperability? Is it the word that covers MPEG-21? yes, but

together with consumer issues – we must view things from a consumer viewpoint. What would the world look like from this viewpoint?

Remark: thinking about all possible business models is impossible; people will be smarter than we. Any framework must be flexible.

Remark: the consumer doesn’t interact with standards, but with applications. And don’t look at things only from consumer perspective, but do give it a higher priority!

42

Remark: it is impossible to get the consumers themselves in the loop This may be changing with the Internet. Consumers give their opinions every day – you only need to listen and read.

Is content-based billing and accounting in the framework? yes, although currently not under that title.

Consumers want free content. So why should we facilitate anything else? They will not pay anything anyway! Not many people seem agree. Content is almost always paid for, but not always by the end-user, so most people believe in a whole range of business models. Consumers will be willing to pay if the value is good enough.

We should take into account content-related commerce.

5328 J. Kishigami et. al. - A proposal to MPEG21, "the Big Picture" - Part 1 - Purpose of the Content ID and a Comment to the MPEG Player Reference Model5329 Y. Takashima et. al. - Part 2 - Technologies for CID -5330 J. Kishigami et. al. - Part 3 - Requirements on Content ID from Digital Archives' Viewpoint These three contributions presented the work and the viewpoints of the JPese Content ID forum (cIDf)

Goals are:1. declaring copyrights2. obtaining the right content (like ISBN for books)3. collecting content distribution records for marketing goals.4. to refer to USge rules5. to protect from illicit USge6. to validate whether content is legal7. managing editing history8. knowing basic attributes like what is there, what does it cost, etc.9. Categorizing for efficient search. (extensible, well-structured, interoperable)10. automated rights approval and royalty distribution.

The proposal was to add the MPEG-21 reference model an ‘editor’ or a ‘secondary creator’ (a re-user) Keith explained that the model essentially covers this already, because the same people can occupy different roles in the model. He will add text to explain this.

An interesting point was that one of the functionalities in the cIDf model (‘Detection’) would involve a ‘net police’ and a ‘net judge’.

Rapid Content ID standardization is needed, says cIDf. It should be global and there should be protection mechanisms. The rest of the chain (security, facilitating transactions) will follow.The discussion on the link with the DOI (Digital Object Identifier) has recently started. (DOI only does the ID, not payment and protection – the same applies to cIDf.) DOI looks at directory services, maintain metadata associated with content

It is beyond question that the Content ID is important to MPEG-21.

5425 G. De Petris, et. al. – Contribution to MPEG-21: An Integration Attempt of MPEG-4 IPMP, OPIMA and SDMI

43

The contributions explains how the three frameworks, each with their own API’s, can be merged into one model. The picture assumes that there are no downloadable decoders, Why not? Because of security reasons, because presenters do not see the benefit. remark: look at the Internet, this is just the model that is happening. Reply: we want to have one single coding technology (because we and the world know it’s the best) and multiple IPMP technologies (because different industries will have different needs).

5533 Jan Bormans, Gauthier Lafruit - Contribution to MPEG-21: Advanced QoS scenariosAnother piece of the puzzle: a QoS framework. QoS means real-time operation, which has implications for both networks and decoders. Most QoS problems can be solved by trading of decoder power and bandwidth. There is no standardized way of doing that.

Remark: You need to package the content for the network, and the package content for the device. This issue is important for providers. Perhaps we need a negotiation language to achieve this, but we shouldn’t forget no-return channel scenarios.

5627 IEC TC 100 - Liaison from IEC TC 100 on Collaboration with WG 11 (SC 29 N 3370)TC100 works on multimedia equipment. It is structured as follows:TA1 Digital receiving equipment, including the interfacesTA2 Color managementTA3 InfraRed SystemsTA4 Pulse and Digital Systems.(Works on Audio)

SC100B works on all types all types of storage including home servers – they also look at implications of Intellectual property. SC100D works on cable distribution systems.

Other interesting issues are: Dbook – based on DIGITAG work (digital broadcast) Residential gateway

Future work on MPEG-21A call for contributions is not yet appropriate. First we must get a good understanding internally, and then we can talk to other bodies and groups.The New Work Item will be about understanding the framework (the technical report), not emphasizing new standardization items. That may come later, within MPEG or outside of it.The result of the activity must be a Technical Report that:1. Describes the MM framework2. Identifies areas of standardization

The mode of operation is as follows:: Describe problem within MPEG Call for contributions, first internally, then publicly Organize workshops with invited speakers Getting in contact with other parties Make it Official

The starting point is to look at interoperability from the user perspective – and derive the issues from that point of view. Look at the ‘ideal world’ – how would things work? We will describe functionality that is transparent to the user, but perform the same functions as today.

44

The decision is to organize an open (publicly accessible) workshop in Noordwijkerhout, during the MPEG meeting, on Monday after the plenary and Tuesday until 16:00 hours.A number of organizations were identified as important to the workshop; the program will be finalized in the AHG (N3167).

45

Annex 5

Systems reportSource: Olivier Avaro, Chair

OverviewThe main results of the meeting from the Systems Sub-group perspective are:

MPEG-2 Systems

1. Carriage of MPEG-4 on MPEG-2 Transport (ISO/IEC 13818-1/FDAM 7): Production of the final text.

2. FDAM 2 for DSM-CC: Production of the final text.

MPEG-4 Systems Version 1

1. Systems Version 1 Part 1 Study of Corrigendum (ISO/IEC 14496-1/COR1)2. Systems Version 1 Conformance FDIS (ISO/IEC 14496-4 FDIS)


3. Systems Version 2 Part 1 FDAM (ISO/IEC 14496-1/FDAM1)4. Systems Version 2 Part 6 FDAM (ISO/IEC 14496-6/FDAM1)5. Draft MPEG-J Explanatory Report6. Systems Version 2 Conformance (ISO/IEC 14496-4/CD)


7. Request for Amendment for 14496-1 Amendment 2 (BIFS Extensions, Flexible Timing Model)

8. Text for ISO/IEC 14496-1 Amendment 2 PDAM9. DMIF Roadmap


10. Request for Amendment for 14496-1 Amendment 311. MPEG-4 Textual Format (Call for proposal)

MPEG-4 Systems Exploratory Areas

12. Requirements for Multi-users Worlds

MPEG-7 Systems

13. MPEG-7 DDL WD 1.014. MPEG-7 Systems WD 0.1

A detailed report can be found below.

General issuesThe following contributions have been reviewed:

N° Title5294 Systems October 1999 (Melbourne) Meeting Report5484 Proposal on editing procedure of ISO/IEC 14496-1/2/6 FDAM15295 Summary of Voting: ISO/IEC FDIS 14496-1

1. M5294: The report was approved without comments.

46

2. M5484: This issue was discussed in the HOD. The following was agreed : Version 2 will be create as a delta document making the assumption that the corrigendum to Version 1 is approved. Version 1 corrigendum will be integrated in the Version 1 specification. Carsten kindly proposed to be the editor of this new version of the specification (Thanks Carsten !).

3. M5295: The version 1 specification was massively approved.

A joint WG1/WG11 meeting was held on a standard format for sequences of images (from timed slide-shows to higher frame rates), such as might be produced by a digital still camera, possibly with audio annotation. The meeting report can be found in W3151. In order to initialize the collaboration between WG1 and WG11 on this topic, the Systems sub-group proposed a first draft of proposed tools (W3207). This proposal should be tuned to WG1 requirements.

MPEG-2 Systems

Amendment 7The following contributions on the carriage of MPEG-4 over MPEG-2 have been reviewed :

N° Title5405 JNB comments on the study of ISO/IEC 13818-1/FDAM75586 Comments on Draft amendment 7 to ISO/IEC 13818-15526 DNB Comments to the Amendment 7 to ISO/IEC 13818 and the Audio parts of MPEG-

2 and MPEG-4.

As a result the final text of the amendment has been produced.

MPEG-2 DSM-CC

Amendment 2The following contributions on the carriage of MPEG-4 over MPEG-2 have been reviewed :

N° Title5289 Summary of Voting: ISO/IEC 13818-6/FPDAM 2

As a result the final text of the amendment has been produced.

MPEG-4 Systems

Corrigendum 1General

The following contributions have been reviewed :N° Title

5396 Comments on ISO/IEC 14496-1/DCOR 15418 Comments on ISO/IEC 14496-6/DCOR15402 JNB comments on ISO/IEC 14496-1 (Systems) DCOR5555 Incomplete specifications in the Systems FDIS5428 OCR fields in ESD

4. M5396, M5402, and M5418: Several issues discussed and addressed (start time/stop time). Open Close: Instantiation of the node. Relative URLs.

47

5. M5555: Report of incomplete specification from implementation. All discussed and resolved..

6. M5428: Move the OCRstreamflag and OCR_ES_ID fields from SLConfigDescriptor to ESD. Issue resolved.

The conclusion have been summarized and documented in the text of corrigendum 1.

IPMP


5315 Input on the Registration Authority for MPEG-4 IPMP5550 Implementation and synchronization considerations on IPMP and Audio BIFS decoders

embedded in the MPEG-4 System Layer

The IPMP activity is now mostly taking place outside the Systems group since the discussions are essentially discussions on requirement.

Remaining V1 issues were:requirements for the registration authority: the issue is finalized and SC29 can proceed so that the registration authority is set up (see doc. on Registration Authority for MPEG-4 IPMP); the lack of clarity on the usage of the URLs has been resolved and is documented in the V1 corrigendum.

Amendment 1General

The following comments have been reviewed:N° Title

5290 Summary of Voting: ISO/IEC 14496-1/FPDAM 15401 JNB comments on ISO/IEC 14496-1 (Systems) FPDAM5630 Late Comments on ISO/IEC 14496-1/FPDAM 1 (SC 29 N 3371)

The conclusion have been implemented in the final version of the Amendment 1of MPEG-4 Systems.

Elementary stream management

The following contributions have been reviewed:N° Title

5567 Towards a normative ESI?5587 SL layer & ESI interface clarifications5588 Possible improvements to the ESI interface

It’s was clarify the ESI is a conceptual interface and that there were no value in standardizing it.

Scene Description


5598 Study on Systems FPDAM (BIFS part)5579 Specification of an Advanced BIFS node for an Application Window5326 Comments on ISO/IEC 14496-1/FPDAM1

48


MP4

The following contributions on have been reviewed :N° Title

5395 Comments on MP4


MPEG-J

The following contributions on MPEG-J have been reviewed:N° Title

5601 Edit List for MPEG-J part of Systems DAM-15616 Input Draft Text for MPEG-J Explanatory Report

7. M5601 Edit List for MPEG-J subpart of Systems DAM-1)Contribution m5601 was reviewed in detail. Most of the changes requested in the document have been accepted. Most editorial comments about the text is accepted.8. M5616 Input Draft Text for MPEG-J Explanatory ReportContribution 5616 and its update were reviewed in detail. The output document "Draft of MPEG-J Reference Explanatory Report" was produced.The conclusion have been implemented in the final version of the Amendment 1of MPEG-4 Systems.

It was agreed that the JavaStreamHeader description text needs further clarification.Review new API changes based on implementation:

MPDecoder It was agreed that OD and ESDescriptor types will be added to the API for ease of

implementation. Methods to access OD of a Node, and ESIDs of an OD will also be added.

ResourceMagagerExplanatory text about the expiration of non-class Data has to be included.Informative Annex on the usage of the different APIs and accessing a scene has to be included.API descriptions : This primarily involves more descriptions in the API javadocs and introductions. In Net, Cap, Scene APIs missing documentation to be added.

Upstream Management

The following contributions on MPEG-4 upstream management have been reviewed:N° Title

5313 Tutorial example of MPEG-4 backchannel5361 ISO/IEC 14496-1 (Systems) FPDAM-1 ballot comment of BNB5462 Draft text of the informativ annex on MPEG-4 upstream walkthrough

9. M5462, M5361 and M5313: Review of the informative on the upstream channel walkthrough. Clarification with the DecoderSpecificInfo.

Carriage of MPEG-4 on the Internet

49

The following contributions on the carriage of MPEG-4 on the Internet have been reviewed :N° Title

5568 Coding-Tool Configuration Packet for MPEG-4 Video

10. M5568: No one to present the contribution. Basically something that has to be presented in IETF.

Amendment 2General

A new amendment has been initiated supporting the following functionality: Flex time Server Command Media Control Extern Proto Coding of multiple value fieldsAll scene description aspects should be considered for joint specification with Web3D.

Scene description


5339 New BIFS Sensors for V3: refinement and architecture5340 Ideas for other applications and BIFS sensors for V35522 Proposal for upstreamming with ServerCommand node5371 Liaison from Web3D on Proposed X3D SufraceNode for Consideration in Aplication

Texture Discussion (BIFS) (SC 29 N 3349)

11. M5522: No one to present the contribution. Functionality for V3. Should not be limited to index face set. More text is needed. Version 3 : Newpred for BIFS to correct error transmission. Version 3 or Version 4.

12. M5371: Some duplication of functionality with existing specification. Other functionality should be considered for this amendment.

13. M5340, 5339: Proliferation of sensors should be avoided. Only the generic ones were retained in the VM.

FlexTime

The following contributions have been reviewed :N° Title5307 Advanced Synchronization Model (FlexTime Model)5319 Augmenting the TemporalForm node for Intra-Object Synchronization

The text of the Flextime specification was promoted to CD and is now open to National Body comments.

Amendment 3 General

The following contribution has been reviewed :N° Title5621 Requirements for an MPEG-4 Authoring Format

50

As a consequence, a new amendment has been initiated supporting a textual format for MPEG-4. A call for proposal has been issued. The specification will be jointly developed with AIC and Web3D.

Exploratory activitiesMulti-user application

The following contribution has been reviewed :N° Title5623 Requirements on MPEG-4 Multi-users Applications

The activity should be clarified further (AHG activity). The effort will be conducted jointly with Web3D.

Advanced 3D Animation

The following contributions have been reviewed :N° Title5541 Proposal for a packed editing command for multi-valued fields5520 Analysis of existing tools for animated 3D model coding

The activity should be clarified further (AHG activity). The effort will be conducted jointly with Web3D and SNHC.

MPEG-4 DMIF

GeneralThe following contributions on DMIF have been reviewed :

N° Title5591 Proposal for a DMIF Roadmap

14. M5591: Where we are (file, broadcast , interactive), where we go (Conformance, DAI Syntax, QoS architecture, Stream Control, Multiuser, H.245 mapping (Ericson, Bosh, Docomo), MPEG-4 over RTP).

Amendment 1The following contributions on DMIF have been reviewed :

N° Title5293 Summary of Voting: ISO/IEC 14496-6/FPDAM 15590 Update on the new DAI syntax proposal

Comments have been taken into account to construct the final text of the amendment. The DAI Syntax will remain informative.

RSTPThe following contributions on RTSP have been reviewed:

N° Title5581 Activities on RTSP/MPEG4 mapping in IETF MMUSIC5584 MPEG-4 over IP: preliminary results using MPEG/IETF AVT payload and DAI/RTSP

mapping

15. M5581: Short report to make people aware. Already implemented. Is going to be documented.

51

16. M5584: Review the DMIF spec.

The mapping RTSP/MPEG4 is being developed. Possible RTSP changes will be contributed to IETF MMUSIC.

MPEG-4 Conformance

Profiles and LevelsGeneral


5597 Proposals on MPEG-4 Version 2 Advanced AudioBIFS Conformance and Level Definitions

5622 Issues on MPEG-4 Profile and Level Indication5352 Update of level definition for Scene Graph Profiles17. M5352, M5597: Proposal considered and conclusion implemented in the Amendment 1. 18. M5622: Proposal accepted and conclusion implement in both the Corrigendum 1 and the

Amendment 1.

MPEG-J Profiles

The current profiles document was reviewed. The Decoder API was moved from the Interactive Profile to the Personal profile. Decoder Functionality APIs were removed from the Interactive Profile as it is not included in Version-2. The name of the "Interactive" profile was changed to Main Profile.

Conformance Systems Version 1The following contributions have been reviewed :

N° Title5294 Summary of Voting: ISO/IEC FCD 14496-45463 Comments on FCD ISO/IEC 14496-45632 Late Comments on ISO/IEC FCD 14496-4 (SC 29 N 3373)5556 Some minor comments on the Systems Conformance Scene Dump format5327 Im1Player A Bitstream Compliance Verification Tool5341 Update of ENST/”MPEG-4 PC” tools for content creation

19. M5294, M5463, M5632: Served the construction of the final text of the conformance part.20. M5556, M5327, M5341 : The work plan for production of test streams for MPEG-4 Systems

conformance as been updated in N3152. Other companies are encourage to join this crucial effort for the validation of the standard. More test streams can be integrated in the standard in the form of amendment.

Conformance Systems Amendment 1 General

The following contribution has been reviewed and served to update the current text of the Version 2 conformance.

N° Title5337 WD 4.1 of ISO/IEC 14496-4 Amendment 1

MPEG-J API Conformance

52

MPEG-J Part of the Conformance FPDAM was reviewed. The following companies agreed to start the bitstream exchange before March 2000.

Personal Profile:1. Scene (France Telecom, AT&T, IBM)2. Resource (Sun, CSELT)3. Decoder (Sun, AT&T)4. Network (Sun, CSELT)

Main Profile:SI and SF (Philips, CSELT)

MPEG-4 Reference Software

GeneralThe following general issues were discussed: Matthew Leditchke will take over the maintenance of the 3D player. Errors in the script node will be fixed by Conexant.

The usual documents on Im1 status and implementation work plan have been produced and the following demonstrations have been presented:

N Title Author5554 MPEG-4 over IP: preliminary results using

MPEG/IETF AVT payload and DAI/RTSP mappingRoberto Castagno, et al.

5545 Demonstration of Real time Video Transmission System

Shigeru Fukunaga

5608 MPEG-4 Over Multiple Transports/Protocols Michelle Kim, et al.Xxxx Face and Body Animation Tolga Capin and Eric Petajan

Reference Software Amendment 1General

The following contributions have been reviewed and served the construction of the reference software for the Amendment 1.

N° Title5303 Release Notes Update for 3D Mesh Coding Integration into The Pact 3D Player5334 Software Updates of PACT 3D Player with Support for Hierarchical 3D Meshes5445 IM1 Core code + authoring tools version 4.05450 BIFS/OD Encoder version 4.0

MPEG-J

a) List of Implemented APIsIt was agreed that the proposal made in Melbourne by the French National body for node creation through MPEG-J will be accepted as it was implemented. As this functionality was agreed to be very useful, there was consensus in moving these APIs from the VM to the FDAM-1.

53

b) CVS serverUnnecessary files in the CVS server were deleted. This made it easy for accessing these files using a modem. Integration work plan

The integration work plan described in the table below was agreed upon and given to the IM-1 chair to be included in the IM-1 status report and work plan.

c) Core.Integrating the new core (with changes for MPEG-J) to the new 2D Player (to be released soon with Core 4.0) has to be done. This is reflected in the work plan.

d) MPEG-J Demo Work plan: There were extensive discussions on the plan for a demonstration of MPEG-J in the 51st

WG11. Currently there has been some commitment towards this from different companies.

Reference Software Amendment 2Flextime

The following contributions have been reviewed:N° Title5307 Advanced Synchronization Model (FlexTime Model)5319 Augmenting the TemporalForm node for Intra-Object Synchronization

Flextime has been implemented in Im1. Massive changes were made on the core to implement the specification. The Im1 team was too short in time to check the validity of the implementation. The implementation should be validated by next meeting.

MPEG-7 Systems

GeneralThe following contributions have been reviewed:N° Title5376 Validation experiment work plan from the viewpoint of office applications5377 Introduction of Multimedia Document Description Tool and Time Cost for Semantic

Annotation5378 The Need for an MPEG-7 Applications Evaluation Group5379 First Outline of MPEG-7 Vertical Applications Description Tools

The application experiment and evaluation work will be carried out with the requirement sub-group.

First sketch of MPEG-7 Systems have been discussed and are documented in MPEG-7 Systems WD0.1. Contributions are needed to make the work progress.

DDLOverview

Due to the unfortunate lack of manpower the DDL group concentrated its effort on adapting the version 3.0 of the DDL in order to produce the Working draft 1.0. Furthermore during joint

54

meetings, the DDL group provided clarifications of the DDL and hints for a better use of the DDL.

Reviewed documents

The DDL group reviewed the following documents:

N° TitleM5409 MPEG-7 Description Definition Language Document V 3M5424 Status of the MPEG 7 parser, Claude Seyrat (UPMC)

21. M5409: the contribution describes the latest version of the DDL before the MAUI meeting. This document served as a basis for the WD which can be considered as the v4.0 DDL document.

22. M5424: the contribution describes the status of the parser developed at UPMC. The presented parser is conformant to version 2.0 of the grammar. Some v3.0 features were already incorporated. The UPMC will work on making the parser conformant to the WD version of the DDL.

Achievments

The DDL group adapted the DDL grammar in order to include the following features:1. Named DSTypeRef and DtypeRef2. Typed references3. To avoid terminology ambiguities due to the new “named DSTypeRef” feature, the group

changed some keyword names.4. Interpretation of inheritance and refinement rules: The interpretation of inheritance allows polymorphism in description Refinement rules for attributes have been defined

Based on the preceding results, the DDL group produced the following documents:1. The first version of the Working Draft

Comment: In order to avoid any ambiguity in the DDL Working Draft, the Group decided to not integrate any XML schema part. The tracking of XML schema and its mapping should be kept on a separate document. This document could not be finalized during the meeting but should be provided by the next meeting.

2. The DDL part of the “MPEG-7 overview document”

Joint meeting sessions

The DDL group assisted the MDS group during joint meetings on the following subject:1. Reviewed the open issues, provided some solutions, identified areas of future work 2. Provided hints for writing DSs3. Highlighted the interpretation of some of the DDL constructs4. Helped writing correct description schemes during breakout meetings5. Presented the parser availability6. Helped define the architecture of the XM for DSs in order to facilitate the future introduction

of a DDL Parser.

Open issues

Finally the group updated the list of the open issues based on the concerns of the MMDS group:1. Should we specify the query capabilities of MPEG-7 compliant applications ?2. Ability to manage Hierarchical Thesauri

55

3. Mechanism to reference Description Schemes constructs in the description (ex: an enumeration of all the DS names defined in the schema).

4. Matrix and arrays (possibly parameterized).5. Some solutions have been proposed but need to be improved.6. Refinement mechanism of the element model need to be analyzed.7. We need to precise the interpretation rules of description schemes and the mapping to

description.8. We need a clarification of the concept of attribute in the MPEG-7 context.9. The data inheritance issue has been discussed in a joint meeting with MMDS Group. We

conclude that:10. Data inheritance mechanism on elements will not be provided by the DDL.11. Data inheritance mechanism only on attributes becomes a open issue. 12. DDL Group recommends the use of references.Remark: It is not an exhaustive list of open issues (c.f. Working Draft of the DDL)

Future work

Different actions have been identified:1. To continue building the parser;

Update the parser in order to make it conformant to the WD version of the DDL Define DOM API of the Description parser and keep in mind the probable future

integration of the parser in the XM architecture2. To continue assisting and advising the DS and D groups with encoding their Descriptors and

Description Schemes in the DDL and to refine the DDL as required to satisfy their requirements;

3. To continue work on DDL open issues e.g.: representation of large arrays (color histograms) and matrix data inheritance of attributes, links to procedural code;

4. To update the DDL document to reflect changes to the DDL and XML Schema Language between the 50th and the 51th meetings.

5. To continue the liaison with W3C’s relevant working groups. This includes: continued tracking of XML Schema developments; investigation of how to extend XML Schema to satisfy MPEG-7 requirements; input of specific MPEG-7 requirements to the relevant W3C Working Groups, such as

XML Schema, XLink, Xpointer, XPath; dissemination of relevant W3C WG documents back to MPEG-7.

56

Annex 6

Multimedia Description Schemes report

Source: Philippe Salembier, Ana Benitez, Jose Martinez, John Smith, Toby Walker

Overview:The main activity during the week has been to:

1. Review the results of the validation and core experiments started at the Melbourne meeting,

2. Select DSs for inclusion in the XM and in the WD,

3. Clarify the strategy for software implementation and define the corresponding workplan,

4. Review input contributions,

5. Define new Core experiments.

Kick-off of the workDuring the kick-off session, the working strategy and structure of the MDS group was discussed. The Generic AV DS document was decided to be frozen at version 0.8. In the future, the developments of the Description Schemes will be done through core experiments and inclusion of the relevant entities in the XM or in the WD. The Generic AV DS (v0.8, M5380) can still be used for reference but will not evolve anymore. Four break-out groups were created with the following mandates:

Break-out on core-experiment: Analyze experiments results, Make recommendation of entities to be included in the XM or WD, Define workplan for software integration, Define new core experiments (Chairs: Ana. B. Benitez, Toby Walker).

Break-out on meta/media: Analyze and continue the harmonization work with relevant standards, Recommend entities to be promoted to XM and WD (Chair: Jose Martinez).

Break-out on Semantic DS: Make a proposal of improved semantic DS and define workplan for validation (Chair: Riccardo Leonardi).

Break-out on Application: Study the issues related to the testing of the MPEG-7 standard through applications (Chair: Neil Day).

Finally, John Smith was appointed as deputy-chair. His function will include among other things leading the work in absence of the MDS chair and in case multiple meetings.

Review of Validation and Core Experiments resultsNum. Contributions

M5348 Results of Validation Experiment for MPEG-7 Model-DS on Visual DataM5355 The Second Report of the Verification Experiment on MPEG-7 DS from the Viewpoint

of Video Editing and ProductionM5364 Report on "Validation Experiments for Universal Multimedia Access (UMA)", Part 2M5365 Report on Validation Experiment on Ordered Relation GraphsM5414 Results of Spatio-temporal Region DS Core/Validation ExperimentM5415 Results of Validation Experiments on Summarization DSM5419 Results of the Digital Patient Record Validation Experiment for MPEG-7

57

Num. ContributionsM5431 Report of the AHG on MPEG-7 DS Validation and Core ExperimentsM5432 Report on the Validation Experiment on the Syntactic DS, the Semantic DS, and the

Syntactic/Semantic Link DSM5437 Report on Validation Experiments for Universal Multimedia Access (UMA) - Part 1M5440 Report on Core/Validation Experiments of Weight DSM5441 Report on Validation Experiments of Summary DSM5458 Validation Experiment Result Report on Summary DSM5459 Validation Experiment Result Report on Ordered RelationM5460 Report of Validation/Core Experiment on Weight DSM5464 Experimental Result on Weight DSM5468 Cross-Verification Results of Spatio-temporal Region DSM5492 Results of the Validation Experiments on Summary DSM5575 Results of the Validation Experiment on Summary Description SchemesM5576 Workplan and Status of the Validation Experiment on Summary Description SchemesM5605 Validation Experiment on the Ordering Key DS and an Unified Syntax for the Weight

DSM5618 Results of Validation Experiment on Weight DS(DS4: Point of view DS)

All these contributions were discussed in the AHG meeting during the weekend before the meeting and also at the beginning of the week.

VE on Syntactic/Semantic DSs

M5432Report on the Validation Experiment on the Syntactic DS, the Semantic DS, and the Syntactic/Semantic Link DS

A. B. Benitez, D. Zhong, S.-F. Chang, C. Saraceno, M. Kangas, B. J. BrownSummary:Summary: The experiment validated the Syntactic DS (Video Segment DS, Moving Region DS,

and Segment Relation Graph DS), the Semantic DS (Object DS and Event DS), and the Syntactic/Semantic Link DS in a retrieval and visualization scenario. The experiment generated several descriptions of syntactic and semantic elements for video shots in the MPEG-7 content set. It also implemented a system that retrieved video shots based on their syntactic and/or semantic descriptions and visualized the results.

VE on Summarization DS

M5415Results of Validation Experiments on Summarization DSO. Hori, T. Kaneko, K. Yamamoto,

M5441Report on Validation Experiments of Summary DSS. Sull, J.-R. Kim, Y. Kim,

M5458Validation Experiment Result Report on Summary DSK. Yoon, S. B. Jun,

M5492Results of the Validation Experiment on Summary DSM. Kim, J.-G. Kim, H. S. Chang, J. Kim

M5575Results of the Validation Experiment on Summary Description SchemesP. van Beek, I. SezanSummary: The validation experiment validated the Summarization DS, mainly from the point of view of browsing and visualization applications. The experiment generated different instantiations of the Summary DS for material in the MPEG-7 content set and other material, and shot boundary data that was used to generate various summaries. The

58

participants validated the Summary DS on paper, as well as using a software parser and working software systems.

VE on Model DS

M5348Results of Validation Experiment for MPEG-7 Model-DS on Visual DataJ. R. Smith, U. Srinivasan, C. Leung, Summary: The Model DS validation experiment validated Cluster DS, the Examples DS, the Cluster Set DS, Example Set DS, Probability Model Classifier DS, Classifier DS, and Gaussian DS in an image searching and filtering scenario. This experiment generated descriptions instantiating the DSs for the images and labels prepared by the video group for color queries. It has also implemented a demo browsing system based on the descriptions.

VE on Variation DS

M5437Report on Validation Experiments for Universal Multimedia Access (UMA) – Part 1

J. R. Smith, S. Sull, C. Christopoulos, Y. Abdeljaoued, A. B. Benitez, M5364Report on Validation Experiments for Universal Multimedia Access (UMA) – Part 2

A. Perkis, J. Chicharo, S. Jensen, Summary: Part 1 of the experiment validated the fidelity attributes in the Variation DS in a Universal Multimedia Access (UMA) scenario. The experiment generated and collected different audio and video variations for video sequences in the MPEG-7 content set. It also simulated how the selection of a variation of the content could be made to maximize fidelity under different client capability conditions based on the variation fidelity values.Part 2 of the experiment validated the variation type attribute of the Variation DS in a Universal Multimedia Access scenario. The experiment run simulations with visual quality comparisons between different variations of video sequences in the MPEG-7 content set for different client terminal capabilities.

CE on Weight DS

M5440Report on Core/Validation Experiments of Weight DSS. Sull, J.-R. Kim, Y. Kim,

M5460Report of Validation/Core Experiment on Weight DSJ.-S. Lee,

M5464Experimental Result on Weight DSS. Sekiguchi, Y. Isu, H. Nishikawa, K. Asai,

M5605Validation Experiment on the Ordering Key DS and an Unified Syntax for the Weight DSL. Rossi, N. Adami, A. Bugatti, R. Leonardi,

M5618Results of Validation Experiment on Weight DS (DS4: Pointofview DS)K. Emura, T. Munetsugu, Summary: The experiment validated the functionality of weighting schemes for the Generic AV DS and generated the syntax for them. This experiment generated Hint DS as the new primitive DS, which fully supports DescriptorUSgeDS, PointOfViewDS, FidelityDS, and OrderingKeyDS. And, as a DS inherited from Hint DS, Weight DS was generated as the primitive DS due to Weight DS' additional attributes like reliability. The experiment validated the syntax by specifying the DSs in XML schema and by instantiating them. Several implemented systems validated the functionality. There were some concerns some of the specialized Weight DSs overlap with the Summarization DS and the User Preferences DS.

CE on Ordered Relation Graph

59

M5365Report on Validation Experiment on Ordered Relation GraphsA. Divakaran, A. Vetro, K. Peker, H. Sun,

M5459Validation Experiment Result Report on Ordered RelationK. Yoon, S. B. Jun, J.-M. Song, Summary: The Ordered Relation Graph validation experiment validated and studied the utility of order as a normative type of relation and reported on applications using order. This experiment generated descriptions of ordered relation graphs for material in the MPEG-7 content set together with syntactic and semantic descriptions.

CE on Spatio-Temporal Region DS

M5414Results of Spatio-temporal Region DS Core/Validation ExperimentO. Hori, T. Kaneko

M5468Cross-Verification Results of Spatio-temporal Region DS

d).1.1 Yasuaki Tokumo, Shuichi WatanabeSummary: The core experiment evaluated the SpatioTemporalRegion DS in terms of MPEG-7 evaluation criteria in a hypermedia application. It was jointly analyzed with the video group (see the video group report).

AE on Video Production and Editing

M5355The Second Report of the Verification Experiment on MPEG-7 DS from the Viewpoint of Video Editing and ProductionM. Shibata, Summary: The video production/editing application experiment validated the Generic DS from the viewpoint of video editing and production targeting the Desktop Program Production (DTPP) system developed by NHK.

AE on Digital Patient Record

M5419Results of the Digital Patient Record Validation Experiment for MPEG-7P. Porskamp, M. Veenstra, D. Velthausz, Summary: The experiment validated the Generic DS from the viewpoint of a digital patient record application for Gait Analysis, a typical application within the medical domain. The experiment generated the description of a collection of multimedia patient records using the Collection DS, the Semantic DS, the Syntactic DS, and the Syntactic/Semantic Link DS in the Generic AV DS. The existing medical application was redesigned to accept MPEG-7 as the interoperability format between clients-services and services-services.

Description Schemes Promoted to XM and WDAt the Maui meeting, a decision was made on which description schemes (and descriptors) to promote to either Working Draft (WD) or Experimentation Model (XM). The initial decisions were made by the breakout group, which reported back to the main MDS group each day. After discussion, the final decisions were made and approved by the entire MDS group.

Only those description schemes and descriptors defined in the Generic AV DS Version 8.0 [M5380] were considered for promotion. Note, that the Media/Meta description schemes were handled in a separate breakout group. Description schemes that had been proposed earlier but that were not included in M5380, as well as description schemes proposed at the Maui meeting, were not considered for promotion. All such description schemes will have to challenge those in the XM/WD as described in the MPEG-7 development process [N3159].

60

The following criteria were used to decide whether to promote a description scheme to either WD or XM:Two or more independent parties had validated the description scheme in a validation or core experiment reported to the Maui meeting. The validation results showed no significant problems with the description scheme. The description scheme was felt to be stable. For Media/Meta DS the harmonization work was used as a basis for validation. There were several DSs for which the group made a decision that while the DS had not been validated explicitly as such, it had been implicitly validated by other core experiments. In determining whether to promote a description scheme to WD or XM, the group assessed the amount of validation, the stability of the description scheme, and the “centrality” of the description scheme to the MPEG-7 Multimedia Description Scheme. The decision of which description schemes to promote is summarized in the following table. Unless indicated explicitly otherwise, whenever a description scheme has been promoted to XM or WD status, the items (data types, descriptors and sub-description schemes) defined in the same section of M5380 have also been promoted to the same status (XM or WD).

Note: All description schemes promoted to WD are also included in the XM.DescriptionScheme

VerificationStatus

Notes

XM

WD

6bitinteger8bitinteger16bitintegerVectorIntegerVectorRealMatrixIntegerMatrixRealDiagonalRealDiagonalInteger

Verified by: M5348.

Temporal descriptorsTimeDDurationDIncrDurationDRelTimeDRelIncrTimeD

The group made the decision that these Ds are mature enough to promote to WD.

The promotion includes all descriptors and data types defined in sections 3.2 of M5380.

HistogramDEntity and Relation DSInclude: Reference, Entity-Node, DirectionType,Relation, Relation-Graph

Verified in M5432. May need integration with DDL linking.

Temporal LocatingMediaTimeDSTimeDS

Group made the decision that this DS is sufficiently mature to promote to WD.

Does not include the Date DS

Media LocatorsMedia Locator DSVideo Segment Locator DSAudio Segment Locator DSSound Locator DSImage Locator DS

Verified by: M5415, M5441, M5458, M5492, M5575, and M5576.

Does not include the Region Locator (currently in CE).

61

DescriptionScheme

VerificationStatus

NotesX

M

WD

Media Url InlineMedia is not specified

Weight DS Verified by: M5440, M5460, M5464, M5605, M5618:

Only the unified syntax is promoted to WD. The original proposals are promoted to neither XM nor WD.

Segment DSSegment Composition DS

Verified by: M5432, M5355.

The Segment Level D, Segmentation Criterion D, and Segment Hint D have not been verified. Therefore, the WD Segment Composition DS will not include these descriptors.

Video Segment DS Verified by: M5432, M5355.

The Mosiac DS and Editing Effect DS are not included in the WD

Still Region DS Not yet verified, but group made decision to promote to XM.

Moving Region DS Verified by: M5432, M5355.

The Mosaic and Editing Effect DS are not included in the WD or XM.

Audio Segment DS Not yet verified, but group made decision to promote to XM.

The Temporal Sound DS, Audio Spectrum DS, Sound Surrounding DS, Placement DS, and Editing Effect DS are not included in the WD or XM for this DS.

Segment-Relation Graph DS Verified by: M5432, M5355.

The graph has been promoted to XM. However, only the following relations have been included in the XM: the 13 Allen temporal relations.

Summarization DS Verified by: M5415, M5441, M5458, M5492, M5575, M5576

Reference to SegmentReference to Program Has not been sufficiently

verified to be promoted to WD.

62

DescriptionScheme

VerificationStatus

NotesX

M

WD

Summary DS Verified by: M5415, M5441, M5458, M5492, M5575, M5576

Only the following types re be included in the Summary Type: keyVideoClips, keyFrames, keyEvents, keyRegions, keyAudioClips, thumbnail.

Hierarchical Summary DS Verified by: M5415, M5441, M5458, M5492, M5575, M5576.

Hierarchical Summary Level DS

Verified by: M5415, M5441, M5458, M5492, M5575, M5576.

Highlight Level DS Verified by: M5415, M5441, M5458, M5492, M5575, M5576.

All descriptors and attributes in this DS have been promoted to WD, including fidelity.

Highlight Segment DS Verified by: M5415, M5441, M5458, M5492, M5575, M5576.

Sequential Summary DSFrameProperty DS

Verified by: M5415.

Model DS Verified by: M5348.Probability Model DS Verified by: M5348. Does not include all DSs.

Only the Gaussian DS has been promoted to XM and WD.

Gaussian DS Verified by: M5348.Analytic Models DS Verified by: M5348. Only the Linear Classifier

DS was not promoted to WD status. All other contained DSs were promoted to WD.

Cluster DS Verified by: M5348.Examples DS Verified by: M5348.Probability Model Classifier DS

Verified by: M5348.

Classifier DS Verified by: M5348.Cluster Set DS Verified by: M5348.Example Set DS Verified by: M5348.Variation DS Verified by: M5437.Unique IdentifierControlled term

Met/Med harmonization

63

DescriptionScheme

VerificationStatus

NotesX

M

WD

Language entitiesAnnotation DSPerson DS (minimal version)Place DSMedia informationMedia Coding DSMedia Instance DSMedia Profile DSMedia Identification DSMedia Format DS


Creation Meta informationCreation DSClassification DSCreation Purpose DSRelated Material DS


USge Meta informationRight DSPublication DSCost DSIncome DSFinancial DS


Table 1. Description Schemes Promoted to XM or WD.

Structure and Semantic DSsNum. Contributions

M5580 A New Design for Semantic DSM5611 A Useful Generalization of the Syntactic/Semantic Division and its Application to

CollectionsM5593 Experiments for Multiple Level Classification of Visual DescriptorsM5446 MPEG-7 Basic Ontology and Linguistic/Data/Semantic LinksM5475 Thesauri and the Annotation DSM5384 Content DS

M5580A New Design for Semantic DS, A. Murat Tekalp, Rajiv MehrotraOutcome: This proposal discussed the advantages of having a standalone semantic description. In particular, it was claimed that for specific applications, the structure DS was an overhead. The interest of having media locators within the semantic DS was also highlighted. During the discussion the structure of the current Semantic was criticized and missing functionalities were presented. As a result, the proposers were encouraged to participate in the breakout group on Semantic DS to redesign the DS and to plan for experiments for validation.

64

M5611A Useful Generalization of the Syntactic/Semantic Division and its Application to Collections, Hawley K. Rising III, Ali J. TabatabaiOutcome: This contribution made an analysis of the structure and semantic DS from a theoretical viewpoint. The review focussed on the notions of TOC and Index, recommended to change the names of the DS and to add some constraints. As several validation experiments, this proposal was recommending the inclusion of semantic notions in the structure DS. Concerning the Semantic DS, the proposers were also encouraged to joint the corresponding break-out group.

M5593Experiments for Multiple Level Classification of Visual Descriptors, Alejandro Jaime, Corrine JorgensenOutcome: This contribution was highlighting the necessity to support a broad range of semantic descriptors and proposed a 10 level structure of concepts. Several experiments of indexing using these 10 level concepts were presented. The conclusion was that the classification corresponding to the 10 conceptual levels can accommodate almost all attributes and can guide human indexer. There was no request of specific syntax change but it is proposed to use this work as guideline to review the Semantic DS.

M5446MPEG-7 Basic Ontology and Linguistic/Data/Semantic Links, HASIDA KoitiOutcome: This document proposed a complete reorganization of the structure and the semantic DSs. It was argued that instead of a dichotomy there was a hierarchical relation between structure and semantic. The proposed hierarchy was defined as:

linguistic entities strutural entities semantic entities.It is a rather radical reorganization of the Structure/SemanticDS. Propose also to define a linguistic DS. Further study is required to see the implication of this proposal.

M5475Thesauri and the Annotation DS, Eric RehmOutcome: This proposal discussed the interest of having a controlled vocabulary. A modification of the Annotation DS was defined to use the annotation as free text or as thesaurus with a string. This modification was also proposed for the classification DS. The group agreed that this was indeed a good improvement of the DS. The thesaurus linking mechanism may be improved in the future.

M5384Content DS, Jose M. MartinezOutcome: This text was extracted from an initial specification of the Meta information DS and was forwarded to the semantic break-out for contribution to the definition of the core experiment on the semantic DS.

Met/MedNum. ContributionsM5385 Report of the AHG on Media and Meta DSs and Harmonization with other Schemes, M5387 Current status and open issues of the MedMet DSsM5474 Related Material DSM5478 System vs. Component Format/Coding in the Media Profile DSM5386 First attempt on MPEG-7 harmonization with SMPTE

65

M5387Current status and open issues of the MedMet DSs, Jose M. Martinez (on behalf of the MedMet AHG)Outcome: the current status and open issues regarding the MedMet DSs were presented as it was the starting point for the MedMet Break-out Group work. The MedMet DSs were upgraded during the Break-out Group work and, based on the harmonization work, promoted to XM and/or WD (the MetaMeta DS was removed).

M5385Report of the AHG on Media and Meta DSs and Harmonization with other Schemes, Jose M. Martinez (on behalf of the MedMet AHG)Outcome: the document presents the report of the work carried out within the MedMet AHG between Melbourne and Maui.

M5474Related Material DS, Eric RehmOutcome: the document presents an upgrade for the Related Material DS within the Creation MetaInformation DS. It was used to upgrade the resulting Related Material DS promoted to WD during the Maui meeting.

M5478System vs. Component Format/Coding in the Media Profile DS.Eric RehmOutcome: the document presents an upgrade for some components of the Media Profile DS within the Media Information DS. It was used to upgrade the resulting Media Profile DS promoted to WD during the Maui meeting.

M5386First attempt on MPEG-7 harmonization with SMPTE, Mike Cox, Jose M. Martinez, Peter MulderOutcome: the document presents the results of the work carried out within the MedMet AHG between Melbourne and Maui for harmonization with SMPTE. These results were proposed, accepted and therefore used as Validation Procedure for the MedMet DSs in order to be promoted to XM and/or WD during the Maui meeting.

New DSs and RelationsNum. ContributionsM5346 An Analytical Description Scheme for Video Editing WorkM5481 Weight DS Extension for Affect-based Content CharacterizationM5610 A Proposed Structure for the KeyFrame DSM5367 Properties of Relations in Entity-Relation Graphs

M5346An Analytical Description Scheme for Video Editing Work, Rosa Ruiloba, Philippe Joly, Cedric Thienot, Claude SeyratOutcome: The proposal targets the detailed description of the editing work. It makes a review of the editing process: Rushes, Transition effects, composition effect (inclusion of video in video, screen split in two component, text overlay). The group discussed the level of details that is needed for MPEG-7. Maybe a full description of the editing work goes beyond the scope of the standard. The proposal is not fully ready for integration and further work is needed. NHK proposed to collaborate with the proposers in this issue.

M5481Weight DS Extension for Affect-based Content Characterization,

66

Y. Shibata, Y. Takahashi, M. Kamada, E. Osawa, et al.Outcome: This contribution proposed an affect-based content characterization. The goal is to be able to describe affective information about content (based on human content interaction). The application to trailer creation has been discussed. The weight DS syntax is already accommodating this kind of functionality. A core experiment will be proposed (N3120).

M5610A Proposed Structure for the KeyFrame DS, Ali J. Tabatabai, Hawley K. Rising IIIOutcome: This proposal discussed the need of a flexible mechanism to attach signal-based and semantic based attribute. For the key-frame notion, it was proposed to create a primitive DS to be used in several places of the Generic AV DS. Moreover, flexibility on the attributes could be provided by KLV. Finally, a dynamic binding can be based on a Command DS. The command DS may be used to update the description. Two functionalities: command DS and runtime instantiation have system implications and were further discussed jointly with the system group. The conclusion was to further study this issue within the framework of the MPEG-7 System AHG.

M5367Properties of Relations in Entity-Relation Graphs, Ajay Divakaran, Ana Belen BenitezOutcome: Propose to standardize a few basic mathematical relations for the entity-relation graphs. The discussion raised the issue of the interest of normalizing just the properties and not the exact semantic of the relations. A methodology has to be found to define the minimal list of relations to be made normative. The proposer were encouraged to continue the work on this issue.

Model DSNum. ContributionsM5488 Considerations on Structure of Synthetic Model DS for Describing Human Body ObjectM5603 A Refinement on Probability Model DS

M5488Considerations on Structure of Synthetic Model DS for Describing Human Body Object, Han-kyu Lee, M. Kim, J. Kim, N.-K. Kim,H.-K. KimOutcome: This document proposed a unification of the two proposals made at Melbourne for Synthetic model DS. New DSs have been specified and the descriptors have been removed from the initial proposal presented in Melbourne. More work is needed to develop a full specification of the sub-DSs (attribute, Ds, DS and relations). Proposers are encouraged to work on these aspects. If video Ds are involved, the proposers should have them evaluated in the video group first. Furthermore, proposers were asked to consider contributing to new entities to the conceptual model.

M5603A Refinement on Probability Model DS, A. Aydin Alatan, Ajay DivakaranOutcome: Proposal of new model DSs and correction of a few mistakes found in the Generic AC DS document. A validation experiment has been launched at this meeting (N3119).

67

Summarization DSNum. ContributionsM5434 Consideration on MPEG-7 Summary DS for Audio ContentsM5435 Improvement on the Summary DS for Collection of SummariesM5442 Improved notion of the fidelity for efficient browsing, M5451 Searchable and Presentable Aspects of SummaryDSM5493 An efficient Hierarchical Summary DS for event based summarizationM5566 A proposal for the modification of the Sequential Summary DS

M5434Consideration on MPEG-7 Summary DS for Audio Contents, Masaru Sugano, Yasuyuki NakajimaOutcome: This document proposed a set of summary types for Audio, that have direct mappings to video summary types: Thumbnail, Key words/notes/sounds, Key audio clips, Key Events, Key Streams, Key Audio Objects, etc. The proposers were recommended to join the summary core experiment (N3121). It was also suggested to work towards the validation of audio-visual data (i.e., news video) rather than pure audio.

M5435Improvement on the Summary DS for Collection of Summaries, Masaru Sugano, Yasuyuki NakajimaOutcome: This contribution discusses ways to describe a collection of summaries. The drawbacks resulting from the use of the current DSs included in the summarization DS were highlighted: redundant descriptions: duplication of summary type, parsing efficiency: the whole description has to be parsed. It was suggested to introduce the CollectionSummaryDS. The group recommendation was pursue this work and to create a validation work dealing with all aspects of the SummaryDS (N3121).

M5442Improved notion of the fidelity for efficient browsing, Sanghoon Sull, Jung-Rim Kim, Yunam KimOutcome: The main goal of this contribution was informative. It discussed scalable summarization using the notion of fidelity. It did not request any syntax change of the Summarization DS since this functionality was already taken into account in the last validation work.

M5451Searchable and Presentable Aspects of SummaryDS, Dulce Ponceleon, Nevenka DimitrovaOutcome: This paper presented several possible generalization and improvements of the summarization: textual information should be included (summaries should go beyond AV), Time-scale modulation (fast/slow audio), synchronization of audio and slide show, etc. A large number of these improvements were judged as being interesting. The extension to search functionality however was found to be out of the scope of this DS. The proposers were encouraged to join the CE on Summarization (N3121).

M5493An efficient Hierarchical Summary DS for event based summarization, Munchurl Kim, Jae-Gon Kim, Hyun Sung Chang, J. KimOutcome: This contribution presented the drawbacks of the current hierarchical summary DS for event-based description. Two minor modifications are proposed and core (mini) experiment was proposed (N3121).

M5566A proposal for the modification of the Sequential Summary DS, Yousri Abdeljaoued, Touradj Ebrahimi, C. Christopoulos

68

Outcome: This contribution proposed the addition of data significance value to the sequential summary DS. This modification was proposed to support the adaptation of the number of frames, images, or audio-clips according to the user preferences or the client device capabilities in addition to the smart fast playback already supported. The recommendation of the group was to join the core experiments for summary (N3121).

Joint Meeting with the Requirements groupNum. ContributionsM5344 Report of the Ad-hoc Group on a Generic MPEG-7 DS Conceptual ModelM5343 MPEG-7 Description Schemes Principle Concept List (V0.4), M5614 An update of the Mathematical model for Description SchemesM5308 A Proposal for In-Vehicle ProfilesM5309 Interactive Agent using In-Vehicle ProfilesM5310 Business Model of Interactive Media Communication

M5344Report of the Ad-hoc Group on a Generic MPEG-7 DS Conceptual Model, John R. Smith, Hawley K. Rising III, Uma Srinivasan

M5343MPEG-7 Description Schemes Principle Concept List (V0.4), John R. Smith (on behalf of MPEG-7 C. Modeling AHG)Outcome: The AHG has maintained and updated the list of principle concepts. The group recommends to continue the work on this list. The AHG also investigated various modeling approaches and concluded that the Extended Entity Relation was the more appropriate (however, extension will be needed for spatio-temporal relation). In the future, the principle list will be used to define useful "package" of DSs and to create relations between DSs and application. Also it will be used as a basis for the definition of the MDS section of the standard. It was decided to make the principle list a public output document.

M5614An update of the Mathematical model for Description Schemes, Hawley K. Rising IIIOutcome: This document addresses the problem of imprecise definitions of DSs from a theoretical viewpoint. The approach is to work from a set of primitive concepts, and by means of careful definitions, define what is meant by some of the concepts used in describing DSs.

M5308A Proposal for In-Vehicle ProfilesMikio Sasaki, Katsushi Asami

M5309Interactive Agent using In-Vehicle ProfilesMikio Sasaki, Katsushi Asami

M5310Business Model of Interactive Media CommunicationMikio SasakiOutcome: The three documents were jointly presented. The main objective of the application is to allow the safe and convenient access to the information while driving a vehicle. The proposal presented many aspects of the entire system. During the discussion, the group felt that a large part of this application was going far from multimedia content description and was out of the scope of the standard. The proposer were encouraged to launch an application Experiment to further study the relation between the In-Vehicle application and the MPEG-7 standard.

69

User preferencesNum. ContributionsM5336 Report of AHG on User Descriptions in MPEG-7M5335 User Descriptions in MPEG-7

M5336Report of AHG on User Descriptions in MPEG-7, I. Sezan and K. Yoon (Chairmen)

M5335User Descriptions in MPEG-7, I. Sezan and K. Yoon (On behalf of AHG on User Desc.)Outcome: These two contributions presented the main recommendations of the AHG as well as a precise specification of the User preferences DS. It was agreed to start a validation experiment at the Maui meeting (N3128).

Demo on Non normative toolsNum. ContributionsM5377 Introduction of Multimedia Document Description Tool and Time Cost for Semantic

Annotation

M5377Introduction of Multimedia Document Description Tool and Time Cost for Semantic Annotation, Yuki Wakita, Takayuki Kunieda, Neil DayOutcome: This contribution demonstrated a non-normative tool for indexing of video shots. It also assessed the time needed to instantiate the equivalent of the structure DS: the indexing time corresponds roughly to 30 times the duration of the original sequence! Moreover, the description of a 16min movie is 385 Kb in XML format. The author concluded that new tools are needed to have an efficient indexing. In the future, they will develop such tools and use an MPEG-7 compliant description.

Joint Meeting with IPMPDuring the meeting, the IPMP people reviewed and refined the requirements they planned on proposing to the Requirements group for MPEG-7. Some of the IPMP concepts and their possible modeling using DSs were discussed. It was concluded that the IPMP group needs to contribute their expertise in developing IPMP-related DSs. This work will be carried out within the framework of the IPMP AHG.

Joint Meeting with System (and Requirements)Num. ContributionsM5516 Mechanism for Checking Link Validity in General AV DSM5378 The Need for an MPEG-7 Applications Evaluation GroupM5379 First Outline of MPEG-7 Vertical Applications Description Tools

M5516Mechanism for Checking Link Validity in General AV DS, Sanghoon Sull, Sangwook Oh, Seong Soo ChunOutcome: The group felt that the solution proposed by this paper to check the validity of a link was a good starting point. However, it was concluded that further work is required on the linking issue and the validation mechanism. This will be done in the AHG on linking (N3217).

70

M5378The Need for an MPEG-7 Applications Evaluation Group, Neil Day, Takayuki Kunieda

M5379First Outline of MPEG-7 Vertical Applications Description Tools, N. Day, T. Kunieda, Y. Wakita, N. Takahashi, M. IwasakiOutcome: These proposals discussed the need to validate the work in MPEG-7 with real and complete applications. It was concluded that an AHG should be created on this issue (N3169).

Joint meeting with ISG and DDL on Software integration

The issue of Software Integration Policy for MPEG-7 DSs into the XM was discussed during several joint meetings. The XM software aims to be a set of normative and non-normative components for validating the different parts of MPEG-7 standard such as Ds, DSs, and the DDL. Currently, only video descriptor tools have been integrated into the MPEG-7 XM software [N2964]. For the integration of MPEG-7 DSs into the XM software, the following considerations have been taken into account: (1) a DS is specified using the MPEG-7 DDL; (2) the instantiations of a DS are stored as XML documents; (3) the instantiations can have complex graph-like structures; and (4) the description parser would provide a sophisticated API (extending XML DOM) to access the description data in memory.

The main points of the policy for integrating MPEG-7 DSs into the XM software are summarized below:1. The normative component of a DS is the DDL specification of the DS and its components.

There is no normative software for a DS because the Description Parser API manages the description data. Therefore, individual C++ classes do not need to be created for each DS.

2. The non-normative software for a DS consists of, at least, one extraction and one matching application as for Ds. Extraction and matching tools would populate or access the description data in memory using the description parser API. The description parser is not going to be available until the next MPEG meeting. Therefore, an XML parser with a DOM API will be used temporarily, to be replaced later by the description parser API (which will very likely be a superset of the XML parser DOM API). The following software components will have to be integrated for each extraction and matching tool: 1. C++ classes conforming to the interface in the XM architecture and2. the application that uses these C++ classes.

The schedule for integrating MPEG-7 DSs into the XM software has been divided into three stages which are summarized below:1. Initial guidelines for integrating MPEG-7 description schemes into the XM software will be

distributed on MPEG-7 DS CE and the general XM reflectors.2. Several example description schemes will be integrated into the XM software in order to

complete the guidelines for this process. The Annotation DS and the Video Segment DS are two of the candidates.

3. Proponents will integrate their extraction/matching tools into the XM software based on the guidelines.

More information about the policy and the schedule for integrating MPEG-7 DSs into the XM software is included in [N3115].

New Core & Validation experiments:

71

The following experiments were approved during the Maui meeting:Core

ExperimentDescription

SchemesParticipants Summary of

Experiment.N3127: Work Plan for Validation of the RegionLocatorDS

Region Locator DS MitsubishiSiemens

Validating Region Locator DS using hyperlinking of MPEG-7 information with image or video data

N3130: CE on Media Transcoding Hint DS

Media Transcoding Hint DS

Columbia UniversityIBMUni. of WollongongNTNUKR UniversityEricssonSony

Variations of content are delivered depending on network conditions and/or user requests based on the utility function and scaling profile descriptors in the Media Transcoding Hint DS.

N3126: CE on the Segment Hint DS

Segment Hint DS IBMKR UniversityColumbia University

Visual content using hints provided by Segment Hint DS. Evaluation of improvement of trans-coding with and without segments hints.

N3125: CE for Mosaic DS

Mosaic DSMosaic & Warping Parameters DS

EricssonEPFLHHI

Video summarization using mosaic and mosaic-based representation for video editing

N3123: CE on the Semantic DS

Semantic DSObject DSEvent DSEvent-Object Relation GraphStructure-Semantic Relation Graph

Columbia U.KodakElectrotechnical LabHitachiSony, USSUNY Buffalo, U. of BresciaU. of Rochester

Demonstrate the effectiveness of the proposed semantic DS, represent multiple descriptions of the semantic content associated with multimedia documents of different categories

N3124: CE on the Space&Frequency View DS

Multi-resolution Level DSSpace Frequency View DS

IBMToshibaSiemensColumbia University

Progressive and interactive retrieval in which views of image and video data are retrieved by the user.

N3121: CE on Summary Description Schemes

Sequential Summary DSFrame Property DSHierarchical Summary DS

ToshibaSharp Labs of AmericaIBM AlmadenKR UniversityEPFLEricssonKDDETRILG-CITMatsushitaPhilips

Quick audiovisual discovery, fast browsing, navigation, smart playback of audio, image and/or video content

72

Core Experiment

DescriptionSchemes

Participants Summary of Experiment.

N3119 CE on Probability Model DS

Probability Model DS including: Gaussian DSHigher-order Gaussian DSHMM State DS,Discrete Probability DSAnalytic Model DSCluster DSExample DSProbability Classifier DS

IBMNJITMitsubishiSony

Searching and filtering scenario in which video and collections of images are searched by content and are classified into a set of classes

N3122: CE on Collection Structure DS

Collection Structure DS(Annex of M5380)

Columbia UniversityIBMSonyTelematica Instituut

Browsing of images and syntactic/semantic categories based on descriptions of an image collection by the Collection Structure DS.

N3120: CE on the Weight DS

Subjective DS (Within Weight DS)

SonyLG CITKR UniversityMitsubishiMatsushitaUniversity of Brescia

Summaries will be generated for several movies based on video segment weights obtained through the affect-based content characterization of the movies

N3128: CE on User Preferences

User Preferences DS

NTNU/UofWLG-CITMatsushitaSharpKR UniversityEricssonEPFL

Delivery of summaries to users based on communication bandwidth, resources, and user preferences specified using User Preference DS.Filtering, user identification, delivery of summaries, among others, in an AV management system with persistent storage based on descriptions of User Preferences DS.

N3129: CE on the Media Title DS

Media Title DS Sharp Labs of Americasingingfish.com

On-paper validation of the Media Title DS expressed in MPEG-7 DDL. A demo system implementing the DS may be provided.

N3131: CE on Measuring Entropy for Descriptors and Description Schemes

MPEG-7 Ds and DSs

SonyUniversity at Buffalo, State University of New York

Measure the entropy on Ds in a classification test to assess quality for MPEG-7 Ds and DSs.

N3132: MPEG-7 Application

Generic DS NHKTelematica Instituut

Video production/editing application

73

Core Experiment

DescriptionSchemes

Participants Summary of Experiment.

Experiments Ricoh Company, Ltd Digital patient record applicationOffice application

Table 2: Description Schemes in Core Experiments.

List of Output documents:Number TITLE

3112 MPEG-7 Multimedia Description Schemes XM3113 MPEG-7 Multimedia Description Schemes WD3114 Supporting information for MPEG-7 Description Schemes3115 MPEG-7 Software integration policy & Workplan for DS integration3116 MPEG-7 Principle Concept List 3117 Mathematical Models for Description Schemes Concepts3118 Multimedia Description Schemes FAQs3119 Core Experiment on the Probability Model DSs3120 Core Experiment on the Weight DS 3121 Core Experiment on the Summary DS3122 Core Experiment on Collection Structure DS3123 Core Experiment on the Semantic DS3124 Core Experiment on Space and Frequency View DS3125 Core Experiment on Mosaic DS3126 Validation Experiment on Segment Hint DS3127 Core Experiment on Region Localization3128 Core Experiment on User Preferences3129 Core Experiment on the Media Title DS 3130 Core Experiment on Media Transcoding Hint DS3131 Core Experiment on Measuring Entropy for Descriptors and Description Schemes3132 MPEG-7 Application Experiments

List of AHG related to the MDS activities:Number TITLE3133 AHG on Editing the Multimedia Description Schemes XM and WD3134 AHG on MPEG-7 Media/Meta DSs and Harmonization with Other Schemes3135 AHG on MPEG-7 DS Validation and Core Experiments 3136 AHG on DDL integration of MPEG-7 Ds and DSs3137 AHG on Conceptual Modeling 3138 AHG on MPEG-7 Semantic information representation3139 AHG on User Preferences 3217 AHG Linking3216 AHG on MPEG-7 Systems

Next targets for the Noordwijkerhout meeting:

74

Perform the software integration of the DSs that have been promoted during the Maui meeting to XM and WD.

Perform the Validation and Core experiments approved during the Maui meeting and review the results.

Continue the Met/Med harmonization work with relevant standards. DDL integration of audio, visual and MDS entities. Update the XM and WD. Define and plan new core experiments

75

Annex 7

Video Report (MPEG-2 and MPEG-4)Source: T. Sikora, Chairman MPEG Video Subgroup

MPEG-2Content Data Extension (Jeff Mc Veigh, Patty Gray)

Tasks to be completed during the 50th meeting in Maui:1. Response to the JNB contribution requesting that the work on the amendment not

continue to use the extension start codes.2. Study of the existing data types to determine if they were properly defined.3. Determination of the readiness of the working draft to advance to PDAM status.

Results of the work done by the break out group:

In response to the JNB contribution, we developed an alternative syntax to be used in the reserved area of the picture_header known as extra_information_picture. Although the use of a reserved extension start code is technically preferable, using the extra_information_picture did produce a sound solution. This alternative also allows manufacturers of the non-compliant products some additional time to decide how to correct their problem. It is recognized, however, that the use of extra_information_picture may cause other arguably compliant decoders to encounter failures as the specification clearly states that the extra_bit_picture bit must be set to '0'. Jeff McVeigh (Intel) generated test bitstreams that set this bit to '1' and uploaded them onto a public website. A more complete set of test bitstreams with examples of the data types and payloads defined in the PDAM document will be generated and made available in approximately one month.

The study of the existing data types resulted in the removal of two data types, a refinement of the definition of the remaining four, and the addition of one. The data type known as edit_timecode was removed as no consensus or clear set of requirements from the production community could be obtained. SMPTE, meeting during the same week as MPEG, was actively discussing timecode issues and hope to have a draft of a timecode proposal during the next few months. We felt that we should proceed with the other data types in this amendment as a finalized timecode proposal and requirements may require up to one year to define. The repeat_frame data type was removed because it could not be encoded in a backward compatible way for existing decoders. The data type, padding_bytes, was added to allow for an encoder to add blank bytes into the picture header to be used during the VBV calculation, and then filled in later in a subsequent encoding pass.

Towards the end of the week, almost all active participants felt the draft had reached a stable state and that it should be advanced to PDAM status. This included Tahara (Sony) who represented the JNB and their concerns.

Special thanks to Aharon Gill (Zoran), Katie Cornog (Avid), and Katsumi Tahara (Sony) for their time and expertise.

Current status: AHG has been started and the reflector is active.

76

Intel's MPEG decoder has been modified to correctly parse the data proposed in the PDAM.

Intel's development of a PC-based tool to insert the new syntax into an existing DVD MPEG-2 bitstream is nearly complete and will be used at Warner Bros to execute an experiment during Q1'00.

Output for Maui- Justification for MPEG-2 Video Elementary Stream supplemental information- Text of ISO/IEC 13818-2.2/PDAM1

Schedule for development- PDAM1 Dec 1999- FPDAM1 Mar 2000- FDAM1 July 2000- AMD1 Oct 2000

MPEG-4 Version 1Version 1 Conformance (Michael Frater)

The main task for the conformance of version 1 is to process the ballot comments and to finalize the bitstreams. At this meeting we processed and accepted all comments from the National Bodies of Poland, SG, JP, Germany and documented the disposition of comments as an output document.

The Conformance text was edited and approved with an additional period of 4 weeks from the end of the Maui meeting set aside for editing.

On the issue of the missing bitstreams, the remainder volunteers were identified and the following parties shall create the following streams and deliver them before the 15 th of January 2000.

Sarnoff & TI, Still Texture Streams SS-8 to 13 GI, GE-1 to 12, 14, 18 to 25 Sony (nee EPFL), SH-1 to 6

Version 1 Corrigendum 1 (Guido Heising)

The main task for the version 1 corrigenda was to process all the NB comments and related comments. This task was complicated by the fact that the Version 2 Text will go the FDAM at this meeting and will incorporate the COR1 changes. Therefore at this meeting we did the following.

We processed the comments from the National Body of Germany and JP. There were a total of 14 editorial and 1 technical comment from Germany and 2 editorial and 3 technical comments from JP. All of which were accepted with three accepted with modification.

We also reviewed the related comments from the input documents m5347 (low latency sprites and greyscale) m5373 (vop_coded, VCV) m5570 (mi in VBV, t0) m5485 (higher layer syntax in VBV)

Finally we review the DCOR to identify editorial errors.

77

From this we produced and approved the Study of ISO/IEC 14496-2/DCOR1 as an output document at this meeting. The editing of the PDAM shall take into account of the DCOR1 as well as this document.

MPEG-4 Version 2Version 2 Video (Yuichiro Nakaya)

At this meeting a total of 5 new video profiles ACE, ARTS, Core Scalable, Advanced Core, Advanced Scalable Texture were added (there are the other 3D Mesh and Hierarchical 3D Mesh SNHC profiles) to the Version 2 standard.

The Video group reviewed the list of companies that have provided Patent Statement to the Convenor and believes that the list is adequate.

The Video group also addressed the NB Ballot Comments. We received comments on the natural video from Belguim, Germany, JP, KR and US. The comments were processed and a Disposition of Comments was produced at the meeting.

The Issue of JNB Comment on the document editing procedure (m5484) was also discussed in great lengths. It seems that there will be an inconsistency in the text of the Version 1 and Amendment since the COR1 only applies to the IS and not the Amendment and the Amendment contains the uncorrected version text. The final conclusion is to promoted the ISO/IEC 14496-2 FPDAM1, which is a replacement document to ISO/IEC 14496-2 FDIS, to FDAM1 introducing the corrections of ISO/IEC 14496-2 COR1.

Version 2 Conformance (Sven Bauer)

The conformance text is in a very final stage. At this meeting, it was only necessary to do minor editing and add the conformance of the newly added profiles. The Video group approved the Text of ISO/IEC 14496-4/PDAM1 and a Schedule for Bitstream Creation was established.

The following companies has volunteered to create the necessary bitstreams between now and the next meeting:

NTT, Hitachi, Uni Hannover, Bosch, Fujitsu, Siemens, HHI, Oki, Samsung, Sharp, Sarnoff, Sony, Optivision.

Version 2 Software Reference Model (Eishi Morimatsu/Norio Ito)

The Ad Hoc group on the software reference model integration and maintenance reported that all tools OBSS, CGD, ST, NewPred and MAC has been integrated. The group is currently verifying combination of tools. Some examples are MAC tools, 1/4 Pel and OBSS, DRC and OBMC.

There were a total of 152 test bitstreams defined, more than 80% generated and crossed checked.

Besides that the video group also went through the list of bugs that were found in version 1 and has identified parties that will work on them.

A schedule for clean up and final bitstream checking was defined at the meeting and the work will be carried out between now and the next meeting.

VM Editing (Shigeru Fukunaga)

78

VM 14.2 (m5477) was produced prior to the MAui meeting and was submitted as an input document. At the meeting the document was worked on and now most of the DCOR revisions are added into the VM. The video group approved a new release of VM 15.0 at this meeting and further development is expected until the next meeting.

MPEG-4 Version 3 & 4Version 3 Studio Profile (Yoichi Yagasaki)

The ad hoc meeting was held on Sunday evening and break out meeting on Tuesday afternoon.The output document of Maui meeting is Working Draft 3.0(N3094). The updated part of WD3.0 from WD2.0 is introduction of uncompressed mode.

WD3.0 also has annex H (Study for PDAM) which consists of Refinement of Multiple Auxiliary Channel Switching VLC Table Coding Tool Initial Draft of Core Studio Profile

The issue of the ITU liaison on the standardization of the IDCT was also discussed. It was determined that the work currently being undertaken in MPEG and ITU are sufficiently different.

The current scheduled plan for this amendment is PDAM in March 2000.

Fine Granularity Scalability (Jens-Rainer Ohm)

From the Core Experiments performed in Fine Granularity Scalability, improvement in visual quality was observed by application of frequency weighting and selective enhancement. These elements were included in the FGS WD 3.0, which was issued at Maui. Until Noordwijkerhout meeting, further experiments are planned, which are related to

- combination of FGS with Temporal Scalability, still requiring verification of syntax elements;

- further improvement in coding efficiency by entropy coder optimization and enhancement layer prediction.

A workplan was set up, by which it can be expected that FGS syntax will be stable in Noordwijkerhout, such that a PDAM could be started in March 2000.

Visual quality evaluation showed that FGS is in average (over a large range of instantaneous network rates) comparable to a simulcast solution, while under dynamic network conditions (changes in network rate, errors) FGS quality is better. It was decided to further explore the dynamic behavior, especially by using longer sequences and better error models.

Output at this meeting - WD 3.0 for FGS- VM 3.0 for FGS- Core Experiment Description for FGS

MPEG-4 GeneralEncoder Optimization (Tihao Chiang/Huifang Sun)

79

Fast Motion EstimationOn the request by the SG National Body, it was decided not to include the description in FDAM1 as it is felt that the proposed MPEG-4 Technical Report on non-normative tools is a more appropriate place.

There will be a case study in the Ad Hoc on the proposed MPEG-4 Technical Report on non-normative tools. Both the proposals will be used.

DS was removed from the VM15.0 specification as it was suggested by ISG since it was proven to be inferior to the MVFAST and AZS techniques. However, it should be noted that there were objecting to doing so since there is now a mismatch between the VM text and VM software.

A new document was created to document the Case Study for Technical Report on Non-Normative Tools.

Remaining Items1) On the Integer IDCT, the code has been ported from MPEG-2 reference software to

MPEG-4 codec. Mitsubishi and NCTU will bring the verification next time. Note that this is an implementation of a non compliant IDCT and therefore we need to consider this issue with regards to the requirements of Part 7.

2) On the Global motion estimation contribution, the proposal has no verification and is asked to bring one next time.

3) On the B-VOP rate control, Samsung has agreed to continue investigating the bug that stall them. More volunteers are solicited.

It was agreed that all these 3 items above were still in the early stages and no Core Experiments are defined at this meeting.

MPEG-4 - Technical Report

At this meeting the HoD group raised the proposal that Consideration be given to the publication of a technical report as part 7 of ISO/IEC 14496 for the purpose of describing non-normative tools that offer significant enhancements to the performance of software contained in part 5.

Some questions were raised: Donation in the form of source code modules. API must be defined to fit into the part 5 reference software. Significant improvements must be demonstrated. Should be on non-normative parts of the standard i.e. must not change the normative parts of

the standard and should be compliant. Optimization of an implementation which is compliant to the standard is acceptable.

Multiple tools possible? How to decide when this is allowed? Copy right issues.

An Ad Hoc group was created to study the feasibility and process.

MPEG-4 General

80

Video Resolution 2.4.4 (Melbourne Meeting)Based on the above resolution a Call for Evidence Justifying the Testing of Video Coding Technology issued at this meeting. The following areas are identified as possible but not the only areas where comparisons are sought: Low and Medium Bitrate Coding Efficiency Content Based Coding Spatial and Temporal Scalability Robustness in Error Prone EnvironmentsReferenced made to the various verification test reports already conducted and reported in previous meeting as a guide.

MPEG-4 Overview (Sven Bauer)The Video group also contributed to the MPEG-4 overview with the following people providing the text in the following areas: Version 2 (Sven Bauer) Version 3 (Yoichi Yagasaki) Version 4 (Jens-Rainer Ohm) Profiles (Simon Winder) List of Products (Simon Winder)

Review of the FAQ (Mark Pickering/Manish Singhal)The FAQ was reviewed and it was agreed that the FAQ is grossly outdated and should be revised. An Ad Hoc to be created and it is expected to arrive at a revised version of the FAQ by the next meeting.

Joint MeetingsProfile Situation (Joint Video & Req)

The Requirements and the Video groups met to discuss the profile issues. The main discussions centered on the Streaming Video Profile. There were many different issues raised for trying to establish a profile at this meeting. The final conclusion from the discussion was that the discussion on FGS is mainly on the tool and is therefore and issue that Video shall deal with. The issue of the SVP profile is actually two parts. The two parts are streaming profile itself and the base layer of the profile. It was decided that the streaming profile should wait for the maturity of the FGS tool. As for the base layer profile it was seen as wit really was - an attempt to get a single layer profile between simple and core that supports B-VOP and Interlace. The people supporting this profile are ask to make a submission to request for this profile stating clearly the requirements before it is discuss in the Requirements group.

The commitment of companies for conformance of version 2 was also sought at this meeting.

AC prediction (Joint ISG & Video)

A proposal showing the evaluation of the computational complexity of AC prediction was also presented. This proposal presented an evaluation for the divide and storage requirements of AC prediction and it was claimed to be expensive. The joint group decided that the ISG Criteria for Complexity Evaluation should be on

81

• Software implementation (PC Platform)

• Implementation Specific (ASIC, 32 bit architecture)

Fast Motion Estimation (Joint ISG & Video)

At this meeting we established that the fast motion estimation was an informative tool. There were two proposals MVFAST and Advanced Zonal Search. On top of that there is the Diamond Search which is already in the VM. The evaluation criterion for Video evaluation is that the quality must not have degradation. This has already been established for all the proposal.

We established the ISG criterion and agreed that the rough estimation using the number of search point is sufficient for the evaluation of the performance of this tool as it was felt that it is not necessary to go into the implementation specific details.

ISG then reported that they have done the evaluation and that both Advanced Zonal Search and the MVFAST are significantly faster than the Diamond Search. MVFAST is three times faster than DS and The AZS is two times faster than DS. What is not conclusive is that whether MVFAST is significantly better than the AZS. It was felt that the sequences used were not critical enough and that by defining critical sequences the results will become clear. Therefore it was proposed that the exercise be carried out in an Ad Hoc group.

MPEG 7 Main Highlights: Writing/editing of the first version of the WD started. Working draft editors are: Leszek

Cieplinski, Jens Ohm, Sylvie Jeannin, Munchurl Kim. The document (W3069) is based on the latest version of the XM document, with non-normative parts removed. The editors converted entire XM document into WD, pending the decision of the video which descriptors should be promoted. However, It was decided that only the histogram and grid structure descriptors should be included in the WD at Maui meeting. This decision was based on the fact that other descriptors were not fully integrated into software XM and therefore could not be extensively tested or challenged by alternative technology. It was also expected that such a decision will accelerate the XM integration process.

MPEG7 video overview document was created, based on contributions from companies which technologies were included/accepted for the XM.

Joint meeting with MDS and Systems took place on Tuesday. The main topic discussed was the latest version of the DDL and how to express D/DS in the DDL.

Joint meeting on XM development with Stephan, reviewed the XM status. It appears that the XM learning curve is quite steep, and therefore integration takes longer then expected. Inevitably, as with most “new” software there are numerous bugs which hinder/delay the integration. In addition, there are two platforms to support (Windows NT and Linux) and only one person developing/integrating the software (Stefan). It was decided, that a group of companies working with Windows NT platform should help Stephan to iron out the problems with compilation. It was also decided that a very strict policy would be introduced which will not allow promotion to WD of any technology that had not been fully integrated into the software XM well before the meeting. In addition any technology which is not integrated at subsequent meeting will be removed from the XM document.

The video group accepted the MPEG Copyright statement and Copyright policy, as described in the output document.

The following decisions were taken and approved:(Colour and Texture)

82

1. Include in the XM the following new descriptors: The Colour Layout Descriptor, Colour Structure Histogram Descriptor, Compact Colour Descriptor.

2. Add the HMMD colour space to the list of default colour spaces. (Thus there are three default colour spaces: RGB, HSV and HMMD)

3. Include in the XM the texture descriptor, which resulted from an agreement between parties proposing several technologies for texture descriptors. The new descriptor consists of the browsing descriptor and homogeneous texture, and is technically superior to competing proposals.

4. Include in the XM the Edge histogram descriptor, after successful CE and modifications that ensure that it does not overlap the colour descriptor via its luminance component.

5. Urgently start a new CE (CT-1) which will determine interoperability of descriptors using different colour spaces. If satisfactory solution for interoperability cannot be determined, the list of recommended colour spaces will be pruned.

6. Start CE (CT-2) on interoperability between compact colour descriptors and histogram/dominant colour descriptors. NEC to explore interoperability of the colour layout descriptor with the dominant colour. Philips to explore interoperability of the compact colour descriptor with the histogram descriptor.

7. Start CE (CT-3): Colour-structure histogram: Interoperability with colour histogram and different structuring (size) elements.

8. Start Core Experiment on the extension to the dominant colour (CT-4). Significantly improved results over XM were reported and the proposed extension is fully compatible with the current dominant colour from XM. Decision on inclusion to the XM could not be taken as there was no formal CE with this technology.

(Shape/Motion)9. Include two examples of the non-normative similarity measure for the POMD Descriptor

into XM. 10. Continue the CE on 3D shape with the objective to complete second stage by the march

meeting. 11. Start CE on the modifications to the region-based shape descriptor, based on the proposal

and results presented at Maui.12. Start CE on the contour-based CE which with the objective to determine the

performance-size curve for various descriptor sizes.13. To include spatio-temporal region (DS) into XM

ColourColour layout: M5356, M5544

This is a very compact descriptor (64 bits) which captures the layout of the colour within the image. The matching function is very simple to compute. The proposers shown that combination of grid layout and dominant colour to achieve similar functionality would result in a very bulky descriptor and a complex matching function. An example of the functionality of the descriptor: sketch to image matching is shown. The descriptor ouperforms pure colour descriptors.

Colour structure Histogram CSH (Sharp): M5572, M5456A structuring element is used to compute local histograms. Structuring elements is 15x15 pixels.Extended set of queries to demonstrate the use of the extended histogram. Results are comparable to the standard histogram for large (>128) number of bins. Significant improvements can be observed for histograms with smaller number of bins, where CSH consistently outperforms standard histogram by a factor of two in descriptor size (e.g. to achieve similar retrieval performance, CSH is half the size of the standard histogram). Cross verification results are provided by LG. The binary format of the CSH is the same as standard histogram, so one could argue that this is just an extraction issue (non-normative). However, the descriptors using

83

the colour blob histogram and the standard histogram extraction are not interoperable! (This again leads to the issue of what has to be normative and what non-normative – it seems we have to look at this one case-by-case). The proposers agreed to fix the dimensions of the structuring element in order to . Since the results of Colour structure Histogram were significantly better then the standard histogram it was finally decided to include CSC in the XM. The proposers are to investigate the issue of interoperability with standard histogram. If this problem can be resolved satisfactorily, the issue will become a non-normative extraction. If it cannot be resolved, a decision between standard and CSH histogram will be to be taken.

Compact Colour descriptor (Haar transform on histogram): M5345

It is a compact descriptor that combines a grid structure and colour. The group believes that this is a good example of (lossy) coding of the layout+dominant colour that is very compact and is very useful for a class of applications with very strong requirements for low storage and computational cost. We recommend that this be accepted for the XM. The participants should explore interoperability with the existing XM descriptors (including the one recommended from CT-6). A new CE for this experiment was defined at this meeting.

HMMD Colour Space: Recommendation: M5455, M5574

The video group recommends that the HMMD colour space be added to the set of default colour spaces to the XM. CT-1 will continue with the following objectives: a) interoperability between histograms computed using different colour spaces in the

XM (e.g, query feature vector in RGB and the database uses HSV).b) How this affects the retrieval performance?c) Interoperability when the number of bins is different.

Participants: HHI: for RGB + HSV; LGCIT: HMMD + HSV; HMMD + RGB;

Motion CE Spatio-Temporal Region:

M5413 Report of Spatio-temporal Region DS Core/Validation Experiment - Experimental Conditions and Test Data, ToshibaM5414 Results of Spatio-temporal Region DS Core/Validation Experiment, ToshibaThe group reviewed documents and results. The experimenters followed to the CE conditions and workplans decided at the last meeting. Conclusion is that: The experiment results show that the proposed DS performs well for hyperlinking (visualization) application. The question is whether this validation through hyperlinking is acceptable by the DS group. If yes the performances are judged satisfactory to allow inclusion in XM. If now further validation should be performed based on DS recommendations. There are currently two parts in this DS (FigureTrajectory and ParameterTrajectory). Figure trajectory performs slightly better in average than parameters trajectory, but not in all cases. The video group wonders if having two methods in a DS has some drawbacks (in similar ways than for instance having two Ds for the same functionality endanger interoperability). Question is to be answered by DS group, and depending on this both methods may stay or one may be selected.

CE Motion Trajectory:M5400 Quantitative analysis for the motion trajectory descriptors in MPEG-7, ETRIThe group was not yet convinced by this proposal, which proposes to fix the time intervals between keypoints, in the motion trajectory XM descriptor. It needs further proof of evidence. Besides, a cross-check partner has to be found. ETRI proposes to continue this work and demonstrate further evidence if a partner can be found, which could also cross-check the results. If a cross-check partner can not be found, the CE will be dropped.

84

ETRI is currently discussing with Panasonic SG on this topic.CE POMD:

M5372 Video EMU Retrieval Using Parametric Object Motion, RochesterM5394 Results of MPEG-7 Core Experiment on Parametric Object Motion, HHIM5594 Motion Content Set: New Contributions, INTM5595 Similarity Measures for Motion-based Retrieval, INTThe group reviewed these documents and results. Two methods are proposed for a matching technique to associate with parametric motion descriptor of XM. The first method is based directly on the motion parameters: it is of very low complexity but has problems for comparing motions expressed by different models. The second one first calculates the velocity field associated with the motion parameters, and then calculate the similarity between the fields: it is more complex but more flexible, and can compare any motions in a transparent way. These are complementary features. Therefore, the recommendation is to accept both methods in the XM.

Motion activity improvements contributionsThe group reviewed all contributions relating to motion activity. M5602 Comparison of Motion Activity Descriptors based on Activity Ground Truth, NJIT Extensive work was acknowledge, concerning ground truth specification and extraction technique studies. The work will be continued together with the initial motion activity proposers. Localized motion activityM5412 Cross Verification Results of Spatial Localized Motion Activity D, ToshibaM5465 Spatial Localized Motion Activity for MPEG-7, Sharp M5466 Experimental Methods for Spatial Localized Motion Activity D, SharpM5467 Results of Experiments for Spatial Localized Motion Activity D, SharpM5506 Results of Similarity based Retrieval by using Spatial Localized Motion Activity D, SharpThese documents present proposal and results for spatial localized motion activity. The proposers simplify their proposal since last meeting, as recommended. The group felt that that indeed the proposed description captures the feature of localization of motion activity in images. Results were convincing. However, it was not clear which concrete applications this feature specifically allows, or which functionality it adds to the actual motion activity descriptor in the framework of a concrete application. Recommendation is thus for the proposers to show at the next meeting how this proposal either usefully adds to the applications/functionality provided by the current descriptor, or how this proposal improves significantly the performance of the current descriptor for an already identify application/functionality.

Content Creation Group activitiesM5411 Report of the ad-hoc group on Object-based content creation for MPEG-7The group acknowledge the work of the Object-based content creation AHG. As coordination work is not really needed anymore, decision was taken to cancel this AHG, ETRI kindly offering to operate and manage the content repository ftp site.

Shape

Region-based shape descriptor M5472

A new extraction for the region-based shape descriptor was presented. It improves the results for region-based scale/rotation invariance (+5.4%), but worsens the result for similarity retrieval (-3.3%).On average there is some improvement (+1.05%). The proposers requested a core experiment, which was established.

85

3D Shape descriptor M5592,

20 There is just one proposal on the table (INT), as IBM withdrew before the Maui meeting. The first part of the CE was completed and crossed-verified. The second part was not completed due to delays with 3D models data. The CE continues. (Note after the meeting – the data is now available for purchase from ViewPoint – quotations were sent to all interested parties late December 1999).

21

Link between 2D-3D descriptors – shape and other features M5388A new proposal on relation between the 3D and 2D descriptors was presented. It addresses an important question how to perform matching when full 3D (say 3D shape descriptor) and a 2D descriptor relating to a particular view are known. Three examples of 2D-3D relations are considered for shape and texture with very promising results. A Core Experiment is established.

Contour-based shape descriptor.New results showing very good performance and robustness to affine transformation were presented. A core experiment was established to determine the performance/descriptor size curve for the contour-shape descriptor and to optimise the coding of the parameters.

Other contributions:M5417 Presented a proposal for Region localisation descriptor based on polygon approximation. A discussion on this was lead by J. Ohm. The issue of region localisation was already addressed by the RegionLocatorDS, (proposed and accepted in MDS group), which supports several formats of shape coding, including MPEG-4 shape only, segmentation maps and polygon approximation. The discussion followed if MPEG-4 shape coding is a good solution for the region localisation. The proposers were arguing that MPEG-4 is more complex and less efficient compared to polygon approximation and that polygon approximation should be chosen as MPEG-7 shape coding for region localisation. Joern Ostermann commented that an extensive study by MPEG4 shown that there is no difference in computational complexity between the MPEG-4 scheme and polygon approximation. M. Bober argued that RegionLocatorDS gives flexibility to choose the best solution for the problem, and that the content creator can choose between three techniques, including MPEG-4 shape-only coding.In addition, some results on shape matching using polygon approximation were presented. The absolute results for polygon-based descriptor were not as good as provided by the contour-based shape descriptor, and the size of the polygon-based descriptor was not given so the relative performance between the two descriptors could not be compared.

M5559 proposed shape and shape deformation descriptor based on complex Fourier Transform. Very few results were shown so it was impossible to judge the performance. The proposers were asked to perform experiments on the CE-1 database and to provide more examples showing the performance/application of the shape deformation component.

M5507 proposed a new technique for shape descriptor. However, no detailed performance results were presented and it was not possible to compare the performance. The proposers were asked to follow the methodology of the CE-1 shape experiment and to provide a complete set of results.

ResolutionsThe Video group decided that all descriptors which were approved for the XM before Maui, should be integrated within the XM by 20 January 2000. If a descriptor is not integrated by

86

20/01/2000, then the descriptor cannot be promoted to the WD during the March 2000 meeting. Any descriptor not integrated by the March meeting should be removed from the XM document. The descriptors approved during the Maui meeting should be integrated by 22/03/2000., otherwise they will be removed from the XM. If they are integrated by 15/01, then they can be considered for WD during the March meeting. By integration we mean that the code can be compiled and successfully run on at least one architecture by Stephan Herrmann.

87

Annex 8

Audio Subgroup Report for the 50th MPEG Meeting

Source: S. Quackenbush, Chair Audio SubgroupEditor: D. Thom

Opening of the meetingThe MPEG Audio Subgroup meeting was held during the 50th meeting of WG11 in Maui, Hawaii, December 6 to 10, 1999. The list of participants is given in Annex A-1. The Chair welcomed the delegates to the meeting and outlined the work for the five days.

Administrative matters

Approval of agendaThe agenda, as presented in Annex A-II, was discussed, edited and approved.

Melbourne meeting reportThe Audio Subgroup Melbourne meeting report, October 1999, had been previously distributed by email and was approved.

Allocation of contributionsAll contributions (see Annex A-IV) were allocated to the agenda and were discussed either in the task groups or in Audio plenary. The Chair brought other relevant documents from Test, Systems, and Requirements to the attention of the group.

Communications from the ChairThe Chair summarised the issues raised at the Sunday evening Chair’s meeting and indicated the priorities for the week’s work.

Joint meetingsJoint meetings over the course of the week are listed here and are reported on below.Day Time Meeting With Topic LocationTuesday 1800-1900 MMDS m5434 (MPEG-7 DS for Audio

Contents)MDS

Wednesday 1130 - All Conformance m5597 (Advanced Audio BIFS)

Systems

Wednesday 1630 – 1730 ISG, MDS, Audio XM contribution MDSThursday 0900 - 1000 Requirements Review of proposed V2 profiles

and levelsAudio

Thursday 1000 - 1100 Systems Review of backchannel Audio

Received National Body Comments and Liaison mattersThe following National Body Comments and Liaison responses were prepared:NB Comm.

Title Source Author of Response

88

5302 Liaison Statement from ITU-T SG 16 Q.11 and Q.14 on a Generic Capability Parameter Addition for H.245 MPEG-4 Audio (SC29N3321)

Liaison M. Iwadare

5637 Transmission of Music Metadata Liaison J. Flemming5423 German NB Comments on MPEG-4 as Coder for

VoIP ApplicationsGeneral K. Brandenburg

5470 JP NB Comments on MPEG-4/Audio Transport Stream Format

General M. Iwadare

5526 Dutch NB Comments to the Amendment 7 to ISO/IEC 13818 and the Audio parts of MPEG-2 and MPEG-4.

General R. Funken

Task GroupsEight task groups were convened for the duration of the MPEG meeting, as shown in ANNEX-III. Results of task group activities are reported below.

Record of Audio plenary, joint meetings and task group activities

Audio plenary discussionsThe chair advised the Audio group that, at least for this meeting, the Audio report would be a collaborative effort. Specifically, that each task group should contribute a summary of key decisions so that these can be recorded in the Audio report and hence do not need to be re-visited in future meetings. Those contributions are shown in the section “Task group discussions.”The chair felt that some presentations needed the visibility provided by Audio plenary, or that some discussions were sufficiently contentious that they needed to be held in Audio plenary. These topics are indicated in the Audio agenda, and are reported on in the following paragraphs.There were three input documents on statements of support for MPEG-4 profiles. The first confirmed that NEC has withdrawn its support for the high quality audio profile, the second indicated that NTT will support Mobile AUdio Internetworking profile, and the third indicated that Samsung will support the natural audio profile.The Liaison inputs from ITU-T and the related comment from JNB were discussed. Proposed syntax for LATM was reviewed, and the chair took issues from the proposal to the Wednesday evening chairs meeting. At the chairs meeting it was agreed that, using H.245, terminal equipment could signal both the MPEG-4 profile and level and the individual audioObjectTypes that the terminal equipment was able to handle. Typically, the equipment would signal every audioObjectTypes in the signalled profile and level. Systems will have the required Audio profile and level table available for review by ITU-T by Jan 31.The issue of using the EP tool with AAC SSR profile was discussed. The consensus was that the EP tool will not be used with any Version 1 audio object type. Furthermore, that since the AAC SSR audio object type has not defined a error resilient bitstream syntax that supports unequal levels of error protection, there will not be a Version 2 AAC SSR plus EP tool audio object type.The Draft Report on MPEG-4 Version 2 Verification Test was reviewed. Constructive comments on the specific and overall conclusions of the report were incorporated into the text. The very good results of the test were publicised within the Audio group.The Audio chair insured that all contributions were reviewed either in task groups or in Audio plenary. As indicated in the previous sections, when task group activity could not progress, those discussions were moved to Audio plenary. In this way, plenary activity was primarily that of the Audio chair sharing information with the group or getting task group status from the group, or approval of output documents, audio resolutions and ad hoc group mandates.

Joint meeting discussions

89

Impromptu joint meeting on MPEG-7 source code policy

There was a joint meeting between the Convenor and the MDS, Video and Audo subgroup chairs to discuss a revision of the MPEG policy on contribution of normative and non-normative software on MPEG-7. The revised policy was reviewed in Audio plenary, was agreed to by Audio members, and is shown in resolution 3.9.6 of the Maui meeting.

Joint meeting on Conformance

In a Joint Conformance Meeting with Video and Systems, it was agreed that the MPEG-4 version 2 conformance document will be a delta document to the version 1 conformance document. Conformance for Advanced Audio BIFS was discussed, and it was agreed that Advanced Audio BIFS would be part of audio conformance.

Joint meeting with MDS and ISG on XM contribution

This was a very valuable meeting in which Audio experts learned more about the XM and how the results of our core experiments can be integrated into it. However, it was clear from the discussions that significant aspects of the DDL parser and associated tools are not yet complete. For this reason, the Audio group agreed that there could be an interim period in which C++ classes and associated C++ code can be used instead of DDL code and the DDL parser for integrating accepted technology into the XM.

Joint meeting with requirements on MPEG-4 profiles and levels

Audio presented the proposed audio profiles and levels to requirements. After some discussion, these were accepted for incorporation into the MPEG-4 version 2 text. Requirements requested a table listing each proposed audio profile and the companies that pledge to deliver bitstreams that will test some aspect of that profile.

Joint meeting with systems on upstream channel

The Audio syntax for upstream channel was reviewed. The meeting was very valuable since some technical errors in the Audio proposal were identified and corrected.

Task group discussionsMPEG-4 conformance task group

Version 1.

The AHG report was presented during the Monday plenary and therefore was not presented again. With respect to V1, the Ballot comments were reviewed by the task-group. A summary is provided in the following table. Ballot Comment Topic DoCGerman NB Include changes reflected in “Study on Doc.” AcceptedGerman NB AAC AcceptedJPese NB Include changes reflected in “Study on Doc.” AcceptedJPese NB Provide bitstreams & ref. output and sw work planJPese NB Modify CELP bitstream description acceptedSwiss NB SA acceptedFinnish NB Replace table of LTP bitstreams Accepted

The task-group agreed that software required for calculating conformance criteria other than the “RMS measurement”, such as PNS-Measurement, segmental SNR, Cepstral criteria and the

90

criteria required for HVXC, shall be made available together with the bitstreams by Jan 31, 2000. Version 2.5354 was presented by Sanae Hotani. There was a short discussion related to EP tool levels. The conclusion is to include the document into the PDAM document and update it based on the results of the discussion related to levels for the EP Tool. 5503 was presented by Toshiyuki Nomura. In this document. the information for the silence compression tool is missing, but this will be added. 5505 was presented by Takehiro Moriya. The conformance criteria is the same as for V1. Two bitstreams are currently defined, but this will probably be extended to the whole V1 set with V2 syntax. No further discussions. 5514 was presented by Sunh-Hee Park. This document contains some refinements and updates related to the BSAC part of conformance. No further discussion. 5530 was presented by Heiko Purnhagen giving procedures how to test HILN.5597 was discussed in a joint session with systems. Based on this, there was a more general discussion about the method of conformance testing that could apply to the Advanced Audio BIFS. The conclusion is, that it is appropriate to define a “subjective conformance procedure” (as they did e.g. in SNHC)

MPEG-7 task group

The task group identified four core experiments to carry forward from this meeting: Speech Recognition, Timbre Similarity, Sound Effects and Music Genre.The task group recommended that the XM integration strategy follow the methodology utilized by the video descriptor XM integration. This is because of the current status of the DS/DDL group XM integration, and because of the common bottom-up, functional nature of the current DSs in CE.The Contribution M5408 was examined, and it made for a good framework for all audio descriptors, as well as a good contribution to the MedMet activities. Michael Casey added a section providing a good framework for signal-oriented descriptors. The resulting output document should be discussed on the reflector.Jim Flemming presented the liaison from the RIAA. The MPEG-7 Audio experts found this presentation very interesting and hope that the RIAA will continue to interact with the MPEG Audio group. The task group members agreed with the SOL license agreement for the content set, following the changes we proposed. The issue remains as to who signs “for MPEG.”The Time-series and Scale Tree datatypes were examined and discussed, and both require more information and a CE to validate them. The group approved the requirement clauses in the Time-series document. They were brought to the Requirements group, but, alas, it was too late in the week to be discussed. It should be put forth as an input to the next meeting.The idea of “obvious” audio descriptors is now officially gone. Any such thing requires a full proposal for a CE.The task group made the following CE-related observations: All CEs need to familiarize themselves with current time representationsSpeech CE: Hand-tuned “optimal” lattices will be very helpful for demonstration and XM integration

purposes. We need to devise the right test conditions to show the DS favorably. The MPEG-7 Audio experts should attempt to get other experts at AT&T involvement this

core experiment.Instrument Timbre CE: The CE should make a definitive statement on its scope and applications.

91

There should be thought given to re-use of the more generic low-level descriptors. The specific CE experimental methodology should be solidified and provided to other

potential experimenters. The proposers are encouraged to prepare integration materials in parallel with the validation

activities.Sound Effects Classification CE: The proposers need collaborators for verification.

Task group on FAQ:

M5532 was reviewed and edited in HTML by D Thom, J Herre, A Lindsay & H Purhagen for consistency and accuracy. Because HTML files do not highlight revisions, it was decided to have a review period until Feb 1, 2000 for additional editing.

Meeting deliverables

Press statementThe Chair prepared the Audio part of the press statement by the Audio chair. It was reviewed and approved.

Dispositions of CommentsThe DoC were reviewed and approved.

Responses to NB commentsResponses to the national body comments listed above were reviewed, approved and given to Liaison.

Liaison statementsResponses to the liaison inputs listed above were reviewed, approved and given to Liaison.

Recommendations for final plenaryA list of Audio recommendations were reviewed and approved.

Establishment of new Ad-hoc GroupsThe following ad-hoc groups were established:Title Chair No. MeetingAHG on MPEG-4 Audio Reference Software Editing H. Purnhagen

B. Teichmann3085 No

AHG on Audio part of MPEG-4 Version 1 FDIS and Version 2 PDAM Conformance

T. Mlasko S Hotani

3086 No

AHG on MPEG-4 Audio Version 2 FDAM Editing S-W Kim 3087 NoAHG on MPEG-7 Audio Core Experiments Chair: A. Lindsay

Co-chair: J. Koehler, M Casey, G Peeters, S. Quackenbush

3088 Sunday before meeting

Approval of output documentsAll output documents, shown in Annex-IV, were presented to Audio plenary and were approved.

Future activities

92

Schedule of future meetingsIf any ad-hoc group meeting are to occur, they will be on the weekend immediately preceding the Noordwijkerhout MPEG meeting.

Agenda for next meetingThere was not sufficient time to present the agenda for the Noordwijkerhout Audio subgroup meeting, although it is shown in Annex A-V of this report.

A.O.B.There was none.

Closing of the meetingThe chair noted that, despite the hectic pace of work, this week was an especially important milestone: MPEG-4 version 1 and version 2 are essentially complete, and MPEG-4 version 1 and version 2 conformance are well on their way to completion. He congratulated all MPEG Audio members who contributed to this very large effort, and he fully expects that we will all be proud of this impressive standard. To commemorate this occasion, the meeting was adjourned for a group picture on the beach, with the volcanoes of Maui in the background!

Annex A-I: Meeting Participants ListName First Name Cou

ntryAffiliation e-mail address

Brandenburg Karlheinz DE FhG-IIS [email protected] Mike US M E R L [email protected] Markus CH ETHZ [email protected] Frank DE T-Nova [email protected] James US RIAA [email protected] Ralf F.M. NL Philips Consumer

[email protected]

Garner Philip UK Canon [email protected] Bernhart DE FhG-IIS [email protected] Perfecto FR IUA/Cuidad [email protected] Jürgen DE FhG-IIS [email protected] Daniel DE FHG [email protected] Sanae JP NTTDoCoMo [email protected] Jyn FI Nokia Res Center [email protected] Masahiro JP NEC [email protected] Sang-Wook KR Samsung [email protected] YeonBae KR Samsungi [email protected] Sunghan KR ETRI [email protected] Youngmoo US MIT Media Lab [email protected] Joachim DE GMD [email protected] Kalero FIN Nokia Res Center Kalervo@[email protected] Guyader Alain FR France Telecom [email protected]

m.frLindqvist Morgan SE Ericsson morgan.lindqvist@era-

t.ericsson.seLindsay Adam BE Starlab [email protected] Toshio JP NTT DoCoMo [email protected] Torsten DE Bosch [email protected]

93

mailto:[email protected]



mailto:Kalervo@[email protected]















Name First Name Country

Affiliation e-mail address

Moriya Takehiro JP NTT [email protected] Masayuki JP Sony [email protected] Toshiyuki JP NEC [email protected] Sung-Hee KR Samsung [email protected] Geoffroy FR Ircam/cuidat [email protected] Ged SG Infineon [email protected] Heiko DE Uni Hannover [email protected] Schuyler US AT&T [email protected] Ralph DE FhG [email protected] Jens DE Thomson

[email protected]

Sugaro Masaru JP KDD [email protected] David US Mitsubishi [email protected] Toguri Yasuhiro JP Sony [email protected]äänänen Mauri FI Nokia Res. Center [email protected]

Annex A-II: Agenda for the Maui Audio Meeting1. Opening of the meeting2. Administrative matters

2.1. Approval of agenda2.2. Approval of Melbourne meeting report2.3. Allocation of contributions2.4. Communications from the Chair2.5. National Body Comments and Liaison

matters5302 Liaison Statement from ITU-T

SG 16 Q.11 and Q.14 on a Generic Capability Parameter Addition for H.245 MPEG-4 Audio (SC29N3321)

5637 Transmission of Music Metadata5357 German NB Comments on

MPEG-7 Binary Code and Copyright Issues

5359 German NB Comments on the Assessment of Watermarking Technology for MPEG

5423 German NB Comments on MPEG-4 as Coder for VoIP Applications

5470 JP NB Comments on MPEG-4/Audio Transport Stream Format

5495 UK National Body Position on Document Management Technologies

5526 Dutch NB Comments to the Amendment 7 to ISO/IEC 13818 and the Audio parts of MPEG-2

94








and MPEG-4.5631 Finland NB Late Ballot

Comment on ISO/IEC 14496-33. Audio plenary issues

3.1. General interest documents 5296 Summary of Voting: ISO/IEC FDIS 14496-3

5363 Draft Report on MPEG-4 Version 2 Audio Verification Test

5420 Information on the Philips System standards & Licensing website

5532 MPEG Audio FAQ Update5584 Using the mp4 library to create

AAC audio bitstreams3.2. MPEG-4 Version 1 corrigenda Do we need one? yes

LTP and AAC PCE restrictions.3.3. MPEG-4 Version 1 software status Audio composition? Seems OK3.4. MPEG-4 Version 2 Patent Statements Complete? yes3.5. MPEG-4 Version 2 Profiles 5471 Withdrawal of support for an

MPEG-4 Version 2 Audio Profile

5511 Supporting Statement of MPEG-4 Version 2 Audio Profile

5515 Support statement for the Natural Audio Profile

3.6. MPEG-4 Version 2 technical issuesAudio transportBackchannel

Status? Role of audio transport vs. role of systems.

3.7. MPEG-4 Reference Software framework re-write

3.8. Call for Audio Testing Pro/Con?3.9. MPEG-7

3.9.1. Software agreement3.9.2. Contribution to XM3.9.3. Review terms for access to new

material

What comments can we make as a result of m5508 and other CE?C++ code to XM.DDL expression in our own document.

4. Joint Meetings4.1. Systems 5597 Proposals on MPEG-4 Version 2

Advanced AudioBIFS Conformance and Level Definitions

4.2. MMDS 5434 Consideration on MPEG-7 Summary DS for Audio Contents

4.3. Requirements OK Review of audio profiles and levels

5. Task group activities5.1. MPEG-4 V1 Corrigenda5.2. MPEG-4 Conformance 14496-4 (FDIS Dec 5301 Study on the audio part of

95

99) and 14496-4/Amd 1 (PDAM Dec 99) MPEG-4 Version 1 Conformance

5397 Report of the Ad-hoc Group on MPEG-4 Audio Version 1 and Version 2 Conformance

5354 The conformance methods of EP-tool for MPEG-4 audio version2

5503 A Draft Conformance Testing of ER CELP for MPEG-4 Version 2 Audio

5505 Proposal of Conformance Test Procedure for the ER-TwinVQ Object Type

5514 A description of conformance testing for ER Fine Granule Audio

5530 Information on HILN conformance testing

5.3. MPEG-4 Audio 14496-3/Amd 1 (FDAM Dec 99)

5292 Summary of Voting: ISO/IEC 14496-3/FPDAM 1

5287 Study on MPEG-4 Version 2 Audio FPDAM

5306 FhG Comments on Study on MPEG-4 Audio Version 2 FPDAM

5353 The level of EP-tool for MPEG-4 audio Version2 profile

5406 AHG Report on MPEG-4 Audio version 2 Study on FPDAM Editing

5427 Proposed modifications to error resilient syntax of CELP objects

5509 Proposal of Editorial Corrections of 14496-3 FPDAM 1 (ER-TwinVQ Object Part)

5525 Revisions of V2 text for ER HVXC object

5529 Study of HILN in 14496-3 FPDAM1 and proposed revisions

5.4. MPEG-4 Reference Software 14496-5/Amd 1 (FPDAM Dec 99)

5527 Report of AHG on MPEG-4 Audio Version 2 Reference Software editing

5.5. MPEG-4 Version 2 Technical Issues 5571 Report of the ahg on MPEG-4 Version 2 Audio Technical Matters

5.5.1. Audio Transport5.5.2. Backchannel

96

5.6. MPEG-4 Version 2 Verification Tests 5331 Workplan for MPEG-4 Audio Version 2 Verification Test

5332 AHG Report on MPEG-4 Version 2 Audio Verification Tests

5531 Study on HILN subjective quality

5.7. MPEG-7 Audio 15938-3 (WD Dec 99) 5562 Report of AHG on MPEG-7 Audio Matters

5.7.1. Core experiments 5422 Audio CE on Instruments Description (Timbre Similarity)

5508 Current Status of Spoken Content DS

5.7.2. Audio DS and D 5408 Obvious Audio Ds / DSs5.7.3. Other 5443 Scale Tree Update

5444 A Set of Scalable Audio Descriptors

5540 Evaluation of the MPEG-7 speech DS

5585 Core experiment proposal for a description scheme for music program creation

5604 Sound Effects CE5615 Time Series Style Descriptors

5.8. MPEG Audio Web Page / FAQ / FTP site / Audio press statement contribution

6. Discussion of unallocated Contributions7. Meeting deliverables

7.1. Press statement7.2. Dispositions of Comments7.3. Responses to NB comments7.4. Liaison statements7.5. Recommendations for final plenary7.6. Establishment of new Ad-hoc Groups7.7. Approval of output documents

8. Future activities8.1. Agenda for next meeting was not presented or approved

9. A.O.B.10. Closing of the meeting

Annex A-III: Task Groups at the Maui Audio Meeting1. MPEG-4 Audio 14496-3/COR1 (DCOR Mar 00)

Chair: M. VäänänenMandate:

Start corrigendum to correct LTP errors and any other errors that can be identified at this meeting.

97

2. MPEG-4 Conformance 14496-4 (FDIS Dec 99) Conformance 14496-4/Amd 1 (PDAM Dec 99)Chairs: T. Mlasko, S. Hotani, V1 Mandate:

Produce FDIS text based on input documents Produce DoC of FCDProduce workplan for 1) obtaining and cross-checking all bitstreams 2) converting all

bitstreams to mp4 file format.V2 Mandate:

Produce PDAM text Produce workplan for 1) progressing text and 2) obtaining and cross-checking all

bitstreams

3. MPEG-4 Audio 14496-3/Amd 1 (FDAM Dec 99)Chair: S-W KimMandate:

Produce FDAM textProduce DoC of FPDAM

4. MPEG-4 Reference Software 14496-5/Amd 1 (FPDAM Dec 99)Chairs:H. PurnhagenMandate:

Produce status report of V1 and V2 reference software and work plan if appropriate for both software errors and software re-write

5. MPEG-4 Version 2 Technical IssuesChair: B. GrillMandate:

Finalize open issues on:Audio TransportBackchannel syntaxAny other technical issues

Edit 14496-3/Amd1 text as appropriateStart Version 3 if appropriate

6. MPEG-4 Version 2 Verification TestsChair: R. SperschneiderMandate:

Produce final report on MPEG-4 Version 2 Verification TestsUpdate V2 Audio overview text in N2995

7. MPEG-7 Audio 15938-3 (WD Dec 99)Co-Chairs: A. Lindsay, S. QuackenbushMandate:

Prepare contribution to XMGenerate work plans for new and ongoing CE.Contribute to

MPEG-7 process document (edit N2999)MPEG-7 overview document (edit m5519)

Review conditions for acceptance of new test material

98

Consider Audio contribution to Generic A/V DS (edit m5380)Consider output document on obvious Audio D and DS (edit m5408)

8. MPEG Audio Web Page / FAQ / FTP site / Press ReleaseChair: D. ThomMandate:

Answer FAQ for posting on web pagePublicize all public FTP sites for MPEG Audio information, conformance bits or source

codeDraft Audio contribution to 50th meeting press release

Annex A-IV: Input and Output Documents

Contributed documentsThe following documents were contributed to the Audio Subgroup and were considered during this meeting:

No. Section Source Title5423 General German National Body Comments on MPEG-4 as Coder for

VoIP Applications5526 General Jean H.A. Gelissen (editor),

Falf Funken, Jan van der Meer

DNB Comments to the Amendment 7 to ISO/IEC 13818 and the Audio parts of MPEG-2 and MPEG-4.

5287 Audio AHG on MPEG-4 Audio V2 Study on FPDAM Editing

Study on MPEG-4 Version 2 Audio FPDAM

5292 Audio SC 29 Secretariat Summary of Voting: ISO/IEC 14496-3/FPDAM 1

5296 Audio ISO/IEC ITTF via SC 29 Secretariat

Summary of Voting: ISO/IEC FDIS 14496-3

5301 Audio Audio Conformance AHG (Torsten Mlasko)

Study on the audio part of MPEG-4 Version 1 Conformance

5302 Liaison ITU-T SG 16 Q.11 and Q.14 via the SC 29 Secretariat

Liaison Statement from ITU-T SG 16 Q.11 and Q.14 on a Generic Capability Parameter Addition for H.245 MPEG-4 Audio (SC29N3321)

5306 Audio Ralph Sperschneider FhG Comments on Study on MPEG-4 Audio Version 2 FPDAM

5331 Audio Ralph Sperschneider Workplan for MPEG-4 Audio Version 2 Verification Test

5332 Audio Ralph Sperschneider AHG Report on MPEG-4 Version 2 Audio Verification Tests

5353 Audio Sanae Hotani, Toshiro Kawahara, Norio Nakamura

The level of EP-tool for MPEG-4 audio Version2 profile

5354 Audio Sanae Hotani, Toshiro Kawahara, Norio Nakamura

The conformance methods of EP-tool for MPEG-4 audio version2

5363 Audio Schuyler Quackenbush, Ralph Sperschneider, Frank Feige

Draft Report on MPEG-4 Version 2 Audio Verification Test

5397 Audio Torsten Mlasko, Takehiro Moriya

Report of the Ad-hoc Group on MPEG-4 Audio Version 1 and Version 2

99

No. Section Source TitleConformance

5406 Audio Sang-Wook Kim AHG Report on MPEG-4 Audio version 2 Study on FPDAM Editing

5408 Audio Silvia Pfeiffer Obvious Audio Ds / DSs5420 Audio Ralf Funken Information on the Philips System

standards & Licensing website5422 Audio Geoffroy Peeters, Perfecto

Herrera, Xavier AmatriainAudio CE on Instruments Description (Timbre Similarity)

5427 Audio Torsten Mlasko, Ertugrul Akyuez

Proposed modifications to error resilient syntax of CELP objects

5434 MMDS Masaru Sugano, Yasuyuki Nakajima

Consideration on MPEG-7 Summary DS for Audio Contents

5443 Audio A. de Cheveigni Scale Tree Update5444 Audio A. de Cheveigni, G. Peeters A Set of Scalable Audio Descriptors5470 Audio The National Body of JP JNB Comments on MPEG-4/Audio

Transport Stream Format5471 Audio Masahiro Iwadare, Toshiyuki

NomuraWithdrawal of support for an MPEG-4 Version 2 Audio Profile

5503 Audio Masahiro Serizawa, Toshiyuki Nomura, Masahiro Iwadare

A Draft Conformance Testing of ER CELP for MPEG-4 Version 2 Audio

5505 Audio Takehiro Moriya, Takeshi Mori, Naoki Iwakami, Akio Jin

Proposal of Conformance Test Procedure for the ER-TwinVQ Object Type

5508 Audio Philip N. Garner, Jason P. A. Charlesworth

Current Status of Spoken Content DS

5509 Audio Takehiro Moriya, Takeshi Mori, Naoki Iwakami, Akio Jin

Proposal of Editorial Corrections of 14496-3 FPDAM 1 (ER-TwinVQ Object Part)

5511 Audio Takehiro Moriya Supporting Statement of MPEG-4 Version 2 Audio Profile

5514 Audio Sung-Hee Park, Yeon-Bae Kim

A description of conformance testing for ER Fine Granule Audio

5515 Requirements Samsung Electronics Co., Ltd (Jae-Seob Shin)

Support statement for the Natural Audio Profile

5525 Audio Yuji Maeda, Masayuki Nishiguchi

Revisions of V2 text for ER HVXC object

5527 Audio Heiko Purnhagen, Bodo Teichmann

Report of AHG on MPEG-4 Audio Version 2 Reference Software editing

5529 Audio Heiko Purnhagen, Bernd Edler, Nikolaus Meine

Study of HILN in 14496-3 FPDAM1 and proposed revisions


Information on HILN conformance testing


Study on HILN subjective quality

5532 Audio Heiko Purnhagen, Silvia Pfeiffer

MPEG Audio FAQ Update

5540 Audio Joachim Koehler Evaluation of the MPEG-7 speech DS

100

No. Section Source Title5562 Audio Adam Lindsay (on behalf of

the MPEG-7 Audio AHG)Report of AHG on MPEG-7 Audio Matters

5571 Audio Bernhard Grill, Masahiro Iwadare

Report of the ahg on MPEG-4 Version 2 Audio Technical Matters

5584 Audio S. R. Quackenbush Using the mp4 library to create AAC audio bitstreams

5585 Audio S. R. Quackenbush Core experiment proposal for a description scheme for music program creation

5596 Audio Mauri Vaananen, Juha Ojanpera

A description of conformance testing for ER AAC LTP

5597 Audio Riitta Vaananen, Jyri Huopaniemi

Proposals on MPEG-4 Version 2 Advanced AudioBIFS Conformance and Level Definitions

5637 Audio J Flemming RIAA Transmission of Music Metadata5631 Audio NB of Finland via the SC 29

SecretariatLate Comments on ISO/IEC 14496-3/FPDAM 1 (SC 29 N 3372)

Output DocumentsThe following output documents were produced in whole or part by the Audio Subgroup. Those shown in Italics were approved for public release.Title NoMPEG-4 Version 1 Audio Conformance FDIS 3067DoC on Audio Conformance FCD 3066Doc of ISO/IEC 14496-3/FPDAM 1 3057Text of ISO/IEC 14496-3/FDAM1 3058MPEG-4 Version 2 Audio Conformance PDAM 3065Status and Workplan of MPEG-4 Version 2 Audio Reference Software 3071Audio Contribution to MPEG-7 WD 3072Proposed Corrections to 14496-3, MPEG-4 Version 1 Audio 3073Workplan for MPEG-4 Audio Conformance 3074Report on MPEG-4 Version 2 Audio Verification Test 3075Information on Use of MPEG-4 Low-overhead Audio Transport Multiplex (LATM)

3076

MPEG-7 Audio Core Experiment Methodology 3077Framework for MPEG-7 Audio Descriptors 3078Status of MPEG-7 Audio Core Experiments 3079Workplan for MPEG-7 Speech Recognition Core Experiment 3080Workplan for MPEG-7 Timbre Similarity Core Experiment 3081Workplan for MPEG-7 Sound Effects Core Experiment 3082Workplan for MPEG-7 Music Genre Core Experiment 3083Audio FAQ Update 3084

Annex A-V: Agenda for the Noordwijkerhout Audio Meeting1. Opening of the meeting2. Administrative matters

2.1. Approval of agenda

101

2.2. Approval of Melbourne meeting report2.3. Allocation of contributions2.4. Communications from the Chair2.5. Joint meetings2.6. Received National Body Comments and Liaison matters2.7. Review of MPEG-4 FDIS and FDAM ballots2.8. Review of MPEG-4 Version 1 conformance bitstream status2.9. MPEG-4 Reference Software 14496-5/Amd 1 (FDAM Mar 99)

2.9.1. Reference software framework re-write3. Task group activities

3.1. MPEG-4 Conformance 14496-4/Amd 1 (FPDAM Jul 99)3.2. MPEG-7 Audio 15938-3 (CD Oct 00)

3.2.1. Audio Core Experiment Methodology3.2.2. Core experiments3.2.3. Contribution to Generic A/V DS3.2.4. Promotion of MPEG-7 Audio

3.3. MPEG Audio Web Page / FAQ / FTP site3.4. MPEG Audio press statement contribution

4. Discussion of unallocated Contributions5. Meeting deliverables

5.1. Press statement5.2. Dispositions of Comments5.3. Responses to NB comments5.4. Liaison statements5.5. Recommendations for final plenary5.6. Establishment of new Ad-hoc Groups5.7. Approval of output documents

6. Future activities6.1. Agenda for next meeting

7. A.O.B.8. Closing of the meeting

102

Annex 9SNHC meeting report

SNHC Meeting SummarySNHC objectives for the Maui meeting in Version 1 MPEG-4 were to check the development of conformance testing. For Version 2, objectives were to review on the SNHC part of Visual FDAM 1 and resolve the reference S/W management for Version 2. Version 2 Conformance was also discussed with a workplan till March 2000. Review and discussion on generic 3D animation took place with a plan to issue a Call for Proposals in March 2000.

SNHC Objectives & Work ItemsAn outline of more detailed meeting objectives and work is given below:

MPEG-4 Version 1A. Conformance 1.Face Animation Conformance

a. Conversion of test data streams to MP4 file format

MPEG-4 Version 2A. 3D Model Coding

1. Visual FDAM review2. Conformance

a. Reference S/W (with maintenance)b. Integration to IM-1c. Conformance testing with bit streams

3. Upstreaming

B. Face and Body Animation1. Integration to IM-1a. MP-4 file format

2. Profiles and levels for body animation jointly with face animationa. SimpleFBA profile

3. Part 5 Reference Software4. Visual FDAM review

MPEG-4 Version 3A. Generic 3D Animation1. Review of input contributions

SNHC Contributions & Related ReviewNumber Group Title

Section Source5291 MPEG-4 Summary of Voting: ISO/IEC 14496-2/FPDAM 1

Video/SNHC SC29 Secretary5407 MPEG-4 JNB comments on ISO/IEC 14496-2(Visual) DCOR1

Video/SNHC The National Body of JP (JNB)5320 MPEG-4 Body Animation Tables for Hand Deformations

SNHC Sunil Hadap, Laurent Moccozet, Tom Molet, Igor Pandzic5518 MPEG-4 Release Notes Update for Body Animation Integration into the im1 3D Player

SNHC Tolga K. Capin, Fabrice Vergnenegre5558 MPEG-4 Support on FBA Simple, 3D Hybrid, and Version 2 Complete Graphics Profiles

Requirements Tolga K. Capin (EPFL), Eric Petajan (Lucent)4564 MPEG-4 Detailed Hands and Clothes BATs for Body Animation

SNHC Laurent Moccozet, Sunil Hadap

103

5350 MPEG-4 SNHC Report of AHG on SNHC Conformance SNHC G. Taubin, M. Hahn, and F. Bossen

5523 MPEG-4 The conformance methods for SNHC 3DMCSNHC Mahnjin Han, Gabriel Taubin, Frank Bossen

5349 MPEG-4 3D Mesh Coding Reference SoftwareSNHC G. Taubin

5521 MPEG-4 3DMC Reference SoftwareSNHC Sungjin Kim, Mahnjin Han, Mun-Sup Song

5303 MPEG-4 Release Notes Update for 3D Mesh Coding Integration into The Pact 3D PlayerSystems Ioana M. Martin, Gabriel Taubin

5334 MPEG-4 Systems Software Updates of PACT 3D Player with Support for Hierarchical 3D MeshesSystems Ioana M. Martin, Gabriel Taubin

5313 MPEG-4 Tutorial example of MPEG-4 backchannelSystems Gauthier Lafruit

5522 MPEG-4 Proposal for upstreamming with ServerCommand nodeSystems Do-Kyoon Kim, Sungjin Kim, Mahnjin Han, Mun-Sup Song, Euee S. Jang

5368 MPEG-4 Liaison from Web3D on VRML/MPEG-4 Convergence (SC 29 N 3346)Liaison Web3D Consortium via the SC 29 Secretariat

5369 MPEG-4 Liaison from Web3D on MPEG-4 Technical Review (SC 29 N 3347)Liaison Web3D Consortium via the SC 29 Secretariat

5520 MPEG-4 Analysis of existing tools for animated 3D model codingSNHC Do-Kyoon Kim, Mun-Sup Song, S.Y. Jung, Mahnjin Han, Euee S. Jang

Output Document EditorsDocument EditorOverview of MPEG-4 Frank BossenMinutes of the Meeting Mahnjin HanV2 SNHC Conformance Body Animation Eric Petajan 3D Model Coding Gabrial TaubinV2 Visual FPDAM editing Mahnjin Han, Frank Bossen, Tolga Capin, Eric PetajanSNHC Work Plan Mahnjin HanPress Release Eric Petajan

SNHC & Joint MeetingsThe following schedule of joint meetings and SNHC work was agreed:

Joint in Monday Tuesday Wednesday Thursday FridayWG11 9:00am-1:30pm

WG11 Plenary9-11:00am

WG11 Plenary2-10:00pm WG11

PlenarySNHC 3-5pm SNHC

Plenary – meeting agenda, work plan, V1 Conformance

9– 12 am SNHC Plenary – Profile issue, V2 FBA, V2 3DMC: Afternoon,

4-7pm SNHC Plenary- animation coding

4:20 - 6pm SNHC Plenary – Liaison, Patent Statements,Visual Test, FAQ, deliverables, Press release

9-11 am SNHC Plenary

4-6pm SNHC Plenary –Conformance, FDAM changes

9-11:30 SNHC Plenary approval

Systems 2-4pm SNHC & Systems - V2 integration + surface node

11.30 - 1:00 SNHC & Systems –

Conformance

2-4 pm SNHC, Systems,

11am-12pm SNHC, Systems –

advanced 3D animation with

12-1pm SNHC, Requirements,

104

Joint in Monday Tuesday Wednesday Thursday FridayRequirements-

complete graphics profile

Systems – multiuser profile,

textual format

2-4pm SNHC, Systems - Im-1 implementation

Requirements 5-6 pm SNHC, Requirements - SNHC profiles

8:30-9am Requirements, SNHC – profiles

3.30 – 4pm SNHC, Requirements - 3D mesh profiles

Meeting Work & Results

Version 1 - ConformanceFace animation will be jointly done with body animation (version 2), because the two will be implemented in IM-1 3D player together. The conformance testing of FBA and 2D mesh is expected to be done by March. The conformance document of Version 1 is due Feb. 1, 2000.

Version 2 - ProfilesIn Version 2, Simple FBA profile is supported. Other profiles (e.g., 3D mesh, hierarchical 3D mesh, and hybrid) will remain in ‘Profiles under considerations’ document. It was noted that for Version 2 profiles the conformance testing has to be finished by March 2000 before the elevation of conformance document to the next level. For Simple FBA profile, Lucent, EPFL, and AT&T are the supporting companies.

For 3DMC related profiles, decision was made that we support it in Version 3 with the following schedule:

• 99/12/10: Approval of Visual PDAM2

• 99/12/13: SC29 sends out the Visual PDAM2

• 00/03/18: Editorial Group produces draft of FPDAM and DOC of PDAM

• 00/03/20: Plenary approves FPDAM and DOC of PDAM

• 00/03/20: SC29 sends out Visual FPDAM 2

• 00/07/20: Editorial Group starts producing draft of DAM and DOC of FPDAM

• 00/07/20: Editorial Group compares draft with actual comments

• 00/07/21: Plenary approves FDAM 2 and DOC of FPDAM 2

Version 2 – FDAM Reference S/WAt this meeting, it is noted that Version 2 Ref. S/W is the superset of Version 1 Ref. S/W. For the maintenance of FDAM Ref. S/W, the following companies are identified. FBA: EPFL/Lucent, INT 3DMC: Samsung

Version 2 - ConformanceFor FBA, some progress was made. For the complete conformance testing, it was needed to have a firm implementation of the S/W in IM-1. Through the joint meeting with Systems, we identified a volunteer (Matthew Leditschke) who will be responsible for the development of 3D player, which will be the basis of FBA confomance testing. The integration of SNHC tools into the 3D player is expected to be done by the end of FEB 2000.

105

For 3DMC, Maui is the starting point for conformance. Samsung and IBM have volunteered to do conformance testing on 3DMC. From Maui, Samsung will maintain the reference S/W of 3DMC. In Im-1, hierarchical 3D node is implemented. The IndexedFaceSet node is not implemented with 3DMC, though. There was an reordering of indices when 3DMC is applied on IFS node. Gabriel Taubin and Julien Signes will come up with a proper text by March. By the next meeting the group expects that we make some progress in porting 3DMC into Im-1.

Version 2 - UpstreamCe_parameters has find its way in upstreaming in Version 2. There was another proposal from Samsung to provide an upstream capability in 3DMC. This issue is modified by the proposer and has become a Systems issue to reload ‘any’ node in Systems when it is broken. This issue will be handle with Systems in March.

After Version 2 – Generic 3D animationCompared to the attention it has obtained recently, generic 3D animation is still at the discussion stage. We had some input documents with more evidences to pursue it after Version 2. The group decided to continue the AHG till March. Upon successful completion, the group will issue a Call for Proposals in March.

Output Documents from MauiSNHC produced or contributed to the following output documents from Maui: DoC on ISO/IEC 14496-2/AMD1 FPDAM N3055 Text of ISO/IEC 14496-2/AMD1 FDAM N3056 Text of ISO/IEC 14496-2/AMD2 FDAM N3146

- Version 3 3DMC profiles Text of SNHC part of ISO/IEC 14496-4 FDIS (Conformance) N3067 Text of SNHC part of ISO/IEC 14496-4/AMD1 (Conformance) N3065 SNHC Work plan on conformance N3147

Ad Hoc Groups for MarchThe following AHGs were established by SNHC:

Doc. No. TitleN3149 AHG on FBAN3150 AHG on study of generic 3D animationN3148 AHG on 3DMC conformanceN3172 AHG on Editing the Version 3 Visual Profiles

Maui ParticipantsMany thanks to the following individuals who participated in SNHC meetings in Maui:Name Company Country E-mailDokyoon Kim Samsung AIT KR [email protected] Petajan Lucent - Bell Labs US [email protected] S. Jang Samsung AIT KR [email protected] Bossen EPFL CH [email protected] Taubin IBM Research US [email protected] Ostermann AT&T Research US [email protected] Jin Han Samsung AIT KR [email protected] K. Capin EPFL CH [email protected] Ho K-JIST KR [email protected]

106

Annex 10ISG meeting report

Source: ISG ChairEditor: Marco Mattavelli (EPFL)

OverviewFour main topics have been the subjects of the Implementation Subgroup activity in Maui:

1. MPEG-7 XM development for video/audio descriptors and Descriptor Schemes,2. SNHC QoS through back-channel,3. IPMP interface implementation issues,4. Complexity of video tools:

- Encoder optimization: motion estimation algorithm complexity- AC prediction complexity for the Video Streaming Profile.

Input contributions w.r.t. the above items are summarized according to the following table:

MPEG-7 XM developmentM5317 Report of AHG on XM Development. Stephan Herrmann

QoS BackchannelM5313 Tutorial example of MPEG-4 back-channel. Gauthier Lafruit

IPMP interface implementation issuesM5425 MPEG-4 IPMP, OPIMA and SDMI: An

integration attempt.M. Balestri, S. Dal Lago, G. De Petris, A. Varesio CSELT

M5547 Necessity of basic regularity of IPMP and summary of past discussions.

Itaru Kaneko

M5448 IPMP-delay for trans-coding of MPEG-4. Itaru KanekoM5550 Implementation and synchronization

considerations on IPMP and Audio BIFS decoders embedded in the MPEG-4 Systems layer.

Marco Mattavelli, Giorgio Zoia

Complexity of video toolsM5528 Report of Ad Hoc Group on Encoder

Optimization.Tihao Chiang and Huifang Sun

M5321 Hardware/software complexity issues between two fast motion estimation algorithms: the Advanced Diamond Zonal Search with Embedded Radar (ADZS-ER) and Diamond Search (DS).

Alexis M. Tourapis, Oscar C. Au, Ming L. Liou, and Guobin Shen

M5453 Report on Performance of Fast Motion Estimation using Motion Vector Field Adaptive Search Technique (MVFAST).

Prabhudev Irappa Hosur and Kai-Kuang Ma.

M5476 Verification Report of Performance Evaluation of Fast Motion Estimation Algorithm: MVFAST.

Chengyu Xiong, Weisi Lin

Detailed Report

108

MPEG-7 XM developmentSeveral joint meetings have been devoted to discuss and agree on the steps for the integration of Description Schemes in the MPEG-7 XM. The main obstacle for the integration of both normative and non-normative Description Schemes is the non-availability of the DDL parser that is currently under development.The solution found and on which a consensus has been reached is, on one side, to wait for the first preliminary API of the DDL parser for the actual XM integration, while in parallel to develop the applications corresponding to the Description Scheme as C++ classes. Then, when the DDL parser API will be fully defined the development of the application can be finalized. In more details, this means that DS proposers will start by using a XML parser to develop the C++ application (this will enable to start integration but without the validation). Then this C++ application development will be updated when the DDL parser will be ready (including the validation). Such solution seems to be the one that minimize the development overhead and that enable DS proposers to begin the development of the application in C++ using the XML parser, in parallel with the DDL parser development. So as to minimize the upgrade of the developed code to be used by the DDL parser, the DDL parser API should be developed as a superset of the existing XML API. Other decisions taken to further proceed with the integration of video and audio tools are:

A single ad-hoc group is established for the integration of video audio and DS, The integration of missing video tools has been rescheduled. Strict dead-lines for the

availability of software have been approved by the video group. Late submissions will not be considered.

Integration of audio tools has been scheduled.Contributions have also been provided to generate output documents N3115 “MPEG-7 software integration policy and work-plan for DS integration” and N3158 “MPEG-7 Overview V.1”.

SNHC QoS through back-channelThe mechanism of attaching a single upstream to a single or a group of elementary down-streams for QoS purposes (e.g. SNHC_QoS, Video_NewPred) has been verified and approved, jointly with systems. An example reported in contribution M5313: “Tutorial example of MPEG-4 back-channel”, including all aspects of the back-channel mechanism has extensively been discussed for verification purposes. As a result, the Belgian National Body FPDAM-1 ballot comments have been accepted and adequately addressed for Systems Amendment-1.

IPMP interface implementation issuesSeveral relevant contributions have been presented during a ISG-Requirement-IPMP Systems joint meeting. One contribution considered the integration of MPEG-4 IPMP, OPIMA and SDMI functionalities into a unique system (M5425). Such integration is assumed to be done on a single processing platform. Another contribution (M5547) considers the requirements for developing and implementing separately IPMP systems and MPEG-4 systems. Such requirements are essentially the possibility of different companies of developing and assembling into a secure system standard MPEG-4 Systems implementations and proprietary IPMP Systems namely: -independent design, -design level integration and -final assembly. A related contribution (M5548) present the difficulties of implementing such scenario without explicitly considering the delay of IPMP processing in the standard IPMP interface. Another contribution (M5550) consider the possible implementation scenarios of IPMP Systems that satisfy the “MPEG-4 System Decoder Model” and therefore can use the appropriate synchronization mechanisms. According to this contribution only implementations based on a single scheduler can be synchronized using the MPEG-4 System mechanisms. Therefore, “embedding” of IPMP “de

109

facto” decoders into standard MPEG-4 Systems implementations are the only possible implementation scenarios that can guarantee synchronization of elementary streams.

Discussions tried to answer the following questions: Do we agree with the analysis of the problem of M5550? IPMP “de facto” decoders can only be implemented embedding them in the Audio/Video decoders or in the System layer (i.e. the same OS and scheduler)? Is that acceptable from the requirement point of view of independent development of standard MPEG-4 Systems and proprietary IPMP implementations?

No consensus was reached for the answers to these fundamental questions and the IPMP ad-hoc group will continue to study the problems and will try to come to consensus and conclusions for the next meeting.

Complexity of video toolsEncoder Optimization: fast motion estimation.

Two candidate algorithms have been analyzed. They are both based on a recursive diamond search and shows equivalent performances in terms of PSNR. The main difference of the two algorithms is given by the way the diamond search is propagated and by the criterion used to stop the recursive diamond search iteration. The overall complexity of the algorithm is given by the number of search points (number of block matchings), the complexity of the operations related to the definition of the search propagation and the stop criterion is negligible. A more in depth complexity analysis although could measure differences in complexity it is likely to result equivalent to the simple statistic of the number of search points. ISG remarks that the sequence set used to evaluate performances does not seem appropriate for the evaluation of motion estimation algorithms for which more critical sequences such as sport one should be considered. The set used for MPEG-4 video standardization has in fact been chosen for other purposes than motion estimation evaluations.Therefore ISG has proposed to further proceed with tests and simulations using more critical sequences and presenting some additional results regarding the statistics of PSNR losses and the number of block matchings. Document N3141 “Experimental conditions for evaluating encoder motion estimation algorithm” reports the work plan with the schedule of the various steps, the new sequences to be used, and the requirements on the results to be presented during next meeting for the selection of the best algorithm.

AC/DC prediction complexity for the Stream video profile

Implementation complexity concerns have been raised about the inclusion of AC prediction tool in the Stream Video profile. Currently AC/DC is included in the simple profile. Such concerns are based on the complexity of the division operation and on the amount of memory necessary to store the information from the MBs relevant for the prediction. A preliminary example of implementation (not included in an official contribution) shows a size of 16K gates while for instance IDCT complexity is about 25K gates. These results are based on a very straightforward implementation, where no cost/performance analysis is provided.

The relevant question to be answered is whether the high complexity of AC prediction is justified by the coding efficiency gain at medium bit-rates.

The conclusion of the discussion was that a final decision could not be made on the basis of these incomplete evidences. An ad-hoc group will consider contributions for both ASIC based analysis and media processor implementation analysis.

The formal mandate of the ad-hoc is:

110

To assess the additional relative complexity of AC prediction tool in the SVP profile by considering typical implementations on media processors and on ASICs.

To evaluate such complexity considering the coding gain of AC prediction tool in the SVP profile.

MiscellaneousAudio BIFS Conformance

Because some of ISG members were absent at the Maui meeting, a number of discussions have been postponed or reduced to their simplest form: Contributions w.r.t. audio conformance have mainly been analyzed by the audio group with

minor intervention of ISG members (mainly informal discussions with audio and audio conformance chairman).

Multimedia Framework contribution.

A preliminary contribution for the inclusion of QoS work items considering the multimedia terminal as integral part of transmission system has been presented (M5533 “Proposal for MPEG-21: Advanced QoS”, Jan Bormans, Gauthier Lafruit). This proposition is derived directly from previous ISG activities regarding decoder QoS for both natural and synthetic video coding.

111

Annex 11Liaison report

The Liaison Group considered the following Maui input documents

M5302 ITU-T H.245 audio – parameter “maxA1-sduAudioFrames” is missing in MPEG-4 audio

M5607 “ H.263++ & H.26L IDCT specs may differ in ITU & MPEG

M5297 SC32 USge of ISO11179 Metadata Registries – entering metadata

M5298 “ USge of ISO11179 Metadata Registries – queries

ATSC Synchronized Data Broadcast in MPEG-2, the problem of PCR discontinuities.

W3C Happy to liaise with MPEG-7 on XML Schema & Xlinks, Xpointrs and Xpath

OCLC Jane Hunter is liaising, but SC29 is proceeding slowly

M5267 IEC TC100 Response to our Liaison. Kate Grant is liaising.

M5299 “ Framework for standardization of MM Equipment

M5300 “ UI guidelines for MM Equipment

“ Colour measures – Default RGB Color Space (sRGB)

Web3D Plan for convergence of MPEG-4 with Web3D – evaluation of technology and IPR (patents)

M5368 “ Ballot for Web3D to vote on approving convergence of MPEG-4 and VRML97

M5369 “ Review of MPEG-4 Technology

M5370 “ Review of MPEG-4 IPR

M5371 “ X3D – new node “Surface Node” for ApplicationTexture

M5637 RIAA Transmission of Music Metadata

E&S SEDRIS Information

ITU-R Request for Liaison message

IOC Offer of audiovisual content for use in MPEG standardization

List of Organizations with which Liaisons ExistOrganization Responsible IndividualAES Karlheinz BrandenburgAGICOA Didier MaryCEN Francoise PreteuxCISAC Dominique YonDIG Kats IshiiIEC TC100 Kate GrantIETFINTELSAT Surendra SatijaISO / TC 46 / SC 9 Albert Simmonds and Keith HillITU-T SG16 Q11 Yoshihiro KikuchiITU-R WP10C Karlheinz Brandenburg

112

OCLC Jane HunterSMPTE Mike Vetter and Stephen LongW3C Jane Hunter and Frank NackWeb3D (VRML) Julien Signès

113