synchronized multimedia interface language (smil) and w3c multimedia e-commerce course november 20,...

Synchronized Multimedia Interface Synchronized Multimedia Interface Language (SMIL) and W3CLanguage (SMIL) and W3C

Multimedia E-Commerce CourseMultimedia E-Commerce Course

November 20, 2000November 20, 2000

Mike Christel and Alex HauptmannMike Christel and Alex Hauptmann

CarnegieMellon© Copyright 2000 Michael G. Christel and Alexander G. Hauptmann 2

World Wide Web ConsortiumWorld Wide Web Consortium

• W3C, W3C, founded in October 1994: http://www.w3.org/http://www.w3.org/

• Purpose: develop common protocols that promote Purpose: develop common protocols that promote WWW’s evolution and ensure its interoperability, in part WWW’s evolution and ensure its interoperability, in part by providing:by providing:• A repository of information about the WWW for developers A repository of information about the WWW for developers

and users, especially specifications about the weband users, especially specifications about the web• A sample code implementation to embody and promote A sample code implementation to embody and promote

standards standards • Various prototype and sample applications to demonstrate Various prototype and sample applications to demonstrate

use of new technology use of new technology


More on W3CMore on W3C

• Directed by Tim Berners-Lee, inventor of the WWW Directed by Tim Berners-Lee, inventor of the WWW

• 481 members as of 11/18/2000 (370 Members as of 481 members as of 11/18/2000 (370 Members as of 11/11/1999)11/11/1999)• vendor-neutral forumvendor-neutral forum• http://www.w3.org/Consortium/Member/Listhttp://www.w3.org/Consortium/Member/List

• Specifications must be formally approved by the Specifications must be formally approved by the membership via W3C Recommendation trackmembership via W3C Recommendation track


W3C Recommendation TrackW3C Recommendation Track

• Working DraftWorking DraftWork in progress, commitment to pursue work in areaWork in progress, commitment to pursue work in area

• Candidate RecommendationCandidate Recommendation

Proposed for implementation experience and Proposed for implementation experience and feedbackfeedback

• Proposed RecommendationProposed RecommendationSent to the Advisory Committee for reviewSent to the Advisory Committee for review

• RecommendationRecommendationConsensus within W3C, appropriate for widespread Consensus within W3C, appropriate for widespread

deployment and promote W3C's missiondeployment and promote W3C's mission

(source: http://www.w3.org/Consortium/Process/)(source: http://www.w3.org/Consortium/Process/)


Some W3C SpecificationsSome W3C Specifications

• User Interface DomainUser Interface DomainHypertext Markup Language (HTML), Cascading Style Hypertext Markup Language (HTML), Cascading Style

Sheets (CSS), Document Object Model (DOM), SMILSheets (CSS), Document Object Model (DOM), SMIL

• Technology and Society DomainTechnology and Society DomainPlatform for Internet Content Selection (PICS), Resource Platform for Internet Content Selection (PICS), Resource

Description Framework (RDF), Platform for Privacy Description Framework (RDF), Platform for Privacy Preferences (P3P)Preferences (P3P)

• Architecture DomainArchitecture Domain

Hypertext Transfer Protocol (HTTP), Extensible Markup Hypertext Transfer Protocol (HTTP), Extensible Markup Language (XML)Language (XML)

...Follow links at http://www.w3.org/ for more details…...Follow links at http://www.w3.org/ for more details…


XML: A Quick IntroductionXML: A Quick Introduction

• XML is a method for putting structured data in a text fileXML is a method for putting structured data in a text fileextensible, support for internationalization/localization, extensible, support for internationalization/localization,

platform-independentplatform-independent

• XML makes use of tags in ‘<‘ ‘>’ but is not HTMLXML makes use of tags in ‘<‘ ‘>’ but is not HTMLmeaning of tags like “<p>” and attributes left to the meaning of tags like “<p>” and attributes left to the

application that reads itapplication that reads it

• XML is text, not binary, but is not meant to be readXML is text, not binary, but is not meant to be readXML has strict parsing rules; more verbose that binary but XML has strict parsing rules; more verbose that binary but

zip/gzip, cheap storage, etc. minimize that disadvantagezip/gzip, cheap storage, etc. minimize that disadvantage

• XML is license-free, platform-independent and well-XML is license-free, platform-independent and well-supported, a W3C standard since February 1998supported, a W3C standard since February 1998


SMIL: Synchronized Multimedia SMIL: Synchronized Multimedia Integration LanguageIntegration Language

• SMIL 1.0 specification, June 15, 1998, available at: SMIL 1.0 specification, June 15, 1998, available at: http://www.w3.org/TR/REC-smil/http://www.w3.org/TR/REC-smil/

• W3C SMIL references available at: W3C SMIL references available at: http://www.w3.org/AudioVideo/http://www.w3.org/AudioVideo/

• Purpose: a new language for scheduling multimedia Purpose: a new language for scheduling multimedia presentations where audio, video, text and graphics are presentations where audio, video, text and graphics are combined in real-timecombined in real-time

• Result: an XML application enabling authors to specify Result: an XML application enabling authors to specify what should be presented what should be presented whenwhenXML implies: extensible, support for internationalization and XML implies: extensible, support for internationalization and

localization, platform-independent, editable by hand, etc.localization, platform-independent, editable by hand, etc.


SMIL 2.0SMIL 2.0

SMIL 2.0 has the following two design goals: SMIL 2.0 has the following two design goals:

• Define an XML-based language that allows authors to Define an XML-based language that allows authors to write interactive multimedia presentations. Using SMIL write interactive multimedia presentations. Using SMIL 2.0, an author can describe the temporal behavior of a 2.0, an author can describe the temporal behavior of a multimedia presentation, associate hyperlinks with multimedia presentation, associate hyperlinks with media objects and describe the layout of the media objects and describe the layout of the presentation on a screen. presentation on a screen.

• Allow reusing of SMIL syntax and semantics in other Allow reusing of SMIL syntax and semantics in other XML-based languages, in particular those who need to XML-based languages, in particular those who need to represent timing and synchronization. For example, represent timing and synchronization. For example, SMIL 2.0 components are used for integrating timing SMIL 2.0 components are used for integrating timing into XHTML and into SVG.into XHTML and into SVG.


SMIL TimelineSMIL Timeline

Oct 1996: Workshop Oct 1996: Workshop

Mar 1997: W3C working group on synchronized multimedia Mar 1997: W3C working group on synchronized multimedia

Jun 1998: W3C Recommendation (SMIL 1.0)Jun 1998: W3C Recommendation (SMIL 1.0)

Sep 1998: Timed Interactive Multimedia Extensions for Sep 1998: Timed Interactive Multimedia Extensions for HTML (HTML+TIME), W3C NoteHTML (HTML+TIME), W3C Note

http://www.w3.org/TR/NOTE-HTMLplusTIMEhttp://www.w3.org/TR/NOTE-HTMLplusTIME

Feb 1999: 2nd Working Group on Synchronized Multimedia Feb 1999: 2nd Working Group on Synchronized Multimedia (SYMM) formed (SYMM) formed

http://www.w3.org/TR/NOTE-SYMM-http://www.w3.org/TR/NOTE-SYMM-modules modules

Aug 1999: First working draft (SMIL “Boston”) releasedAug 1999: First working draft (SMIL “Boston”) released


SMIL Timeline, ContinuedSMIL Timeline, Continued

Dec 1999: Chinese Translation of SMIL 1.0 Dec 1999: Chinese Translation of SMIL 1.0

Jan 2000: Realslideshow 2.0 from RealNetworks Jan 2000: Realslideshow 2.0 from RealNetworks

Jan 2000: Internet Explorer 5.5 Preview by Microsoft Jan 2000: Internet Explorer 5.5 Preview by Microsoft (supports selected modules of SMIL “Boston” (supports selected modules of SMIL “Boston”

draft) draft)

Jan 2000: Apple QuickTime 4.1 now a SMIL 1.0 playerJan 2000: Apple QuickTime 4.1 now a SMIL 1.0 playerhttp://www.w3.org/TR/NOTE-http://www.w3.org/TR/NOTE-

HTMLplusTIMEHTMLplusTIME

Jun 2000: Fourth Working Draft of SMIL “Boston” Jun 2000: Fourth Working Draft of SMIL “Boston” released released

Oct 20, 2000: Last call for comments on Public Working Oct 20, 2000: Last call for comments on Public Working Draft of SMIL 2.0Draft of SMIL 2.0


SYMM-WG ContributorsSYMM-WG Contributors

Synchronized Multimedia Synchronized Multimedia Working Group contributing Working Group contributing organizations:organizations:

Compaq, CSELT, CWI, Compaq, CSELT, CWI, France Telecom, Gateway, Glocomm, France Telecom, Gateway, Glocomm, IBM, INRIA, Intel, Macromedia, IBM, INRIA, Intel, Macromedia, Microsoft, Netscape/AOL, NIST, Nokia, Microsoft, Netscape/AOL, NIST, Nokia, Oratrix, Panasonic, Philips, Oratrix, Panasonic, Philips, RealNetworks, WGBH, W3CRealNetworks, WGBH, W3C


Example of Contributing Org. AgendasExample of Contributing Org. Agendas

• WGBH a contributor to Synchronized Multimedia Working WGBH a contributor to Synchronized Multimedia Working Group Group

• WGBH partner in CPB/WGBH National Center for WGBH partner in CPB/WGBH National Center for Accessible Media Accessible Media

http://main.wgbh.org/wgbh/pages/ncam/http://main.wgbh.org/wgbh/pages/ncam/aboutncam.htmlaboutncam.html

• Agenda: “make media accessible to underserved Agenda: “make media accessible to underserved populations such as disabled persons…”populations such as disabled persons…”

• ““NCAM is currently working with the W3C to add to SMIL the ability NCAM is currently working with the W3C to add to SMIL the ability to pause the video track and play extended audio descriptions, which to pause the video track and play extended audio descriptions, which will provide blind users with more information than would otherwise will provide blind users with more information than would otherwise be possible. This capability will be part of the next version of SMIL.”be possible. This capability will be part of the next version of SMIL.”http://main.wgbh.org/wgbh/pages/ncam/webaccess/pivot/pvtaxxmm.htmlhttp://main.wgbh.org/wgbh/pages/ncam/webaccess/pivot/pvtaxxmm.html


SMIL 1.0 ModulesSMIL 1.0 Modules

• StructureStructure

• Meta (“who”, “what”)Meta (“who”, “what”)

• Layout (“where”)Layout (“where”)

• Timing and Synchronization (“when”)Timing and Synchronization (“when”)

• Media Elements (“what”)Media Elements (“what”)

• LinkingLinking

• Content ControlContent Control

SMIL High Level Document Structure SMIL High Level Document Structure <smil><smil>

<head><head><meta><meta>



</meta></meta>

<layout><layout>



</layout></layout>

</head></head><body><body>



</body></body></smil></smil>


SMIL 1.0 StructureSMIL 1.0 Structure

<smil><smil><head><head></head></head><body><body></body></body>

</smil></smil>

These elements used in documents based upon traditional These elements used in documents based upon traditional SMIL profiles.SMIL profiles.


SMIL 1.0 MetaSMIL 1.0 Meta

<smil><smil><head><head>

<meta … /><meta … /></head></head>

</smil></smil>

The meta elements contain information describing the The meta elements contain information describing the document, either to inform the human user or to assist document, either to inform the human user or to assist some automation, e.g.,some automation, e.g.,

<meta name="title" content=”My Italy Trip"/><meta name="title" content=”My Italy Trip"/>

<meta name="copyright" content="©1998 WGBH" /><meta name="copyright" content="©1998 WGBH" />

<meta name="base” content="http://billswin.edu/Italy/"/><meta name="base” content="http://billswin.edu/Italy/"/>


SMIL 1.0 LayoutSMIL 1.0 Layout

<smil><smil><head><head>

<layout><layout>

<root-layout … /><root-layout … />

<region id=“R1” … /><region id=“R1” … />

<region id=“R2” … /><region id=“R2” … />

</layout></layout></head></head>

</smil></smil>

Includes the <layout>, <root-layout> and <region> Includes the <layout>, <root-layout> and <region> elements, and related attributes. elements, and related attributes.


SMIL 1.0 Layout ExamplesSMIL 1.0 Layout Examples

<root-layout height="300" width="450" background-<root-layout height="300" width="450" background-color="#FFFFFF" title="Venezia!"/>color="#FFFFFF" title="Venezia!"/>

<root-layout height="450" width="625" background-<root-layout height="450" width="625" background-color="black"/>color="black"/>

<region id="title" left="5" top="150" width="400" <region id="title" left="5" top="150" width="400" height="200" z-index="1"/>height="200" z-index="1"/>

<region id="videoregion" top="0" left="0" height="240" <region id="videoregion" top="0" left="0" height="240" width="352"/>width="352"/>

SMIL 1.0 Timing and SynchronizationSMIL 1.0 Timing and Synchronization

• Sequence and parallel timelines, via <seq> and <par>

• Timing control properties, via “begin”, “end”, “repeat”, “dur”

• “The accuracy of synchronization between the children in a parallel group is implementation-dependent”• “soft synchronization” vs. “hard synchronization”

• how to realize hard synchronization

<smil><body>

</body>

</smil>


Hard vs. Soft SynchronizationHard vs. Soft Synchronization

• Hard synchronization: Hard synchronization: player synchronizes the children in the "par" (parallel play) element to a common clock

• Soft synchronization: each child of the "par" element has its own independent clock

audio

videoHard sync. case 1: stop audio, e.g., wait to regain lip synchronization

audio

videoHard sync. case 2: drop video frames, e.g., narrative

audio

video Soft sync: audio unaffected


SMIL 2.0 New Focus on Sync. BehaviorSMIL 2.0 New Focus on Sync. Behavior

<par><par> <animation src="..." /><animation src="..." /> ...... <par id="speech" syncBehavior="canSlip" ><par id="speech" syncBehavior="canSlip" > <video src="speech.mpg" syncBehavior="locked" /><video src="speech.mpg" syncBehavior="locked" /> <audio src="speech.au" syncBehavior="locked" /><audio src="speech.au" syncBehavior="locked" /> </par></par> ......</par></par>

There is also a There is also a syncTolerancesyncTolerance that is referenced when that is referenced when syncBehavior is “locked”, to ignore a given amount of syncBehavior is “locked”, to ignore a given amount of slew without forcing resynchronizationslew without forcing resynchronization

SMIL 1.0 Media ElementsSMIL 1.0 Media Elements

<smil><smil>

<body><body>

</body></body>

</smil></smil>

• Includes the media declaration elements <text>, <img>, <audio>, <video>, <textstream>, <animation>, and <ref>

• all absolutely positioned elements (animation, img, ref, text, textstream and video) are contained within a single containing block defined by the root-layout element

SMIL 1.0 LinkingSMIL 1.0 Linking

<smil><smil><body><body>

</body></body>

</smil></smil>

• Includes the <a> and <anchor> elements, e.g., <a href="http://www.w3c.org" ><a href="http://www.w3c.org" >

<text src="media/w3c.txt" region="w3c” <text src="media/w3c.txt" region="w3c” begin="14.05s" dur="15.95s" />begin="14.05s" dur="15.95s" />

</a></a>• Timing applied to HTML <a> and <area> tags could

provide much or all of SMIL functionality; hence, linking modules under review

SMIL 1.0 Content ControlSMIL 1.0 Content Control

<smil><smil>

<head><head>

</head></head><body><body>

</body></body>

</smil></smil>

Content control based on “<switch>” and test attributes:

system-bitrate, system-captions, system-language, system-system-bitrate, system-captions, system-language, system-overdub-or-caption, system-required, system-screen-size, overdub-or-caption, system-required, system-screen-size, system-screen-depth system-screen-depth

synchronized multimedia interface language (smil) and w3c multimedia e-commerce course november 20,...

Documents