ncl 2.0: integrating new concepts to xml modular languages · ncl 2.0: integrating new concepts to...

NCL 2.0: Integrating New Concepts to XML Modular Languages

Heron V. O. Silva Rogério F. Rodrigues

Luiz Fernando G. Soares Dpto de Informática – PUC-Rio

R. Marquês de São Vicente, 225 Rio de Janeiro – 22453-900 – Brazil

{heron|rogerio|lfgs}@inf.puc-rio.br

Débora C. Muchaluat Saade Dpto de Eng. de Telecom – UFF

R. Passo da Pátria, 156 Niterói – 24210-240 – Brazil

[email protected]

ABSTRACT This paper presents the main new features of Nested Context Language (NCL) version 2.0. NCL 2.0 is a modular and declarative hypermedia language, whose modules can be combined to other languages, such as SMIL, to provide new facilities. Among the NCL 2.0 new features, we can highlight the support for handling hypermedia relations as first-class entities, through the definition of hypermedia connectors, and the possibility of specifying any semantics for a hypermedia composition, using the concept of composition templates. Another important goal of this paper is to describe a framework to facilitate the development of NCL parsing and processing tools. Based on this framework, the paper comments several implemented compilers, which allow, for instance, the conversion of NCL documents into SMIL specifications.

Categories and Subject Descriptors I.7.2 [Document Preparation]: Markup languages, Languages and systems, Hypertext/hypermedia

General Terms Languages, Standardization.

Keywords NCL, hypermedia connector, composition template, XConnector, XTemplate, SMIL, framework for parsing and processing XML.

1. INTRODUCTION NCL – Nested Context Language – is a declarative language for hypermedia document authoring. The first version of NCL [1] was specified as a unique XML DTD. This paper presents NCL version 2.0, which is specified in modules. This approach allows the combination of NCL modules in different language profiles,

following the proposal of other standard multimedia/hypermedia languages [10].

Besides the new modular structure, NCL 2.0 introduces new facilities to its previous version. Some of them facilitate the authoring process, while others improve the language expressiveness. Among the new concepts we can highlight: the definition of hypermedia connectors and connector bases (to permit link authoring through the reuse of relation definitions); the definition of templates for hypermedia compositions (allowing the specification of semantics for a generic hypermedia composition); the switch element definition (a refinement for the specification of content alternatives); the descriptorSwitch element definition (a refinement for the specification of presentation alternatives); and the possibility of defining duration cost functions for elastic time computation. These new concepts will be further discussed in the next sections.

Why one more authoring language? NCL has never had as its goals to be an entirely new authoring language or to substitute any other language. Instead, it tries to use as much as possible the syntax and semantics defined in standard languages. For example, NCL inherited several modules from SMIL [10]. However, some NCL modules are not found in other hypermedia languages. This is indeed the real contribution expected from NCL, allowing not only the creation of specific languages according to user needs, but mainly the integration of NCL 2.0 modules into other languages. Therefore, the incorporation of new facilities into languages such as XLink [14], SMIL 2.0 [10] and XMT-O [3], guided the NCL 2.0 conception.

In order to promote the use of NCL 2.0, a framework was built, providing code structuring and reusability for the development of NCL document processing tools. The framework was used to implement several compilers. Each compiler is able to receive an NCL document as input, and to generate, as output, a new document description. Among them, one compiler was developed for converting NCL documents into SMIL 2.0 ones. From the experience obtained with the development of this framework, another framework for processing and converting SMIL documents into other representations (for instance, NCL) was also implemented.

Therefore, this paper has two main goals. The first one is to provide a complete view of NCL 2.0 main new concepts,

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. DocEng’04, October 28–30, 2004, Milwaukee, Wisconsin, USA. Copyright 2004 ACM 1-58113-938-1/04/0010...$5.00.

pointing the language features as a support for extending the expressiveness of declarative hypermedia languages in general. The second goal is the description of the framework for parsing and compiling NCL documents, and the presentation of the corresponding instantiated processing tools, besides a processing tool for converting SMIL documents into NCL (instance of another framework).

The paper is organized as follows. Section 2 summarizes the new version of NCL and describes the main features introduced by the language. In order to better illustrate the language, NCL document samples are depicted. Section 3 proposes how some NCL concepts can be explored to augment the expressiveness of other XML languages. Section 4 describes the framework for parsing and processing NCL documents and its instantiations, i.e., the implemented NCL document processing tools. A brief discussion about the framework for parsing and processing SMIL documents is also presented in Section 4. Section 5 offers comparisons with related work and presents conclusions and future steps.

2. NESTED CONTEXT LANGUAGE 2.0 Following the same modularization approach of SMIL 2.0, the functionality of NCL 2.0 is partitioned into eleven functional areas, which are partitioned again into modules. As aforementioned, SMIL modules were reused as much as possible.

Table 1 presents NCL modules, organized by NCL functionalities. The complete definition of NCL 2.0, using XML Schema, is available in http://www.telemidia.puc-rio.br/products/ncl.

Table 1. NCL 2.0 modules and functional areas

Functional Area Modules Structure Structure Components SMIL 2.0 BasicMedia

BasicComposition Interfaces MediaInterface

CompositionInterface AttributeInterface SwitchInterface

Connectors Xconnector CompositeConnector

Linking Linking Composition Templates XTemplate

XTemplateUse Timing BasicTiming

AdaptableTiming Layout BasicLayout Presentation Specification BasicDescriptor

CompositeDescriptor Presentation Control TestAttributes

ContentControl DescriptorControl

Metainformation SMIL 2.0 Metainformation

A first benefit of defining a language using a modular approach is the possibility of creating language profiles. A profile groups language modules, defining an appropriate subset of the language

functionalities for building a specific class of documents. As an example, the NCL 2.0 Language Profile is the complete profile of NCL 2.0 language. It includes all NCL modules and provides all facilities for declarative authoring of NCL documents. Besides the NCL 2.0 Language Profile, other profiles may be built, such as the ones described in Table 2.

Table 2. NCL 2.0 profile examples

Profile Name Descr iption

Basic Media Profile

Allows the creation of hypermedia documents without composite nodes and composite connectors. This profile includes Structure, BasicMedia, MediaInterface, XConnector, Linking and BasicDescriptor modules.

Basic Linking Profile

Allows the creation of links and link bases without composite connectors. This profile includes Structure, XConnector and Linking modules.

XConnector Profile

Allows the creation of simple hypermedia connectors. This profile includes only the XConnector module, which is independent from other NCL 2.0 modules

XTemplate Profile

Allows the creation of hypermedia composition templates and simple connectors. This profile includes XConnector and XTemplate modules.

An NCL document is represented by a composite node, which can contain a set of media nodes or other composite nodes, recursively. Composite nodes may also contain a set of links relating its component nodes, as illustrated in Figure 1. The composition element, representing a composite node, is defined in the BasicComposition module of NCL.

In order to allow the creation of links touching node content parts (anchors) or node attributes, the language provides the definition of interface points. For composite nodes, an interface point called port (Figure 1) can export an interface point of one of its component nodes. Interface point elements are defined in the NCL Interfaces functional area.

Anchors are defined via the area element (Figure 1), which extends the syntax and semantics of the homonym element defined by SMIL and XHTML [13]. Besides allowing the specification of anchors representing spatial portions, through the coords attribute (as in XHTML), and the definition of anchors representing temporal portions, through begin, end and dur (as in SMIL), the area element permits the definition of textual anchors. Textual anchors have a text attribute, which defines the anchor string, and a position attribute, which defines the initial position of the string in the text. Instead of using the position, textual anchors may also use the occurrence attribute, informing the string ith occurrence in the text (e.g. “2” for the second string occurrence). The area element can also define an anchor based on the number of the initial and final audio samples or video frames, through attributes first and last. Finally, the area may be specified just through an anchorLabel attribute. In this case, it is responsibility of the node player to map this label to a content fragment. For instance, as illustrated in Figure 1, the areas with anchorLabel attributes equal to “singer” represent the markup tag with “singer” id in files lyrics01.htm and lyrics02.htm.

NCL links are created by making reference to a hypermedia connector, which defines the relation type without defining the related participants, as it will be discussed in Section 2.2. Thus,

a link must also define a set of binds associating connector roles to node interface points (the link participants). Since a composite node port exports an interface point of an internal node, links among nodes contained in different compositions can be defined (l inkA in Figure 1).

In NCL 2.0, links are grouped into link bases. The set of links of a composite node can be given by the union of several link bases. Besides node reuse, allowed in the previous version of NCL, NCL 2.0 also allows link and link base reuse. The Linking module is responsible for the definition of the l ink and l inkBase elements.

<ncl > . . . <composition i d=" songBook" > <composition i d=" sambaSong" > <por t i d=" por t Samba" component =" samba" / > <audi o i d=" samba" sr c=" samba. wav" . . . > <ar ea i d=" par t 1" begi n=" 2s" end=" 15s" / > <ar ea i d=" par t 2" begi n=" 17s" end=" 30s" / > </ audi o> <t ext i d=" l yr i cs1" sr c=" l yr i cs01. ht m" . . . > <ar ea i d=" si nger Anc1" anchor Label =" si nger " / > </ t ext > <t ext i d=" l yr i cs2" sr c=" l yr i cs02. ht m" . . . > <ar ea i d=" si nger Anc2" anchor Label =" si nger " / > </ t ext > <t ext i d=" si nger 1" sr c=" si nger . ht ml " . . . / > <linkBase> <l i nk i d=" l i nk1" xconnect or =" st ar t s. xml " > <bi nd component =" samba" r ol e=" on_x_pr esent at i on_begi n" / > <bi nd component =" i mg1" r ol e=" st ar t _y" / > </ l i nk> <l i nk i d=" l i nk2" xconnect or =" st ar t s. xml " > <bi nd component =" samba" por t =” par t 1” r ol e=" on_x_pr esent at i on_begi n" / > <bi nd component =" l yr i cs1" r ol e=" st ar t _y" / > </ l i nk> . . . </ l i nkBase> </ composi t i on> <composition i d=" bossaSong" > <por t i d=" por t Bossa" component =" bossa" / > <audi o i d=" bossa" sr c=" bossa. wav" . . . / > . . . </ composi t i on> <linkBase> <l i nk i d=" l i nkA" xconnect or =" meet s- st ar t . xml " > <bi nd component =" samba" por t =" por t Samba" r ol e=" on_x_pr esent at i on_end" / > <bi nd component =" bossa" por t =" por t Bossa" r ol e=" st ar t _y" / > </ l i nk> </ l i nkBase> </ composi t i on> . . . </ ncl >

Figure 1. NCL document example

The next subsections comment the main new features introduced by NCL 2.0.

2.1 Presentation Control The presentation specification of an NCL node is done independent of the node definition, using the descriptor entity [1]. Different presentation specifications can be defined for any given node by associating different descriptors to it. A descriptor can be associated to a node depending on the node type (audio,

video, text, image, composition, etc.), depending on the node itself, depending on a composition that contains the node, depending on a link that touches the node or even depending on the reader’s will. If a node has more than one descriptor specified, the node final presentation descriptor results from the combination of all of them, according to cascading rules [1].

The descriptor element is defined in the BasicDescriptor module. The definition of descriptor elements should be done in the document head, inside the descriptorBase element. A descriptor contains a set of optional attributes, which should be used according to the type of the object to be presented: temporal attribute dur defined by the BasicTiming module, which specifies an explicit ideal duration for the object; an attribute named costFunction defined by the AdaptableTiming module, which can specify metrics for adjusting the content presentation duration; an attribute named player, which identifies the presentation tool to be used; an attribute named region, which refers to a layout region defined by elements of the BasicLayout module, among others. Moreover, aiming at specifying adaptable presentations, each descriptor can actually specify a set of alternative descriptors, instead of a single one. The set of alternative descriptors (called descriptorSwitch in NCL) provides presentation alternatives for the same node. Among the alternatives, one may be selected depending on the presentation platform and final user characteristics. The descriptorSwitch element is specified by the DescriptorControl module.

Another entity introduced by NCL 2.0 is a node that groups a set of alternative nodes, named switch node. This entity can also be used for specifying adaptable hypermedia documents considering contextual information. The ContentControl module specifies the switch element. NCL switch element is similar to the SMIL homonym, but with some differences that will be explained further on.

Figure 2 shows an example of the use of descriptors, switches and duration cost functions. Descriptor d1 is referenced by the img element. It simply references an imageRegion. Descriptor d2 is a descriptor switch, thus allowing multiple alternatives. If the systemBitrate is greater than 56000, descriptor d2Audio will be chosen, otherwise d2Text will be selected. Both text nodes t1 and t2 reference the same descriptor switch d2. If d2Audio is chosen as the final descriptor for these texts, they will be played as synthesized audio, because a TtS (Text to Speech) presentation tool has been set as the player attribute of the descriptor. In this case, the synthesized speech should ideally last for 30 seconds, but it can be shrunk or stretched by 6 seconds (20%). Moreover, a linear cost specifies the presentation quality degradation if this kind of adjustment is applied. Author constraints, together with cost functions, can define metrics for elastic time computation algorithms in hypermedia formatters [8]. If d2Text is chosen, the texts will be shown in the textRegion with their original formats.

In the example, the switch element is defined inside the composition element. It has two text alternatives, depending on the systemLanguage – one version in English and another in Portuguese. Clearly, there are four possibilities for this document: two versions with audio and two versions with text. It is also clear the distinction between content adaptation and presentation adaptation.

Different from SMIL, test attributes for selecting a node alternative are defined in a presentation rule base. Rules can be simple (as illustrated in Figure 2) or compound, and can be reused in more than one switch element. Moreover, rules are not defined through node attributes, as in SMIL, but through bindRule elements. The main reason for adopting this approach is that NCL allows (different from SMIL) the same node to be reused in more than one composition, using the idref attribute [1]. Besides the containment of the bindRule element, NCL switch element differs from SMIL homonym since it can also contain generic composition elements.

<ncl > <head> . . . <pr esent at i onRul eBase> <pr esent at i onRul e i d=" r ul eBdw" var =" syst emBi t r at e" op=" gt " val ue=" 56000" / > <pr esent at i onRul e i d=" r ul eEn" var =" syst emLanguage" op=" eq" val ue=" en" / > <pr esent at i onRul e i d=" r ul ePt " var =" syst emLanguage" op=" eq" val ue=" pt - br " / > <pr esent at i onRul e i d=" r ul eI mg" var =" i mageAval i abl e" op=" eq" val ue=" t r ue" / > </ pr esent at i onRul eBase> <cost Funct i onBase> <cost Funct i on i d=" speechFunct i on" xsi : t ype=" l i near " del t aShr i nk=" 20%" del t aSt r et ch=" 20%" mi nDur Cost =" 2" maxDur Cost =" 2" / > </ cost Funct i onBase> <descr i pt or Base> <descr i pt or i d=” d1” r egi on=” i mageRegi on” r ul e=” r ul eI mg” / > <descr i pt or Swi t ch i d=” d2” > <bi ndRul e r ul e=" r ul eBdw" component =" d2Audi o" / > <descr i pt or i d=” d2Audi o” r egi on=” audi onRegi on” dur =" 30s" cost Funct i on=" speechFunct i on" pl ayer =” t ext Synt het i zer ” / > <descr i pt or i d=” d2Text ” r egi on=” t ext Regi on” / > </ descr i pt or Swi t ch> </ descr i pt or Base> </ head> <body> . . . <composi t i on . . . > <swi t ch> <bi ndRul e r ul e=" r ul eEn" component =" t 1" / > <bi ndRul e r ul e=" r ul ePt " component =" t 2" / > <t ext i d=" t 1" descr i pt or =" d2" . . . / > <t ext i d=" t 2" descr i pt or =" d2" . . . / > </ swi t ch> . . . </ composi t i on> . . . </ body> </ ncl >

Figure 2. NCL document example with presentation control.

Analogous to SMIL, rules can be directly applied to a node, without needing to put the node into a switch. In this case, the rule should be defined as a descriptor attribute. The idea is to allow the NCL formatter to test if the object should participate in the presentation depending on contextual information. For instance, in Figure 2, the image will be shown only if the user enables this feature (imageAvailable test attribute is set to true).

In order to adopt the same structure defined for node switches, and to avoid authors misunderstanding the descriptor rule attribute, the rules for selecting a descriptor from a descriptor switch are also specified using the bindRule element.

2.2 Connectors As previously mentioned, an NCL link refers to a hypermedia connector, which defines the relation semantics. The connector functionality brings new facilities not found in any other hypermedia authoring language, handling hypermedia relations as first class entities [7]. The XConnector module allows the definition of xconnector (hypermedia connector) and connectorBase elements.

A connector is defined by a set of interface points (called roles) and a glue. Each connector role specifies the role of a participant, and the glue describes how participants interact, completing the definition of the relation semantics.

The XConnector module allows the definition of connectors representing synchronization and reference relations. More details about XConnector can be found in [7]. NCL also presents the concept of composite connectors, defined in the CompositeConnector module.

Figure 3 depicts a hypermedia connector and its reuse when defining links. A l ink element has an id attribute and an xconnector attribute, which refers to a hypermedia connector URI. In order to associate nodes to connector roles in a specific link, a bind element has three basic attributes. The first one is called role, which refers to a connector role. The second one is called component, which is used for identifying the node. Finally, the third attribute is called port1, used for making reference to the node interface point (see Figure 1).

xconnector role

xconnector R

node anchor/port/attribute

A C

D

B

link l1R

bind

link l2R

xconnectorxconnector role

xconnector Rxconnector R

node anchor/port/attributeanchor/port/attribute

A C

D

B

A C

D

B

link l1R

link l1RR

bindbind

link l2RR

Figure 3. Connector concept.

2.3 Composition Templates The Composition Templates functionality also brings new facilities not found in any other hypermedia authoring language [7]. The XTemplate module allows the definition of xtemplate (hypermedia composition templates) and templateBase elements. A composition template can be used for embedding semantics into a composite node.

Figure 4 shows an example of a template (audio-with-subtitles) synchronizing an audio node with a logo, several text nodes and a sequence of images. When the audio node starts playing, the logo and the first image node are presented. The logo is presented until the end of the audio. The first image is exhibited during some time and then fires the presentation of a new image. The process repeats through a series of images but the total duration of the image slide show should coincide with the audio duration

1 Note that the port attribute of a bind element can refer to any node interface point, that is, an area (anchor), an attribute, a port or a portSwitch element. When the port attribute is not specified, the connector role applies to the whole node content.

(constraint relation [7]). In parallel with the image presentation, each text node (subtitle) is synchronized with its corresponding audio track. Moreover, each text node offers means for navigating to another text node by user interaction.

Also in Figure 4, the template is used (referenced) by a composition named "sambaSong" that specifies a samba to be the audio node, two lyrics parts to be the subtitle nodes, three image nodes to compose the slide show, and a text giving information about the samba singer. After processing the template (see Section 4), the final document ("processed sambaSong") contains the original composition including the semantics given by the template (image node "logo" and several synchronization links).

img1 img2 imgk

subt1 subt2 subtn

text

...

...

track1 track2 trackn

img 2lyrics1

lyrics2

samba

img 1

S1 S2 SnF1 F2 Fn

S0

H1

C0

M1 Mk-1

img 3

singer

Relation Types (Connectors)Si : star tsFi : finishesM i : meets-star tCi : end constraintHi : hyper l ink

img 2lyrics 1

lyrics 2

samba

img 1

img 3

singer

M1

M2

C0

S0

H1

H2

S1

S2

F2

F1+ =

CompositionsambaSong

processedsambaSong

Composition template “ audio-with-subtitles”

audio

logo

S’ 1 F’ 1

logo

S’ 1 F’ 1

Figure 4. Composition template example

The definition of a composition template is done by a vocabulary, which defines component types, relation types (modeled by connectors) and interface points. A composition template also defines a set of constraints on vocabulary elements, the inclusion of component instances (modeled by nodes) and relationships among them (modeled by links). The XTemplate module is presented in details in [7].

When a language profile uses the XTemplate module, it should include a module for the definition of connectors, such as XConnector. Connectors give the semantics of links defined in a template.

The XTemplateUse module allows NCL composite nodes to use templates. This module defines the xtemplate attribute, which refers to a template URI, and the label attribute, which specifies a label for document nodes and interface points. When a language profile uses this module, it should add xtemplate attribute to composition elements, and label attribute to media objects, compositions and interface point elements, in order to define the associations for template processing. Figure 5 shows the sambaSong composition specified with the template presented in Figure 4. The processed sambaSong composition was already shown in Figure 1. The same template could be applied to bossaSong. As it can be observed, the use of the template spares the author from defining the "logo" image node and the link base inside the compositions.

3. ADDING NCL FACILITIES TO OTHER LANGUAGES As aforementioned, one of the main goals of the NCL 2.0 project was to provide facilities that could be incorporated into other authoring languages. Reference [7] discusses how XConnector and XTemplates modules can be used for adding new facilities to XLink [14]. Considering extensions to SMIL 2.0, a profile called SMIL+XTemplate could be created. This profile would allow the creation of other types of time containers, besides the well-known par, seq and excl elements, which could be used to author SMIL documents.

In order to support the definition of templates, BasicComposition, XConnector and XTemplate modules should be added to the profile. The BasicComposition module introduces the composition element, which may acquire the semantics defined by any composition template. The XConnector module is needed for the definition of connectors used for link creation inside templates. Besides adding these three modules, two new language modules would have to be created, as follows. The first new module would comprise the definition of SMIL links using connectors, similar to what was done in the Linking module of NCL 2.0. The other new module would provide the use of composition templates. This latter module would add the label attribute to SMIL media objects, anchors and standard time containers, similar to what was done in the NCL XTemplateUse module. This new container would allow the use of composition templates inside SMIL documents.

<ncl > . . . <composi t i on i d=" sambaSong" xtemplate="audio-with-subtitles.xml"> <por t i d=" por t Samba" component =" samba" / > <audi o i d=" samba" sr c=" samba. wav" label="audio". . . > <ar ea i d=" par t 1" f i r st =" 2s" l ast =" 15s" label="track"/ > <ar ea i d=" par t 2" f i r st =" 17s" l ast =" 30s" label="track"/ > </ audi o> <t ext i d=" l yr i cs1" sr c=" l yr i cs01. ht m" label="subt". . . / > <ar ea i d=" si nger Anc1" anchor Label =" si nger " label="subtAnchor"/ > </ t ext > <t ext i d=" l yr i cs2" sr c=" l yr i cs02. ht m" label="subt". . . / > <ar ea i d=" si nger Anc2" anchor Label =" si nger " label="subtAnchor"/ > </ t ext > <t ext i d=" si nger 1" sr c=" si nger . ht ml " label="text". . . / > </ composi t i on> . . . </ ncl >

Figure 5. NCL document composition using a template.2

In order to better understand the difference in concepts between NCL and SMIL, Figure 6 shows a SMIL document with semantics similar to the one represented by Figure 1.

2 The complete NCL files and XSLT style sheets used for

processing this example document are also available at http://www.telemidia.puc-rio.br/products/ncl/.

<smi l > . . . <seq i d=" songBook" > <par i d=" sambaSong" > <audi o i d=" samba" sr c=" samba. wav" dur =" 30s" . . . > <ar ea i d=" par t 1" begi n=" 2s" end=" 15s" / > <ar ea i d=" par t 2" begi n=" 17s" end=" 30s" / > </ audi o> <seq> </ seq> <t ext i d=" l yr i cs1" sr c=" l yr i cs01. ht m" begi n=" par t 1. begi n" end=" par t 1. end" . . . / > <t ext i d=" l yr i cs2" sr c=" l yr i cs02. ht m" begi n=" par t 2. begi n" end=" par t 2. end" . . . / > <t ext i d=" si nger 1" sr c=" si nger . ht ml begi n=” l yr i cs1. cl i ck; l yr i cs2. cl i ck" . . . / > </ par > <par i d=" bossaSong" > <audi o i d=" bossa" sr c=" bossa. wav" . . . > </ par > </ seq> . . . </ smi l >

Figure 6. SMIL document example cor responding to NCL example in Figure 1

The SMIL document is simpler and smaller than the NCL processed specification. Maybe this is not so evident because, for space restriction reasons, only two of the twelve NCL synchronization links are shown in sambaSong composition link base (Figure 1). However, the use of templates counterbalances this NCL relative disadvantage (Figure 5), besides allowing the definition of other composition types.

The NCL samba-with-subtitles composition template defines synchronization semantics that needs the definition of an internal sequence composition in SMIL and the explicit declaration of a logo image node. These definitions were not necessary in Figure 5 document. This is not so critical in this particular example, but when more complex and elaborated structures are defined (synchronization relationships needing many nesting levels of sequence and parallel compositions, automatically introduction of several nodes, etc.) and repeated in several document parts (or even different documents), templates can significantly reduce the authoring effort.

It is worth mentioning that two basic templates (smilSeq and smilPar) could be created to add the semantics for sequence and parallel temporal relationships, respectively, to NCL compositions. As an example, Figure 7 shows the songBook composition, of Figure 1 and Figure 6, specified using a particular type of smilSeq template, named smilSeqComp. This template synchronizes a list of NCL compositions in sequence, where the compositions are linked through their ports labeled seqNodePort. The definition of composition ports prevents links to cross composition boundaries, therefore ensuring the document compositionality property. This feature is extremely important for checking document properties formally [7]. The example in Figure 7 also illustrates the definition of nested composition template references.

<ncl > . . . <composi t i on i d=" songBook" xt empl at e=" smilSeqComp" > <composi t i on i d=" sambaSong" l abel =" seqNode" xtemplate="audio-with-subtitles.xml"> <por t i d=" por t Samba" component =" samba" l abel =" seqNodePort" / > . . . </ composi t i on> <composi t i on i d=" bossaSong" l abel =" seqNode" xtemplate="audio-with-subtitles.xml"> <por t i d=" por t Bossa" component =" bossa" l abel =" seqNodePort" / > . . . </ composi t i on> </ composi t i on> . . . </ ncl >

Figure 7. NCL document structured using template nesting

As aforementioned, the main advantage of composition templates is the possibility of defining arbitrary relation structures that can be reused, thus simplifying the authoring task. Furthermore, when using templates, NCL connectors can be explored for relation specifications. Connectors are more expressive than the begin and end events of SMIL for node synchronization (Figure 6). For example, connectors can use different event transitions in its specifications, as starts, stops, aborts, ends, pauses, resumes and abortsFromPaused for defining causality relations. Moreover, connectors can define multipoint and constraint relations. All these features allow authors to define more precisely their presentation wishes [7].

4. PROCESSING NCL DOCUMENTS In order to promote the use of NCL, a framework was developed, providing code structuring and reusability for the implementation of NCL processing tools. The framework aims at facilitating the development of compilers for translating NCL documents into different languages or document models.

The framework was modeled using an object-oriented approach and implemented in Java, using the JAXP package [4]. It is composed of ten classes representing the NCL Functional Areas (except Metainformation) and one class providing the basic methods for reading and compiling a document. The framework presented in this section is defined for the NCL 2.0 Language Profile. Nevertheless, it could be simplified for other profiles, like the ones exemplified in Table 2.

Figure 8 depicts the framework logic. The framework implements methods for traversing the DOM tree representing the XML document, following the Template Method design pattern [2]. This pattern defines generic algorithms (concrete method implementations) for each class representing each functional area, and delegates certain steps to the functional area subclasses (compiler implementation of functional areas), through the definition of abstract methods. As an example, consider the following abstract methods of the NclComponentsParser class for the Components functional area: createComposition, createBasicMedia and addChildToComposition. The first one is invoked when a composition object should be handled. Analogously, the second one is called for creating a media object. Finally, the third one should be implemented to insert the composition or media object created into the parent composition object. When implemented,

these abstract methods will actually allow creating and manipulating objects of the target data structure (right-lower arrow in Figure 8).

Templates

NCL Document

NCL Document Processed

Java Objects

Abstract Methods

NCL Parser Framework

Templates

NCL Document

NCL Document Processed

Java Objects

Abstract Methods


Figure 8. Framework for parsing and processing NCL documents.

The space limitation does not allow a deep discussion of each class in this paper. However, let us give a general idea about another important class related to templates. The TemplateProcessor class represents the Composition Templates functional area. This class is responsible for processing a composition element that uses a template, and returning another composition element with the template semantics added. This class also implements a method that analyzes the whole document, processes the templates on all compositions3, and generates the final NCL document with the processed compositions (right-upper arrow in Figure 8).

The template processing implementation is illustrated in Figure 9. First, the original composition element is cloned. Component instances defined in the template are then added to the cloned composition (e.g., the logo specified in Figure 4). Afterwards, the first XSL transformation [15] is created to apply the template semantics to the cloned composition. After applying the first transformation (Template XSLT), the processed composition is built. However, one last step is still necessary: all constraints [7] specified in the template are used to create another XSL transformation (Constraint XSLT), which generates error reports if constraints are not met.

Figure 9. Template processor

The framework for parsing and processing NCL has already four instantiations, as illustrated in Figure 10. The first instance is the NCL-NCM Compiler, which reads an NCL document and

3 When the document has more than one composition element

referring to templates (as in Figure 7), the processor starts with the innermost composition, and then proceeds with the outer ones.

generates NCM objects, based on Java, for the HyperProp Formatter (an NCL player) handling [9]. The data structure generated can be used in the HyperProp System (and its authoring tools). Since both NCL and HyperProp are based on the same conceptual model, the implementation of the abstract methods and classes was straightforward.

The second framework instance is the NCL-Formatter Compiler. It reads an NCL document and generates objects following the internal data structure of the HyperProp Formatter [8]. The abstract methods and classes were specialized based on the formatter pre-compiler functions, which build a data structure driven to document presentation control. Before the implementation of this compiler, an NCL document had to be imported to the system using the NCL-NCM Compiler (previously implemented inside the HyperProp authoring tools), and then converted using the formatter hard-coded NCM-Formatter Compiler. NCL documents can now be played directly by the formatter, without the support of the NCM model and the use of HyperProp authoring tools. This compiler is particularly useful when only the formatter is available on the presentation platform. Devices with limited resources, like mobile phones and interactive TV set-top boxes can take advantage of this implementation.

Another framework instance is the NCL-SMIL Compiler. It reads an NCL document and generates a SMIL 2.0 document. Some NCL elements, like the layout element, were easily adapted to the SMIL specification. Other modules that were brought into NCL from SMIL were also quite simple to compile. NCL composition elements are mapped to SMIL par containers, while link synchronization is done via SMIL events.

NCLDocument


NCMObjects

NCL-NCMCompiler

Formatter Objects

NCL-FormatterCompiler

SMILDocument

NCL-SMILCompiler

Templates, Connectors, ...

NCLDocument

NCL-NCLCompiler

Switch EvaluationCompiler

NCLDocument

NCLDocument


NCMObjects

NCL-NCMCompiler

Formatter Objects

NCL-FormatterCompiler

SMILDocument

NCL-SMILCompiler

Templates, Connectors, ...

NCLDocument

NCL-NCLCompiler

Switch EvaluationCompiler

NCLDocument

Figure 10. NCL compiler framework instantiations

The main difficulties to implement the NCL-SMIL Compiler occurred with entities that are present only in NCL, such as descriptors, ports and connectors. The current implementation only considers the descriptor information that matches SMIL attributes, such as the region attribute, which is copied to all elements that refer to the descriptor. Considering the port element issue, a table is constructed mapping each port into a document component. This table is used to determine link endpoints. The major problem, as expected, is concerned with the use of connectors. To overcome this problem, a class was

CompositeElement

Template

Template ProcessorCloned

Composite

Template XSLTXSLT

Processor

Constraint XSLT

Processed Composite

XSLTProcessor

Add Component

Instances

CompositeElement

Template

Template ProcessorCloned

Composite

Template XSLTXSLT

Processor

Constraint XSLT

Processed Composite

XSLTProcessor

Add Component

Instances

created to simulate the behavior of a connector in SMIL: SmilConnector. When compiling a connector, an instance of a SmilConnector is created, determining what should be done in the SMIL document to provide the semantics represented by the connector. This class tries to extract, from the connector definition, which SMIL elements and which begin and end event attributes could be used to provide the connector semantics. This algorithm usually works well when the connector uses only starts and stops transitions. Unfortunately, some NCL connector semantics cannot be expressed in SMIL documents.

The last compiler developed is the NCL-NCL Compiler. This compiler implements all abstract methods of the framework simply generating a copy of the XML input document. This compiler was designed to become an adaptive parser framework for NCL documents. An instance of this adaptive framework was implemented to compute all the NCL switch and descriptorSwitch elements that can have their presentation rules resolved before starting the document presentation. It generates another NCL document with the chosen element replacing the switch (or descriptorSwitch), or another switch (or descriptorSwitch) element with fewer alternatives. This kind of compiler is used on server-side and proxy adaptations to enable sending an adapted document to the client, instead of the original requested one.

Based on the NCL compiler framework implementation, we developed a similar framework for parsing SMIL documents and converting them into other representations. The framework was instantiated by other four compilers: SMIL-SMIL, SMIL-NCL, SMIL-NCM and SMIL-Formatter.

Analogous to NCL-NCL compiler, the first compiler was extended to implement SMIL adaptation strategies at server-side. SMIL-NCM, SMIL-NCL and SMIL-Formatter reinforce NCL and SMIL interoperability.

5. RELATED AND FUTURE WORK SMIL 2.0 is the W3C recommendation for authoring interactive multimedia documents. Now, the question raised in Section 1, "Why proposing a new authoring language if there are standards with similar goals?", can be considered more deeply.

Representing relationships through compositions with embedded semantics, which is the SMIL time containers approach, makes the authoring task easier, as authors can specify with a single composition what would be alternatively specified using several links. On the other hand, with this approach, more complex relationships among document components must be built using a hierarchy of basic compositions. This is not always desirable, since it obliges the document logical organization to match its temporal presentation structure.

Specifying synchronization relationships using links, which is the NCL approach since its first version [1], preserves compositions for creating context relationships among document components. Besides that, the use of links for defining temporal synchronization inside a composition allows an author to define any type of synchronization relationship among a group of components. This approach gives more flexibility, since the author does not need to be limited to predefined types of

compositions provided by the language. Therefore, the author is able to structure the document independently of its synchronization relationships.

Given the pros and cons of using compositions and links to represent relationships, the ideal scenario would be to provide all the possibilities to the author: the use of compositions for logical structuring, the use of links to specify synchronization, and the use of compositions with semantics, such as temporal semantics, when the logical structure matches the presentation structure. It is even better if the authoring language allows the creation of composition types with the semantics desired by the author, to be used according to his(her) needs. Thus, the presentation structure of a document could be adapted to its logical structure, and not the opposite.

NCL 2.0 provides all these possibilities to the author through the concept of composition templates. Predefined composition types, as the ones provided by SMIL, can be seen as templates. Using the XTemplate module, new composition templates can be created, generalizing the composition types an authoring language provides.

These new paradigms justified the specification of a new modular language, whose modules can be used for extending and incorporating new facilities into existent standards. Following this procedure, the XLink+XTemplate extension was proposed [7]. An ongoing work is the implementation of the SMIL+XTemplate profile, using XConnector and XTemplate to extend SMIL functionalities.

Besides the main difference about the use of compositions, other comments can be highlighted when comparing NCL and SMIL. Apart from the possibility of specifying content alternatives, through the switch element, identical to what is provided by SMIL, NCL 2.0 allows the specification of presentation alternatives for the same node. Presentation alternatives can be specified through the descriptorSwitch element, which can be associated to a node by a link, by a composition or directly by the node itself. In SMIL, the node presentation characteristics are attached to the node, except the spatial layout information. In order to specify presentation alternatives for a node in a SMIL document, it is necessary to define a switch element containing repetitions of an element referring to the same content and associating different presentation information to them. In NCL, the author can specify just one node with the desired content, and associate a set of alternative descriptors, represented by a descriptorSwitch element.

NCL provides multipoint links representing relationships with causal or constraint semantics, according to the connector used. Moreover, the same relationship can relate different types of events, besides the traditional selection and presentation events, usually comprised. SMIL 2.0 only provides single-headed links, which can be fired by temporal events. The possibility of using events in the temporal specification of a SMIL 2.0 document gave sufficient flexibility to the language, compared to its previous version. However, in order to specify complex relationships, the author must combine the use of par, seq and excl time containers with the use of events and links. In NCL 2.0, even the most complex relation can be abstracted by a single connector and used just like the simplest one.

The XConnector module comprises the definition of synchronization and reference relations. However, the connector concept can be applied to define any relation type. For instance, we can have a version connector to represent the ancestor/descendant relation used in mechanisms for object versioning control, or we can have a channel connector to represent the publish-subscribe paradigm used in notification mechanisms. Based on new types of connectors, the XTemplate module can be used for defining new composition semantics, different from the spatial and temporal semantics already available. For example, we can create composition templates to group versions of the same object and its derivation graph. The development of new language modules to create other types of relations is in our plan for future work.

Another issue to be commented is about the flexible temporal behavior of objects. Since NCL 1.0, object durations can be defined with ideal, minimum and maximal values. This facility was included in SMIL only in its version 2.0. However, NCL allows, in addition, the specification of cost functions to help the formatter making temporal adjustments during document presentation, aiming at reducing presentation quality loss.

It is important to highlight that, although NCL is a hypermedia authoring language, the use of its modules are not restricted to extending other hypermedia languages. As an example, there is a current project to define workflow patterns [12] and to extend workflow languages [11] using NCL templates and connectors.

In order to allow these extensions, we are currently working on the specification of XTemplate as a new language with the possibility of defining XTemplate profiles. The goal is to provide a more general way for defining component templates (not only composition templates) and to allow, for example, the use of template facilities without obliging the use of hypermedia connectors.

With the MPEG-4 standard [3, 6], other declarative languages are emerging for specifying multimedia documents (or scenes): XMT-A and XMT-O [3, 5] are examples. We intend to extend the processing tools in order to investigate the possibility of converting NCL documents into these formats. Since XMT-O is a SMIL-hosting language, and XMT-O can be directly converted into XMT-A, almost all NCL-SMIL conversion code may be reused. The result of these conversions could be useful for NCL document transmission using MPEG-4 transport protocols.

Although this paper presents a complete overview of the NCL version 2.0 declarative language for authoring hypermedia documents, the most important modules of NCL 2.0 are, with no doubt, the ones that provide the definitions of connectors and composition templates. These two modules offer new contributions to hypermedia authoring languages, considering relations as first-class entities and allowing the definition of any semantics for a composite node.

The implementation of a framework for parsing and processing NCL documents is another contribution of this work. The framework includes a template processor that uses standard XSL transformations to generate a final NCL document without templates. Thus, the use of composition templates is transparent to the translation of NCL documents into another language or model.

From the experience obtained with the development of NCL and SMIL compiler frameworks, common characteristics could be identified. Therefore, another ongoing work is the specification of a meta-framework for semi-automatic generation of frameworks for building compilers of modular XML authoring languages.

Another approach under evaluation is to generate the structure of the SMIL document (in the NCL-SMIL conversion) from the temporal graph used by the HyperProp Formatter to control the document presentation. We believe that this graph may give an optimized structure of SMIL containers, permitting to identify sequence and parallel patterns, thus reducing the number of SMIL events used to express the semantics of the original NCL document.

The complete reference for NCL language can be found in http://www.telemidia.puc-rio.br/products/ncl/. The address http://www.telemidia.puc-rio.br/products/compilers/ contains the implementations of the compilers described in this paper. Finally, the downloadable version of the HyperProp formatter is available in http://www.telemidia.puc-rio.br/products/formatter/.

6. REFERENCES [1] Antonacci M.J., Muchaluat-Saade D.C., Rodrigues R.F.,

Soares L.F.G. Improving the expressiveness of XML-based Hypermedia Authoring Languages, Proc. of the Multimedia Modeling Conference'2000, Nagano, Japan, 2000.

[2] Gamma, E. et al. Design Patterns: Elements of Reusable Object-Oriented Software. Addison Wesley, 1995.

[3] ISO/IEC 14496-1, “ Information technology – Coding of audio-visual objects – Part 1: Systems - Amendment 2: Textual Format” , May 2002.

[4] “JAXP – Java API for XML Processing” , http://java.sun.com/xml/jaxp.

[5] Kim M., Wood S., Cheok L.T. Extensible MPEG-4 Textual Format (XMT), Proceeding of the ACM Multimedia 2000, Los Angeles, USA, 2000, pp. 71-74.

[6] Koenen R. Overview of the MPEG-4 Standard, Technical Report - ISO/IEC JTC1/SC29/WG11 N4668, 2002.

[7] Muchaluat-Saade D.C., Soares L.F.G. XConnector & XTemplate: Improving the Expressiveness and Reuse in Web Authoring Languages, The New Review of Hypermedia and Multimedia Journal, Taylor Graham Publisher, 8, pp. 139-169, 2002.

[8] R.F. Rodrigues, L.F.G. Soares, Inter and Intra Media-Object QoS Provisioning in Adaptive Formatters, ACM Symposium on Document Engineering – DocEng’03, Grenoble, France, November 2003.

[9] Soares L.F.G., Rodrigues R.F., Muchaluat-Saade D.C. Modeling, Authoring and Formatting Hypermedia Documents in the HyperProp System, ACM Multimedia Systems Journal, 8(2):118-134, March 2000.

[10] Synchronized Multimedia Integration Language (SMIL 2.0), W3C Recommendation, August 2001.

[11] Thatte, S. et al , Business Process Execution Language for Web Services (BPEL4WS 1.1). http://dev2dev.bea.com/techtrack/BPEL4WS.jsp

[12] van der Aalst, W.M.P. et al, Workflow Patterns. Distributed and Parallel Databases, 14(3), p. 5-51, July 2003. http://tmitwww.tm.tue.nl/research/patterns/.

[13] XHTML 1.1 - Module-based XHTML, W3C Rec., May 2001.

[14] XML Linking Language (XLink) Version 1.0, W3C Rec., June 2001.

[15] XSL Transformations (XSLT) Version 1.0. W3C Recommendation, November 1999.