EMML is a standard model proposed by mashup vendors

Arun Viswanathan is a Technology Lead with the Java Centre of Excellence in SETLabs, Infosys Technologies, Bangalore. He works on Java and Java EE technologies. He can be reached at Arun_Viswanathan01@infosys.com.

Mashups are an architectural style that combines data and/or content from different data sources or sites. Mashups are normally differentiated based on the use, architecture style, and data. While consumer mashups have been in use for a while, we're now see them moving into the enterprise. In a general sense, what differentiates a consumer mashup from an enterprise mashup scene is that enterprise mashups are built following standard guidelines such those promoted by the Open Mashup Alliance (OMA), a standard model proposed by mashup vendors.

The OMA is a federation of some of the top vendors in mashup space. The alliance has been set up with the aim to have a standardized, consistent, and interoperable way of developing and deploying the mashups. The principles of the organization are open and the specification and runtime implementation are made available to anyone to use through the Creative Commons License.

The OMA defines an Enterprise Mashup Markup Language (EMML) which is used to define mashups in a standardized manner. The mashup defined thus can be deployed in any of the mashup runtimes which is implemented as per the specifications provided by the OMA.

In this article, I examine at the importance of OMA, the mashup architecture proposed by OMA, and the ease with which developers can create, deploy, and test a mashup developed as per the EMML specification.

The Need for Standard Mashup Development

There are several reasons why I think a standard mashup development platform is a necessity:

The mashup space is fragmented with lots of vendors. Currently there is no single dominant mashup platform.

All applications do not provide a web API which is the one of the main inputs for a mashup.

There is also splintering of widget models developed by large enterprises.

Access to accurate data sources within the network is required in a consistent manner.

The mashup marketplace is still developing. Since there was no dominant platform there was a need to provide a standard way of developing mashups to accelerate their use. A unified mashup model:

Can increase the software quality

Lower cost of IT since they wouldn't need to develop their own ways of mashing

Bring about a standardization and consistency of approach

Help drive innovation

Improves the portabilit of mashup designs and allows interoperability of mashup solutions, thus reducing the risk and cost to organizations that are adopting enterprise mashups.

Provide an easy way to create mashups from different sources

The Enterprise Mashup Markup Language (EMML)

OMA defines an XML markup language -- EMML -- for creating enterprise mashups. EMML is declarative mashup domain-specific language which eliminates complex and procedural programming requirement to create mashups. It is an open specification and the language is free to use. EMML will thus remove any vendor lock-in and allows portability of the mashup solution. OMA has released the EMML specification, EMML schema, and an open source reference implementation where EMML scripts can be deployed and tested. Figure 1 illustrates the EMML architecture.

[Click image to view at full size]

Figure 1: EMML architecture

An EMML file is the mashup script that has a ".emml" extension and uses the Enterprise Mashup Markup Language. The mashup script defines the services, operations and responses to be constructed based on the results generated. The input to the EMML script could be through any

of the data sources such as XML, JSON, JDBC, Java Objects or Primitive types. EMML provides a uniform syntax to call any of the service styles -- REST, SOAP, RSS/ATOM, RDBMS, POJO or web clipping from HTML pages. Complex programming logic is also supported through an embedded scripting engine which supports JavaScript, Groovy, JRuby, XPath and XQuery. The EMML script is then deployed on to any J2EE compliant application server where the EMML runtime has been deployed. The mashup is then accessible as REST service using a URL with the mashup name. The Mashup service returns the result in XML format.

The Structure and Content of a Mashup Script

The Mashup script has <mashup> as the root element. Within the root element, you can add other elements to define variables, invoke other services, and process the results to return the mashup result. EMML provides a large set of elements that let you perform different operations easily. Some of the operations supported by EMML include:

Declare mashup input, output parameters and variables to hold input/output for mashup scripts

Declare RDBMS data sources that define connection details.

Issue statements to execute a SQL query to a data source.

Invoke component services such as REST or SOAP web services and HTML- or RSS-based web sites.

Web clipping or screen scrapping functionality for HTML from any URI using XHTML.

Transform intermediate results such as assign one variable to another, filter, group, sort, annotate data across heterogeneous services.

Defining a script to execute at runtime.

Combining results from different component services

Defining the Mashup Output, Input or Intermediate Variables.

Controlling the Mashup Processing Flow through if, for, while, and parallel elements.

Defining and using custom mashup statements with macros.

Send messages to the console or logs to support debugging.

Adding Metadata to Mashups to provide user or any other information.

The result of a mashup script can also be used within another mashup script.

Developing and Deploying Mashups

To illustrate what's involved in building EMML-based mashups, I create Stock Ticker mashup and deploy it in the EMML Runtime reference implementation deployed on Tomcat Web Server. The sample mashup uses web clipping from Google Finance for the stock price details based on ticker code from a HSQL database which is combined with a company specific news feed from Yahoo Finance News to display a real-time stock ticker for few of the popular stocks. The sample also showcases the use of JavaScript by adding some information to the final result. The complete XML and EMML source code for the StockTicker example is available here.

I've tested this example with:

Apache Tomcat 6.0 or any other J2EE compliant Application Server EMML Runtime Engine Reference Implementation 1.0

HSQL library (included in the EMML Runtime)

The first step, of course, is to install and setup the deployment environment:

1. Download the latest Apache Tomcat server from the Apache site (http://tomcat.apache.org/download-60.cgi).

2. Download the EMML related zip file from the OMA site (http://www.openmashup.org/download).

3. Deploy the EMML reference runtime engine web application emml.war in the server by copying the war file to the %TOMCAT_HOME%/webapps folder where %TOMCAT_HOME% is the folder where Tomcat has been installed.

4. Copy the testing class emmlclient.class to %TOMCAT_HOME%/webapps/emml/WEB-INF/classes. This is used to test the mashup through the command line. In this example, however, I will be testing through the URL.

For this example, I'm using a HSQL database to retrieve the stock names and company names. The HSQL server needs to be started before the mashup can be tested.

1. Go to the %TOMCAT_HOME%/temp folder and start the HSQL database using this command:

java -classpath ..\webapps\emml\WEB-INF\lib\hsqldb.jar org.hsqldb.Server

2. From the same folder, start the HSQL Database Manager GUI using this command:

java -cp ..\webapps\emml\WEB-INF\lib\hsqldb.jar org.hsqldb.util.DatabaseManagerSwing

3. Create a table of the name STOCK_TICKER using this query:

CREATE TABLE STOCK_TICKER (STOCKNAME VARCHAR, COMPANYNAME VARCHAR);

4. Insert some values into the STOCK_TICKER table using this query:

INSERT INTO STOCK_TICKER VALUES ('INFY', 'Infosys');INSERT INTO STOCK_TICKER VALUES ('GOOG', 'Google');INSERT INTO STOCK_TICKER VALUES ('MSFT', 'Microsoft');INSERT INTO STOCK_TICKER VALUES ('YHOO', 'Yahoo');

Developing and Deploying a Mashup Script

Here are the steps necessary to create and execute a mashup as per the EMML specification:

1. The first step is to create an emml script as per the specifications. Open a new XML file in any editor and add the following content to it. Save the file as StockTicker.emml in your local hard drive.

<mashup xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xhtml="http://www.w3.org/1999/xhtml" xsi:schemaLocation="http://www.openemml.org/2009-04-15/EMMLSchema ../schemas/EMMLSpec.xsd" xmlns="http://www.openemml.org/2009-04-15/EMMLSchema" name="StockTicker"></mashup>

2. Next add the input and output variables to the mashup script. The output of the mashup could be of type boolean, date, document, number, string or any token that identifies a data type defined by a service. The input of the mashup can only be one of the primitive types. Input cannot be of document type.

<!-- Input and Output tags to declare input/output parameters to the mashup. There can be only one output for a mashup --><input name="ticker" type="string" default="INFY"/><output name="result" type="document"/>

3. For this mashup, I retrieve the list of stock codes and company names from a HSQL database based on which the details are retrieved from Google Finance and Yahoo Finance News sites. EMML provides tags to configure and connect to any database and retrieve the required values. Here is the code that does this:

<!-- Connect to a SQL data source --><datasource url="jdbc:hsqldb:hsql://localhost:9001" username="sa" password="" />

<!-- Execute select statement to retrieve the stock and company name --> <sql query="select stockname, companyname from STOCK_TICKER" outputvariable="itemNames"/>

The output is stored in the variable itemNames as an XML document in this format:

<records><record>

<stockname>INFY</stockname><companyname>Infosys</companyname>

</record>

</records>

4. We would now need to retrieve the stock price information and news for the list of stocks retrieved above. EMML provides a many conditional operations that allow developers to control the processing flow. We use <foreach> tag to loop through a set of nodes:

<foreach variable="value" items="$itemNames/records/record"><!-- Retrieve the ticker code and the company name from the SQL query

result --><assign fromexpr="$value/stockname/string()" outputvariable="ticker"/><assign fromexpr="$value/companyname/string()"

outputvariable="companyname"/></foreach>

5. EMML provides a common command to invoke any publically accessible web service or website using the <directinvoke> operation. I access the Google Finance web page and retrieve the financial information for the ticker through the web clipping feature provided by EMML. Web Clipping converts the HTML result of any URI into XHTML and gives you a clip of the required content from the web page:

6.7. <foreach variable="value" items="$itemNames/records/record">8.9. <template expr="http://finance.google.com/finance?q={$ticker}"

outputvariable="wholeURL"/>10.11. <!-- invoke the Google Finance web page that has stock

information -->12. <directinvoke outputvariable="clipresult"

endpoint="$wholeURL"/>13.14. <assign fromexpr="$clipresult//xhtml:div[@class='g-section

sfe-break-bottom-8']" outputvariable="clipresult2"/>15. <assign fromexpr="$clipresult2//xhtml:h3/string()"

outputvariable="$company"/>16. <assign fromexpr="$clipresult//xhtml:span[@class='pr']"

outputvariable="clipresult3"/>17. <assign fromexpr="$clipresult3//xhtml:span/string()"

outputvariable="$price"/>18. <assign fromexpr="$clipresult//xhtml:div[@id='price-change']"

outputvariable="clipresult4"/>

19. <assign fromexpr="$clipresult4//xhtml:span[@class='chg']/string()" outputvariable="change"/>

20. 21. </foreach>22.

The company name, price, and percentage change details are retrieved from the web page and stored in temporary variables.

23. Next I retrieve the news feed for the provided company using Yahoo Finance News RSS feed. As mentioned earlier this is also accessed using the <directinvoke> command. EMML also provides various commands to transform intermediate results using commands such as filter, sort, group, script, etc. In this sample, I only retrieve the first three news results and title, link, description details from the RSS feed:

<foreach variable="value" items="$itemNames/records/record">

<!-- Get the news feed from Yahoo Finance News --><template expr="http://news.search.yahoo.com/news/rss?

p={$companyname}" outputvariable="newsURL"/>

<!-- Add the additional values to the URL using JavaScript --><script type="text/javascript" inputvariables="newsURL"

outputvariable="newsURL"><![CDATA[ newsURL += "&ei=UTF-8&fl=0&x=wrt";]]></script>

<directinvoke endpoint="$newsURL" outputvariable="companynews"/>

<!-- Filter RSS feed using xpath regular expression function to retrieve only 3 news feeds --><filter inputvariable="companynews"

filterexpr = "/rss/channel/item[position() = (1 to 3)]"outputvariable="companynews"/>

<select inputvariable="$companynews" outputvariable="$companynews" selectexpr="/rss/channel/item"> <columns>

<column>title</column><column>link</column><column>description</column>

</columns></select>

</foreach>

The company news retrieved from the RSS is stored in the temporary variable companynews.

24. Using the required content stored in the temporary variables, a custom result xml is prepared using the <appendresult> command:

<foreach variable="value" items="$itemNames/records/record">

<!-- build the xml result --><appendresult outputvariable="result">

<stock><ticker>{$ticker}</ticker><company>{$company}</company><price>{$price}</price><change>{$change}</change><news>{$companynews}</news>

</stock></appendresult>

</foreach>

25. Finally an additional information tag is added to the final result using JavaScript:

<!-- Add a final source and generated time using JavaScript --><script type="text/javascript" inputvariables="result" outputvariable="result"><![CDATA[var m_names = new Array("January", "February", "March",

"April", "May", "June", "July", "August", "September", "October", "November", "December");

var d = new Date();var curr_date = d.getDate();var curr_month = d.getMonth();var curr_year = d.getFullYear();var curr_hour = d.getHours();var curr_min = d.getMinutes();

// Add order details to EMML Variable - resultvar info = <info/>;info.source = "Google Finance and Yahoo Finance News";info.datetime = "Auto Generated on " + curr_date + "-" +

m_names[curr_month] + "-" + curr_year + " at " + curr_hour + ":" + curr_min;

result.info = info;]]></script>

: Deploying the Mashup Script

When it is time to deploy the script:

1. Start by copying the StockTicker.emml file to the %TOMCAT_HOME%/webapps/emml/WEB-INF/classes folder. To deploy a mashup,

the emml file needs to be copied to the WEB-INF/classes folder of the emml web application and updated in the mashups.xml.

2. Update the mashups.xml files in the %TOMCAT_HOME%/webapps/emml/WEB-INF/classes folder:

3.4. <!--5. Mashups can be invoked using their logical name OR script name.6. e.g.7. http://xyz:8080/emml/name/operation?param1=value1&param2=value28. http://xyz:8080/emml/script/operation?param1=value1&param2=value29. -->10. <mashups>11. <mashup script="StockTicker.emml"/>12. </mashups>13.

14. Restart the Tomcat server

15. Go to the URL http://localhost:8080/emml/StockTicker. All mashups registered with the EMML engine can be accessed by their logical name or file name. The full syntax for a mashup service URL is http://host:port/emml/mashup-file-without-extension[/operation-name][?param-name=value&param-name=value...]

16. The following result is displayed in the browser:

17.18. <?xml version="1.0" encoding="UTF-8"?>19. <xml>20. <stock>21. <ticker>INFY</ticker>22. <company>Infosys Technologies Limited (ADR)</company>23. <price>56.41</price>24. <change/>25. <news>26. <rss>27. <channel>28. <item>29. <title>Infosys welcomes projected reduction in fiscal

deficit (New Kerala)</title>30. <link>http://us.rd.yahoo.com/dailynews/rss/search/Infosys/

SIG=11lda6h80/*http%3A//www.newkerala.com/news/fullnews-59951.html</link>

31. <description>Bangalore, Feb 26 : Efforts to bring down the fiscal deficit to 5.5 per cent from 6.8 per cent was a positive development as the country strengthens its growth after recovery from the global recession, Infosys CEO and MD S Gopalakrishnan said today.</description>

32. </item>33. <item>34. <title>India Stock Index Fluctuates; Infosys Rises,

Mahindra Retreats (Bloomberg)</title> <link>http://us.rd.yahoo.com/dailynews/rss/search/Infosys/SIG=124h235gl/*http%3A//www.bloomberg.com/apps/news?pid=20601091&sid=a62MHSs.HJJ0</link>

35. <description>Feb. 24 (Bloomberg) -- Indian stocks fluctuated. Infosys Technologies Ltd. gained after its rating was raised at Macquarie Group Ltd. Mahindra & Mahindra Ltd. fell for a fifth day amid concern the government may increase car taxes.</description>

36. </item>37. <item>38. <title>Infosys Raised to Outperform at Macquarie on

Earnings Growth (Bloomberg)</title>39. <link>http://us.rd.yahoo.com/dailynews/rss/search/Infosys/

SIG=1242cqk17/*http%3A//www.bloomberg.com/apps/news?pid=20601091&sid=aLrwFpe_ExMs</link>

40. <description>Feb. 24 (Bloomberg) -- Infosys Technologies Ltd. , Indias second-largest software-service provider, had its rating raised to outperform at Macquarie Group Ltd. on potential earnings growth.</description>

41. </item>42. </channel>43. </rss>44. </news>45. </stock>46. 47. <info>48. <source>Google Finance and Yahoo Finance News</source>49. <datetime>Auto Generated on 26-February-2010 at 21:1</datetime>50. </info>51. </xml>52.

53. An error message such as this is displayed in the browser if there are any exceptions while processing the mashup. A detailed stack trace is also logged in the Tomcat log files. EMML also provides the <display> and <assert> commands for help in debugging the mashup script:

54.55. <error>56. <errorMessage>1002</errorMessage>57. <errorMessage><![CDATA[Could not get JDBC Connection; nested exception

is org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (socket creation error)]]> </errorMessage>

58. </error>59.

Conclusion

This article presented an overview of OMA and its mashup specification. EMML supports an extensive set of operations and commands to handle simple to complex processing needs. EMML also supports the results of one mashup as input to another. All in all, this should give you a head start for developing more complicated mashups by referring to the detailed EMML documentation provided in the OMA portal.