integrating peer to peer networks and web services with the grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/tutorial2/fox" euri="gxos://gridform/ggf4/tutorial2/fox" email="[email protected]"mail="[email protected]"

11

Integrating Peer to Peer Networks and Web Services with the Grid

PTLIU Laboratory for Community GridsGeoffrey Fox, Marlon Pierce, Shrideep Pallickara, Choonhan Youn

Computer Science, Informatics, PhysicsIndiana University, Bloomington IN 47404

http://grids.ucs.indiana.edu/ptliupages/presentations/ggf4feb02

[email protected]

Part of GGF4 Tutorial Toronto February 17 2002 Grid Programming with Web Services, Software Components and

Portal Development Tools

04/19/2304/19/23 uri="gxos://gridform/ggf4/tutorial2/fox" email="[email protected]"uri="gxos://gridform/ggf4/tutorial2/fox" email="[email protected]" 22

Classic Multi-Tier Architecture

Web Server

Object BrokerIIOP

Specialized Java Server

HTTP

RMI

Client Tier

Old and New Useful Backend Systems

Back-end TierRaw Resources

Middle TierServers

Web ServiceSOAP + WSDL

RelationalDatabase

“Flat” FileSystem


Object Web View of the World I “Everything electronic” is a resource

• Computers

• Programs

• Data (from sensors to this presentation to email to databases)

• People

“Everything electronic” is a distributed object “All External Interfaces” are defined in XML whether this is a

property (data-structure) or a method (service, function, subroutine)• We can assume that a data-structure property has getproperty() and

setproperty(value) methods to act as interface

• All resources have interfaces

All resources are linked by messages with format and transport protocol to be discussed but as interfaces defined in XML, message structure must be specifiable in XML


Object Web View of the World II All resources have a URI which can be a URL (location) to URN

(property tag)• I could be ssn://abc/de/fghk or my homepage

http://www.w3.org/Addressing/


The 3 Critical XML Interfaces Data (sort of

dumb) Information User

Rendering

(Virtual) XML Data Interface

Raw Data

Processing

Information/Knowledge

(Virtual) XML Information Interface

Render to XML Display Format

RawResources

Server

Clients

(Virtual) XML Rendering Interface


What is a Virtual XML Interface We can specify interfaces in XML but we are not required to

implement in XML. Example 1:We aren’t likely to change syntax of mail

Reply-to:Geoffrey Fox <[email protected]>To: Geoffrey Fox <[email protected]>Subject: A Test for Tutorial

A simple mail messageGeoffrey Fox [email protected] FAX 8128567972Phones Cell 315-254-6387 Home 812323919

But we could specify and indeed store in XML with transport done using conventional SMTP.

So conventional mail is easy to give a virtual XML specification for withname:value becoming<name>value</name> but this is not only way


Mail in XML <mailasxml uri=“gxos://mail/users/gcf/sent/2002/february/290” >

<smtpheaders><reply-to email=“[email protected]” >Geoffrey Fox</reply-to><to email=“[email protected]” >Geoffrey Fox</to> <subject>A Test for Tutorial</subject>

</smtpheaders><smtpbody> <message whitespace="collapse"> A simple mail message</message> <signature personuri=“ssn://123/45/6789” whitespace=“preserve” > Geoffrey Fox [email protected] FAX 8128567972 Phones Cell 315-254-6387 Home 812323919 </signature> </smtpbody>

</mailasxml> Such an interface could be used by “Messaging as a Web service” which

could perhaps integrate e-mail, text chat, instant messenger, voicemail


Quick and Dirty mail as XML If your boss orders you to produce a quick messaging web

service then you might instead define: <kwikmailasxml whitespace=“preserve” >

Reply-to:Geoffrey Fox <[email protected]>To: Geoffrey Fox <[email protected]>Subject: A Test for Tutorial

A simple mail messageGeoffrey Fox [email protected] FAX 8128567972Phones Cell 315-254-6387 Home 812323919

</kwikmailasxml> This is allowed but of course relies on an SMTP parser not an

XML parser to decode message Our messaging web service would allow either form but put

them on different ports. Internally we would convert to the common XML representation


XML in a Nutshell So we have seen .xml files – XML Instances or the

equivalent of Objects in Java There are corresponding .xsd or Schema files which are

the equivalent of Classes in Java• Schema happen to be defined in XML syntax

• Schema totally replace DTD’s

• Originally Schema defined “things”/”properties”/”data” At times in this tutorial, we will use .wsdl files which

define Web Service Interfaces• The wsdl specification separates into components that specify

structure of service which is equivalent to “Abstract Method or Interface” and those that specify actual service (equivalent of program and its methods)


XML Rendering Interfaces The simplest example is XHTML which is HTML in an XML Syntax

• See http://www.w3schools.com/xhtml/default.asp• <BR> in HTML MUST become <br/>• <p> must become <p>Material in Paragraph</p> etc.

Another good example is SVG (Scalable Vector Graphics) which is 2D Vector Graphics and can represent• Macromedia Flash• PowerPoint and StarOffice• Adobe Illustrator/Photoshop etc.• See http://grids.ucs.indiana.edu/courses/xinformatics/svg

Here is a snippet of SVG from above link illustrating text, matrix transformations, positioning and styles<text transform="matrix(1 0 0 1 220.687 216.7256)"

style="fill:#66FF00;stroke:none;"> <tspan x="5" y="0">Spring Semester 2001 MWF 2:30 pm - 3:20 pm </tspan> </text>


XML in a Nutshell Concluded We can browse some examples at:

http://grids.ucs.indiana.edu/courses/xinformatics/educexample and on the following page

Note Schema are labeled by URI’s – their namespaces which allow one to build class libraries• WSDL files are also referenced by their URI’s (namespaces) in the XML

(SOAP) message communicating between 2 Web services

• Note version number in URI http://grids.ucs.indiana.edu/courses/xinformatics/xmlschemaspring02.ppt http://www.w3.org/TR/xmlschema-0/ are reasonable links Always remember that XML defines (potentially complicated)

objects and it can specify both data and methods this could have profound implications for architecture and implementation of Enterprise infrastructure

Use Castor (http://castor.exolab.org/) to link XML & Java “automatically”


A Sample of XML <?xml version="1.0"?> <okc version="3"

xmlns="http://grids.ucs.indiana.edu/okc/schema/admin/ver/3"> <event> <comment> This is a Test </comment> <sender email="[email protected]">Geoffrey Fox</sender> <distribution> Community Grids Project Reports </distribution> <organization></organization> <update createuri="gxos://okc/newsgroups/cgreports/$next“ /> <keywords></keywords> <subject></subject> <message/> <filingdate> 2/6/2002 </filingdate> <cg:messageType

xmlns:cg=“http://grids.ucs.indiana.edu/okc/schema/cg/ver/1” > Test </cg:messageType> </event> </okc>

http://grids.ucs.indiana.edu/schemas/commgrids-v1.xsdhttp://grids.ucs.indiana.edu/schemas/okc-v3.xsd

Schema are here


http://xml.apache.org/soap/faq/faq_chawke_smtp.html

http://www.w3c.org/2000/xp/Group/1/11/16/soap12-primer.html#SMTP

SOAP over HTTP Structure SOAP defines a message with

envelope, header and body The SOAP standard also defines a

binding to HTTP which is equivalent to methodology for delivering message

The SOAP body can specify data as well as fault processing information

SOAP can be transported over standard mail by just putting XML in body of mail – see

HTTP Delivery with address


SOAP Example from W3C I We take November 16 tutorial:

http://www.w3c.org/2000/xp/Group/1/11/16/soap12-primer.html

This is a SOAP Message – defining a reservation request Header has key meta-data of application (namelist m: and

instructions to SOAP (intermediate actor) processors

End HeaderStart Body

Start Header

Start Envelope


SOAP Example (Concluded)

This has the SOAP Body with actual request

End BodyEnd Envelope

Start Body


SOAP Binding to

SMTPfrom W3C

Same example as before

This is very straightforward!


Remarks on SOAP Structure If you look at networking protocols, at each layer one

specifies a header and a body• HTTP has a header telling system where to send information

and some high level specification of what to get

• In example, we see that body of HTTP message is full SOAP envelope and this envelope has SOAP Header and Body interpreted by “application”

• At a lower level in protocol stack, TCP/IP and ATM etc. have their own headers and bodies

• It is the standard Russian Doll situation – each doll has a body (contained inside it) which is the full doll of higher level protocol

HTTPSOAP

SOAP Body


SOAP Attachments

This uses same approach as email or HTTP and is a general way of including binary data in XML – but not understood by most parsers!

Very important?

XML links to MIMEAttachment using absolute or relative URI’s


What is a Web Service I A web service is a computer program running on either

the local or remote machine with a set of well defined interfaces (ports) specified in WSDL

In principle, computer program can be in any language (Fortran .. Java .. Perl .. Python) and the interfaces can be implemented in any way what so ever• Interfaces can be method calls, Java RMI Messages, CGI

Web invocations, totally compiled away (inlining) but The simplest implementations involve XML messages

(SOAP) and programs written in net friendly languages like Java and Python

Web Services separate the meaning of a port (message) interface from its implementation


What is a Web or Grid Service II? The Web is distributed system allowing communities to access

seamlessly heterogeneous resources from heterogeneous clients• Resources are web-pages, instruments, Object repositories,

Simulation codes running on supercomputers …. A Service is a generic application or capability respecting

standards (general web and application specific) allowing multiple providers to compete on a given service• For a simulation to be a web service, we want it to have

standard interfaces so be run from standard “portals” and access “standard format” data

Back endCapability

Middle TierBroker

Portal is customizableUser interface

Resource


Role of Web Services Define interfaces of web applications so that computer-computer

interactions are enabled• Defines virtual XML for all system and application services

WSDL is XML versions of Class and Method definitions SOAP is XML version of message UDDI or WSIL catalogs WSDL based services enabling precise

linkage of them WSFL and WSCL are candidate linkage languages

Security Catalog

PaymentCredit Card

Warehouseshipping

WSDL interfaces

WSDL interfaces


etc.XML WS to WS Interfaces

(Virtual) XML Knowledge (User) Interface

Clients

(Virtual) XML Data Interface

Raw DataRawResources

Raw Data

WSWS

Web Service (WS)

WS

WSWS WS WSWS

Render to XML Display Format

(Virtual) XML Rendering Interface


Some Sort of XML Protocol Stack Protocols are built in a layered

fashion – one on top of another At higher (highest) level a

protocol becomes an API (Application Interface)• Some reserve protocol to denote

the representation of a process involving some negotiation but to my naïve view there is no consensus as to difference between an API and a protocol

We are expanding application (top) layer of OSI protocol stack

UDDI or WSILUDDI or WSIL

WSFLWSFL

WSDLWSDL

SOAP or RMISOAP or RMI

HTTP or SMTP or IIOP or

RMTP


RMTP

TCP/IPTCP/IP

Physical Network

Physical Network


Details of Application Layer UDDI finds where programs are

• remote( (distributed) programs are just Web Services

WSFL links programs together WSDL defines interface (methods,

parameters, data formats) SOAP defines structure of message

including serialization of information HTTP is negotiation/transport

protocol TCP/IP is layers 3-4 of OSI Physical Network is layer 1 of OSI

UDDI or WSILUDDI or WSIL

WSFLWSFL

WSDLWSDL

SOAP or RMISOAP or RMI


RMTP


RMTP

TCP/IPTCP/IP

Physical Network

Physical Network


Role of UDDI with WSDL UDDI plays a role like find or grep under UNIX or Google on the

web or yellow pages in the real world• A critical difference is that UDDI aims at finding resources matching

precise information• Suppose we were a shopping cart service and told by a customer that

we need to contact the SuchandSuch online credit card system• We would not only need to find www.suchandsuch.com but also

identify the exact URL’s for “approval” and “payment” channels – we would need to get the syntax correct for exchanging data

For instance, we would need to know if system used US Canadian or HK dollars

• So UDDI knows about WSDL Interfaces and this knowledge enables programs with such interfaces to link together

UDDI and WSDL together enable dynamic applications Note “service” just means web-based application built according

to agreed interaction rules – such “plug and play” software usually called “components”


Software Engineering and WSDL/SOAP? Software Engineering is a field whose goal is agreed by most

to be important• Namely to codify the building of software so that it is

more re-usable, modular, easier to maintain and easier to build with teams

• However there is not so much agreement as to how to do this and if the field has established itself

WSDL is part of component-based software engineering methodology of the web

As the Web could be a set of processes and threads on a single PC or Palmtop, WSDL can in fact provide software engineering for conventional sequential or parallel computing.

WSDL/SOAP addresses same issues as calling sequences and library methodology in conventional programming


WSDL and SOAP We will use the Gateway Computing Portal as an

example (http://www.gatewayportal.org)• It is largely built using CORBA with a Java Server Pages

front end• http://community.ucs.indiana.edu:8004/GCWS/BatchScriptGen/Main.jsp

Several capabilities have been interfaced using WSDL• Job Submission (11 Methods including execute local and

remote command, copy files etc. as well as Submit Job)• Manage WebFlow Session (67 Methods)• Generate Batch Script (just 1 method but two

implementations developed – one at SDSC and one at Indiana – with UDDI to manage)

• Each is one service – could have used finer grain services• Sample files are at

http://grids.ucs.indiana.edu/ptliupages/presentations/ggf4feb02


Gateway Architecture As needed DoD approved Kerberos/SecureID security,

does not usually link to standard Grid technology (Globus)

backend

Globusor submit to localqueue system

Corba middle tierSECIOP withJava modules

JSP SSL

backend

SOAP wrapperJSP Servlet

CORBA

WSDL

Will replacehistorical CORBABy EJB

Add CastorJava to XML

Pre WSDL Post WSDL


Basic Portal SOAP Interaction

Client WebBrowser(HTML)

User Interface

Web Server

HTTP(s)

UI Server Components(including SOAP Client)

SOAP over HTTP

SOAPServer

Service Components

The deployment descriptor maps the SOAP request to thecorrect service component (ApacheSOAP specific).


Server-Server Communication for Basic SOAP Requests

We have two servers:• User interface (UI) server. Includes components for basic

functionality (like managing user sessions) plus SOAP client modules.

• SOAP server. Converts SOAP requests into calls to the underlying service.

User interacts with UI to make requests for services. UI server constructs SOAP request and acts as the SOAP client.

For example, if UI server is Tomcat, developer uses Apache SOAP’s Java API implementation to create JavaBean components to act as SOAP clients.

SOAP server executes the request (generate batch script) and returns response.

Note that the UI’s SOAP component must match the interface on the SOAP server. This is where WSDL will come in.

UI server formats display for user.


SOAP and Gateway Portal I Having specified service in WSDL, the run-time is implemented in SOAP which is “just” an XML header (info

needed by transport – empty here) and body Here is SOAP transported by HTTP message This is execLocalCommand WSDL operation to run one particular command (ls) on current WebFlow directory

Specify ls as

Argument of operation

HTTP Header

SOAP Envelope and body


And this is the result of ls sent back to client in SOAP over HTTP

SOAP and Gateway Portal II

HTTPHeader

SOAPEnvelopeand body


WSDL Abstractions WSDL abstracts a program as an entity that does

something given one or more inputs with its results defined by streams on one or more outputs.

Functions are defined by method name and parametersmethodname(parm1,parm2, … parmN)• Where parameters are “Input” “Output” or both

In WSDL, we will have a Web Service which like a (Java or CORBA Program) can be thought of as a (distributed) object with many methods• Instead of a function call, the “calling routine” sends an

XML message to the Web Service specifying methodname and values of the parameters

• Note name of function is just another parameter


7 Primitives in WSDL types: which provides data type definitions used to describe the

messages exchanged. message: which represents an abstract definition of the data being

transmitted. A message consists of logical parts, each of which is associated with a definition within some type system.

operation– an abstract description of an action supported by the service.

portType: which is a set of abstract operations. Each operation refers to an input message and output messages.

binding: which specifies concrete protocol and data format specifications for the operations and messages defined by a particular portType.

port: which specifies an address for a binding, thus defining a single communication endpoint.

service: which is used to aggregate a set of related ports


Discussion of 7 WSDL Primitives types specify data-structures which are equivalent to arguments of methods message specifies collections of types and is equivalent to set of

arguments in a method call. Note that it is an “abstract method” in Java terminology

operation is a a collection of input output and fault messages; there are 4 types of operation one-way(service just receives a message), request-response(RPC), solicit-response, notification (services pushes out a message)

portType represents a single channel that can support multiple operations. It is “abstract” as specified as a set of operations. It is equivalent to a “interface or abstract class” in Java

binding tells you transport and message format for a porttype (which can have multiple bindings to reflect say performance-portability trades)

port combines a binding and an endpoint network address (URL) and is like a “class instance”

service consists of multiple ports and is equivalent to a “program” in Java


Basic Portal WSDL Interaction

Client WebBrowser(HTML)

User Interface

Web Server

HTTP(s)

UI Server Components(including WSDL clients)

SOAPServer

SOAP over HTTP

Service Components

WSTK compilesWSDL into client stubs that can be used by UI components


WSDL-Based Portal Services Basically the same as SOAP server-server

interaction described previously. We can use Web Services Toolkit (WSTK) from

IBM to generate java client stubs from the WSDL interface. These stubs can be used to create “local” java objects in the UI server that use SOAP RPC to talk to server. So the WSTK creates CORBA-like remote objects.

We still use SOAP+HTTP under the hood to make the method calls.


WSDL Definitions and Namespaces<?xml version="1.0" encoding="UTF-8" ?> <definitions name="BatchScriptService" targetNamespace="http://yourserver/BatchScriptService"

xmlns="http://schemas.xmlsoap.org/wsdl/"xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"

xmlns:tns="http://yourserver/BatchScriptService" xmlns:xsd="http://www.w3.org/1999/XMLSchema“ >

Default Schema (null namespace) is WSDLsoap namespace is SOAP binding of WSDLtns namespace is this WSDL filexsd namespace defines (ancient) XML Schema


WSDL Message Example<message name="submitRequest"> <part name="xmljob" type="xsd:string"/></message><message name="submitResponse"> <part name="response" type="xsd:string"/></message>

For the batch script service, we pass the XML description of the job as a string and get back the script as a string.In general, any XML primitive or complex types can be used in messages.

We could improve our service by defining a BatchScript complex type.


WSDL portTypes Example <portType name="BatchScriptServicePortType"> <operation name="batchGen"> <output message="tns:submitResponse"

name="submitResponse"/> <input message="tns:submitRequest"

name="submitRequest"/> </operation> </portType>

A portType corresponds to a Java class, so if we compile this WSDL to make client stubs, we will generate a BatchScriptServiceBinding.java class.


WSDL SOAP Binding Example<binding name="BatchBinding" type="tns:BatchScriptServicePortType"> <soap:binding style="rpc" transport="http://schemas.xmlsoap.org/soap/http"/> <operation name="batchGen"> <soap:operation soapAction=""/> <input> <soap:body use="encoded“ namespace="urn:BatchScriptService" encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/> </input> <output> <soap:body use="encoded" namespace="urn:BatchScriptService“

encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/> </output> </operation></binding>

note binding’s “type” attribute points back

to the portType tag by name


WSDL Ports and Services<service name="BatchScriptService"> <documentation>BS stands for Batch Script </documentation> <port binding="BatchBinding”

name="BatchPort"> <soap:address location=

"http://yourserver/soap/servlet/rpcrouter/"/> </port> </service></definitions>

ports are concrete implementations of portTypes and point back to a particular binding (by name). They also point to the specific location of a server that implements the service.A service is a collection of one or more ports.


Portal UDDI InteractionClient Web

Browser(HTML)

User Interface

Web Server

HTTP(s)

SOAPServer

SOAP over HTTP

(1)

(2)

UDDI Server

SOAP over HTTP

(3)

UI Modules include bothWSDL and UDDI clients


SOAP+UDDI+WSDL

UI server now includes both WSDL and UDDI client components.

UI’s UDDI components are, for example JavaBeans using UDDI4J API to interact with the UDDI.

The UDDI client components can be used to build administrator HTML/JSP interfaces to add, modify UDDI entries on the server.

UDDI client components can also be used to build search interfaces for regular users.

Note the UDDI client implementation uses SOAP for RPC to talk to the server.


Steps 1-3 on Figure

1. User requests that the UI server performs a search for a particular service.

2. UI server now acts as a client to the UDDI server, gets back the search results, plus URLs for the SOAP servers and the WSDL for the desired service.

3. User selects the desired service provider from UI’s display. UI then becomes client to the SOAP server and executes the service.


Virtual XML for MPI I In days gone by, we used to study:

MPI_SEND(buf, count, datatype, dest, tag, comm) [ IN buf] initial address of send buffer (choice) [ IN count] number of elements in send buffer (nonnegative integer) [ IN datatype] datatype of each send buffer element (handle) [ IN dest] rank of destination (integer) [ IN tag] message tag (integer) [ IN comm] communicator (handle)

Now our SPMD code is a Web service running in some large scale parallel machine – perhaps integrated with an Apache server running on node• Need to support distributed and shared memory

We want to define all interfaces in WSDL – at least all of them which could be “external” ports


Virtual XML for MPI II datatype becomes an XML Schema specifying structure of

transmitted data – could either be a URI for Schema or its realization• There would be a built-in schema suitable for handling standard

Fortran/C datatypes

buf is a handle to an XML instance of Schema datatype count is still an integer (xs:integer where xs is Schema

namespace xmlns:xs=http://www.w3.org/2001/XMLSchema)• For some bindings count is implied by buf

dest becomes a URI with special notation such as legacympi://procnum/local/nnn for processor numbers

tag becomes an instance of (new) mpitag Schema comm becomes an instance of (new) mpicommunicator Schema


BackendResources

Middle Tier

Client

JavaC++ ..Proxy

HPCEngine

MPI ..

SOAP IIOPRMI …..

XML basedInterfaces

WSDL

Database

JavaC++ ..Proxy

PersistentStorage

ParallelI/O …

SimulationComponent

DataComponent

MPP

HPCC andWSDL


Virtual XML for MPI III So one defines mpi.wsdl with all 100 or so methods

defined as (abstract) operations• We will define several bindings including the classic HPCC

method calls for Fortran C C++ and Java with either distributed memory or shared memory mode

• For C++ and Java we would implements bindings for either “object” or “packed consecutive byte” interpretation of buf

Note one can use both static and run-time compilation to map XML specified services into particular implementations

MPI_INIT() might be implemented with JXTA (peer-to-peer advertisments) to find nodes; the communicator might be transmitted by SOAP while buf transmission uses MPICH

This brings Web Services to MPI but also MPI to web services


Compiling for WSDL

Single Address space

Method1Method2

stack

WebServer

WebApplication1

WebServer

WebApplication2

WSDL Messages

Shared Memory Distributed System


How to Integrate XML/Web Services into an Enterprise I

You are the CIO of a little known country’s (call it Quagmire) education system and been tasked to position it as a 2005 leader in technology enhanced education

Your initial review reveals a multitude of incompatible existing information systems distributed over Quagmire’s 50 campuses

Your cradle to grave community education mandate (which got the President – your uncle – elected) implies integration of all aspects of education in your information system

You note that performances issues are much less important in this application than interoperability (it currently takes months to get anything done so what's an extra millisecond or so at a WSDL interface) , so even in current primitive state Web services look attractive


Enterprise Web Services II We start a requirements analysis and a catalog of existing “raw

resources”. This will lead to the clean internal XML(WSDL) interfaces QIF and to the external XML interfaces (QEF) to the raw resources

Quagmire Internal Framework (QIF) We find that The Grid community has defined computing

interfaces which we will adopt to support infrastructure needed for use of simulations and data repositories in curriculum

We find that Microsoft, the Liberty Alliance and the Grid have Security schemes but are relieved that with a little bit of XSLT (transformations) we can map between them

We join the IMS consortium and adopt their standards for learning meta-data and user related information

We adopt the AICC and ADL SCORM standards for “learning objects” (course pages)


Enterprise Web Services III We define Quagmire educational agents which use QIF plus

RDF/DAML/OIL to be able to capture educational goals We expect commercial standards compliant solutions will be

available for most other needed Educational web services We find that we must interface with a dreadful collection of

external raw resources coming from legacy PeopleSoft SAP Oracle and Microsoft products• As our new model is object (XML) and open standards based we build a

new QII (Quagmire Information Infrastructure) supporting QIF objects We analyze structure of legacy systems and their relation to QIF

and define an external framework QEF which captures much of the raw resources• This Schema is extensible to capture quirks of each legacy system• We build a set of Web services that convert QEF into QIF

We inform vendors that we will only install QIF compliant systems in all future purchases


Some Science Web Services These build on general (community) web services


Some General Grid Web Services


Education as a Web Service Can link to Science as a Web Service and substitute educational

modules “Learning Object” XML standards already exist from IMS/ADL

http://www.adlnet.org – need to update architecture Web Services for virtual university include: Registration Performance (grading) Authoring of Curriculum Online laboratories for real and virtual instruments Homework submission Quizzes of various types (multiple choice, random parameters) Assessment data access and analysis Synchronous Delivery of Curricula Scheduling of courses and mentoring sessions Asynchronous access, data-mining and knowledge discovery Learning Plan agents to guide students and teachers


RDF in a Nutshell

The essence of RDF is a statement which has a subject and predicate and an object

The “subject” has the property “predicate” with value “object” The Web Page has the title “RDF Model and Syntax” Mary had a husband called John Statements can specify metadata but more generally any facts are

expressible in this way One can associate multiple objects with a subject and so build up

a general node of a tree with parent as subject and objects as children.

DAML and OIL extend RDF with more powerful relations and ontologies

Subject ObjectPredicate


RDF Tree Fragment

Subject

Object

Object

Object

Same or DifferentPredicates

Object

Object

Object

Class

Agatha Henry Fred

Same Predicate: Student

Fred@jim Freddie B-

DifferentPredicates

email Nickname

grade

General

Specific


An Example of RDF and Dublin Core <rdf:RDF xmlns:rdf=“http://www.w3.org/1999/02/22-rdf-syntax-

ns#” xmlns:dc=“http://purl.org/metadata/dublin_core#”> <rdf:Description about=“http://www.dlib.org”> <dc:Title>D-Lib Program - Research in Digital

Libraries</dc:Title><dc:Description>The D-Lib program supports the community of people with research interests in digital libraries and electronic publishing. </dc:Description><dc:Publisher>Corporation For National Research Initiatives</dc:Publisher><dc:Date>1995-01-07</dc:Date>

<dc:Subject>• <rdf:Bag> <rdf:li> Research; statistical methods</rdf:li>

<rdf:li> Education, research, related topics</rdf:li><rdf:li> Library use Studies </rdf:li> </rdf:Bag>

</dc:Subject> <dc:Type>World Wide Web Home Page</dc:Type> <dc:Format>text/html</dc:Format> <dc:Language>en</dc:Language> </rdf:Description> </rdf:RDF>


ResourceDatabase

ResourceSoftware

Software

XMLSkin

Peer to Peer Community Grid

XMLSkin

MessageOr EventBasedInterConnection XML Defined Resources

connected by XML defined messagesImplementation of resource andconnection may or may not be XML


P2P Grid Architecture Note proposed architecture – resources communicating via

messages – is already peer-to-peer as we consider everything democratically – clients, data sources, backend resources

Peer-to-peer systems (P2P) include Napster, Gnutella and Avaki discussed in another tutorial

JXTA (http://www.jxta.org) is attractive as has very clear architecture and primitives such as peers (users), peer groups , advertisements (XML messages), pipes (message queues), rendezvous (repository of advertisements and other places to find things)

ResourcesResources

Resources

Resources

Queued Events

JXTA Peers communicate via messages with XML topics used to match service providers with service users

Different mechanism than UDDI for registering and discovering resources


Classic Grid Architecture

Database Database

Netsolve

Neos

SecurityPortal

Composition

Portal

Resources

Clients Users and Devices

Middle TierBrokers Service Providers

Typically separate Clients Servers Resources


DatabaseDatabase

EventBroker

EventBroker

EventBroker

EventBroker

Resources

Resources

ResourcesResources

Peer-to-Peer Grid


Semantic Grid & Digital Brilliance Peer to Peer networks teach us that we can build “small worlds”

where distance between nodes is logarithmic in number of nodes Consider a Grid of WSDL services linked (through UDDI)

together

• This is spirit of Semantic web – RDF/DAML/OIL metadata enables meaningful linkage

We do not need to link everybody but only to establish “small world” routes

Physics analogies suggest that phase transitions will occur when “enough” nodes are linked – one will get nodes to align in the direction of new knowledge

This suggests ways of quantifying value of metadata induced linkages and areas where one “should” add more WSDL specifications


Publish/Subscribe as a Web Service We can implement messaging subsystem (between WSDL

resources) with either direct messages or by a queued system where you publish messages to queues and subscribe as receiver to particular queues• Natural asynchronous collaboration model which is in fact fast enough for

synchronous collaboration There are many different publish/subscribe models

• JMS is a cluster of central servers• JXTA is a very dynamic Peer to Peer model where pipes are queues and

topics (metadata) are service advertisements Implement JMS API with JXTA protocol – different WSDL

bindings here have different fault tolerance/reliability semantics• Could use JMS as long distance “carrier” between JXTA peers• JXTA provides higher performance than JMS for nearby recipients

Pallickara built an intermediate dynamic message broker subsystem which can link JMS and JXTA see:

http://grids.ucs.indiana.edu/ptliupages/projects/narada/


Single Server P2P Illusion

JMSBroker

Database


Broker Network

Database

Resource

Broker

Broker

Broker

Broker

Broker

Broker

Software multicastP2P: Brokers are clients


Community P2P Grid

JMSServer

Database

JXTARendezvous

GXOSServer Generate “Automatically”

GXOSServer


Narada JXTA/JMS Integration http://grids.ucs.indiana.edu/ptliupages/projects/narada/ Is a network of event brokers which can reliably deliver XML

specified events Using openJMS selection module, becomes a distributed or

conventional Java Message Service Linking special JXTA

proxies, it can link JXTA communities


Event Web Service

Web Service 1

(Virtual)Queue

Web Service 2

Destination Source Matching FilterRouting workflow

WSDLPorts

WSDLPorts


Some Research Issues for P2P Grid What happens to programming languages when data structures

are defined in XML How do we manage a sea of virtual XML?

• Register, find and link objects• This is “distributed operating system of the world” ?

What happens to databases when everything is an Object defined in XML and transformed by Java?

How and when do we compile virtual XML • Convert slow XML message to super fast method call on stack

How do we implement services such as Security and collaboration over a range of grain sizes

Supporting all “grain sizes” we get some sort of dynamic fractal world which looks like XML objects exchanging XML messages at all scales• Not well supported by centralized services (P2P problem)

Semantic Grid: as metadata increases, objects link together forming digital brilliance – a phase transition in information space


Collaborative Web Resources Collaboration is “just” sharing objects What about Collaborative Web Services ?

• You can in some cases do this automatically just by multicasting messages from service to clients

• This is achieved by service publishing messages and clients subscribing

Many applications do not expose all state changes• E.g. when I edit PowerPoint slide, PowerPoint does not tell the world by

sending an (XML) message

Solved by shared event collaboration model and requires one to view user interface as a “port” in WSDL sense and treat “event handlers” (mouseover, click etc.) as messages in WSDL

Groove Networks does use XML front end to COM interfaces• More elegantly can use W3C DOM for (the few) documents (SVG is one)

and “universal event handlers”

Interesting research area


Collaboration as a Web service I


Future Collaboration Web Service

integrating peer to peer networks and web services with the grid

Documents