the enterprise web services bus routable, reliable and publish/subscribe web services

The Enterprise Web Services Bus

Routable, Reliable and Publish/Subscribe Web Services

INFOSYS 290, Section 3Web Services: Concepts, Design and

Implementation

Adam Blumablum@good.com

Agenda

• Bus-style integration versus point to point integration

• WS-Addressing

• WS-ReliableMessaging

• WS-Eventing

The Problem with Point to Point

• Web services enable ad hoc integration without negotiating transports – And potentially not even payload formats

• But connectivity is still built point to point– Origin and destination of messages are determined

by the underlying protocol

• Many business events are of interest to many parties– It would be more efficient to send messages once to a

“bus” and have it transmitted to all participants which are interested

What’s Needed for Bus-Style Integration over Web Services

• Remove destinations and routing from the transport

• Allow applications to subscribe to business events of interest

• Insure reliable delivery of messages in standardized way

Routable Web Services

• The Problem– Web service destinations are today expressed

in the physical transport– E.g. the http destination address

• The Solution– Use a SOAP header to reflect the address

Sample WS-Addressing Message

Endpoint References Consist Of • Address

– URI for the endpoint• Reference Properties

– Optional properties which fully qualify endpoint reference– Opaque to consuming app

• Reference Parameters– Optional parameters supplied to endpoint for particular interaction – Opaque to consuming app

• PortType– Optional reference to the interface supported

• ServiceName– Optional WSDL service reference

• ServiceName/@PortName– Optional WSDL port

• Policy– Optional WS-Policy

Sample Endpoint Reference

<wsa:EndpointReference xmlns:wsa="..." xmlns:fabrikam="...">

<wsa:Address>

http://www.fabrikam123.example/acct

</wsa:Address>

<wsa:PortType>

fabrikam:InventoryPortType

</wsa:PortType>

</wsa:EndpointReference>

Binding to SOAP

• Copy address property to destination header field of SOAP message

• Each ReferenceProperty and ReferenceParameter becomes SOAP header

SOAP Binding Sample - EndpointReference

<wsa:EndpointReference xmlns:wsa="..." xmlns:fabrikam="..."> <wsa:Address>http://www.fabrikam123.example/acct</wsa:Address> <wsa:ReferenceProperties>

<fabrikam:CustomerKey>123456789

</fabrikam:CustomerKey></wsa:ReferenceProperties> <wsa:ReferenceParameters> <fabrikam:ShoppingCart>ABCDEFG</fabrikam:ShoppingCart></wsa:ReferenceParameters>

</wsa:EndpointReference>

SOAP Binding SampleSOAP Header

<S:Envelope xmlns:S="http://www.w3.org/2003/05/soap-envelope" xmlns:wsa="..." xmlns:fabrikam="... "> <S:Header> ... <wsa:To>

http://www.fabrikam123.example/acct</wsa:To> <fabrikam:CustomerKey>

123456789</fabrikam:CustomerKey> <fabrikam:ShoppingCart>

ABCDEFG</fabrikam:ShoppingCart> ... </S:Header> <S:Body> ... </S:Body>

</S:Envelope>

Message Information Header • To - destination

– The address (URI) of intended receiver – Mandatory• Action

– URI identifying uniquely semantics of message - Mandatory• From

– Endpoint reference of the origin• ReplyTo

– Endpoint reference of intended receiver of replies• FaultTo

– Endpoint reference of where to send faults• MessageID

– URI which uniquely identifies message in time and space• RelatesTo

– Pair of values which indicates how one message is related to another message

Sample Message Information Headers

<S:Envelope xmlns:S="http://www.w3.org/2003/05/soap-envelope" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:f123="http://www.fabrikam123.example/svc53">

<S:Header>

<wsa:MessageID>uuid:aaaabbbb-cccc-dddd-eeee-ffffffffffff </wsa:MessageID> <wsa:ReplyTo> <wsa:Address>http://business456.example/client1

</wsa:Address> </wsa:ReplyTo> <wsa:To S:mustUnderstand="1">mailto:joe@fabrikam123.example</wsa:To> <wsa:Action>http://fabrikam123.example/mail/Delete</wsa:Action>

</S:Header> <S:Body>

<f123:Delete> <maxCount>42</maxCount> </f123:Delete> </S:Body> </S:Envelope>

A Reply to that Message<S:Envelope xmlns:S="http://www.w3.org/2003/05/soap-envelope"

xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:f123="http://www.fabrikam123.example/svc53"> <S:Header>

<wsa:MessageID> uuid:aaaabbbb-cccc-dddd-eeee-wwwwwwwwwww </wsa:MessageID>

<wsa:RelatesTo>uuid:aaaabbbb-cccc-dddd-eeee-ffffffffffff

</wsa:RelatesTo> <wsa:To S:mustUnderstand="1">

http://business456.example/client1 </wsa:To>

<wsa:Action>http://fabrikam123.example/mail/DeleteAck</wsa:Action>

</S:Header> <S:Body><f123:DeleteAck/></S:Body>

</S:Envelope>

Sample WSE CodeSoapEnvelope env = new SoapEnvelope();env.Context.Addressing.Action

=String.Format("urn:chat:message"); EndpointReference epr = new

EndpointReference("soap.tcp://theirowntoys.com/adam")); env.Context.Addressing.ReplyTo = new ReplyTo(epr);env.CreateBody(); // must create body before setting it env.Body.InnerXml = String.Format(

"<x:messagexmlns:x='urn:chat'><user>{0}</user><msg>{1}</msg> </x:message>", user, msg);

... EndpointReference epr = new EndpointReference( new

Uri("soap.tcp://theirowntoys.com/lauren")); SoapSender ss = new SoapSender(epr); // this fills in the

Toss.Send(env);

….Creates Following Message<soap:Envelope

xmlns:wsa="http://schemas.xmlsoap.org/ws/2003/03/addressing" xmlns:wsse="http://schemas.xmlsoap.org/ws/2003/06/secext" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:Header>

<wsa:Action>urn:chat:message</wsa:Action> <wsa:ReplyTo>

<wsa:Address>soap.tcp://theirowntoys.com/adam </wsa:Address>

</wsa:ReplyTo> <wsa:MessageID>uuid:59bc1ebb-40aa-4508-9a69-

5b148d04d697 </wsa:MessageID> <wsa:To>soap.tcp://theirowntoys.com/lauren</wsa:To> ...

</soap:Header> <soap:Body>

<x:message xmlns:x="urn:chat"> <user>adam</user> <msg>hi</msg>

</x:message> </soap:Body>

</soap:Envelope>

Routing with WS-Referral• <?xml version="1.0" ?>• <r:referrals

xmlns:r="http://schemas.xmlsoap.org/ws/2001/10/referral">• <r:ref>• <r:for>•

<r:exact>http://localhost/RouterService/StockService.asmx</r:exact>

• </r:for>• <r:if />• <r:go>•

<r:via>http://localhost/StockService/StockService.asmx</r:via>• </r:go>•

<r:refId>uuid:fa469956-0057-4e77-962a-81c5e292f2ae</r:refId>• </r:ref>• </r:referrals>

Implementations of WS-Addressing

• Microsoft WSE 2.0

• Apache Axis!– http://ws.apache.org/ws-fx/addressing/

(Davanum Srinivas at CA)– Part of WS-FX

• http://ws.apache.org/ws-fx/• See WSS4J for WS-Security!

• Systinet

WS-Addressing References

• WS-Addressing Specification– http://www.w3.org/Submission/2004/SUBM-

ws-addressing-20040810/

• MSDN Article on Moving from WS-Routing to WS-Addressing– http://msdn.microsoft.com/library/default.asp?

url=/library/en-us/dnwebsrv/html/wsroutetowsadd.asp

WS-ReliableMessaging

Why a Reliable Messaging Standard?

• Base Web services standards (particularly SOAP over http) do not address guaranteed delivery of messages Applications build reliable messages into their business logic– Each in their own, possibly inconsistent way

• Standards-based interoperability requires a reliable messaging method of communication

• Routing and publish/subscribe exacerbates the problem – Since there is no direction connection to the other system– If there are multiple recipients (e.g. bus) we cannot program to

each of their unique business logic programmed for reliable messaging

Requirements for Reliable Messaging

• AtLeastOnce delivery

• AtMostOnce delivery

• ExactlyOnce delivery

• InOrder message sequencing

Reliable Messaging Model

Sequence

Sequence Responses

RM Client Code (IBM ETTK)

import com.ibm.ws.rm.client.Context;

Context wsrContext = new Context();wsrContext.setMessageNumber(1);wsrContext.setLastMessage(true);wsrContext.setSendAckRequested(true);wsrContext.setSendCreate(true);wsrContext.setSendTerminate(true);call.setProperty(Context.RM_CONTEXT_PROPERTY_NAME, wsrContext);

System.out.println("Sending the PO...");

call.setOperationName( new QName("http://po.demo.wsrm.ibm.com", "submitPO"));

call.invoke( new Object[] { "12345", "12345445", new Integer(1), new Float(12.10)});

System.out.println("Message was delivered.");

(Source code in c:/ettk/services/demos/wsrm/client )

Three MessagesContext wsrContext = new Context(); wsrContext.setSequenceIdentifier(groupId);

wsrContext.setMessageNumber(1); Call call = (Call)service.createCall(portQName); call.setTargetEndpointAddress(url); call.setProperty(Context.RM_CONTEXT_PROPERTY_NAME, wsrContext); call.setOperationName( new QName("http://po.demo.wsrm.ibm.com",

"submitPO")); EndpointReference epr = UtilsWSA.createEndpointReference(url); // sets ToWSAddressingOutboundContext context =

WSAddressingContextFactory.createWSAddressingOutboundContext(epr);

context.setFrom( UtilsWSA.createEndpointReference( "http://" + ETTKConstants.SERVER_HOSTNAME + ":" + listenerPort + "/wstk/services/rm"));

UtilsWSA.associateContextWithCall(context, call); System.out.println("Delivering message number 1."); call.invokeOneWay( new Object[] { "11111A", "11111111", new

Integer(1), new Float(11.11)})

….More MessagesgroupId = wsrContext.getSequenceIdentifier(); wsrContext = new Context(); wsrContext.setSequenceIdentifier(groupId); wsrContext.setMessageNumber(3); wsrContext.setLastMessage(true); Call call3 = (Call)service.createCall(portQName); call3.setTargetEndpointAddress(url); call3.setProperty(Context.RM_CONTEXT_PROPERTY_NAME, wsrContext); call3.setOperationName( new QName("http://po.demo.wsrm.ibm.com", "submitPO")); UtilsWSA.associateContextWithCall(context, call3);

Runnable r3 = new Runnable() // async { public void run() {

System.out.println("Delivering message number 3."); call3.invokeOneWay( new Object[] { "33333C", "33333333", new Integer(1),

new Float(33.33)}); }

}; wsrContext = new Context(); wsrContext.setSequenceIdentifier(groupId); wsrContext.setMessageNumber(2); call = (Call)service.createCall(portQName); call.setTargetEndpointAddress(url); call.setProperty(Context.RM_CONTEXT_PROPERTY_NAME, wsrContext); call.setOperationName( new QName("http://po.demo.wsrm.ibm.com", "submitPO")); UtilsWSA.associateContextWithCall(context, call);System.out.println("Delivering message number 2.");call.invokeOneWay( new Object[] { "22222B", "22222222", new Integer(1), new

Float(22.22)});

OutputRunning the client code... Starting the response listener... Delivering message number 1. Delivering message number 3. Delivering message number 2.

Server-side output (server.out): Received PO with OrderID : 11111A Received PO with Part ID : 11111111 Received PO with Quantity: 1 Received PO with Amount : 11.11 Received PO with OrderID : 22222B Received PO with Part ID : 22222222 Received PO with Quantity: 1 Received PO with Amount : 22.22 Received PO with OrderID : 33333C Received PO with Part ID : 33333333 Received PO with Quantity: 1 Received PO with Amount : 33.33

Systinet Hello World

• import org.systinet.wasp.webservice.ServiceClient;• import org.systinet.wasp.sequence.Sequence;• public static void main(String[] args) throws Exception {

• String serverURL = System.getProperty("systinet.demo.server.url", "http://localhost:6060");

• // lookup of HelloService• ServiceClient client = ServiceClient.create(serviceWSDLURL);• client.setServiceURL(serverURL + servicePath);

• // create new sequence for one request• Sequence seq = Sequence.createOutputSequence(client);• seq.setLength(1);

• // call HelloService and print out a response message• Call call = client.createCall("hello");• System.out.println("Sending 'world' and waiting for the ACK and response...");• System.out.println(call.invoke(new Object[]{"world"}));• }

Systinet Multiple Messages

• private static final String servicePath = "/demo/reliability/OneWayService";• private static final String serviceWSDLURL = "resource:/demo/reliability/OneWayService.wsdl";

• private static final String sequencePrefix = "http://systinet.com/demo/reliability/oneway/";• private static final long sequenceLength = 3;

• String serverURL = System.getProperty("systinet.demo.server.url", "http://localhost:6060");

• // create a service client• ServiceClient client = ServiceClient.create(serviceWSDLURL);• client.setServiceURL(serverURL + servicePath);

• // register the sequence listener to the ServiceClient• SequenceListener listener = new OutSequencesListener();• Sequence.addSequenceListener(listener, client);

•

Systinet Multiple Messages// load all my previous sequences// this causes that all messages which are not acknowledged are delivering to the destination againSequence[] seqs = Sequence.loadOutputSequences(sequencePrefix, client);for(int i = 0; seqs != null && i < seqs.length; i++) {

System.out.println("Delivering loaded sequence " + seqs[i].getID());

• // if the sequence is incomplete we will continue with next invocations; the length is 'sequenceLength' but

• // the current length is equaled to number of already queued messages in the sequence• // before making invocations the loaded sequence must be activated for used processing

(client)• if(seqs[i].getCurrentLength() != seqs[i].getLength()) {

• System.out.println("Continuing in sequence " + seqs[i].getID());• seqs[i].setActive(true);• sendMessages(client, seqs[i].getCurrentLength() + 1);• }• }• // create a new sequence with length of 'sequenceLength'• // new created sequence is automatically activated for used processing (client)• Sequence seq = Sequence.createOutputSequence(sequencePrefix, true, client);• seq.setLength(sequenceLength);• System.out.println("Created new sequence " + seq.getID() + " (persistence=" + seq.isPersistent()

+ ")");• // make new one-way invocations in the sequence• sendMessages(client, 1);

Systinet Server Side• public void send(String message) {• // is the incoming message in a sequence?• Sequence seq =

Sequence.getActiveInputSequence();• if(seq != null) {• System.out.println("Reliable message: " + message + " (seq =

" + seq.getID() +• ", number = " + seq.getCurrentMessageNumber() + ")");• }

else {• System.out.println("Non-reliable message: " + message);• }• }

Sandesha Mods to Axis

Implementations

• IBM Emerging Technologies Toolkit– http://www.alphaworks.ibm.com/tech/ettk

• Systinet– http://www.systinet.com/products/wasp_jserve

r/overview

• Apache Sandesha– http://ws.apache.org/ws-fx/sandesha/

WS-ReliableMessaging References

• Spec– http://msdn.microsoft.com/library/default.asp?url=/

library/en-us/dnglobspec/html/ws-reliablemessaging.asp

• Article– http://www-106.ibm.com/developerworks/

webservices/library/ws-rmimp/

• IBM Page– http://www-106.ibm.com/developerworks/

webservices/library/ws-rm/

WS-Eventing

WS-Eventing Specification

• Sponsored by Microsoft, Tibco, BEA, Sun

• Four main request messages– Subscribe– Renew– GetStatus– Unsubscribe

Point to Point Integration

Content-Based Routing Topology

Subscribe Grammar

Subscribe Message<s12:Envelope

xmlns:12'http://www.w3.org/2003/05/soap-envelope'xmlns:wsa='http://schemas.xmlsoap.org/ws/2003/03/addressing'xmlns:wse='http:// schemas.xmlsoap.org/ws/2004/01/eventing'xmlns:eri='http://electronicsretailer.com/inventory'><s12:Header> 

<wsa:Action>http://schemas.xmlsoap.org/ws/2004/01/eventing/Subscribe 

</wsa:Action><wsa:To>http://electronicsretailer.com/CBRService</wsa:To><wsa:ReplyTo>

<wsa:Address>http://electronicsretailer.com/inventory</wsa:Address>

</wsa:ReplyTo><wsa:MessageID>

uuid:d7c5276b-de29-4313-b4d4-b3425b200840 </wsa:MessageID> 

</s12:Header><s12:Body>

<wse:Subscribe><wse:Delivery>

<wse:NotifyTo><wse:Address>

http://electronicsretailer.com/inventory/HandleNewProduct.asp </wse:Address><wsa:ReferenceProperties><eri:MySubscriptionID>1234</eri:MySubscriptionID></wsa:ReferenceProperties> 

</wse:NotifyTo></wse:Delivery><wse:Expires>2004-06-26T21:07:00.000-08:00</wse:Expires> <wse:Filter xmlns:er='http://electronicsretailer.com/schemas'>

/s12:Envelope/s12:Body/er:NewProduct</wse:Filter></wse:Subscribe>

</s12:Body></s12:Envelope>

Subscribe Response Grammar

SubscribeResponse<s12:Envelope

xmlns:12'http://www.w3.org/2003/05/soap-envelope'xmlns:wsa='http://schemas.xmlsoap.org/ws/2003/03/addressing'xmlns:wse='http:// schemas.xmlsoap.org/ws/2004/01/eventing'xmlns:eri='http://electronicsretailer.com/inventory'> <s12:Header> 

<wsa:Action>http://schemas.xmlsoap.org/ws/2004/01/eventing/SubscribeResponse </wsa:Action><wsa:To>http://electronicsretailer.com/inventory</wsa:To><wsa:RelatesTo>uuid:d7c5276b-de29-4313-b4d4-b3425b200840 </wsa:RelatesTo> 

</s12:Header><s12:Body>

<wse:SubscribeResponse><wse:SubscriptionManager>

<wsa:Address>http://electronicretailer.com/cbrservice</wsa:Address>

<wse:ReferenceParameters><wse:Identifier>uuid:5005cfe6-c2c6-4296-9c3a-80b9ad111813</wse:Identifier></wse:ReferenceParameters>

</wse:SubscriptionManager><wse:Expires>2004-08-01T00:00:00-000-00:00</wse:Expires>

</wse:SubscribeResponse></s12:Body>

</s12:Envelope>

A “Notification” (any matching message)…

<s12:Envelopexmlns:12'http://www.w3.org/2003/05/soap-envelope'xmlns:wsa='http://schemas.xmlsoap.org/ws/2003/03/addressing'xmlns:wse='http://schemas.xmlsoap.org/ws/2004/01/eventing'xmlns:eri='http://electronicsretailer.com/inventory'xmlns:er='http://electronicsretailer.com/schemas'><s12:Header> 

<wsa:Action>http://electronicsretailer.com/schemas/NewProduct</wsa:Action><wsa:To>http://electronicsretailer.com/inventory</wsa:To><eri:MySubscriptionID>1234</eri:MySubscriptionID>

</s12:Header><s12:Body>

<eri:NewProduct><eri:ProductID>AC-MP471</eri:ProductID><eri:ProductName>Acme 128MB Portable MP3 Player</eri:ProductName><eri:ProductCategory>MP3 Players</eri:ProductCategory><eri:Price>47.56</eri:Price></eri:NewProduct>

</s12:Body></s12:Envelope>

Sample Subscription Client Code• bool result=true;•• CBRSubscribe.CBRService.Subscribe subscription=new CBRSubscribe.CBRService.Subscribe();• subscription.NotifyTo=new CBRSubscribe.CBRService.EndpointReferenceType();• subscription.NotifyTo.Address=new CBRSubscribe.CBRService.AttributedURI();• subscription.NotifyTo.Address.Value=notifyAddress;• if (subscriptionFilter.Length>0){• subscription.Filter=new CBRSubscribe.CBRService.MessagePredicateAssertion();• subscription.Filter.Value=subscriptionFilter;• }• subscription.Expires=expireDate;

• CBRSubscribe.CBRService.Eventing objEventing=new CBRSubscribe.CBRService.Eventing(cbrServiceURL);• CBRSubscribe.CBRService.SubscribeResponse subscribeResponse=new

CBRSubscribe.CBRService.SubscribeResponse();try {

subscribeResponse=objEventing.SubscribeOp(subscription);• }• catch {

– result=false;• }

• if (!subscribeResponse.Id.StartsWith("uuid:"))• result=false;

Changes over First Draft

• No requirements for WS-Addressing delivery

• GetStatus message

• SubscriptionManager abstraction

WS-Eventing Implementations

• My WS-Eventing general purpose client– http://adamblum.com/cbrsubscribe.zip– Needs to be updated to August 2004 spec!

• Systinet’s Content-Based Router– http://www.systinet.com/contentbasedrouting

Competitors to WS-Eventing

• WS-Notification– IBM et al– Used in WSDM

• JMS– Sonic– Fiorano

• Message queueing/bus providers– Tibco (behind WS-Eventing)– MQSeries

WS-Notification Differences

• Topic abstraction– Necessary?

• Notification brokers– Necessary?

• Subscription Manager?– Yes!

• http://blum.typepad.com/coarsegrained/2004/02/wsnotification_.html

WS-Eventing References

• The Spec– http://msdn.microsoft.com/webservices/understanding/specs/def

ault.aspx?pull=/library/en-us/dnglobspec/html/ws-eventing.asp– WSDL:

http://schemas.xmlsoap.org/ws/2004/08/eventing/eventing.wsdl

• My Web Services Journal article– http://www.sys-con.com/story/?storyid=44362&DE=1

• Interesting Survey of CBR methods– Covers WSE via WS-Referral (shown in this deck), Indigo via

“rules manager” and Systinet WASP– http://www.c-sharpcorner.com/Code/2004/June/

ContentBasedWS.asp

Putting It All Together:A Web Services Bus

• WS-Addressing for removing reliance on physical direct connections

• WS-ReliableMessaging to insure multiple disconnected recipients all receive message

• WS-Eventing to ease communications with multiple recipients

• http://blum.typepad.com/coarsegrained/2003/11/index.html