The Diary of a Fence-sitting SOA Geek

Dr Mark LittleTechnical Development Manager, Red Hat

2 Red Hat

Background

• Research into fault-tolerant distributed systemssince 1986– Arjuna, Argus, Isis/Horus, Emerald, Xerox, …– DCE, DCOM, CORBA, JavaRMI, HTTP, Web Services, …

• Active in OMG, OASIS, W3C, JCP, GGF andothers

• Involved with Web Services since 1999– Co-author of a number of WS-* specifications and standards

• Involved with REST/HTTP since at least 2000– W3Objects

3 Red Hat

Overview• Evolution of distributed systems

– 40 years in 5 minutes• WS-* and REST/HTTP

– Apples and Oranges, or Apples and Broccoli?• Where should you start?

– Donʼt panic!• Lessons learned

– And re-learnt• Donʼt believe Snake-oil Salesmen

– On both sides of the divide!

4 Red Hat

The debate

• No REST bashing– We all know that REST/HTTP is just one approach to REST

• No WS-* bashing– And specifically no WSDL bashing please

• These types of debate have raged throughouthistory– BetaMax vs VHS– Blu-Ray vs HD-DVD– England vs France– MacDonalds vs BurgerKing

• Hybrid systems are the norm– Very few places can afford to rip-n-replace

5 Red Hat

Back to the future?

6 Red Hat

January 1st 1951

• Dear diary– John next door has a computer-thingy called UNIVAC

• Not sure what he expects to be able to do with it. It’s a bit big!• Apparently it’s good at math!

– International Business Machines think the world will onlyneed a handful of them

• And that punch cards are the way of the future

7 Red Hat

January 1st 1962

• Dear diary– I just got a computer. These things are a lot smaller and

faster than they used to be (I can put a desk in the sameroom!)

– Spacewars developed in 200 hours on PDP-1– Jim and I decided to try to connect his computer with mine

• We spent about 4 months getting the wires connected, checkedand working

8 Red Hat

January 1st 1969

• Dear diary– Today we created ARPANet– Managed to connect 4 computers together on a network

• UCLA• Stanford Research Institute• UC Santa Barbara• University of Utah

– Heterogeneity is a real pain!– I canʼt see this scaling!

• Update– Invented email– Can send simple messages (1971)– Invented telnet (1972)– Invented FTP (1973)

9 Red Hat

January 1st 1970 (The Epoch!)

• Dear diary– Invented Remote Procedure Call (RPC)– First used in Unix back in the 1970ʼs to aid distributed development

• Try to make distribution opaque• Leverage a well known pattern: local procedure calls

– Invented UDP and TCP

• RPC is NOT dead– Neither is it pining for the fjords– Integrated Systems Architecture (ISA), Open Network Computing

(ONC), the Open Software Foundation Distributed ComputingEnvironment (OSF/DCE), the Object Management Group CommonObject Request Broker Architecture (CORBA), DCOM, J(2)EE

10 Red Hat

January 1st 1990

• Dear diary– A physicist has created something called the World Wide

Web• What does he know compared to us Computer Scientists?!

Hypertext? Hypercard maybe!• It’ll never catch on!

11 Red Hat

January 1st 1999

• Dear diary– Invented Web Services

• Users want to glue together existing distributed systems• The firewall vendors won’t open up to CORBA or DCOM, so

we’ll show them!– HTTP is a useful transport after all

12 Red Hat

January 1st 2005

• Dear diary– Spent the last 5 years going backwards to distribution

transparency• It feels like 1970 all over again!

13 Red Hat

Distributed Systems

• Web Services, WWW, CORBA, DCOM, …– All examples of distributed systems

• Same fundamental laws– Develop “entity”

• Define the UOW it supports– Search for “entity”

• Agree “entity” offers the desired capability or UOW– Request “entity” to perform UOW

• Create a network-transferable message• Send the message

– Maybe try to make the remote interaction appear local– Maybe do some “enterprise” work as well

• Security, transactions, replication etc.

14 Red Hat

Different approaches

• Event Driven Architectures– Pub/Sub

• Service Oriented Architectures• Resource Oriented Architectures• Message Oriented Architectures

– MOM– Pub/Sub

• Tuple Spaces– E.g., Linda

• RPC based• Synchronous or asynchronous• Group communication based

– Reliable or unreliable

• There is no such thing as a global panacea!

15 Red Hat

Tightly coupled

• Client and server technologies based on RPC– Hide distribution

– Make remote service invocation look the same as localcomponent invocation

• Unfortunately this leads to tightly coupledapplications– Changes to the IDL require re-generation of stubs

• And dissemination of new code

• Or errors will occur during interactions

– Such applications can be brittle• Hard to control the infrastructure as needed

• No quiescent period

16 Red Hat

Stub example

Node X Node Y

Client Process

CSO

Server Process

SSO O

17 Red Hat

SOA in a nutshell

• SOA is an architectural style to achieve loosecoupling– A service is a unit of work done by a service provider to

achieve desired end results for a consumer

• SOA is deliberately not prescriptive about whathappens behind service endpoints– We are only concerned with the transfer of structured

data between parties

• SOA turns business functions into services that canbe reused and accessed through standard interfaces.– Should be accessible through different applications

over a variety of channels

18 Red Hat

Achieving loose coupling

• SOA employs two architectural constraints– A small set of simple and ubiquitous interfaces to all

participating software agents. Only generic semantics areencoded at the interfaces. The interfaces should beuniversally available for all providers and consumers

• Yes, it’s often based on specific interfaces for each service• But does not need to be

– Descriptive messages constrained by an extensible schemadelivered through the interfaces. No, or only minimal, systembehavior is prescribed by messages. A schema limits thevocabulary and structure of messages. An extensibleschema allows new versions of services to be introducedwithout breaking existing services

• Sounds vaguely REST-like

19 Red Hat

But …

• At the one extreme– Defining specific service interfaces, akin to IDL

• Easier to reason about the service• Limits the amount of freedom in changing the implementation

• At the other extreme– Single operation (e.g., doWork)

• More flexibility is changing the implementation– Well, almost …

• More difficult to determine service functionality a priori– Need more service metadata

• There are degrees of coupling and you shouldchoose the level that is right for you– Not specific to distributed systems

20 Red Hat

The message and stack

21 Red Hat

Service stack example

Message listener

Network

Routing validator(e.g., objectpresent?)

Manage non-functional data

Work dispatcher

22 Red Hat

What are the pros and cons?

• The same requirements are present throughoutthe stack– Split differently between the infrastructure and the

“application”• Uniform interface allows for generic

infrastructural support– Caching, extremely loose coupling– Can push more requirements on to the “developer”– Requires more from external contract meta-data

• Specific interface allows for more limited genericsupport– Targeted caching, application semantics– Impacts less on the “developer” but may cost in terms of

flexibility

23 Red Hat

REST/HTTP in a nutshell

• Original HTTP specification talked about adding newcommands– GET only in 0.9– GET, HEAD, POST, extension-method in 1.0– Now we have up to 8 different verbs

• Changes to “interface” occur but users aren’t affected• Many good distributed systems approaches

– Dynamic content negotiation– Caching– Scalability

• It is NOT a transport

24 Red Hat

Not machine driven?

• Often stated that REST/HTTP is only suitable forhypertext– “Because thatʼs the way the majority of the Web works”

• Weak argument– Itʼs still a distributed system based on resources– OK, humans fill in the “gaps” in contract definition– Extra infrastructure support could help

• Anyone remember URN name servers (1994/1995)?

• Not enough “application” standards– Good point

• Lack of good tooling– JAX-RS, WCF, …

25 Red Hat

What about Web Services?

• Popular integration approach– XML– HTTP

• Other transport bindings are possible• Developed with machine-to-machine interactions

in mind• Not specific to SOA

– Web Services began life as CORBA-over-HTTP– XML-RPC– WS-RF and WS-Addressing

• Web Services+SOA gives benefits– Loose coupling– Interoperability– Enterprise capabilities, e.g., security and transactions

26 Red Hat

Enterprise realities

• Customers want interoperability ofheterogeneous systems

• They want guaranteed delivery of messages– Even in the presence of failures such as network partitions

• They want transactions– NOT ACID transactions!

• They need audit trails– Sarbanes-Oxley anyone?

• They need bullet-proof security– Sarbanes-Oxley

• They need machine-readable contracts with SLAs

27 Red Hat

WS-* Architecture

28 Red Hat

Standards

• The Web is a series of standards– URIs– HTTP– HTML

• Universal adoption has to count for something!• REST/HTTP is ubiquitous

– Communication interoperability, which is a good start• Also why WS-* standardised on HTTP

– But application semantic interoperability is not there (yet)• Take a minimum of 5 years to do

• But …

29 Red Hat

Transaction example

• REST transactions by HP in 2000– Yes, customers want to coordinate business interactions,

even if only atomically• http://<machine>/transaction-coordinator

Performing a GET returns a list of all transactionsknow to the coordinator (active and recovery)

• http://<machine>/transaction-coordinator/begin?<ClientID>Performing a PUT will start a new transaction andreturn a URL /transaction-coordinator/<id>

30 Red Hat

However …

• WS-* has driven protocol interoperability– More so than CORBA, DCE, Java

• Native data and protocol bridging• Clearly defined semantics for transactions,

security, reliable messaging– Not specific to HTTP

31 Red Hat

Zen and the art of [R..S]OA

• Are all entities in a distributed system services?• Are they all resources?• Why canʼt we mix-and-match?

– In fact, isnʼt a service a resource?– Isnʼt an object a resource?

• Before ROA and SOA there were just distributedsystems– Distributed-Oriented Computing? DOC? (EDOC)– Internet-Oriented Computing? IOC

• What about the Intranet?

• And a lot of work on scalability, fault toleranceetc.

32 Red Hat

REST or (SOA?)WS-*?

• SOAP/HTTP– WS-* vendors have spent a lot of time ensuring it integrates with

legacy systems• How many people remember that WS-* is supposed to also be about

Internet scale computing?– Thatʼs important for out-of-the-box and interoperable deployments– But REST as a general architectural approach has merits even in

SOA

• REST/HTTP has “simplicity” and (relative) ease of use– No vendor-lockin at the infrastructure level– Is precisely for Internet scale computing as well– Remember that a lot of cool Web applications arenʼt RESTful

33 Red Hat

What can REST learn from WS-*?

• Uniform interface isnʼt enough for complexapplication requirements– Standardise on the application protocol semantics– Ad hoc does not scale and leads to interoperability

nightmares• Popularise using HTTP outside of the browser

– Yes, there are examples and implementations, but they arethe exception to the rule

• Just because the Web “works” is not sufficientreason to assume itʼs right for everything– When was the last time you rode a horse to work?

34 Red Hat

What can WS-* learn from REST?

• Donʼt abuse transports, they donʼt like it!• Adopt SOA principles

– No more WS-RF please!– Adopt WS-Context in favour of WS-Addressing “extensions”

• Late binding is good– But extremely late binding can be a burden

• KISS and make up– Occams Razor

• Infrastructure support for commonservices/resources simplifies development

• The Web uses HTTP

35 Red Hat

The combination

• WS-* used within the firewall– REST principles could still help– SOAP is not fast!

• REST/HTTP for between firewalls– HTTP is not for performance– Improved application protocol standards

• Get the bridging between WS-* and REST/HTTPright– Leverage all of HTTP where possible– Definitely not easy to do, but …

• WS-* between firewalls?– Unlikely to see massive adoption

36 Red Hat

13th March 2008

• What have we learnt from the last 40 years?– One size does not fit all!– Use the right tool for the right job

• If all you’ve got is a hammer then of course everything lookslike a nail!

37 Red Hat

Conclusions

• WS-* and REST/HTTP arenʼt perfect– But what is (DCOM, DCE, CORBA, JavaRMI, …)?

• R&D on large scale distributed systems began inthe late 80ʼs and early 90ʼs– Started before the Web

• E.g., EU Broadcast and Cabernet projects– Addressed many of the same areas

• But had no good use cases!• Was seen as too esoteric!

• The Web is here to stay!– It is a deployment platform for “enterprise” applications– It is an “enterprise” service bus