building a soa with infrastructure, application, and orchestration … · 2009-07-30 ·...

International SOA Conference 2009

Building A SOA With Infrastructure, Application, And Orchestration Services From The Ground Up –Design Decisions Illustrated In A B2B Case Study

Max Dolgicer

Managing Director

International Systems Group, Inc.

[email protected]

http://www.isg-inc.com

2


Table of Contents

� Case study: Chauffeured Services Company

� Project overview

� Three dimensions of business partner integration

� Service Oriented Integration Architecture

� Definition Of Service Layering

� Logical architecture and business event walkthrough

� Verifying SOA principles

� Designing service interfaces

� Designing the Schemas

� Interface consolidation

3


Table of Contents (cont.)

� WSDL and XML Schema

� Do we need SOAP?

� “REST-like” Services

� ROI through service reuse

� Conclusions

4


Case Study: Chauffeured Services Company

� The Company provides premium limousine services

� The clientele of The Company comprises � Individual customers including many celebrities� Corporate clients � Travel agencies

� The Company provides services to its customers via� Subsidiaries� Affiliates� Licensees and farm outs

� In addition, there are external sources of business information� E.g. Global Distribution Systems (GDSs)

5


Development, Internal + External Integration

� Overall project focus on development of new business “applications” including re-implementation of outdated legacy systems

� Strong emphasis on services

� A limited number of internal integration points

are required:

� Fax, email, Oracle, 2 legacy systems

� B2B Integration is a major requirement

� Subject matter of this case study

6


B2B Integration Strategy

Booking Agents

Dispatch

Chauffeurs

Back Office / Customer Service

BusinessAviationSystems

Travel Web Sites

Customers

GDS Systems

Electronic Distribution

Channels

Customer Intranet

Sites

Supply Chain

Partners

Service Network

B2B Gateway

Core Application Services

Reservations

Reservations And Profiles Services

Passengers

Accounts

Price Quotes

Service Providers

Locations

DispatchingServices

BillingServices

7


B2B Gateway

� The B2B Gateway enables The Company to � Receive reservations from a multitude of partners

electronically

� Accept them into the back office reservations application automatically

� Previously, reservations were received via email and the call center and had to be re-keyed into the reservation system

� The system is required to support the following functionality:� Create Reservation

� Modify Reservation

� Retrieve Reservation

� Cancel Reservation

� Quote Request

8


Business Process Walk-Through

B2B Gateway

Reservations & Profiles Services

DispatchingServices

BillingServices

76

5 4

3

21

8

9

9


Business Process Walk-Through

� A complete business process that covers all phases of a car service booking consists of the following steps:

1. A customer or travel agent request the creation of a new reservation

2. The Reservation system makes a new reservation

3. The Dispatching system picks up the new reservation

4. The Dispatching system sends a job allocation to a driver through a 3rd-party wireless network

5. The driver receives the job allocation

6. The driver reports end-of-job billing information via the 3rd-party wireless network

7. The Billing system receives the end-of-job information and calculates the bill

8. The Billing system sends and invoice to the customer

9. The customer receives the invoice

10


Project Scope

1. Encapsulation of the legacy reservation system� Shield business partners from legacy data formats

2. Connectivity to GT3� GT3 provides access to Global Distribution Systems

(GDS) for receiving automated reservation requests� GDS examples: Sabre, Apollo

3. Definition and implementation of a B2B protocol standard for the chauffeured services industry

4. Definition of a common XML-based data architecture

� The diversity of systems that are connected to the B2B Gateway (i.e. partner systems, legacy systems, newly developed applications) necessitates the definition of a comprehensive data architecture

11


Project Scope (cont.)

5. Integration with a 3rd party wireless system

� The B2B Gateway needs to mediate between a 3rd party wireless system such that the Chauffeurs smart phones can access The Company’s back office applications

6. Implementation of a SOA foundation

� The items listed above constituted the primary goals of the project

� However, The Company wanted to capitalize on the opportunity and build a SOA foundation

� Common infrastructure services and common application services, i.e. services that can be reused across several business partner integrations.

12


Three Dimensions of Business Partner Integration

� From a technical perspective, integrating with a new business partner requires addressing three

issues:

1) Data

� Each business partner uses different data formats

� Data transformation/translation from each partner’s format to The Company’s internal standard format is required

� The standard data format is used for the information that flows through the integration infrastructure

� It is a standardized superset of information that the integration endpoints (i.e. applications or services) employ

� Reduces the number of point to point transformations that need to be built and maintained

� This concept is sometimes referred to as logical decoupling

13


Three Dimensions of Business Partner Integration

� From a technical perspective, integrating with a new business partner requires addressing three

issues:

2) Connectivity

� The mechanism to connect to the business partner over the Internet

� E.g. Web Services, Sockets, MQSeries

3) Business process

� The business rules that a business partner follows in order to conduct a specific transaction (e.g. to modify a reservation, to cancel a reservation etc)

� These business partner integration issues always follow patterns � applicability of

service oriented integration architecture

14


Service Oriented Integration Architecture

CSI Reservation

Services

CSI Reservation

Services

CSI Reservation

Services

CSI Reservation

Services

CSI Reservation

Services

Business Partners

Vettro Inbound Request Service

GT3 Reservation

Request Service

GT3 Customer

Cancel Service

CSI Reservation

Services

Reservation Service Wrapper

City of Service

Calculation

Infrastructure Services

Orchestration Services Application Services

Vettro GT3Future

Partners

Sockets, Web Service (HTTP)

Message Store SecurityCommunication Notification Transformation

Invoice Service

15


Definition Of Service Layering

� Orchestration services:

� Orchestration services manage the flow of business transactions between The Company and the business partner

� They also map the partner requests to activities that are internal to The Company

� Reservation Request service

� Implements a business process where a business partner system sends a reservation request to The Company

� Vettro Inbound Request service

� Implements a business process that receives messages from a 3rd-party wireless network provider for interaction with the chauffeurs

16



� Application services:

� These are business process agnostic

� They are used by multiple business processes that implement connections with different partners (e.g. GT3, NetJets etc)

� Legacy reservation system wrapper service

� Encapsulates the existing reservation application

� The legacy system is a object oriented application with proprietary data formats

� There is an inbound and an outbound wrapper service

� City of Service calculation service

� Calculates the location of the nearest service hub based on pick-up and drop-off location

� Invoice service

� Encapsulates the billing system in order to provide external partners limited access

17



� Infrastructure services:� This layer contains services that are technology

specific� Message store service

� Captures an audit trail of all messages that are exchanged with business partners

� Communication service

� Provides an abstraction to several communication mechanisms� HTTP, RMI, JMS, SOAP

� Notification service

� Facilitates sending a notification message to operations personnel, e.g. via email

� Security service

� Provides authentication, encryption and decryption

� Transformation service

� Transforms data between partner formats, internal standard format, and the format of the legacy back office systems

18


Logical Architecture - Inbound

CSI Reservation

Services

CSI Reservation

Services

CSI Reservation

Services

CSI Reservation

Services

CSI Reservation

Services

(External) B2B Clients


Orchestration ServicesApplication Services

GT3

XML over

HTTP Post

Message Store Service

Security Service

Notification Service

Transformation Service

Identity Management Service

VettroOther

GT3 Reservation

Request Service

SocketsXML over

HTTP Post

GT3 Customer

Cancel Service

Message Store Presentation

Service

Socket Listener

EPCAB

Core Applications

Dispatch

CESRes

Reservation Inbound Service

B2B Gateway

JM

SS

OA

PR

MI

HT

TP

Co

mm

un

icati

on

Serv

ice


Vettro Inbound Request Service

City of Service

Calculation Service

CSI Reservation

Services

Reservation Outbound

Service

7

6

5

4

3

2

1

8

911

12

13

14

15

16 17

18

19

20

10

19


Sample Business Event Walkthrough

� A request for a new reservation is received from GT3; the B2B Gateway processes the request and returns a response to GT3:

1. The GT3 System sends a message

2. The message is received by the Sockets Listener, which is listening on a well-defined port for messages from GT3

3. The arrival of the message triggers the Orchestration Service of the GT3 Channel to be activated

4. The Orchestration Service calls the Security Service in order toauthenticate that the request came from GT3 based on User-ID / Password (and potentially a certificate)

5. The Orchestration Service checks if the GT3 system sent a duplicate request message (for example, due to a network problem)

6. The Orchestration Service calls the Message Store Service to save a copy of the request message in the message store such that it can later be retrieved and correlated to the response message

20


Sample Business Event Walkthrough (cont.)

7. The Orchestration Service checks if the data in the request message contains all the required information and verifies the format

8. The Orchestration Service calls the Transformation Service to transform the data into the Common XML format

9. The Orchestration Service calls the City of Service Calculation service to determine the city of service based on the pick-up and drop-off information in the request message

10.The Orchestration Service determines the type of the request (create, change, cancel). It then uses the Communication Service to send the XML over HTTP to the Reservation Inbound Service

11.The Reservation Inbound Service receives the XML

12.The Reservation Inbound Service calls the Transformation Serviceto transform the data into the format of the legacy reservation application

13. It then calls the reservation application

14.When the reservation application returns to the Reservation Inbound Service, the Reservation Inbound Service calls the Transformation Service to transform the data back into the Common XML format

21


Sample Business Event Walkthrough (cont.)

15. The Orchestration Service checks if the Reservation Inbound Service declined the reservation request

16. The Orchestration Service calls the Transformation Service to transform the data from the Common XML format into one of the response message formats that the GT3 system expects

17. The Orchestration Service calls the Message Store Service to save a copy of the response message in the message store such that itcan later be retrieved and correlated to the request message

18. The Orchestration Service inserts a user-ID and password into the response message

19. The Orchestration Service calls the Security Service in order toencrypt the message before it is sent to GT3

20. The Orchestration Service provides the response message to the Socket Listener, which sends the response message back to the GT3 System

22



� The B2B Gateway supports business partners that want to utilize

� automated reservation requests

� retrieve billing or car status information

� The business partners use their own, internal B2B client software

� They connect to an orchestration web service through HTTP

� Except for GT3, which requires a socket listener

� The orchestration services utilize a number of application services that are part of the B2B Gateway

23



� Similarly, the orchestration services utilize the infrastructure services

� Message Store to log messages, Transformation, etc.

� The core applications (Billing, Reservations, Dispatching) are logically and physically separate from the B2B Gateway

� The orchestration services use different means of communication to interact with them

� HTTP, JMS, etc.

24


Logical Architecture - Outbound


Message Store Service

Security Service

Notification Service

Transformation Service

Billing

Dispatching

Reservations


(External) B2B Servers Hudson XML

Bridge

Vettro Messaging

Server

HTTP SOAP JMS RMI

Communication Service

Hudson Outbound Request Service

Orchestration Services

Vettro Outbound Request Service


JM

SS

OA

PR

MI

HT

TP

Co

mm

un

icati

on

Serv

ice


Service

City of Service

Calculation Service


Invoice Service

XML over HTTP

Core Applications

B2B Gateway

5

4

3 21

25


Sample Business Event Walkthrough

� When “Dispatching” (a core application) needs to send a job allocation to a driver it pushes the information to the Vettro Outbound Request orchestration service:

1. Are dispatch request is generated by the Dispatch application inthe core system and put on a JMS queue

2. The communication service forwards the message to the VettroOutbound Request Orchestration service in the B2B Gateway

3. The orchestration service calls the Transformation Service to transform the data from the Common XML format into one of the response message formats that the Vettro system expects

4. The orchestration service uses the Communication Service to send the XML over HTTP to the 3rd-party wireless Web Service (Vettro Messaging Service)

5. The Vettro Messaging Service receives the dispatch request and notifies a driver through wireless communication

26


Verify SOA Principles

� SOA principle: reusability

� Application and infrastructure services provide business process agnostic functionality that can be reused across different business processes

� Reservation Service Wrapper:

� Under certain circumstances a change reservation request will require a cancellation followed by a rebooking

� Don’t put into orchestration service that is business process specific

� Implement something like this in the application service that isbusiness process agnostic and can be reused across processes

27



� SOA principle: reusability

� Future reuse: add functionality to accommodate broader reuse opportunity

� Security service:

� The current project requires authentication based on user-ID and password

� Add the capability to authenticate a business partner through certificates

� Notification service:

� The current project requires notifications based on email

� Add the capability to send out notification to another device

28



� SOA principle: reusability� Reusability is a key characteristic of SOA that needs

to be addressed during service design

� The data architecture is a key contributing factor to the reuse

� Define common message formats

� All newly developed services adhere to a XML-based standard for the data entities that are input and output of a service

� This essentially defines the interface of a service

� Legacy systems continue to use proprietary data formats� They are integrated into the SOA through wrapper services that

transform the data to the standard formats

� Some external systems (i.e. business partners) use proprietary formats, others adhere to a standard that The Company has defined

29


Designing Service Interfaces

� Reservation Request Orchestration Service operations:

� Create Reservation

� A request for the creation of a new reservation is received from a business partner

� Modify Reservation

� A request for the modification of an existing reservation is received from a business partner

� Retrieve Reservation

� A request to retrieve information about an existing reservation is received from a business partner

30


Designing Service Interfaces

� Reservation Request Orchestration Service operations (cont.):

� Cancel Reservation

� A request to cancel an existing reservation is received from a business partner

� Quote Request

� A request for a rate quote is received from a business partner

31


Designing The Schemas

� Schema Componentization & Type Reuse

� The Schemas for the standard B2B Gateway are designed in a 3-layer hierarchy

Create Reservation

Schema

Chauffeured Services Request & Response Message Schemas

OTA Common Types

Chauffeured Services Common Types

Modify Reservation

Schema

Cancel Reservation

Schema

Retrieve Reservation

Schema

Rate Quote Schema

32


Schema Componentization & Type Reuse

� Bottom layer:

� Types defined by the OpenTravel Alliance (OTA)

� They are common to several travel industries, e.g. airline, rental car, hotel

� Middle layer:

� Types defined by The Company for the chauffeured services industry

� They are common across different companies that service this industry segment

� Top layer:

� The Schemas defined The Company that represent the 5 reservation requests

33



� Many data entities occur in most of the Schemas

� They should be defined once and then reused across all Schemas

� Should follow established enterprise entity models (if available)

� Foster a standardization of XML components (i.e. elements and attributes) within the IT organization

� Results in significant reuse potential� New Schemas can be created very rapidly

� Usually, the new Schemas contain very little new types, but rather new combinations of existing types

� Data transformation logic can be segmented and reused

� Standardizes the data semantics that application services have to deal with

34



OTA Common Types

ReserveConfirmType

PersonNameType

GivenNameSurName

MiddleName NameTitle

AddressType

CountryCityName

StreetNmbr PostalCode

CustLoyaltyType

RateQualifierCoreType

FlightNumberType

Chauffeured Services Common Types

RateQualifierCoreType

VehicleRateTotal

CancelPolicy

ReservationID

Reservation Response

Chauffeured Services Transactions

PickUpLocation PassengerOfRecord

ReserveConfirmationDropOffpLocation

35


Data Integration Architecture

GT3 CSI


Application ServicesGT3

Future Partners

City of Service

Calculation

Reservation

Billing

Dispatching

Core Applications


Hudson

Vettro

Vettro

Vettro

Hudson Vettro

Common XML Format

Common XML Format

Proprietary Data Formats





Service

Invoice Service

36


Interface Consolidation

� There are 5 conceptual operations that make up the Reservation Request Service Interface

� The concrete service interface can be designed in the following ways:

� One interface with 5 operations and the associated input and output data items

� 5 different interfaces, whereby each interface has no explicit operation

� Invoking the service constitutes an implicit “operation call”

37


Interface Consolidation

� Design decision:

� Design the Reservation Request service as 5 distinct orchestration services

� Each orchestration service is exposed to the business partners as a Web Service

� The business partners invoke distinct Web Services that deal with the requested business function

� i.e. create reservation, modify reservation, etc.

� This achieves overall simplicity� Although it causes some duplication of code and proliferation

of orchestration services

38


WSDL + XML Schema

� When WSDL is used to describe the interface of a Web Service, the parameters of each operation need to be defined

� Requires type definitions

� There are 3 options for type definitions in WSDL

� Define all types explicitly in the type section of the WSDL

� No – types should be defined in reusable Schemas

� Define types in a Schema and reference needed elements individually in WSDL

� No – unnecessary duplication

� Define all types in a Schema and reference the entire schema in WSDL as a complex type

39


WSDL + XML Schema

� Define all types explicitly in the type section of the WSDL:

<types>

<xsd:schema …

<xsd: element name=“CreateReservationType”>

<xsd:complexType>

<xsd:sequence>

<xsd:element name=“="PickUpDateAndTime" type="xs:dateTime”/>

<xsd:element name=“="PickUpLocation" type="xs:string”/>

</xsd:sequence>

</xsd:complexType>

</xsd: element>

</xsd:schema>

</types>

� These types are then referenced in the “message”section of the WSDL

40


WSDL + XML Schema

� Define all types in Schema and reference the entire Schema as complex type in WSDL:

<types>

<xsd:schema …

<xsd:import “….. import the Schema here ….

<xsd: element name=“CreateReservationType”>

<xsd:complexType>

<xsd:sequence>

<xsd:element name=“Create“ type=“CreateReservation”/>

</xsd:sequence>

</xsd:complexType>

</xsd: element>

</xsd:schema>

</types>

� Again, this type is then referenced in the “message”section of the WSDL

41


Do We Need SOAP?

� Advantages of SOAP

� Many of the higher-layer WS*- standards are based on utilizing the SOAP header

� E.g. reliable messaging, security, transactions

� BUT:

� Some of these standards are immature

� They are not required for our project

� Disadvantages of SOAP

� Adds another layer of complexity

� Often used in RPC-style service invocation

� Service requests are mapped to method calls on components

� Results in tighter coupling of the Web Service provider and consumer

42


“REST-like” Services

� Many of the services that make up the B2B Gateway have been designed in a “REST-like”fashion, but …

� They are not completely REST since they don’t follow the concept of utilizing all the HTTP commands

� They are all based on the HTTP Post command

� E.g. the “Cancel Reservation” service should be based on HTTP Delete

� The “Retrieve Reservation” service should be based on HTTP Get

� They do not follow a strict addressing of resources based on URLs

� This had been dictated by the requirement to follow an industry standard for The Company’s business

43


Service Reuse

� The success of the B2B Gateway depends to a large degree on service reusability� The potential for reusing a service is significantly

improved when the SOI follows a proper layering of the services

� Orchestration, application, and infrastructure services

� Application and infrastructure services provide business process agnostic functionality that can be reused across different business processes

� The importance of the data architecture can not be over emphasized� Key contributor to reusability in a SOI

� Define a hierarchical structure of reusable data entities

� Developing new service interfaces as well as the code that implements a service becomes a predictable task

44


Service Reuse

� An initial investment into the architecture is required

� The overall SOI needs to be defined

� The first architected sub-project does not benefit, since it contributes services for later reuse

� We can distinguish 3 types of projects:

� Non-architected

� High cost, limited reuse, no contribution to SOA

� Unique partner integration based on SOI

� About 50% less cost based on reuse

� Standard (CSI) based partner integration

� About 90% less cost based on standards

45


Service Reuse

-2

0

2

4

6

8

10

12

-2 0 2 4 6 8 10 12 14

Contribution to SOI

Exploitation of SOI

GT3 V1.0

Architecture

Unified eRescode base

CSIGT3 V2

VettroVirgin Atlantic

Bubble size = cost of project

GT3 Web Service

GT3 V1.9

Hudson

Partner XPartner YPartner Z

46


ROI Through Service Reuse

� One approach to ROI in a SOI is to measure service reuse across a number of projects

� This means that a service is used in the context of different business processes

� Requires that application services and infrastructure services are designed such that they are autonomous and agnostic of the business process context within which they are executed

� The B2B Gateway project reused a number of services there were built in the core application modernization project

47



� Application service reuse� The following application services were reused:

� Reservation Service Wrapper – provides a service oriented interface by encapsulating the legacy reservation system

� City of Service Calculation Service – calculates the nearest service center based on passenger pick-up and drop-off location data

� Infrastructure service reuse� The following infrastructure services were reused:

� Transformation Service – converts data between XML representation and legacy system formats and business partner formats

� Communication Service – encapsulates different communication mechanisms

� Security Service – provides functionality like authentication and encryption

48


ROI Model For Software Reuse

� A ROI model needs to consider two factors:

� Cost savings based on service reuse

� Additional cost in designing and implementing reusable services

� Popular software ROI model:

� J.S. Poulin and J.M. Caruso in “A Reuse Measurement and Return on Investment Model,”Proceedings of the Second International Workshop on Software Reusability, Lucca, Italy, March 1993

� Dates back in time, but the fundamental assumptions are still correct

� The numbers have to be adjusted for today’s kind of projects

� I.e. given today’s technologies, application development approaches, and tools

49


ROI Model For Software Reuse

$40 (multiplier 1.33)

Multiplier: 1.2

$150Cost of non-reusable line of code multiplied by 1.5

Cost per reusable line of code

$5 (multiplier 0.16)

Multiplier: 0.1

$20 per line of reused code

Cost of non-reusable line of code multiplied by 0.2

Cost of reusing code engineered for reuse

$5,000$15,000 per defect

The cost of repairing defects once they have been promoted into production code.

Cost of defect repair

1.01.0The number of defects contained in a typical project’s production code; averaged over 10 or more projects.

Defects per 1000 lines of code

$30$100Cost for all development exercises of a project divided by number of lines of code; averaged over 10 or more projects.

Cost per non-reusable line of code

Case study

Poulin SOA values

Original Poulin Numbers

RationaleMetric

50



Lines of code Infrastructure Services reused 7,200

Lines of code Application Services reused 4,800

Total lines of reused code 12,000

Message store service 3000

Notification service 2000

6 CSI orchestration services 1800

2 GT3 orchestration services 600

2 Vettro orchestration services 600

Sockets protocol handler 2500

XML message content validation 2500

Configuration 1000

Total lines of code built from scratch 14,000

Cost without reuse $780,000

Cost with reuse $480,000Savings $300,000

Defect repair cost without reuse $130,000

Defect repair cost with reuse $70,000

Savings $60,000

Total savings $360,000

Savings percentage 46%ROI 100%

Reused code from Core Application project

Code built from scratch

51


Case Study Conclusions: Project Risks

� Components vs. services

� The development organization has been specializing in J2EE and Spring

� Moving to SOA requires a paradigm shift

� This was sometimes met with resistance

� Budgeting for reuse

� The cost savings of service reuse can easily be demonstrated

� However, developing for reuse incurs additional upfront cost

� Business pressures have sometimes forced a shorter term view on the project

52


Case Study Conclusions: Benefits

� The B2B Gateway provides The Company with two key benefits: insulation and automation

� It insulates The Company’s business partners from the intricacies of the core applications and business processes

� It insulates The Company’s core applications from any dependencies on business partners

� They can evolve independently, relying on the B2B Gateway to “make up” for any API changes

� The B2B Gateway facilitates automation of a diversity of business processes

� They range from dealing with reservation requests over exchange of billing information to wireless communication with drivers

building a soa with infrastructure, application, and orchestration … · 2009-07-30 ·...

Documents