pragmatic business value of using service orientated architecture

blackmarble

the strategic IT asset for your organisation

PRAGMATIC BUSINESS VALUE OF USING SERVICE ORIENTATED ARCHITECTURE AND IMPLEMENTING AN ENTERPRISE SERVICE BUS

blackmarble

FBCS, CEng

Senior Architect

Black Marble LTD

Robert Hogg

blackmarble

Agenda

• What is SOA?• Connecting Systems• Demo• Governance• Getting the Return

blackmarble

blackmarble

Key Business System Drivers

Businesses need:• Flexibility

• Agile Systems• Rapid Development Cycle• Extensibility• Collaboration – local/national/global

• Rigidity• Reliability• Concrete • Guarantees and Risk Management• Compliance with governance

blackmarble

• Service Oriented Architecture: enables loose coupling, interoperability, management of change, and operation of business services in a governable environment. Business Services operating in a well run SOA can be composed into business process that align IT with the business.

SOA TerminologyTwo Definitions of SOA

• Service Oriented Architecture: a software design approach in which key functions are built as reusable components which implement industry standards for interoperable communications.

• Service Oriented Architecture: enables loose coupling, interoperability, management of change, and operation of business services in a governable environment. Business Services operating in a well run SOA can be composed into business process that align IT with the business.

• Service Oriented Architecture: enables loose coupling, interoperability, management of change, and operation of business services in a governable environment. Business Services operating in a well run SOA can be composed into business process that align IT with the business.

• Service Oriented Architecture: enables loose coupling, interoperability, management of change, and operation of business services in a governable environment. Business Services operating in a well run SOA can be composed into business process that align IT with the business.

• Service Oriented Architecture: enables loose coupling, interoperability, management of change, and operation of business services in a governable environment. Business Services operating in a well run SOA can be composed into business process that align IT with the business.

blackmarble

State of SOA• Perceptions

– SOA = IT centralization/standardization– “SOA is dead” (Anne Thomas Manes, Burton Group; January 2009)

– Cloud computing will replace SOA

• Reality– SOA does not solve all enterprise IT issues– SOA is complex, and difficult– Core value of SOA is agility, but needs vary– SOA demands change, but organizational resistance is

often the biggest impediment– Need to focus on architecture, not technology

blackmarble

SOA Manifesto

Service orientation is a paradigm that frames what you do.

Service-oriented architecture (SOA) is a type of architecture

that results from applying service orientation.

Business value over technical strategy

Strategic goals over project-specific benefits

Intrinsic interoperability over custom integration

Shared services over specific-purpose implementations

Flexibility over optimization

Evolutionary refinement over pursuit of initial perfection

.

blackmarble

Technology model

TECHNOLOGY

Business model

BUSINESS

Business siloed by IT

Solutions impose

constraintsInward

technology driven view

Focus on document exchange

IISSiebel

AS400Tandem

Applications

Finance

Logistics

R&D

The Benefits of SOA

• Align Business and IT

• Increase Agility • Reduce Costs

Source: Dr Arvindra Sehmi

BUSINESS

TECHNOLOGY

Soft boundaries(process and services)

Processes

Services

Service model

Business model

Technology model

Outwardbusiness-

driven view

Requirements and solutions

closely aligned

Focus on

service

Focus on

contract

blackmarble

Common Myths about SOAMyths Facts

1. SOA is a technology 1. SOA is a design philosophy independent of any product technology or industry trend.

2. SOAs require web services 2. SOAs may be realized via web services but using web services will not necessarily result in a SOA.

3. SOA is new and revolutionary. 3. EDI, CORBA, and DCOM were conceptual examples of SOA.

4. SOA ensures the alignment of IT and business

4. SOA is not a methodology

5. A SOA Reference Architecture reduces implementation risk

5. SOAs are like snowflakes, not two are the same.

6. SOA requires a complete technology and business process overhaul

6. SOA should be incremental and built on your current investments.

7. We need to build a SOA 7. SOA is a means, not an end.

blackmarble

Common Myths about SOA #2Myths Facts

SOA begins at the firewall

SOA Aligns the Business With IT

SOA Governance Fixes Everything

SOA begins at the firewall

People Align Business With IT

Governance needs to span all of IT,not just SOA

blackmarble

the strategic IT asset for your organisation

CONNECTING SYSTEMS

blackmarble

Connecting Application is Not EnoughPoint-to-Point Integration Inhibits Agility

• Point-to-Point– Complex Interface– Redundant Logic– Doesn’t Scale– Lacks Visibility

• Business Impact– Delays Response

to changing business needs

JD EdwardsService

SAPService

Java Appl.

Service

.Net Appl.

Service

CICS Service

AS/400Service

MS CRM

Service

OracleService

blackmarble

“End Point/Service Oriented Spaghetti”

15 Service Providers22 Service Consumers

Guess how many Service Providers and Consumers are in this Spaghetti?

Even a few dozens of services pose quite a plumbing challenge!

blackmarble

Introduction to ESB

“A Web-services-capable infrastructure that supports intelligently directed communication and mediated relationships among loosely coupled and decoupled biz components.”

- Gartner Group

blackmarble

Enterprise Service Bus (ESB)

E-COMMERCE

CRMBUSINESS PARTNER

HR

ERP

BizTalk was positioned as a Hub and spoke

Static Receive Port

Static Send Port

Hard Coded Map Name

Static Schema

Static Schema

Static Rece

ive Port

Static Send Port

Hard Coded Map

NameStatic S

chema

Static Sch

ema

Static Receive Port

Static Send Port

Hard Coded Map

Name

Static Schema

Static Schema

Most decisions are made and locked in at Design time or at Deployment not at Run time!

Any change is a re-development or a system re-

configuration!

We’ve allowed the easy to use tools to define our

architecture

Now we’re saying it can be an Enterprise Service Bus?

blackmarble

We need to agree on what an Enterprise Service Bus is and what it does.

• Message Broker• Message Transformation• Message Validation• Message-Oriented Middleware• Service Orchestration• Adaption

blackmarble

ESB Core

• Configurable, reusable architecture• Addresses common concerns generically - Routing,

Transformation, Exception Handling, Monitoring etc.• Enforces Consistency and Unification• Leverages modern Standards (WS-*), still supports

Legacy• Central point for Service Monitoring and Administration• Increases agility (through added Abstraction, Loose

coupling)• Faster implementation times (through Configuration vs.

Coding)

blackmarble

Agility thru Service CompositionService

ConsumerService

ConsumerService

Consumer

Service Provider

Service Provider

Service Provider

Invocation & Orchestration

Transport ProtocolConversion

Data Format Transformation

Location & VersionTransparency

Message InteractionsSupport

En

terp

rise

Ser

vice

Bu

s

Error Handling & Repair

blackmarble

Relevant Pattern ImplementationESB Guidance

Policy Centralization

RulesCentralization

Event-Driven

Messaging

Data Format Transformation

Protocol Bridge

MessageRouter

Message Broker

Legacy Wrapper

Architectural Design

Patterns

Message Routing

Scatter-Gather

Routing Slip

Recipient List

Message Transformation

Content Enricher

Data Model Transformation

ESB Usage Patterns

Repair and Resubmit VETO/

VETRO

Gateway

?

Reply Forward

?

Metadata CentralizationPerimeter

Service Router

Content-BasedRouter

blackmarble

ESB Toolkit - New Abstraction Layer on Top of BizTalkDeclarative, Meta-data, Policy and Configuration –Driven.

Transform ServiceRouting Process Orchestration

ProtocolAdaptation

End Point ResolutionPub/Sub ServiceService Consumers Service Providers

1.Transform my message2.Determine which endpoint I

need3.Route my message4.Route the response to a

second service5.Return the final result to me

On Ramp Off Ramp

Transform my message

Resolve a service end point address for me

BizTalk ESB Toolkit

blackmarble

Web Services

BizTalk

On-ramps Off-rampsItinerary Services ResolversAdapters

Exception

Microsoft ESB Guidance ArchitectureManagement Portal

blackmarble

Core Web Services

Resolver Web Service

Transformation Web Service

UDDI Web Service

Exception Web Service

Operations Web Service

BizTalk Send Ports

Off-Ramps

BizTalk Receive Ports

On-Ramps

BizTalk ESB Toolkit Architecture

Exception Management

Store

ESB Management Portal

Provisioning Framework

Reports

Alerts

ESB Toolkit Core Itinerary Services

Exception Management Framework

Exception Logger

Exception Handler

Fault Processor

Resolver-Adapter Provider Framework

Resolvers (…)Adapter Providers(…)

UDDI 3.0 BAM Generic SOAPSend

Generic WCFSend

Generic JMSSend

CustomSend

Pipeline

Pipeline

Pipeline

Custom Pipeline

ItineraryStore

Generic SOAPReceive

Generic WCFReceive

Generic JMSReceive

CustomReceive

Pipeline

Pipeline

Pipeline

Custom Pipeline

Route Service

Custom Service

Transform Service

Transformation Engine

Business Rules Engine

Orchestration Engine

BizTalk Pub/Sub Engine

blackmarble

ItineraryServices:

Message Processor

Resolvers:Context Finder

AdapterProviders:Adapter Properties

The Core ESB Toolkit Components

ItineraryMediation Policy

blackmarble

Itinerary Concept

• This is the heart of ESBG –Allows us to implement the ESB pattern

as opposed to a Hub/Spoke pattern

• It provides the runtime flexibility that BizTalk doesn’t have by default

blackmarble

Itinerary Mechanism

Itinerary ConceptLogical overview

Orchestration Based ESB Services

RouteTransform

Messaging Base ESB Services

RouteTransfor

m

Itinerary CacheMessageBox

Itinerary Processing

Generic On-

Ramps

Itinerary

Message body

blackmarble

Three Ways to Assign Itineraries

Itinerary is resolved on the server’s ESB On-Ramp receiving port via configurable resolver

Service

Proxy

Client resolves itinerary via resolver service and then sends it in a WCF/SOAP Header request

AdaptiveServiceClient

Client sends an itinerary in a WCF/SOAP Header request

AdvancedServiceClient

blackmarble

BizTalk Dynamic Send Port

Itinerary Based Routing & Processing

• Light-weight service sequencing & composition– Invokes itinerary

(internal) and external services

– Dynamic service context resolution at runtime

• Maps internal service invocation to BizTalk service containers– pipelines in ports and

orchestrations.

Pub/Sub Engine

BizTalk Receive Port

Routing Service

On Ramp Receive

Pipeline

Resolver

Adapter Provider

Off Ramp Send

Pipeline

The “heart” of the ESB Toolkit

BizTalk Orchestration

ItineraryService1

Resolver1Resolver1

Pub/Sub Engine

blackmarble

Itineraries DeploymentDesign Time

XML File

ItineraryRepository

Test/Run Time

Itinerary

OnRamp

Itinerary Selector

ESB Processing

Itinerary &Message

Message

ItineraryRepository

BizTalk Deployment Tools

blackmarble

Resolvers

For runtime flexibility services are not hardcoded to specific endpoints or maps

Service end-point metadata can be determined at runtime

Resolvers are called by services to set an artifact (map , itinerary) or endpoint metadata

blackmarble

ESB Toolkit Resolver

• Can resolve:– Message delivery endpoints– Maps for transformations– Endpoint configuration– Custom service metadata– Itinerary to be associated with a message

• Endpoint/Map resolution with:• UDDI 2.0/3.0• XPATH• Static• Business Rules Engine (BRE)

• Itinerary resolution with:• Itinerary (BRE)• Itinerary Static

blackmarble

Resolver Mechanism

ESB Services

A resolver configuration string contained within the

itinerary is provided to the resolver manager

Resolver Mechanism

Static

UDDI

SQL XPath

BRE

WSMEX

Resolver Manager

The resolver mechanism is able to load different

resolvers based on where the runtime data is stored

Rules Engine

Service Registry Database

blackmarble

ESB Resolvers

Endpoint Resolution

UDDI 2.0

Artifact Resolution

ESB Resolvers

Static Static BRE

BRE Itinerary - StaticXPATH Itinerary - BRE

UDDI 3.0

ESB Toolkit V2

ESB Guidance V1

blackmarble

ESB Adapter Provider

• The following Adapter Providers supplied:– WCF-BasicHTTP– WCF-WsHTTP– WCF-Custom

• WCF-SQL

– FTP– WebSphere MQ– File– SMTP

• Custom– any BizTalk adapter (that support dynamic ports) can be

exposed as an adapter provider

blackmarble

ESB Adapter Providers

FTP

ESB Adapter Providers

WebSphere MQ

File

Custom

WCF-BasicHTTP

WCF-WsHTTP

WCF-Custom

ESB Toolkit V2

ESB Guidance V1

SMTP WCF-SQL

blackmarble

ESB Exception Management Portal

blackmarble

DEMO

blackmarble

GOVERNANCE

blackmarble

SOA Governance Architecture

Publish/SubscribeInfrastructure

ESB Core ServicesVirtualized Service

Off-ramp

IIS

Web ServiceOn-ramp

ManagementPoint

Governance Portal

Governance Server

Security Enforcement

SLA Enforcement

Service Registry

Service Metrics Collection

Service Configuration Metrics Reporting

Metrics Reporting Service Lifecycle Management

PhysicalService

blackmarble

ESB Governance Integration

• Integrated with governance solutions from AmberPoint and SOA Software

• Specific functionality varies by governance provider, but typically include:– Service metrics– Policy-driven security– Policy-driven service-level agreement– Alerting

blackmarble

Defining Solutions As A Set Of Capabilities

Mapping ServiceRoutingProcess

Orchestration

ProtocolAdaptation

End Point Resolution

Pub/Sub Service

WSDL Schema

WSDL Schema

Service Consumer

1.Transform my message2.Determine which endpoint I

need3.Route my message4.Route the response to a

second service5.Return the final result to me

blackmarble

the strategic IT asset for your organisation

GETTING THE RETURN

blackmarble

Business Value

• Broaden availability of Information and Capability

• Exception based processing• Shorten business process cycles• Regulatory compliance • Alignment of systems with the business

blackmarble

Limitations

• Not all Capabilities are modular• Issues over Transfer of Control and Ownership • Modelled process do not implement all activities, e.g.

Human intervention • Difference between managing documents and

business processes

blackmarble

Road to failure about for Implementing SOA

• Typically a one or more of:– Lack of Comprehension of Business needs.– IT first Design.– Poor Management.– Poor implementation.– No Need for SOA

blackmarble

Key Lessons

• Critical to have a single shared vision by business and IT• Enterprise logging and monitoring critical for finding

and resolving problems• Apply Solid change management

– Test Change Management before initial rollout• Governance, Governance, Governance• Consider an Agile rollout but do it well.• After Rollout keep talking to the business and keep the

value of the investment visible by showing agility.

blackmarble

Loosely coupled messaging environment

Policy driven message level routing and mediation

Itinerary based processing

Enterprise Service Bus - Summary

Higher levels of reuse

Lower operational costs

Dynamic business changes

Business and exception analytics

Business requirements in the drivers seat

What it gives you…

Why you want it…

blackmarble

Where Next?

• Online content–http://www.msdn.com/esb –http://www.codeplex.com/esb –http://www.soa-manifesto.org/

blackmarble

How to Contact Us

01274 841305

www.blackmarble.cominformation@blackmarble.com

@blackmarble

blackmarble

the strategic IT asset for your organisation

APPENDIX

blackmarble Endpoint A

UDDI Repository

Version PolicySchema

Metadata

Endpoint Metadata

blackmarble Endpoint A

UDDI Repository

Endpoint BEndpoint C

Intelligent Routing

blackmarble Endpoint A

UDDI Repository

Endpoint BEndpoint C

Transformation

blackmarble Endpoint A

UDDI Repository

Endpoint BEndpoint C Endpoint D

Orchestration

blackmarble

Vertical Market Application

DatabaseMainframe

Application

J2EE Application.NET ApplicationWeb Service

Endpoint

Adaption

blackmarble Service Consumer

AsynchronousService

SynchronousService

Message Exchange Patterns

Publishing/SubscribeService