Demystifying ESB patterns
Colin Gniel and David Arnold, November 2009
Agenda
• ESB – A Recap
• SOA Reference Architecture
• IBM ESB offerings
• IBM ESB Positioning in the SOA Market
– Best fit, Hybrid and “Federated ESB”
• What is a Best Fit ESB
• What is a Hybrid ESB
• What is a “Federated ESB”
Assets Layer of disjointed, poorly understood enterprise assets, preventing reuse
Architectures Tightly-coupled architectures hindering IT flexibility
Skills IT skills shortage and silos limiting staff productivity and mobility
Processes and toolsDuplicate processes, tools and infrastructures limiting collaboration
Investments Increasing maintenance costs limiting flexibility for new investments
E01-EDI
Data Warehouse(Interfaces to and from the
Data Warehouse are not
displayed on this diagram)
G02 - GeneralLedger
A05 - AP
S01 - SalesCorrections
I01 POReceiving
I03 Return toVendor
I06 WarehouseManagement
M a inframe apps - B lue
PC/NT apps - G reen
Unix apps - Y ellow
3rd party interface - Orange
Lines: Colors have no special meaning.They are to help make the diagram easier toread.For More Information: See the databasecontaining information about eachapplication: Application V4.mdb
S06 - Credit App
P15 EES EmployeeChange Notice
OTHER APPS - PC
AP - Collections/Credit
TM - Credit Card DB
ACCTS REC APPS - PC
990COR
Bad Debt
Benefical Fees
Beneficial Reconcil
JEAXF
JEBFA
JEBKA
JEDVA
JESOA
JEVSA
JEVSF
NSF
TeleCredit Fees
INVENTORY CONTROL APPS - PC
Code Alarm
Debit Receivings
Devo Sales
Display Inventory
In Home
Junkouts
Merchandise W ithdrawl
Promo Credits
RTV Accrual
Shrink
AP Research - Inv Cntrl
AP Research-Addl Rpts
Book to Perpetual Inventory
Close Out Reporting
Computer Intelligence Data
Count Corrections
Cross Ref for VCB Dnlds
Damage W rite Off
Debit Receivings
DFI Vendor Database
Display Inventory Reconcil
Display Inventory Reporting
INVENTORY CONTROL APPS - PCDPI/CPI
IC Batching
Inventory Adj/Count Correct
Inventory Control Reports
Inventory Levels
Inventory Roll
Merchandise W ithdrawl
Open Receivings
PI Count Results
PI Time Results from Inv
Price Protection
Sales Flash Reporting
Shrink Reporting
SKU Gross Margin
SKU Shrink Level Detail
USM
VCB Downloads
Journal Entry Tool Kit
Scorecard - HR
L02-ResourceScheduling(Campbell)
P09 - P17Cyborg
M02 - Millennium
M03 - Millennuim 3.0
Banks - ACH and Pos toPay
Cobra
B01 - StockStatus
S03-Polling
P14 On-line NewHire Entry
CTS
Plan Administrators(401K, PCS, Life,
Unicare, SolomonSmith Barney)
D01 Post LoadBilling
I04 HomeDeliveries
I02 -Transfers
Arthur Planning
I07 PurchaseOrder
I12 EntertainmentSoftware
I05Inventory Info
E13E3 Interface
S04 - Sales Posting
V01-Price ManagementSystem
I10 Cycle PhysicalInventory
I55 SKUInformation
K02Customer Repair
TrackingI35 Early Warning
System
B02 MerchandiseAnalysis
I13- AutoReplenishment
U18 - CTO
Intercept
I09 Cycle Counts
E02-EmployeePurchase
Texlon 3.5
ACH
Stock Options
I17 Customer PerceivedIn-Stock
U16-Texlon
SiteSeer
C02 - CapitalProjects
F06 - FixedAssets
US Bank ReconFile
Star Repair
EDICoordinator
Mesa Data
NEW Soundscan
NPD Group
AIG W arranty Guard
Resumix
Optika
Store BudgetReporting
P16 - Tally Sheet
Cash Receipts/Credit
S05 - HouseCharges
Ad Expense
L01-PromoAnalysis
V02-PriceMarketingSupport
BMP - Busperformance Mngt
StoreScorecard
I11 PriceTesting
Valley Media
P09Bonus/HR
I15 Hand ScanApps
Roadshow
POS
S08 - VertexSalesTax
A04 - CustRefund Chks
Equifax
ICMS Credit
CellularRollover
S09 - DigitalSatelliteSystem
NPD,SoundScan
Sterling VANMailbox (Value)
I18SKU Rep
X92-X96Host to AS400
Communication
S02 -Layaways
Washington,RGIS,
Ntl Bus Systems
V04-SignSystem
I14 Count CorrectionsNARM
P01-EmployeeMasterfile
I06 - CustomerOrder
FrickCo
UAR - Universal AccountReconcilliation
DepositoryBanks
S07 - CellPhones
S11 - ISPTracking
AAS
Fringe PO
Cash Over/Short
L60 MDFCoop SKU Selection
Tool
SKUPerformance
SupplierCompliance
1
I35 - CEI
ASIS
Misc Accounting/Finance Apps - PC/NTCOBA (Corp office Budget Assistant)PCBS(Profit Center Budget System)
Merchandising Budget
AIMSMerch Mngr Approval
Batch ForcastingAd Measurement
AIMS Admin
AIMSReportingAd
Launcher
V03- MktReactions
SpecSource
CTO2.Bestbuy.com
RebateTransfer
SignSystem
CopyWriter'sWorkspace
ELTPowerSuite
StoreMonitor
AIS Calendar
Stores & Mrkts
Due Dates
Smart Plus
InsertionsOrders
BudgetAnalysis Tool
Print CostingInvoice App
AIS Reports
BroadcastFilter
Smart PlusLauncher
GeneralMaintenance
Printer PO
PrinterMaintenance
VendorMaintenance
Vendor Setup
Connect 3
Connect 3Reports
Connect 3PDF Transfe
Spec SourceSKU Tracking
S20-SalesPolling
Prodigy
PSP
In-HomeRepair
WarrantyBillingSystem
Process Servers(Imaging)
Prepared by Michelle Mills
Page 1 of 2
Actual application architecture of an IBM customer
J2EE
Linux
Palm / PDABlackberry
SAP / Oracle
VB / C++
PackagedApps
Improving Business Agility and the challenge of integrating existing applications
System i
System z
CICS / IMS
Web 2.0
Softwarearcheology
“Legacy modernisation is morphing into a strategic function. IT can't afford to toss away reliable application transactions indiscriminately.“
-- Phil Murphy, Forrester Research, April 2007
Outsourced vendor
Packaged app providerAcquired dev team
Testing team
Core dev team
Regional SI
LOB Leaders
Web team
Architecture team
Java/ . Net
Enterprise Service Bus
An ESB enables flexible SOA connectivity for integrating business applications and processes as services
Converts between different transport protocols
ESB promotes Agile ConnectivityThe Enterprise Service Bus (ESB)
Matches & routes communications between services
Connects everything to everything
Distributes Business events
Transforms between different data formats
Communications Layer
ESBService Enrichment,
Composition, Mediation,Connection
• Match & Route communications between services
• Converts between transport protocols• Transforms between data formats• Identifies and distributes bus events
Communications Layer(Messaging)
• Connect everything in your SOA• Leverage existing skills and assets• Support business growth and agility• Deliver fast ROI
A Cross-sectional view of the ESB
SOA Foundation Reference Architecture
Business Application
Services
Strategy and Planning Services
Enterprise Service Bus
Access Services
Partner Services
Process Services
Information Services
Interaction Services
Business Services and Events
Lifecycle Services
Asset and Registry Services
Development Services
Management Services
Infrastructure Services
Enables collaboration between
people, processes & information
Business-driven Enterprise Architecture and Standards
Manages diverse data and
content in a unified manner
Integrated
environment for
design and
creation of solution
assets
Connect with trading partners Build on a robust, scaleable, and
secure services environment
Facilitate interactions with
existing information and
application assets
Manage and
secure services,
applications &
resources
Optimizes throughput, availability and utilization
Orchestrate and automate
business processes
Supports the specification of enterprise business
solutions through business architecture
WebSphere DatapowerWebSphere MB
WebSphere MQ
WebSphere ESBWSRR
ESB builds upon the Communications Layer
Only WebSphere delivers the most complete ESB solution
– Adds “awareness” of what the messages contain on the backbone
– Adds message mediation, message transformation and content-based routing
– Underpins BPM and Application Integration in an SOA
WebSphere MQ family Embedded Messaging
WebSphere DataPowerIntegration Appliance
Purpose-built hardware ESB
with simplified deployment
and hardened security
WebSphereMessage Broker
ESB for universal connectivity and
transformation in heterogeneous
IT environments
WebSphere ESBBuilt on WebSphere
Application Server for an
integrated SOA platform
Communications Layer
IBM ESB Positioning within the SOA Market
Enterprise Service Bus
An architectural pattern within Service Oriented
Architecture rather than a product.
In nearly all cases a recommended best practice.
Best Fit ESB
Hybrid ESB
“Federated ESB”
Best Fit ESB
• What is a Best Fit ESB
– Single technology (product)
– Maybe multiple instance for scaling and availability
– Often some compromise required
• Non-functionals: throughput, response time
• Functionals: extension through coding required
– Runtime registry interaction not necessarily required
• When a “Best Fit” ESB
– Size of organisation
– Skills availability
– Scope of requirements
– Single Service Domain
Best Fit – WebSphere Message BrokerWhen to Consider?
• You are an existing WebSphere Message Broker user
Leverage existing WMB skills, while migrating to 7.0
• You have extensive heterogeneous infrastructures including
both standard and non-standards-based
Applications
Protocols
Data formats
• You have extensive MQ skills and infrastructure
• You are using Industry formats such as SWIFT, EDI, ACORD
• You need extensive pre-built mediation support
• You are implementing more complex messaging and integration patterns
• You have non-XML or complex transformation needs
• Reliability and extensive transactional support are key requirements
• You need high performance with horizontal and vertical scaling
• You use WebSphere Application Server
– Your team has skills with WAS Administration and Java coding
• You are now or planning on developing business
processes using WebSphere Process Server
– Common tooling, programming model, and runtime
• You are integrating with ISV business applications hosted
on WAS or 3rd party solutions which extend and support
WAS
• You want to mediate between Web services and existing
systems using JMS and WebSphere JCA Adapters
• Reliability and extensive transactional support are key
requirements
• You want to minimize your server investment by co-
hosting WebSphere services and ESB in one application
server
Best Fit – WebSphere ESBWhen to Consider?
• Ease of use is a pre-dominant consideration
– Simple experience of drop-in installation and admin-based
configuration with no or minimal development required
• You are transforming between XML-and-XML or XML-
and-any other format
• You are using XML-based or WS-Security extensively
• Your interaction patterns are relatively simple
• You require use of advanced Web services standards
• Your mediation requirements are met by the existing DP
mediations and minimal extensibility is needed
• You need to minimize message latency when adding an
ESB layer
• You are doing extensive XML processing combined with
high performance requirements
• Your ESB must be in production very quickly
Best Fit – WebSphere DatapowerWhen to Consider?
Hybrid ESB
• What is a Hybrid ESB
– Also termed, “Multi-node” or “Distributed” ESB
– Connectivity centric patterns
– Engineered for each customer to provide the optimal SOA value and footprint – powerful but small.
– Focused on connectivity and runtime optimization
– All about “feeds and speeds”
– Multiple technologies (products) to build an ESB
– Little compromise on Functional or non-Functional requirements
– Runtime registry recommended
• When a Hybrid ESB
– Single Service Domain
– Broad range of functional requirements
– Challenging non-functional requirements
Technology1 Technology3
ESB
Technology2
1414
Total
Spend
Requests /
Second
Custom ESB
Other Vendor ESB
WebSphere Message Broker
Hybrid ESB (with
appliance
Hybrid Bus(with appliance) enables cost effective high performance
1
2
• DataPower: Extreme Performance,
processing of standard messages at the
highest possible rates, low resource
consumption
• Ease of configuration and deployment
• Perimeter Security
• Message Broker: Universal Connectivity,
higher value ESB functions requiring greater
flexibility and lightweight orchestration
• For proprietary and packaged application
integration requiring hosted adapters or
custom extensions
Hybrid ESB and B2B Gateway ServicesKey
Portal Cluster – Mobile Users
Mobility Extensions,
Portal Server
Web Content Services
Mobility Extensions
Portal Server
Web Content Services
Load Balancing WebSphere Partner Gateway
Load Balancing WMB Instances
Backend Systems
BRAVO WISNET Other
External Information
Providers
ReutersRace
CalendarOther
Load Balancing Datapower XI50s
WMQ
External Information
Providers
SkySocial
NetworkingOther
Security
Services
Mediation
Services
Legacy
B2B
Gateway
Services
Complex
Mediation and
Service
Composition
Industry
Support
Services
Protocol
Switching
Services
Pe
rsis
ten
ce
Complex
Integration
Patterns
Integration Layer
Gateway
Services
Integration
B2B
Integration/B2B
Portal Cluster – Web Users
Mobility Extensions,
Portal Server
Web Content Services
Mobility Extensions
Portal Server
Web Content Services
“Federated ESB” – Federation of Services
• What is a “Federated ESB”
– Multiple ESBs working together
Maybe a combination of Hybrid and Best Fit ESBs
– Focused on non-runtime optimization, governance, system management, monitoring and security.
– Runtime registry integral part of the ESB
– Pre-requisites
Federated security
Federated systems management
Federated Governance
• When a “Federated ESB”
– Multiple Service Domains
• Example: Domain per Business Unit
• Domain per Geography
– Shared Services across business units
– Existing ESB technologies in business units
– Existing registries in business units
Service Domain Service Domain
Service
Federation
Service Domain
ESB 1
ESB 2
ESB 3
ESBs providing the Federated Connectivity to enable the Federation of Services across
multiple Service Domains in an Enterprise.
Analyst Views 2007 – the Future is Federated
ESB Adoption Trends - Gartner
“80% of large companies will have ESBs or similar SOA infrastructure products from three or more vendors in 2009. (0.7 probability)”
“…half of all large companies will apply a systematic, federated approach to managing their disparate SOA domains and ESBs in 2009. (0.7 probability)”
– Roy Schulte, Gartner, “Succeeding With Multiple SOA Service Domains and Disparate ESBs”, May 2007
ESB Adoption Trends - Forrester
“Some of Forrester's most advanced clients got on the ESB bandwagon years ago and today find that they have an embarrassment of riches, with multiple ESBs. This outcome is almost inevitable for larger enterprises; no single ESB today can satisfy all requirements equally well across the whole of a large enterprise.”
– Larry Fulton, Forrester, “Shaping Your Middleware Strategy to Benefit from ESBs, May 2007
The Future is Here!
“FederatedESB”
ServiceSecurity Service
ManagementService
Governance
Service Federation – going beyond connectivity Which Links Together Multiple, Disparate Domains
A “Federated ESB” is an enterprise-spanning connectivity infrastructure of multiple ESBs working together to extend
service reuse across as well as within domains.
Domain
Domain
Domain
Federated Connectivity Patterns
• Peer to Peer (direct routing)
– Peers share services directly
– All domains are service providers and consumers
• Parent/child (direct routing)
– Children do not share services
– Parent is a provider of services
– Parent is a consumer of child services
– Children are consumers of parent services
• Brokered (centralized routing)
– Children share services via the brokering domain
– Parent and children are both providers and consumers of services
1 2
2
B
1
2
P
1
P P P
21 21 21
3
• Most, if not all, ESB architectures will include a Registry and Repository
– WSRR implementation maybe beyond first deployment phase
• Non-Runtime Considerations
– Lifecycle management (ex. Conformance) in ESB
– Service Cataloging
• Runtime
– Need runtime lookup of runtime service endpoint locations for dynamic routing
– Need to use service metadata at runtime –policy driven ESB
– Use real-time information about service use in the ESB provided via WSRR using ITCAM for SOA
• Key to Service Federation
When To Add A Service Registry to an ESBWhen to Consider WSRR?
Role of the registry in ESB Patterns
Best Fit Hybrid Federated
Development Recommended Highly
Recommended
Essential
Runtime Optional Recommended Highly
Recommended
Federation model (Service Connectivity Management)
Atom/APP based protocol used
to add, update, remove resources
Federation
Console
DomainFederation
Connectivity
Provider
Service
Group
Registry
Proxy
Service
Endpoints
SFM
Coordinator
• WSRR v7.0 includes product in support of this model
– Injection of service proxies into buses
– Projection of service proxy meta data into the appropriate
registry/registries
• Supported Registries
• WSRR,UDDI
• Support ESB products
– WMB and WESB (DP to come next year)
Service Domains, Service Providers and proxies
Proxies in consumer and/or provider domain allow:
Control points for policy enforcement
Identity mapping between trust domains
A’A’’ AA’
A’ AA’’
Provider domainConsumer domain
Service Registry Service Registry
ESBESB
Delegation Delegation