Research WorkshopCalit2, November 7, 2007

Alexander Gruler, Sabine RittmannTechnische Universitaet Muenchen, Germany

Joint work with Ingolf H. Krueger, Matthew Arrott, Barry Demchak,Vina Ermagan, Emilia Farcas, Claudiu Farcas, Massimiliano Menarini

CSE Department – Calit2

University of California, San Diego

Representative Rich Services Papers• M. Arrott, B. Demchak, V. Ermagan, C. Farcas, E. Farcas, I. H. Krüger,

and M. Menarini, “Rich Services: The Integration Piece of the SOA Puzzle,” in Proceedings of the IEEE International Conference on Web Services (ICWS), Salt Lake City, Utah, USA. Jul. 2007.

• B. Demchak, C. Farcas, E. Farcas, and I. H. Krüger, “The Treasure Map for Rich Services,” in Proceedings of the 2007 IEEE International Conference on Information Reuse and Integration (IRI), Las Vegas, USA. IEEE, Aug. 2007.

Motivation

• Dramatic increase in distribution and complexity of software systems

– Business/Enterprise Systems– Technical/Embedded Systems

• Shift from stand-alone to networked systems

• Internet and Wireless Networks– key enabling technologies for advanced services

• Convergence between business and technical systems:– Telecommunication/Networking– Embedded Systems

Overview

• Background and Motivation

• State of the Art and Challenges of SOA Integration

• Rich Services

• Examples: Chat, Next-Generation Ocean Observatories, Rich Feeds

• Deployment Strategies for Rich Services using ESB Technology

• Summary and Outlook

Web Services – State of the Art• Several W3C standards backed by industry

– separation of concerns (HTTP/SOAP), – data marshaling (XML), – interface descriptions (WSDL)

• Service composition, Semantic web– Active research with results such as OWL-S

• Business workflows– Several approaches such as BPEL, WSCL, WS-CDL

Web Services – State of the Art• Addressing cross-cutting concerns

– Separate step through UDDI, WS-Security, etc

• Enterprise Service Bus technologies for deployment– Message-oriented middleware (MOM)– Flexible plug-in architecture– Rich set of data adapters/connectors for rapid connections

• Transition from logical architecture to ESB implementation is still ad-hoc

Challenges• Address crosscutting architectural concerns

– such as policy management, governance, and authentication

• Still maintain a lean implementation and deployment flavor?

• Horizontal: interplay at the same logical or deployment level of– application services– the corresponding crosscutting concerns

• Vertical: hierarchical decomposition into sub-services– the environment is shielded through encapsulation from

– their structural and behavioral complexity– the form of their composition

Rich Services – Why/What?

“To boldly go where no service has gone before”.

• an extension of the service notion, based on an architectural pattern• Dynamic adaptation

– new services can be introduced at runtime– no need to change or adapt the implementation of existing services

• Manage the complexity of a system-of-systems – decomposing into primary and crosscutting concerns– providing flexible encapsulation for these concerns– generating a model that can easily be leveraged into a deployment

• Workflow management– Service choreography at the infrastructure or application level

Rich Services: Scalable Service Integration

Messenger

Router/Interceptor

Policy

Ser

vice

/Dat

aC

onne

ctor

Messenger

Router/Interceptor

Failure Manager

...

<<Rich Service>> S

Ser

vice

/Dat

aC

onne

ctor

...

<<Rich Service>> S.n

Service/DataConnector }<<

Rich Infrastructure

Services>>

EncryptionService/Data

Connector

LoggingService/DataConnector

Failure Manager

Service/DataConnector

...

Service/DataConnector

S.1

Service/DataConnector

S.2

Service/DataConnector

}<<

Rich Application Services

>>

S.n.2

Service/DataConnector

S.n.m

Service/DataConnector

}

<<Rich

Application Services

>>

S.n.1

Service/DataConnector

Service/DataConnector

Logging

Service/DataConnector

Encryption

Service/DataConnector

Policy ...

Service/DataConnector

Service/DataConnector

<<Rich

Infrastructure Services

>>}

From tightly to l o o s e l y coupled systems

a hierarchically decomposed structure supporting“horizontal” and “vertical” service integration

Rich Services – Core• Main entities of the architecture

– Service/Data Connector - interaction between the Rich Service and its environment

– the Messenger and the Router/Interceptor - communication infrastructure

– Rich Services - encapsulate various application and infrastructure functionalities

• Rich Application Services– interface directly with the Messenger– provide core application functionality

• Rich Infrastructure Services – interface directly with the Router/Interceptor– provide infrastructure and crosscutting functionality– Examples: policy monitoring/enforcement, encryption, authentication

Composite Chat• Disparate Chat Systems• Rich Services for Enterprise Chat

– Systems of Systems– Service Oriented Architectures

Integration of existing solutionsFlexibility in configuration and management

Legacy and emergent capabilitiesTrust between domains

SecurityGovernance

Provisioning and policiesScalability

Disconnected operationDegraded serviceLow bandwidthPoint failures

Composite Chat Logical Architecture

Chat System 1

Enterprise Integration Layer

Chat Integration Layer (Chat System 3)

Chat System 2

Directory Service

Presence System Service

Authorization Service

Monitor Service

Logging System

Suspicious Activity Monitor

Chat Proxy

Service

Bandwidth Management

Logging System

Gateway

Monitor Service

Management Service

Presence System Service

Directory Service

DOS Monitor

Suspicious Activity Monitor

Chat System

CertificateAuthority Service

CertificateAuthority Service

Management Service

Authorization Service

} RIS

} RAS

} RIS

} RAS

Rich Services – Development Process

Rich Services Virtual Network

Rich ServicesRAS4

Services

Service S1

Roles

U1

U2

U3

U4

U5

Use Case Graph

ConcernsC1 C2 C3

C4CC1

CC2CC3

Domain Model

R1 R2

R3 R4

R5 R6

R1 R2

msg

R3

CC1CC2

Role Domain Model

R1 R2

R3 R4

R5 R6

CC1 CC2 CC3

Router/Interceptor

Messenger/Communicator

RAS1 RAS2

CC1 CC4 CC5

Router/Interceptor

Messenger/Communicator

RAS5 RAS6RAS3

S/D

S/D

RIS:

RIS:

Serv

ice

Elic

itatio

nR

ich

Serv

ice

Arc

hite

ctur

e

RAS7

System of Systems Topology

H1 H2

H3

H5

H6

H7

H8

H9H4

RAS1 RAS2 RAS3

RAS5 RAS6 RAS7

Infrastructure Mapping

H1:RAS1 H2:RAS2

H3:CC1

H5:RAS2

H6:RAS5

H7:RAS7H8:RAS7

H9:RAS6

H4:RAS3

Opt

imiz

atio

n ImplementationRAS1 RAS2

RAS3 RAS4

RAS5 RAS6

RAS7 CC1

CC2 CC3

CC4 CC5

Ana

lysi

s

Syn

thes

is

Ana

lysi

s

Iden

tific

atio

n

Def

initi

on

Con

solid

atio

n

Refinement

Hierarchic composition

Refinement

Logical Model

Syst

em A

rchi

tect

ure

Def

initi

on

Logical Architecture Loop

Deployment Loop

Example: Trading System

EnterpriseSupplier

S/D Connector S/D Connector

Store

CashDesk

S/D Connector

EncryptionS/D Connector

Router / Interceptor and Messenger / Communicator

Router / Interceptor

Messenger / Communicator

Bank

S/D Connector

Courier

S/D Connector

Customer

S/D Connector

Enterprise Management

System

S/D

CahsierLight Display

Printer Card Reader

Barcode Scanner

Cashbox System Cache

Sale System

Store Mgmt System

S/D

Store Inventory

S/D

StoreManager

S/D

StockManager

S/D

Dispatcher

S/D

Enterprise Repository

S/D

EnterpriseManager

S/D

Loggin AuditingStore Failure Management

Enterprise Failure ManagementS/D

QoS MonitorS/D

Logging CashDesk Failure Management

EncryptionS/D

Encryption

duplicate

Logical linkRecovered link

Enterprise serverStore serverCash desk PC

Store clientEnterprise client

Other entities

Ocean Observatory example

Global Scale Observatory Modeling Facility Research LaboratoryS/D Connector S/D Connector S/D Connector

Regional Cabled Observatory

Observatory Service

S/D Connector

Identity Authentication Policy Accounting LoggingS/D ConnectorS/D ConnectorS/D ConnectorS/D ConnectorS/D Connector

Identity Authentication Policy Accounting LoggingS/DS/DS/DS/DS/D

Router / Interceptor and Messenger / Communicator

Identity LoggingData

Service

S/D

Router / Interceptor

Messenger / Communicator

...

Acquisition scheduler

Science Instrument Web Service

Matlab Processing

Engine

Power Monitor

Instrument

Service Repository

S/D Connector

Storage

S/D Connector

Scheduler

S/D Connector

Resource Repository

S/D Connector

State Management

S/D Connector

Router / Interceptor and Messenger / Communicator

. . .

INTERNET

Service Rep.

. . .

State Mng.

S/D

S/D

S/D

Rich Feeds• Problems

– Research data feeds accessible over time– Needs for particular feeds cannot be predicted– Future restrictions and constraints can’t be anticipated

• Objectives– Capture Research Data Feeds– Expose Datasets– Remain Flexible and Extensible

Policy System

Logging System

RESCUE

ODBC Adapter

Visualization Tool

Research Data FeedDatabase

Authorization Monitor

Failure Detection/Mitigation

Rich Feeds Logical Architecture

• Today’s Data Feeds– Traffic– Trackable Objects

• Today’s Visualizations– Google Maps– Google Earth (very soon)

Composite Chat• Disparate Chat Systems• Rich Services for Enterprise Chat

– Systems of Systems– Service Oriented Architectures

Integration of existing solutionsFlexibility in configuration and management

Legacy and emergent capabilitiesTrust between domains

SecurityGovernance

Provisioning and policiesScalability

Disconnected operationDegraded serviceLow bandwidthPoint failures

Composite Chat Logical Architecture

Chat System 1

Enterprise Integration Layer

Chat Integration Layer (Chat System 3)

Chat System 2

Directory Service

Presence System Service

Authorization Service

Monitor Service

Logging System

Suspicious Activity Monitor

Chat Proxy

Service

Bandwidth Management

Logging System

Gateway

Monitor Service

Management Service

Presence System Service

Directory Service

DOS Monitor

Suspicious Activity Monitor

Chat System

CertificateAuthority Service

CertificateAuthority Service

Management Service

Authorization Service

} RIS

} RAS

} RIS

} RAS

Example: MULE as deployment system

•MULE Enterprise Service Bus–Relatively new technology with great potential–Ad-Hoc development process, needs new SOA perspective–Rich Services are a perfect match

Security – Authentication and Authorization

MULE BackboneEnd-to-End Data Transformation

Web Portal BPEL Web

ServicesJ2EE/EJB/

Servlet SAP IBM AS400

JBI (JSR-208)

File/FTP/SFTP

JMS, MQ Series,

ORACLE AQ

TCP, MCAST,

SSL

Caching (Distrib.)

Frameworks(Spring)

GRID,JavaSpace

E-CommEmail, IM

Ser

vice

/ Dat

aC

onne

c tor

Deployment using MULE• MULE or similar ESB for deployment architecture

Service ConnectorAdapter

Mule RouterEncryptionInterceptor

LoggingInterceptor

SanitizerRouter

Mule UMO Component

Mule TransformersMessage Receivers

Connector Dispatcher <<Rich Service>>

Service/DataConnector

Web service

WSDL

SOAP

Mule Transport (Messenger)Support: Jms, SOAP, Http, etc...

Mule (Router/Interceptor)