middleware for high availability and scalability in multi-tier and service-oriented architectures

Middleware for Middleware for High Availability and High Availability and

ScalabilityScalability in Multi-Tier and Service-in Multi-Tier and Service-

OrientedOrientedArchitecturesArchitectures

© Francisco Pérez-SorrosalAdvisor: Marta Patiño-Martínez

Distributed Systems Laboratory (DSL/LSD)

Universidad Politécnica de MadridMadrid, Spain

MotivationMotivation

Ph.D. Thesis © Francisco Pérez-Sorrosal

2

DataServices

Com

pany

A

Apps.

Stateful Data &Transactions

EXPENSIVE

May 13th 2009



3

Data

Services

Com

pany

B

Services

DataServices

Apps.

Apps.

Apps.

Data

StateConsistency

Cluster

Replica

May 13th 2009



4

Service A Com

pany

AC

ompa

ny BService B

Service C

Critical Service

CompositeApplication

May 13th 2009

OutlineOutline• High Availability (HA) and Scalability

in Multi-Tier Architectureso Protocols for High Availability in MTAso A Protocol for HA and Scalability in MTAs

• High Availability in Service-Oriented Architectureso WS-Replication Framework

• Conclusion• Publications


5May 13th 2009

Multi-tier Multi-tier Architectures: Architectures:

MotivationMotivation• Great success of MTAs

o CORBA, .NET & J(2)EE

• Cache requires concurrency controlo Serializabilityo Synchronization with the

underlying database

• Many databases provideo Classical isolation levels +

Snapshot IsolationPh.D. Thesis © Francisco Pérez-Sorrosal

6

Cache

ApplicationServer

May 13th 2009

HA and Scalability in HA and Scalability in MTAs: ContextMTAs: Context

• J2EE application serverso Transactional Services:

• ACID Transactions (JTA)• Advanced Transactions (Activity Service) • Our implementation available at http:\\jass.objectweb.org

o Component Model: Enterprise Java Beans (EJBs)• Stateless (SLSB) and Stateful (SFSB) Session Beans, Entity

Beans (EB) & Message-Driven (MDB)

• When replicating EJBs:o SLSBs & MDBs don’t keep state => NOT Replicatedo SFSB beans keep client-related state across requests o EBs represent persistent data in a datasource


7May 13th 2009

Horizontal Replication Horizontal Replication (DB Replication)(DB Replication)


8

BOTTLENECK& SINGLE POINT

OF FAILURE

X X

(X)

X

ReplicationProtocol

May 13th 2009

Horizontal ReplicationHorizontal Replication(App. Server (App. Server Replication)Replication)


9

BOTTLENECK& SINGLE POINT

OF FAILURE

(X)

X

(X) (X)

ReplicationProtocol

May 13th 2009

Horizontal ReplicationHorizontal Replication(AS and DB Replication)(AS and DB Replication)


10

COMPLEX

(X) (X) (X)

X X X

ReplicationProtocols

May 13th 2009

Our Solution: Vertical Our Solution: Vertical ReplicationReplication

• No single bottleneck• No single point of failure• Only one replication protocol


11

Unit ofReplication

ReplicationProtocol

May 13th 2009

OutlineOutline

• High Availability (HA) and Scalability in Multi-Tier Architectureso Protocols for High Availability in MTAso A Protocol for HA and Scalability in MTAs


• Conclusions• Publications


12May 13th 2009

Protocols for HA in Protocols for HA in MTAsMTAs

• Consider session data (SFSBs) and persistent data (EBs)• Are transaction aware & mask failures transparently• Approach: Vertical Replication + Primary-Backup


13

App.Server

DB

Primary

App.Server

DB

Backup

Client

ClusterPrimary

GCS

May 13th 2009

Our protocols offer...Our protocols offer...• Data consistency in all the replicas

o Vertical replication + transaction managemento 1-copy correctness

• Exactly-once executiono The client performs a request only once and gets the

results also only once

• High available transactionso The replication protocols are transaction-awareo Transactions are not aborted if the primary fails

• Different interaction patternso 1 Req/1 Tx, N Req / 1 Tx, 1 Req / N Txs and N Req / M Txs


14May 13th 2009

N requests/1 transaction: N requests/1 transaction: GoalsGoals

• Support transactional conversations o Several client requests inside a single

transaction

• Upon failover, resume the conversation from the last interactionoDo not abort ongoing transactions


15May 13th 2009

N requests/1 transactionN requests/1 transaction


16

T1Client invocations inside T1

T2

Client invocation that requires a new TX T2

May 13th 2009

N Req / 1 TX Replication N Req / 1 TX Replication Protocol: Primary (Begin)Protocol: Primary (Begin)


17

TM

Backup

Intercep

tors

Primary

beginTx

Backup

beginTx + TxId

Client

TxId

TxId

May 13th 2009

Replication Protocol: Replication Protocol: Backup (Begin)Backup (Begin)


18

TxTable

Intercep

tors

Backup

beginTx + TxId

Store TxId

May 13th 2009

Replication Protocol: Replication Protocol: Primary (Invocation)Primary (Invocation)


19

BackupIntercep

tors

Primary

EJB

EJB

EJB

EJB

Backup

TxId + Bean changes + Response

ClientTxIdResponse

May 13th 2009

Replication Protocol: Replication Protocol: Backup (Invocation)Backup (Invocation)


20

Uncommitted Table

Intercep

tors

Backup

Response Table

Apply

Save

SFSB

TxId+ Bean changes + Response

EB

May 13th 2009

Replication Protocol: Replication Protocol: Primary (Commit/Abort)Primary (Commit/Abort)


21

TM

BackupIntercep

tors

Primary

commit/ abort Tx

Backup

Commit/abort Tx + TxId

Client

EB DBEBEB

May 13th 2009

Replication Protocol: Replication Protocol: Backup (Commit/Abort)Backup (Commit/Abort)


22

Uncommitted Table

Intercep

tors

Backup

Apply

SFSB

commitTx + TxId

EB

EBDBEB

EB

May 13th 2009

Replication Protocol: Replication Protocol: FailoverFailover


23

Intercep

tors

BackupPrimary

TxTable

Uncommitted Table

Apply

SFSB EB

TM

For each non-completed Tx

...

Create

Response Table

Client

TxId

Response

NOT FOUND,

RE-EXECUTE REQUEST

May 13th 2009

Evaluation: ECPerfEvaluation: ECPerf

• Benchmark to evaluate the throughput and scalability of J2EE Application Servers

• Emulates the processes involved in a supply-chain management scenario

• The load is measured as the Injection Rate (IR)o # of clients = IR * 5

• Throughput is given in Benchmark Business Operations per Minute (BBOps/Min)


24May 13th 2009

Experiment SetupExperiment Setup

• JBosso Non-replicated

• JBoss Primary-Backupo Only SFSB replicationo Shared DB

• Our replication protocolo Primary-Backup +

Vertical replicationo SFSB & EB replicationo Transaction awarePh.D. Thesis © Francisco Pérez-Sorrosal

25

J2EE App.Server

Host 2

Client

DB

Host 3

Host 1

J2EE App.Server

Host 2

Client

Cluster

DB

Host 3

Host 1

J2EE App.ServerBackup

Host 4

Cluster

J2EE App.Server

Host 2

Client

DB

Host 3

Host 1

J2EE App.ServerBackup

Host 4

DBBackup

Host 5

May 13th 2009

ECPerf: ThroughputECPerf: Throughput


26

17

20%

21

May 13th 2009

ECPerf: Response TimeECPerf: Response Time


27

ECPerf Limit

May 13th 2009

OutlineOutline





28May 13th 2009

Limitations of Current Limitations of Current Middleware for HA in Middleware for HA in

MTAsMTAs

• Mismatch between isolation at Application Server and DBMSo Current application servers do not work

correctly with SI databases

• Snapshot Isolation (SI) has become the “de-facto” standard isolation level

• Current middleware does not scale-out stateful applications consistently


29May 13th 2009

Our Protocol for HA and Our Protocol for HA and Scalability in MTAs…Scalability in MTAs…

• Is consistent, high available and scalable• Includes a SI cache at the middleware

level for correctness and performance in a single replica

• SI cache is combined with replication for scalability and fault-tolerance in a cluster

• Vertical replicationo Only-one replication protocol coordinates the

execution of transactions and the propagation of changes in a cluster


30May 13th 2009

Snapshot IsolationSnapshot Isolation


31

Start Timestamp = 10Reads the version of TReads the version of TPrevious TX T has written X, so C=10

Start Timestamp = 11Reads X written by T2Validation succeedsIncrement C and set new version for X

New private version of XCounter of Committed TXs Conflict with T2Read-only Tx -> No validation

May 13th 2009

A Protocol for HA and A Protocol for HA and Scalability in MTAsScalability in MTAs


32

App.Server

DB

Replica 1

App.Server

DB

Replica 2

Client

Cluster

Client

May 13th 2009

Protocol FeaturesProtocol Features• Transactions: Started at the same time in AS and DBS• SI Cache: Maintains a certain number of versions to

o Avoid accesses to the DBo Guarantee conflict detection

• Conflicts: o Locally: Detected on-the-fly (Pessimistic)o Remotely: Detected on a validation phase

• Other Issues: o Creation and Deletion of Components (CRUD Ops.)o Garbage Collectiono Session Replication o Failure Handling (Transparent failover of clients)o Replica Recovery


33May 13th 2009

How the Multi-version How the Multi-version Cache WorksCache Works


34

Y:bX:a

Update x

x = e

T1Update y Read xT2 Read yCommit

y = fx = a

X:e Y:f

y = b

T1: STS=10 T2: STS=10CTS=11

Ver =11 Ver =11Ver = -1 Ver = -1

Ccounter=10Ccounter=11

Start T1Start T2

Update y

conflict

Abort

abortcommit

Cache

Read x

May 13th 2009

Cache ReplicationCache Replication


35

Replica 2Replica 1

X:a

Cache

Update x

x = e

T1 T2Commit

x= f

x = a

X:e

x = a

T1: STS=10 T2: STS=10CTS=11

Ver =11 Ver =11

Ver = -1 Ver = -1

Ccounter=10Ccounter=11

Start T1

Update x

Ccounter=10

Commit

Ccounter=11

X:a

X:e

Cachex = e

T1: STS=10

CTS=11GCS

T1: STS=10X=e

T2: STS=10 x = f

T2: STS=10 x = f

T1: STS=10 x = e

Conflict

Abort T2

Abort

Conflict

Commit Commit

Read x

May 13th 2009

Throughput Throughput (SPECjAppServer)(SPECjAppServer)


36

Baselines

Our Protocol

May 13th 2009

Response Time: Read-only Response Time: Read-only TxnTxn


37May 13th 2009

Response Time: Update Response Time: Update TxnTxn


38May 13th 2009

OutlineOutline





39May 13th 2009

HA in SOA: MotivationHA in SOA: Motivation

• Some Web Services are critical for the interaction among organizations and should remain available despite failures

• WS-Replication Framework helps on replicating these critical Web Services


40May 13th 2009

HA in SOA: The WS-HA in SOA: The WS-Replication FrameworkReplication Framework

• WS-Replication is a framework that eases the replication of WSso SOAP-based web services

• Properties:o Respects WS autonomyo Provides transparent fault-tolerance

• Components:o Deployer toolo WS-Multicast serviceo WS-Dispatcher


41May 13th 2009

Background: Active Background: Active ReplicationReplication


42

ServiceReplica 1

ServiceReplica 2

ServiceReplica 3

m2 m2 m2m1 m1 m1

Client Client

m1 m2

May 13th 2009

WS-Replication: WS-Replication: Invoking a Replicated Invoking a Replicated

Service IService I


43

Replica 1

WS-Proxy

transport

WSWS-Dispatcher

WS Replica 2

WS-Proxy

transport

WS-Dispatcher

WS-Multicast

WS

WS-Multicast

WS

Client

May 13th 2009

WS-Replication: WS-Replication: Invoking a Replicated Invoking a Replicated

Service IIService II


44

Replica 1

WS-Proxy

transport

WSWS-Dispatcher

WS Replica 2

WS-Proxy

transport

WS-Dispatcher

WS-Multicast

WS

WS-Multicast

WS

Client

May 13th 2009

WS-Replication WS-Replication Evaluation: SetupEvaluation: Setup


45

Zurich

Bologna

Montreal

Madrid

Critical Service

Load Generator

May 13th 2009

WS-I & WS-CAF WS-I & WS-CAF IntegrationIntegration


46

WS-CAF

WS-Rep.

Zurich

WS-CAF

WS-Rep.

Bologna

WS-CAF

WS-Rep.

Montreal

DB

Madrid Host 1 Madrid Host 2

WS-IApplic.

ClientEmulator

May 13th 2009

WS-CAF ReplicationWS-CAF Replication


47

10%

May 13th 2009

WS-CAF ReplicationWS-CAF Replication


48

May 13th 2009

OutlineOutline• High Availability (HA) and Scalability in

Multi-Tier Architectureso Protocols for High Availability in MTAso A Protocol for HA and Scalability in MTAs




49May 13th 2009

ConclusionsConclusions• We have developed a set of replication and

recovery protocols for providing consistent high availability and scalability to multi-tier applications

• Main contributions:o Transaction-aware replicationo Exactly-once execution of client requestso Deal with several interaction patternso Scalability through a replicated SI cache in the app.

servero Online recovery (Not presented because the lack of

time)• Results show that the proposed protocols are

affordablePh.D. Thesis © Francisco Pérez-Sorrosal

50May 13th 2009

ConclusionsConclusions• We have also developed a framework to

provide high availability to SOAs• WS-Replication provides seamless

replication to critical WSs• Adequate engineering proved to provide

affordable performance• Evaluation of a realistic application in WANs

has shown a quite reasonable overhead


51May 13th 2009

OutlineOutline• High Availability (HA) and Scalability in

Multi-Tier Architectureso Protocols for High Availability in MTAso A Protocol for HA and Scalability in MTAs




52May 13th 2009

PublicationsPublications• Jorge Salas, Francisco Pérez-Sorrosal, Marta Patiño-Martínez and

Ricardo Jiménez-Peris. WS-Replication: a Framework for Highly Available Web Services. WWW, 2006.o Acceptance rate: 11 %o Percentile top 0 % in Microsoft’s Libra (WWW category)

• Francisco Pérez-Sorrosal, Marta Patiño-Martínez, Ricardo Jiménez-Peris and Bettina Kemme. Consistent and Scalable Cache Replication for Multi-tier J2EE Applications. Middleware, 2007. o Acceptance rate: 20 %o Percentile top 12 % in Microsoft’s Libra (Dist. And Parall. Computing

category)

• Francisco Pérez-Sorrosal, Marta Patiño-Martínez, Ricardo Jiménez-Peris and Jaksa Vuckovic. Highly Available Long Running Transactions and Activities for J2EE Applications. ICDCS, 2006.o Acceptance rate: 13 %o Percentile top 3 % in Microsoft’s Libra (Dist. And Parall. Computing category)


53May 13th 2009

Thank You!Thank You!

QUESTIONS?


54May 13th 2009

middleware for high availability and scalability in multi-tier and service-oriented architectures

Technology

ricardo jim

thesis francisco

proxy transport

multicast

single pointof

multicast

data services

web services