real life use of spring integration with rabbitmq

© 2013 SpringOne 2GX. All rights reserved. Do not distribute without permission.

Real life use of Spring Integration with RabbitMQ

By Durai Arasan,

Director of Architecture, ETRADE Financial

Monday, September 9, 13

About Me

2

Enterprise Java Technology Leader

Monday, September 9, 13

Agenda

§Background§Things that matter§Our ways to solve§Demo§Q&A

Monday, September 9, 13

Once upon a time ...

4Monday, September 9, 13

In the beginning ...

• Early adopters of services• Large set of services

–Anything and everything is a service–Many granular services

• Payload formats–Mixed set of formats–Proprietary

5Monday, September 9, 13

Things that matter

6

Top requirements for our need

Monday, September 9, 13

Things that matter

§Performance–Slow performance makes users leave–Instant response is expected

LOADING ...

Monday, September 9, 13

Things that matter

§Burst Load–Flood gates open–Zero to Sixty in no time

Source: mashable.com

Monday, September 9, 13

Things that matter

§Service Chains–Granular services end up in service chains–Parallel processing of services–Need to meet SLA–Keeping it in same context

Monday, September 9, 13

Things that matter

§Simple Services–Keep services strictly to do business logic–POJOs driven–Container to do the heavy lifting

• Routing, Filtering, Transforming, Splitting and aggregating handlers and many more

Client Service

Request

Response

Monday, September 9, 13

Things that matter

§Separation of concerns–Request Isolation

• Bottleneck with all requests in one queue• Priority queues

–Process Isolation• Impacting other services

Monday, September 9, 13

Things that matter

§Services in new Languages–New languages are adopted widely–Gone are the days of few languages–Choose a language specific to a task

•C, Java, Python, Ruby, Node.js and Perl

Monday, September 9, 13

Summary: Things that matter

ServicesPlatform

Web, Mobile

Integration,Batch

Meets the demand for new

& old

Lean& Fast

13Monday, September 9, 13

How did we solve?

14Monday, September 9, 13

Enterprise Integration Pattern• Messaging is core to solve integration• Blueprint for solving

–Payload Transformation–Protocol Converters –Filters–Aggregation (scatter gather)–Claim check pattern–and others ...

15Monday, September 9, 13

Goal: Lean Services Platform• Build with Messaging• Assemble components that work best• Meet all things matter• Build to support

–Legacy Services–New Services

16Monday, September 9, 13

RabbitMQ• Popular AMQP implementation• High Performance• Robust Messaging• Large number of connections • Runs on many operating systems• Clients for many languages• Open Source Support• AMQP Extensions (DLX, AE etc)

17Monday, September 9, 13

Spring Integration• Based on EIP: Lightweight ESB & Service Host• Out of the box support for RabbitMQ• POJOs driven services• Gateways

–Configuration driven–Extendable for other impls

• Handlers for all use cases• Built-in JMX support

18Monday, September 9, 13

Architecture

19Monday, September 9, 13

Integrating Web Services

20Monday, September 9, 13

Spring Integration + Web Services• Used as an IDL rather than full-fledged container• Web Services Endpoints

–jax-ws–jax-rs

• No need to re-invent• Support for multiple transport• Messages are routed from broker to SI

21Monday, September 9, 13

Example JAX-WS Service Interface

@WebService(name="Payment")public interface PaymentService

{   @WebResult(partName="AccountRequest") @WebMethod(operationName="getAccount")

public AccountRequest getAccount(@WebParam(partName="AccountId")int id) throws AccountRequestException;

}

22Monday, September 9, 13

Example JAX-RS Service Interface@WebService(name="REST/Payment")public interface PaymentService { @GET @Path("/account/{id}") @WebMethod(operationName="account") public AccountRequest getAccount (@PathParam("id") int id);

@POST @Path("/pay/{accountId}") @Consumes({"application/json", "text/xml"}) @WebMethod(operationName="payWithId") public PaymentResultResponse payWithAccountId (@PathParam("accountId") int accountId,

ItemRequest paramItemRequest);}

23Monday, September 9, 13

Why DSL?• Custom name space support in SI• Simple things simple

–Avoid repetitive patterns –Enables standard patterns

• Complex things possible–Add additional on demand

24Monday, September 9, 13

Example w/o DSL<!-- INBOUND GATEWAY -->

<int-amqp:inbound-gateway request-channel="echoInputChannel" queue-names="svc_echoQueue" concurrent-consumers="10" connection-factory="rabbitConnection"/>

<!-- UNMARSHALLER --><!-- the handler that processes the request from the inbound gateway --><int:chain id="unmarshaller" input-channel="echoInputChannel" output-channel="echoServicesChannel" ><int-xml:unmarshalling-transformer unmarshaller="jaxb" /><int:service-activator expression="payload.getValue()"></int:service-activator></int:chain>

<!-- ROUTER --><!-- a router based on payload that dispatches the requests on it's input channel to the appropriate service activator --><int:payload-type-router input-channel="echoServicesChannel"> <!-- based on the unmarshalled object type route to different channels --> <int:mapping type="com.etrade.schemas.etsvc.EchoSleepT" channel="echoSleepChannel" /> <int:mapping type="com.etrade.schemas.etsvc.EchoIncrementT" channel="echoIncrementChannel" /> <int:mapping type="com.etrade.schemas.etsvc.EchoPayloadT" channel="echoPayloadChannel" /></int:payload-type-router>

<!-- REQ/REPLY SERVICES --><!-- messaging endpoints that are request and reply producing, and put all their outputs to the output channel that is then received by the marshaller--><int:service-activator id="echoSleepService" input-channel="echoSleepChannel" ref="echoImpl" method="sleep" output-channel="echoOutChannel" /><int:service-activator id="echoIncrementService" input-channel="echoIncrementChannel" ref="echoImpl" method="increment" output-channel="echoOutChannel"/><int:service-activator id="echoPayloadService" input-channel="echoPayloadChannel" ref="echoImpl" method="echoPayload" output-channel="echoOutChannel"/>

<!-- MARSHALLER --><!-- the handler that processes the response from the services, the anonymous replyChannel of the outbound gateway creates the bridge --><int:chain input-channel="echoOutChannel"> <int-xml:marshalling-transformer marshaller="jaxb" id="marshaller" result-type="StringResult"/> <int:object-to-string-transformer /></int:chain>

<!-- BEAN DECLARATION --><bean id="echoImpl" class="com.etrade.service.echo.impl.EchoServiceImpl"/>

<!-- ADMINISTRATION, RABBIT CONFIGS --><rabbit:admin connection-factory="rabbitConnection"/><rabbit:connection-factory id="rabbitConnection" /><rabbit:template id="amqpTemplate" connection-factory="rabbitConnection" /><!-- sets up the exchange --><rabbit:queue name="svc_echoQueue" auto-delete="false" durable="true" exclusive="false"></rabbit:queue><rabbit:direct-exchange name="etsvc_exchange"> <rabbit:bindings> <rabbit:binding queue="svc_echoQueue" key="svc_echoSleep"/> <rabbit:binding queue="svc_echoQueue" key="svc_echoPayLoad"/> <rabbit:binding queue="svc_echoQueue" key="svc_echoIncrement"/> </rabbit:bindings></rabbit:direct-exchange>

25Monday, September 9, 13

Example w/ DSL<!-- transformers, routers, channels, service-activators are now assimilated!! :) -->

<beans> <!-- gateway declaration --> <etrade:inbound-gateway service-interface="com.etrade.service.echo.Echo" ref="echoService" concurrent-consumers="5"/> <!-- bean declaration --> <bean id="echoService" class="com.etrade.service.echo.impl.EchoServiceImpl" /></beans>

26Monday, September 9, 13

Use of standard WSDL Tools • Wsdl2java to produce java interfaces • Java2wsdl to produce wsdl• Bindings for wsdl (jaxb, et al)• Transport in wsdl (http, amqp)• Use of standard wsdl extensions

27Monday, September 9, 13

Payloads over AMQP• Standard service development

–Using well known annotations (jax-ws & jax-rs)–Deployed under AMQP container

• JAX-RS–All media types (text/xml, application/octet-stream etc)

• JAX-WS–SOAP–XML/POX

28Monday, September 9, 13

SI + Web Services Flow

29Monday, September 9, 13

Demo

30

Generic Sample Service

Monday, September 9, 13

Next Steps

31

Spring IO:

Monday, September 9, 13

Summary• This model works for us• Performance meets our needs• Async Services help manage load• Decoupling of clients & endpoints• Platform that can run legacy and new services• Heterogeneous deployments in to single container

32Monday, September 9, 13

Learn More. Stay Connected.

We are Hiring!career.etrade.com

Twitter: twitter.com/duraiarasanLinkedIn: linkedin.com/in/duraiarasan Facebook: facebook.com/duraiarasan

Monday, September 9, 13

Q&A

Monday, September 9, 13