© 2013 Spring, by Pivotal

Chris RichardsonAuthor of POJOs in ActionFounder of the original CloudFoundry.com

@crichardsonchris.richardson@springsource.comhttp://plainoldobjects.com

Decomposing applications for deployability and scalability

@crichardson

Presentation goalHow decomposing applications

improves deployability and scalability

and simplifies the adoption of new

technologies

@crichardson

About Chris

@crichardson

(About Chris)

@crichardson

About Chris()

@crichardson

About Chris

@crichardson

About Chris

http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/

@crichardson

vmc push About-Chris

Developer Advocate

@crichardson

Agenda

The (sometimes evil) monolith

Decomposing applications into services

Developing and deploying services

How do services communicate?

@crichardson

Let’s imagine you are building an e-commerce application

@crichardson

Tomcat

Traditional web application architecture

Browser

WAR

MySQL Database

ShippingService

AccountingService

InventoryService

StoreFrontUI

developtest

deploy

Simple to

Apache

scale

@crichardson

But there are problems with a monolithic architecture

@crichardson

Intimidates developers

@crichardson

Obstacle to frequent deployments

Need to redeploy everything to change one component

Interrupts long running background (e.g. Quartz) jobs

Increases risk of failure

Fear of change

Updates will happen less often

e.g. Makes A/B testing UI really difficult

@crichardson

Overloads your IDE and container

Slows down development

@crichardson

Shipping team

Accounting Engineering

Obstacle to scaling development

E-commerce application

@crichardson

WAR

Shipping

Accounting

InventoryService

StoreFront UI

Shipping team

Accounting team

Inventory team

UI Team

Obstacle to scaling development

@crichardson

Lots of coordination and communication required

Obstacle to scaling development

I want to update the UI

But the backend is not working

yet!

@crichardson

Requires long-term commitment to a technology stack

@crichardson

Agenda

The (sometimes evil) monolith

Decomposing applications into services

Developing and deploying services

How do services communicate?

@crichardson

@crichardson

The scale cube

X axis - horizontal duplication

Z axis

- data

partit

ioning

Y axis - functional

decomposition

Scale b

y split

ting s

imilar

thing

s

Scale by splitting

different things

@crichardson

Y-axis scaling - application level

WAR

ShippingService

AccountingService

InventoryService

StoreFrontUI

@crichardson

Y-axis scaling - application level

Store front web application

shipping web application

inventory web application

Apply X axis cloning and/or Z axis partitioning to each service

AccountingService

StoreFrontUI

accounting web application

ShippingService

InventoryService

@crichardson

Partitioning strategies...

Partition by verb, e.g. shipping service

Partition by noun, e.g. inventory service

Single Responsibility Principle

Unix utilities - do one focussed thing well

@crichardson

Partitioning strategies

Too few

Drawbacks of the monolithic architecture

Too many - a.k.a. Nano-service anti-pattern

Runtime overhead

Potential risk of excessive network hops

Potentially difficult to understand system

Something of an art

@crichardson

http://highscalability.com/amazon-architecture

http://techblog.netflix.com/

http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf

http://queue.acm.org/detail.cfm?id=1394128

Real world examples

@crichardson

There are drawbacks

@crichardson

Complexity

@crichardson

Multiple databases &

Transaction management

@crichardson

Implementing and deploying features that span multiple

services

@crichardson

When to use it?In the beginning: •You don’t need it •It will slow you down

Later on:•You need it•Refactoring is painful

@crichardson

But there are many benefitsScales development: develop, deploy and scale each service independently, e.g. update UI independently

Simplifies distributed development and outsourcing

Improves fault isolation

Eliminates long-term commitment to a single technology stack

Modular, polyglot, multi-framework applications

@crichardson

Two levels of architectureSystem-level

ServicesInter-service glue: interfaces and communication mechanisms

Slow changing

Service-level

Internal architecture of each serviceEach service could use a different technology stack

Pick the best tool for the jobRapidly evolving

@crichardson

If services are small...

Regularly rewrite using a better technology stack

Adapt system to changing requirements and better technology without a total rewrite

Pick the best developers rather than best <pick a language> developers ⇒ polyglot culture

@crichardson

The human body as a system

@crichardson

50 to 70 billion of your cells die each day

@crichardson

Yet you (the system) remain you

@crichardson

Can we build software systems with these characteristics?

http://dreamsongs.com/Files/WhitherSoftware.pdf

http://dreamsongs.com/Files/DesignBeyondHumanAbilitiesSimp.pdf

@crichardson

Agenda

The (sometimes evil) monolith

Decomposing applications into services

Developing and deploying services

How do services communicate?

@crichardson

Services come in all shapes and sizes

@crichardson

Example service: Spring MVC@Controllerclass TwilioController { @Autowired var surveyManagementService: SurveyManagementService = _

@RequestMapping(value = Array("/begincall.html")) @ResponseBody def beginCall(@RequestParam("From") callerId: String) = { surveyManagementService.findSurveyByCallerId(callerId) match { ... case Some(survey) => <Response> <Say>{ survey.prompt }</Say> <Gather action="handleresponse.html" method="POST" numDigits="1"> { for ((choice, index) <- survey.choices zipWithIndex) yield <Say>Press { index } for { choice }</Say> } </Gather> <Say>We are sorry you could not decide</Say> <Hangup/> </Response> } }

@crichardson

Example: standalone app

main()

Spring Integration

@crichardson

Example service: NodeJSvar express = require('express') , http = require('http') , amqp = require(‘amqp’) ....;

server.listen(8081);...var amqpCon = amqp.createConnection(...);

io.sockets.on('connection', function (socket) { function amqpMessageHandler(message, headers, deliveryInfo) { var m = JSON.parse(message.data.toString()); socket.emit(‘tick’, m); }; amqpCon.queue(“”, {}, function(queue) { queue.bind(“myExchange”, “”); queue.subscribe(amqpMessageHandler); });});

Simple portable way to deliver AMQP messages

to the browser

@crichardson

Example service: Sinatra

require 'sinatra'

post '/' do phone_number = params[:From] registration_url = "#{ENV['REGISTRATION_URL']}?phoneNumber=#{URI.encode(phone_number, "+")}" <<-eof <Response> <Sms>To complete registration please go to #{registration_url}</Sms> </Response> eofend

@crichardson

Service deployment options

VM or Physical Machine

Linux Container/LXC

JVM

JAR/WAR/OSGI bundle/...

Isolation, manageability

Density/efficiency

You could do this yourself but ...

@crichardson

PaaS dramatically simplifies deployment

Applica'on  Service  Interface

OSS community

Private  Clouds  

PublicClouds

MicroClouds

Data Services

Other Services

Msg Services

vFabric Postgres

vFabric RabbitMQTM

Additional partners services …

@crichardson

Cloud Foundry features

One step deployment: vmc push

Single application

Service-oriented application

Easy platform service provisioning: vmc create-service

Simple scaling: vmc instances app-name +/- N

Health monitoring and automated recovery

@crichardson

Benefits of PaaSSimplifies and automates deployment

Eliminates barriers to adding new service

Eliminates barriers to using a new platform service

Imposes conventions: packaging, configuration and deployment

Enforces consistency

Eliminates accidental complexity

@crichardson

Agenda

The (sometimes evil) monolith

Decomposing applications into services

Developing and deploying services

How do services communicate?

@crichardson

Inter-service communication options

Synchronous HTTP ⇔ asynchronous AMQP

Formats: JSON, XML, Protocol Buffers, Thrift, ...

Asynchronous is preferredJSON is fashionable but binary format

is more efficient

@crichardson

StoreFrontUI

wgrus-store.war

AccountingService

wgrus-billing.war

InventoryService

wgrus-inventory.war

ShippingService

wgrus-shipping.war

MySQL

RabbitMQ(Message Broker)

Asynchronous message-based communication

@crichardson

Benefits

Decouples client from server: client unaware of server’s coordinates (URL)

Message broker buffers message when server is down/slow

Supports a variety of communication patterns, e.g. point-to-point, pub-sub, ...

@crichardson

Drawbacks

Additional complexity of message broker

Request/reply-style communication is more complex

@crichardson

Spring IntegrationProvides the building blocks for a pipes and filters architecture

Enables development of application components that are

loosely coupled

insulated from messaging infrastructure

Messaging defined declaratively

@crichardson

Synchronous REST

ShippingService

StoreFrontUI

wgrus-store.war

AccountingService

wgrus-billing.war

wgrus-shipping.war

InventoryService

wgrus-inventory.war

MySQL

REST

...

Pros and cons of RESTPros

Simple and familiar

Request/reply is easy

Browser and firewall friendly

No intermediate broker

Cons

Only supports request/reply

Server must be available

Client needs to know URL(s) of server(s)

@crichardson

Spring MVC makes REST easy@Controllerpublic class AccountController {

@Autowired private MoneyTransferService moneyTransferService; @RequestMapping(value = "/accounts/{accountId}", method = RequestMethod.GET) @ResponseBody public AccountInfo getAccount(@PathVariable String accountId) { Account account = moneyTransferService.findAccountByid(accountId); return makeAccountInfo(account); }

@RequestMapping(value = "/accounts", method = RequestMethod.POST) @ResponseStatus( HttpStatus.CREATED ) public void createAccount(@RequestBody AccountInfo accountInfo, UriComponentsBuilder builder, HttpServletResponse response) { ... }

URL matching &

destructuringobject ⇒

XML/JSON

XML/JSON ⇒

object

@crichardson

About Hypertext As The Engine Of Application State (HATEOAS)

$ curl http://cf-auto-scaler.cloudfoundry.com{"links": [ {"rel":"autoscaledapps", "href":"http://cf-auto-scaler.cloudfoundry.com/autoscaledapps"}]}

The well known URL

Linklink type

$ curl http://cf-auto-scaler.cloudfoundry.com/autoscaledapps{"content":[ {"name":"vertx-clock", "links":[ {"rel":"self","href":"http://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock"}, {"rel":"rules","href":"http://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock/rules"} ] }],...}

Links to act on this app

@crichardson

Spring HATEOAS@Controller@RequestMapping(value = "/autoscaledapps")public class AutoscaledAppController {

@RequestMapping(value = "/{appName}", method = RequestMethod.GET) public HttpEntity<AutoscaledAppResource> get(@PathVariable String appName) { AutoscaledAppResource ar = new AutoscaledAppResource(appName); ar.add(linkTo(AutoscaledAppController.class)

.slash(appName).withSelfRel()); ar.add(linkTo(AutoscaledAppController.class)

.slash(appName).slash("rules").withRel("rules")); return new HttpEntity<AutoscaledAppResource>(ar); } ...}

https://github.com/SpringSource/spring-hateoas

public class AutoscaledAppResource extends ResourceSupport {

private String name;

@crichardson

Consuming RESTful WS

RestTemplate restTemplate = new RestTemplate();

AccountInfo accountInfo = new AccountInfo(...);URI accountUrl = restTemplate.postForLocation("http://localhost/accounts", accountInfo);

ResponseEntity<AccountInfo> accountInfoResponse = restTemplate.getForEntity(accountUrl, AccountInfo.class);

Assert.assertEquals(HttpStatus.SC_OK, accountInfoResponse.getStatusCode());AccountInfo accountInfo2 = accountInfoResponse.getBody();...

@crichardson

The Spring REST shell$ rest-shellhttp://localhost:8080:> baseUri http://cf-auto-scaler.cloudfoundry.comhttp://cf-auto-scaler.cloudfoundry.com:> discoverrel href =======================================================================autoscaledapps http://cf-auto-scaler.cloudfoundry.com/autoscaledapps

http://cf-auto-scaler.cloudfoundry.com:> follow --rel autoscaledapps

http://cf-auto-scaler.cloudfoundry.com/autoscaledapps:> post --from src/test/resources/examplejson/createapp1.json --follow true

1 files uploaded to the server using POSThttp://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock:> discoverrel href ================================================================================self http://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock rules http://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock/ruleshttp://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock:> follow --rel rules http://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock/rules:> post

--from src/test/resources/examplejson/createrule1.json --follow true1 files uploaded to the server using POSThttp://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock/rules/idle:> uphttp://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock/rules:> up

@crichardson

Writing code that calls services

@crichardson

The need for parallelism

ProductDetails

Controller

Product Details

Recommendations

Reviews

getProductDetails()

getRecomendations()

getReviews()

Call in parallel

DisplayProduct

@crichardson

Futures are a great concurrency abstraction

An object that will contain the result of a concurrent computation http://en.wikipedia.org/wiki/Futures_and_promises

Hides the underlying concurrency mechanism: threads or async

Future<Integer> result = executorService.submit(new Callable<Integer>() {... });

Java has basic futures. We can do much better....

@crichardson

Better: Futures with callbacks

val f : Future[Int] = Future { ... } f onSuccess { case x : Int => println(x) } f onFailure { case e : Exception => println("exception thrown") }

Guava ListenableFutures, Java 8 CompletableFuture, Scala Futures

@crichardson

Even better: Composable Futuresval f1 = Future { ... ; 1 }val f2 = Future { ... ; 2 }

val f4 = f2.map(_ * 2)assertEquals(4, Await.result(f4, 1 second))

val fzip = f1 zip f2assertEquals((1, 2), Await.result(fzip, 1 second))

def asyncOp(x : Int) = Future { x * x}val f = Future.sequence((1 to 5).map { x => asyncOp(x) })assertEquals(List(1, 4, 9, 16, 25), Await.result(f, 1 second))

Scala Futures

Transforms Future

Combines two futures

Transforms list of futures to a future containing a list

@crichardson

Using Scala futuresdef callB() : Future[...] = ...def callC() : Future[...] = ...def callD() : Future[...] = ...

val future = for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d

val result = Await.result(future, 1 second)

Two calls execute in parallel

And then invokes D

Get the result of DScala Futures

@crichardson

Handling partial failures

Service A Service B

Down?Slow?

Down?Slow?

@crichardson

About Netflix> 1B API calls/day

1 API call ⇒ average 6 service calls

Fault tolerance is essential

http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

@crichardson

How to run out of threads

Tomcat

Execute thread pool

HTTP Request

Thread 1

Thread 2

Thread 3

Thread n

Service A Service B

If service B is down then thread

will be blocked

XXXXX

Eventually all threads will be blocked

@crichardson

Their approachNetwork timeouts and retries

Invoke remote services via a bounded thread pool

Use the Circuit Breaker pattern

On failure:

return default/cached data

return error to caller

https://github.com/Netflix/Hystrix

@crichardson

Summary

@crichardson

Monolithic applications are simple to develop and deploy

BUT have significant drawbacks

@crichardson

Apply the scale cube

Modular, polyglot, and scalable applications

Services developed, deployed and scaled independently

@crichardson

Cloud  Provide

r  Interface

Applica'on  Service  Interface

Private  Clouds  

PublicClouds

MicroClouds

Data Services

Other Services

Msg Services

.js

Cloud Foundry helps

@crichardson

Questions?

@crichardson chris.richardson@springsource.comhttp://plainoldobjects.com - code and slides

www.springsource.org

78

Learn More. Stay Connected.

Download now: • springsource.org/spring-tool-suite-download (eclipse plugin)• springsource.org/download/community (zip)• springsource.org/spring-framework#maven (maven)

Monthly Newsletter: springsource.org/news-eventsTwitter: twitter.com/springsourceYouTube: youtube.com/user/SpringSourceDevRSS Feed: feeds.feedburner.com/springsource/OEVELinkedIn: springsource.org/linkedin