Decomposing applications for deployability and scalability

Chris Richardson

Author of POJOs in ActionFounder of the original CloudFoundry.com

@crichardsoncrichardson@vmware.comhttp://plainoldobjects.com/

1

Presentation goal

2

How decomposing applications improves

deployability and scalability

vmc push About-Chris

3

Developer Advocate for CloudFoundry.com

Signup at http://cloudfoundry.com cfopentour2012

4

Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?

Let’s imagine you are building an e-commerce application

5

Tomcat

Traditional web application architecture

6

Browser

WAR

MySQL Database

ShippingService

AccountingService

InventoryService

StoreFrontUI

developtestdeploy

Simple to

Apache

scale

But there are problems with a monolithic architecture

7

Users expect a rich, dynamic and interactive experience

8

Java Web Application

Browser

HTTP Request

HTML/Javascript

Old style UI architecture isn’t good enough

Real-time web ≅ NodeJS

Obstacle to frequent deployments§Need to redeploy everything to change one component§Interrupts long running background (e.g. Quartz) jobs§Increases risk of failure

Fear of change

§Updates will happen less often§e.g. Makes A/B testing UI really difficult

9

Overloads your IDE and container

10Slows down development

11

Shipping team

Accounting team

Engineering

Obstacle to scaling development

E-commerce application

12

WAR

Shipping

Accounting

InventoryService

StoreFrontUI

Shipping team

Accounting team

Inventory team

UI team

Obstacle to scaling development

13

Lots of coordination and communication required

Obstacle to scaling development

14

Requires long-term commitment to a technology

stack

15

Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?

16

The scale cube

17

X axis - horizontal duplication

Z axis

- data

parti

tionin

g

Y axis - functionaldecomposition

Scale

by sp

litting

simila

r

thing

s

Scale by splitting different things

Y-axis scaling - application level

18

WAR

ShippingService

AccountingService

InventoryService

StoreFrontUI

Y-axis scaling - application level

19

Store front web application

shipping web application

inventory web application

Apply X axis cloning and/or Z axis partitioning to each service

ShippingService

AccountingService

InventoryServiceStoreFrontUI

accounting web application

Partitioning strategies§Partition by verb, e.g. shipping service§Partition by noun, e.g. inventory service§Single Responsibility Principle§Unix utilities - do one focussed thing well

20

Something of an art

Real world examples

21

http://highscalability.com/amazon-architecture

Between 100-150 services are accessed to build a page.

http://techblog.netflix.com/

http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf

http://queue.acm.org/detail.cfm?id=1394128

There are drawbacks

22

Complexity

23

See Steve Yegge’s Google Platforms Rant re Amazon.com

Multiple databases =

Transaction management challenges

24

When to use it?

25

In the beginning: •You don’t need it •It will slow you down

Later on:•You need it•Refactoring is painful

But there are many benefits§Scales development: develop, deploy and scale each service independently§Update UI independently§Improves fault isolation§Eliminates long-term commitment to a single technology stack

26

Modular, polyglot, multi-framework applications

Two levels of architecture

27

System-level

ServicesInter-service glue: interfaces and communication mechanismsSlow changing

Service-level

Internal architecture of each serviceEach service could use a different technology stackPick the best tool for the jobRapidly evolving

If services are small...§Regularly rewrite using a better technology stack§Adapt system to changing requirements and better technology without a total rewrite§Pick the best developers rather than best <pick a language> developers ⇒ polyglot culture

28

Fred George “Developer Anarchy”

The human body as a system

29

50 to 70 billion of your cells die each day

30

Yet you (the system) remain you

31

Can we build software systems with these characteristics?

32

http://dreamsongs.com/Files/WhitherSoftware.pdf

http://dreamsongs.com/Files/DesignBeyondHumanAbilitiesSimp.pdf

33

Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?

Inter-service communication options§Synchronous HTTP ⇔ asynchronous AMQP

§Formats: JSON, XML, Protocol Buffers, Thrift, ...

§Even via the database

34

Asynchronous is preferred

JSON is fashionable but binary format is more efficient

StoreFrontUI

wgrus-store.war

AccountingService

wgrus-billing.war

InventoryService

wgrus-inventory.war

ShippingService

wgrus-shipping.war

MySQL

35

RabbitMQ(Message Broker)

Asynchronous message-based communication

Benefits§Decouples caller from server§Caller unaware of server’s coordinates (URL)§Message broker buffers message when server is down/slow

36

Drawbacks§Additional complexity of message broker §RPC using messaging is more complex

37

Writing code that calls services

38

The need for parallelism

39

Service A

Service B

Service C

Service D

b = serviceB()

c = serviceC()

d = serviceD(b, c)

Call in parallel

Java Futures are a great concurrency abstraction

40

http://en.wikipedia.org/wiki/Futures_and_promises

Akka’s composable futures are even better

41

Using Akka futures

42

def callB() : Future[...] = ...def callC() : Future[...] = ...def callD() : Future[...] = ...

val future = for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d

val result = Await.result(future, 1 second)

http://doc.akka.io/docs/akka/2.0.1/scala/futures.html

Two calls execute in parallel

And then invokes D

Get the result of D

Spring Integration§Implements EAI patterns§Provides the building blocks for a pipes and filters architecture§Enables development of application components that are•loosely coupled•insulated from messaging infrastructure

§Messaging defined declaratively

43

Handling failure

44

Service A Service B

Errors happen in distributed systems

About Netflix

45

> 1B API calls/day1 API call ⇒ average 6 service calls

Fault tolerance is essential

http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

Use timeouts and retries

46

Never wait forever

Errors can be transient ⇒ retry

http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

Service A

Service B

Use per-dependency bounded thread pool

47http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

Runnable 1

Runnable 2

Runnable ...

bounded queue

Task 1

Task 2

Task ...

bounded thread pool

Limits number of outstanding requests

Fails fast if service is slow or down

Use a circuit breakerHigh error rate ⇒ stop calling temporarily

Down ⇒ wait for it to come back up

Slow ⇒ gives it a chance to recover

48http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

On failureReturn cached data

Return default data

Fail fast

49http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

AvoidFailing

Summary

50

Monolithic applications are simple to develop and deploy

BUT have significant drawbacks

51

Apply the scale cube

52

X axis - horizontal duplicationZ ax

is - d

ata pa

rtitio

ning

Y axis - functionaldecomposition

§Modular, polyglot, and scalable applications§Services developed, deployed and scaled independently

Questions?

@crichardson crichardson@vmware.comhttp://slideshare.net/chris.e.richardson/

www.cloudfoundry.com promo code: cfopentour2012