decomposing applications for deployability and scalability (cf india july/august 2012)
DESCRIPTION
Latest version of the decomposing applications for scalability and deployability talk. From Cloud Foundry July/August India tour.TRANSCRIPT
Decomposing applications for deployability and scalability
Chris Richardson
Author of POJOs in ActionFounder of the original CloudFoundry.com
@[email protected]://plainoldobjects.com/
1
Presentation goal
2
How decomposing applications improves
deployability and scalability and
How Cloud Foundry helps
About Chris
3
dsply ‘About Chris’;
4
RPG
(About Chris)
5
About Chris()
6
About Chris
7
About Chris
http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/
8
vmc push About-Chris
9
Developer Advocate for CloudFoundry.com
Signup at http://cloudfoundry.com
10
Agenda The (sometimes evil) monolith Decomposing applications into services How do services communicate? Presentation layer design How Cloud Foundry helps
Let’s imagine you are building an e-commerce application
11
Tomcat
Traditional web application architecture
12
Browser
WAR
MySQL Database
ShippingService
AccountingService
InventoryService
StoreFrontUI
developtestdeployscale
Simple to
Apache
But there are problems
13
Users expect a rich, dynamic and interactive experience on mobile devices and desktop
14
Java Web Application
Browser
HTTP Request
HTML/Javascript
Old style UI architecture isn’t good enough
Real-time web ≅ NodeJS
Obstacle to frequent deployments Need to redeploy everything to change one component Interrupts long running background (e.g. Quartz) jobs Increases risk of failure
Fear of change
Updates will happen less often e.g. Makes A/B testing UI really difficult 15
Overloads your IDE and container
16Slows down development
Obstacle to scaling development
17
!=Scalable development
WAR
Shipping
Accounting
InventoryService
StoreFrontUI
Forces teams to synchronize development efforts Teams need to coordinate updates
Requires long-term commitment to a technology stack
Switching technology stack ⇒ touches entire application Painful and rarely done Makes it difficult to incrementally adopt newer technologies You end up being stuck with the technology you started with
18
19
Agenda The (sometimes evil) monolith Decomposing applications into services How do services communicate? Presentation layer design How Cloud Foundry helps
3 dimensions to scaling
20
X axis - horizontal duplicationZ ax
is - d
ata pa
rtitio
ning
Y axis - functionaldecomposition
Scale by cloning
Scale
by sp
litting
simila
r thin
gsScale by splitting different things
Y axis scaling - functional partitioning Splits monolithic application into a set of services Each service implements a set of related functionality Partitioning schemes:•Partition functionality by noun or by verb•Single Responsibility Principle•Unix utilities - do one focussed thing well
21
Something of an art
Y axis scaling - application level
22
WAR
ShippingService
AccountingService
InventoryService
StoreFrontUI
Y axis scaling - application level
23
Store front web application
shipping web application
inventory web application
Apply X axis cloning and/or Z axis partitioning to each service
ShippingService
AccountingService
InventoryServiceStoreFrontUI
billing web application
Real world examples
24
http://highscalability.com/amazon-architecture
Between 100-150 services are accessed to build a page.
http://techblog.netflix.com/
http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
There are drawbacks Complexity: development, deployment, etc. Multiple databases ⇒ transaction management challenges Deciding when to use it• In the beginning: you don’t need it and it will slow you down•When you do need it: refactoring existing code is painful
25
See Steve Yegge’s Google Platforms Rant re Amazon.com
But there are many benefits Scales development: develop, deploy and scale each service
independently Improves fault isolation Enforces well defined interfaces between components Eliminates long-term commitment to a single technology
stack
26
Modular, polyglot, multi-framework applications
Two levels of architecture System level•Defines the inter-service glue: interfaces and
communication mechanisms•Slow changing
Service level•Defines the internal architecture of each service•Far fewer constraints on technology•Each service could use a different technology stack•Pick the best tool for the job•Rapid evolving
27
Versus
If services are small... Regularly rewrite using a better technology stack Adapt system to changing requirements and better
technology without a total rewrite Pick the best developers rather than best <pick a
language> developers ⇒ polyglot culture
28
Fred George “Developer Anarchy”
The human body as a system
29
50 to 70 billion of your cells die each day
30
Yet you (the system) remain you
31
Can we build software systems with these characteristics?
32
http://dreamsongs.com/Files/WhitherSoftware.pdf
http://dreamsongs.com/Files/DesignBeyondHumanAbilitiesSimp.pdf
33
Agenda The (sometimes evil) monolith Decomposing applications into services How do services communicate? Presentation layer design How Cloud Foundry helps
Inter-service communication options Multiple collaborating services ⇒ need a communication
mechanism Many choices:•Synchronous ⇔ asynchronous•Transports: HTTP, AMQP, ...•Formats: JSON, XML, Protocol Buffers, Thrift, ...•Even via the database
Distributed application ⇒ error handling strategies
34
Asynchronous is preferred
JSON is fashionable but binary format is more efficient
StoreFrontUI
wgrus-store.war
AccountingService
wgrus-billing.war
InventoryService
wgrus-inventory.war
ShippingService
wgrus-shipping.war
MySQL
35
RabbitMQ(Message Broker)
Asynchronous message-based communication
Benefits and drawbacks Benefits•Decouples caller from server•Caller unaware of server’s coordinates (URL)•Message broker buffers message when server is down/slow
Drawbacks•Additional complexity of message broker •RPC using messaging is more complex
36
Writing code that calls services
37
Composable futures Problem:•Service A needs to call services B and C and then D•Makes sense to call B and C parallel•Yet most concurrency APIs are low-level, error-prone etc
Solution:•Use Akka composable futures = really nice abstraction
38
val futureB = callB()val futureC = callC()
val futureD = for { b <- futureB.mapTo[SomeType] c <- futureC.mapTo[SomeType] d <- callD(b, c) } yield d
val result = Await.result(futureD, 1 second). asInstanceOf[SomeOtherType]
http://doc.akka.io/docs/akka/2.0.1/scala/futures.htmlhttp://en.wikipedia.org/wiki/Futures_and_promises
Two calls execute in parallel
And then invokes D
Get the result of D
Spring Integration
Builds on Spring framework
Implements EAI patterns = high-level of abstraction for building message based applications
Provides the building blocks for a pipes and filters architecture
• Pipes = message channels
• Filters = endpoints that filter, transform, route messages and integrate with external messaging infrastructure
Enables development of components that are
• loosely coupled
• insulated from underlying messaging infrastructure
39
Handling failure
40
Service A Service B
Errors happen in distributed systems
Use timeouts and retries
Never wait foreverErrors can be transient ⇒ retry
41http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
Service A
Service B
Use per-dependency bounded thread pool
42http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
Runnable 1
Runnable 2
Runnable
bounded queue
Task 1
Task 2
Task ...
bounded thread pool
Limits number of outstanding requests
Fails fast if service is slow or down
Use a circuit breakerHigh error rate ⇒ stop calling temporarily
Avoids calling service that has issues
Down ⇒ wait for it to come back up
Slow ⇒ gives it a chance to recover
43http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
On failure - don’t fail tooReturn cached dataReturn default data
Invoke custom error handler
44http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
45
Agenda The (sometimes evil) monolith Decomposing applications into services How do services communicate? Presentation layer design How Cloud Foundry helps
Modular application
Choice of presentation layer technology+
Redeploy UI frequently/independently
46
NodeJS is the fashionable technology
47
Many JavaScript client frameworks have a NodeJS counterparte.g. socket.io
NodeJS example
48
var http = require('http');var fs = require("fs");
http.createServer(function (req, res) { fs.readFile('somefile.txt', function (err, data) { if (err) throw err; res.writeHead(200, {'Content-Type': 'text/plain'}); res.end(data); });}).listen(1337, '127.0.0.1');
console.log('Server running at http://127.0.0.1:1337/');
Handle HTTP
request Handle file read
NodeJS isn’t the only game in town
49
JVM-based http://vertx.io/
A modern web application
50
Browser
Service 1
Service 2
...
HTML 5Application
Socket.ioclient
Events
RESTful WS
Server Application
Socket.ioserver
Node JS
NodeJS - using RESTful WS and AMQP
51
Node JS
Service
RabbitMQ Service
REST
AMQP AMQP
RESTRequests
Eventssocket.io
Updating the UI is easy
52
Update the UI independently of rest of system Easily run A/B tests Enables fast iteration of the UI
http://theleanstartup.com/principles
53
Agenda The (sometimes evil) monolith Decomposing applications into services How do services communicate? Presentation layer design How Cloud Foundry helps
Traditional tools: monolithic applications
54
Developing modular apps is more difficult Many more moving parts to manage•Platform services: SQL, NoSQL, RabbitMQ•Application services: your code
Who is going to setup the environments:• the developer sandbox?• ...•QA environments?
55
But Cloud Foundry helps...
Applica'on Service Interface
Data Services
Other Services
Msg Services
Easy polyglot application deployment and service provisioning
vFabric Postgres
vFabric RabbitMQTM
Additional partners services …
OSS community
Private Clouds
PublicClouds
MicroClouds
Creating a platform service instance$ vmc create-service mysql --name mysql1Creating Service: OK
$ vmc services......
=========== Provisioned Services ============
+-------------+---------+| Name | Service |+-------------+---------+| mysql1 | mysql |+-------------+---------+
Multi-application manifest - part 1--- applications: inventory/target: name: inventory url: cer-inventory.chrisr.cloudfoundry.me framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: si-rabbit: type: :rabbitmq si-mongo: type: :mongodb si-redis: type: :redis
58
Path to application
Required platform services
Multi-application manifest - part 2 store/target: name: store url: cer-store.chrisr.cloudfoundry.me framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: si-mongo: type: :mongodb si-rabbit: type: :rabbitmq
59
Path to application
Required platform services
One command to create platform services and deploy application
$ vmc push Would you like to deploy from the current directory? [Yn]: Pushing application 'inventory'...Creating Application: OKCreating Service [si-rabbit]: OKBinding Service [si-rabbit]: OKCreating Service [si-mongo]: OKBinding Service [si-mongo]: OKCreating Service [si-redis]: OKBinding Service [si-redis]: OKUploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK Uploading (12K): OK Push Status: OKStaging Application 'inventory': OK Starting Application 'inventory': OK Pushing application 'store'...Creating Application: OKBinding Service [si-mongo]: OKBinding Service [si-rabbit]: OKUploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK Uploading (5K): OK Push Status: OKStaging Application 'store': OK Starting Application 'store': ... 60
vmc push:•Reads the manifest file•Creates the required platform services•Deploys all the applications
Micro Cloud Foundry: new developer sandbox
61
Open source Platform as a Service project
App Instances Services
10.04
A PaaS packaged as a VMware Virtual Machine
Use as a developer sandbox
• Use the services from Junit integration tests
• Deploy your application for functional testing
• Remote debugging from STS
Using Caldecott to tunnel into your services
62
Caldecott = TCP over HTTP
63
Your computer
Caldecott gem
Cloud Foundry
Caldecott application
ServiceHTTP native
protocol
Port NNN
Service client
nativeprotocol
Using Caldecott…$ vmc tunnel1: mysql-135e02: mysql1Which service to tunnel to?: 2Password: ********Stopping Application: OKRedeploying tunnel application 'caldecott'.Uploading Application: Checking for available resources: OK Packing application: OK Uploading (1K): OK Push Status: OKBinding Service [mysql1]: OKStaging Application: OK Starting Application: OK Getting tunnel connection info: OK
Service connection info: username : uMe6Apgw00AhS password : pKcD76PcZR7GZ name : d7cb8afb52f084f3d9bdc269e7d99ab50
Starting tunnel to mysql1 on port 10000.1: none2: mysqlWhich client would you like to start?: 2
…Using CaldecottLaunching 'mysql --protocol=TCP --host=localhost --port=10000 --
user=uMe6Apgw00AhS --password=pKcD76PcZR7GZ d7cb8afb52f084f3d9bdc269e7d99ab50'
Welcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 10944342Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL),
Release 12.5, Revision 188
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
Running JUnit test with Caldecott
66
Configure your test code to use port + connection info
Summary Monolithic applications are simple to develop and deploy
BUT have significant drawbacks
Applying the scale cube
Decomposes your application into services Enables scaling for transactions and data volumes Tackles application complexity Enables scaling for development Enables frequent, independent deployments Make it easy to leverage other technologies
AND
Cloud Foundry simplifies the development and deployment of modular, service-based applications 67