From concept to cloud: A look at modern software

developmentChris Richardson

Author of POJOs in ActionFounder of the original CloudFoundry.com

crichardson@vmware.com@crichardson

plainoldobjects.com

From concept to cloud: A look at modern software

development

Chris RichardsonAuthor of POJOs in Action,

Founder of the original CloudFoundry.com crichardson@vmware.com, @crichardson

plainoldobjects.com

About Chris

(About Chris)

About Chris()

About Chris

About Chris

http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/

vmc push About-Chris

Developer Advocate for

Signup at http://cloudfoundry.com

Promo code: sgce2012

Dreams of my childhood

Published in 1968

The vision

HAL 9000

.... is an artificial intelligence

became operational on

12 January 1992.....

So how are we doing?

The reality

Floor cleaning robot

The reality

http://en.wikipedia.org/wiki/IBM_Watson cluster of ninety IBM Power 750 servers with a total of 2880 POWER7 processor

cores and 16 Terabytes of RAM.

We still have a long way to go before we can

build HAL

But despite that....

Today, a small team of people can easily build an application that’s used by millions of people world-wide

http://highscalability.com/blog/2012/5/7/startups-are-creating-a-new-system-of-the-world-for-it.html

Endpoints everywhere

The web

Cloud Computing

http://en.wikipedia.org/wiki/Cloud_computing

How did we get here?

Many of the fundamentals of computing were

invented before many of us were born

Computer hardware

1948 - First computer

http://en.wikipedia.org/wiki/ENIAC

1940

1950

1960

1970

1980

1990

2000

201030 tons167 m2

150 KW

40 divs/second

1956 - Hard drive

http://boingboing.net/2010/06/24/ibm-hard-disk-drive.html

3.7 Mbit

Weighed over a ton

Designed to fit through a door

1971 - microprocessor

http://en.wikipedia.org/wiki/Microprocessor

4 bit

108-740 kHz

2,300 transistors

The power of Moore’s law

http://www.gotw.ca/publications/concurrency-ddj.htm

Hard disk capacity

In my career1982 RM 380Z4 Mhz 8-bit Z8032K RAM100K floppy

2012MacBookPro2.5 Ghz Quad core 64 bit Intel i7256K/core + 8M cache,16G RAM512G SSD drive

Machrone’s law

“...the price of the computer you want to buy

will always be $5000*...”

* Today it’s $3000

Wirth’s law

“Software is getting slower more rapidly than hardware becomes

faster.”

Programming languages

1958 - Lisp

http://en.wikipedia.org/wiki/Lisp_(programming_language)

1940

1950

1960

1970

1980

1990

2000

2010

garbage collection dynamic typing

self-hosting compiler tree data structures

(defun factorial (n) (if (<= n 1) 1 (* n (factorial (- n 1)))))

1960 - Algol 60

http://en.wikipedia.org/wiki/Algol_60

1940

1950

1960

1970

1980

1990

2000

2010 Many languages

including Java are derived from Algol

procedure Absmax(a) Size:(n, m) Result:(y) Subscripts:(i, k); value n, m; array a; integer n, m, i, k; real y;begin integer p, q; y := 0; i := k := 1; for p:=1 step 1 until n do for q:=1 step 1 until m do if abs(a[p, q]) > y then begin y := abs(a[p, q]); i := p; k := q endend Absmax

1967 - Simula

http://en.wikipedia.org/wiki/Simula

1940

1950

1960

1970

1980

1990

2000

2010

class-based object-oriented

programming

So what have programming language designers been

doing since then?

1995 - Java

Not innovative BUT

Brought garbage collection, object-oriented programming, exception handling, safety ... to mainstream

developers

Gosling “Java is a blue collar language”

google “Gosling the feel of Java”

1940

1950

1960

1970

1980

1990

2000

2010

Tools and techniques

1980s - Cool IDEs for Lisp and Smalltalk

Class hierarchy

Editor

Repl

Debugger

ClassList

1940

1950

1960

1970

1980

1990

2000

2010

1990s• CVS - Lockless version management

• DVCS - distributed version control

• Refactoring

• Agile/XP starts to become popular

• Mosaic web browser

• Web crawlers

http://www2.jpl.nasa.gov/sl9/1940

1950

1960

1970

1980

1990

2000

2010

2000s - online developer community

• Explosion of open-source

• Binary artifact repositories - 41K maven.org

• Github.com social coding - 1M projects!

1940

1950

1960

1970

1980

1990

2000

2010

2000s - DevOps, NoOps, Continuous Deployment, etc

• Developers responsible for testing, operations, ...

• Continuous deployment: update production many times a day

• Chaos monkeys - constantly verify that the system can tolerate failurehttp://highscalability.com/blog/2012/5/7/startups-are-creating-a-new-system-of-the-world-for-it.html

1940

1950

1960

1970

1980

1990

2000

2010

William Gibson

“The future is already here – it's just not evenly distributed.”

Ideas take a long time to diffuse through the software

community

Where is here?

Polyglot applications NoSQL databases Cloud Computing

Until ~2004 Java was the one language to

rule them all

Some developers became frustrated with

the perceived complexity of Java

Ruby on Rails became popular

• Web application framework for Ruby

• Significantly more productive

• Simplicity of Convention over Configuration

• Motivated the Java community to improve:

• e.g. Spring became even easier to use

But...

…Highlighted problems with Java

• Tedious collection processing

• Painful object construction code

• Tedious XML processing

• Types: verboseness and complexity of generics

• Limited support for DSLs

Reopened the debate about programming languages

Alternative languages

• Ruby

• Groovy

• Clojure

• Scala

• JavaScript/NodeJS

Closuresscala> val numbers = Array(1,2,3,4,5) numbers: Array[Int] = Array(1, 2, 3, 4, 5)

scala> val oddNumbers = numbers.filter((x => x % 2 == 1)) oddNumbers: Array[Int] = Array(1, 3, 5)

scala> val doubled = numbers.map(_ * 2)doubled: Array[Int] = Array(2, 4, 6, 8, 10)

XML processing

scala> val xml = <foo><bar>baz</bar></foo>xml: scala.xml.Elem = <foo><bar>baz</bar></foo>

scala> (xml \ "bar").textres9: String = “baz”

Meta-programming

Computer programs that modify themselves

Grails persistence methodsclass Customer { String name}

Customer c = new Customer("John Doe") if (!c.save()) fail "validation failed: ${c.errors}" Customer c2 = Customer.get(c.id) 

def customers = Customer.findAllByName(“Fred”)

Sinatra DSL example

require 'sinatra'

get '/hi' do "Hello World!"end

A Ruby-based Domain Specific Language for web programming

So what should we do with these languages?

Tomcat

Traditional web application architecture

Browser

WAR

MySQL Database

developtest

deployscale

ShippingService

BillingService

InventoryService

StoreFrontUI

Simple to

ApacheCustomer

Management

But there are problems

• Inadequate for modern, real-time web applications (need NodeJS)

• Obstacle to frequent deployments

• Slows down IDE and dev/test cycle

• Obstacle to scaling development

• Requires long term commitment to technology stack

Modular, polyglot application architecture

RabbitMQ

NodeJS

Inventory Service Shipping Service

Billing Service Inventory Database

Order Database

Standalone“headless” Spring/Java applications

Spring/Scala web application

Customer Database

Desktop Browser Native Mobile application HTML5 mobile application

StoreUI StoreUI StoreUI

StoreUIJavascriptAsynchronous,

scalable communication

CustomerManagement

Rails

Real world examples

http://highscalability.com/amazon-architecture

Between 100-150 services are accessed to build a page.

http://techblog.netflix.com/

http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf

http://queue.acm.org/detail.cfm?id=1394128

There are drawbacks• Complexity: Architectural, development,

deployment

• Deciding when to use it

• In the beginning: you don’t need it and it will slow you down

• When you do need it: refactoring existing code is painful

But there are many benefits

• Scales development: focused two pizza devops teams

• Deploy services independently

• Scale services independently

• Improves fault isolation

• Enforces well defined interfaces between components

• Eliminates long-term commitment to a single technology stack

Modular, polyglot applications

If services are small...

• Regularly rewrite using a better technology stack

• Pick the best developers rather than best <pick a language> developers ⇒ polyglot culture

• Adapt system to changing requirements and better technology without a total rewrite

Moreover: you are not the same you ...• 50 to 70 billion of your cells die each day

• Cell lifetimes:

• hours - some white blood cells

• days - stomach lining cells

• years - bone cells

• lifetime - brain cells

• Yet you (the system) remains intact

http://dreamsongs.com/Files/WhitherSoftware.pdf

http://dreamsongs.com/Files/DesignBeyondHumanAbilitiesSimp.pdf

Can we build software systems with these

characteristics?

Too much technical debt ⇒ component death?

Polyglot applications NoSQL databases Cloud Computing

Relational databases have been the place to

store your data

Limitations of relational databases

• Scaling

• Despite Moore’s law vertical scaling is limited/expensive

• Horizontal scaling is limited

• Distribution

• Updating schema

• O/R impedance mismatch

• Handling semi-structured data

Solution: Use NoSQLBenefits

• Higher performance

• Higher scalability

• Richer data-model

• Schema-less

Drawbacks

• Limited transactions

• Relaxed consistency

• Unconstrained data

NoSQL databases

http://nosql-database.org/ lists 122+ NoSQL databases

Data Model Examples

Key-value Redis, Voldemort

Document MongoDB, CouchDb

Extensible columns/Column-oriented

Cassandra, Hbase, SimpleDB, DynamoDB

Graph Neo4j

• Advanced key-value store

• Very fast

• Optional persistence

Redis

K1

K2

K3

V1

V2

V2

MongoDB: fast, scalable, document orientedServer

Database: Food To Go

Collection: Restaurants

{ "_id" : ObjectId("4bddc2f49d1505567c6220a0") "name": "Ajanta", "serviceArea": ["94619", "99999"], "openingHours": [

{ "dayOfWeek": 1, "open": 1130, "close": 1430 },{ "dayOfWeek": 2, "open": 1130, "close": 1430 }, …

]}

BSON = binary JSON

Sequence of bytes on disk è fast i/o

Datacenter 2

Cassandra cluster

Datacenter 1

Cassandra cluster

Cassandra: very scalable

Node 1

Node 2

Node 3

Node 4

Node 1

Node 2

Node 3

Node 4

Application Application

The future is polyglot persistence

IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg

e.g. Netflix• RDBMS• SimpleDB• Cassandra• Hadoop/Hbase

Polyglot applications NoSQL databases Cloud Computing

Let’s imagine that you want to deploy an app...

• Do you know how much hardware to buy?

• Can you afford it?

• How long does it take to approve, buy and install?

• Who is going to set it up?

• What happens if the traffic increases 10x?

• Can we afford a test lab?

Cloud computing empowers us to deal

with these challenges...

Cloud computing defined

IT delivered as a service

Over the internet

Self-service

Pay per use

SaaS

PaaS

IaaS

The three layers of cloud computing

Wednesday October 18, 2006....

.... spoke at the Oakland JUG

AWS evangelist

Sign up and deploy your application a few minutes later

• Login using your existing Amazon account

• Select the web

Benefits of IaaS

• Agility

• Pay per use

• Elasticity

• Focus on deploying your application

But you need to configure and maintain the operating systems, the app servers and the

databases etc.

SaaS

PaaS

IaaS

Need to move up the stack

PaaS

= Application deployment

and management

Service provisioning

+

Simple, Open, Flexible,

Scalable

The Open Platform as a Service

Deploy and scale applications in seconds, without locking yourself into a single cloud

Applica'on  Service  Interface

Data Services

Other Services

Msg Services

Easy polyglot application deployment and service provisioning

vFabric Postgres

vFabric RabbitMQTM

Additional partners services …

OSS community

Private  Clouds  

PublicClouds

MicroClouds

CloudFoundry.COM - Multi-tenant PaaS operated by VMware

Runtimes & FrameworksServices

vCenter / vSphere

CloudFoundry.COM (beta)

Infrastructure

Micro Cloud FoundryTM – Industry first

downloadable PaaS

Runtimes & FrameworksServices

Your Laptop/PC

Micro Cloud Foundry

Single VM instance of Cloud Foundry

that runs on a developer’s MAC or PC

CloudFoundry.ORG - Community open-source project

CloudFoundry.ORG

DownloadCode

Setup Environment

Deploy Behind FirewallBOSH

Apache2 license

Your Infrastructure

Sinatra + Redis

require 'sinatra'require 'redis'

configure do $r = Redis.new(:host => '127.0.0.1', :port => '6379') if !$r end

get '/' do "Hello World! " + $r.incr("hitcounter").to_send

http://sgce2012.cloudfoundry.com/

Connect to Redis

Increment hit counter

Consuming 3rd party cloud services

Thousands of 3rd party services

http://www.programmableweb.com/apis/directory

Number of APIs increasing exponentially

• Predominantly REST

• Predominantly JSON

• > billion API calls/day: Twitter, Google, Facebook, Netflix, Accuweather, ...

• Increasing number of API-only companies

http://www.slideshare.net/jmusser/j-musser-apishotnotgluecon2012

Cloud service trends

Diverse

SendGrid

GeoNamesTwilio

Tropo

Google

Mashery

Yahoo

Amazon

Face.comPaypal

Cybersource Aviary.com

WorldAddressesAlchemyAPI

Amazon Simple Storage Service (S3)

• Bucket

• has a globally unique name, e.g. cloudtools.org.photos

• contains objects

• Objects - 1 byte – 5 Tbytes

• Pay per use: storage + bandwidth

• Accessed via RESTful API

• PUT – create bucket or object

• GET – retrieve object

• DELETE – delete an object 1T objects June 2012

face.com - face detection and recognition

http://api.face.com/faces /detect.json?api_key=&api_secret=&urls=<url>&detector=Aggressive&attributes=all

Twilio - Telephony and SMS as a service

• REST API

• Allocate phone numbers

• Make and receive phone calls

• Send and receive SMS messages

• Pay per use:

• Phone number – per month

• Phone calls - per-minute

• SMS – per SMS sent or received

• Example: Salesforce – SMS-based voting for 19,000 conference attendees

Expose your APIs and

build an ecosystem

Where are we going?

Going beyond today’s pop culture

But love is blind and lovers cannot seeThe pretty follies that themselves commit;For if they could, Cupid himself would blush To see me thus transformed to a boy.

William Shakespeare, The Mechant of Venice

Gartner hype curve

http://www.gartner.com/technology/research/methodologies/hype-cycle.jsp#

Concurrency

Clock speeds have plateaued

http://www.gotw.ca/publications/concurrency-ddj.htm

Multi core is the future

• Dual core A5 for iPhone 4S

• Quad core Intel i7 for laptops

• 10 core Intel Xeon Processor

• 3072 core NVidia Tesla Kepler

Sometimes we can use request-level parallelism and let the container worry about it

BUT not always...

But concurrent programming is difficult

The problem=

Mutable state that’s shared between multiple

threads (using locks)

Not thread safepublic class InventoryTracker { private int inStock; private int sold = 0; public InventoryTracker(int initialInventory) { this.inStock = initialInventory; } public int getInStock() { return inStock; } public int sell(int units) { inStock -= units; sold += units; return inStock; }}

Thread safepublic class InventoryTracker { private int inStock; private int sold = 0; public InventoryTracker(int initialInventory) { this.inStock = initialInventory; } public synchronized int getInStock() { return inStock; } public synchronized int sell(int units) { inStock -= units; sold += units; return inStock; }}

• Single threaded runtime ⇒ no concurrency

• Runs an event loop that waits for i/o events

• i/o event ⇒ callback function

• Updates state

• Publishes events ⇒ triggers more callbacks)

• Initiates i/o, registering callbacks

• Returns

NodeJS - Event driven

Mutable state that’s shared between multiple threads

NodeJS examplevar http = require('http');var fs = require('fs');var url = require('url');

var inStock = 100;var sold = 0;

http.createServer(function (req, res) { var quantity = parseInt(url.parse(req.url, true).query.quantity); inStock -= quantity; sold += quantity; var message = "inStock=" + inStock + ", sold=" + sold; res.end(message);}).listen(1337, "127.0.0.1");

console.log('Server running at http://127.0.0.1:1337/');

HTTP request handler

http://127.0.0.1:1337?quantity=1

• Actor = object that asynchronously exchanges immutable messages with other actors

• Actor processes one message at a time

• Sends messages

• Updates its private mutable state

• Creates new actors

• No shared state

Scala Actors

Mutable state that’s shared between multiple threads

Scala examplecase class Sell(quantity: Int)case class Sale(inStock : Int, sold : Int)

class InventoryTracker(initialInventory : Int) extends Actor { var inStock = initialInventory var sold = 0 def receive = { case Sell(quantity) => inStock -= quantity sold += quantity sender ! Sale(inStock, sold) }}

Update state

Message types

Send reply

case class Order(quantity: Int)

class StoreFront(inventoryTracker: ActorRef) extends Actor { var numberOfOrders = 0

def receive = { case Order(quantity) => numberOfOrders += 1 inventoryTracker ! Sell(quantity) case Sale(inStock, sold) => println("numberOfOrders=" + numberOfOrders + ",inStock=" +

inStock + ", sold=" + sold) }}

Send message

Handle reply

• Eliminates mutable state

BUT

• I/O and state management require Monads

• Monads are extremely difficult to learn

Pure functional programming

Mutable state that’s shared between multiple threads

Clojure - a modern LISP• Practical functional programming language

• Runs on the JVM

• Core data structures are immutable

BUT

• Ref = object that holds a mutable reference to a value

• Refs can only be updated within a transaction

• Software Transaction Memory

• Atomic, Consistent, Isolated

• No explicit locksMutable state that’s

shared between multiple threads

Clojure STM example

(def it (make-inventory-tracker 100))

(sale it 25)

(defn sale [tracker quantity] (dosync (let [new-in-stock (alter (:in-stock tracker) - quantity) new-sold (alter (:sold tracker) + quantity) ] [new-in-stock new-sold] )))

(defrecord InventoryTracker [in-stock sold])

(defn make-inventory-tracker [initial-inventory] (InventoryTracker. (ref initial-inventory) (ref 0)))

dosync=

a transaction

Update refs

Create refs

Final thoughtsSoftware development is an old

... yet immature profession

Multi-core programming is a challenge

BUT

The cloud enables anyone with a good idea to create an application that touches the lives of millions of people

Thank you!

Chris Richardson

Author of POJOs in ActionFounder of the original CloudFoundry.com

crichardson@vmware.com@crichardson

plainoldobjects.com

Signup @ cloudfoundry.com Promo code: sgce2012