reactive supply to changing demand

Reactive Supply to Changing Demand

Jonas Bonér Typesafe

CTO & co-founder @jboner

Scalable Trait: React 2014

Outline

1. Introduction

2. Scale Up

3. Scale Out

4. Bring It Together

2

Outline

1. Introduction

2. Scale Up

3. Scale Out

4. Bring It Together

3

4

4

What is Scalability?

6

“The house in which Amdahl wakes up very early each day and rules with an iron fist.”

- Martin Thompson (originally Gil Tene)

6

“The house in which Amdahl wakes up very early each day and rules with an iron fist.”

- Martin Thompson (originally Gil Tene)

7

“Capable of being easily expanded or upgraded on demand” - Merriam Webster Dictionary

8

“A service is said to be scalable if when we increase the resources in a system, it results in increased performance

in a manner proportional to resources added.” - Werner Vogels

Scalability vs Performance

Why do we need to Scale on Demand?

The rules of the game

have changed

12

Apps in the 60s-90s were written for

Apps today are written for

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

Slow networks

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

Slow networks Fast networks

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

Slow networks Fast networks

Few concurrent users

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

Slow networks Fast networks

Few concurrent users Lots of concurrent users

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

Slow networks Fast networks

Few concurrent users Lots of concurrent users

Small data sets

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

Slow networks Fast networks

Few concurrent users Lots of concurrent users

Small data sets Large data sets

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

Slow networks Fast networks

Few concurrent users Lots of concurrent users

Small data sets Large data sets

Latency in seconds

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

Slow networks Fast networks

Few concurrent users Lots of concurrent users

Small data sets Large data sets

Latency in seconds Latency in milliseconds

14

14

14

14

Cost Gravity is at Work

15

Cost Gravity is at Work

15

Outline

1. Introduction

2. Scale Up

3. Scale Out

4. Bring It Together

16

UPScale

18

Modern CPU architecture

18

Modern CPU architecture

18

Modern CPU architecture

The CPU is gambling. Taking bets.

19

Maximize Locality of Reference

Minimize Contention

Common points of

22

ApplicationPhysical

contention

23

Neverever

23

Neverever

Block

23

Neverever

24

GO

Async

24

GO

Divide & Conquer

25

26

27

Single Writer Principle

27

Single Writer PrincipleIO deviceProducers

SERIAL & CONTENDED

27

Single Writer PrincipleIO deviceProducers

SERIAL & CONTENDED

IO deviceProducers Actor or Queue

BATCHED & UNCONTENDED

The Role of Immutable State

28

The Role of Immutable State

28

The Role of Immutable State

• Great to represent facts

• Events

• Database snapshots

28

The Role of Immutable State

• Great to represent facts

• Events

• Database snapshots

• Less ideal for a “working” data set

• Persistent data structures can increase contention

• Instead use a Share Nothing Architecture with mutable state within each isolated processing unit

28

29

Needs to be async and non-blocking all the way down

29

Needs to be async and non-blocking all the way down

…or Amdahl’s Law will hunt you down

29

Needs to be async and non-blocking all the way down

…or Amdahl’s Law will hunt you down

30

NOTHINGshare

Scale on Demand

Outline

1. Introduction

2. Scale Up

3. Scale Out

4. Bring It Together

32

OUTScale

Distributed Computing is the !

• Mobile

• Cloud Services, REST etc.

• NOSQL DBs

• Big Data

34

NEW NORMAL

What is the essence of distributed computing?

35

What is the essence of distributed computing?

To try to overcome that 1. Information travels at the speed of light

2. Independent things fail independently

35

36

Node Node Node

36

Node

Middleware

Node Node Node

36

Node

Cluster/Rack/Datacenter

Cluster/Rack/Datacenter

Cluster/Rack/Datacenter

Middleware

Node Node Node

36

Node

37

Peter Deutsch’s 8 Fallacies of

Distributed Computing

37

1. The network is reliable

Peter Deutsch’s 8 Fallacies of

Distributed Computing

37

1. The network is reliable2. Latency is zero

Peter Deutsch’s 8 Fallacies of

Distributed Computing

37

1. The network is reliable2. Latency is zero3. Bandwidth is infinitePeter Deutsch’s

8 Fallacies of Distributed Computing

37

1. The network is reliable2. Latency is zero3. Bandwidth is infinite4. The network is secure

Peter Deutsch’s 8 Fallacies of

Distributed Computing

37

1. The network is reliable2. Latency is zero3. Bandwidth is infinite4. The network is secure5. Topology doesn't change

Peter Deutsch’s 8 Fallacies of

Distributed Computing

37

1. The network is reliable2. Latency is zero3. Bandwidth is infinite4. The network is secure5. Topology doesn't change6. There is one administrator

Peter Deutsch’s 8 Fallacies of

Distributed Computing

37

1. The network is reliable2. Latency is zero3. Bandwidth is infinite4. The network is secure5. Topology doesn't change6. There is one administrator7. Transport cost is zero

Peter Deutsch’s 8 Fallacies of

Distributed Computing

37

1. The network is reliable2. Latency is zero3. Bandwidth is infinite4. The network is secure5. Topology doesn't change6. There is one administrator7. Transport cost is zero8. The network is homogeneous

Peter Deutsch’s 8 Fallacies of

Distributed Computing

The Graveyard of Distributed Systems

• Distributed Shared Mutable State

•⇒ EVIL (where N is number of nodes)

• Serializable Distributed Transactions

• Synchronous RPC

• Guaranteed Delivery

• Distributed Objects • “Sucks like an inverted hurricane” - Martin Fowler

38

N

Maximize Locality of Reference

40

Sticky Load Balancing & Sharding

40

41

Buddy Replication

41

42

Consistent Hashing

42

Minimize Contention (Coordination in the Cluster)

Strong Consistency

45

Linearizability

“Under linearizable consistency, all operations appear to have executed atomically in an order that is consistent with the

global real-time ordering of operations.” - Herlihy & Wing 1991

Strong Consistency Protocols

47

CAPTheorem

47

CAPTheorem

Eventual Consistency

49

CRDTCvRDTs/CmRDTs

50

“In general, application developers simply do not implement large scalable applications assuming

distributed transactions.” -­‐  Pat Helland

Life beyond Distributed Transactions: an Apostate’s Opinion

51

“State transitions are an important part of our problem space and should be modeled within our domain.”  

- Greg Young

Domain Events

52

"The database is a cache of a subset of the log.” - Pat Helland

The Event Log

The Event Log

• Append-Only Logging

• Database of Facts

• Two models:

• One single Event Log

• Strong Consistency

• Multiple sharded Event Logs

• Strong + Eventual Consistency

53

NOTHING

54

share

Scale on Demand

TRANSPARENCY

56

location

Outline

1. Introduction

2. Scale Up

3. Scale Out

4. Bring It Together

57

58

58

59

Data Center

59

Data Center

ClusterClusterMachineMachineJVMJVMNodeNodeThreadThreadCPUCPUCPU Socket

CPU Socket

CPU Core

CPU Core

CPU L1/L2 Cache

CPU L1/L2 Cache

60

Scaling Up and Out is essentially

the same thing

Questions?

Reactive Supply to Changing Demand

Jonas Bonér Typesafe

CTO & co-founder @jboner

Scalable Trait: React 2014

Thanks for listening