Download - Reactive Supply To Changing Demand
Reactive Supply to Changing Demand
Jonas Bonér Typesafe
CTO & co-founder @jboner
Scalable Trait: React 2014
Outline
1. Introduction
2. Scale Up
3. Scale Out
4. Bring It Together
2
Outline
1. Introduction
2. Scale Up
3. Scale Out
4. Bring It Together
3
4
4
What is Scalability?
6
“The house in which Amdahl wakes up very early each day and rules with an iron fist.”
- Martin Thompson (originally Gil Tene)
6
“The house in which Amdahl wakes up very early each day and rules with an iron fist.”
- Martin Thompson (originally Gil Tene)
7
“Capable of being easily expanded or upgraded on demand” - Merriam Webster Dictionary
8
“A service is said to be scalable if when we increase the resources in a system, it results in increased performance
in a manner proportional to resources added.” - Werner Vogels
Scalability vs Performance
Why do we need to Scale on Demand?
The rules of the game
have changed
12
Apps in the 60s-90s were written for
Apps today are written for
12
Apps in the 60s-90s were written for
Apps today are written for
Single machines
12
Apps in the 60s-90s were written for
Apps today are written for
Single machines Clusters of machines
12
Apps in the 60s-90s were written for
Apps today are written for
Single machines Clusters of machines
Single core processors
12
Apps in the 60s-90s were written for
Apps today are written for
Single machines Clusters of machines
Single core processors Multicore processors
12
Apps in the 60s-90s were written for
Apps today are written for
Single machines Clusters of machines
Single core processors Multicore processors
Expensive RAM
12
Apps in the 60s-90s were written for
Apps today are written for
Single machines Clusters of machines
Single core processors Multicore processors
Expensive RAM Cheap RAM
12
Apps in the 60s-90s were written for
Apps today are written for
Single machines Clusters of machines
Single core processors Multicore processors
Expensive RAM Cheap RAM
Expensive disk
12
Apps in the 60s-90s were written for
Apps today are written for
Single machines Clusters of machines
Single core processors Multicore processors
Expensive RAM Cheap RAM
Expensive disk Cheap disk
12
Apps in the 60s-90s were written for
Apps today are written for
Single machines Clusters of machines
Single core processors Multicore processors
Expensive RAM Cheap RAM
Expensive disk Cheap disk
Slow networks
12
Apps in the 60s-90s were written for
Apps today are written for
Single machines Clusters of machines
Single core processors Multicore processors
Expensive RAM Cheap RAM
Expensive disk Cheap disk
Slow networks Fast networks
12
Apps in the 60s-90s were written for
Apps today are written for
Single machines Clusters of machines
Single core processors Multicore processors
Expensive RAM Cheap RAM
Expensive disk Cheap disk
Slow networks Fast networks
Few concurrent users
12
Apps in the 60s-90s were written for
Apps today are written for
Single machines Clusters of machines
Single core processors Multicore processors
Expensive RAM Cheap RAM
Expensive disk Cheap disk
Slow networks Fast networks
Few concurrent users Lots of concurrent users
12
Apps in the 60s-90s were written for
Apps today are written for
Single machines Clusters of machines
Single core processors Multicore processors
Expensive RAM Cheap RAM
Expensive disk Cheap disk
Slow networks Fast networks
Few concurrent users Lots of concurrent users
Small data sets
12
Apps in the 60s-90s were written for
Apps today are written for
Single machines Clusters of machines
Single core processors Multicore processors
Expensive RAM Cheap RAM
Expensive disk Cheap disk
Slow networks Fast networks
Few concurrent users Lots of concurrent users
Small data sets Large data sets
12
Apps in the 60s-90s were written for
Apps today are written for
Single machines Clusters of machines
Single core processors Multicore processors
Expensive RAM Cheap RAM
Expensive disk Cheap disk
Slow networks Fast networks
Few concurrent users Lots of concurrent users
Small data sets Large data sets
Latency in seconds
12
Apps in the 60s-90s were written for
Apps today are written for
Single machines Clusters of machines
Single core processors Multicore processors
Expensive RAM Cheap RAM
Expensive disk Cheap disk
Slow networks Fast networks
Few concurrent users Lots of concurrent users
Small data sets Large data sets
Latency in seconds Latency in milliseconds
14
14
14
14
Cost Gravity is at Work
15
Cost Gravity is at Work
15
Outline
1. Introduction
2. Scale Up
3. Scale Out
4. Bring It Together
16
UPScale
18
Modern CPU architecture
18
Modern CPU architecture
18
Modern CPU architecture
The CPU is gambling. Taking bets.
19
Maximize Locality of Reference
Minimize Contention
Common points of
22
ApplicationPhysical
contention
23
Neverever
23
Neverever
Block
23
Neverever
24
GO
Async
24
GO
Divide & Conquer
25
26
27
Single Writer Principle
27
Single Writer PrincipleIO deviceProducers
SERIAL & CONTENDED
27
Single Writer PrincipleIO deviceProducers
SERIAL & CONTENDED
IO deviceProducers Actor or Queue
BATCHED & UNCONTENDED
The Role of Immutable State
28
The Role of Immutable State
28
The Role of Immutable State
• Great to represent facts
• Events
• Database snapshots
28
The Role of Immutable State
• Great to represent facts
• Events
• Database snapshots
• Less ideal for a “working” data set
• Persistent data structures can increase contention
• Instead use a Share Nothing Architecture with mutable state within each isolated processing unit
28
29
Needs to be async and non-blocking all the way down
29
Needs to be async and non-blocking all the way down
…or Amdahl’s Law will hunt you down
29
Needs to be async and non-blocking all the way down
…or Amdahl’s Law will hunt you down
30
NOTHINGshare
Scale on Demand
Outline
1. Introduction
2. Scale Up
3. Scale Out
4. Bring It Together
32
OUTScale
Distributed Computing is the !
• Mobile
• Cloud Services, REST etc.
• NOSQL DBs
• Big Data
34
NEW NORMAL
What is the essence of distributed computing?
35
What is the essence of distributed computing?
To try to overcome that 1. Information travels at the speed of light
2. Independent things fail independently
35
36
Node Node Node
36
Node
Middleware
Node Node Node
36
Node
Cluster/Rack/Datacenter
Cluster/Rack/Datacenter
Cluster/Rack/Datacenter
Middleware
Node Node Node
36
Node
37
Peter Deutsch’s 8 Fallacies of
Distributed Computing
37
1. The network is reliable
Peter Deutsch’s 8 Fallacies of
Distributed Computing
37
1. The network is reliable2. Latency is zero
Peter Deutsch’s 8 Fallacies of
Distributed Computing
37
1. The network is reliable2. Latency is zero3. Bandwidth is infinitePeter Deutsch’s
8 Fallacies of Distributed Computing
37
1. The network is reliable2. Latency is zero3. Bandwidth is infinite4. The network is secure
Peter Deutsch’s 8 Fallacies of
Distributed Computing
37
1. The network is reliable2. Latency is zero3. Bandwidth is infinite4. The network is secure5. Topology doesn't change
Peter Deutsch’s 8 Fallacies of
Distributed Computing
37
1. The network is reliable2. Latency is zero3. Bandwidth is infinite4. The network is secure5. Topology doesn't change6. There is one administrator
Peter Deutsch’s 8 Fallacies of
Distributed Computing
37
1. The network is reliable2. Latency is zero3. Bandwidth is infinite4. The network is secure5. Topology doesn't change6. There is one administrator7. Transport cost is zero
Peter Deutsch’s 8 Fallacies of
Distributed Computing
37
1. The network is reliable2. Latency is zero3. Bandwidth is infinite4. The network is secure5. Topology doesn't change6. There is one administrator7. Transport cost is zero8. The network is homogeneous
Peter Deutsch’s 8 Fallacies of
Distributed Computing
The Graveyard of Distributed Systems
• Distributed Shared Mutable State
•⇒ EVIL (where N is number of nodes)
• Serializable Distributed Transactions
• Synchronous RPC
• Guaranteed Delivery
• Distributed Objects • “Sucks like an inverted hurricane” - Martin Fowler
38
N
Maximize Locality of Reference
40
Sticky Load Balancing & Sharding
40
41
Buddy Replication
41
42
Consistent Hashing
42
Minimize Contention (Coordination in the Cluster)
Strong Consistency
45
Linearizability
“Under linearizable consistency, all operations appear to have executed atomically in an order that is consistent with the
global real-time ordering of operations.” - Herlihy & Wing 1991
Strong Consistency Protocols
47
CAPTheorem
47
CAPTheorem
Eventual Consistency
49
CRDTCvRDTs/CmRDTs
50
“In general, application developers simply do not implement large scalable applications assuming
distributed transactions.” -‐ Pat Helland
Life beyond Distributed Transactions: an Apostate’s Opinion
51
“State transitions are an important part of our problem space and should be modeled within our domain.”
- Greg Young
Domain Events
52
"The database is a cache of a subset of the log.” - Pat Helland
The Event Log
The Event Log
• Append-Only Logging
• Database of Facts
• Two models:
• One single Event Log
• Strong Consistency
• Multiple sharded Event Logs
• Strong + Eventual Consistency
53
NOTHING
54
share
Scale on Demand
TRANSPARENCY
56
location
Outline
1. Introduction
2. Scale Up
3. Scale Out
4. Bring It Together
57
58
58
59
Data Center
59
Data Center
ClusterClusterMachineMachineJVMJVMNodeNodeThreadThreadCPUCPUCPU Socket
CPU Socket
CPU Core
CPU Core
CPU L1/L2 Cache
CPU L1/L2 Cache
60
Scaling Up and Out is essentially
the same thing
Questions?
Reactive Supply to Changing Demand
Jonas Bonér Typesafe
CTO & co-founder @jboner
Scalable Trait: React 2014
Thanks for listening