message driven architecture enables elasticity, resilience and availability
TRANSCRIPT
Message / Event Driven Arch
Tim Harper, Software Craftsman
Me
Books
Books
+ anything out of this
guy’s mouth
Messaging / Event Oriented Arch
Tim Harper, Software Craftsman
Why use privates in code?/** * INTERNAL API */ private[akka] object StreamLayout {
// compile-time constant final val Debug = false
final def validate(m: Module, level: Int = 0, doPrint: Boolean = false, idMap: mutable.Map[AnyRef, Int] = mutable.Map.empty): Unit = { val ids = Iterator from 1 def id(obj: AnyRef) = idMap get obj match { case Some(x) => x
Why use privates in code?/** * INTERNAL API */ private[akka] object StreamLayout {
// compile-time constant final val Debug = false
final def validate(m: Module, level: Int = 0, doPrint: Boolean = false, idMap: mutable.Map[AnyRef, Int] = mutable.Map.empty): Unit = { val ids = Iterator from 1 def id(obj: AnyRef) = idMap get obj match { case Some(x) => x
Why use privates in code?/** * INTERNAL API */ private[akka] object StreamLayout {
// compile-time constant final val Debug = false
final def validate(m: Module, level: Int = 0, doPrint: Boolean = false, idMap: mutable.Map[AnyRef, Int] = mutable.Map.empty): Unit = { val ids = Iterator from 1 def id(obj: AnyRef) = idMap get obj match { case Some(x) => x
Why use privates in code?
• Good API design
Why use privates in code?
• Good API design
• It’s easier to add functionality than remove it / change it
Why use privates in code?
• Good API design
• It’s easier to add functionality than remove it / change it
• Less guarantees = less coupling
Why use privates in code?
• Good API design
• It’s easier to add functionality than remove it / change it
• Less guarantees = less coupling
Don’t “pay the cost of a rich API without any of the benefits”
Why use privates in code?
“Private” return value?trait Publisher[T] { def subscribe(sub: Subscriber[T]): Unit } trait Subscription { def requestMore(n: Int): Unit def cancel(): Unit } trait Subscriber[T] { def onSubscribe(s: Subscription): Unit def onNext(elem: T): Unit def onError(thr: Throwable): Unit def onComplete(): Unit }
“Private” return value?trait Publisher[T] { def subscribe(sub: Subscriber[T]): Unit } trait Subscription { def requestMore(n: Int): Unit def cancel(): Unit } trait Subscriber[T] { def onSubscribe(s: Subscription): Unit def onNext(elem: T): Unit def onError(thr: Throwable): Unit def onComplete(): Unit }
“Private” return value?trait Publisher[T] { def subscribe(sub: Subscriber[T]): Unit } trait Subscription { def requestMore(n: Int): Unit def cancel(): Unit } trait Subscriber[T] { def onSubscribe(s: Subscription): Unit def onNext(elem: T): Unit def onError(thr: Throwable): Unit def onComplete(): Unit }
Private result?
Private result?• Schedule now or then
Private result?• Schedule now or then
• Run it here or there
Private result?• Schedule now or then
• Run it here or there
• Recovery, prioritization, etc.
Private result?• Schedule now or then
• Run it here or there
• Recovery, prioritization, etc.
Don’t “pay the cost of a synchronous API without any of the benefits”
Reactive Manifesto
Reactive Manifesto
Reactive Manifesto
It all starts here.
Event Oriented
• Flip the arrows
Event Oriented
Concert added to database
Event Oriented
Concert added to database
Send Email to User
Event Oriented
Concert added to database
Send Email to User
Schedule Ingestion Task
Event Oriented
Concert added to database
Send Email to User
Notify SalesForce
Schedule Ingestion Task
Event Oriented
Concert added to database
Send Email to User
Record in ElasticSearch
Notify SalesForce
Schedule Ingestion Task
Event Oriented
Concert added to database
Send Email to User
Record in ElasticSearch
Notify SalesForce
Schedule Ingestion Task
Event Oriented
Concert added to database
Send Email to User
Record in ElasticSearch
Notify SalesForce
Schedule Ingestion Task
Such knowledge
Event Oriented
Concert added to database
Send Email to User
Record in ElasticSearch
Notify SalesForce
Schedule Ingestion Task
Such knowledge
Very couple
Event Oriented
Concert added to database
Send Email to User
Record in ElasticSearch
Notify SalesForce
Schedule Ingestion Task
Such knowledge
Very couple
Wow
Event Oriented
Concert added to database
Record in ElasticSearch
Send Email to User
Schedule Ingestion Task
Notify SalesForce
Event Oriented
Concert added to database
Record in ElasticSearch
Send Email to User
Schedule Ingestion Task
Notify SalesForce
Event OrientedConcert added
to database
Record in ElasticSearch
Send Email to User
Schedule Ingestion Task
Notify SalesForce
Announce
Listen for announce
Listen for announce
Listen for announce
Listen for announce
Event Oriented
Notify SalesForce
Listen for promotion
event
Listen for new concert event
Listen for concert update
event
Functional Reactive Programming
Event X happened@ time Y
Functional Reactive Programming
Event X happened@ time Y
AS TRUE TODAY AS
WHEN IT WAS WRITTEN
Functional Reactive Programming
A @ 1:00
B @ 1:01
C @ 1:05
D @ 1:06
Functional Reactive Programming
A @ 1:00
B @ 1:01
C @ 1:05
D @ 1:06
F @ 1:00
G @ 1:01
H @ 1:05
I @ 1:06
FRPA @ 1:00
B @ 1:01
C @ 1:05
D @ 1:06
REDUCE (cache layer)
FRPA @ 1:00
B @ 1:01
C @ 1:05
D @ 1:06
REDUCE (cache layer)
It’s beginning to look
a lot like CQRS
FRP / CQRS
• “Travel through time”
• Retain all data (UPDATE = DELETE PRIOR STATE)
• Full audit log
• Separation of read / write concerns; et. al
Messaging Patterns
• Queue
• Topic
• Exchange
Orange 1
Messaging Patterns: Direct
Orange 1
Publish
“oranges”
“oranges” queue
Orange 3
Orange 2
Orange 1
Messaging Patterns: Direct
Publish
“oranges”
“oranges” queue
Orange 3
Orange 2
Orange 1
Orange 1
Messaging Patterns: Broadcast / Fanout
Orange 1
Publish
“fruit” exchange
“fruit”
Orange 3
Orange 2
“products”
Orange 3
Orange 2
Orange 1
Messaging Patterns: Broadcast / Fanout
Publish
“fruit” exchange
“fruit”
Orange 3
Orange 2
“products”
Orange 3
Orange 2
Orange 1
Orange 1
Orange 1
Messaging Patterns: Broadcast / Fanout
Publish
“fruit” exchange
“fruit”
Orange 3
Orange 2
“products”
Orange 3
Orange 2
Orange 1
Orange 1
AKA PubSub
Orange 1
Messaging Patterns: Topic / PubSub
Orange 1
Publish
“fruit” exchange topic: fruit.orange
“all-fruit” fruit.*
Apple 6
Apple 4
“apples” fruit.apples
Apple 6
Apple 4
Orange 1
Messaging Patterns: Topic / PubSub
Publish
“fruit” exchange topic: fruit.orange
“all-fruit” fruit.*
Apple 6
Apple 4
“apples” fruit.apples
Apple 6
Apple 4
Orange 1
Orange 1
Messaging Patterns: Topic / PubSub
Publish
“fruit” exchange topic: fruit.orange
“all-fruit” fruit.*
Apple 6
Apple 4
“apples” fruit.apples
Apple 6
Apple 4
Orange 1
“Yo dawgs,
this thing happened”
Message Broker Benefits
• Messages are routes and enqueued while app not booting
• Centralized point of coordination
• Replication, persistence, etc.
• If you don’t use one then you will build one.
Delivery guarantees
• At least once
• At most once
Delivery guarantees
Network Process
acksend receive
Broker
Delivery guarantees
Idempotence
• fn(fn(x)) = fn(x)
• Great idea in general
• (NOT impotency)
Idempotence
• The operation to perform is a function of the current state and next command.
Cons(in which it is acknowledged that everything has a cost)
Cons
• Complexity moved, not entirely solved.
• (although, functional discipline aids in reducing state complexity)
• Difficulty grokking message flow (DOCS!!!)
Cons
• Tracing errors can be difficult.
Cons
• Rate-limiting with high-availability can be complex.
ConsDDOS YOUR DATABASE
WITH THIS ONEWEIRD TRICK