Upload File

events on the outside, on the inside and at the core (jfokus jfokus2016)

  • Home
  • Software
  • Events on the outside, on the inside and at the core (jfokus jfokus2016)
77
@crichardson Events on the outside, on the inside and at the core Chris Richardson Founder of Eventuate.io Founder of the original CloudFoundry.com Author of POJOs in Action @crichardson [email protected] http://microservices.io http://eventuate.io http://plainoldobjects.com

Upload: chris-richardson

Post on 15-Apr-2017

5.802 views

Category:

Software


3 download

Report

TRANSCRIPT

Page 1: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Events on the outside, on the inside and at the coreChris Richardson

Founder of Eventuate.io Founder of the original CloudFoundry.com Author of POJOs in Action

@crichardson [email protected] http://microservices.io http://eventuate.io http://plainoldobjects.com

Page 2: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Presentation goal

Events play a key role in applications

Event sourcing enables the event-driven enterprise

Page 3: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

About Chris

Page 4: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

About Chris

Consultant and trainer focusing on

microservices

http://www.chrisrichardson.net/

Page 5: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

About Chris

Founder of a startup that is creating

a platform that makes it easy for application developers to write

microservices

http://eventuate.io

Page 6: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

For more information

https://github.com/cer/event-sourcing-examples

http://microservices.io

http://plainoldobjects.com/

https://twitter.com/crichardson

http://eventuate.io/

Page 7: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Agenda

Events on the outside

Events on the inside

Events at the core with event sourcing

Designing event-centric domain model

Page 8: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Events on the outside

Application A

Application B

Application C

Event X

Event YEvent Z

Page 9: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

What’s an event?

http://www.merriam-webster.com/dictionary/event

Page 10: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Examples of events

Page 11: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Instacart: event-driven grocery shopping

Page 12: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

An event is produced when…

Creation or update of a business object

Attempt to violate a business rule

Page 13: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Who consumes an event?

Order Application

HumanNotification Service

Dashboard/ Monitoring

Shipping Application

OrderCreated

Email SMS …

Page 14: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

How to deliver events?

Page 15: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Polling for events

HTTP

Periodically poll for events

Atom Publishing Protocol (AtomPub)

Based on HTTP

Head is constantly changing

Tail is immutable and can be efficiently cached

High-latency, inefficient

Page 16: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Using WebSockets

Browser

Web Socket

STOMP

Client application

Service

Web Socket

STOMP

ApplicationSUBSCRIBE

MESSAGEMESSAGE

Low latency, more efficient, but what about past events?

Page 17: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Webhooks = user-defined HTTP callback

Client Service

register(events, callbackUrl)

POST callbackUrl

POST callbackUrl

…

https://en.wikipedia.org/wiki/WebhookLow latency, more efficient, but what

about past events?

Page 18: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Github webhookshttps://developer.github.com/webhooks/

Installed on an organization or repository

e.g. POST /repos/:owner/:repo/hooks

Available events:

push - push to a repository

fork - repository is forked

pull_request - assigned, unassigned, …

push - push to a repository

…

Page 19: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Twilio - Telephony and SMS as a service

Twilio Your Application

TwiML doc

HTTP GET/POST

REST API

Manage resourcesSend SMS

Initiate voice calls

Webhooks handle incoming SMS and voice calls

VoiceSMS

Phone number ⇒ SMS URL + VOICE URL

Page 20: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Integration hubs - Zapier, IFTTTApplication abstraction:

Triggers - events published by application: polling or Webhooks

Action - operation supported by application, e.g. REST API end points

Page 21: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

WebHooks =

web friendly publish/subscribe

Page 22: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Enterprise integration patterns

http://www.enterpriseintegrationpatterns.com/patterns/messaging/

Page 23: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

messaging system

Messaging-based IPC

Sender RecipientChannelmessage

Page 24: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Example messaging systems

Page 25: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

The event-driven enterprise

App A App BApp X

App Y

Firewall

Message Broker

webhooks

WebSockets

AtomPub

App Z App C

MsgBrokerClient Msg

BrokerClient

MsgBrokerClient

App D

MsgBrokerClient

Inside the firewallOutside the firewall

Page 26: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Agenda

Events on the outside

Events on the inside

Events at the core with event sourcing

Designing event-centric domain model

Page 27: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Events on the insideApplication

Service A

Service B

Service C

Event X

Event YEvent Z

Event X

Event Y

Page 28: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Traditional monolithic architecture

Simple to develop

test deploy scale

Shopping Cart

CustomersOrders

Mobile client RESTAPI

Browser WebUI

MessageBroker

Adapter

MySQLAdapter

RDBMS

…

Message Broker

Page 29: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

But that leads* to monolithic hell

For large and/or complex applications…

Page 30: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Today: use a microservice, polyglot architecture

Orders

Customers

…Shopping UI

Mobile Client

Browser

API Gateway

Order management

RESTAPI

CustomerManagement

RESTAPI

….

RESTAPI

OrderDatabase

(MongoDB)MongoDBAdapter

CustomerDatabase(Sharded MySQL)

MySQLAdapter

Page 31: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

But now we have distributed data management

problems

Page 32: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Example: placing an order

Order Service Customer Service

Order Database

Customer Database

Order #1 Customer #1

No 2PC

No ACID

NoSQL SQL

Page 33: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Customer management

How to maintain invariants?

Order management

Order Service

placeOrder()

Customer Service

updateCreditLimit()

Customer

creditLimit ...

has ordersbelongs toOrder

total

Invariant: sum(open order.total) <= customer.creditLimit

?

Page 34: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Use an event-driven architecture

Services publish events when something important happens, e.g. state changes

Services subscribe to events and update their state

Maintain eventual consistency across multiple aggregates (in multiple datastores)

Synchronize replicated data

Page 35: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Event-driven application architectureApplication

Service A Service B

Message Broker

Service C

MsgBrokerClient

MsgBrokerClient

MsgBrokerClient

Service D

MsgBrokerClient

Page 36: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Order ManagementOrder

id : 4567 total: 343 state = CREATED

Customer Management

Customer creditLimit : 12000 creditReservations: {}

Customer creditLimit : 12000 creditReservations: { 4567 -> 343}

Order id : 4567 total: 343 state = OPEN

Eventually consistent credit checking

Message Bus

createOrder()

Publishes:Subscribes to:

Subscribes to:

publishes:

OrderCreatedEvent

CreditReservedEvent

OrderCreatedEvent CreditReservedEvent

Page 37: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Now there are two problems to solve….

Page 38: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Problem #1: How to design eventually consistent business logic?

More on that later….

Page 39: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Problem #2: How to atomically update database and publish an event

Order Service

Order Database

Message Broker

insert Order

publish OrderCreatedEvent

dual write problem

?

Page 40: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Failure = inconsistent system

Order Service

Order Database

Message Broker

insert Order

publish OrderCreatedEvent

X

Page 41: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Two-phase commitX

Page 42: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Agenda

Events on the outside

Events on the inside

Events at the core with event sourcing

Designing event-centric domain model

Page 43: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Just publish events

Application

Database

Message Broker

update

publish

X

Page 44: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Event sourcingFor each aggregate (business entity):

Identify (state-changing) domain events

Define Event classes

For example,

ShoppingCart: ItemAddedEvent, ItemRemovedEvent, OrderPlacedEvent

Order: OrderCreated, OrderCancelled, OrderApproved, OrderRejected, OrderShipped

Page 45: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Persists events NOT current state

Order

status ….

101 ACCEPTED

Order tableX

Page 46: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Persists events NOT current state

Event table

Entity type Event id

Entity id

Event data

Order 902101 …OrderApproved

Order 903101 …OrderShipped

Event type

Order 901101 …OrderCreated

Page 47: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Replay events to recreate state

Order

state

OrderCreated(…) OrderAccepted(…) OrderShipped(…)

Events

Periodically snapshot to avoid loading all events

Page 48: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

The present is a fold over history

currentState = foldl(applyEvent, initialState, events)

Page 49: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Events at the coreService

Aggregate A

Aggregate B

Aggregate C

Event X

Event YEvent ZEvent Z

Event Y

Page 50: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Domain logic = event-driven aggregates

CustomerAggregateOrder

Aggregate

Order CreatedEvent Check Credit

Command

Credit ReservedEvent

Approve OrderCommand

Create OrderCommand

External request

Emits events Event ⇒ Command

Page 51: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Request handling in an event sourced application

HTTP Handler

Event Store

pastEvents = findEvents(entityId)

Order

new()

applyEvents(pastEvents)

newEvents = processCmd(SomeCmd)

saveEvents(newEvents) (optimistic locking)

Order Service

apply(newEvents)

Page 52: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Event Store publishes events consumed by other services

Event Store

Event Subscriber

subscribe(EventTypes)

publish(event)

publish(event)

Customer

update()

Customer Service

Page 53: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Event Store publishes events consumed by other services

Event Store

Event Subscriber

subscribe(EventTypes)

publish(event)

publish(event)

CQRS View

update()

Service Xyz

send notifications

…

Page 54: Events on the outside, on the inside and at the core (jfokus jfokus2016)

Event store = database + message broker

Hybrid database and message broker

Implementations:

Home-grown/DIY

geteventstore.com by Greg Young

http://eventuate.io (mine)

Event Store

Save aggregate

events

Get aggregate

events

Subscribe to events

Page 55: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Benefits of event sourcingSolves data consistency issues in a Microservice/NoSQL based architecture

Reliable event publishing: publishes events needed by predictive analytics etc, user notifications,…

Eliminates O/R mapping problem (mostly)

Reifies state changes:

Built in, reliable audit log,

temporal queries

Preserved history ⇒ More easily implement future requirements

Page 56: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Drawbacks of event sourcing…

Requires application rewrite

Weird and unfamiliar style of programming

Events = a historical record of your bad design decisions

Must detect and ignore duplicate events

Idempotent event handlers

Track most recent event and ignore older ones

…

Page 57: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

… Drawbacks of event sourcing

Querying the event store can be challenging

Some queries might be complex/inefficient, e.g. accounts with a balance > X

Event store might only support lookup of events by entity id

Must use Command Query Responsibility Segregation (CQRS) to handle queries ⇒ application must handle eventually consistent data

Page 58: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Agenda

Events on the outside

Events on the inside

Events at the core with event sourcing

Designing event-centric domain model

Page 59: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Use the familiar building blocks of DDD

Entity

Value object

Services

Repositories

Aggregates ⟸ essential

Page 60: Events on the outside, on the inside and at the core (jfokus jfokus2016)

About AggregatesGraph consisting of a root entity and one or more other entities and value objects

Each core business entity = Aggregate: e.g. customer, Account, Order, Product, ….

Reference other aggregate roots via primary key

Often contains partial copy of other aggregates’ data

Order

OrderLine Item

quantity productId productName productPrice

customerId

Address

street city …

Page 61: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Domain model = collection of loosely connected aggregates

Order

OrderLine Item

quantity

…

Addressstreet city …

Customer

Productname price

Page 62: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Product service

Customer serviceOrder service

Easily partition into microservices

Order

OrderLine Item

quantity

…

Addressstreet city …

Customer

Productname price

Page 63: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Transaction = processing one command by one aggregate

No opportunity to update multiple aggregates within a transaction ⇒ event driven eventual consistency between aggregates

If an update must be atomic (i.e. no compensating transaction) then it must be handled by a single aggregate

Therefore, aggregate granularity is important

Page 64: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Product service

Customer serviceOrder service

Transaction scope = service

Order

OrderLine Item

quantity

…

Addressstreet city …

Customer

Productname price

Page 65: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Aggregate granularity

Consistency Scalability/ User experience

Customer

Order

Product

Customer

Order

Product

Customer

Order

Product

Page 66: Events on the outside, on the inside and at the core (jfokus jfokus2016)

Designing domain eventsRecord state changes for an aggregate

Part of the public API of the domain model ProductAddedToCart

id : TimeUUID senderId: UUID productId productName productPrice …

Required by aggregate

Enrichment: Useful for

consumers

Event metadata

Page 67: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Example event

Page 68: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Designing commandsCreated by a service from incoming request

Processed by an aggregate

Immutable

Contains value objects for

Validating request

Creating event

Auditing user activity

Page 69: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Example command

Page 70: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Various programming models

“Traditional Java” mutable object-oriented domain objects

https://github.com/cer/event-sourcing-examples/tree/master/java-spring

Functional Scala with immutable domain objects

https://github.com/cer/event-sourcing-using-scala-typeclasses

Hybrid OO/Functional Scala with immutable domain objects

https://github.com/cer/event-sourcing-examples/tree/master/scala-spring

Page 71: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

OO = State + Behavior

creditLimit creditReservations : Map<OrderId, Money>

Customer

List<Event> process(CreateCustomerCommand cmd) { … } List<Event> process(ReserveCreditCommand cmd) { … } … void apply(CustomerCreatedEvent anEvent) { … } void apply(CreditServedEvent anEvent) { … } …

State

Behavior

Page 72: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Familiar concepts restructured

class Customer {

public void reserveCredit( orderId : String, amount : Money) {

// verify

// update state this.xyz = … }

public List<Event> process( ReserveCreditCommand cmd) { // verify … return … new CreditReservedCredit(); }

public void apply( CreditReservedCredit event) { // update state this.xyz = event.xyz }

Page 73: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Kanban board example

Page 74: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

ArchitectureCreate/update boards

and tasks

Change notifications Materialized views

Event Store

Page 75: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Demo

Page 76: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

Summary

Events are a central to modern applications

Events integrate applications

Events maintain data consistency in a microservices architecture

⇒

Build events into the core of your application using event sourcing

Page 77: Events on the outside, on the inside and at the core (jfokus jfokus2016)

@crichardson

@crichardson [email protected]

http://eventuate.iohttp://plainoldobjects.com http://microservices.io

Questions?


Inside Outside Text

Fiat Outside Sortimo Inside

Looking Outside. Thinking Inside

Inside-Outside Merleau PontyYoga

Inside outside abetween

Nissan Outside Sortimo Inside

Inside Outside (Pp Tminimizer)

Accordion inside & outside

Looking Inside / Looking Outside

MERCEDES-BENZ OUTSIDE – SORTIMO INSIDE · MERCEDES-BENZ OUTSIDE – SORTIMO INSIDE Made for professionals

Peugeot Outside Sortimo Inside

Inside Outside

Outside Front Outside Flap GLUE AREA Inside Flap Inside ... · Outside Front Outside Flap GLUE AREA Inside Flap Inside Pocket GLUE AREA . Created Date: 7/6/2018 12:03:11 PM

Outside In Inside Out

Ford Outside Sortimo Inside

OUTSIDE AND INSIDE LIQUIDITY∗

Inside outside, outside in

Respiratory System Discovery Stations. 1. The Lungs Inside/Outside Outside Lungs Inside Lungs

Learning inside, learning outside

Inside and outside geometry

T_Looking outside/Looking inside

November 2013- Outside Inside

STRANGE NOVITÀ - Ceramica Artistica Due · inside nocciola outside nocciola inside bianco outside bianco inside grigio outside grigio inside sbiancato outside sbiancato inside nero

Inside outside in between

VALUABLE OUTSIDE. INVALUABLE INSIDE. · 2019-10-03 · VALUABLE OUTSIDE. INVALUABLE INSIDE. BEST OF THANE OUTSIDE. SHAPOORJI PALLONJI INSIDE. ... Mumbai Domestic Airport 300 m 3.6

Page-04 · 35 16" 215 8 7 16 " 9 10 11 cl outside inside outside inside outside inside primary door secondary door 21 2" 23 16" 35 16" outside inside outside inside 215 16" 41 16"

Inside, Outside, In Between

Inside, Outside, Inbetween

Outside/Inside school shots

INSIDE OUTSIDE - USModernist

inside, outside and inbetween

Inside, Outside & In-between

EMERGING ISSUES: INSIDE VERSUS OUTSIDE MANAGED LANES€¦ · EMERGING ISSUES: INSIDE VERSUS OUTSIDE MANAGED LANES. FINAL Inside Versus Outside Managed Lanes January 2010 Atlanta Regional

UFFO: Outside / Inside

Inside and outside