the current messaging landscape

Messaging

Events Everywhere

Michael LaingArchitect, Edge Engineering

New York Timesmichael.laing@nytimes.com

We’ll talk about:• nyt aбrik (my project)⨍• Messaging• Messaging Open Source

Software• Use cases (Events/Demo)

A Global Mesh with a Memory

Millions of users

Dozens ofinternal clients

1988:(Harvard)

30,000

2020:(Global) 50,000,000,000

We’ll talk about:• nyt aбrik (my project)⨍• Messaging• Messaging Open Source

Software• Use cases (Events/Demo)

Messaging:

• What is it?• Why should I care?

Message

Event

Metric

Header

Body

Why should I care? Decoupling

by wmasterj and Fabrice TIERCELIN - Wikipedia

Why should I care? Decoupling

How does this happen?• Asynchronous interaction

A BMessages

Instances

Processes

Functions

AsyncScope

Time

P C

Idle

Idle

Idle

sync

async

Time

P C CP

Idle

Idle

Idle

Idle

sync vslatency

We’ll talk about:• nyt aбrik (my project)⨍• Messaging• Messaging Open Source

Software• Use cases (Events/Demo)

Broker

LAN

WAN

: + flexibility

LAN/WAN

DatabaseSemanticsFor access

:

+ scalability

GW GW …GW GW GW …GW

Region A Region B …

GW

Push Pullsnd rcv

µSvc …

: + IoT

2020:(Global) 50,000,000,000

:Written in C

+ IoT

GW

Push Pullsnd rcv

µSvc …∅mq

: + very fast∅mq

GW

Push Pull snd rcv

µSvc …NSQ

: + fastNSQ

GW

Push Pullsnd rcv

µSvc …NSQ

What if…

GW GW …GW GW GW …GW

Region A Region B …

Then:

… …

Region A Region B …

GW GW GW GW GW GW

Becomes:

: + offsets

We’ll talk about:• nyt aбrik (my project)⨍• Messaging• Messaging OSS• Use cases (what about

Events & Demo? Hang on…)

: + flexibility

: + very fast∅mq

: + fastNSQ

: + IoT

: + offsets

“Mobile is about moments, shorter and more fragmented. It’s important we string these together,” said Paul Muret, the vice president for display, video and analytics at Google.

“We need to understand the desires of consumers in each point in time to understand their context and intent.”

Events

2020:(Global) 50,000,000,000

Observations• SQL considered harmful for global

online distributed systems• Web is dead• Mobile is maturing• The excitement: Things & Events

Event Sourcing

…a design pattern that captures all changes to an application's state as a sequence of events…

Event Sourcing

…synchronizes in distributed systems around the concept of optimistic concurrency…

Use Case: XWORDCharacteristics• Multi-user• Distributed

Globally• Clients can

disconnect

Implementation• Message Bus• NoSQL

Michael LaingArchitect, Edge Engineering

New York Timesmichael.laing@nytimes.com