building distributed systems with ravendb nscbcon ny by oren eini & ayende rahien

Building Distributed Systems with

RavenDB Oren Eini / Ayende Rahien

Hibernating Rhinosayende@ayende.com

Distributed systems look like…

Where do I go?

Your distributed system…

RavenDB is…

• No SQL Document Database• .NET / JVM / Node.JS / Python / REST / etc clients• High performance• Scalable & distributed• Safe by default• ACID

• And those are just the highlights…

Use RavenDB for?

• Saga state• Timeout data• Subscription storage• Deduplication• Service Control

Use RavenDB for?

What about you!?

Entities & Aggregates

Complex object graphs

Static aggregation via map/reduce

Endpoint specific database

Distributed Data: the wrong way

Sharing data

Distributed data, properly!

HR Sales

Manufacturing

Inside an endpoint

Sales

Q

Msg handlers

Inside an Endpoint’s RavenDB Server

• Sales DB – all the data for this endpoint• HR DB, Mfg DB – Relevant data for this endpoint from another

location

• Physical separation of data. • Clear data ownership• Isolation

Data distribution

• RavenDB Replication• SQL Replication• Background ETL processes• Dedicated messages• Clear data ownership• What about duplication?

CQRS: The world via events

• Endpoints send event messages to subscribers.• Subscribers use events to create a view of the world that pertain to

their needs.• ShippmentArrivedEvents means to that we can increase our stock levels.

• How does it work with RavenDB?• Store events as messages.• Process via streams or via map/reduce.

Trivial example

• OrderProcessedEvent – generated by Fulfillment BC• OrderId• CustomerId• TotalAmount• Lines (product, quantity, etc)

• In Sales endpoint, need to know how much customer bought and what.

Map/reduce to the rescue

// mapfrom e in docs.OrderProcessedEventsfrom l in e.Linesselect new {

e.CustomerId, e.OrderDate.Year,l.Product,l.Count

}

// reducefrom r in results group r by new { r.CustomerId, r.Product, r.Year } into gselect new {

g.Key.CustomerId,g.Key.Year,g.Key.Product,Count = g.Sum(x=>x.Count)

}

Answers?

• Immediate (pre-computed)• Which customers bought which products this year?• Which customer didn’t buy product they previous bought?• Etc…

High availability

• RavenDB Replication

• Zero to crash in…

Load balancing

RavenFS

• Distributed, replicated file system• Large files• Many files• Optimized replication support

RavenFS & The Data Bus

• Source code: http://goo.gl/51MRWG

• Better (explicit)• Upload to Raven FS on endpoint #A• Replicate transparently to endpoint #B• Get notified & process file.

Small tricks

• My sagas are in RavenDB…• What can I do with that?

Out of the box with Service Insight

More interesting, reporting on top of your sagas…

• SQL Replication to dedicated reporting database• Map/Reduce on your sagas.• Live view of everything that goes on.

• Note, separate licensing

Questions?