Simplify Your Life with CQRSFinding clarity and unity in a complicated world

Joel MasonTechnical Lead

Aesynt@jamason05

What will be in this talk?

Use CQRS to simplify data persistence Use CQRS to improve the “*-ilities” of applications Use CQRS to bring teams together Use CQRS to help define the behavior of our systems

What won’t be in this talk?

A lot of detail about how to implement CQRS into a system A lot of theory about the finer points of CQRS, DDD or any

other related practices An authoritative talk on CQRS…I’m a practitioner—nothing

less, nothing more

What is CQRS?

CQRS stands for Command Query Responsibility Segregation…At its heart is a simple notion that you can use a different model to update information than the model you use to read information.

– Martin Fowler

Is CQRS too confusing?So if this is the first picture you have ever seen…it probably seems so

Is CQRS too confusing?

In actual ity I would say no…but it can be made very complicated, very quickly.

Imagine a typical N-tier system…

PresentationInput Validation

Commands/Queries/Service CallsYet More Validation

Probably Business LogicWhich Probably Exist in Business Logic

Which Then Read and Write From A DALWhich Probably Exists as a Bunch of

DTO’s/Sprocs/Whatever

CQRS can be as simple as this…

PresentationInput Validation

Commands/Service CallsProbably Business LogicWhich Then Write to A DAL

Which Probably Exists as a Bunch of DTO’s/Sprocs/Whatever

Queries to Generate Views

…or it could be scaled out to something like this…“Hint: If your technical solution is more complicated than your original problem you're probably doing it wrong.” –Rob Ashton

Some CQRS Myths

Myth #1: CQRS == Event Sourcing

Myth #2: CQRS requires an eventual consistent read store

Myth #3: CQRS requires message queuing/bus/async calls

Myth #4: Commands are “Fire and Forget”

Myth #5: Read Models have to be eventually consistent

Myth #6: CQRS lets you escape consistency problems and eliminate concurrency violations

Myth #7: CQRS is easy

CQRS isn’t so much an architectural pattern…it’s a pattern of thought

“…[CQRS is not an answer], but is one of many questions to be answered” – Udi Dahan

CQRS broken in real life

• A “hard” inquiry impacts your score negatively

• …but wait a “soft” inquiry has no impact on your score…

Race Conditions Don’t Exist…In Business…

…but they do in Software

Here are a couple places where this has bit me

• Labels printed that contents must be incremented one at a time

• Easy to leave a hole in the timing where a request can come in mid-transaction

• Can result in wrong number being returned

Server side Appilcation

External Device

Requests Number for a label

Gets label number and then marks the records as ‘Printed’

Can be run multiple times from multiple places Also gets

incremented when a new order comes in

Oh…and it gets initialized when a particular action is taken

Example #1: Label Printing

Here are a couple places where this has bit me

• Every few days in the middle of the night users complain of application becoming unusable

• Errors increase due to DB deadlocks

• Support has heavier workload and is unable to do anything to relieve issues for users

Application

Application queries for normal use

An intensive ordering calculation kicks off that takes about 15 minutes to complete

Example #2: Reorder Calculation

How do we try to handle race conditions?

We can (try to) trick the timing…I’ve had this discussion

…or we sprinkle in some locks…

…maybe we blame the environment…

…perhaps we hope and pray…

…or sometimes we just ignore the problem.

What does CQRS suggest?

• Put a read model in for the labels

• Take in the command(s) and save them into an event store

• Process the event store periodically and modify the read model

Solution #1: Label PrintingServer side Appilcation

Write Store

External Device

Read Store

Initialization Increment

What does CQRS suggest?

• Have a process that transforms the data into a separate set of tables for the calculation

• Track inventory changes as they come in and save them in a way that helps the calculation

• When the calculation runs, store off the results in a model or models that support the downstream processes

Solution #2: Reorder Calculation

Server side Appilcation

Write Store

Read Store

Lots of inventory items

Items to Reorder

Optimize for your writes

• If events happen at different times, then save them in different tables

• If something happens a whole lot, then make it as quick as possible

• Modify your technologies to optimize for writes

Imagine an order at Starbucks…

• An order is placed• It is paid for• It is made• It is received

Traditionally we would make a table something like this…

OrderId

Size Type Milk Made Paid Received

But since we are optimizing for the writes we can do this…

This is about a separation of concerns in time instead of by types of data

OrderId Size Type

MilkPlacedOrder

OrderIdPaidOrder

OrderIdMadeOrder

OrderIdReceivedOrder

Optimize for your reads

• Introduce read models• Optimize your table

indexes for the queries you run

• Choose a different store all together

Now to optimize our read for Starbucks

Order

Made

Paid

Received

View

Simplify your supportability

• Having a copy of what you wrote is really helpful

• Having a copy of what you read is really helpful

CQRS, meet BDDA match made in acronym heaven!

Use CQRS and BDD to provide a vision for your team

Continuous attention to technical excellence and good design enhances agility. –Agile Principles

Focus on answering the “question” of CQRS

Interview your Product Owner on the behavior of a write and the behavior of a read

What is the behavior of a write?

What makes a write valid or invalid? How often do we anticipate this event

happening? Is there any tolerance for missing this

event? Do we need to be able to recover from

a missed event? What should we do if there is an error? Does this event have to provide

feedback to the user? How do we anticipate this event

impacting the system? How many of these events are going

to being happening at a time? Are these events coming from more

than one place?

What is the behavior of a read?

How often will this information need to be retrieved?

How up to date does this information need to be?

How fast does the information need to be returned to the user?

How much information is usually going to be returned in a typical query?

Who is going to be looking at this information?

What parts of this information need to be historical?

What is this information going to be used for?

Use what you find out to layout your behavioral tests

Given a set of labels that has just been initializedWhen a device requests labels to printThen no labels will be returned to the device

Given a label that has just been initializedWhen a new order comes inThen the label number is incremented by 1

Given a label that has just been initializedWhen a new order comes inThen the label number is incremented by 1

Take your behaviors and use these to help you drive your design

• Do you need your information to return very fast and you can handle at least a small amount of staleness? Maybe use a read model.

• Do you only ever need to run a calculation once a day and it can run most any time? Maybe you don’t need any optimization.

• Do you aggregate 500,000 records a day and another user may look at that information once a week? You probably need to optimize your writes and periodically calculate your read.

Twitter Case Study

• Before applying CQRS twitter would keel over now and then

• Optimized for the writes• Fanout pre-builds

everybody’s feeds• Works great except

when one celeb tweets another celeb

Drive your behavioral tests into component tests

• Component tests should be relatively simple and straight-forward

• Should be very little doubt from testers or developers about what is actually going on

• The fact that these are simple also is an indication that you are building something that is supportable

When shouldn’t I use CQRS as part of my architecture?

As a thought pattern…you probably should always at least answer the question

Are you doing something that really is simple? Are you doing something where only one actor is going to be

writing to a data store at a time? CRUD You have a user base or a business that can’t tolerate any

change

Thank You!Slides will be posted at www.slideshare.net/joelamason/

Joel MasonTechnical Lead, Aesynt@jamason05www.craftoverart.comjoelamason@gmail.com

Resources• http://martinfowler.com/bliki/CQRS.html• http://

codeofrob.com/entries/cqrs-is-too-complicated.html

• http://www.udidahan.com/2009/12/09/clarified-cqrs/

• http://www.udidahan.com/2011/10/02/why-you-should-be-using-cqrs-almost-everywhere

• http://lostechies.com/jimmybogard/2012/08/22/busting-some-cqrs-myths/

• http://lostechies.com/derickbailey/2010/03/08/cqrs-performance-engineering-read-vs-read-write-models/

• http://www.infoq.com/presentations/Real-Time-Delivery-Twitter