decomposing the monolith using microservices that don't give you pain
TRANSCRIPT
Dennis Doomen | @ddoomen | Aviva Solutions | The Continuous Improver
Also known as the Microservices Premium
Hybrid of Front-end
Technologies
Multiple patterns for the same problems
Long compile time
Large source control
repositories
Long-running unit tests
Too much coupling
Limitedteam scalability
Devs need to know everything
No
Isola
tion
Productivity drops over time.
• The network is reliable
• Latency is zero
• Bandwidth is infinite
• The network is secure
• Topology doesn't change
• There is one administrator
• Transport cost is zero
• The network is homogeneous.
https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing
The First Law of Distributed Computing
Very scalable, very reliable
Very difficult to debug
distributed problems
Unreliable network requires
complicated techniques
Requires message
bus/broker
End-to-end testing very unpractical
Very mature DevOps culture is a prerequisite
Advanced monitoring and
visualization is a must
Security can not be an afterthought
High operational maintenance.
Can be deployed/upgraded
independently.
Monolith
Microservice
Microservice
Microservice
HTTP API
HTTP API
HTTP API
OWIN component released through
MyGet/Nuget
No network I/O at all
Can have its own database instance, shared with monolith,
or shared storage service
Microservice ownsthe schema
Separate repos with distinct owners.
Runs in-process, thus easier debugging
New version requires
redeployment of the monolith.
Loose coupling through HTTP
Less scalability options
HTTP is not the fastest
serialization format
Relies on libraries, not frameworks.
Client
IIS
Console
WinSvc
Unit Test
HTTP request
HTTP response
Middlewareenvironment
handler
next
Middleware
IDictionary<string, object>
Func<IDictionary<string, object>, Task>
Task
Func<Func<IDictionary<string, object>, Task>, Func<IDictionary<string, object>, Task>
>
Monolith
Microservice
HTTP API
Service Registry
Contract documented through
Swagger
Webhooks
Microservice
Microservice
Provides OWIN-aware HttpClient to connect to service
Event payload exposed as Swagger
(e.g. SwaggerHub)
Subscribers register URLs or in-process
OWIN message handler
API/payload consumers use liberal JSON interpretation
Circuit Breaking to handle unreliable
consumers
At-least-once payload delivery
Subcribers can request replays.
Payload includes ID to help idempotency
Each request requires correlation ID that flows
through all APIs, webhooksand services
Aggregates
Command Handlers
Queries
Commands
Events
Upconverters
Value Objects
Query Handlers
SchemaMigrations
Projectors
DataImporters
Query API
Jobs
Projections
HTML
CSS
JS
API Controllers
Projectors
Projections
RegistrationsWebhook API
Command API
Event Storage
Payload Definitions
Outer layers depend on inner layers
Domain is persistency ignorant
Can be tested independently
For eventual consistent business
rulesMaster data
To support domain queries
For microservices that provide a (partial) UI or
management screen
Based on Domain Driven Design
Supported by LibLog, TinyIoc
Designed according to the 12 Factor
principles
UI / HTTP APICommand
Service
ChangeUserEmailHandler
UserUnit of Work
User Projector
DAL
Read DB
Write DB
ChangeUserEmailCommand
Execute query
Get<User>(identity)
Invoke method
Event Store
Load(events)
Apply
Get changes
Dispatcher Events
Handle(UserEmailChangedEvent)
Submit changes
History
Dennis Doomen | @ddoomen | The Continuous Improver
Monolith
Microservice
HTTP API
Event Store
Events
Payload
Projector
Subscription Request
Payload ‘event store’
Projectors use LiquidProjections
Have their own checkpoints
Projects fine-grained events into payload
projections
Projector
Payload schema is documented using
Swagger
ProjectorProjector
Webhook Projectors
Separate instance per subscription
Subscriber HTTP API
‘Project’ payload ‘event store’ into
HTTP calls
Projected Data
Uses the incoming payloads as an ‘event
store’
Payload
Use Circuit Breakers to handle
slow/unavailable subscribers
SQL DB Schema changes using
FluentMigratoror using NoSQL
DB
Domain
Event Store
Events
App
RDBMS
EventsProjection
EventsProjectionProjection
ProjectorOptimized for
specific queries
Separate projections
database NoSQL
Projector
Projection-specific storage
Projector
HTML
Raw SQL or Dapper
Run asynchronously
Great for sharding
Dennis Doomen | @ddoomen | The Continuous Improver
Events
Transaction 6
Transaction 5
Transaction 4
Transaction 3
Transaction 2
Transaction 1
Temporal Projector
Read Store
Time
Projected until this point
Immutable, thus auditable and SOX
compliance
Dennis Doomen | @ddoomen | The Continuous Improver
Domain
Event Store
Events
App
EventsProjection
EventsProjectionProjection
ProjectorApplication projections
RDBMS
Reporting Projector
Traditional reporting model
Asynchronous
OLAP
Dennis Doomen | @ddoomen | The Continuous Improver
StreamId: User-Dedo, Rev: 5
Persisted State Changes
StreamId: User-Dedo, Rev: 4
Grant Role “Editor”
Change Password 2nd
StreamId: User-Dedo, Rev: 4
StreamId: User-Dedo, Rev: 3
Create User
Grant Role “Owner”
Add Phone Number
Change Password 1st
StreamId: User-Dedo, Rev: 5
Revoke Role “Owner”
StreamId: User-Dedo, Rev: 6
Grant Role “Editor”
Time
StreamId: User-Dedo, Rev: 7
Changed Password 2nd
Dennis Doomen | @ddoomen | The Continuous Improver
Events
1
2
3
6297
6298
6298
Projections
Documents
Groups
Users
Objects
Folders
Microservice V2
Changes Queries
Events
1
2
3
6299
6300
6301
Projections
Documents
Groups
Users
Objects
Folders
Microservice V1
ChangesQueries
Migration Process
Dennis Doomen | @ddoomen | The Continuous Improver
Events
Aggregate Root
Entity
Entity
Value Object
Aggregate Root
Aggregate Root
Aggregate Root
Entity
Value Object
Value Object
Dennis Doomen | @ddoomen | The Continuous Improver
• Asynchronous projections
• Idempotency of projections
• Self-tracking
• Projection autonomy -> more duplication
• Projection tracking and prediction.
• Functional archiving
• Clear separation between critical and auxiliary data -> prioritization
• Partitioning
• Dynamic rebuilding (after bugs, schema changes, etc).
• Projections that crash– Column constraints
– Changes in data invariants
– Unexpected projection dependencies
– Lookups vs stream dependencies.
• (Unexpected) projection dependencies– Synchronous -> asynchronous and existing
bugs
– Aggregrate dependencies.
• Bugs – Causing large event streams.
Prerequisites
- Microservice owns ‘schema’
- Supports upgrading and downgrading
- NoSQL preferred, but RDBMS can work too.
Rebuild schema in-place
– Rebuilding using event store
– Requires tracking API
– Involves down-time.
Disallow breaking changes
– Only new ‘columns’ and ‘tables’
– Partial rebuilding using event store
– Requires forwards compatibility.
Rebuild side-by-side
– Rebuilding using event store
– Requires tracking API
– Requires compatibility with previous ‘schema’
– Previous ‘schema’ is removed in the next release.
Microservice
Bunch of classes that should be used directly
X
No inheritance
Uses composition over inheritance
internally
Convenience classes that don’t hide the
magic
Library
Abstract Package
Interfaces
Delegates
DTOs
Only depend on abstract packages…
Stable Package
…or depend on more stable packages
Auxiliary Package
Classes that are not used together do not
belong togetherOptional Dependency
DependencyPackage
Consumers should not be faced with
optional dependencies
• Delegates
• Source-only Packages
• Merge dependencies
Monolith
Microservice
Microservice
Microservice
HTTP API
HTTP API
HTTP API
Apply routing conventions
Versioning using Accept Headers,Response Headers or
Routes
Optimize body using AVRO or Protobuf
Cross-service tracing using OpenTracing
Replace OWIN with (upcoming) in-process gRPC
Common contracts for diagnostics,
monitoring
Add KPI dashboards, e.g. Sumologic
1 Build a (container-based) cloud capability.
2 Deploy monolith to the cloud.3 Automate your delivery pipeline.4 Full end-to-end responsibility5 Break-up delivery teams and code6 Evaluate status-quo on micro-services
(products, platforms, etc)7 Slide-and-dice microservices one-by-
one.
• Liquid Projections – The Examplehttps://github.com/liquidprojections
• Bliki: MonolithFirsthttps://dzone.com/articles/martin-fowler%E2%80%94bliki
• In-process gRPChttps://github.com/grpc/grpc-go/issues/906
• Protobuf in WebAPIhttp://www.infoworld.com/article/2982579/application-architecture/working-with-protocol-buffers-in-web-api.html
• Ingredients for well-design OWIN componentshttp://www.continuousimprover.com/search/label/OWIN%20Recipes
• The good, the bad and the ugly of Event Sourcinghttp://www.continuousimprover.com/search/label/event%20sourcing