modularity and domain driven design - a killer combination - t de wolf & s van den ende
DESCRIPTION
Applying domain driven design in a modular fashion has implications on how your data is structured and retrieved. A modular domain consists out of multiple loosely coupled sub-domains, each having their own modular schema in the database. How can we migrate and evolve the database schema’s separately with each new sub-domain version? And how do we match this with reporting and cross-domain use cases, where aggregation of data from multiple sub-domains is essential? A case study concerning an OSGi-based business platform for automotive services has driven us to solve these challenges without sacrificing the hard-worked-on modularity and loose coupling. In this presentation you will learn how we used Modular Domain Driven Design with OSGi. 'Liquibase' is elevated to become a first class citizen in OSGi by extending multiple sub-domains with automatic database migration capabilities. On the other hand, 'Elasticsearch' is integrated in OSGi to become a separate search module coordinating cross-domain use cases. This unique combination enabled us to satisfy two important customer requirements. Functionally, the software should not be limited by module boundaries to answer business questions. Non-functionally, a future-proof platform is required in which the impact of change is contained and encapsulated in loosely coupled modules. Bios: Tom De Wolf Tom is currently technical lead of a Java software factory delivering multiple projects for multiple clients. His background skills include software architecture, designing & implementing enterprise systems using the Java eco-system. He has specialised in building OSGi based platforms using Agile methodologies and at the same time shaping a technological base platform for future ACA projects. His background as a PhD. in Computer Science at the Katholieke Universiteit Leuven enabled Tom to travel the world and present at multiple international conferences. Stijn van Den Ende Stijn is CTO at ACA IT-Solutions. His background skills include software architecture, designing & implementing enterprise class systems based on XML & WebServices, Enterprise Application Integration and B2B integration. He also has a strong background in agile methods. Instructing Java, Patterns, Architecture & XML courses is another way of his involvement with the JEE technology. He advises customers with regards to their enterprise system blueprints, and how to streamline their development process while defining and maintaining the technical roadmap for ACA IT-Solutions.TRANSCRIPT
MODULARITY AND DOMAIN DRIVEN DESIGN
a killer combination?
Tom De Wolf Architect
[email protected] @tomdw
Stijn Van den Enden CTO
[email protected] @stieno
www.aca-it.be
initial development maintenance phase
Software Design Customer SatisfactionSeparation of Concerns
Low coupling High Cohesion
B AB
A• Change A impacts all
modules = costly • Change B requires split
of module = costly
• Change A only impacts other module if api change
• Change B limited to module
Encapsulate Source of Change
Predictable Cost of ChangeConstant change Business Driven
Aim for 1-on-1 mapping from business changes onto software constructs
Source of Change = Business
Functional Modularisation
Vehicle
Option
Make
Warranty
Model
Transport
Location
Domain Driven Design Without modularity
BusinessPartner
AddressContact
consumer supplier
transports
owner
from to
Vehicle
Option
Make
Warranty
Owner
Model
Transported Item
Transport
Transport Supplier
Transport Consumer
Vehicle Context Transport Context
Domain Driven Design Modularity with Bounded Contexts BusinessPartner AddressContact
Business Partner Context
Locationfrom to
transport
vehicle
MODULAR DATABASE SCHEMA
• cross domain database structures not allowed
• queries cannot cross domain boundaries
NO foreign key
business partner
UUID MAKE MODEL OWNER_ID
djdfjfdkjdk221 Ford C-Max BE1234567
TRANSPORTED_ITEM_ID FROM CONSUMER_IDdjdfjfdkjdk221 454 BE1234567
LOCATION_ID ADDRESS454 Sunstreet 23
foreign key
VAT_NUMBER NAME
BE1234567 Company A
BENEFITS …
• Domain modules can migrate independently
to next version!
• Database schema is internal to the domain
bundle, i.e. NOT part of the API!
• Persistence and/or database technology can differ between modules in one system!
M modular database migration
cross domain transactionsTxS cross domain search and reporting
CHALLENGES
Mmodular database migration
MDev Machine 1
MIGRATION CHALLENGE
Dev Machine 2
Continuous Integration
Test environment
Acceptance environment
Production environment
Version 1.0.0
Version 1.0.1
Version 1.0.2
Version ?
Version ?
Version ?
environment x
Module 1 Module 2 Module 3Version 1.0.0
Version 3.0.0
Version 2.0.2
Manually track which scripts have run on which environment for which module ?
MLIQUIBASE
DATABASECHANGELOG table to track executed changesets for each environment
In versioned source codeXML based DSL for changeSets
www.liquibase.org
MMODULAR LIQUIBASE
transportvehicle business partner
deployment 1
deployment 2
deployment 3
Migrate modules separately Migrate @deploy of module
migrate to initial version migrate to initial version migrate to initial version
no db changes, no migration migrate to version 2 migrate to version 2
migrate to version 3
MTHE EXTENDER PATTERN
bundle A
bundle B
bundle C
bundle D
extender bundle
Extension Pattern ? !
no match/OSGI-INF/liquibase/db.changelog-master.xml
osgi-liquibase-extender
framework dependencyLiquibase
change sets domain A
change sets domain B
change sets domain C
• Only extender depends on Liquibase framework
• Update of single bundle triggers Liquibase update
• New Liquibase process for each matching bundle
Tcross domain transactionsx
JTA OR NOT?
• No persistence layer — direct jdbc/sql access — 1 datasource
• Multiple datasources
• Different types of persistence layers (JPA, NoSQL, …)
• JPA persistence layer in each domain bundle
• instead of one big persistence unit
• even on 1 datasourceT1 transaction spanning multiple modular domains
JTA not needed
JTA required
x
MODULAR PERSISTENCE
T osgi-datasource XA enabled
vehicle-domaintransport-domain
javax.transaction.TransactionManager
transaction provider e.g. atomikos
javax.sql.DataSource
persistence context
persistence context
commitx
Scross domain search
Scross domain search
SEARCH the tale of the evil joins
NOcross
module search
A. Search encapsulated in a separate module leveraging the functionality of other domain modules
search-module
domainA
domainB
x
x
Repository
Repository
Stop
• requires specific implementation for every search • complexity increases with number of modules • inability to leverage power of the datastore
SearchAPI
domainA
B. Search is using a separate query model
search-module
domainB
SearchAPI EventProvider
Update Events
Update
$ curl -XPUT ‘localhost:9200/vehicle/external/1’ -d ‘{ “make”: “BMW”, “VIN”: “30203232012102”}’
index document_type
id
Index a document
$ curl -XPOST ‘localhost:9200/vehicle/external/1/_update’ -d ‘{ “doc”: {“make”: “Alpina”}}’
Update a document
index document_type
id
$ curl -XPOST localhost:9200/vehicle/_search?pretty -d ‘{"query":{"match":{"make":"BMW"}}}'
index
Querying an index
$ curl -XPOST localhost:9200/vehicle/_search?pretty -d ‘{"query":{"match":{"mark":"BMW"}}}'{ "took" : 3, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 2, "max_score" : 0.30685282, "hits" : [ { "_index" : "vehicle", "_type" : "external", "_id" : "1", "_score" : 0.30685282, "_source" : {"make": "BMW", "VIN": "30203232012102"} }, { "_index" : "vehicle", "_type" : "external", "_id" : "2", "_score" : 0.30685282, "_source" : {"makes": "BMW", "VIN": "30203232012112"} } ]}}!
Querying an index
• Distributed (shards/replicas)
• Advanced Quering (lucene/geo/aggregation)
• Facets (terms/ranges/histogram)
• API (HTTP/Java/Java Testing)
• …{
1Populate the index
1Populate the index
package be.vabfs.search.api;!public interface SearchIndexService { void update(Object entityToUpdate); void delete(Object entityToRemove);! void populate();! void clear();!}
search
SearchIndexService
domainB
Entity
EntityListener
@PrePersist @PostUpdate @PostRemove
1Populate the index
search
SearchIndexService
update(entity)
Synchronise on Transaction
Track updated entities
A
Map entities to searchData
B
Bulk update search index
C
1Populate the index
Map entities to searchData
B
package be.vabfs.search.api.data;!import java.util.List;!public interface SearchDataProvider {! Class<? extends Object> getEntityClass(); List<SearchData> index(Object entity);!}
domainB
Entity
SearchData
1Populate the index
Map entities to searchData
B
SearchDataProvider
domainB
Entity
SearchData
1Populate the index
A
search
SearchIndexService
update(entity)
Synchronise on Transaction
Track updated entities
Bulk update search index
C
Map entities to searchData
B SearchDataProvider
domainB
A
2Enhancing Search
2Enhancing Search
2Enhancing Search
search
SearchService
package be.vabfs.search.api;!import …!public interface SearchService { …! SearchResults query(String query, int start, int rows, String sort, String sortOrder, String documentType); List<String> options(String field, String... documentTypes); List<SearchCriterion> availableCriteria( String... criteriaCategories); }
2Enhancing Search
package be.vabfs.search.api.criteria;!import java.util.List;!public interface SearchCriteriaProvider { String getProviderCategory(); List<SearchCriterion> getAvailableCriteria();}
package be.vabfs.search.api.criteria;!public interface SearchCriterion {! String getName(); String getKey(); String getUnitName(); SearchCriterionType getType(); SearchCriterionQueryType getQueryType(); String getDocumentType();!}
"vehicle.mileage""mileage"
SearchCriterionType.NUMBERSearchCriterionQueryType.RANGE_NUMBER
"SERVICE_ITEM"
"km"
2Enhancing Search
search
SearchService
domainB
SearchCriteriaProvider
domainB
LESSONS LEARNED
@deployment time migration = RISK
• have CI build continuously migrate current production data
Refactor wrong modularisation requires unwanted migration dependencies
• reset/flatten migration change sets to restore modularisation
Migration
Cross domain search and reportingEffort to integrate search is limited
• due to dynamic osgi service model
Advanced search functionality is possible
WHAT IS NEXT?
Multiple module versions active? Multiple schema versions active?
• e.g. feature try-out to limited customer base
Downgrade to older module version?
• Previous schema with new data?
Hot deploy while users are firing requests?
• Migration still busy = failure
Migration
Cross domain search and reportingDomain Specific Query Language
!
Exploiting elastic search capabilities beyond search, e.g. reporting
MModular migration
Liquibase extender @ DeployTCross domain transactions
Distributed JTA transactionsx SCross domain search
Elasticsearch view
Functionally not limited by domain module boundaries to answer business questions
Non-functionally future-proof platform with impact of change contained in loosely coupled domain modules
Tom De Wolf Architect
[email protected] @tomdw
Stijn Van den Enden CTO
[email protected] @stieno
www.aca-it.be