The Importance of Data Model Change Management
March 8, 2017Joy RuffProduct Marketing [email protected]
2
Agenda
Enterprise data trends Development methodologies Communicating through data models Considerations for change management Sprint-based modeling activities Summary Q&A
3
Enterprise data trends
Increasing volumes, velocity, and variety of Enterprise Data30% - 50% year/year
growth
Decreasing % of enterprise data
which is effectively utilized5% of all Enterprise
data fully utilized
Increased risk from data
misunderstanding and non-compliance$600bn/annual cost
for data clean-up in U.S.
4
Evolving Database
Ecosystems
Volume, Velocity, Variety
Keeping pace with the rapid growth of data, change and compliance
Agile Development
Cycles
Maximizing IT
Infrastructure
ComplianceLimited Resources
Data Professionals Need the Right Tools
5
Waterfall vs Agile
Data Modeling
6
Data model usage & understanding
We don’t use data models
Other
Our data team does most data models but developers also build them as needed
Our database administrators own data modeling
Developers develop their own data models
We have a data modeling team that is responsible for data models
0% 10% 20% 30% 40%
13%
3%
16%
19%
31%
18% Completely understand
20%
Understand somewhat60%
Don’t un-derstand
17%
I don’t know3%
87%
What is your organization’s approach to data modeling?
How well does your organization’s technology leadership team understand the value of using data
models?
7
8
Why we need data models: Much more than a picture
Full Specification• Logical• Physical
Descriptive metadata• Names• Definitions (data dictionary)• Notes
Implementation characteristics• Data types• Keys• Indexes• Views
Business rules• Relationships (referential
constraints)• Value Restrictions (constraints)
Security classifications + rules Governance metadata
• Master Data Management classes
• Data quality classifications• Retention policies
9
Benefits of data modeling
Design• Manage redundancy• Integrate and rationalize• Increase quality
Use & Maintain• Increase discoverability• Improve comprehension
• Data dictionaries• Business glossaries
Province
Province ID (ProvinceID) NUMERIC(9,0)
Country ID (CountryID) (FK) NUMERIC(9,0)
Province Name (ProvinceName) VARCHAR(50)Province Code (ProvinceCode) VARCHAR(3)
Business Value: LowMaster Data Class: Reference
County
County ID (CountyID) NUMERIC(9,0)
Province ID (ProvinceID) (FK) NUMERIC(9,0)County Name (CountyName) VARCHAR(50)
Business Value: LowMaster Data Class: Reference
Municipality
Municipality ID (MunicipalityID) NUMERIC(9,0)
County ID (CountyID) (FK) NUMERIC(9,0)Province ID (ProvinceID) (FK) NUMERIC(9,0)
Elevation UOM (ElevationUOM) (FK) VARCHAR(10)
Municipality Name (MunicipalityName) VARCHAR(50)
Municipality Latitude (MunicipalityLatitude) NUMERIC(13,9)Municipality Longitude (MunicipalityLongitude) NUMERIC(13,9)Municipality Elevation (MunicipalityElevation) NUMERIC(9,3)
Business Value: LowMaster Data Class: Reference
Country
Country ID (CountryID) NUMERIC(9,0)
Country Name (CountryName) VARCHAR(50)Country Code (CountryCode) VARCHAR(3)
Business Value: LowMaster Data Class: Reference
Address
Address ID (AddressID) NUMERIC(9,0)
Address Type ID (AddressTypeID) (FK) NUMERIC(9,0)
Municipality ID (MunicipalityID) (FK) NUMERIC(9,0)Province ID (ProvinceID) (FK) NUMERIC(9,0)Country ID (CountryID) (FK) NUMERIC(9,0)
Address Line 1 (AddressLine1) VARCHAR(50)Address Line 2 (AddressLine2) VARCHAR(50)Address Line 3 (AddressLine3) VARCHAR(50)Address Line 4 (AddressLine4) VARCHAR(50)Postal Code (PostalCode) VARCHAR(20)
Business Value: LowMaster Data Class: Master
Address Type
Address Type ID (AddressTypeID) NUMERIC(9,0)
Address Type Name (AddressTypeName) VARCHAR(50)
Business Value: LowMaster Data Class: Reference
10
The Need for Common Understanding
11
Apply meaning with business glossaries
Maximize understanding of the core business concepts and terminology of the organization
Minimize misuse of data due to inaccurate understanding of the business concepts and terms
Improve alignment of the business organization with the technology assets (and technology organization)
Maximize the accuracy of the results to searches for business concepts, and associated knowledge
12
Data model change management considerations
Needs to work with any workflow style – not just sprint-based Fine-grained check-in and check-out capability Method to associate model changes to requirements and list
them in a change management control center Audit trail of changes made – what was done and why, to
demonstrate compliance for data governance Ability to compare models to databases and other models, and
identify changes that need to be merged into the source or target Capability to create branches from a model baseline and merge
them back in or roll back to restore a previous release Ability to generate the necessary DDL code to implement the
desired changes into the database
13
Agile data modeling considerations
Primary focus is enablement of the team• Can not be perceived as an obstacle/gatekeeper
Iterative work style• Managing changes during sprints• Implementing database changes with DDL
Collaboration is paramount• Cross-project focus• Enterprise data perspective
Traceability – what changed and why• Data lineage can show change impacts• Audit reporting for data governance
14
Managing changes during agile sprints
15
Start of sprint preparation
Participate fully in sprint planning Ensure there is a “Named Release” as of
completion of previous sprint• Always have a baseline for
compare/merge! Submodels
• Structure by relevant topic/subject area• At story level if necessary to facilitate
communication• Roll up to parent level submodels
16
In-sprint activities
Modeler fully engaged in daily stand-up meetings Model change workflow
• Model each change, associating with appropriate task/user story• Generate incremental DDL script(s) and post
• Use a robust script naming convention, particularly if utilizing automated build systems
Different work approaches• Some designs will be originated “pushed” by data modeler• Others may be “pulled” from developer “sandbox”
• Analyze, amend and “push” back out• Compare/merge and redesign as appropriate• Ensure developer uses the officially sanctioned script
• Use submodels for audience specific perspective• Use data model design patterns
Maintain the discipline!
17
End of sprint wrap-up
Create “Named Release” at end of Sprint• Serves as baseline for start of next sprint• Serves as baseline for comparison at ANY later point
Create delta DDL script by using compare/merge• Based on Named Release from end of the previous sprint
Create full database DDL script• Can be used to easily create “sandbox” databases quickly
Ensure the model(s) have been published Participate fully in sprint planning and retrospectives
• Lessons learned• Celebrate the successes
18
Summary
Important factors for effective data model change management• Working style
• Agile (sprint-based) or Waterfall• Collaborative rather than gatekeeper
• Consistency in communication• Data dictionaries• Business glossaries
• Traceability• Track why changes are made, not just what• Correlate with development changes• Implement model version control• Provides audit trail for compliance / governance