geek sync i in database automation we trust
Post on 29-Jul-2015
42 Views
Preview:
TRANSCRIPT
In Database Automation We Trust
May 13th , 2015
2
About Yaniv Yehuda
Co-Founder and CTO of DBmaestro
Spent the last years raising awareness about the challenges around database development and deployment, and how to support database Continuous Delivery.
3
About DBmaestro
• The leading provider of DevOps for Database• Database development and deployment automation
4
Agenda
▪ Continuous Delivery is coming
▪ CD benefits
▪ But what about the database?
▪ Database version control
▪ Database deployment automation
▪ Database impact analysis
▪ Stopping the CD line
▪ Q&A
5
DevOps & CD: a must for every company
▪Every business is an IT business
▪Customers demand that you deliver new features faster
− Agile Development
− Process Automation
− DevOps
▪Can’t wait 6 months for that next waterfall release…
▪ If you don’t, your competitor probably will
6
Continuous *what*?
Continuous Integration
Continuous Delivery
Continuous Deployment
!
?
7
What is Continuous Integration?
• Principles and practices
• Been around for a while
▪ Focus on streamlining development ￭ Developers integrate code into shared repository￭ Each check-in is verified￭ Automated builds￭ Automated tests￭ High visibility – a feedback loop
▪ Easier & quicker to find problems, less back tracks => short integrations
8
And Continuous Delivery?
▪ Next step after continuous integration
▪ Becoming lean, and even more Agile￭ Make sure each change is releasable
• Develop-> build-> test-> move to staging-> acceptance test￭ Build a process to release with a push of a button
• Deploy to production-> test production
▪ Actual deployment to production in manually actuated
=> Ensure risk mitigation and high efficiency
9
Continuous Delivery
Image from Wikipedia
Fail
Pass
10
Continuous Deployment
• Automating changes all the way to production￭ Develop-> build-> test-> move to staging-> acceptance test->
deploy to production-> test production
• Makes sense for Facebook, Amazon, i.e. - for large SAAS solutions.￭ A/B test new features￭ Throttle traffic to new releases etc…
▪ Not for everyone…
11
How Do I Measure Success?
• More rapid changes
• Fewer changes backed out
• Better collaboration
• Fewer defects
• Ultimately better service
• Happy customers
• Profitability
12
Continuous Delivery moving ahead!
▪ Recently Conducted DBmaestro Survey￭ Over 350 participants
13
Why Continuous Delivery?
14
But…what about the database?
15
What about the database?
16
Down to 14% !!!
• Based on the questions they answered, only 14% from the ones reported doing CD for DB, are actually performing basic CD practices!!!
• The rest are plugging the automated process with various manual steps…
• Why?• What is so special about the database?
17
The Weakest Link In a Chain ???
•Old adage but true• The database is often neglected and therefore can
become the weakest link• Manual processes
•Database/Code Silos exist…• Don’t always communicate effectively• Need to follow same procedures & best practices
•Essential from a compliance and business point of view
•Should be the strongest link
18
A quick poll
How often do you or others make changes to your development databases?
• Daily
• Few times a week
• Once every few weeks
• Almost never
19
The Database is a constraint
20
CD for the database
• In many organizations, changes to databases are not included in the CD picture
• Too much risk – seems to be very challenging
• Too little awareness of the packaging, deployment, promotion etc. concepts used for application code
• Insufficient tool support for these concepts
21
Manual steps in an automated process…
• At first you think: It is easy! We can handle it!
• Then your start fighting a loosing game…
• And then you have to speed up!!!
22
So why not move forward?
• Mistrust…
23
What is the problem?
• Root Causes for issues:– Challenging manual version control process– Static deployments code overrides– Dynamic deployments unaware of version control– No release automation red-flags – don’t know when to stop the line…
24
Another quick poll
How do you source control your database changes?
• We use an automated/dedicated database source control solution
• We manually save scripts to an application-code source control solution
• We keep track with excel sheets and other lists
• We don't
25
Two isolated processes
•Version Control Process (file based)
Development Process
Check-Out Script
Modify Script
Get updated Script from DB
Check-In Script
Compile Scriptin DB
Debug Scriptin DB
?
??
?
A
A’
Version control repository is out of sync from the tested database and cannot act as a Single Source of Truth
26
Scripts & version control
▪Challenges:− Code-overrides
− Working on the wrong revisions
− Scripts do not always find their way to the version control solution
− Out of process updates go unnoticed
− Hard to locate outdated update scripts
▪Playing safe? What we really need: − The actual code of the object
− The upgrade script
− A roll-back script
27
86%Rate this as a risk Factor
64%Rate this as a HIGH risk Factor
Scripts… Build Once Deploy Many
X1.11.1.11.11.21.31.41.51.61.7
Int QA Stage Prod
Database Deploy ScriptEnvironment
Re-Base (due to defects)
DevDev
DevModel
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.11.11.41.7
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.61.6 1.7
Out of Process Change
X
X
X
XX
? 1.1.1
X
a
a
28
29
Scripts are static…
▪ Scripts
− Hard to test in their entirely (holistically)
− Hard to test due to colliding dependencies
− Need to run in a specific order…
− Much harder to deal with project scope changes
▪ Scripts, unless super sophisticated:
− Unaware of changes made in the target environment
− Time passed from their coding to the time they are run
− Potentially overriding production hot-fixes or work done in parallel by another team
30
60%of those manually building scripts have
to fix or tweak them regularly as part of a deployment process
31
89%Rate this as a risk Factor
60%Rate this as a HIGH risk Factor
32
Gaining Trust!
Coordinated Process Traceability
Start in the Beginning
No Out-of-Process Changes
Impact Analysis
Automation
Task Based Development
Well Defined Processes
33
Version control: Integrated & Enforced process
34
Dealing with challenges…
▪Integrated Database Version Control process
− Leverage proven version control best practices • Forcing check in & out for changes• Labels• etc..
− No code-overrides
− Always working with the correct revision
− All changes are documented
− Always know who did what, when, why and from where
− No out-of-process changes
▪ No time spent on manual coding of the change scripts
1.11.21.31.41.51.61.7
*
Int QA Stage Prod
Database Deploy Script
Environment* Execute the same script being executed at the Stage environment
Re-Base (due to defects)
DevDev
DevModel
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.1 1.4
1.4 1.7
1.1.1 1.7
1.11.1 1.1
1.41.7
File Based Version Control
Out of Process Change
1.1.11.7 1.1.11.7
Validate
Build & deploy on demand
35
36
Using tools
Test cases using compare & sync tools:
An index exists in source (QA) but not in target (Production)
What should we do? Add the index or not?
37
Compare & sync tools
Safe to automate?Sure… (?)
38
Compare & sync tools
Safe to automate?No. Requires manual inspection…
39
Challenges…
Compare & sync tools:
▪ Are great for finding out what is out of sync
▪ Not so great for automating deployments
▪ Requires manual inspection
▪ Requires detailed knowledge regarding each change as part of the process
▪ Is unaware of any changes that occurred before the time it ran
▪ Has no knowledge of changes that took place at the target environment
▪ Unable to deal with conflicts & merges between different teams
Mistrust AGAIN… So…no automation… We fear for automating problems into production and a major risk!!!
40
Another poll? Last one!
How much automation do you have in your database deployments?
• None, scripts are written manually
• Writing scripts manually, executing them as part of an automated process
• Generating scripts automatically, reviewing them manually
• We have a fully automated CD process (databases updates are created and executed automatically)
41
We need to leverage
knowledge from version control
42
Safety net
Static script based deployment
43
Deploying changes if needed
DevelopmentBaseline
Previous Label /
Production Golden Copy
Production
If we had the index in the baseline =>
we should take it down from production…
(Deploy Change)
44
Or protecting target environment…
DevelopmentBaseline
Previous Label /
Production Golden Copy
Production
BUT… If no index in baseline =>
we should protect the NEW index on production!!!
(Protect Target)
45
Safe?
Simple, right?NO! we are going to BREAK production without even knowing…
46
Dealing with conflicts => merging changes
47
Conflict Resolving – Meta Data/Content
48
Safety Net Deployment Automation
￭ Raise red flags on conflicts ￭ Support out-of-process changes￭ Utilize baseline aware analysis￭ Understand the nature of the changes
Source vs. Baseline
Target vs. Baseline
Action
= = No Action
≠ = Deploy Changes
= ≠ Protect Target
≠ ≠ Merge Changes
49
Impact Analysis! not Damage Control…
Raise red flags to stop the line…
if requires human intervention
50
Safety Net For Deployment Automation
Database Safe Deployment Automation:• Leverage one source of truth (baselines & previous revisions)
• Flexible scope (deploy multi schema to single task or work item)
• Run as a batch process (repeatable & consistent)
• Integrates to ALM (labels, CRs, Continuous Integration & Delivery)
• Deal with conflicts & merges to match code agility
Can raise red flags to stop the line…
if requires human intervention
51
To summarize CD…
•Automate “everything”• Package the deployment of database changes along with all your other application
components to give a unified picture
•Move the process upstream• Easily promote the same package (including database changes!) from one environment to
the next, handling environment-specific differences automatically
•Create the deployment pipeline
52
What does DBmaestro offer?Database Enforced Change Management solution
+ Database version control+ Enforce best practices+ Plugs into the ALM (change request, tickets & work items)+ Database merge & change impact analysis+ Know who can do what, where, when & why
DevOps Solution for databases + Baseline aware deployment automation, rollback & recovery+ Reduce database deployment issues+ Plugs into release management & Continuous Delivery
Allows you to package, verify, deploy and promote database changes just as you would do with application code…
putting you in a position to build a full delivery pipeline…
53
Change Policy Enforcement
54
Impact Analysis
Head over to our community to see how everyoneresponded to the poll questions.
Poll Questions
community.idera.com/forums
top related