how database devops will help you - redgate...how database devops will help you to stay ahead of the...

How Database DevOps Will Help You

To Stay Ahead Of The Competition

Derek Campbell, Solutions Architect at Octopus Deploy

Agenda

• Background

• DevOps

• Benefits

• Roadblocks

• Questions

Background

Previous Work Experience

• Release Manager

• Lead DevOp

• System Architect

• System Administrator

What are the issues we can solve?

• Databases out of pace with application development

• Lack of Traceability

• Inability to Roll back

• Databases are the bottleneck

• Releases less frequent with increased risk

• Efficiency – Manual work takes more time.

System Development Life Cycle

Development Team

Operations Team

Knowledge Silos

• Development Team• Development roadmap

• Feature requirements

• Application specific knowledge

• Operations Team• Performance statistics

• Support tickets submitted

• Infrastructure specific knowledge

Typical Communication Between Teams

• “This service is down please fix it”

• “Your service is consuming 16 GB of RAM, fix it now”

• “I have a call from a customer and they say your application is

slow”

• “I got called out last night because this SQL Job failed. Fix it.”

• “It worked in Dev”

A Lack of Trust

Lack Of Trust

Slows Everything

Down

It doesn’t need to

be Perfect!!

Example

• Application ready for deployment

• Development: • “We agreed with the customer to deploy a new feature next week”

• Operations:• “Every time you deploy your application you increase our support

tickets volume”• “Every time you deploy your application consumes more resources”

• New Policy: Min 2 Week Notice For Deployments

Massive Deployments

• Multiple Sprints prior to going to production

• Multiple Features bundled together prior to going

to production

• Long Waterfall release cycles (quarter or months)

Perfect Storm

• Quarterly Releases

• Lots of changes

• Out of sync databases

• Manual Deployments

Everyone Is Unhappy

• Business Owners• “This isn’t going to work now, we need to add the following

features to make this viable”

• “Fine. Fine. Fine. Deploy it as is. I’ll add it my next release doc”

Everyone Is Unhappy

• Users• Not what they wanted.

• How long until the application is no longer relevant

• Head to competitors

Everyone Is Unhappy

• Security• Data Compliance

• Patches

DevOps

DevOps: Development and Operations

Teams Working Together Towards a Common

Goal

Dev + Ops = DevOps

Work toward a Common Goal

• 1 deployment a month

• 1 deployment a week

• Zero Downtime Deployments

• 10 deployments a day

• Fixing all critical bugs within 2 business days

Common Goal

• Performance tests before each release

• Decrease deployment time

• Increase response time on critical methods

• Decrease resource usage

Example: Performance Monitoring

• Dev Team Problem: Bad Performance

• Ops Team Problem: Babysitting Apps

• DBAs: Increase in resource consumption

• Common Problem: What is causing the slow

down

Example: Performance Monitoring

• Solution: New Relic• Ops: Train developers on how to use NewRelic

• Developers: Train Ops and DBAs what to look for (sprocs, stack traces, etc)

• DBAs: Look for slow running sprocs, or sprocs getting hit thousands or millions of times

Example: Automated Database Deployments

• Dev Team Problem: Hours spent manually creating delta scripts

• DBA Problem: No consistency between teams or environment databases, no auditing of scripts, no testing of scripts.

• Common Problem: 30 minutes - 2 hours per deployment per team

• Solution: Use Redgate tooling with Octopus Deploy to automate database deployments

• Result: • 5 - 20 minute deployments• One release a quarter down to one release every 10 days

Benefits

Speed - Release When Ready

• At Octopus Deploy we release as soon as the feature is ready

• Monthly releases of major features, daily releases of bug fixes, performance improvements, and UI improvements

• Use “release bots”, TeamCity, Slack and Octopus Deploy to deploy Octopus Deploy

Example - Fixing Critical Bug

• Customer reported bug to their process through

support channels at Octopus Deploy Tuesday

morning

• Discussed bug during the day

• Fix released overnight

• In customer’s hands by Wednesday morning

Example - Better Performance

• Application averaging 2000 requests per minute

• After deployment average response time went

from 20 ms to 200 ms

• Identify slow performance and deploy release

before first support ticket came in

Staying Ahead Of Competition

• Respond to customer feedback

• Fix critical bugs

• Experiment and A/B testing

• Keep company relevant

• Faster time to market

Staff

• Higher staff engagement

• More interesting work

• Better work/life balance

• Cross Skilling and Self Improvement

• Decentralized responsibility

Compliance

• Auditing

• Standardization

• Reduce Errors

• Protect Sensitive data

• Increased Trust

Getting Started

Start Small

• Attend meetings

• Discuss issues and complaints

• Listen and Learn

• Pick a common problem

• Use existing tooling

Go a bit bigger

• Solve a larger problem

• Pilot Application / Group

• New Tooling

• New Processes

Tooling

• Build: TeamCity / VSTS / Jenkins / Bamboo

• Unit Testing: NUnit / MSTest / XUnit

• Integration Testing: Selenium / Postman

• Deploy: Octopus Deploy / Redgate (for DBs) / VSTS Release / DACPAC

• Monitoring: NewRelic / DataDog / DynaTrace / SQL Sentry

• Error Logging: Splunk / Seq / PaperTrail

DevOps is People,

Processes and

Tools

Roadblocks

People

• “We’ve been doing it this way for 10 years”

• “Sure what we have is a little bumpy, but it works”

• “I’m too busy to change”

• “This was tried once before and it failed”

• “What you proposed won’t work for me or my

team, we’re special”

Time

• Time to implement changes

• Each new change requires multiple iterations

• Deadlines and other conflicting priorities

Cost

• Tools cost money

• Cost per hour

• Consultants

• Training

• Return of Investment

Breaking Through Roadblocks

• Technology is the easy part

• Pilot team / project

• Small changes

• Don’t aim for perfection at first

• Provide clear and measurable stats

• Be Transparent

Anyquestions?

Thank You!