death to manual deployments
DESCRIPTION
Manual deployments of software are slow, error prone, and generally miserable. Let's talk about getting rid of them and replacing them with automation.TRANSCRIPT
11
Death to Manual Deployments!
Implementing Reliably Consistent Deployments Across Environments
22
Eric MinickTechnical [email protected]
• Background as a developer, tester and tools guy
• Helping customers implement build & deploy for the past 8 years
33
We hate manual deployments
• They’re slow• They aren’t consistent across environments• You fail in production• You don’t know who deployed what, where or
when
44
Why do people like manual deployments?
• I feel more in control from the command line
• It’s how we’ve always done it
• I know what I did and see the output of the commands I ran
• When something goes wrong, I can be the hero and fix it
55
What about scripts?
• I don’t have a manual deployment: I have scripts!– How many scripts?– What order?– Who has permissions to run them?– Are they the same across environments?– What else do you need to do?
• Scripts are only part of a fully automated deployment.
66
We love automated deployments
• They’re NOT slow• They ARE consistent across environments• You DON’T fail in production• You DO know who deployed what, where or
when
77
Automated deployments provide benefits
• Provide a platform you can build on– continuous deploys, smoke tests, other goodies
• Empower people while increasing control– stop waiting on other people– stop being harassed by other people
• Better, Faster, Cheaper (Pick any 3)
88
The Program
• Today’s Manual Deployments• Elements of an Automated Solution• Best Practices with Automated Deployments
99
The Program
• Today’s Manual Deployments• Elements of an Automated Solution• Best Practices with Automated Deployments
1010
… you might have a manual deployment
• Hand-offs are through email (or file shares)• Developers, QA, and Ops deploy differently• Tweaks communicated on an as-broken basis• Deployments steps are a huge document• The term “deployment weekend” is used• Developers have production passwords
1111
Hand-offs are through email (or file shares)
• They’re slow• They aren’t consistent across environments• You fail in production• You don’t know who deployed what, where or
when
1212
Developers, QA, and Ops deploy differently
• They’re slow• They aren’t consistent across environments• You fail in production• You don’t know who deployed what, where or
when
1313
Tweaks communicated on an as-broken basis
• They’re slow• They aren’t consistent across environments• You fail in production• You don’t know who deployed what, where or
when
1414
Deployments steps are a huge document
• They’re slow• They aren’t consistent across environments• You fail in production• You don’t know who deployed what, where or
when
1515
The term “deployment weekend” is used
• They’re slow• They aren’t consistent across environments• You fail in production• You don’t know who deployed what, where or
when
1616
Developers have production passwords
• They’re slow• They aren’t consistent across environments• You fail in production• You don’t know who deployed what, where or
when
1717
If only people were more disciplined…
• We could make deployment docs– we could capture everything– we could update them– we could follow them, perfectly, every time.
• We could have audit spreadsheets– we could capture everything– we could update them
1818
People aren’t machines
• People: good at using creative judgment
• Machines: good at consistency
Automated deployments leverage the strengths of people and machines.
Alistar Cockburn, “Characterizing People as Non-Linear, First-Order Components in
Software Development”
1919
DevOps isn’t natural
• Conway’s Law: organizations are constrained to produce solutions that copy the structure of their organization
• Ops wants to maintain control– doesn’t care about DEV, can’t keep up with the
frequent changes• Development doesn’t want responsibility for
production deployments
• QA lacks influence in DEV & PROD
2020
The Program
• Today’s Manual Deployments• Elements of an Automated Solution• Best Practices with Automated Deployments
2121
Elements of an automated solution
• Automated process• Coordinating tiers• Environment definitions• Artifact repository• Access control• Audit trail
2222
Automated process
Status Quo• Don’t exist• Incomplete• Multiple order-dependent
scripts• Multiple versions per
environment (or machine)• Uncontrolled
What we need• Single script reused across
environments• Controlled access to
deployment steps• Auto record what happened
Fast, consistent and repeatable
2323
Coordinating Tiers / Components
Status Quo• Based on oral history• In a Release Manager’s
spreadsheet• Total plan based on
documented changes for each release– Redesign every time
What we need• A model for whole systems• Standard deployment based
on changes to all pieces and parts
• Actual deployment runs the needed subset.
Move all tiers, services, config etc together
2424
Environment definitions
Status Quo• Configured manually• Details embedded in
binaries, scripts, property files, ini files
• Environment information either uncontrolled or in source control
What we need• Environments a first-level
entity• Properties controlled at the
environment level• Secure storage of sensitive
values (passwords)
Per-environment configuration and passwords
2525
Artifact repository
Status Quo• Is email a repository• File share on the network• Source control for binary control• Uncontrolled• Crude control over what to keep
What we need• Definitive Software Library• Consistent set of artifacts across
environments• Tamper-proof artifacts• Automated retention policies
based on deployments
Know what you’re getting
2626
Access control
Status Quo• Controlled by who can log
onto the machine• Access tends to grow over
time in response to crisis• Access control distributed
across machines
What we need• Role-based access control• Access control by
environment• Single source of access
control• LDAP / Active Directory
authentication (or authorization)
Who can deploy to which environment
2727
Audit trail
Status Quo• What audit trail?• I have the spreadsheet• Our Change Management
System has what we think we did
• Audit trail starts at PROD
What we need• Win at Clue:
– Who– What– Where– When (better than Clue!)
• Require no additional work• End-to-end traceability
Know what happened
2828
Solution Benefits
• We have confidence that our production deployments will succeed
• QA can take new builds faster• DEV deployments fast enough to keep up with
our rate of change• We have an audit trail from day one• We have separation of duties
2929
The Program
• Today’s Manual Deployments• Elements of an Automated Solution• Best Practices with Automated Deployments
3030
Best Practices with Automated Deployments
• Deferred deployments• Self-service deployments• Automatic deployment• Automatic smoke-test• Automated rollbacks• Continuous production deployment
3131
Deferred deployments
• Story: I want to do my deployment at 11 pm on Friday after my users are offline.
• Solution: Schedule deployment for desired time, go home. Wait for notification of success or failure.
3232
Self-service deployments
• Story: QA wants to be able to take a new build when they need one.
• Solution: Use per-environment access control to allow the QA people to deploy on-demand, but only to the test environment.
3333
Automatic deployment
• Story: Want to keep the development test server in sync with the latest changes in source control.
• Solution: After building, automatically deploy to the development server.
3434
Automatic smoke-test
• Story: I want to know if something breaks immediately.
• Solution: After deploying, automatically trigger a suite of automated tests and report the results.
3535
Automated rollback
• Story: If my deployment fails I want to restore the previous build.
• Solution: When a deployment step fails, trigger the restoration of the last successfully deployed build.
3636
Continuous production deployments
• Story: When a change is committed it should be deployed to production if it doesn’t break anything.
• Solution: After deployment to test environment run automated test suites. If they all pass, trigger a deployment to production, then test that production deploy was successful. Rollback on failure.
3737
Summary
• Manual deployments are slow and broken because they require people to act like machines
• Computers make good machines
• Automated deployments provide a solid foundation for further improvement
3838
References
http://urbancode.com/resources• Deployment Automation Basics• Lean Build & Deployment Automation• ITIL Release Management and Automation
Blogs.urbancode.com Twitter.com/UrbanCodeSoftFacebook.com/UrbanCodeSoft
3939
Yes, UrbanCode sells products for this
• AnthillPro– Continuous Delivery including build, test and
deployment.
• uDeploy– Model driven deployments focused on the
coordination problem. Integrated with a variety of build tools.
