continuous delivery - takeaways
Post on 14-Jan-2017
22 Views
Preview:
TRANSCRIPT
§Continuous delivery
1. Antipatterns
2. Proposed model
3. Common delivery issues
Agenda
§Antipatterns
Deploying software manually• production of extensive, detailed documentation that describes the
steps to be taken and the ways in which the steps may go wrong
• reliance on manual testing to confirm that the application is running
correctly
• frequent calls to the development team to explain why a deployment is
going wrong on a release day
• frequent corrections to the release process during the course of a
release
• environments in a cluster that differ in their configuration
• releases that take more than a few minutes to perform
• unpredictable releases that often have to be rolled back or run into
unforeseen problems
Deploying to a production-like environment only after development complete
• testers, if involved by this stage, have tested the system on
development machines
• releasing into staging is the first time that operations people interact
with the new release
• dev team assembles the correct installers, configuration files, db
migrations and deployment documentation to pass to the deployment
team without having tested them in an environment that looks like
production or staging
• little/no collaboration between the dev and deployment teams
Manual configuration management of production environments• having deployed successfully many times to staging, the deployment
into production fails
• different members of a cluster behave differently
• operations team take a long time to prepare an environment for a
release
• cannot step back to an earlier configuration of your system
• servers in clusters have, unintentionally, different versions of OS, 3rd
party infrastructure, libraries or patch levels
§Proposed model
Software delivery principles
Create a repeatable, reliable release process
Automate almost everything
Keep everything in version control
If it hurts, do it more frequently and bring the pain forward
Build quality in
Done means released
Everybody is responsible for delivery process
Continuous improvement
The deployment pipeline
- deployment pipeline = an automated implementation of your application’s build, deploy, test, and release process
- makes every part of the process of building, deploying, testing, and releasing software visible to everybody involved
- improves feedback so that problems are identified and resolved as early as possible- enables teams to deploy and release any version of their software to any environment
at will through a fully automated process
Blue-Green Deployments
- blue and green = 2 identical versions of production environment- deploy new version to the blue environment- when ready, change the router to point to blue – blue becomes production- if something goes wrong, switch the router back to green- careful with the db!- if budget allows, blue and green can be completely separate replicas- if not, use 2 copies of your application running side by side on the same environment
as blue and green
Canary Releasing
§Common delivery
issues
Infrequent/buggy deployments - symptomsProblem - long time to deploy the build, brittle deployment process
Symptoms
• long time for bugs to be closed by testers
• long time for stories to be tested or signed off by the customer
• testers are finding bugs that developers fixed a long time ago
• nobody trusts the UAT, performance, or CI environments, and
people are skeptical as to when a release will be available
• showcases rarely happen
• the application can rarely be demonstrated to be working
• the team’s velocity (rate of progress) is slower than expected
Infrequent/buggy deployments - causes• deployment process not automated
• not enough hardware available
• hardware and OS configuration not managed correctly
• deployment process depends on systems outside the team’s control
• not enough people understand the build and deployment process
• testers, developers, analysts, and operations personnel not collaborating
sufficiently during development
• developers not disciplined about keeping the application working by
making small, incremental changes, and so frequently break existing
functionality
Poor application quality - symptomsProblem - Delivery teams failing to implement an effective testing
strategy
Symptoms
• regression bugs keep popping up
• the defect number keeps increasing even when your team spends
most of its time fixing them
• customers complain of a poor-quality product
• developers groan and look horrified whenever a new feature request
arrives
• developers complain about the maintainability of the code
• an ever-increasing amount of time to implement new functionality, the
team starts falling behind
Poor application quality - causes• testers do not collaborate with developers during feature development
• stories or features “DONE” without comprehensive automated tests,
being signed off by testers, or being showcased to users from a production-
like environment
• defects routinely entered into a backlog without being fixed on the spot
with an automated test to detect regression problems
• developers or testers without enough experience developing automated
test suites
• team doesn’t understand the most effective types of tests to write for the
technology or platform
• developers working without enough test coverage
Poorly managed CI - symptomsProblem - build process not properly managed
Symptoms
• developers don’t check in often enough (at least once a day)
• commit stage permanently broken
• high number of defects
• long integration phase before each release
Poorly managed CI - causes• automated tests take too long to run
• commit stage takes too long to run (ideally under 5 minutes, more than
10 minutes unacceptable)
• automated tests fail intermittently, giving false positives
• nobody empowered to revert check-ins
• not enough people understand and can change the CI process
Poor configuration management - symptomsProblem - environments can’t be commissioned, and applications
installed reliably, using an automated process
Symptoms
• mysterious failures in production environments
• new deployments are tense, scary events
• large teams dedicated to environment configuration and
management
• deployments to production often have to be rolled back or patched
• unacceptable downtime of production environment
Poor configuration management - causes• UAT and production environments are different
• poor or badly enforced change management process for making
changes to production and staging environments
• insufficient collaboration between operations, data management and
delivery teams
• ineffective monitoring of production and staging environments to detect
incidents
• insufficient instrumentation and logging built into applications
• insufficient testing of the non-functional requirements of applications
§Thanks!
top related