frequent releases reduce risk
DESCRIPTION
TRANSCRIPT
Releasing Frequently Reduces Risk
Owen RogersTwitter: @exortech
http://exortech.com/blog
1 year
48 releases
~ 1 release / week
10+/day
50+/day
continuous deployment
crazy
sea change
competitive advantage
evolve software
resolve problems
faster than the competition
capability
respond to change
▪ Individuals and interactions over processes and tools
▪ Working software over comprehensive documentation
▪ Customer collaboration over contract negotiation
▪ Responding to change over following a plan
Agile Manifesto
proposition
increase riskfrequent releases
increase riskreduce
frequent releases
1) expose
2) mitigate
?
can you deploy daily?
“we can’t do that...”
(3 minutes, 3 reasons)
3 common excuses
1. not enough time to test
1. not enough time to test
2. disruptive to users
1. not enough time to test
2. disruptive to users
3. too much overhead
1. not enough time to test
2. disruptive to users
3. too much overhead
1. not enough time to test
1. not enough time to test
Risk: cost of bugs getting into production
1. not enough time to test
Risk: cost of bugs getting into production
Assumptions:
1. not enough time to test
Risk: cost of bugs getting into production
Assumptions:• we know what our customers want
1. not enough time to test
Risk: cost of bugs getting into production
Assumptions:• we know what our customers want• bugs are expensive
1. not enough time to test
Risk: cost of bugs getting into production
Assumptions:• we know what our customers want• bugs are expensive• testing takes a long time
1. not enough time to test
Risk: cost of bugs getting into production
Assumptions:• we know what our customers want• bugs are expensive• testing takes a long time• all bugs can be found in test
1. not enough time to test
Risk: cost of bugs getting into production
Assumptions:• we know what our customers want• bugs are expensive• testing takes a long time• all bugs can be found in test
1. not enough time to test
Assume: we know what our customers want
testing: did we build it right?
1. not enough time to test
Assume: we know what our customers want
risk: did we build the right thing?
1. not enough time to test
Assume: we know what our customers want
$O
1. not enough time to test
Assume: we know what our customers want
$O(well, negative $ actually)
1. not enough time to test
Assume: we know what our customers want
1. not enough time to test
Assume: we know what our customers want
• we understand our customer’s needs
1. not enough time to test
Assume: we know what our customers want
• we understand our customer’s needs• our customers know what they want
1. not enough time to test
Assume: we know what our customers want
• we understand our customer’s needs• our customers know what they want• our customers will still want what we have when we give to them
1. not enough time to test
Assume: we know what our customers want
validated learning about customers
1. not enough time to test
Assume: we know what our customers want
simplest solution to validate hypothesis
1. not enough time to test
Assume: we know what our customers want
split test
1. not enough time to test
Assume: we know what our customers want
“deliver fast enough that the customer doesn’t have time
to change their mind”
1. not enough time to test
Risk: cost of bugs getting into production
Assumptions:• we know what our customers want• bugs are expensive• testing takes a long time• all bugs can be found in test
1. not enough time to test
Assumption: bugs are expensive
$ million bug
1. not enough time to test
Assumption: bugs are expensive
= $10 billion
1. not enough time to test
Assumption: bugs are expensive
= 120M users/day
1. not enough time to test
Assumption: bugs are expensive
bugs are inevitable
1. not enough time to test
Assumption: bugs are expensive
speed of response
1. not enough time to test
Assumption: bugs are expensive
continuous monitoring
1. not enough time to test
Assumption: bugs are expensive
1. not enough time to test
Assumption: bugs are expensive
1. not enough time to test
Assumption: bugs are expensive
1. not enough time to test
Assumption: bugs are expensive
1. not enough time to test
Risk: cost of bugs getting into production
Assumptions:• we know what our customers want• bugs are expensive• testing takes a long time• all bugs can be found in test
1. not enough time to test
Assumption: testing takes a long time
small changes
1. not enough time to test
Assumption: testing takes a long time
continuous integration
1. not enough time to test
Assumption: testing takes a long time
continuous testing
1. not enough time to test
Assumption: testing takes a long time
test automation
1. not enough time to test
Assumption: testing takes a long time
always releaseable
1. not enough time to test
Risk: cost of bugs getting into production
Assumptions:• we know what our customers want• bugs are expensive• testing takes a long time• all bugs can be found in test
1. not enough time to test
Assumption: all bugs can be found in test
= 40,000 photos/sec
1. not enough time to test
Assumption: all bugs can be found in test
= 6 Pb of storage
1. not enough time to test
Assumption: all bugs can be found in test
1. not enough time to test
2. disruptive to users
3. too much overhead
2. disruptive to users
2. disruptive to users
Risk: new releases will annoy users
2. disruptive to users
Risk: new releases will annoy users
Assumptions:
2. disruptive to users
Risk: new releases will annoy users
Assumptions:• releases incur downtime
2. disruptive to users
Risk: new releases will annoy users
Assumptions:• releases incur downtime• users will notice changes
2. disruptive to users
Risk: new releases will annoy users
Assumptions:• releases incur downtime• users will notice changes• changes are visible to all users
2. disruptive to users
Risk: new releases will annoy users
Assumptions:• releases incur downtime• users will notice changes• changes are visible to all users
2. disruptive to users
Assumption: releases incur downtime
patch releases?
2. disruptive to users
Assumption: releases incur downtime
good will
2. disruptive to users
Assumption: releases incur downtime
zero-downtime deployment
2. disruptive to users
Assumption: releases incur downtime
redundancy
2. disruptive to users
Risk: new releases will annoy users
Assumptions:• releases incur downtime• users will notice changes• changes are visible to all users
2. disruptive to users
Assumption: users will notice changes
2. disruptive to users
Assumption: users will notice changes
2. disruptive to users
Assumption: users will notice changes
version?
2. disruptive to users
Risk: new releases will annoy users
Assumptions:• releases incur downtime• users will notice changes• changes are visible to all users
2. disruptive to users
Assumption: changes are visible to all users
big-bang release
2. disruptive to users
Assumption: changes are visible to all users
options
2. disruptive to users
Assumption: changes are visible to all users
2. disruptive to users
Assumption: changes are visible to all usersOptions:
2. disruptive to users
Assumption: changes are visible to all usersOptions:
• release to user subset (private beta)
2. disruptive to users
Assumption: changes are visible to all usersOptions:
• release to user subset (private beta)• rollout to some servers
2. disruptive to users
Assumption: changes are visible to all usersOptions:
• release to user subset (private beta)• rollout to some servers• downgrade feature
2. disruptive to users
Assumption: changes are visible to all usersOptions:
• release to user subset (private beta)• rollout to some servers• downgrade feature• disable feature (feature darkmode)
2. disruptive to users
Assumption: changes are visible to all usersOptions:
• release to user subset (private beta)• rollout to some servers• downgrade feature• disable feature (feature darkmode)• defer commit
2. disruptive to users
Assumption: changes are visible to all usersOptions:
• release to user subset (private beta)• rollout to some servers• downgrade feature• disable feature (feature darkmode)• defer commit• defer release
2. disruptive to users
Assumption: changes are visible to all users
options = $$$
2. disruptive to users
Assumption: changes are visible to all users
“release is a marketing decision”
2. disruptive to users
Assumption: changes are visible to all users
bonus:
1. not enough time to test
2. disruptive to users
3. too much overhead
3. too much overhead
3. too much overhead
Risk: cost of a release is greater than the benefit of its changes
3. too much overhead
Risk: cost of a release is greater than the benefit of its changes
Assumptions:
3. too much overhead
Risk: cost of a release is greater than the benefit of its changes
Assumptions:• high coordination overhead
3. too much overhead
Risk: cost of a release is greater than the benefit of its changes
Assumptions:• high coordination overhead• releases are risky
3. too much overhead
Risk: cost of a release is greater than the benefit of its changes
Assumptions:• high coordination overhead• releases are risky• deployment takes a long time
3. too much overhead
Risk: cost of a release is greater than the benefit of its changes
Assumptions:• high coordination overhead• releases are risky• deployment takes a long time
3. too much overhead
Assumption: high coordination overhead
functional silos
3. too much overhead
Assumption: high coordination overhead
dev vs. ops
3. too much overhead
Assumption: high coordination overhead
devs don’t know prod
3. too much overhead
Assumption: high coordination overhead
ops don’t know code
3. too much overhead
Assumption: high coordination overhead
shared accountability
3. too much overhead
Assumption: high coordination overhead
3. too much overhead
Assumption: high coordination overhead
shared accountability
3. too much overhead
Risk: cost of a release is greater than the benefit of its changes
Assumptions:• high coordination overhead• releases are risky• deployment takes a long time
3. too much overhead
Assumption: releases are risky
“if it hurts, do it more often”
3. too much overhead
Assumption: releases are risky
incremental change
3. too much overhead
Assumption: releases are risky
more or less same system
3. too much overhead
Assumption: releases are risky
practice makes perfect
3. too much overhead
Risk: cost of a release is greater than the benefit of its changes
Assumptions:• high coordination overhead• releases are risky• deployment takes a long time
3. too much overhead
Assumption: deployment takes a long time
risk: manual steps
3. too much overhead
Assumption: deployment takes a long time
automated deployment
3. too much overhead
Assumption: deployment takes a long time
one-step process
Summary
?
Frequent Releases?
concerns
1. not enough time to test
2. disruptive to users
3. too much overhead
risks
1. bugs getting into production
2. new releases will annoy users
3. cost of a release is greater than the benefit of its changes
localized risk
1) expose
underlying risks
Risks:• do we know what customers want?• can we respond fast enough to problems?• can we detect problems when they happen? • can we limit the impact of changes?• can we deploy without downtime?• can we build production-ready software?
2) mitigate
mitigation strategies
Strategies:• validated learning about customers• split testing• continuous monitoring• continuous integration • test automation • zero-down time deployment• deployment options• operation levers• automated deployment
Thank You!
Shameless Plug
Nov 2 - 5• Eric Evans
• Michael Feathers
• Martin Fowler
• Jeff Patton
• Mary Poppendieck
• Michael Nygard
• Linda Rising
• Johanna Rothmann
Releasing Frequently Reduces Risk
Owen RogersTwitter: @exortech
http://exortech.com/blog
1. not enough time to test
2. disruptive to users
3. too much overhead
1. bugs getting into production
2. new releases will annoy users
3. cost of a release is greater than the benefit of its changes