agility requires safety
TRANSCRIPT
![Page 1: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/1.jpg)
AGILITYrequires
SAFETY
![Page 2: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/2.jpg)
Every startup has the same story:
![Page 3: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/3.jpg)
“We don’t have time for best practices.”
![Page 4: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/4.jpg)
![Page 5: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/5.jpg)
![Page 6: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/6.jpg)
![Page 7: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/7.jpg)
![Page 8: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/8.jpg)
![Page 9: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/9.jpg)
![Page 10: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/10.jpg)
You can’t go faster by being reckless
![Page 11: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/11.jpg)
Think of cars on a highway
![Page 12: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/12.jpg)
What happens if everyone jams down on the gas?
![Page 13: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/13.jpg)
![Page 14: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/14.jpg)
To go fast, a car needs not only a powerful engine…
![Page 15: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/15.jpg)
But also powerful brakes.
![Page 16: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/16.jpg)
As well as seat belts, airbags, bumpers, and auto-pilot
![Page 17: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/17.jpg)
For cars and for software, speed is limited by safety
![Page 18: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/18.jpg)
What are the seat belts, brakes, & self-driving cars of
software?
![Page 19: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/19.jpg)
This talk is about safety mechanisms
![Page 20: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/20.jpg)
That make it possible tobuild software quickly
![Page 22: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/22.jpg)
Founder of
Atomic Squirrel
atomic-squirrel.net
![Page 23: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/23.jpg)
PAST LIVES
![Page 25: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/25.jpg)
1. Brakes2. Bulkheads3. Autopilot4. Safety catch5. Speedometer6. Warning lights7. Seat belt
Outline
![Page 26: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/26.jpg)
1. Brakes2. Bulkheads3. Autopilot4. Safety catch5. Speedometer6. Warning lights7. Seat belt
Outline
![Page 27: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/27.jpg)
Good brakes stop your car before you run into something
![Page 28: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/28.jpg)
Continuous integration stops buggy code before it goes into production
![Page 29: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/29.jpg)
Imagine your goal is to build the International Space Station
![Page 30: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/30.jpg)
Each team designs and builds their component in isolation
![Page 31: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/31.jpg)
You launch everything into space and hope it all comes together
![Page 32: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/32.jpg)
![Page 33: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/33.jpg)
I thought the Russians were going to build the bathrooms?
![Page 34: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/34.jpg)
Weren’t the French supposed to do the wiring?
![Page 35: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/35.jpg)
Everyone is using the metric system, right?
![Page 36: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/36.jpg)
Teams working for a long time with incorrect
assumptions
![Page 37: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/37.jpg)
Finding this out when you’re in outer space is too
late
![Page 38: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/38.jpg)
This is the result of “late integration”
![Page 39: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/39.jpg)
Lots of teams working in isolation on separate branches
![Page 40: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/40.jpg)
Before attempting a massive merge at the very end
![Page 41: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/41.jpg)
MERGE CONFLICT
![Page 42: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/42.jpg)
The alternative is “continuous integration”
![Page 43: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/43.jpg)
Where everyone regularly merges their work
![Page 44: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/44.jpg)
The most common approach is
trunk-based development
![Page 45: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/45.jpg)
Everyone works on a single branch (trunk)
![Page 46: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/46.jpg)
That can’t possibly scale to a lot of developers, can it?
![Page 47: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/47.jpg)
Uses trunk-based development for 1,000+ developers
![Page 48: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/48.jpg)
Uses trunk-based development for 4,000+ developers
![Page 49: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/49.jpg)
Uses trunk-based development for 20,000+ developers
![Page 50: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/50.jpg)
![Page 51: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/51.jpg)
Wouldn’t you have merge conflicts all the time?
![Page 52: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/52.jpg)
If you merge (commit) regularly, conflicts are rare.
![Page 53: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/53.jpg)
And those that happen are from a day of work—not months.
![Page 54: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/54.jpg)
Commit early and often.
![Page 55: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/55.jpg)
Small commits are easier to merge, test, revert, review
![Page 56: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/56.jpg)
![Page 57: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/57.jpg)
Wouldn’t there constantly be broken code in trunk?
![Page 58: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/58.jpg)
Build Build Build Build
Not if you run a self-testing build after every commit
Build Build Build Build Build Build Build
![Page 59: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/59.jpg)
Build Build Build Build
It should compile your code and run your automated tests
Build Build Build Build Build Build Build
![Page 60: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/60.jpg)
Build Build Build Build
If a build fails, a developer must fix it ASAP or revert the commit
Build Build Build Build Build Build Build
![Page 61: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/61.jpg)
Of course, this depends on having good automated
tests
![Page 62: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/62.jpg)
Tests give you the confidence to make changes
quickly
![Page 63: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/63.jpg)
JUnit version 4.11
...
Time: 6.063
OK (259 tests)
How long would it take you to do 259 tests manually?
![Page 64: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/64.jpg)
What should you test?
![Page 65: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/65.jpg)
Everything!
![Page 66: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/66.jpg)
Everything!
![Page 67: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/67.jpg)
It’s a trade-off between: 1. Likelihood of bugs2. Cost of bugs 3. Cost of testing
![Page 68: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/68.jpg)
Likelihood of bugs is higher for complex code and large
teams
![Page 69: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/69.jpg)
Cost of bugs is higher for some systems (payments,
security)
![Page 70: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/70.jpg)
Cost of tests is higher for integration and UI tests
![Page 71: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/71.jpg)
“Without continuous integration, your software is broken until somebody proves it works, usually during a testing or integration stage.
![Page 72: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/72.jpg)
With continuous integration, your software is proven to work (assuming a sufficiently comprehensive set of automated tests) with every new change—and you know the moment it breaks and can fix it immediately.”
![Page 73: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/73.jpg)
1. Brakes2. Bulkheads3. Autopilot4. Safety catch5. Speedometer6. Warning lights7. Seat belt
Outline
![Page 74: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/74.jpg)
Ships have bulkheads to try to contain flooding to one area.
![Page 75: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/75.jpg)
You can split up a codebase to contain problems to one area.
![Page 76: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/76.jpg)
Code is the enemy: the more you have, the slower
you go
![Page 77: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/77.jpg)
Project SizeLines of code
Bug Density Bugs per thousand lines of code
< 2K 0 – 25
2K – 6K 0 – 40
16K – 64K 0.5 – 50
64K – 512K 2 – 70
> 512K 4 – 100
![Page 78: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/78.jpg)
As the code grows, the number of bugs grows even
faster
![Page 79: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/79.jpg)
“Software development doesn't happen in a chart, an IDE, or a design tool; it happens in your head.”
![Page 80: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/80.jpg)
The mind can only handle so much complexity at once
![Page 81: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/81.jpg)
One solution is to break the code into multiple
codebases
![Page 82: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/82.jpg)
Instead of depending on the source of another module
/moduleA
/moduleB /moduleC /moduleD
/moduleE
![Page 83: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/83.jpg)
You depend on a versioned artifact from that module
moduleA-0.3.1.jar
moduleB-3.1.0.jar moduleC-9.8.0.jar moduleD-1.4.3.jar
moduleE-0.5.6.jar
![Page 84: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/84.jpg)
This provides isolation from changes in other modules
moduleA-0.3.1.jar
moduleB-3.1.0.jar moduleC-9.8.0.jar moduleD-1.4.3.jar
moduleE-0.5.6.jar
![Page 85: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/85.jpg)
You already do this: guava-18.0.jar
jquery-2.2.0.js
![Page 86: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/86.jpg)
Advantages of artifacts:
1. Isolation2. Decoupling3. Faster builds
![Page 87: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/87.jpg)
Disadvantages of artifacts:
1. Dependency hell2. No continuous
integration3. Hard to make global
changes
![Page 88: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/88.jpg)
Another option is to break the codebase into services
![Page 89: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/89.jpg)
In a monolith, you use function calls within one process
A.a()
B.b() C.c() D.d()
E.e()
![Page 90: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/90.jpg)
With services, you pass messages between processes
http://A/a
http://B/bhttp://C/c
http://D/d
http://E/e
![Page 91: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/91.jpg)
Advantages of services:
1. Technology agnostic2. Scalability3. Isolation
![Page 92: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/92.jpg)
Disadvantages of services:
1. Operational overhead2. Performance overhead3. I/O, error handling4. Backwards compatibility5. Hard to make global
changes
![Page 93: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/93.jpg)
1. Brakes2. Bulkheads3. Autopilot4. Safety catch5. Speedometer6. Warning lights7. Seat belt
Outline
![Page 94: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/94.jpg)
Autopilot prevents accidents caused by human error
![Page 95: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/95.jpg)
Automated deployments prevent accidents caused by human error
![Page 96: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/96.jpg)
Deploying code can be painful
![Page 97: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/97.jpg)
“If it hurts, do it more often.” – Martin Fowler
![Page 98: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/98.jpg)
The deployment process should be:
![Page 99: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/99.jpg)
![Page 100: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/100.jpg)
That means you should never deploy or configure
manually
![Page 101: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/101.jpg)
> ssh [email protected]
__| __| __| _| ( \__ \ Amazon ECS-Optimized Amazon Linux AMI 2015.09.d ____|\___|____/
[ec2-user ~]$ sudo apt-get install ruby
Don’t do this
![Page 102: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/102.jpg)
Or this
![Page 103: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/103.jpg)
Instead, automate everything
![Page 104: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/104.jpg)
The gold standard is theblue-green deployment
![Page 105: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/105.jpg)
Let’s say you have version 0.0.1 of your app deployed
![Page 106: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/106.jpg)
First, deploy version 0.0.2 on a duplicate set of servers
![Page 107: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/107.jpg)
If everything looks good, switch the load balancer over to 0.0.2
![Page 108: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/108.jpg)
Four main categories of deployment automation
tools:
![Page 109: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/109.jpg)
1. Configuration management: Chef, Puppet, Ansible, Salt
![Page 110: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/110.jpg)
- name: Install httpd and php yum: name={{ item }} state=present with_items: - httpd - php
- name: start httpd service: name=httpd state=started enabled=yes
- name: Copy the code from repository git: repo={{ repository }} dest=/var/www/html/
Imperative scripts to configure servers and deploy code
![Page 111: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/111.jpg)
2. Provisioning tools: Terraform, CloudFormation, Heat
![Page 112: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/112.jpg)
resource "aws_instance" "example" { ami = "ami-b960b1d" instance_type = ["t2.micro"]}
resource "aws_eip" "ip“ { instance = "${aws_instance.example.id}" depends_on = ["aws_instance.example"]}
Declarative templates that define your infrastructure
![Page 113: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/113.jpg)
3. Virtual machines: VMWare, VirtualBox, Packer, Vagrant
![Page 114: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/114.jpg)
{ "builders": [{ "type": "amazon-ebs", "source_ami": "ami-de0d9eb7", "instance_type": "m1.medium", "ami_name": "example-packer-ami-{{timestamp}}" }], "provisioners": [{ "type": "shell", "inline": [ "sudo apt-get -y update", "sudo apt-get -y install httpd php” ] }]}Images of configured servers
![Page 115: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/115.jpg)
4. Containers: Docker, rkt, LXD
![Page 116: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/116.jpg)
FROM ubuntu:12.04
RUN apt-get update && apt-get install -y apache2 php
ENV APACHE_RUN_USER www-dataENV APACHE_LOG_DIR /var/log/apache2
EXPOSE 80
CMD ["/usr/sbin/apache2", "-D", "FOREGROUND"]
Lightweight images of configured servers
![Page 117: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/117.jpg)
These tools allow you to define your infrastructure
as code
![Page 118: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/118.jpg)
That way, you can version it, review it, test it, and
reuse it.
![Page 119: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/119.jpg)
1. Brakes2. Bulkheads3. Autopilot4. Safety catch5. Speedometer6. Warning lights7. Seat belt
Outline
![Page 120: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/120.jpg)
Elisha Otis demoing elevatorfree-fall
safety in 1854
![Page 121: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/121.jpg)
The safety elevator patent
![Page 122: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/122.jpg)
The safety catches are locked by default
![Page 123: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/123.jpg)
Only an intact cable can unlock thelatches
![Page 124: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/124.jpg)
This elevator provides safety by default
![Page 125: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/125.jpg)
Feature toggles provide safety by default
![Page 126: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/126.jpg)
New feature, part 1
New feature, part 2
New feature, part 3
If a large new feature takes many commits, wouldn’t a user see it in an unfinished state?
![Page 127: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/127.jpg)
<section id="new-section"> <!-- Code for new section--></div><section id="original-section"> <!-- Code for original section--></section>
Let’s say you were adding a new section to your website.
![Page 128: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/128.jpg)
<% if toggles.enabled("new-section") %> <section id="new-section"> <!-- Code for new section--> </div><% end %> <section id="original-section"> <!-- Code for original section--></section>
Wrap new code in a conditional that looks up a feature toggle
![Page 129: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/129.jpg)
<% if toggles.enabled("new-section") %> <section id="new-section"> <!-- Code for new section--> </div><% end %> <section id="original-section"> <!-- Code for original section--></section>
Toggles are off by default, so users won’t see unfinished work
![Page 130: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/130.jpg)
development: feature_toggles: new-section: true
production: feature_toggles: new-section: false
You can enable feature toggles in a config file.
![Page 131: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/131.jpg)
> curl http://feature.toggles/
{ "development": { "new-section": true }, "production": { "new-section": false }}
Or you could create a web service for feature toggles.
![Page 132: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/132.jpg)
> curl http://feature.toggles/?user=123
{ "development": { "new-section": "A" }, "production": { "new-section": "B" }}
It could return different, complex values for each user.
![Page 133: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/133.jpg)
And provide a web UI for configuring toggles.
![Page 134: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/134.jpg)
This allows you to quickly turn features on or off.
![Page 135: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/135.jpg)
<% if toggles.get("new-section") == "A" %> <section id="new-section-bucket-a"> <!-- Code for new section, version A --> </div><% elsif toggles.get("new-section") == "B" %> <section id="new-section-bucket-b"> <!-- Code for new section, version B --> </div><% end %>
This allows A/B testing
![Page 136: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/136.jpg)
1. Brakes2. Bulkheads3. Autopilot4. Safety catch5. Speedometer6. Warning lights7. Seat belt
Outline
![Page 137: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/137.jpg)
A speedometer tells you how fast you’re driving
![Page 138: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/138.jpg)
Monitoring tells you how your product is performing
![Page 139: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/139.jpg)
“If you can’t measure it, you can’t fix it.” – David Henke
![Page 140: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/140.jpg)
There are many types of monitoring
![Page 141: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/141.jpg)
Availability metrics: is my product up or down?
![Page 142: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/142.jpg)
Useful tools: Keynote, Pingdom, Uptime Robot, Route53
![Page 143: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/143.jpg)
Business metrics: what are my users doing in the product?
![Page 144: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/144.jpg)
Useful tools: Google Analytics, KISSMetrics, Mixpanel
![Page 145: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/145.jpg)
Application metrics: how is my application performing?
![Page 146: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/146.jpg)
Useful tools: New Relic, CloudWatch, Datadog
![Page 147: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/147.jpg)
127.0.0.1 - - [10/Oct/2000:13:55:36] "GET /apache_pb.gif HTTP/1.0" 200 232664.242.88.10 - - [07/Mar/2004:16:05:49] "GET /twiki/bin/ HTTP/1.1" 401 12846127.0.0.1 - - [28/Jul/2006:10:22:04] "GET / HTTP/1.0" 200 221664.242.88.10 - - [07/Mar/2004:16:06:51] "GET /twiki/bin/Twiki/" 200 452364.242.88.10 - - [07/Mar/2004:16:10:02] "GET /mailman HTTP/1.1" 200 6291127.0.0.1 - - [28/Jul/2006:10:27:32] "GET /hidden/ HTTP/1.0" 404 7218192.168.2.20 - - [28/Jul/2006:10:27:10] "GET /cgi-bin/try HTTP/1.0" 200 339564.242.88.10 - - [07/Mar/2004:16:11:58] "GET /twiki/bin/view/" 200 735264.242.88.10 - - [07/Mar/2004:16:20:55] "GET /twiki HTTP/1.1" 200 5253Log files are also a form of application-level monitoring
![Page 148: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/148.jpg)
127.0.0.1 - - [10/Oct/2000:13:55:36] "GET /apache_pb.gif HTTP/1.0" 200 232664.242.88.10 - - [07/Mar/2004:16:05:49] "GET /twiki/bin/ HTTP/1.1" 401 12846127.0.0.1 - - [28/Jul/2006:10:22:04] "GET / HTTP/1.0" 200 221664.242.88.10 - - [07/Mar/2004:16:06:51] "GET /twiki/bin/Twiki/" 200 452364.242.88.10 - - [07/Mar/2004:16:10:02] "GET /mailman HTTP/1.1" 200 6291127.0.0.1 - - [28/Jul/2006:10:27:32] "GET /hidden/ HTTP/1.0" 404 7218192.168.2.20 - - [28/Jul/2006:10:27:10] "GET /cgi-bin/try HTTP/1.0" 200 339564.242.88.10 - - [07/Mar/2004:16:11:58] "GET /twiki/bin/view/" 200 735264.242.88.10 - - [07/Mar/2004:16:20:55] "GET /twiki HTTP/1.1" 200 5253Useful tools: loggly, logstash, Papertrail, Sumo Logic
![Page 149: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/149.jpg)
Server metrics: how is my server performing?
![Page 150: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/150.jpg)
Useful tools: Nagios, Icinga, Munin, collectd, CloudWatch
![Page 151: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/151.jpg)
1. Brakes2. Bulkheads3. Autopilot4. Safety catch5. Speedometer6. Warning lights7. Seat belt
Outline
![Page 152: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/152.jpg)
Warning lights notify you if something is wrong
![Page 153: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/153.jpg)
Alerting systems notify you if something is wrong
![Page 154: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/154.jpg)
You can’t look at metrics 24/7. Alerting systems can.
![Page 155: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/155.jpg)
Useful tools: PagerDuty, VictorOps
![Page 156: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/156.jpg)
For a full list of monitoring and alerting tools, see:
hello-startup.net/resources
![Page 157: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/157.jpg)
1. Brakes2. Bulkheads3. Autopilot4. Safety catch5. Speedometer6. Warning lights7. Seat belt
Outline
![Page 158: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/158.jpg)
Seat belts help you survive crashes
![Page 159: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/159.jpg)
High availability helps you survive crashes
![Page 160: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/160.jpg)
Stateless servers: multiple instances, multiple zones
![Page 161: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/161.jpg)
Load balancer routes around server or zone outages
![Page 162: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/162.jpg)
Auto-recovery mechanism brings server back after outage
![Page 163: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/163.jpg)
Stateful servers: multiple instances, multiple zones
![Page 164: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/164.jpg)
Replication to one or more standby servers
![Page 165: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/165.jpg)
Load balancer switches to standby server in case of outage
![Page 166: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/166.jpg)
Auto-recovery mechanism brings server back after outage
![Page 167: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/167.jpg)
Test your recovery process regularly.
![Page 168: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/168.jpg)
1. Brakes2. Bulkheads3. Autopilot4. Safety catch5. Speedometer6. Warning lights7. Seat belt
Outline
![Page 169: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/169.jpg)
Speed is limited by safety
![Page 170: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/170.jpg)
Two cars can drive at 80mph in opposite directions safely…
![Page 171: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/171.jpg)
Because of two yellow lines
![Page 172: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/172.jpg)
It’s worth the time to put these safety mechanisms in
place
![Page 173: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/173.jpg)
![Page 174: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/174.jpg)
For more info, see
Hello, Startup
hello-startup.net
![Page 175: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/175.jpg)
Questions?
![Page 176: Agility Requires Safety](https://reader035.vdocuments.us/reader035/viewer/2022070515/587aa8111a28abed218b4dd5/html5/thumbnails/176.jpg)
F1 racecar: Takayuki SuzukiHighway traffic: Oran ViriyincyCar accident: ER24 EMS (Pty) Ltd.Road: Nicolas RaymondBWM: Andy DurstSelf-driving car: Steve JurvetsonBus: Roland TanglaoTail lights: Tony WebsterUSS South Dakota: WikimediaCrash test dummy: Wikimedia
Elisha Otis: WikimediaOtis Elevator: WikimediaSpeedometer: Dawn HopkinsDashboard lights: Jim LarrisonSeat belt: WikimediaGoogle repo stats: Rachel PotvinISS: WikimediaFire: PeteMartin Fowler: Wikimedia
Image credits