the anatomy of continuous deployment at scale
TRANSCRIPT
The Anatomy of Continuous Deployment
At Scale
John Viner (Development Manager)
CTO Summit 2016
Who is Envato?
We run a two-sided Marketplace for creative assets that has paid out $450 million to our
“Authors” in the last 10 years7 million registered
users10 million items for
sale
The Business
25,000 requests a minute160m Page Views a Month
1 sale a second at Peak
120,000 lines of code (Ruby on Rails)25,000 tests
75 contributors, 10 years old
The Tech
Continuous Deployment vs Continuous Delivery
“Continuous Delivery is a software development discipline where you build
software in such a way that the software can be released to production at any time.”
“Continuous Deployment is a software development practice in which every code
change is deployed to production, automatically.”
Foundational Engineering Capabilities
Visible Operational Monitoring
Feature Toggle Tooling
Rollback Capability
Automated Build and Deploy Toolset
Comprehensive Automated Test Suite
Quality Code
Continuous Deployment
Foundational Organisational Behaviours
MTTR Focus
No QA / Approval Gates
Access to Production
Blameless Post Incident Reviews
Developers on First Line Support
Pairing and/or Peer Review
Continuous Deployment
The Lifecycle of a Change at Envato
Pull Request (PR) for all changes - GithubDeveloper Merges PR with at least 1 review
Automated Tests executed (no perf tests) - Buildkite
Automated Deploy executed on passed test - Samson + CodeDeploy
Rolling Deploy of Application with Zero DowntimeSchema Migrations with Zero Downtime
The Thumbs Up
Communication
The Numbers for June 2016
~ 40 developers3000 total builds (2.5 builds p/dev/day)
500 Pull Requests Closed400 deployments (20 p/day, 0.5 p/dev/day, peak 25 per day)
4.5 hour cycle time (from first commit to deploy - working hours)
8000 deployments in the last 2 YearsPull Request Size - 37 lines changed , 3 files
changed
Scaling the Team
Myths Dispelled
Continuous Deployment is only for:• Small Teams• Small Scale• Low Commercial Impact of Outage
The Benefits for Envato
1. Reduced Risk.• Small Changes are Easier to Identify and Rollback (or
forward)2. Increases Speed to Market• Small Batch Efficiency - Faster Throughput
3. Increases Developer Happiness• Code in Production makes for happy devs• Supporting Production increases empathy with customer
How to Get Started
• Start on a Small System • Encourage a Culture of Developer
Ownership• Invest in Test Automation• Automate Your Build and Deployment
Pipeline• Invest in Feature Toggles
• Measure Your Cycle Time
• Optimise for Mean Time to Recovery (not Failure)
Thank you!