continuous deployment at etsy - sxsw 2012 lean startup track

Continuous Deployment at Etsy

Chad Dickerson, CEOchad@etsy.com / @chaddickerson

March 10, 2012SXSW

1.4B page views per month$525MM sales on Etsy in 201139MM unique visitors / month800K shops / 150 countries100+ engineers

By the numbers

“The Etsy Way”http://etsy.me/etsyway

Our simple definition of continuous deployment:Small, frequent changes

to production.

Continuous deployment transformed Etsy

6/05 9/08 12/09 9/10

Timeline

Etsy launches I joined

First 1-button deploy; 7

deploys in ’09

StatsD(graphs!)

12/10 12/11

3,029 deploys

in ‘10

10,068deploys

in ‘11

12/12

10x growth

September 2008

“There are issues to address now and there will most certainly be bumps on the technology side as Etsy grows, but I want all of you Etsians to know that it is an honor and a privilege to be serving the Etsy community as your CTO. If you can give me some time and understanding here in my early days at Etsy, I hope to make you proud.”

- An Honest Beginning for Etsy’s New CTO (company blog)

http://www.etsy.com/blog/en/2008/tech-update-an-honest-beginning-for-etsys-new-cto/

Problems!

What was missing at Etsy at the time

• Culture of transparency!

• Monitoring

• Metrics / Graphing

• Automated testing

• Error log analysis

The present

6/05 9/08 12/09 9/10

Timeline

Etsy launches I joined

First 1-button deploy; 7

deploys in ’09

StatsD(graphs!)

12/10 12/11

3,029 deploys

in ‘10

10,068deploys

in ‘11

12/12

10x growth

Confidence as a Service

Fred

Board members deploy!

Dogs deploy!

How we got there

Most important issue?

Culture.

Peopleware:“the major problems of

our work are not so much technological as

sociological in nature”

“Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.”

Conway’s Law:

- Melvin Conway, 1968

Fix people and team issues first.

Communication

Insist on transparency

Operability reviews

http://www.slideshare.net/jallspaw/go-or-nogo-operability-and-contingency-planning-at-etsycom

Blameless post-mortems

http://www.slideshare.net/jallspaw/advanced-postmortem-fu-and-human-error-101-velocity-2011

Key tools at Etsy

Deployinator:One button deploy

https://github.com/etsy/deployinator

StatsD:Graphs, democratized

https://github.com/etsy/statsd

Code talks.

Make graphs!

Yes! No.

Dashboard Driven Development**

**60,000 graphs, ~30 of the most important ones on deploy dashboard

Errors*

*Vertical lines are deploys!

Key business metrics*

*Vertical lines are deploys!

Automated Testing

http://codeascraft.etsy.com/2011/04/20/divide-and-concur/

What continuous deployment gives Etsy

Agility: Always Be Shipping

MTTD / MTTR

Stability & predictability

Encourages experimentation

Developer happiness

DevOps

Scales trust and risk

“Not being in a state to deploy is a matter of liability. It's like having the only fire exit blocked. You ignore it at everyone's peril.”

— Lacy Rhoades, Etsy Engineer

“an ecosystem where the lazy, excitable and impatient really shine”

— Andrew Morrison, Etsy Engineer“The Product Hacking Ecosystem”

http://codeascraft.etsy.com/2012/01/04/the-product-hacking-ecosystem/

ResourcesCode as Craft bloghttp://codeascraft.etsy.com/

Slides (including these)http://www.slideshare.net/etsy

Github http://github.com/etsy/

Join in the fun http://www.etsy.com/careers/