Why continuous delivery needs devops, and why devops needs

infrastructure-as-code

Sriram Narayan

@sriramnarayan

25-Oct-2012

about me

• Part of ThoughtWorks Studios Go team

• Have consulted as Tech Principal, Agile Coach, organization development coach

• Been a manager, leadership development coach and innovation facilitator

• Written production code in Java, C#, Ruby and Python

2010

2012

About ThoughtWorks Studios

Training on agile foundations, project management, testing, continuous delivery

www.thoughtworks-studios.com/services/agile-workshops

Radar and Anthology

Executive summary

• Why does CD need devops?

– Because you can’t have CD with a siloed org

– devops helps blur the strict boundary between dev and ops.

• Why does devops need IaC?

– IaC helps create a common currency between dev and ops

– Helps skills crossover

definitions

Continuous delivery is

an approach to delivering software that

reduces the cost, time, and risk

of delivering incremental changes to users.

ThoughtWorks Anthology, March 2008

I think of it as agile software delivery/release instead of agile software development

1. Your software is releasable

throughout its lifecycle

2. Your team prioritizes keeping the software releasable over working on new features

3. Anybody can get fast, automated feedback on the production readiness of your systems any

time somebody makes a change to them

4. You can perform push-button releases of any version of the software on demand.

It is a high bar

And it isn’t easy

devops

theagileadmin.com/what-is-devops/

system administrators participating in an agile development process alongside developers and using a many of the same agile techniques for their systems work wikipedia

a response to the interdependence of software development and IT operations. It aims to help an organization rapidly produce software products and services.

Q. Why does CD need devops?

A. Because you can’t have CD with a siloed org and devops helps blur the

strict boundary between dev and ops.

A typical setup (pre-devops)

http://www.ediblegeography.com/wp-content/uploads/2012/09/steel-silo-460.jpg

A typical anti-pattern (post-devops)

http://www.ediblegeography.com/wp-content/uploads/2012/09/steel-silo-460.jpg

Silo trouble

• What is a silo?

• Why are silos bad for CD?

• What causes silos?

– Understand the causes so that we can take preventive measures

What is a silo?

http://wiki.answers.com/Q/What_are_organizational_silos

Organizational silos…are (often) dysfunctional units or departments within an enterprise, characterized by their tendency to protect themselves...they are sometimes the result of unchecked "empire building" by middle management. A less harsh attempt… Anything that tends to localize a (technical) competency and create handoffs in a delivery value stream.

Why are silos bad for CD?

• Silos mean handoffs

• Handoffs

– Cannot be continuous or real-time

– Communication protocol (e.g. SPOC, minutes)

– Work queues, turn around times, follow ups, priority escalations

• Can only work with large batch size

But...

We all want shorter cycle times

• This requires small batch size

• Handoffs double when batch size halves

– Too much communication overhead

– Just not responsive enough

• Ergo, we reduce hand-offs by tearing down silos

Things that promote a silo

• Org structure, reporting hierarchy

• Geographic separation

• Speciality tools

– Limited licenses to go around

– e.g. salesforce accounts, expensive deployment tools

• Speciality teams

1. Extreme case: dev team has no people from other functions (7 silos)

2. Many orgs now merge development, build and testing into one team (5 silos)

3. devops tries to merge operations into what then becomes the IT team

Towards autonomous cross-functional teams

7 silos 5 silos no silos

ARCHITECTURE ARCHITECTURE

DEVELOPMENT

BUILD

DATABASE

TESTING DATABASE

OPERATIONS OPERATIONS

MAINTENANCE MAINTENANCE 3 cross-functional teams

Silo mitigation – team and org design

• Organize along business outcomes

• Create semi-autonomous teams that own outcomes and minimize geographic separation

• Distribute specialists among the teams rather than create a speciality team

– Responsiveness over Efficiency

• It’s ok if specialist utilization falls

– Generalizing specialist, T-shaped people

Silo mitigation – choice of tools

• Use tools & techniques that blur boundaries between specialists rather than define them

– Same source code repo for app, build and deploy

– Same wiki tool, work management tool

• One over-arching tool for the entire continuous delivery value stream

• Instead of different tools for CI and deployment

http://www.thoughtworks.com/radar

Oct 22, 2012

Q. Why does devops need infrastructure-as-code?

A. To create a common currency between dev and ops, which in turn helps mutual

transfer of skills and insights

infrastructure-as-code

• Infrastructure described via code

– Server and network config, base builds, packages, environments, services and config files

• A domain model for infrastructure

• First class infrastructure primitives that allow for a declarative description of desired state

• The phoenix test

– Bare metal, infrastructure code and data backup

The radar on infrastructure-as-code July 2011

March 2012

Oct 2012

http://wiki.opscode.com/display/chef/Chef+Concepts+as+UML

domain model

From Bryan Beery http://faodata.blogspot.in/2012/04/chef-testimonial-data.html

Chef: resources in a recipe

Chef: data bag

Chef: scripting a master-slave config

IaC samples

Common currency Application code

Infrastructure code

The power of text

• Text lends itself better to collaboration and automation

– Non-proprietary version control, diff and merge

– So much tooling is available

• Code and config in the form of text descriptors makes transactions between specialists more fluid.

IaC facilitated crossover

What devs learn from ops

• Resilience

• Cloud, Virtualization

• Data centres

• Inventory

• Backups & Audits

• Monitoring

What ops learn from devs

• Domain modelling

• Common vocabulary

• Continuous integration

• Everything in version control

• DRY

• Automated testing

Putting it all together

build and unit test acceptance

deploy-Performance

deploy-QA

deploy-production

App Source

pkg repo Infra Code

source, tests

binaries

recipes etc

target nodes

pipeline trigger

publish artifacts

devops

continuous delivery

Infrastructure-as-code App dev

one cross-functional team of generalizing specialists

text descriptors in version control

One tool for traceability, orchestration, visualization

iterative, not a linear flow from left to right

discover analyze validate

architect design

develop build

test acceptance

performance exploratory

provision inventory

audit

deploy upgrade rollback

monitor backup

Thank you

Please use the Q&A box for questions or comments