BUILDING AND MANAGING RELIABLE INFRASTRUCTURE

Mandi Walls, Chef SoftwareIPExpo Europe, October 8, 2015

With Chef and Chef Delivery

WHO IS CHEF?• Founded about 8 years ago• Seattle-based, with offices in London and San

Francisco• Customers include web-natives, retail,

financial services• Core product, Chef, is client/server

configuration management

WHOAMI?• Director of Consulting and Customer

Success EMEA• Joined Chef in November 2011• @lnxchk• These slides will be uploaded

• http://www.slideshare.net/lnxchk

CORE CHEF

• Sophisticated toolset to produce reliable infrastructure

• Tested, versioned, recorded

THE CODED BUSINESS

• Infrastructure as Code – the underlying idea that computing components should be describable in software, versioned, and tested

• Expanded to include risk-reduction, value acceleration, and expanded trust

EASIER TO FIX BUGS BEFORE CODE GETS TO

PRODUCTION

PUSHING MORE INTO PRE-DEPLOY• Testing with ChefSpec, Test Kitchen,

Food Critic• Additional system controls via Chef

Audit• Don’t get to production and discover

that the security rules prevent the app from working

• Full 360° view of the application, no assumptions, everything documented in code

NEED A TOOL THAT MAKES SUBMITTING, APPROVING, AND

BUILDING FAST AND EASY

WHY DELIVERY?• Production of a repeatable pipeline for

software delivery• Software isn’t creating value at the end

of the build; it creates value after deployment

• Application of key concepts to all code, including the code that builds other code

NOT JUST APPLICATION CODE

• We’re now working with infrastructure code, too

• It should be auditable and traceable• Incorporate good change management

behaviors• Reduce risk

CHEF DELIVERY

HOW IT WORKSCLI

WebBrowser

Job DispatchPush

Chef Servererlang

Deliveryerlang

pgSQL

DBgit

SCM

Build NodeBuild NodeBuild Nodes

U-071982-C

U-071982-C

THE PIPELINE CONCEPT• The build server should reflect modern

distributed infrastructures• Preference for smaller, independent,

loosely coupled but dependent services to deliver a fully functioning application

• Individual software projects ship on their own schedule, but must integrate!

FAILURE TO SUCCESSFULLY INTEGRATE MEANS THE BUILD

SHOULD FAIL BEFORE PRODUCTION

STABLE PIPELINESteps

Create a new change1

Test ChangeLocally2

Verification Tests

SubmitChange3

ReviewChange4

Build ArtifactsAcceptance

Tests

ApproveChange5

Release Process

DeliverChange6

manual automated

U-071982-C

THE SHARED PIPELINE

• Across various products and teams• Coordination, integration• New releases can trigger refreshes on

dependent components

COMMON PIPELINEOne Pipeline

Delivery Pipeline - Shared

union rehearsal delivered

Acceptance Pipelines - Private

review approve deliverChangeCookbook [A]

review approve deliverChangeCookbook [B]

review approve deliverChangeApplication [A]

review approve deliverChangeApplication [B]

U-071982-C

Infrastructure & Applications

UNIFIED SHAPE• The stages are fixed. No change goes to

production without flowing through the stages• Stages include specific phases to reduce

confusion and increase predictability and stability – these are made up of tasks you’re probably already doing

• Your goal is to build software that has value for you, not bikeshed on the pipeline. We did that.

COMMON PIPELINESteps manual automated

VerifyLint

SyntaxUnit

BuildMergeLint

SyntaxUnit

QualitySecurityPublish

ProvisionDeploySmoke

FunctionalCompliance

Acceptance

UnionProvisionDeploySmoke

FunctionalCompliance

RehearsalProvisionDeploySmoke

FunctionalCompliance

DeliveredProvisionDeploySmoke

FunctionalCompliance

Stages

customizable

Verify BuildAcceptance

UnionRehearsalDelivered

SubmitChange3

ReviewChange4

ApproveChange5

DeliverChange6

Chef DeliveryCreate a new

change1Test Change

Locally2

Workstation

U-071982-C

CUSTOMIZE ACTIVITIES• Within the phases, the actions taken are

customized to meet the needs of the application• Syntax checking, usability testing, etc, can

make use of existing tools• Test nodes can be provisioned on a variety of

platforms and environments to meet application needs

BUILD COOKBOOK├── recipes   ├── default.rb

├── lint.rb ├── syntax.rb ├── unit.rb ├── quality.rb ├── security.rb ├── publish.rb ├── provision.rb

   ├── deploy.rb ├── smoke.rb

   └── functional.rb

U-071982-C

BUILD COOKBOOK• Treating the definition of the build

process with the same care as the code that is going through it

• Allows for versioning of the build definition, tracking of changes, linking of new build definition to new components

PHASE EXECUTION

log "Running unit"

repo = node['delivery_builder']['repo']

execute “run my junit tests" do command "mvn test" cwd repoend

U-071982-C

COOKBOOK WORKFLOW

U-071982-C

Supermarket

Chef Server

review approve deliverChangeCookbook

Node Node Node

Node Node Node

Node Node Node

Publi

sh

Publish

Update

APPLICATION WORKFLOW

U-071982-C

review approve deliverChangeApplication

Node Node Node

Node Node Node

Node Node Node

Deploy1 2 3

2 2 3

3 3 3

U-071982-C

CREATE NEW PIPELINES

• Project code repository• Build cookbook *versioned!*• Project configuration file *also

versioned!*

CONFIG.JSON

{ "version": "1", "build_cookbook": "./.delivery/build_cookbook", "build_nodes": { "default": [“name:*buildnode-*”],

"unit": [“name:*-unittest*”], "lint": [“name:*-linttest*”],

"build": [“name:*-builder*”] }}

U-071982-C

• Delivery gives you a mechanism to create fully-described build process for complex applications

• The build process itself benefits from the same version control and software practices the code uses

• Prevent bugs and regressions from getting to production with full integration retrospective testing

MORE ABOUT DELIVERYhttps://www.chef.io/deliver

y/

VISIT USBOOTH GG8

UPCOMING EVENTS• Next London Chef Meetup: October 20

• http://www.meetup.com/Chef-Users-London• Chef Cookbook Workflow @ AWS Popup Loft

• https://www.chef.io/blog/event/chef-cookbook-workflow-2/ • Chef Community Summit

• November 3-4 in London• https://www.chef.io/summit/london/

NEW TO CHEF?

• https://learn.chef.io/• Online trainings, in-person classroom

training, dates announced regularly

U-071982-C