Dive into CHEF

Eduardo S. Scarpellini@escarpellini

Aug, 2016

Agenda

- Architecture;- Main tools;- Cooking;

- cookbooks, recipes, resources, templates, etc;

- Knife;- LWRPs;- Tests;- Best practices;

Architecture

Main tools

● Development workstation:○ chef-dk;

■ cookbook standardized skeletons, linting and testing tools, style checking;○ knife;

■ swiss knife: cookbook/node management, search, many plugins (AWS, OpenStack,

VMWare, etc);○ chef-kitchen (or any other testing tool);

■ runs acceptance criterias in a brand new/isolated environment (docker, vagrant, etc);

● Nodes:○ chef-client;○ ohai;

■ node attributes: platform and kernel details, network configuration, hostname, cpu and

memory data, etc;

Cookbook building blocks

● Metadata;○ author, version, dependencies, documentation, etc;

● Resources;○ desired state, several types (package, service), name+parameters, notifications/subscriptions;

● Attributes;○ global hash table, precedence/override;

● Files and templates;○ static or dynamic files (erb templates) that should be synchronized with nodes;

● Recipes;○ collection of resources;

● Libraries, definitions and LWRPs;○ custom resource types, ruby functions;

Cookbook anatomychef generate <cookbook, recipe, attribute, lwrp, ...>

attributes

files repository

recipes

templates repository

cookbook

Recipes: basics

● Ruby DSL;● Collection of resources;

○ package, cookbook_file, directory,

template, service, execute, user,

group, cron, dsc_resource, etc.

● Interaction with attributes;○ user defined or ohai;

● Everything is evaluated in the same order that they are declared;

● Can include other recipes;

{ yum install …apt-get install …msiexec.exe /i …

type + name

} parameters

notification

{ service … systemctl … sc.exe …

myapp/recipes/default.rb

Recipes: a bit more complexohai attribute

Powerful of Ruby: syntax, variables, data structures, conditionals, loopings, libraries, gems, etc.

include external code

local variables

attribute

notification timer

Requires attribute definition:

myapp/recipes/default.rb

myapp/attributes/default.rb

Templatesohai attributeCombine plain text with

Ruby code (<% … %>) for variable substitution and flow control. loop + attribute +

local variables

myapp/templates/default/haproxy.cfg.erb

myapp/attributes/default.rb

haproxy.cfg

What else...

● Attributes:○ defined/accessed in/from attribute files, recipes, roles, environments and/or node JSONs;○ precedence/overwritten (node.override > node.set > node.default);○ accessed via node['key'] or node.key

● Data bags:○ global variables stored as JSON files;○ can be encrypted (suitable for passwords, shared keys, etc);○ accessed via Chef::EncryptedDataBagItem.load('bag_name', 'bag_item')

● Roles describes nodes (group cookbooks, overwrite attributes);● Nodes (JSON):

○ Node attributes (ohai + defined by cookbooks);○ run_list: list (array) of roles, cookbooks and/or recipes that will run on the node.

Knife

● Provides an interface between a local chef-repo and the Chef server;● Extensible;● Helps to manage:

○ nodes:■ knife node <create, edit, delete ,...>

○ cookbooks, roles and data bags:■ knife cookbook <create, delete, upload, ...>■ knife role <create, edit, delete, ...> ■ knife data bag <create, edit, delete, ...>

○ cloud/hypervisors/servers (plugins):■ knife ec2 server create…■ knife bootstrap <FQDN>…

LWRPs

● Is a simple extension of Chef;● Is implemented as part of a cookbook;● Follows easy, repeatable syntax patterns;● Effectively leverages resources that are built into Chef;● Is reusable in the same way as built-in resources;

LWRPsmyapp/providers/hello_world.rbmyapp/resources/hello_world.rb

resource parameter

usage:

resource nameresource action

resource attributes

Kitchen

● Test cookbook data across any combination of platforms and test suites;● Defined in a YAML file (.kitchen.yaml);● Uses a driver plugin architecture;

○ many cloud providers and virtualization technologies ■ AWS, VMWare, Docker, Vagrant, openstack, etc;

○ automatic chef-server provisioning;■ environments, roles, data bags, dependencies (via Berkshelf), etc;

● Supports all common testing frameworks that are used by the Ruby community;○ bats (bash), minitest, rspec, serverspec;

Kitchen will run all .bats files against all specified platforms

Kitchen

myapp/.kitchen.yml

test/integration/default/bats/lb_haproxy.bats

{

driver plugin

run_list + attributes

bash commands

Kitchenkitchen test

successful

fail

platform/instance

Best practices

● Plan in advance;○ environments, roles, data bags and attributes hierarchy, naming conventions;

● Use foodcritic integrated to your favorite editor (sublime, atom);● Versioning your cookbooks (at scm and at chef-server level);

○ use scm tags;○ use one repository for each cookbook;○ freeze stable versions on chef-server;○ specify versions of the dependencies on metadata.rb;○ specify versions of the application cookbooks either on roles or run_list;

● Use Berkshelf to manage your dependencies;● Use search(...) instead hard-coded attributes whenever it's possible;

○ load-balancer members, cluster members, etc;

Best practices

● Build cookbooks that support external templates;● Avoid repetition: use LWRPs;● Don't reinvent the wheel: https://supermarket.chef.io/

○ don't fork;○ build wrapper cookbooks that match your needs;○ replace original templates when it's needed;

● Write/run tests;○ rely on chef built-in resources and write tests only for your code;

● Let your CI tool take care of your publishing process;○ checkout + run tests + version bump + cookbook upload;

Going further

● Getting started:○ https://docs.chef.io/chef_overview.html○ https://www.linode.com/docs/applications/chef/beginners-guide-chef

○ https://www.digitalocean.com/community/tutorial_series/getting-started-managing-your-infrastr

ucture-using-chef

● Books:○ Chef Infrastructure Automation Cookbook;○ Learning Chef: A Guide to Configuration Management and Automation○ Customizing Chef;

Questions?