zc.buildoutricardo newbery

ric@ncryptedcloud.com

a presentation given in november 2013Donetsk, Ukraine

Sunday, January 5, 14

• Need a process to build the application stack, including all dependencies.

• Requirements:

• repeatable and possible to automate

• concise and easy to understand/modify

• not just Python!

the problem

Sunday, January 5, 14

alternatives• app-get, yum, homebrew, etc.

• make, rake, paver, minitage, etc.

• puppet, chef, ansible salt, etc.

• virtualenv

• pip install -r requirements.txt

• fabric

Sunday, January 5, 14

what is buildout?The Buildout project (zc.buildout) provides support for creating applications. It provides tools for assembling applications from multiple parts, Python or otherwise. An application may actually contain multiple programs, processes, and configuration settings.

The word "buildout" refers to a description of a set of parts and the software to create and assemble them. It is often used informally to refer to an installed system based on a buildout definition.

For example, if we are creating an application named "Foo", then "the Foo buildout" is the collection of configuration and application-specific software that allows an instance of the application to be created. We may refer to such an instance of the application informally as "a Foo buildout"

Sunday, January 5, 14

buildout benefits• An automated and simple process to build and

maintain a consistent software stack across all environments (dev/qa/staging/production)

• Centralized specification of the major dependencies of the software stack, both Python and non-Python

• Centralized specification of the configuration parameters and scripts that define a stack build

Sunday, January 5, 14

a barebones buildoutTwo files: - buildout.cfg - bootstrap.py

Running the bootstrap script grabs the zc.buildout library and some minimal dependencies and creates a “bin/buildout” script.

Sunday, January 5, 14

buildout.cfg[buildout]parts = mydjango

[mydjango]recipe = zc.recipe.eggegg = Django

Sunday, January 5, 14

variable interpolation[buildout]parts = mydjangoeggs = django

[mydjango]recipe = zc.recipe.eggegg = ${buildout:eggs}

Sunday, January 5, 14

extends(buildout.cfg)

[buildout]extends = varnish.cfgparts += mydjangoeggs = Django Pillow

[mydjango]recipe = zc.recipe.eggegg = ${buildout:eggs}

(varnish.cfg)

[buildout]parts = myvarnish

[myvarnish]recipe = zc.recipe.varnishurl = http://...varnish.tar.gz

Sunday, January 5, 14

basic recipes• zc.recipe.egg

Install Python package distributions as eggshttp://pypi.python.org/pypi/zc.recipe.egg

• collective.recipe.templateGenerate a text file from a templatehttp://pypi.python.org/pypi/collective.recipe.template

• collective.recipe.cmdExecute a command linehttp://pypi.python.org/pypi/collective.recipe.cmd

• zc.recipe.cmmihexagonit.recipe.cmmiTwo different recipes to download and build a package using configure/make/make-installhttp://pypi.python.org/pypi/zc.recipe.cmmihttp://pypi.python.org/pypi/hexagonit.recipe.cmmi

• mr.scriptyUse python to write configurationhttp://pypi.python.org/pypi/mr.scripty

Sunday, January 5, 14

more recipes• collective.recipe.omelette

A zc.buildout recipe to create a unified directory structure of installed packages, symlinking to the actual contents, in order to ease navigation and developer inspection.http://pypi.python.org/pypi/collective.recipe.omelette

• mr.developerA zc.buildout extension to ease the development of projects with multiple packages. Provides some configuration sugar and scripts to maintain multiple source-controlled packages within a single buildout.http://pypi.python.org/pypi/zc.recipe.egg

• collective.recipe.environmentA zc.buildout recipe to set or look up environment variableshttp://pypi.python.org/pypi/collective.recipe.environment

• cns.recipe.symlinkA zc.buildout recipe to create symlinks (could do the same with collective.recipe.cmd)http://pypi.python.org/pypi/cns.recipe.symlink

• z3c.recipe.mkdirA zc.buildout recipe to create directories (could do the same with collective.recipe.cmd)http://pypi.python.org/pypi/z3c.recipe.mkdir

Sunday, January 5, 14

keep it simpleThere are too many buildout recipes out there (850+ on pypi alone!)

Avoid the temptation to create yet another recipe when you can do the same operation by just stringing together a few of the basic recipes.

For example, these two are functionally equivalent but the one on the right avoids adding yet another abstraction layer between the semantics of the buildout configuration and the varnish configuration:

[varnish]recipe = zc.recipe.cmmiurl = ${urls:varnish}

[varnish-recipe]recipe = plone.recipe.varnishdaemon = ${varnish:location}/sbin/varnishdbind = 127.0.0.1:8000backends = 127.0.0.1:8080cache-size = 256M

[varnish]recipe = zc.recipe.cmmiurl = ${urls:varnish}

[varnish-config]recipe = collective.recipe.templatepath = etc/varnish.vclinput = templates/${:path}.inoutput = ${buildout:directory}/${:path}

[varnish-runner]recipe = collective.recipe.templateinput = inline: #!/bin/sh exec ${varnish:location}/sbin/varnishd \ -f "${varnish-config:output}" \ -P "${buildout:directory}/var/varnish.pid" \ -a 0.0.0.0:${global:varnish-port} \ -s file,"${buildout:directory}/var/varnish_storage",1G "$@"output = ${buildout:directory}/bin/varnishdmode = 755

Sunday, January 5, 14

more...

• http://en.wikipedia.org/wiki/Buildout

• http://pypi.python.org/pypi/zc.buildout

• http://www.buildout.org/

Sunday, January 5, 14