reaching deployment nirvana using buildout

open sourcenowhere to go but

sixfeetup.com/dw08

ReachingDeployment Nirvana

Using BuildoutClayton Parker

Plone Conference 2008

sixfeetup.com/dw08

Here’s to Plone

sixfeetup.com/dw08

• Buildout

• Parts

• Recipes

• Command Line

• ZopeSkel

• Create Buildouts

• Custom Recipes

To-do List

sixfeetup.com/dw08

Why buildout?

sixfeetup.com/dw08

Eliminate Confusion

sixfeetup.com/dw08

Tracking Dependencies

sixfeetup.com/dw08

Roll Your Own

• Installer

• ZopeSkel

sixfeetup.com/dw08

ZopeSkel

$ easy_install ZopeSkel$ paster create --list-templatesplone3_buildout: A buildout for Plone 3 projects$ paster create -t plone3_buildout mybuildout

sixfeetup.com/dw08

Lingo

sixfeetup.com/dw08

Syntax[buildout]parts = zope2 plone instance

[plone]recipe = plone.recipe.plone

[zope2]recipe = plone.recipe.zope2install

[instance]recipe = plone.recipe.zope2instanceuser = admin:admineggs = ${plone:eggs}products = ${plone:products}

sixfeetup.com/dw08

Advanced Syntax

${part:option}

${plone:eggs}${buildout:directory}option = ${buildout:directory}/somefolder

Variable Substitution

options = foo bar

options += bazoptions -= foo

Option Addition and Removal

sixfeetup.com/dw08

Reserved Characters

: $ % { }

sixfeetup.com/dw08

Parts

sixfeetup.com/dw08

[buildout]parts = instance

[instance]recipe = plone.recipe.zope2instanceuser = admin:admineggs = ${plone:eggs} archetypes.schemaextenderzcml = archetypes.schemaextenderproducts = ${plone:products}environment-vars = TZ America/Chicagohttp-address = 51060zeo-address = 53060debug-mode = onverbose-security = onevent-log-level = debug

sixfeetup.com/dw08

Recipes

sixfeetup.com/dw08

[buildout]parts = zope2 plone instance

[plone]recipe = plone.recipe.plone

[zope2]recipe = plone.recipe.zope2install

[instance]recipe = plone.recipe.zope2instanceuser = admin:admineggs = ${plone:eggs}products = ${plone:products}

sixfeetup.com/dw08

Buildout Directory• bin/

• bootstrap.py

• buildout.cfg

• develop-eggs/

• downloads/

• eggs/

• parts/

• products/

• src/

sixfeetup.com/dw08

Buildout options• bin-directory

• develop-eggs-directory

• eggs-directory

• parts

• parts-directory

• log-level

sixfeetup.com/dw08

Defaults

[buildout]eggs-directory = /home/clayton/.buildout/eggsdownload-cache = /home/clayton/.buildout/downloadszope-directory = /home/clayton/.buildout/zopeindex = http://download.zope.org/ppix

[instance]event-log-level = debug

sixfeetup.com/dw08

Recipes

sixfeetup.com/dw08

Plone Recipes

• plone.recipe.plone

• plone.recipe.zope2install

• plone.recipe.zope2instance

• plone.recipe.squid

sixfeetup.com/dw08

plone.recipe.zope2install

[zope2]recipe = plone.recipe.zope2installurl = ${plone:zope2-url}fake-zope-eggs = trueadditional-fake-eggs = ZODB3skip-fake-eggs = zope.testing zope.component zope.i18n

sixfeetup.com/dw08

In The Wild

sixfeetup.com/dw08

Extending Configuration• buildout.cfg

• profiles/

• base.cfg

• development.cfg

• debug.cfg

• qa.cfg

• prod.cfg

• versions.cfg

sixfeetup.com/dw08

buildout.cfg

[buildout]# the profile we want to useextends = profiles/development.cfg

find-links = http://dist.plone.org http://download.zope.org/distribution/ http://effbot.org/downloads

sixfeetup.com/dw08

base.cfg[buildout]parts = PILwoTK plone zope2 instance zeoserverdevelop = src/my.package

[PILwoTK]recipe = zc.recipe.eggfind-links = http://download.zope.org/distribution/

[zope2]recipe = plone.recipe.zope2installurl = ${plone:zope2-url}fake-zope-eggs = trueadditional-fake-eggs = ZODB3skip-fake-eggs = zope.testing zope.component zope.i18n

sixfeetup.com/dw08

[plone]recipe = plone.recipe.plone

[instance]recipe = plone.recipe.zope2instancezeo-client = Truezope2-location = ${zope2:location}eggs = PILwoTK elementtree ${plone:eggs} my.packagezcml = my.packageproducts = ${plone:products}environment-vars = TZ America/New_York

[zeoserver]recipe = plone.recipe.zope2zeoserverzope2-location = ${instance:zope2-location}zeo-address = ${instance:zeo-address}

sixfeetup.com/dw08

development.cfg[buildout]extends = base.cfg debug.cfgparts += ${debugging:parts} omelette

[instance]user = admin:adminhttp-address = 8080zeo-address = 8100debug-mode = onverbose-security = onevent-log-level = debugproducts += ${buildout:directory}/products ${debugging:debug-products}eggs += ${debugging:eggs}zcml += ${debugging:zcml}

[omelette]recipe = collective.recipe.omeletteeggs = ${instance:eggs}ignore-develop = Trueignores = setuptools

sixfeetup.com/dw08

debug.cfg[debugging]parts = debug-products debug-products-svn ipzope zopepydebug-products = ${debug-products:location}eggs = plone.reload Products.PDBDebugMode Products.DocFinderTab Products.Clouseau Products.PrintingMailHostzcml = plone.reload

sixfeetup.com/dw08

[debug-products]recipe = plone.recipe.distrosurls = ...dcworkflowgraph-0_3.tgz ...PTProfiler-1.2.tgz

[ipzope]recipe = zc.recipe.eggeggs = ipython ${instance:eggs}initialization = import sys, os os.environ["SOFTWARE_HOME"] = "${instance:zope2-location}/lib/python" os.environ["INSTANCE_HOME"] = "${instance:location}" sys.argv[1:1] = "-p zope".split()extra-paths = ${instance:zope2-location}/lib/pythonscripts = ipython=ipzope

[zopepy]recipe = zc.recipe.eggeggs = ${instance:eggs}interpreter = zopepyextra-paths = ${instance:zope2-location}/lib/pythonscripts = zopepy

sixfeetup.com/dw08

versions.cfg[versions]# Use the following from the command line to get the latest versions:# bin/buildout -vvvvv |sed -ne 's/^Picked: //p' | sort | uniqPILwoTk = 1.1.6.4elementtree = 1.2.7-20070827-previewinfrae.subversion = 1.1plone.recipe.distros = 1.3plone.recipe.plone = 3.1.6plone.recipe.zope2install = 2.3plone.recipe.zope2instance = 2.5plone.recipe.zope2zeoserver = 0.13python-openid = 2.2.1my.package = 1.0

sixfeetup.com/dw08

prod.cfg[buildout]extends = base.cfg versions.cfgparts += instance2versions = versions

[instance]zope2-location = ${zope2:location}http-address = 9080zeo-address = 9100eggs += Products.CacheSetupz2-log-level = CRITICALzodb-cache-size = 15000

[instance2]recipe = collective.recipe.zope2clusterinstance-clone = instancehttp-address = 10080

sixfeetup.com/dw08

Command Line

sixfeetup.com/dw08

Baby Steps$ cd path/to/mybuildout

$ python2.4 bootstrap.pyCreating directory 'mybuildout/bin'.Creating directory 'mybuildout/parts'.Creating directory 'mybuildout/develop-eggs'.Generated script 'mybuildout/bin/buildout'.

$ bin/buildout

$ bin/instance start

sixfeetup.com/dw08

Options• -v and -q

• increase and decrease verbosity

• -n and -N

• Newest and non-newest modes

• -O and -o

• online and offline mode

• -t

• socket timeout

sixfeetup.com/dw08

Update your buildout

$ bin/buildout -v

$ bin/buildout -Nvvv

$ bin/buildout -No

$ bin/buildout -t 60

sixfeetup.com/dw08

Assignments

$ bin/buildout instance:debug-mode=on

$ bin/buildout buildout:log-level=70

$ bin/buildout -N instance:debug-mode=on -v

sixfeetup.com/dw08

Commands

$ bin/buildout install

$ bin/buildout -Nv install zope2 instance

$ bin/buildout -nv install instance

sixfeetup.com/dw08

Create Recipes

$ paster create -t recipe my.recipe.example

sixfeetup.com/dw08

Recipe

• Recipe class

• constructor

• install

• update

• uninstall (optional)

sixfeetup.com/dw08

class Recipe: """"A recipe """" def __init__(self, buildout, name, options): self.buildout = buildout self.name = name self.options = options # gather options from other parts here options['notmine'] = buildout['someotherpart']['foobar'] def install(self): """"Install method """ options = self.options location = options['location'] # must return a string, or an iterable of strings return location def update(self): """Update method """ pass

sixfeetup.com/dw08

Real Worldclass Recipe: """infrae.subversion recipe. """

def __init__(self, buildout, name, options): self.buildout = buildout self.name = name self.options = options options['location'] = self.location = os.path.join( buildout['buildout']['parts-directory'], self.name) self.urls = [l.split() for l in options['urls'].splitlines() if l.strip()] self.export = options.get('export') self.newest = ( buildout['buildout'].get('offline', 'false') == 'false' and buildout['buildout'].get('newest', 'true') == 'true' ) self.verbose = buildout['buildout'].get('verbosity', 0)

sixfeetup.com/dw08

Install Method

def install(self): """Checkout the checkouts. """ for (url, name) in self.urls: wc = py.path.svnwc(self.location).join(name) if self.export: raise Exception('Unimplemented feature') if self.verbose: print "Fetch %s" % url wc.checkout(url) return self.location

sixfeetup.com/dw08

def update(self): """Update the checkouts""" if not self.newest: return self.location if self.export: return self.location if self.options.get('ignore_updates', False): return self.location num_release = re.compile('.*@[0-9]+$') part = py.path.local(self.location) for link, sub_path in self.urls: if num_release.match(link): if self.verbose: print "Given num release for %s, skipping." % link continue wc = py.path.svnwc(self.location).join(sub_path) if self.verbose: print "Updating %s" % link wc.update() return self.location

sixfeetup.com/dw08

Wrapping Up• Buildout

• Parts

• Recipes

• Command Line

• ZopeSkel

• Create Buildouts

• Custom Recipes

Plone Deployment Workshop

Register by October 17and save $100!

sixfeetup.com/dw08

sixfeetup.com/dw08

Links

• http://buildout.zope.org

• http://pypi.python.org/pypi/zc.buildout

• https://svn.sixfeetup.com/svn/public/buildout/debug.cfg

• http://www.sixfeetup.com/swag/buildout-quick-reference-card

• http://plone.org/documentation/tutorial/buildout

• http://pypi.python.org

sixfeetup.com/dw08

Photo Credits• http://flickr.com/photos/monsieurlam/2645956083/

• http://flickr.com/photos/_boris/2796908072/

• http://flickr.com/photos/b-tal/163450213/

• http://flickr.com/photos/bullish1974/2648544508/

• http://flickr.com/photos/haydnseek/87432002/

• http://flickr.com/photos/disowned/1158260369/

• http://flickr.com/photos/7603557@N08/2662531345/

• http://flickr.com/photos/julishannon/2151986631/

• http://flickr.com/photos/julishannon/2152778524/

• http://flickr.com/photos/lollyknit/1155225799/

• http://flickr.com/photos/binary_koala/86227485/