Download - A Metadata Ocean in Chef and Puppet
A metadata ocean in Puppet and Chef FOSDEM’14
Marc Cluet DevOps & Automation Advisory Rackspace
http://www.flickr.com/photos/elias_daniel/7470265444/
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Who am I?!Marc Cluet (@lynxman)
More than 16 years of experience as a SysAdmin Founding member of Juju and MAAS while at Canonical Leading a DevOps Engineering Team at Rackspace Likes DevOps, programming and long walks on the beach
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
What is Rackspace?!Founded in 1998 in San Antonio, TX by three guys that wanted to create a hosting company Home of Fanatical Support /o/ Second biggest Public Cloud in the world OpenStack Project co-founder
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
What is Metadata?!
/http://www.flickr.com/photos/boojee/3743753784/
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
What is Metadata?!
http://www.flickr.com/photos/ggvaidya/8256283184/
Metadata is “data about data”!Structural Metadata!Descriptive Metadata!
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Metadata in Chef!Metadata is compiled on the client at execution time!Metadata is always stored in its entirety in the node[] dictionary!$node[var] = val
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Metadata in Chef!You can add metadata in your!
Attribute File!Node/Recipe!Environment!Role!
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Organisation in Chef!
http://docs.opscode.com/essentials_environments.html
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Metadata in Puppet!Metadata is always compiled on execution timeMetadata vars can’t be reassigned!$::var = val
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Metadata in Puppet!You can add metadata in your!!Class vars!!Nodes!!Top Scope!
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Scoping in Puppet!
http://docs.puppetlabs.com/puppet/latest/reference/lang_scope.html
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Hiera in Puppet!Solves metadata fragmentation issues!Defines hierarchy for easy organisation!All in clean and properly styled in either YAML or JSON!
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Hiera in Puppet!:backends: - yaml - json:yaml: :datadir: /etc/puppet/hieradata:json: :datadir: /etc/puppet/hieradata:hierarchy: - "%{::clientcert}" - "%{::custom_location}" - common
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK http://www.flickr.com/photos/musebrarian/3289649684/
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Bad metadata practices!In Chef!
Not keeping consistency between cookbook scope and environment scope!Not removing metadata from nodes when deprecated!Directly assigning metadata in a cookbook without a default attribute!
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Bad metadata practices!In Puppet!
Directly assigning metadata in a class without a default value!Assigning vars in nodes.pp!Not assigning global vars in site.pp!
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK http://www.flickr.com/photos/deepstereo/9616389940/
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Good behaviours!Separate vars from execution!
Do not store vars on the execution area (classes/cookbooks)!
High abstraction, good hierarchy!Metadata is all about context!
!
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Good behaviours!
Global
Environment
Node
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK http://www.flickr.com/photos/monica_andre/3119991125/
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Metadata Across Providers!
Provisioner cloud-init
Config Mgmt Puppet
$dog = cute $::dog = big
WHO IS RIGHT?!?!
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Metadata Across Providers!
http://www.flickr.com/photos/pss/233841567/
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Hard Problem to Solve!
Metadata brain split problem!Gets worse by adding more players!
System view varies!Not same info from bios than from a user level program as facter or ohai!
!
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Possible Solutions!Central DB using something fast!!Redis, mongoDB, mySQL!
Metadata translators might fix it?!!Write for everything!
Creates new SPoF!!Hard to solve without throwing $$!
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Hard Problem to Solve!
http://xkcd.com/927/
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Possible Solutions!
Scope vars from top of chain!Pass them over as ext facts!$dog in cloud-init becomes $::dog in puppet!
27 RACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.CO.UK
RACKSPACE® HOSTING | 5 MILLINGTON ROAD | HAYES, UNITED KINGDOM UB3 4AZ UK SALES: +44 (0)20 8712 6507 | UK SUPPORT: 0800 988 0300 | WWW.RACKSPACE.CO.UK
@lynxman!
http://slideshare.net/lynxmanuk/!
http://www.rackspace.co.uk/technicalservices/ !
!