Download - Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code
![Page 1: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/1.jpg)
Monday, 10 February 14
![Page 2: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/2.jpg)
@peterleschev
Husband, Father of 3 & Atlassian
Build Engineering Team Lead
Peter Leschev
Monday, 10 February 14
![Page 3: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/3.jpg)
A Build Engineering Team’s Journey of
Infrastructure as Code
Monday, 10 February 14
![Page 4: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/4.jpg)
• Build platform & services used internally within the company• 60k builds per month• 35k automated tests for JIRA
Build Engineering today @ Atlassian
Monday, 10 February 14
![Page 5: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/5.jpg)
• 600 build agents (own hardware + EC2 instances)• include SCM clients, JDKs, JVM build tools, databases, headless
browser testing, python builds, NodeJS, installers & more
• Maintain 20 AMIs of various build configurations• 6 Bamboo Servers• maven.atlassian.com / 6 Nexus instances • Monitoring - opsview / graphite / statsd
Build Engineering today @ Atlassian
Monday, 10 February 14
![Page 6: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/6.jpg)
Infrastructure as Code
= Puppet + SCM ?
Monday, 10 February 14
![Page 7: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/7.jpg)
• Manually maintained snowflakes• Started using puppet
3 years ago...
Monday, 10 February 14
![Page 8: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/8.jpg)
Production rollout
puppetmaster
build agents
Monday, 10 February 14
![Page 9: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/9.jpg)
Production rollout failure
puppetmaster
build agents
Monday, 10 February 14
![Page 10: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/10.jpg)
HIGH
NONE
Lifecycle of an infra change
Confidence of Change
Dev Rollout Soak in Prod
Monday, 10 February 14
![Page 11: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/11.jpg)
Monday, 10 February 14
![Page 12: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/12.jpg)
http://atlassian.com/git
https://bitbucket.org/
Monday, 10 February 14
![Page 13: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/13.jpg)
Style in Pull Requests
Monday, 10 February 14
![Page 14: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/14.jpg)
• Automated style checking• Setup automated build that runs checks & posts results• Still need to implement a ratchet build
Puppet Lint https://github.com/rodjek/puppet-lintTim Sharpe
@rodjek
Monday, 10 February 14
![Page 15: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/15.jpg)
HIGH
NONE
Lifecycle of an infra change
Confidence of Change
Dev Code review Rollout Soak in Prod
initial + Code review
Monday, 10 February 14
![Page 16: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/16.jpg)
• Coding on Puppet Master• Culture of manually modifying
production - Configuration Drift
• Impact on Builds
Using Staging for Development
puppetmaster
build agentsbuild agents
staging puppet environment
Monday, 10 February 14
![Page 17: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/17.jpg)
• Easily spin up Infrastructure locally on your laptop• Disposable / reproducible environments• Machine provisioning via Virtual Box / VMWare / AWS• Configuration applied via Shell Scripts / Puppet / Chef• Develop and test infrastructure changes locally
Vagrant http://www.vagrantup.com/Mitchell Hashimoto
@mitchellh
Monday, 10 February 14
![Page 18: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/18.jpg)
Vagrant
Vagrantfile
vagrant basebox
http://www.vagrantup.com/Mitchell Hashimoto
@mitchellh
Monday, 10 February 14
![Page 19: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/19.jpg)
Vagrant
Spins up a local VM to a known state
Destroy the VM when done
Make some puppet changes and then run:
to apply your changes
SSH into your VM using:
to check your changes
http://www.vagrantup.com/Mitchell Hashimoto
@mitchellh
Monday, 10 February 14
![Page 20: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/20.jpg)
HIGH
NONE
Lifecycle of an infra change
Confidence of Change
Dev Code review Rollout Soak in Prod
initial + Code review + Vagrant
Monday, 10 February 14
![Page 21: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/21.jpg)
• Vagrant basebox differences with production machines• Originally using publicly available vagrant baseboxes
• Installed packages biggest differences
• Generating a basebox manually was a painful process
Vagrant != Production
Monday, 10 February 14
![Page 22: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/22.jpg)
VeeweeAutomated
Vagrant basebox generationhttps://github.com/jedi4ever/veewee
Patrick Debois@patrickdebois
Ubuntu installation iso vagrant baseboxVeewee definitions.rbpreseed.cfgpostinstall.sh
+
Monday, 10 February 14
![Page 23: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/23.jpg)
Veeweehttps://github.com/jedi4ever/veewee
AutomatedVagrant basebox generation
Patrick Debois@patrickdebois
Monday, 10 February 14
![Page 24: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/24.jpg)
• Latest basebox generated in CI & published to fileshare• No need to generate baseboxes locally
Basebox generation via CI
Monday, 10 February 14
![Page 25: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/25.jpg)
• VirtualBox Guest additions• Reduced to a minimal
There are still differences!
Monday, 10 February 14
![Page 26: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/26.jpg)
Common Preseed / Postinstall
preseed.cfg postinstall.sh
+
custom ISOsvagrant basebox PXEBoot
Monday, 10 February 14
![Page 27: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/27.jpg)
Packer http://packer.ioMitchell Hashimoto
@mitchellh
Monday, 10 February 14
![Page 28: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/28.jpg)
HIGH
NONE
Lifecycle of an infra change
Confidence in Change
Dev Code review Rollout Soak in Prod
initial + Code review + Vagrant + Veewee
Monday, 10 February 14
![Page 29: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/29.jpg)
Developing locally
Rolling out to production
Broken build agents!
Rolling out to staging
Monday, 10 February 14
![Page 30: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/30.jpg)
• Behaviour Driven Development
Cucumber
Monday, 10 February 14
![Page 31: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/31.jpg)
Cucumber & Vagrant
Vagrant
Custom Provisioner
Virtual Box
VM
puppet apply
cucumber *.features
via ssh
Monday, 10 February 14
![Page 32: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/32.jpg)
• Requires cucumber dependencies to be installed on tested VM
• Tests run within the VM making testing firewall rules harder
Disadvantages
Monday, 10 February 14
![Page 33: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/33.jpg)
HIGH
NONE
Lifecycle of an infra change
Confidence in Change
Dev Code review Rollout Soak in Prod
initial + Code review + Vagrant + Veewee + Cukes
Monday, 10 February 14
![Page 34: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/34.jpg)
But it works on my machine!– Every Developer”“
Monday, 10 February 14
![Page 35: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/35.jpg)
• ‘From scratch’ provisioning• Confidence that you can rebuild in disaster
Continuous Integration
Monday, 10 February 14
![Page 36: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/36.jpg)
The Pets: you give nice names,
you stroke them, and when they get ill,
you nurse them back to health,
taking a long time over it
– Tim Bell, CERN”
“
The Cattle: you give them numbers.
When they get ill, you shoot them
Monday, 10 February 14
![Page 37: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/37.jpg)
HIGH
NONE
Lifecycle of an infra change
Confidence in Change
Dev Code review CI & Rollout Soak in Prod
initial + Code review + Vagrant + Veewee + Cukes + CI
Monday, 10 February 14
![Page 38: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/38.jpg)
Provisioning from scratch is slow
Monday, 10 February 14
![Page 39: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/39.jpg)
Spread out CI
provision VM1
provision VM2
provision VM3
provision VM4
provision VM1
provision VM2 provision VM3
provision VM4Moved from sequentialto parallel provisioning
Monday, 10 February 14
![Page 40: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/40.jpg)
There are so many MacPros you can steal
Monday, 10 February 14
![Page 41: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/41.jpg)
The onesI have my eye on....
Monday, 10 February 14
![Page 42: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/42.jpg)
Profiling Puppet Runs
Add “--evaltrace” to puppet apply
+ =Collect and show the longest occurrences of:“Evaluated in ([\d\.]+) seconds”
Monday, 10 February 14
![Page 43: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/43.jpg)
Profiling Cucumber runs
http://itshouldbeuseful.wordpress.com/2010/11/10/find-your-slowest-running-cucumber-features/
Monday, 10 February 14
![Page 44: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/44.jpg)
• Provision locally & for CI• Faster & different class of problems found• Matches production state
Delta Provisioning
‘from scratch’ provision delta provision
provision VM1
export VM1 fileshare
import VM1 box
provision VM1
on success
Monday, 10 February 14
![Page 45: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/45.jpg)
HIGH
NONE
Lifecycle of an infra change
Confidence in Change
Dev Code review CI & Rollout Soak in Prod
initial + Code review + Vagrant + Veewee + Cukes+ CI + Delta CI
Monday, 10 February 14
![Page 46: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/46.jpg)
Infrequent Releases
Monday, 10 February 14
![Page 47: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/47.jpg)
• Puppet runs impacted running builds• Disabling all the build agents
• Performing the roll out
• git clone / librarian-puppet / symlink update on puppetmaster
• Manually kick off puppet on all the build agents
• Enabling all the build agents
• Set of Puppet environments for every bamboo server
Painful Puppet Rollouts
Monday, 10 February 14
![Page 48: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/48.jpg)
Graceful Service restarts
+Bamboo Agent JVM process watches for touch file & shutdowns when Idle(written as a Bamboo Plugin)
Monday, 10 February 14
![Page 49: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/49.jpg)
• BEFORE - Multiple puppet envs for each Bamboo Server• jbac_staging
• jbac_production
• cbac_staging
• cbac_production
• etc
• AFTER - Changed to use ‘staging’ & ‘production’ only
Puppet Environments
Monday, 10 February 14
![Page 50: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/50.jpg)
• BEFORE: Manually on puppetmaster• git clone the puppet tree
• run librarian-puppet to pull external modules
• Update staging / production symlink
• AFTER: Bamboo build which performs the above steps automatically
Updates on Puppetmaster
Monday, 10 February 14
![Page 51: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/51.jpg)
Less Human interaction +
More automation=
Higher Confidence
Monday, 10 February 14
![Page 52: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/52.jpg)
Less Human Effort =
Increased frequency of releases
Monday, 10 February 14
![Page 53: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/53.jpg)
HIGH
NONE
Lifecycle of an infra change
Confidence in Change
Dev Code review CI & Rollout Soak in Prod
initial + Code review + Vagrant + Veewee+ Cukes + CI + Delta CI + Frequent releases
Monday, 10 February 14
![Page 54: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/54.jpg)
Should I be scared?– Peter Leschev, 3 months ago”“
I’m scared!– Peter Leschev, 3 years ago”“
Monday, 10 February 14
![Page 55: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/55.jpg)
Hipchat integration
Monday, 10 February 14
![Page 56: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/56.jpg)
HIGH
NONE
Lifecycle of an infra change
Confidence in Change
Dev Code review CI & Rollout Soak in Prod
initial + Code review + Vagrant + Veewee+ Cukes + CI + Delta CI + Frequent releases+ Notification
Monday, 10 February 14
![Page 57: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/57.jpg)
HIGH
NONE
Lifecycle of an infra change
Confidence in Change
Dev Code review CI & Rollout Soak in Prod
before after
Monday, 10 February 14
![Page 58: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/58.jpg)
Confidence in Change
or
Finding & fixing problems sooner rather
than later
Monday, 10 February 14
![Page 59: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/59.jpg)
Commit Graph
Monday, 10 February 14
![Page 60: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/60.jpg)
Snowflakes
Pets
Cattle
Stateless Machines
Monday, 10 February 14
![Page 61: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/61.jpg)
We’re still on the Journey
Come join us!
atlassian.com/jobs
Monday, 10 February 14
![Page 62: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/62.jpg)
Questions?
Monday, 10 February 14
![Page 63: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/63.jpg)
Thank you!
Monday, 10 February 14
![Page 64: Puppet Camp Sydney Feb 2014 - A Build Engineering Team’s Journey of Infrastructure as Code](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f40f1b4c90572088b52ff/html5/thumbnails/64.jpg)
Monday, 10 February 14