puppet camp munich 2013: puppet at the centre of everything
DESCRIPTION
"Puppet at the centre of everything" by David Mytton of Server Density presented at Puppet Camp Munich 2013TRANSCRIPT
Puppet at the centre of everything
David Mytton
Woop Japan!
Server Density Architecture
•100 servers - Ubuntu 12.04
Server Density Architecture
•100 servers - Ubuntu 12.04
•50% virtual
Server Density Architecture
•100 servers - Ubuntu 12.04
•Nginx, Python, MongoDB
•50% virtual
Server Density Architecture
•100 servers - Ubuntu 12.04
•Nginx, Python, MongoDB
•25TB data per month
•50% virtual
Use cases
• Config
Use cases
• Config
• Failover
Use cases
• Config
• Code deploys
• Failover
Use cases
• Config
• Code deploys
• System updates
• Failover
• Base manifest
Config
• Base manifest
Config
• Sub manifests
• base
• softlayer
• softlayer-dedicated
• rackspace
• ...
Failover
• /etc/hosts
<%= metrics2IP %> alerts.honshuu.prod
audit.honshuu.prod auth.honshuu.prod
cloud.honshuu.prod datastore.honshuu.prod
inventory.honshuu.prod ...
Failover
• /etc/hosts
108.168.254.33 puppet
Expected
Dealing with outages
Really test your vendors
Dealing with outages
Simulations
Dealing with outages
• Commit
Code deploys
• Commit
Code deploys
• Build
• Commit
Code deploys
• Build
• Deploy
/opt/puppet/bin/mco puppetd runonce -I <host>
• Commit
Code deploys
• Build
• Deploy
• Puppet run
Code deploys
Advantages
APIs
Code deploys
Advantages
APIs
Already there
Code deploys
Advantages
APIs
Already there
System facts
Code deploys
Disadvantages
Slow
Code deploys
Disadvantages
Slow
Puppet run already
Code deploys
Disadvantages
Slow
Puppet run already
Eventually consistent
System updates
• Bash script
System updates
• Bash script
• Remote commands
System updates
• Bash script
• Remote commands
• Notifications
System updates
• Bash script
• Remote commands
• Notifications
• Canaries
•Build master (buildbot): VM x2 CPU 2.0Ghz, 2GB
RAM – $89/m
•Build slave (buildbot): VM x1 CPU 2.0Ghz, 1GB
RAM
– $40/m
•Staging load balancer: VM x1 CPU 2.0Ghz, 1GB
RAM – $40/m
•Staging server 1: VM x2 CPU 2.0Ghz, 8GB RAM
– $165/m
•Staging server 2: VM x1 CPU 2.0Ghz, 2GB RAM
– $50/m
•Puppet master: VM x2 CPU 2.0Ghz, 2GB RAM
– $89/m
•Total: $473/m
Colo experiment
Colo experiment
•Dell 1U R415
•x2 8C AMD 2.8Ghz
•32GB RAM
Colo experiment
•Dell 1U R415
•x2 8C AMD 2.8Ghz
•32GB RAM
•Dual PSU, NIC
Colo experiment
•Dell 1U R415
•x2 8C AMD 2.8Ghz
•32GB RAM
•Dual PSU, NIC
•x4 1TB SATA hot swappable
Dedicated?
Dedicated?
•Hardware replacement
Dedicated?
•Hardware replacement
•Managed/support
Dedicated?
•Hardware replacement
•Managed/support
•Networking
Colo?
Colo?
•Hardware spec/value
Colo?
•Hardware spec/value
•Total cost
Colo?
•Hardware spec/value
•Total cost
•Internal skills?
Colo?
•Hardware spec/value
•Total cost
•Internal skills?
•More fun?!
Colo: Power
•£300-350/kWh/m
•4.5A = £520/m
•9A = £900/m
