scaling devops
DESCRIPTION
Scaling DevOps from the Seattle Scalability Meetup, December 2013TRANSCRIPT
![Page 1: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/1.jpg)
Scaling DevOpsJeffrey HultenWhitepages, Inc.
Wednesday, December 4, 13
![Page 2: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/2.jpg)
Who Am I?
Jeffrey Hulten - @jhulten
Software Engineer, Architecture Improvement at WhitePages, Inc.
Organizer of Seattle DevOps MeetUp
Too Ops for Dev, Too Dev for Ops
Wednesday, December 4, 13
![Page 3: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/3.jpg)
Scaling DevOps
Your Tools Aren’t Enough
Your First 100 Machines
Watching the Horizon
Wednesday, December 4, 13
![Page 4: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/4.jpg)
Your Tools Aren’t Enough
Culture
Deploy a Package
Manage the Metal
Minimize the Surface Area
Growing Pains
Wednesday, December 4, 13
![Page 5: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/5.jpg)
CultureNot Just for Bacteria Anymore
Starts with one person
Infect your friends
Try changing things
Mutation must allow for failure
Requirement for survival
Wednesday, December 4, 13
![Page 6: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/6.jpg)
Deploy a Package
TGZ
OS Package (.deb / .rpm)
Machine Image (AMI)
LXC Image (Docker)
Wednesday, December 4, 13
![Page 7: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/7.jpg)
OS Packages Made Easier
FPM
Package Config Separately
Package Repository
Wednesday, December 4, 13
![Page 8: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/8.jpg)
FPM: Effing Package Management
https://github.com/jordansissel/fpm
$ gem install fpm
Make RPM, DEB, and Solaris packages and Puppet module
Create from directories, RPM, DEB, Gem, PyPi, Tarball and more
Wednesday, December 4, 13
![Page 9: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/9.jpg)
Example: Jenkins
NAME=jenkinsVERSION=1.396
.PHONY: packagepackage: rm -f jenkins.war wget http://ftp.osuosl.org/pub/hudson/war/$(VERSION)/jenkins.war fpm -s dir -t deb -n $(NAME) -v $(VERSION) \ --prefix /opt/jenkins jenkins.war
Wednesday, December 4, 13
![Page 10: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/10.jpg)
Machine Images & Containers
Machine Image
Entire OS
All Services Enabled
Containers
Minimal OS Image
Run One Process (Replace `init`)
Wednesday, December 4, 13
![Page 11: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/11.jpg)
Manage the Metal
Application as Inventory...
Wednesday, December 4, 13
![Page 12: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/12.jpg)
Minimize the Surface Area
What is Surface Area?
Libraries?
Versions?
Components?
Data Stores?
Wednesday, December 4, 13
![Page 13: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/13.jpg)
Minimize the Surface Area
What is Surface Area?
Libraries?
Versions?
Components?
Data Stores?
Deployables...
Wednesday, December 4, 13
![Page 14: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/14.jpg)
Growing Pains
Don’t bottle knowledge
There are no specialists
Think about the smallest solution
Delay your decisions
Wednesday, December 4, 13
![Page 15: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/15.jpg)
Your First 20 Machines
DEMO TIME!
Source at: https://github.com/whitepages/scaledemo-20-machines
AWS CloudFormation, cloud-init...
Wednesday, December 4, 13
![Page 16: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/16.jpg)
Application: Random Numbers as a Service
Wednesday, December 4, 13
![Page 17: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/17.jpg)
CloudFormation
Deploy Stacks
Templates in JSON
Parameters for Reuse
Wednesday, December 4, 13
![Page 18: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/18.jpg)
Template: Parameters"Parameters": { "OperatorEmail": { "Type": "String", "Description": "Email address to notify if there are any scaling operations" }, "InstanceType": { "Description": "WebServer EC2 instance type", "Type": "String", "AllowedValues": [ "t1.micro", "m1.small", ...], "Default": "m1.small", "ConstraintDescription": "must be a valid EC2 instance type." },
... "SSHLocation": { "Default": "0.0.0.0/0", "Type": "String", "MaxLength": "18", "MinLength": "9", "AllowedPattern": "(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})/(\\d{1,2})", "Description": "The IP address range that can be used to SSH to the EC2 instances", "ConstraintDescription": "must be a valid IP CIDR range of the form x.x.x.x/x." } },
Wednesday, December 4, 13
![Page 19: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/19.jpg)
Template: Parameters
"SSHLocation": { "Default": "0.0.0.0/0", "Type": "String", "MaxLength": "18", "MinLength": "9", "AllowedPattern": "(\\d{1,3})\\.(\\d{1,3}).../(\\d{1,2})", "Description": "...", "ConstraintDescription": "..."}
Wednesday, December 4, 13
![Page 20: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/20.jpg)
Template: Mappings"Mappings": { "AWSRegionArch2AMI": { "us-east-1": { "64": "ami-955b79fc" }, "us-west-1": { "64": "ami-6ca89929" }, "us-west-2": { "64": "ami-a8395c98" } }, ...},
Wednesday, December 4, 13
![Page 21: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/21.jpg)
Template: Resources"Resources": { "NotificationTopic": { "Type": "AWS::SNS::Topic", "Properties": { "Subscription": [ { "Protocol": "email", "Endpoint": { "Ref": "OperatorEmail" } } ] } }, ...}
Wednesday, December 4, 13
![Page 22: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/22.jpg)
AutoScaling Group"RandServerGroup": { "Type": "AWS::AutoScaling::AutoScalingGroup", "Properties": { "AvailabilityZones": { "Fn::GetAZs": "" }, "NotificationConfiguration": { "NotificationTypes": [ "autoscaling:EC2_INSTANCE_LAUNCH", "autoscaling:EC2_INSTANCE_LAUNCH_ERROR", "autoscaling:EC2_INSTANCE_TERMINATE", "autoscaling:EC2_INSTANCE_TERMINATE_ERROR" ], "TopicARN": { "Ref": "NotificationTopic" } }, "MinSize": "1", "MaxSize": "100", "LaunchConfigurationName": { "Ref": “..." }}}
Wednesday, December 4, 13
![Page 23: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/23.jpg)
Launch Config"RandLaunchConfig": { "Type": "AWS::AutoScaling::LaunchConfiguration", "Properties": { "SecurityGroups": [{ "Ref": InstanceSecurityGroup" }], "InstanceType": { "Ref": "InstanceType" }, "KeyName": { "Ref": "KeyName" }, "UserData": { "Fn::Base64": { "Fn::Join" : ["",[ "#include\n", "https://.../cloud-config.txt\n" ]]}}, "ImageId": {"Fn::FindInMap": [...] } }},
Wednesday, December 4, 13
![Page 24: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/24.jpg)
Cloud Init
#cloud-configpackages: - ruby1.9.3 - git - build-essential
runcmd: - /usr/bin/gem install bundler - /usr/bin/git clone "https://github.com/whitepages/scaledemo-20-machines.git" /opt/rand - cd /opt/rand && /usr/local/bin/bundle install - cd /opt/rand &&/usr/local/bin/bundle exec randserver &
Wednesday, December 4, 13
![Page 25: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/25.jpg)
Watching the Horizon
Docker / LXC
SmartStack
Akka Cluster / Riak Core
Wednesday, December 4, 13
![Page 26: Scaling DevOps](https://reader034.vdocuments.us/reader034/viewer/2022051210/54b6eff44a7959fd508b468a/html5/thumbnails/26.jpg)
Twitter: @jhulten
Github: jhulten
whitepages
whitepages.com/careers
Wednesday, December 4, 13