2016-02-09 - breakfast seminar - redpill linpro - chef at aptoma - håkon drange
TRANSCRIPT
![Page 1: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/1.jpg)
Håkon DrangeHead of Infrastructure Operations
Chef @ AptomaRedpill Linpro breakfast seminar
February 9th 2016
![Page 2: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/2.jpg)
● Håkon Drange
Introduction
![Page 3: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/3.jpg)
Introduction
![Page 4: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/4.jpg)
● SaaS provider of tools for online news media.● 15 employees (80% tech staff)
About Aptoma
![Page 5: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/5.jpg)
Vision: To help the new media with tools that shapes effective, flexible and profitable news organisations.
About Aptoma
![Page 6: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/6.jpg)
About Aptoma - some customers
![Page 7: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/7.jpg)
The main services we provide
About Aptoma
![Page 8: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/8.jpg)
Good looking editions on mobile devices for “+” subscriptions
DrMobile
![Page 9: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/9.jpg)
Easy and flexible front page editing
DrFront
![Page 10: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/10.jpg)
Efficient article production to any output channel
DrPublish
![Page 11: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/11.jpg)
Video encoding, playback API’s and CDN distribution
DrVideo
![Page 12: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/12.jpg)
Flexible search API for all your media content
DrLib
![Page 13: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/13.jpg)
● All tools delivered as SaaS● The customer builds their business logic and
presentation on top of our APIs
About Aptoma
![Page 14: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/14.jpg)
Internal PaaS project started in 2011
Aptoma Media Platform (AMP)
![Page 15: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/15.jpg)
Vendor agnostic● Cloud providers● VMware● KVM● Physical hardware
Aptoma Media Platform (AMP)
![Page 16: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/16.jpg)
Aptoma Media Platform (AMP)
SaaSDr*
PaaS (AMP)Chef
AWS Cloudformation
IaaS
AWS EC2IBM Softlayer - Zetta - Dell Poweredge
![Page 17: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/17.jpg)
Consistent and predictable platform for the developers● Same Ubuntu● Munin, Nagios, New Relic, log shipping, SSH access,
NTP, firewall, swap, etc.● Same Apache+PHP● Same MySQL● Same MongoDB● etc... Achieved with Chef.
Aptoma Media Platform (AMP)
![Page 18: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/18.jpg)
Primary vendor is AWSPrimary region eu-central-1 (Frankfurt)Secondary region eu-west-1 (Dublin)
IaaS/PaaS providers
![Page 19: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/19.jpg)
Chef
![Page 20: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/20.jpg)
Chef is a configuration management and automation platform from Opscode.
Chef
![Page 21: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/21.jpg)
Can be used in multiple ways:● Chef Server mode● Chef Solo mode
○ With Vagrant
Chef
![Page 22: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/22.jpg)
Building blocks:● Cookbooks contains recipes
○ Write your own cookbook or leverage community cookboks from the Supermarket
○ Or a combination, with wrapper cookbooks
Chef
![Page 23: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/23.jpg)
Building blocks:● Cookbooks contains recipes● A role consists of one or more re-usable recipes● chef-client on the node (server)
Chef
![Page 24: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/24.jpg)
chef-client run:
Chef
![Page 25: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/25.jpg)
Add common bootstrap role first to configure OS platform
Chef
![Page 26: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/26.jpg)
Then add server role● apache-server● nginx-server● varnish-server● mysql-server● mongodb-server
Chef
![Page 27: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/27.jpg)
Then add service role for service specific config.
Chef
![Page 28: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/28.jpg)
service-drfront-web● Adds Apache virtualhosts● memcached on port 11211● Ruby gems● Specific Node.js version
Chef
![Page 29: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/29.jpg)
Base role common-bootstrap on all manually provisioned servers
Chef
![Page 30: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/30.jpg)
Base role common-bootstrap on all manually provisioned servers
knife bootstrap --bootstrap-version 11.18.12 <hostname> --hint ec2 --environment production-v2 -x tech --sudo -r “role[common-bootstrap],role[apache-server],role[service-drfront-web]”
Chef
![Page 31: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/31.jpg)
Base role common-bootstrap-cloud on all automatically provisioned servers with AWS Cloudformation
Chef
![Page 32: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/32.jpg)
“Provision and manage AWS infrastructure as code”
https://aws.amazon.com/cloudformation/
AWS Cloudformation
![Page 33: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/33.jpg)
{ "Description" : "Create an EC2 instance running the Amazon Linux 32 bit AMI.", "Parameters" : { "KeyPair" : { "Description" : "The EC2 Key Pair to allow SSH access to the instance", "Type" : "String" } }, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyPair" }, "ImageId" : "ami-3b355a52" } } }, "Outputs" : { "InstanceId" : { "Description" : "The InstanceId of the newly created EC2 instance", "Value" : { "Ref" : "Ec2Instance" } } }, "AWSTemplateFormatVersion" : "2010-09-09"}
AWS Cloudformation
![Page 34: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/34.jpg)
AWS Cloudformation
![Page 35: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/35.jpg)
1. Develop a template2. Define stack specific parameter values
AWS Cloudformation
![Page 36: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/36.jpg)
AWS Cloudformation
![Page 37: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/37.jpg)
AWS Cloudformation
![Page 38: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/38.jpg)
AWS Cloudformation
![Page 39: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/39.jpg)
AWS Cloudformation
![Page 40: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/40.jpg)
AWS Cloudformation
![Page 41: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/41.jpg)
1. Develop a template2. Parameterize input values3. Define resources
a. Launch Configuration for an Auto Scaling Groupb. Instance type, storage configurationc. OS via AMI IDd. Elastic Load Balancer to attach to (optional)
4. Scaling thresholds5. .. etc
AWS Cloudformation
![Page 42: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/42.jpg)
Now you can create a new stack (a collection of resources)
AWS Cloudformation
![Page 43: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/43.jpg)
Everything gets provisioned automatically
AWS Cloudformation
![Page 44: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/44.jpg)
Consistent and predictable configuration
AWS Cloudformation
![Page 45: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/45.jpg)
Provision stack via AWS Console or API
AWS Cloudformation
![Page 46: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/46.jpg)
So how do we kick off the Chef node bootstrap process?
AWS Cloudformation + Chef
![Page 47: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/47.jpg)
In Cloudformation Launch Configuration user-data:● Download and install publicly available dependencies
"apt-get update; apt-get upgrade -y\n",
"apt-get install -y libffi-dev libssl-dev python-setuptools munin-node\n",
"easy_install pip\n",
"pip install pyopenssl ndg-httpsclient pyasn1 requests s3cmd boto3 awscli\n",
"easy_install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz\n",
AWS Cloudformation + Chef
![Page 48: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/48.jpg)
In Cloudformation Launch Configuration user-data:● Download and install private stuff from S3
○ Aptoma Chef bootstrap validator key○ amp-aws-auto-bootstrap-v1.1.0.py bootstrap script○ chef-client installer
● If all OK, Cloudformation signal OK to the Auto Scaling Group
AWS Cloudformation + Chef
![Page 49: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/49.jpg)
The ASG should now have instances attached to a load balancerELB instance health check /status fails because no application is deployed
AWS Cloudformation + Chef
![Page 50: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/50.jpg)
What about application deployments?
AWS Cloudformation + Chef
![Page 51: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/51.jpg)
We use AWS CodeDeploy.
AWS CodeDeploy
![Page 52: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/52.jpg)
CodeDeploy is a deployment tool from AWS.
AWS CodeDeploy
![Page 53: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/53.jpg)
Nice integration with Auto Scaling
AWS CodeDeploy
![Page 54: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/54.jpg)
A framework, not a complete tool
AWS CodeDeploy
![Page 55: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/55.jpg)
AWS CodeDeploy
![Page 56: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/56.jpg)
1. Provision servers manually or with Cloudformation2. Use the same Chef cookbooks for any IaaS3. Leverage Auto Scaling with Cloudformation and
CodeDeploy
Summary
![Page 57: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/57.jpg)
We achieve:● Automated, repeatable infrastructure with re-usable
code● Predictable and scalable setup● Easy and efficient for developers to set up new services
Summary
![Page 58: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/58.jpg)
That’s all, folks!
![Page 59: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/59.jpg)
Email: [email protected]: @haakondSlides: http://apto.ma/slides20160209hdWWW: http://aptoma.com/
Thanks
![Page 60: 2016-02-09 - Breakfast Seminar - Redpill Linpro - Chef at Aptoma - Håkon Drange](https://reader031.vdocuments.us/reader031/viewer/2022030313/58a9f9871a28abec248b4d0b/html5/thumbnails/60.jpg)
● https://docs.chef.io/_images/chef_run.png● http://docs.aws.amazon.com/gettingstarted/latest/wah-linux/images/architecture_linux.png● http://media.amazonwebservices.com/architecturecenter/AWS_ac_ra_web_01.pdf● https://aws.amazon.com/cloudformation/details/● http://docs.aws.amazon.com/codedeploy/latest/userguide/deployment-steps.html ● https://www.chef.io/● http://www.aptoma.com/ ● http://southparkstudios.mtvnimages.com/shared/characters/adults/chef.png● http://s3.amazonaws.com/opscode-corpsite/assets/121/pic-chef-logo.png
References