(web307) scalable site management using aws opsworks | aws re:invent 2014
Post on 02-Jul-2015
734 Views
Preview:
DESCRIPTION
TRANSCRIPT
November 13, 2014 | Las Vegas, NV
Jonathan Quail / Cliff McCollum, FillZ.com
Story background…
https://www.flickr.com/photos/stevecorey/12844122413
Debian CentOS Amazon
Linux
https://www.flickr.com/photos/bdesham/2432400623
https://www.flickr.com/photos/beigephotos/5334612
https://www.flickr.com/photos/colorhome/4033895230
Monitoring
Provisioning
Logging
Deployment Modeling
Configuration management Auto Scaling Access management
Resource management
github.com/aws/opsworks-cookbooks
supermarket.getchef.com
Hot
tip!
Hot
tip!
• Application servers
• Monitoring servers
• Graphite/dashboards
• Splunk cluster
– fillz-app::setup, fillz-app::configure, fillz-app::deploy
– fillz-base::setup, fillz-base::configure
Setup Configure Deploy Undeploy Shutdown
$ easy_deploy.py deploy –-application=myapp instances –-hosts=host1,host2 \
-–stack-name=my-test-stack --comment=“Deploying myapp to host1/host2”
$ easy_deploy.py deploy –-application=myapp all –-exclude-hosts=host1,host2 \
-–layer-name=backend-servers -–stack-name=my-test-stack \
--comment=“Deploying myapp to rest of the stack”
$ easy_deploy.py deploy –-application=myapp rolling –-layer-name=apiservers \
-–stack-name=my-api-stack --comment=“Deploying myapp behind ELB servers”
Code is available from the Github repo
By permission of Leigh Rubin and Creators Syndicate, Inc.
all_hosts = {}
all_hostgroups = {}
# Gets all (online only) instances in OpsWorks stack
all_instances = search(:node, 'role:*')
all_instances.each do |instance|
node_hostgroups = {}
# add a hostgroup for each layer the instance is in
instance['opsworks']['layers'].each do |layer_id, layer_info|
node_hostgroups[layer_id] = layer_info['name']
end
# add a hostgroup for each availability zone
node_hostgroups[instance['availability_zone']] =
instance['availability_zone']
all_hosts[instance['hostname']] = {
:hostgroups => node_hostgroups,
:private_ip => instance['private_ip']
}
all_hostgroups = all_hostgroups.merge(node_hostgroups)
end
template "/etc/nagios/conf.d/hostgroups.cfg" do
source 'hostgroups.cfg.erb'
owner 'nagios'
group 'nagios'
mode 00640
variables(
:hostgroups => all_hostgroups
)
notifies :reload, 'service[nagios]'
backup 0
end
template "/etc/nagios/conf.d/hosts.cfg" do
source 'hosts.cfg.erb'
owner 'nagios'
group 'nagios'
mode 00640
variables(
:hosts => all_hosts
)
notifies :reload, 'service[nagios]'
backup 0
end
Hot
tip!
<infomercial>
</infomercial>
Wait…Remember that deployment tool? If you call in
the next 20 minutes…it does patching too.
No “4 easy payments of $29.95” this part is open source too
$ easy_deploy.py update --amazon-linux-release=2014.09 --allow-reboots \
instances –-hosts=host1,host2 -–stack-name=my-test-stack \
--comment=“Upgrading host1/host2 to 2014.09 release”
$ easy_deploy.py update all –-exclude-hosts=host1 \
-–layer-name=backend-servers -–stack-name=my-test-stack \
--comment=“Patching layer for ++openssl vulnerability”
$ easy_deploy.py update --allow-reboots rolling \
-–layer-name=apiservers –-stack-name=my-api-stack \
--comment=“Patching api behind ELB servers and reboot for kernel”
“<stack> <name>”
“ bastion-dev fillz-graphite”
“ bastion-dev localhost:8000:fillz-graphite:80”
https://www.flickr.com/photos/wfryer/716965391
/var/log/aws/opsworks/opsworks-agent.log
/var/log/aws/opsworks/opsworks-agent.keep_alive.log
/var/log/aws/opsworks/opsworks-agent.process_command.log
http://blogs.aws.amazon.com/application-
management/post/TxTX72HFKVS9W9/Using-Amazon-CloudWatch-
Logs-with-AWS-OpsWorks
BEFORE AFTER
Time to deploy a new
server2 days + 2 hours 25 minutes
Time to deploy a new
software release2 hours 2 minutes
After-hours alerts 6 / week < 1 / week
BEFORE AFTER
Scheduled downtime per
month5 hours 0 minutes
Software deployment rate 2 / month 2 / week
http://github.com/aws/opsworks-cookbooks
http://supermarket.getchef.com
http://blogs.aws.amazon.com/application-
management/post/TxZX2UI4YSBW3T/Customizing-AWS-OpsWorks-with-Attributes
http://blogs.aws.amazon.com/application-
management/post/TxTX72HFKVS9W9/Using-Amazon-CloudWatch-Logs-with-AWS-
OpsWorks
https://github.com/awslabs/reinvent2014-scalable-site-management
top related