auto scaling on aws
DESCRIPTION
Matt Swain's talk on auto scaling on AWS for the Austin Web Architecture meetup. November 5, 2013.TRANSCRIPT
Auto Scaling on AWS
Matthew SwainSoftware Engineer @ MassRelevance
What is AWS Auto Scaling?
Auto Scaling is Amazon’s hosted service for automatically launching and terminating EC2
instances.
Why Use Auto Scaling
• Save money by only using instances when you need them.
• Scale to accommodate expected and unexpected loads.
• Replace unhealthy servers with healthy ones.
What Makes Up Auto Scaling
• Launch Configurations
• Auto Scaling Groups
• Scaling Policies
• Cloud Watch Alarms
Launch Configurations
Templates that describe the parameters passed at launch-time to your EC2 instances.
Examples: AMI, instance type (c1.xlarge, m1.small, …), security groups, spot price
Auto Scaling Groups
A set of EC2 instances that run a launch configuration.
Scaling Policies
A template describing actions to run against an Auto Scaling Group.
E.g. Start two instances in the ASG named “webservers”
Cloud Watch Alarms
Triggers that can run Scaling Policies based on Cloud Watch metrics (AWS’s built in monitoring suite).
E.g. Run Scaling Policy “Launch Webservers” when CPU exceeds a certain threshold for 5 minutes.
A Simple Web Application in Five Steps
• Create an AMI
• Setup an Elastic Load Balancer (ELB)
• Create a Launch Configuration
• Create Scaling Policies
• Create Cloud Watch Alarms
Prerequisites
• AWS Account
• A running web application on an EBS backed instance.
• AWS Command Line tool: http://aws.amazon.com/cli/
ZSH Auto-complete!
Build an AMI
aws ec2 create-image \ --instance-id i-12345678 \ --name awesome-image-v1.0
output: ami-12345678
Create a Load Balancer
aws elb create-load-balancer \ --load-balancer-name 'my-lb' \ --listeners Protocol=http,\ LoadBalancerPort=80,\InstanceProtocol=http,\InstancePort=80 \ --availability-zones us-east-1d
Create a Launch Configuration
aws autoscaling create-launch-configuration \ --launch-configuration-name awesome-lc-v1.0 \ --image-id ami-12345678 \ --key-name my-keypair \ --securty-groups default \ --instance-type c1.xlarge
Create an Auto Scaling Group
aws autoscaling create-autoscaling-group \ --auto-scaling-group-name awesome-asg \ --launch-configuration-name awesome-lc-v1.0 \ --min-size 1 \ --max-size 10 \ --desired-capacity 1 \ --default-cooldown 120 \ --availability-zones us-east-1d \ --load-balancer-names my-lb \ --health-check-type EC2
Create Scaling Policies
aws autoscaling put-scaling-policy \ --auto-scaling-group-name awesome-asg \ --policy-name awesome-asg-up \ --scaling-adjustment 1 \ --adjustment-type ChangeInCapacity \ --cooldown 300
>> arn:aws:autoscaling<...>:policyName/awesome-asg-up
Create Scaling Policies
aws autoscaling put-scaling-policy \ --auto-scaling-group-name awesome-asg \ --policy-name awesome-asg-down \ --scaling-adjustment -1 \ --adjustment-type ChangeInCapacity \ --cooldown 300
>> arn:aws:autoscaling<...>:policyName/awesome-asg-down
Create CloudWatch Alarms
aws cloudwatch put-metric-alarm \ --alarm-name awesome-cpu-high \ --metric-name CPUUtilization \ --actions-enabled \ --alarm-actions arn:aws:<...>:policyName/awesome-asg-up \ --namespace "AWS/EC2" \ --statistic Average \ --dimensions Name=AutoScalingGroupName,Value=awesome-asg \ --period 300 \ --evaluation-periods 1 \ --comparison-operator GreaterThanOrEqualToThreshold \ --threshold 60.0
Create CloudWatch Alarms
aws cloudwatch put-metric-alarm \ --alarm-name awesome-cpu-low \ --metric-name CPUUtilization \ --actions-enabled \ --alarm-actions arn:aws<...>policyName/awesome-asg-down \ --namespace "AWS/EC2" \ --statistic Average \ --dimensions Name=AutoScalingGroupName,Value=awesome-asg \ --period 300 \ --evaluation-periods 1 \ --comparison-operator LessThanOrEqualToThreshold \ --threshold 20.0
The AWS CloudWatch Console
The AWS CloudWatch Console
Advanced Usage
• IAM Roles - Allow instances to access protected S3 resources
• UserData Scripts - Small bash startup scripts passed to EC2 instances at boot time.
• Chef - Client-Server Configuration Management
UserData Scripts
#!/bin/bash
role=fsbootstrap=as-bootstrap-s3.shs3cmd_pkg=s3cmd-.tar.gzPATH=$PATH:/usr/local/bin
cd /tmpwget http://some-public-bucket/${s3cmd_pkg}tar -xzf ${s3cmd_pkg}cd `basename ${s3cmd_pkg} '.tar.gz'`python setup.py install
cd /s3cmd --config /dev/null get s3://private-bucket/${bootstrap}chmod 755 ${bootstrap}./${bootstrap} ${role} 2>&1 > ${bootstrap}.out
Questions?Twitter: @mswain