building automated control systems for your aws infrastructure
TRANSCRIPT
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Randall Hunt, Developer Evangelist at AWS
October 24, 2016
Building Automated Control
Systems for Your AWS
Infrastructure
AWS Dev Day - Serverless
Randall Hunt - @jrhunt
Developer Evangelist at AWS
Formerly of SpaceX and NASA
Agenda
• Brief Overview of Lambda
• Why Automate?
• Why Lambda for Automation and Control Systems?
• Event-Driven Policy Enforcement
• Lambda as an Infrastructure Control Plane
• Best Practices
Operations and management Scaling
Provisioning and utilization Responsible for Availability
and fault tolerance
Owning Servers Means Dealing With ...
Serverless Compute: AWS Lambda
COMPUTE SERVICE
EVENT-DRIVEN
Run code
without
managing
servers
Code only runs
when it needs to
run
Code is all you need(Native: Java/Python/NodeJS)
Event-driven scaling
Never pay for idle servers Availability and fault tolerance built in
Going Serverless with Lambda
Things To Remember: Lambda Function
Memory = “Power level”
• Higher levels offer more memory and more CPU power
Functions don’t have a notion of state
• Use DynamoDB, S3, or Elasticache
• Wrap your config in a function and call it from your published code
Use the right access control for downstream services
• IAM roles and permissions for AWS services
• VPC for private endpoints
Benefits of Automation Done Correctly
Increased Agility Improved Quality
Improved Innovation
Increased Consistency
Reduced Expenses and
Labor Costs
Reduced Human Errors
Dangers of Incorrect Automation
Code Maintenance Issues Bad Permissions Control Auditing Nightmares
Everyone Blames You!Accountability
Amazon
API GatewayAWS
IoT
Amazon
Alexa
Automatic Infrastructure Cleanup Allow other endpoints and AWS Services to
trigger infrastructure change
Create copies/snapshots of data in response
to events
Integrate / Think outside the box
Amazon EC2 Amazon ECS Amazon ECR
Amazon RDS Amazon
DynamoDB
Instant Core Infrastructure Changes
Amazon
VPC
Amazon
Route 53AWS
CloudFormation
Lambda as a building block for Automaton
Because Lambda is event-driven, it offers a very
powerful framework for Automated Infrastructure
Control Planes and Policy Engines.
AWS Lambda
Near real time reactionEvent
Benefits for Automation with AWS Lambda
Single knob configuration Easy to integrate Logging and Auditability
Rapid delivery and versioning Get started fast
Event-Driven Automation as a building block
Being proactive rather than reactive to changes in your
infrastructure is key. However, event-driven automation can
be part of the solution. We can remove human errors when
we need to be reactive through use of automation.
Amazon
S3
Amazon
DynamoDB
Amazon
Kinesis
AWS
CloudFormation
AWS
CloudTrail
Amazon
CloudWatch
Logs
Amazon
SNSAmazon
SES
Amazon
API Gateway
Amazon
Cognito
AWS
IoT
Cron events
Trigger an event based on Data Store changes Interact with other endpoints and AWS services
to trigger infrastructure change
Respond to custom events as they happen
near real-time
Respond to messaging or scheduled
events in your infrastructure
Event Sources
Policy Enforcement
What do you mean Policy Enforcement?
You have the freedom to define what
policy enforcement means to you!
VPC Policy Enforcement
Lambda Policy Engine
• Allocate Address
• Associate Address
instance instance instance
instance instance instance
instance instance instance
Validate Security Group
instance instance instance
Identity and Access Management Enforcement
Lambda Policy Engine
Validate Role Path
AWS IAM
CreateRole
Verify No Role Elevation
Verify No Resource Level Elevation
Allow Role
Delete Roles
Tagging Enforcement
Lambda Policy Engine
instance instance instance
instance instance instance
RunInstancesValidate Team Tag
Validate Billing Group Tag
Validate Environment Stage Tag
StopInstances
SNS Notification
instance instance instance
instance instance instance
Infrastructure Control Plane
Lambda gives you the freedom to
design the automation you need for
your infrastructure.
Infrastructure Automation
Amazon CloudWatch Events
Auto Scaling Event AWS API Call EC2 Instance State Change
Automatic Security Group Placement
Lambda Policy Engine
instance instance instance
instance instance instance
RunInstancesCheck VPC
Validate Security Group
Remove SG
Add Correct SG
instance instance instance
Do Nothing
Create/Update Route53 Records from Tag
Lambda Policy Engine
instance instance instance
instance instance instance
RunInstances
Do Nothing
instance instance instance
Check VPC
Update DNSGet CNAME Tag
Add DNS Record
instance instance instance
Best Practices
Log Everything
Bite Sized Code
Report Failures
Version Lambda Functions
Use Source Control
Iterate
Recap
• Event-Driven Response to Policy Management and Infrastructure Events
• AWS Config Rules backed by Lambda to visualize compliant versus non-compliant
infrastructure.
• Think outside the box. Number of available CloudWatch Events API Call triggers is
large.
• Don’t forget the ability to schedule AWS Config rule validation, as well as Scheduled
Cloud Watch Event Triggers to Lambda.
• CloudWatch Events can point to multiple Targets (Not just Lambda functions)
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
All attendees will receive a special giveaway gift!
Please join us for the
AWS DevDay Networking Reception
5:00 - 6:30 PM
JW Grand Foyer