AWS Pop-up Loft London

Dave Walker

Specialist Solutions Architect, Security and Compliance

19/04/16

Incident Response and Forensics on AWS

Agenda

• "Expect to be Hacked"

• "Be Prepared" (and "Playing the SIRS Game")

• "That looks Weird..."

• Initial Response Actions

• "Decisions, Decisions..."

• Preserving Evidence, by Service

• Analysis

• Conclusions

“Be Prepared”

• Be Contactable by AWS

• Build an Incident Response Handbook

• Train your People

• Know who to Contact, about What and When

• CloudFormation Everything

• Maintain “Intellectual Property Holding”

• Templates, golden AMIs, Database snapshots

“If it Moves, Log It”

• CloudTrail

• Config

• CloudWatch Logs

• VPC Flow Logs

• ELB logs

• API Endpoint Logs

• Redshift Logs

• …

Identification

• Plenty of logging sources, what to do with them:

Level 1: Log

• Primarily for ‘look back’ analysis.

Level 2: Alarm

• Basic monitoring based in triggers

Level 3: Aggregate, analyze and learn

• Monitor, ingest, analyze and derive

• AWS as a source of event data

Identification – Level 1: Log

Resource

stack

Actors

AWS CloudTrail

Amazon S3

Captures all API interaction

AWS

CloudWatch

Monitors AWS & application

AWS ConfigActions on Captures resourcechanges

Identification – Level 2: Alarm

Resource

stack

Actors

AWS CloudTrail

Amazon S3

monitors

Captures all API interaction

AWS

CloudWatch

Monitors AWS & application

AWS ConfigActions on Captures resourcechanges

alarm

initiates

AWS Config rules

Admin

alarm

Identification – Level 3: Aggregate

Resource

stack

Actors

AWS CloudTrail

Amazon S3

monitors

Captures all API interaction

AWS

CloudWatch

Monitors AWS & application

AWS ConfigActions on

Captures resourcechanges

alarm

initiates

AWS Config rules

Admin

alarm

Aggregate/analyze

Admin

CloudWatch Logs

• What works well

• Simple to set up : http://docs.aws.amazon.com/awscloudtrail/latest/userguide/use-cloudformation-template-to-create-cloudwatch-alarms.html

• Limitations

• Only match terms, phrases, or values in log events.

• No complex matches or regular expressions

CloudWatch Logs

• "ConsoleSignInAnomalyMetricFilter": {

• "Type": "AWS::Logs::MetricFilter",

• "Properties": {

• "LogGroupName": { "Ref" : "LogGroupName" },

• "FilterPattern": "{ ($.eventName = ConsoleLogin) && ($.sourceIPAddress != 55.55.*) }",

• "MetricTransformations": [

• {

• "MetricNamespace": "CloudTrailMetrics",

• "MetricName": "ConsoleSignInAnomalyCount",

• "MetricValue": "1"

• }

• ]

• }

• },

Config Rules

• What works well

• Pre-packaged rules

• Easy to create new rules

• “Config is essentially a CMDB”

• Limitations

• Currently subset of EC2, EBS, VPC, CloudTrail, IAM

“Please, SIRS”

• An Exercise with AWS

• Injection of Events into Logs

• You choose What and When

• Ask your Account Manager

Initial Response Actions

• Get the Right People Involved

• AWS

• Auditors, Regulators

• Forensic Investigators (QIRAs, CESG-recommended firms)?

• Police?

“Decisions, Decisions…”

• “Manage and Mitigate” or “Pursue and Prosecute”?

• First closes open doors, but destroys evidence

• Second (in traditional environments) is a lot more hassle

• Why not do both?

“Decisions, Decisions…”

• Replicate

• Repair

• Redirect

• Ringfence

Preserving Evidence, by Service

Infrastructure services

Container services

Abstracted services

Configuration plus operation

Configuration

Source: “AWS Security Best Practices,” Nov 2013, p7

Best practices: EC2 & VPC

• Create forensic subnets in all VPCs

• All routes black-holed; all traffic stays within

• Create “isolate” security groups to seal off potentially compromised hosts

• Configure all AMIs to automatically mount ENIs when signalled by events

Best practices: EC2 & VPC…

• Configure investigation hosts

• All forensics tools installed and ready to go

• Protect with deny-all SGs on forensics subnet

• Set up write blockers, eg https://github.com/msuhanov/Linux-write-blocker

• Additional VM help available from AWS on request

Analysis

• EIR

- Mandiant

- EnCase

- FTK

- GRR

• Network

- Wireshark

- Moloch

• Memory Capture

- Fastdump

- FTK Imager

- LiME (see Re:Invent, “Incident Response in the Cloud”)

Subnet

10.1.1.0/24

Analysis

Virtual router

Web server10.1.1.60

Internet gateway

Workstation10.1.1.101

Change security

group to “Isolate”

Attach Elastic Network

Interface with security group

“forensics-target”

Elastic Network Interface

10.1.201.60

Security group: “Forensics-target”

(forensics target security group)

Attach Elastic Network

Interface with security group

“forensics-target”

Completely isolated subnet

10.1.201.0/24

Elastic Network Interface

10.1.201.101

Security group: “Forensics-source”

(forensics source security group)

Attach Elastic Network Interface with security group “forensics-source”

Attach Elastic Network Interface with security group “forensics-source”

Conclusion

• Many best practices are free; the rest are cheap

• Configure IAM thoughtfully

• Set up alarms; “snapshot” frequently

• Use rich data emitted by “cloud container” for greater visibility and control

• Use programmable infrastructure to automate detection (and potentially mitigation)

• Always get the right people involved, at the outset

• …and that includes AWS