alfredo reino - monitoring aws and azure

Join the conversation #devseccon

By Alfredo Reino

Monitoring AWS and Azure

Agenda

• Who am I• Why monitor anything• What to monitor• How to monitor Azure• How to monitor AWS• Integrating with SIEMs and MSSPs

Who am I

[insert something funny and slightly self-deprecating here]

Alfredo Reino / @areinoalfredo@aebura.co.ukSecurity Architect

Why monitor anything

• Threat detection (reactive)• Threat hunting (proactive)• Incident response and forensics• Data mining, anomaly detection• Reporting and dashboards• Regulatory and policy requirements• Troubleshooting and root cause analysis

What to monitor

• (Easy answer) Everything and anything!• But• is it possible to log?• is it cost effective to do it?• do you have the storage?• can you make sense of it?• do you have the tools/skillset/capability/time to use it?

• Priorities!

What to monitor

• Using the kill-chain

Generic kill chain

• Recon• Passive recon• Active recon

• Delivery• Internet-facing services• Inbound email• Web browsing• Removable media• Insider / Third-party access

• Exploitation• Internet-facing servers• User endpoint

• Installation• Lateral movement• Elevation of privilege• Persistence• Command and Control

• Actions on target• Access to internal

system and data• Exfiltration• Attack third-party

IaaS/PaaS kill chain

• Recon• Passive recon• Active recon

• Delivery• Internet-facing services• Inbound email• Web browsing• Removable media• Insider / Third-party access

• Exploitation• Internet-facing servers• User endpoint

• Installation• Lateral movement• Elevation of privilege• Persistence• Command and Control

• Actions on target• Access to internal

system and data• Exfiltration• Attack third-party

What to monitor – Shared responsibility

Azure Shared responsibility model https://aka.ms/sharedresponsibility

AWS Shared responsibility modelhttps://aws.amazon.com/compliance/shared-responsibility-model/

What to monitor (IaaS/PaaS)

• Operating System logs from IaaS virtual machines• Application/service logs (webserver, database, etc.)• Performance metrics (CPU, memory, data in/out,

filesystem, …)• Network traffic (at interface or across boundaries)• Other cloud security solutions (WAF, AV, FIM, etc.)• IaaS/PaaS service fabric logs• Audit/management logs (cloud resource access and

management)• Blob Storage/S3

AWS Options

• CloudTrail• Records AWS API calls (usage of Management Console, SDKs, command line tools, and higher-level AWS services

such as AWS CloudFormation).• The recorded information includes the identity of the API caller, the time of the API call, the source IP address of

the API caller, the request parameters, and the response elements returned by the AWS service.• Logs to S3 bucket (possibility of aggregating multiple region or multiple account CloudTrail logs in one S3 bucket)

• CloudWatch• Collects and tracks metrics, collects and monitors log files, sets alarms.• Monitor EC2 instances, WAF, DynamoDB tables, and Amazon RDS DB instances, as well as custom metrics

generated by applications and services, and any log files applications generate.• S3 Server Access Logging

• Track requests for access to S3 bucket. • Each access log record provides details about a single access request, such as the requester, bucket name,

request time, request action, response status, and error code, if any.• VPC Flow Logs

• Log traffic flow in Virtual Private Cloud (VPC), subnets or Elastic Network Interfaces (ENI).• Captures accepted and rejected traffic.• Logs to CloudWatch.

AWS CloudTrail

AWS VPC Flow Logs

Log Management solutions for AWS

• SumoLogic• logs from CloudTrail, VPC Flow, ELB, S3, etc

• Splunk add-on for AWS• logs from AWS Config, Config Rules,

CloudWatch, CloudTrail, Billing, S3, VPC Flow Log, Amazon Inspector, Metadata inputs, etc

• ELK (ElasticSearch+LogStash+Kibana)• logs from applications, OS, ELB, CloudTrail, VPC

Flow, CloudFront, S3, etc

Log Management solutions for Azure

• SumoLogic• logs from Azure Audit Logs, AD access, etc.

• Splunk add-on for Microsoft Cloud• logs from Storage Tables, Storage Blobs, Azure

Service Management APIs and Office 365 Management API.

• ELK (ElasticSearch+LogStash+Kibana)• logs from applications, OS, Storage Blobs, Service

Management APIs, etc.• Azure Log Analytics

• Part of OMS (Operations Management Suite).• Collect logs from agents (Win/Linux), storage,

performance, IIS logs, syslog, etc.

Connecting AWS logs to a SIEM

• Connectors by SIEM vendors• HP ArcSight SmartConnector

• Need to allow inbound SSL to ESM

Connecting AWS logs to a SIEM

• Connectors by SIEM vendors• IBM Qradar

• Native support for AWS CloudTrail using S2 REST API

• Need to import the SSL cert first

Connecting AWS logs to a SIEM

• Connectors by SIEM vendors• Splunk

• Requires “Splunk for AWS” app and “Splunk Add-on for Amazon Web Services”.

• Requires appropriate permissions created on IAM.

• Collects events from Simple Queue Service (SQS) that subscribes to a Simple Notification Service (SNS) events from AWS Config.

Connecting AWS logs to a SIEM

• Connectors by SIEM vendors• ELK Stack (logz.io)

Azure SIEM Integrator

• Integrate with on-premises SIEM (or MSSP)• Logs supported• VM logs• Azure Audit Logs• Azure Security Center alerts

Azure SIEM Integrator

• How to deploy• Install Azlog Integrator on Windows server (on-premises)

• https://www.microsoft.com/en-us/download/details.aspx?id=53324• Needs access to Azure Storage

• Install SIEM log collection agent on same server• Splunk Universal Forwarder• HP ArcSight Windows Event Collector• IBM QRadar WinCollect• …

• Configure SIEM agents for collection • https://blogs.msdn.microsoft.com/azuresecurity/2016/08/23/azure-log-siem-configuration-steps/

• Scalability• On a 8 proc machine – 1 instance of Azlog can process about 277 EPS• On a 4 proc machine – 1 instance of Azlog can process about 17 EPS• Multiple instances of the SIEM Integrators can be run if event volume is high

Azure SIEM Integrator example

Azure SIEM Integrator example

Azure SIEM Integrator example

Threat intel feeds

• Good feeds of IOCs can be invaluable• Integrate threat intel feeds in SIEM/Log Management

solutions• tagging of events• quick searches for malicious activity

• For increased value, maintain your OWN threat intel feed and repository

Endpoint activity monitoring

• Endpoint process activity monitoring tool• such as Carbon Black

• Deploy to IaaS instances• Agent-based blackbox-type recording for

• process activity (creation, termination, child processes)

• filesystem and registry activity• inbound and outbound network connections

• Integration with threat intel feeds• Can integrate (using API) with log

retention solutions• “if log event X then find process tree at the

time for endpoint Y”

Join the conversation #devseccon

Thanks!