Network Security Monitoring with Flow Data

IT Monitoring in Enterprises

NPMD (Network Performance

Monitoring & Diagnostics)

SNMP basics

Flow data for advanced analysis

and troubleshooting

Packet capture for specialties

What about security?

Different technology

Different tools

Different vendors

NPMD and Security

Volumetric

DDoS detection

Anomaly detection

Incident reporting

Neil MacDonald, VP

Distinguished Analyst

Gartner Security & Risk

Management Summit,

London 2015

What is Flow Data?

Modern method for network monitoring – flow measurement

Cisco standard NetFlow v5/v9, IETF standard IPFIX

Focused on L3/L4 information and volumetric parameters

Real network traffic to flow statistics reduction ratio 500:1

Flow data

Start Duration Proto Src IP:Port Dst IP:Port Packets Bytes …

Flow

Export

9:35:24.8 0 TCP 192.168.1.1:10111 -> 10.10.10.10:80 1 40 … 9:35:24.8 0.1 TCP 192.168.1.1:10111 -> 10.10.10.10:80 2 80 …

9:35:25.0 0 TCP 10.10.10.10:80 -> 192.168.1.1:10111 1 40 … 9:35:25.0 0.3 TCP 10.10.10.10:80 -> 192.168.1.1:10111 2 156 … 9:35:25.0 0.5 TCP 10.10.10.10:80 -> 192.168.1.1:10111 3 362 … 9:35:25.0 0.7 TCP 10.10.10.10:80 -> 192.168.1.1:10111 4 862 … 9:35:25.0 0.9 TCP 10.10.10.10:80 -> 192.168.1.1:10111 5 1231 …

Flow Monitoring Principle

Flow Gathering Schemes

Probe on a SPAN port Probe on a TAP Flows from switch/router

Pros • Accuracy

• Performance

• L2/L3/L4/L7 visibility

• Same as „on a SPAN“

• All packets captured

• Separates RX and TX

• Already available

• No additional HW

• Traffic on interfaces

Cons • May reach capacity limit

• No interface number

• Additional HW • Usually inaccurate

• Visibility L3/L4

• Performance impact

Facts • Fits most customers

• Limited SPANs number

• 2 monitoring ports • Always test before use

Use • Enterprise networks • ISP uplinks, DCs • Branch offices (MPLS, …)

Flow-Enabled Devices

Network equipment (routers/switched) Traditional capability known for many years

Firewalls, UTMs, load balancers, hypervisors Ongoing initiative of majority of vendors

Packet brokers and matrix switches Convenient option

Flow-Based Traffic Analysis

Network as a sensor concept (and enforcer) blogs.cisco.com/enterprise/the-network-as-a-security-sensor-and-enforcer

Bridges the gap left by signature-based security

Key technology for incident response

Designed for multi 10G environment

Statistical analysis Volumetric DDoS detection

Advanced data analysis algorithms Detection of non-volumetric anomalies

DDoS Anomaly detection

How L3/L4 Data Helps Security?

Myth 1: Flow is sampled and highly inaccurate.

This is true for sFlow and NetFlow Lite

For NetFlow/IPFIX this depends on flow source

Probes and new network equipment do just fine

Myth 2: Flow is limited to L3/L4 visibility.

This is the original design but today’s flow data come with L2 and L7

extensions (usually using IPFIX)

Myth 3: You need continuous packet capture.

Flows with L7 visibility + on-demand or triggered packet capture is cost

efficient option

Flow vs. Packet Analysis

Strong aspects Weak aspects

Flow data

• Works in high-speed networks

• Resistant to encrypted traffic

• Visibility and reporting

• Network behavior analysis

• No application layer data

• Sometimes not enough details

• Sampling (routers, switches)

Packet

analysis

• Full network traffic

• Enough details for troubleshooting

• Supports forensic analysis

• Signature based detection

• Useless for encrypted traffic

• Usually too much details

• Very resource consuming

Solution?

Take advantage of strong aspects in one solution

Versatile and flexible Probes for visibility into all

network layers – Flowmon long-term strategy

Probes (by Flowmon Networks)

Versatile and flexible network appliances

Monitoring ports convert packets to flows

Un-sampled export in NetFlow v5/v9 or IPFIX

Wire-speed, L2-L7 visibility, PCAPs when needed

L2

• MAC

• VLAN

• MPLS

• GRE tunnel

• OVT

L3/L4

• Standard items

• NPM metrics

• RTT, SRT, …

• TTL, SYN size, …

• ASN

• Geolocation

L7

• NBAR2

• HTTP

• DNS

• DHCP

• SMB/CIFS

• VoIP (SIP)

Use Case: Enterprise Security NBAD: On-demand Triggered Packet Capture

Fighting Advanced Threats

Network visibility is essential

component of new protection

strategies against advanced attacks.

Flowmon ADS Principles

Flo

wm

on A

DS

Machine Learning

Adaptive Baselining

Heuristics

Behavior Patterns

Reputation Databases

Traffic Analysis (Using Flows)

Bridges the gap left by endpoint and perimeter security solutions

Behavior based Anomaly Detection (NBA)

Detection of security and operational issues

Attacks on network services, network reconnaissance

Infected devices and botnet C&C communication

Anomalies of network protocols (DNS, DHCP, …)

P2P traffic, TOR, on-line messengers, …

DDoS attacks and vulnerable services

Configuration issues

SIEM Integration

Event exporting (syslog based)

Links Flowmon <-> Log Management

Special vendor relationships IBM QRadar (whitepaper, integration SW package)

ArcSight native support through CEF

Network Traffic Monitoring

Collection and Behavior Analysis

Flowmon Collector & ADS

Event Collection and Correlation

SIEM system

NetFlow

IPFIX

Syslog

SNMP

Traffic

overview,

anomalies

detected

Attacker

activity (port

scan, SSH

authentica-

tion attack)

Victim of the

attack,

source of

anomalies

Attacker is looking for potential victims

And starts SSH attack

That turns out to be successful

Few minutes

after that

breached

device

starts to

communicate

with botnet

C&C

Botnet

identification

using

Flowmon

Threat

Intelligence

Flow data on

L2/L3/L4

Including L7

visibility

Full packet

capture and

packet trace

(PCAP file)

Analysis of

PCAP file

with botnet

C&C

communica-

tion in

Wireshark

Data

exfiltration

command via

ICMP

Command to

discover RDP

servers

ICMP

anomaly

traffic with

payload

present

PCAP

available,

what is the

ICMP

payload?

Linux

/etc/passwd

file with user

accounts and

hash of

passwords

Looking for

Windows

servers with

RDP

Attack

against RDP

services

Network Against Threats

Flow monitoring including L7

Network Behavior Analysis Full packet capture

Triggered by detection

Use Case: DDoS Protection Volumetric DDoS Detection

Traffic Redirection and Mitigation Control

Enterprise Protection Strategy

Enterprise perimeter scheme

Limited number of uplinks and capacity

In-line DDoS mitigation appliance

All-in-one detection & mitigation out of the box

Volumetric + application (L3/L4/L7) attacks coverage

Up to the uplink capacity!

Internet

CPE

DMZ

LAN

Backbone Protection Strategy

Backbone perimeter specifics

Multiple peering points – routers & uplinks

Large transport capacity – tens of gigabits easily

In-line protection is close to impossible!

Flow-based detection and out-of-path mitigation

Easy and cost efficient to deploy in backbone/ISP

Prevents volumetric DDoS to reach enterprise perimeter

flow export 1. Flow collection

2. DDoS detection

3. Routing control

4. Mitigation control

Flow-Based DDoS Protection

Define customers = protected segments Usually by network subnets (simple)

Configure rules for DDoS detection Multiple types of baselines per protected segment

Set alerts Notify about attacks (humans & systems)

Configure traffic diversion = changes in routing Divert traffic for mitigation of DDoS attack

Configure mitigation control = scrubbing Integration with scrubbing equipment or services

Attack Detection

For each segment, a set of baselines is learned from monitored

traffic. The attack is detected if the current traffic exceeds defined

threshold.

Baseline is learned for:

TCP traffic with specific flags

UDP traffic

ICMP traffic

Attack Reporting

Start/end time

Attack target

Type and status

Traffic volumes during

attack/peace time

Attack targets (top 10 dst IPs,

source subnets, L4 protocols,

TCP flags combinations …)

Response to Attack

Alerting E-mail, Syslog, SNMP trap

Routing diversion PBR (Policy Based Routing)

BGP (Border Gateway Protocol)

BGP Flowspec

RTBH (Remotely-Triggered Black Hole)

User-defined scripting

Automatic mitigation With out-of-band mitigation devices

With services of Scrubbing centers

DDoS Protection Scenario 1

Out-of-path Mitigation

Out-of-Path Mitigation

Internet Service Provider Core

Flow Data Collection

Learning Baselines

Attack

Anomaly Detection

Mitigation

Enforcement

Scrubbing center

Attack path Clean path

Traffic Diversion via

BGP Route Injection

Dynamic Protection

Policy Deployment

incl. baselines and

attack characteristics

Protected Object 1

e.g. Data Center,

Organization,

Service etc…

Protected Object 2

DDoS Protection Scenario 2

Mitigation with BGP Flowspec

BGP Flowspec

Requires dynamic signature of the attack

Provides specific action to take with network traffic

BGP Flowspec rules are based on

Destination Prefix

Source Prefix

IP Protocol

Destination port

ICMP type

ICMP code

BGP Flowspec Rule

BGP Flowspec rules are proposed based on dynamic attack

signature

Manual or automatic trigger is available

Default action can be modified

Rule is pushed to routers via BGP session

BGP Flowspec Scenario

Internet Service Provider Core

Flow Data Collection

Learning Baselines

Attack

Anomaly Detection

Mitigation

Enforcement

Protected Object 1

e.g. Data Center,

Organization,

Service etc…

Protected Object 2

Sending specific

Route advertisement

via BGP FlowSpec

Dynamic signature:

Dst IP: 1.1.1.1/32

Dst Port: 135

Protocol IP: 17

(UDP)

Discard

Dropped traffic for

Dst IP: 1.1.1.1/32

Dst Port: 135

Protocol IP: 17

(UDP)

Flowmon Networks

700+ customers

30+ countries

Strong R&D

background

First 100G probes

in the world

European

origin

is an international vendor devoted to

innovative network traffic &

performance & security monitoring

Customer references

The only vendor recognized in both NetFlow related

Gartner reports – network visibility & security

MAGIC QUADRANT

Technology partner of premium vendors

Flowmon Portfolio

Network Visibility

IT Operations Security

Network

Performance

Monitoring

and

Diagnostics

Application

Performance

Monitoring

Network

Behavior

Analysis

DDoS

Detection

& Mitigation NPMD APM NBA

Flowmon Architecture

Flow export from

already deployed

devices

Flow data export +

L7 monitoring

Flow data

collection,

reporting, analysis

Flowmon modules for advanced flow data analysis

Flowmon Architecture

Flowmon Probes & Collectors

Flowmon extension modules

Flowmon Anomaly Detection

Flowmon DDoS Defender

Application Performance Monitoring

Flowmon Traffic Recorder Network Visibility

Troubleshooting

Flowmon

Collector IPFIX/NetFlow

export

Network Security

Anomaly Detection

Application Performance

Monitoring

DDoS Protection

User Perspective

Next Generation Network Monitoring (NetFlow/IPFIX) Full network traffic visibility

Close to real-time and historical data for LAN & WAN & Internet communications

Network operation & connectivity cost optimization

Effective troubleshooting

Next Generation Network Security (NBA, NBAD) Bridges the gap left by endpoint and perimeter and signature

based security solutions

Behavior-based Anomaly Detection

Detection of polymorphic malwares, zero days attacks, suspicious data transfers, behavior changes and various operational and configuration issues

User Perspective

Full Packet Capture On-demand packet capture for troubleshooting and forensic

analysis producing PCAP files

Traffic capture capabilities on 1G/10G/40G

Distributed architecture

Application Performance Monitoring Agent-less monitoring of all user transactions

No influence on target application

Designed for HTTP/HTTPS and SQL applications

DDoS Protection Flow-based detection of volumetric attacks

Universal deployment scenarios (stand-alone, integrated, with scrubbing center)

Traffic diversion and control of mitigation process

Summary Make Use of Flow Data

Levels of Visibility

SNMP monitoring

Amount of transferred data, number of packet, insufficient

Flow monitoring (based on IP flows)

Traffic structure visibility, anomaly detection and reporting

Packet analysis

For forensics and to deal with specific issues

Flowmon Basic monitoring

Using Flow Data for Security

Keep in mind that there is no silver bullet

Security is balanced combination of

technology, people, processes

Flow data & Probes can help you with

Moving the infrastructure monitoring into next level

Traffic visibility, engineering and troubleshooting

Performance reporting and analysis

Bridging the gap left by signature-based products

Detection and mitigation control of volumetric DDoS

Incident response and on-demand full packet capture

Flowmon Networks, a.s.

U Vodarny 2965/2

619 00 Brno, Czech Republic

www.flowmon.com

Thank you