1

Pegasus: Precision Hunting for Icebergs

and Anomalies in Network Flows

Sriharsha Gangam1, Puneet Sharma2, Sonia Fahmy1

1Purdue University, 2HP Labs

This research has been sponsored in part by GENI project 1723, and by Hewlett-Packard

2

Passive Flow Monitoring

• Detect network congestion, attacks, faults, anomalies, traffic engineering and accounting

• Observe and collect traffic summaries

• e.g., InMon traffic sentinel [InMon] uses sFlow, Cisco’s NetFlow is used in ISPs

Monitoring Data

Collection &Analysis

Network Devices

e.g., switches

[InMon] http://inmon.com

3

Passive Flow Monitoring - Challenges

• Large overhead to collect and analyze fine-grained flow data

• Increasing link speeds, network size and traffico Limited CPU, memory resources at the routerso Millions of flows in ISP networks

• Current Techniques?o NetFlow sampling rate in ISPs ~ 1 in 100 (Internet2)o sFlow packet sampling rate ~ 1 in 2000o Application dependent sketcheso Fine-grained information is lost

4

Will More Resources Help?

• Commercial co-located compute and storageo HP ONE Bladeso Cisco SRE Modules

• Example configurationo 2.20 GHz Core Duo processoro 4 GB RAM, 250 GB HDo 2x10 Gbps duplex bandwidth to switch

• Storage and Analysis of fine-grained flow statisticso Distributed monitoring applications

5

Design Space

Network Overhead

Additional

Compute &

Storage

Acc

urac

y

Ideal Solutio

n

Current Solutions: Sampling and

Sketching

Our Goals: Pegasus - Accurate & low overhead monitoring

Naïve Solutio

n

Impractical

6

Key Class of Applications

• Network bottleneckso Top traffic destinations, sources, and links

• Suspicious port scanning activityo Sources that connect to more than 10% hosts within

time T

• DDoS attack detectiono Destinations with large number of connections or traffic

7

Global Iceberg Detection

• Items with aggregate count exceeding a threshold (S xθ)o Global heavy hitters

• Observations at any single switch/router may not be significant or interestingo E.g., DDoS attack

Monitoring Data

Items contributing > 1% (θ)

traffic?Network Devices

e.g., switches

h1 h2 h4 …

20

10

50

…

h2 h3 h4 …

60

15

20

…

h3 h5 h6 …

50

10

30

…

8

Online Iceberg Detection with

Pegasus• Reduce

communication overhead o Additional compute and

storage

• Precisely detect all global icebergs o zero false positives and

false negatives

• Feedback based iterative approach

High precision

Iterative solution

Comparison of Different Approaches

Network Devices

e.g., switches

Naïve Approac

h Prohibitively large Monitoring Data

Collection &Analysis

(Aggregator)

Sampling and

SketchingLossy Summary:

False +ves and -ves

Pegasus

Lossy Summary: Sketch-sets

i1 i2 i4 …

20

10

50

…

i2 i3 i4 …

60

15

20

…

i3 i5 i6 …

50

10

30

…

Fine-grained data on-demand:

No False +ves or -ves

Monitor

10

1- D Sketch-set Representation

• Sketch-set: Summary representation of a collection of flows, supports set operations

β

Coarse Sketch-set Generation

(Destination IP, Packet Count)

128.41.10.10, 128.41.10.50, 15, 30

128.41.10.110, 128.41.10.150, 100, 110

128.41.10.210, 128.41.10.210, 300, 300Coarse-grained sketch-sets

α

128.41.10.10, 15

128.41.10.20, 20

128.41.10.30, 15

128.41.10.40, 30

128.41.10.50, 25

128.41.10.110, 110

128.41.10.150, 100

128.41.10.210, 300

(startIP, endIP, minPkt, maxPkt)

Example: Destinations IPs receiving more than 200 packets

11

Example

128.41.10.35, 128.41.10.70, 10, 35

128.41.10.100, 128.41.10.120, 90, 130

128.41.10.10, 128.41.10.50, 15, 30

128.41.10.110, 128.41.10.150, 100, 110

128.41.10.210, 128.41.10.210, 300, 300

(startIP, endIP, minPkt, maxPkt)

Coarse-grained Sketch-sets

Monitor 2

Monitor 1

Aggregator

Disjoint Sketch-sets

INTERSECTION SUBTRACTION

Non-icebergs

Query monitors (uncertai

n)

Iceberg

128.41.10.10, 128.41.10.34, 15, 30

128.41.10.35, 128.41.10.50, 10, 65

128.41.10.51 , 128.41.10.70, 10, 35

128.41.10.100, 128.41.10.109, 90, 130

128.41.10.121, 128.41.120.150, 100, 110

128.41.10.110, 128.41.10.120, 90, 240

128.41.10.210, 128.41.10.210, 300, 300

12

Example…Query Response

Aggregator Query:(128.41.10.110, 128.41.10.120)

128.41.10.110, 90128.41.10.120,

130

128.41.10.110, 110

Monitor 2

Monitor 1Lookup relevant

flows

Generate Sketch-sets (finer

granularity)

128.41.10.110, 128.41.10.110, 90, 90

128.41.10.120, 128.41.10.120, 130, 130

128.41.10.110, 128.41.10.110, 110, 110

13

Example…Query Response

Aggregator Query:(128.41.10.110, 128.41.10.120)

Monitor 2

Monitor 1

128.41.10.110, 128.41.10.110, 90, 90

128.41.10.120, 128.41.10.120, 130, 130

128.41.10.110, 128.41.10.110, 110, 110

128.41.10.110, 128.41.10.110, 200, 200

128.41.10.120, 128.41.10.120, 130, 130

Fine-grained sketch-sets

Aggregator

Non-icebergs

Iceberg

14

Evaluation Methodology

• Abilene traceo Netflow records: 11 sites with 1 in 100 sampling for 5

mino Add small flows to revert sampling

• (90% of flows contribute to 20% of traffic, ~ 758K unique flow records)

o Trace is used in [Huang11]

• Enterprise network sFlow traceo sFlow records: 249 switches,1 in 2000 sampling for a

weeko Revert sampling by adding flows

• PlanetLab’s Outgoing Traffico NetFlow records generated at each PlanetLab host

[Huang11] G. Huang, A. Lall, C. Chuah, and J. Xu. Uncovering global icebergs in distributed streams: Results and implications. J. Netw. Syst. Manage., 19:84–110, March 2011

15

Comparison with Sample-sketch

• Sends sampled monitoring data and sketches to the aggregator for iceberg detection

• Uses two main parameterso Sampling intervalo Sketch threshold

• Difficult to decide the parameters

• Can have false positives and false negatives

G. Huang, A. Lall, C. Chuah, and J. Xu. Uncovering global icebergs in distributed streams: Results and implications. J. Netw. Syst. Manage., 19:84–110, March 2011

16

Abilene Trace

For the 5 min trace, θ = 0.08- Naive solution: ≈ 7.63 MB- Pegasus: ≈ 8 KB- Sample-Sketch: ≈ 36 KB

Larg

er

is b

ett

er

Pegasus has lower communication overhead

θ

θ

17

Monitoring Outgoing PlanetLab Traffic

• Example of end-host monitoring system

• Detect accidental attacks and anomalies originating from PlanetLab

• Existing monitoring service: PlanetFlow o Decouples collection

from analysiso Collects 1 TB of data

every month [PF] (naïve approach)

PlanetLab nodes

Monitor

Aggregator

Monitor

NetFlow records generated from outgoing traffic

[PF] http://www.cs.princeton.edu/~sapanb/planetflow2/

18

Pegasus PlanetLab Service

• PlanetLab’s outgoing traffico NetFlow records of ~250 PlanetLab nodeso Online global iceberg detection service

• Global Iceberg detection foro Flow identifier: Destination IP, Source Port, Destination

Porto Flow size: Packet count

19

Pegasus PlanetLab Service

• 15 hour deployment - Pegasus: 403 MB, Naïve: 2.26 GB

• Most outbound traffic to other PlanetLab hosts• 1- Day outgoing traffic:

• CoDNS and CoDeeN don’t produce many icebergs

Source Port Icebergs Destination Port Icebergs

3 (CompressNET) 3 (CompressNET)

8 (unassigned) 0 (Reserved)

22 (SSH) 53 (DNS)

80 (HTTP) 80 (HTTP), 443 (HTTPS)

20

Conclusions• Pegasus: A distributed measurement system

o Commercial co-located compute and storage deviceso Low network overheado High accuracy

• Adaptive aggregation for the global iceberg detectiono Iterative feedback solution

• Experiments from real traces and PlanetLab deploymento low overhead without false +ves and -ves

21

Thank youQuestions?

22

Anomaly Examples• Based on traffic features [Kind09]

[Kind09] Histogram-Based Traffic Anomaly Detection, In IEEE Trans. on Netwk. Service Management

23

Related Work• Threshold Algorithm (TA) [Fagin03]

o Large number of iterations

• Three phase uniform threshold (TPUT) [Cao04]o Accounting data distributions [Yu05]

• Filtering based continuous monitoring algorithms [Babcock03] [Keralapura06] [Olston03]o Send update to aggregator when local arithmetic

constraints fail

[Yu05] Efficient processing of distributed top-k queries. In Proc. of DEXA, 2005

[Cao04] Efficient Top-K Query Calculation in Distributed Networks. In proc. of PODC, 2004

[Fagin03] Optimal aggregation algorithms for middleware. Jour. of Comp. and Sys. Sciences, 2003

[Babcock03] Distributed top-k monitoring. In Proc. SIGMOD, 2003[Keralapura06] Communication-efficient distributed monitoring of thresholded counts. In Proc. of SIGMOD, 2006[Olston03] Adaptive filters for continuous queries over distributed data streams. In Proc. SIGMOD, 2003

24

Sketch-set Granularity - G

• High granularity ⇒ More precise, more expensive representation

• Granularity definition: maxSize – minSize

• Used to determine if more flows should be combined in a sketch-set

• Used to send finer granularity during monitor response (for convergence)

25

Iterative Feedback Algorithm

26

Abilene Trace

β little influence on the communication

cost

27

Enterprise Network sFlow Trace

Larg

er

is b

ett

er

All except one parameter pair (green) has false

positives and negatives

28

Scalability with Number of Monitors

29

Scalability with Number of Monitors

sFlow trace Abilene trace

Larg

er

is b

ett

er