behavior-based worm detectors compared - raid … stafford.pdf · rbs trwrbs d-actm lesg trw ......

Behavior-based WormDetectors Compared

Shad Stafford and Jun LiUniversity of OregonSeptember 15, 2010RAID 2010

This material is based upon work supported by the National Science Foundation under Grant No. CNS-0644434. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

Behavior-based Worm Detectors Compared Shad Stafford and Jun Li

Introduction

2

Snort

Honeycomb

Netbait

Autograph Anagram HonIDS

MRW

Sweeper

Hamsa

Poseidon

PolygraphVigilante

DACODA TaintCheck

COVERS

PAYLHoneyStatEarlyBirdWEW

DSCRBS

TRWRBS

TRWd-ACTM LESG

NetworkTelescope

DarkAddresses

KalmanFilter

ICMP-Blooms

p2p-idsCUSUM

Wormboy

PGD

CoreBehaviors

Markov N-grams

STILL

ProspectorElcano

AVARE

BRO

What is the best worm detector?


TRWRBS

The Detectors

✦ Each of these detectors:‣ is deployable at a single point (gateway)

‣ does not require access to connection payload

‣ is in theory capable of detecting IKEE.B

3

MRWDSC

RBSTRW

PGD

The Internet

Monitor

ProtectedNetwork


Comparing detectors

4

✦ Experiments performed against different traces

✦Different researchers use different metrics

✦We were unable to find direct comparisons of existing detectors


Our contributions

✦ We identify easily deployed detectors that are capable of detecting modern worms like IKEE.B

✦ We measure their performance at detecting worms‣ across a variety of publicly available network traces

‣ against a number of different worm scanning strategies and speeds

✦ We report their performance using a consistent set of metrics‣ based on the scenario of a network administrator who

wants to detect a worm infection within their protected network

5


TRWRBS

✦ Threshold Random Walk‣ Random scanning creates high connection

failure rates

‣ Measure connection failures and compare to normal

‣ Uses sequential hypothesis testing to detect infection

MRWDSC

RBSTRW

PGD

The Detectors

6

Schechter, Jung, and Berger. Fast detection of scanning worm infections (RAID 2004)


TRWRBS

✦ Rate-Based Sequential hypothesis test‣ Worms scan lots of destinations

‣ Non-scanners show exponential distribution of inter-scan times

‣ Uses sequential hypothesis testing to detect infection

The Detectors

7

MRWDSC

RBSTRW

PGD

Jung, Milito, and Paxson. On the adaptive real-time detection of fast-propagating network worms (DIMVA 2007)


TRWRBS

✦ TRW + RBS‣ Product of TRW and RBS likelihood ratios

The Detectors

8

MRWDSC

RBSTRW

PGD

Jung, Milito, and Paxson. On the adaptive real-time detection of fast-propagating network worms (DIMVA 2007)


RBS✦ Multi-Resolution Windows ‣ Monitors number of destinations contacted

over a series of time windows

‣ Each window has an associated threshold

The Detectors

9

MRWDSC

TRWRBS

TRW

PGD

Sekar, Xie, Reiter, and Zhang. A multi-resolution approach for worm detection and containment (DSN 2006)


✦ Destination-Source Correlation ‣ Monitors hosts which receive a connection on

a given port and then begin initiating connections using that port

‣ If connection rate exceeds threshold, alarm is raised

The Detectors

10

MRWDSC

RBSTRWRBS

TRW

PGD

Gu, Sharif, Qin, Dagon, Lee, and Riley. Worm detection, early warning and response based on local victim information (ACSAC 2004)


✦ Protocol Graph Detector ‣ Builds a graph of hosts communicating over a

given protocol (nodes are hosts)

‣ Graph size is normally distributed, abnormal graph size for a given window indicates presence of worm

The Detectors

11

MRWDSC

RBSTRWRBS

TRW

PGD

Collins and Reiter. Hit-list worm detection and bot identification in large networks using protocol graphs. (RAID 2007)


Metrics

✦ False positive rate‣ By host: the number of false alarms raised during a time

period τ (limited to one alarm per host)

‣ By time: Percentage of minutes during time period τ when a false alarm is triggered

✦ False negative rate‣ Percentage of instances where worm infection is not

detected in time period τ

✦ Detection latency‣ The number of outbound worm connections prior to

detection

12


ProtectedNetwork

Measurement Setup

13

The Internet

Monitor


Experiment Procedure

14

BalanceFalse Positives

Step 3)

Establish Thresholds

Training TraceStep 1)

EvaluationEngine Measure

False Positives

Evaluation TraceStep 2)

EvaluationEngine

MeasureFalse negativesand Latency

Evaluation Trace

Step 4)

WormTrace

EvaluationEngine


Worm traffic

✦ Generated with our GLOWS worm emulator‣ host addresses from legitimate trace for internal network

‣ probabilistic model for rest of Internet

✦ Each scenario starts with a single host being infected by an incoming worm connection

15

The Internet(probabilistically modeled)

GatewayProtected Network(discrete event simulation)

= Active host= Dark Address

Wormflow records


0

1

2

3

4

5

trw rbs trwrbs mrw pgd dsc

Fals

e Al

arm

s (li

mit

1 pe

r hos

t) enterprisecampus

departmentwireless

False Positives (by host count)

16


F- vs. Random Scanning

17

TRW TRWRBS

RBS MRW

DSC PGD

0

20

40

60

80

100

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0

20

40

60

80

100

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0

20

40

60

80

100

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0

20

40

60

80

100

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0

20

40

60

80

100

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0

20

40

60

80

100

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

F- (%

of E

xper

imen

ts)

Scans per sec.

✦TRW the most sensitive

✦RBS the least sensitive

✦PGD shows wide variation in performance


Latency vs. Random Scanning

18

TRW TRWRBS

RBS MRW

DSC PGD

# of

sca

ns b

efor

e de

tect

ion

Scans per sec.

✦No one detector is consistently the fastest

✦ Environment again has a big impact

✦ In some scenarios the worm is detected orders of magnitude faster than in other scenarios

0 10 20 30 40 50

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0 50

100 150 200 250 300

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0 50

100 150 200 250 300 350 400 450

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0 200 400 600 800

1000 1200 1400

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0

5

10

15

20

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0 200 400 600 800

1000 1200 1400

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10


F- vs. Topo Scanning

19

TRW vs topo100 TRWRBS vs topo100


TRW vs topoall TRWRBS vs topoall

F- (%

of E

xper

imen

ts)

Scans per sec.

✦TRW’s performance is greatly impaired even with small numbers of neighbors

‣ Unable to detect topo worm with unlimited neighbors

✦TRWRBS shows limited performance degradation

0

20

40

60

80

100

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0

20

40

60

80

100

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0

20

40

60

80

100

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0

20

40

60

80

100

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0

20

40

60

80

100

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0

20

40

60

80

100

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10


Latency vs. Topo Scanning

20

# of

sca

ns b

efor

e de

tect

ion

Scans per sec.

✦TRW latency increases linearly with number of neighbors

✦TRWRBS latency increases more slowly

✦TRWRBS is faster than TRW for fast Topo worms with large neighbor counts



TRW vs topoall TRWRBS vs topoall

0 20 40 60 80

100 120 140

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0 200 400 600 800

1000

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

-1

-0.5

0

0.5

1

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

Not Detected

0 50

100 150 200 250 300 350 400 450

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0 200 400 600 800

1000 1200 1400

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10

0 100 200 300 400 500 600 700

0.005 0.01 0.02

0.05 0.1 0.2

0.5 1 2 5 10


Conclusions

21

✦ No detector stands out as the best in all situations

✦ 1 scan every 10 seconds avoids most detectors in most circumstances‣ This is a relatively fast worm to go undetected

✦ Environment makes a big difference‣ Wireless environment shows game and file-sharing traffic

which causes problems for the detectors


Questions?

22

Shad [email protected]

Dr. Jun [email protected]

http://netsec.cs.uoregon.edu

behavior-based worm detectors compared - raid … stafford.pdf · rbs trwrbs d-actm lesg trw ......

Documents