01/04/2007ecs236 winter 20071 intrusion detection ecs236 winter 2007: intrusion detection #2:...
Post on 19-Dec-2015
219 views
TRANSCRIPT
01/04/2007 ecs236 winter 2007 1
ecs236 Winter 2007:
Intrusion DetectionIntrusion Detection#2: Anomaly Detection
Dr. S. Felix Wu
Computer Science Department
University of California, Davishttp://www.cs.ucdavis.edu/~wu/
01/04/2007 ecs236 winter 2007 2
Intrusion DetectionIntrusion Detection
IntrusionDetection
Model
Input eventsequence Results
Pattern matching
01/04/2007 ecs236 winter 2007 3
Scalability of DetectionScalability of Detection
Number of signatures, amount of analysis Unknown exploits/vulnerabilities
01/04/2007 ecs236 winter 2007 4
Anomaly vs. SignatureAnomaly vs. Signature Signature Intrusion (Bad things happen!!)
– Misuse produces observable bad effect– Specify and look for bad behaviors
Anomaly Intrusion (Good things did not happen!!)– We know what our normal behavior is– Looking for an deviation from the normal
behavior, raise early warning
01/04/2007 ecs236 winter 2007 5
Reasons for “AND”Reasons for “AND”
Unknown attacks (insider threat) Better scalability
– AND target/vulnerabilities– SD exploits
01/04/2007 ecs236 winter 2007 6
Another definition…Another definition… Signature-based detection
– Predefine the signatures of anomalies– Pattern matching
Statistics-based detection– Build statistics profile for expected behaviors
– Compare testing behaviors with expected behaviors
– Significant deviation
Convert our limited/partial understanding/modeling about the target system or protocol into detection heuristics (i.e., BUTTERCUP signatures)
Based on our experience, select a set of “features” that will likely to distinguish expected from unexpected behavior.
01/04/2007 ecs236 winter 2007 8
What is “vulnerability”?What is “vulnerability”?
Signature Detectioncreate “effective/strong/scaleable” signatures
Anomaly Detectiondetect/discover “unknown vulnerabilities”
01/04/2007 ecs236 winter 2007 9
ANDAND(ANomaly Detection)(ANomaly Detection)
Unknown Vulnerabilities/Exploits Insider Attacks
Understand How and Why these things happened
Understand the limit of AND from both sides
01/04/2007 ecs236 winter 2007 11
Intrusion DetectionIntrusion Detection
IntrusionDetection
Model
Input eventsequence Results
Pattern matching
01/04/2007 ecs236 winter 2007 12
Anomaly DetectionAnomaly Detection
IntrusionDetection
Input eventsequence
Pattern matching
01/04/2007 ecs236 winter 2007 13
For each sample of the statistic measure, X
(0, 1]
40%
(1, 3]
30%
(3, 15]
20%
(15, +)
10%
Input Events
iq 1+iq
SAND
01/04/2007 ecs236 winter 2007 14
quantify the anomalies
alarm generation
threshold control
raw events
long term profile
0 5 10 15 20 25 300
“But, which feature(s) to profile??”
function F
01/04/2007 ecs236 winter 2007 15
Statistic-based ANomaly DetectionStatistic-based ANomaly Detection(SAND)(SAND)
choose a parameter (a random variable hopefully without any assumption about its probabilistic distribution)
record its statistical “long-term” profile check how much, quantitatively, its short-term
behavior deviates from its long term profile set the right threshold on the deviation to raise
alarms
01/04/2007 ecs236 winter 2007 16
decay
update
clean
compute thedeviation
alarm generation
threshold control
timer control
raw events long term profile
0 5 10 15 20 25 300
01/04/2007 ecs236 winter 2007 17
False Positive & NegativeFalse Positive & Negative
Long term profile Quantitative measure of the deviation
between long term and target of detection Threshold-based control
01/04/2007 ecs236 winter 2007 18
Long-term ProfileLong-term Profile
Category, C-Training learn the aggregate distribution of a statistic
measure
Q Statistics, Q-Training
learn how much deviation is considered normal
Threshold
01/04/2007 ecs236 winter 2007 19
Long-term Profile: C-TrainingLong-term Profile: C-TrainingFor each sample of the statistic measure, X
(0, 50]
20%
(50, 75]
30%
(75, 90]
40%
(90, +)
10% k bins Expected Distribution, P1 P2 ... Pk , where
Training time: months ∑= =ki ip1 1
01/04/2007 ecs236 winter 2007 20
Long-term Profile: Q-Training (1)Long-term Profile: Q-Training (1)
For each sample of the statistic measure, X
(0, 50]
20%
(50, 75]
40%
(75, 90]
20%
(90, +)
20%
k bins, samples fall into bin samples in total ( ) Weighted Sum Scheme with the fading factor s
iY ′ thiN ′ ∑= ′=′k
i i NY1
01/04/2007 ecs236 winter 2007 21
ThresholdThreshold Predefined threshold, If Prob(Q>q) < , raise alarm
0
0.08
0 5 10 15 20 25 30
Q bins
Probability TH_redTH_yellow
∑= ×′
×′−′=
k
i i
ii
pN
pNYQ
1
2)(
01/04/2007 ecs236 winter 2007 22
Long-term Profile: Q-Training (2)Long-term Profile: Q-Training (2)
Deviation:
Example:
Qmax
the largest value among all Q values
∑= ×′
×′−′=
k
i i
ii
pN
pNYQ
1
2)(
33.21.010
)1.0102(
4.010
)4.0102(
3.010
)3.0104(
2.010
)2.0102( 2222
=××−
+××−
+××−
+××−
=Q
01/04/2007 ecs236 winter 2007 23
Long-term Profile: Q-Training (3)Long-term Profile: Q-Training (3)
Q Distribution [0, Qmax) is equally divided into 31 bins and
the last bin is [Qmax, +)distribute all Q values into the 32 bins
01/04/2007 ecs236 winter 2007 24
Q-MeasureQ-Measure
Deviation:
Example:
Qmax
the largest value among all Q values
∑= ×′
×′−′=
k
i i
ii
pN
pNYQ
1
2)(
33.21.010
)1.0102(
4.010
)4.0102(
3.010
)3.0104(
2.010
)2.0102( 2222
=××−
+××−
+××−
+××−
=Q
01/04/2007 ecs236 winter 2007 26
ThresholdThreshold Predefined threshold, If Prob(Q>q) < , raise alarm
0
0.08
0 5 10 15 20 25 30
Q bins
Probability TH_redTH_yellow
∑= ×′
×′−′=
k
i i
ii
pN
pNYQ
1
2)(
Falsepositive
01/04/2007 ecs236 winter 2007 27
Heidelberg
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0 5 10 15 20 25 30 35Q bins
Probability
NCU
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0 5 10 15 20 25 30 35Q bins
Probability
SingNet
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0 5 10 15 20 25 30 35Q bins
Probability
UIUC
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0 5 10 15 20 25 30 35Q bins
Probability
01/04/2007 ecs236 winter 2007 28
MathematicsMathematics Many other techniques:
– Training/learning– detection
01/04/2007 ecs236 winter 2007 29
Statistical Profiling Long-Term profile:Long-Term profile:
capture long-term behavior of a particular capture long-term behavior of a particular statistic measurestatistic measure
e.g., update once per daye.g., update once per day half-life: 30 updateshalf-life: 30 updates
recent 30: recent 30: 50%50% 31-60: 31-60: 25%25% the newer contributes morethe newer contributes more
01/04/2007 ecs236 winter 2007 30
Statistical Pros and ConsStatistical Pros and Cons Slower to detect - averaging window Very good for unknown attacks - as long as
“relevant measures” are chosen Environment (protocol, user, etc)
dependency– Need good choices on statistical measures– Statistical profiles might be hard to build– Thresholds might be hard to set
01/04/2007 ecs236 winter 2007 31
decay
update
clean
compute thedeviation
alarm generation
threshold control
timer control
raw events long term profile
0 5 10 15 20 25 300
01/04/2007 ecs236 winter 2007 32
Weighted Sum SchemeWeighted Sum Scheme Problems of Sliding Window Scheme
Keep the most recent N pieces of audit records
required resource and computing time are O(N)
12
,2
12
1 +×==
≠×=
+×=
−=
−
−
∑
NYN
ijYY
YY
ki i
jj
ii
Assume K: number of bins Yi: count of audit records falls
into ith bin N: total number of audit records : fading factor
When Ei occurs, update
01/04/2007 ecs236 winter 2007 33
FTP Severs and ClientsFTP Severs and Clients
FTP Client
SHANG
FTP Servers
Heidelberg
NCU
SingNet
UIUC
01/04/2007 ecs236 winter 2007 34
Dropper AttacksDropper Attacks
P%P% Per (K,I,S)Ret (K,S)Ran (K)
Intentional or Unintentional??
01/04/2007 ecs236 winter 2007 35
Periodical Packet DroppingPeriodical Packet Dropping
Parameters (K, I, S) K, the total number of dropped packets in a connection I, the interval between two consecutive dropped packets S, the position of the first dropped packet.
Example (5, 10, 4) 5 packets dropped in total 1 every 10 packets start from the 4th packet The 4th, 14th, 24th, 34th and 44th packet will be dropped
01/04/2007 ecs236 winter 2007 36
Retransmission Packet DroppingRetransmission Packet Dropping
Parameters (K, S) K, the times of dropping the packet's retransmissions S, the position of the dropped packet
Example (5, 10) first, drops the 10th packet then, drops the retransmissions of the 10th packet 5
times
01/04/2007 ecs236 winter 2007 37
Random Packet DroppingRandom Packet Dropping
Parameters (K) K, the total number of packets to be dropped in a
connection
Example (5) randomly drops 5 packets in a connection
01/04/2007 ecs236 winter 2007 38
Experiment SettingExperiment Setting
FTP
Internet
Divert Socket
FTP Client
xyz.zip 5.5M
FTP Server
Attack Agent
Data Packets
01/04/2007 ecs236 winter 2007 39
Impacts of Packet Dropping On Impacts of Packet Dropping On Session DelaySession Delay
5663.4 66
218.4
98.6
108.2
125.8
250.9
62.6
77.1
86.9
260.3
23.626.5
44.6
183.9
0
50
100
150
200
250
300
Session Delay (s)
Heidelberg NCU SingNet UIUC
Normal
RanPD(7)
PerPD(7, 4, 5)
RetPD(7, 5)
01/04/2007 ecs236 winter 2007 40
Compare Impacts of Dropping Compare Impacts of Dropping PatternsPatterns
0
50
0
-10 40
Num ber
Session
He ide lbe rg
0
50
100
150
200
250
300
350
400
450
500
0 10 20 30 40
Nu m b er o f victim p ack ets
Session delay
PerPD
RanPD
RetPD
NCU
0
50
100
150
200
250
300
350
400
450
500
0 10 20 30 40
Nu m b er o f victim p ack ets
Session delay
PerPD
RanPD
RetPD
SingNe t
0
50
100
150
200
250
300
350
400
450
500
0 10 20 30 40
Nu m b er o f victim p ack ets
Session delay
PerPD
RanPD
RetPD
UIUC
0
50
100
150
200
250
300
350
400
450
500
0 10 20 30 40
Nu m b er o f victim p ack ets
Session delay
PerPD
RanPD
RetPD
PerPD: I=4, S=5
RetPD: S=5
01/04/2007 ecs236 winter 2007 41
bone
fire
redwing
light
152.1.75.0
192.168.1.0
172.16.0.0
UDP flood
FTP data
TFN agents
TFN target
FTP client
FTP server
congestion
air
TFN master
01/04/2007 ecs236 winter 2007 42
flood 1, Stop 20
0
2
4
6
8
10
12
0 20 40 60 80 100
Time (s)
Number of Lost Packets
flood 1, Stop 5
0
2
4
6
8
10
12
0 20 40 60 80 100
Time (s)
Number of Lost Packets
flood 5, Stop 10
0
2
4
6
8
10
12
0 20 40 60 80 100
Time (s)
Number of Lost Packets
flood 5, Stop 2
0
2
4
6
8
10
12
0 20 40 60 80 100
Time (s)
Number of Lost Packets
01/04/2007 ecs236 winter 2007 43
TDSAM Experiment SettingTDSAM Experiment Setting
FTP
Internet
Divert Socket
FTP Client
xyz.zip 5.5M
FTP Server
Attack Agent
TDSAM
Data Packets
p1, p2, p3, p5, p4max
reordering counting
01/04/2007 ecs236 winter 2007 44
Heidelberg
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0 5 10 15 20 25 30 35Q bins
Probability
NCU
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0 5 10 15 20 25 30 35Q bins
Probability
SingNet
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0 5 10 15 20 25 30 35Q bins
Probability
UIUC
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0 5 10 15 20 25 30 35Q bins
Probability
01/04/2007 ecs236 winter 2007 45
He ide lbe rg
0
0.05
0.1
0.15
0.2
0.25
0.3
0 5 10 15 20 25 30 35
Q bins
Probability
NCU
0
0.05
0.1
0.15
0.2
0.25
0.3
0 5 10 15 20 25 30 35
Q bins
Probability
SingNet
0
0.05
0.1
0.15
0.2
0.25
0.3
0 5 10 15 20 25 30 35
Q bins
Probability
UIUC
0
0.05
0.1
0.15
0.2
0.25
0.3
0 5 10 15 20 25 30 35
Q bins
Probability
01/04/2007 ecs236 winter 2007 46
Results: Position MeasureResults: Position MeasureHeidelberg NCU SingNet UIUCPosition
nbin=5 DR MR DR MR DR MR DR MR
Normal* - 4.0% - 5.4% - 3.5% - 6.5% -
(10, 4, 5) 99.7% 0.3% 100% 0% 100% 0.0% 100% 0%
(20, 4, 5) 100% 0% 98.1% 1.9% 99.2% 0.8% 100% 0%
(40, 4, 5) 96.6% 3.4% 100% 0% 100% 0% 98.5% 1.5%
(20, 20, 5) 100% 0% 100% 0% 100% 0 % 100% 0%
(20, 100, 5) 98.9% 1.1%. 99.2% 0.8% 99.6% 0.4% 99.1% 0.9%
(20, 200, 5) 0% 100% 76.5% 23.5% 1.5% 98.5% 98.3% 1.7%
PerPD
(100, 40, 5) 0.2% 99.8% 0% 100% 0% 100% 100% 0%
RetPD (5, 5) 84.9% 15.1% 81.1% 18.9% 94.3% 5.7% 97.4% 2.6%
10 0% 100% 42.3% 57.7% 0% 100% 0% 100%RanPD
40 0% 100% 0% 100% 0% 100% 0% 100%
5 98.6% 1.4% 100% 0% 98.2% 1.8% 100% 0%Intermittent
(10, 4, 5) 50 34.1% 65.9% 11.8% 88.2% 89.4% 10.6% 94.9% 5.1%
01/04/2007 ecs236 winter 2007 47
Results: Delay MeasureResults: Delay MeasureHeidelberg NCU SingNet UIUCDelay
nbin=3 DR MR DR MR DR MR DR MR
Normal* - 1.6% - 7.5% - 2.1% - 7.9% -
(10, 4, 5) 97.4% 2.6% 95.2% 4.8% 94.5% 5.5% 99.2% 0.8%
(20, 4, 5) 99.2% 0.8% 98.5% 1.5% 100% 0% 100% 0%
(40, 4, 5) 100% 0% 100% 0% 100% 0% 100% 0%
(20, 20, 5) 96.3% 3.7% 100% 0% 92.6% 7.4% 98.9% 1.1%
(20, 100, 5) 100% 0% 95.3% 4.7% 98.7% 1.3% 100% 0%
(20, 200, 5) 98.6% 1.4% 99% 1% 97.1% 2.9% 100% 0%
PerPD
(100, 40, 5) 100% 0% 100% 0% 100% 0% 100% 0%
RetPD (5, 5) 100% 0% 100% 0% 100% 0% 100% 0%
10 74.5% 25.5% 26.8% 73.2% 67.9% 32.1% 99.5% 0.5%RanPD
40 100% 0% 100% 0% 100% 0% 100% 0%
5 25.6% 74.4% 0% 100% 0% 100% 97.3% 2.7%Intermittent
(10, 4, 5) 50 0% 100% 24.9% 75.1% 0% 100% 3.7% 96.3%
01/04/2007 ecs236 winter 2007 48
Results: NPR MeasureResults: NPR MeasureHeidelberg NCU SingNet UIUCNPR
nbin=2 DR MR DR MR DR MR DR MR
Normal* - 4.5% - 5.8% - 8.2% - 2.9% -
(10, 4, 5) 0% 100% 14.4% 85.6% 29.1% 70.9% 100% 0%
(20, 4, 5) 83.1% 16.9% 94.2% 5.8% 95.2% 4.8% 100% 0%
(40, 4, 5) 100% 0% 97.4% 2.6% 100% 0% 100% 0%
(20, 20, 5) 91.6% 8.4% 92% 8% 93.5% 6.5% 100% 0%
(20, 100, 5) 94.3% 5.7% 92.2% 7.8% 96.4% 3.6% 100% 0%
(20, 200, 5) 0% 100% 96.5% 3.5% 94.8% 5.2% 100% 0%
PerPD
(100, 40, 5) 100% 0% 100% 0% 100% 0% 100% 0%
RetPD (5, 5) 0% 100% 84.7% 15.3% 23.9% 76.1% 46.5% 53.5%
10 0% 100% 0% 100% 100% 0% 100% 0%RanPD
40 100% 0% 100% 0% 100% 0% 100% 0%
5 0% 100% 0% 100% 82.2% 17.8% 100% 0%Intermittent
(10, 4, 5) 50 0% 100% 1% 99% 40% 60% 64.8% 35.2%
01/04/2007 ecs236 winter 2007 49
Results (good and bad)Results (good and bad)
False Alarm Rate less than 10% in most cases, the highest is 17.4%
Detection Rate Position: good on RetPD and most of PerPD
at NCU, 98.7% for PerPD(20,4,5), but 0% for PerPD(100, 40, 5) in which dropped packets are evenly distributed
Delay: good on those significantly change session delay, e.g., RetPD, PerPD with a large value of K
at SingNet, 100% for RetPD(5,5), but 67.9% for RanPD(10)
NPR: good on those dropping many packets at Heidelberg, 0% for RanPD(10), but 100% for RanPD(40)
01/04/2007 ecs236 winter 2007 50
Performance AnalysisPerformance Analysis
Good sites correspond to a high detection rate. stable and small session delay or packet reordering
e.g., using Delay Measure for RanPD(10): UIUC (99.5%) >
Heidelberg(74.5%) > SingNet (67.9%) > NCU (26.8%)
How to choose the value of nbin is site-specific e.g., using Position Measure, lowest false alarm rate occurs when
nbin= 5 at Heidelberg(4.0%) and NCU(5.4%), 10 at UIUC(4.5%) and
20 at SingNet(1.6%)
01/04/2007 ecs236 winter 2007 51
decay
update
clean
compute thedeviation
alarm generation
threshold control
timer control
raw events long term profile
0 5 10 15 20 25 300
01/04/2007 ecs236 winter 2007 52
decay
update
clean
cognitivelyidentify thedeviation
alarm identification
InformationVisualizationToolkit
raw events cognitive profile
01/04/2007 ecs236 winter 2007 54
What is an anomaly?What is an anomaly? The observation of a target system is
inconsistent, somewhat, with the expected conceptual model of the same system
01/04/2007 ecs236 winter 2007 55
What is an anomaly?What is an anomaly? The observation of a target system is
inconsistent, somewhat, with the expected conceptual model of the same system
And, this conceptual model can be ANYTHING.– Statistical, logical, or something else
01/04/2007 ecs236 winter 2007 56
Model vs. ObservationModel vs. Observationthe Model Anomaly Detection
Conflicts Anomalies
It could be an attack, but it might well be misunderstanding!!
01/04/2007 ecs236 winter 2007 57
The ChallengeThe Challenge
Events
Expected Behavior Model
Anomaly Detection
Knowledge about the Target
False Positives & Negatives
01/04/2007 ecs236 winter 2007 58
What is an anomaly?What is an anomaly?
Events
Expected Behavior Model
Anomaly Detection
01/04/2007 ecs236 winter 2007 59
What is an anomaly?What is an anomaly?
Events
Expected Behavior Model
Anomaly Detection
Knowledge about the Target
01/04/2007 ecs236 winter 2007 60
Model vs. ObservationModel vs. Observationthe Model Anomaly Detection
Conflicts Anomalies
It could be an attack, but it might well be misunderstanding!!
01/04/2007 ecs236 winter 2007 61
ChallengeChallenge We know that the detected anomalies can
be either true-positive or false-positive. We try all our best to resolve the puzzle by
examining all information available to us. But, the “ground truth” of these anomalies
is very hard to obtain– even with human intelligence
01/04/2007 ecs236 winter 2007 62
Problems with ANDProblems with AND We are not sure about whatever we want to
detect… We are not sure either when something is
caught… We are still in the dark… at least in many
cases…
01/04/2007 ecs236 winter 2007 63
Anomaly ExplanationAnomaly Explanation How will a human resolve the conflict?
The Power of Reasoning and Explanation– We detected something we really want to detect
reducing false negative– Our model can be improved reduce false
positive
01/04/2007 ecs236 winter 2007 64
Without ExplanationWithout Explanation AND is not as useful?? Knowledge is the power to utilize
information!– Unknown vulnerabilities– Root cause analysis– Event correlation
01/04/2007 ecs236 winter 2007 65
Anomaly ExplanationAnomaly Explanation
the Model Anomaly Detection
Anomaly Analysis and Explanation
EBL
Explaining both the attack and the normal behavior
01/04/2007 ecs236 winter 2007 66
ExplanationExplanation
SimulationExperiments
OrObservatinon
Conflicts Anomalies
01/04/2007 ecs236 winter 2007 67
the Modelmodel-based
event analysis
observed system events
SBL-basedAnomalyDetection
analysisreports
ExampleSelection
Explanation Based
Learning
modelupdate
01/04/2007 ecs236 winter 2007 68
AND AND EXPAND EXPAND
Anomaly Detection– Detect– Analysis and Explanation– Application