para-snort : a multi-thread snort on multi-core ia platform tsinghua university pdcs 2009 november...
TRANSCRIPT
![Page 1: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/1.jpg)
Para-Snort : A Multi-thread Snort on Multi-Core IA Platform
Tsinghua University
PDCS 2009
November 3, 2009
Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue and Jun Li
![Page 2: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/2.jpg)
2
Outline
Introduction of NIDS on IA Some previous work Structure of our system, what’s different? Detailed module design Breaking the bottlenecks Para-Snort Performance Conclusions
![Page 3: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/3.jpg)
3
NIDS on IA platform
NIDS(Network Intrusion Detection System) looks into both header and payload of packets to identify intrusion
Why on IA platform? low price easily to develop flexibility on structure and ruleset
But not so fast as ASICs or FPGA!
![Page 4: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/4.jpg)
4
The structure of NIDS
Snort by Sourcefire Inc. The most popular open
source NIDS on IA platform Preprocess and Detect cost
most computation power
Data acquisition
Decoder
Preprocess
Detect
Output
![Page 5: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/5.jpg)
5
Way to speed up?
Multicore IA platform Leads the trends of higher processor
computation power Need parallel structure of the software Rarely leveraged in existing NIDS
Two previous work: Supra-linear and MultiSnort
![Page 6: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/6.jpg)
6
Supra-linear Packet Processing
Intel Co. in 2006 One data acquisition
component Duplicated other
components No memory sharing ... ...
Packet capture
Packet classification hash
Thread 1
Packet decoder
Preprocessors
Detection engine
Output plug-in
Packet decoder
Preprocessors
Detection engine
Output plug-in
Thread 2 Thread 4
![Page 7: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/7.jpg)
7
MultiSnort
Derek L. Schuff, Purdue University.
With memory sharing Not a clean-cut
modular structure
Minimal Decode
Queue assignment
Full Decode
Preprocessors
Detection engine
Output Module
Packet Capture
Distributed task queues
Full Decode
Preprocessors
Detection engine
Shared Data
... ...
![Page 8: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/8.jpg)
8
Our design – ParaSnort
Based on SnortSP 3.0, a new different branch Modular design Multifunction
processing modules Memory sharing Optimization on core
algorithms Sufficient speedup
Load Balance Module
Output Module
Data Source Module
Packet Queue
Processing ModuleShared Data
Packet Queue
Processing Module
Core 1
Core 2 Core 8
... ...
![Page 9: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/9.jpg)
9
Detailed module design
Data Source data acquisition and decoder
Load Balance dispatches traffic and makes multi-staged processing
Processing Module each is a single thread preprocessors and detection engine easy to develop functions other than intrusion
detection, such as antivirus or URL filtering Output module
Generate alert
![Page 10: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/10.jpg)
10
Optimize Load Balancing
SnortSP 3.0 provides IP hash algorithm Not so balance when there are few flows Three improve methods:
Dst IP
Src port
Src IP
Dst port
protocol
Hash Processing Module
ID
5-tuple hash Join the Shortest Queue
Modified-JSQ Reassign a
flow when it has silenced for a long time
1
23 4
Packet
![Page 11: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/11.jpg)
11
Optimize Multi-pattern Matching
SnortSP 3.0 provides AC algorithm AC works fast, and when there are few matches, the
cache locality is high. But when there are many matches in the traffic, the
cache locality turns bad. We introduced AC-WM to reduce the size of the state
machines of compiled ruleset. While costs much less memory, AC-WM is a bit slower
than AC for ordinary traffics, so users can decide which to use according to their network environment.
![Page 12: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/12.jpg)
12
Para-Snort Performance
![Page 13: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/13.jpg)
13
The Setup
NIDS platform
Testing machine
TCPreplay
eth0
Para-Snort
Testing ServerTesting Server
TestingClientTestingClient
Testing Server
NIDS platform
TestingClient
Para-Snort
eth0 eth1Forwarding
For tcpdump traces For real traffic
two quad-core Xeon E5335 at 2.00GHz4 GB DRAMUbuntu 8.04Linux kernel version 2.6.27
![Page 14: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/14.jpg)
14
![Page 15: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/15.jpg)
15
Performance of 400~800Mbps
1 2 3 4 5 6 70
100
200
300
400
500
600
700
800
900
Processing Engine Threads
Pro
ce
ss
ing
Sp
ee
d (
Mb
ps
)
LL1
LL2CERNET
http
![Page 16: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/16.jpg)
16
Speedup of 4~7, almost linear for LL
1 2 3 4 5 6 71
2
3
4
5
6
7
Processing Engine Threads
Sp
ee
du
p
LL1
LL2CERNET
http
![Page 17: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/17.jpg)
17
Performance of different load balancers
![Page 18: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/18.jpg)
18
Performance of Different Pattern Matching
![Page 19: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/19.jpg)
19
Performance Summary
Good speedup, up to 7. Performance up to 800Mbps
M-JSQ is fastest AC-WM costs less memory, but slower
![Page 20: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/20.jpg)
20
Conclusions
Multi-thread design fully utilizes multi-core CPU
Modular design, multifunction process modules, easy to add modules.
Solve the issues in load balancing and multi-pattern matching
Can be NIPS if inline data source module added.
![Page 21: Para-Snort : A Multi-thread Snort on Multi-Core IA Platform Tsinghua University PDCS 2009 November 3, 2009 Xinming Chen, Yiyao Wu, Lianghong Xu, Yibo Xue](https://reader030.vdocuments.us/reader030/viewer/2022020111/56649e935503460f94b98e18/html5/thumbnails/21.jpg)
21
Questions
Thank You