![Page 1: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/1.jpg)
Suricata and XDP, Performance with a S like Security
É. Leblond
OISF
Nov. 29, 2018
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 1 / 43
![Page 2: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/2.jpg)
1 IntroductionSuricata 101Suricata on live traffic
2 ProblemReconstruction workPacket loss impactElephant flowWork less to get more
3 Suricata BypassIntroducing bypassBypass strategy
4 Extended Berkeley Packet Filter5 AF_PACKET bypass
eBPF bypassXDP support
6 Conclusion
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 1 / 43
![Page 3: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/3.jpg)
Who am I?
Éric LeblondStamus Networks co-founder
Editor of a threat hunting solutionincluding Suricata based appliances
Netfilter core team memberReally low personal activity nowadays
[email protected] time member of OISFSuricata developer
In charge of packet acquisition1118 commits since 2010 (I like small patches)
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 2 / 43
![Page 4: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/4.jpg)
About the journey
Adding bypass feature to Suricata2 years of development to see less and get more doneUsing kick ass technologies before their documentation has been written.
Figure: Summary of talk objectives
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 3 / 43
![Page 5: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/5.jpg)
1 IntroductionSuricata 101Suricata on live traffic
2 ProblemReconstruction workPacket loss impactElephant flowWork less to get more
3 Suricata BypassIntroducing bypassBypass strategy
4 Extended Berkeley Packet Filter5 AF_PACKET bypass
eBPF bypassXDP support
6 Conclusion
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 3 / 43
![Page 6: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/6.jpg)
What it is not ?
https://twitter.com/randomuserid/status/1012474246503845888É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 4 / 43
![Page 7: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/7.jpg)
A signature based IDS
From individual datagram to detectionGet packet per packetReconstruct to application layerRun detection engine
Key pointsGPLv2Owned by OISF foundation10 years oldScalability via multithreadingWritten in C and Rust
Example signature
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 5 / 43
![Page 8: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/8.jpg)
Suricata NSM features
Supported protocolsProtocol analysis: http, ftp, smtp, tls, ssh smb, dcerpc, dns, nfs, ntp, ftp-data, tftp,ikev2, krb5, dhcpProtocol recognition: imap, msn
Log example
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 6 / 43
![Page 9: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/9.jpg)
What it is ? or how to please developers
https://twitter.com/randomuserid/status/1012705279098490880
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 7 / 43
![Page 10: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/10.jpg)
File related features
File analysisMagic computation and in file data matchChecksum computation and file extraction to diskSupported protocols: http, smtp, smb, ftp, nfs
Fileinfo example
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 8 / 43
![Page 11: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/11.jpg)
1 IntroductionSuricata 101Suricata on live traffic
2 ProblemReconstruction workPacket loss impactElephant flowWork less to get more
3 Suricata BypassIntroducing bypassBypass strategy
4 Extended Berkeley Packet Filter5 AF_PACKET bypass
eBPF bypassXDP support
6 Conclusion
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 8 / 43
![Page 12: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/12.jpg)
Suricata live modes
Intrusion Detection SystemAF_PACKET capture method under LinuxGet raw packet from cardDo complete analysis
Intrusion Prevention System
Netfilter with NFQUEUE on LinuxKernel ask userspace for decision onpackets
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 9 / 43
![Page 13: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/13.jpg)
1 IntroductionSuricata 101Suricata on live traffic
2 ProblemReconstruction workPacket loss impactElephant flowWork less to get more
3 Suricata BypassIntroducing bypassBypass strategy
4 Extended Berkeley Packet Filter5 AF_PACKET bypass
eBPF bypassXDP support
6 Conclusion
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 9 / 43
![Page 14: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/14.jpg)
1 IntroductionSuricata 101Suricata on live traffic
2 ProblemReconstruction workPacket loss impactElephant flowWork less to get more
3 Suricata BypassIntroducing bypassBypass strategy
4 Extended Berkeley Packet Filter5 AF_PACKET bypass
eBPF bypassXDP support
6 Conclusion
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 9 / 43
![Page 15: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/15.jpg)
Suricata reconstruction and normalization
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 10 / 43
![Page 16: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/16.jpg)
1 IntroductionSuricata 101Suricata on live traffic
2 ProblemReconstruction workPacket loss impactElephant flowWork less to get more
3 Suricata BypassIntroducing bypassBypass strategy
4 Extended Berkeley Packet Filter5 AF_PACKET bypass
eBPF bypassXDP support
6 Conclusion
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 10 / 43
![Page 17: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/17.jpg)
Impact of loosing packets
MethodologyUse a sample trafficModify the pcap file to have specified random packet lossDo it 3 times par packet lossGet graph out of that
Test dataUsing a test pcap of 445Mo.Real traffic but lot of malicious behaviorsTraffic is a bit old
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 11 / 43
![Page 18: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/18.jpg)
Alert loss by packet loss
Some numbers10% missed alerts with 3% packets loss50% missed alerts with 25% packets loss
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 12 / 43
![Page 19: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/19.jpg)
The case of file extraction
Some numbers10% failed file extraction with 0.4% packets loss50% failed file extraction with 5.5% packets loss
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 13 / 43
![Page 20: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/20.jpg)
1 IntroductionSuricata 101Suricata on live traffic
2 ProblemReconstruction workPacket loss impactElephant flowWork less to get more
3 Suricata BypassIntroducing bypassBypass strategy
4 Extended Berkeley Packet Filter5 AF_PACKET bypass
eBPF bypassXDP support
6 Conclusion
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 13 / 43
![Page 21: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/21.jpg)
The elephant flow problem (1/2)
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 14 / 43
![Page 22: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/22.jpg)
The elephant flow problem (2/2)
Ring buffer overrunLimited sized ring bufferOverrun cause packets lossthat cause streaming malfunction
Ring size increaseWork aroundUse memoryFail for non burst
Dequeue at NQueue at speed N+M
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 15 / 43
![Page 23: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/23.jpg)
1 IntroductionSuricata 101Suricata on live traffic
2 ProblemReconstruction workPacket loss impactElephant flowWork less to get more
3 Suricata BypassIntroducing bypassBypass strategy
4 Extended Berkeley Packet Filter5 AF_PACKET bypass
eBPF bypassXDP support
6 Conclusion
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 15 / 43
![Page 24: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/24.jpg)
Stream depth method
Attacks characteristicIn most cases attack is done at start of TCP sessionGeneration of requests prior to attack is not commonMultiple requests are often not even possible on same TCP session
Stream reassembly depthReassembly is done till stream.reassembly.depth bytes.Stream is not analyzed once limit is reachedIndividual packet continue to be inspected
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 16 / 43
![Page 25: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/25.jpg)
1 IntroductionSuricata 101Suricata on live traffic
2 ProblemReconstruction workPacket loss impactElephant flowWork less to get more
3 Suricata BypassIntroducing bypassBypass strategy
4 Extended Berkeley Packet Filter5 AF_PACKET bypass
eBPF bypassXDP support
6 Conclusion
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 16 / 43
![Page 26: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/26.jpg)
1 IntroductionSuricata 101Suricata on live traffic
2 ProblemReconstruction workPacket loss impactElephant flowWork less to get more
3 Suricata BypassIntroducing bypassBypass strategy
4 Extended Berkeley Packet Filter5 AF_PACKET bypass
eBPF bypassXDP support
6 Conclusion
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 16 / 43
![Page 27: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/27.jpg)
Introducing bypass
Stop packet handling as soon as possibleTag flow as bypassedMaintain table of bypassed flowsDiscard packet if part of a bypassed flow
Bypass methodLocal bypass: Suricata discard packet after decodingCapture bypass: capture method maintain flow table and discard packets ofbypassed flows
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 17 / 43
![Page 28: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/28.jpg)
Bypassing big flow: local bypass
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 18 / 43
![Page 29: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/29.jpg)
Bypassing big flow: capture bypass
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 19 / 43
![Page 30: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/30.jpg)
Implementation
Suricata updateAdd callback functionCapture method register itself and provide a callbackSuricata calls callback when it wants to offload
NFQ bypass in Suricata 3.2Update capture register functionWritten callback function
Set a mark with respect to a mask on packetMark is set on packet when issuing the verdict
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 20 / 43
![Page 31: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/31.jpg)
Suricata NFQ and bypass
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 21 / 43
![Page 32: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/32.jpg)
1 IntroductionSuricata 101Suricata on live traffic
2 ProblemReconstruction workPacket loss impactElephant flowWork less to get more
3 Suricata BypassIntroducing bypassBypass strategy
4 Extended Berkeley Packet Filter5 AF_PACKET bypass
eBPF bypassXDP support
6 Conclusion
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 21 / 43
![Page 33: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/33.jpg)
Stream depth bypass
Stop all treatment after bypassGo beyond what is currently doneDisable individual packet treatment once stream depth is reached
Activating stream depth bypassSet stream.bypass to yes in YAML
TLS bypassencrypt-handling: bypass
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 22 / 43
![Page 34: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/34.jpg)
Selective bypass
Ignore some trafficIgnore intensive traffic like NetflixCan be done independently of stream depthCan be done using generic or custom signatures
The bypass keywordA new bypass signature keywordTrigger bypass when signature matchExample of signature
pass h t t p any any −> any any ( content : " s u r i c a t a . i o " ; \ \h t tp_hos t ; bypass ; s id :6666; rev : 1 ; )
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 23 / 43
![Page 35: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/35.jpg)
And now AF_PACKET bypass
What’s neededSuricata to tell kernel to ignore flowsKernel system able to
Maintain a list of flow entriesDiscard packets belonging to flows in the listUpdate from userspace
eBPF filter using mapseBPF introduce mapsDifferent data structures
Hash, array, . . .Update and fetch from userspace
Looks good!
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 24 / 43
![Page 36: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/36.jpg)
1 IntroductionSuricata 101Suricata on live traffic
2 ProblemReconstruction workPacket loss impactElephant flowWork less to get more
3 Suricata BypassIntroducing bypassBypass strategy
4 Extended Berkeley Packet Filter5 AF_PACKET bypass
eBPF bypassXDP support
6 Conclusion
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 24 / 43
![Page 37: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/37.jpg)
Extended Berkeley Packet Filter
Berkeley Packet FilterVirtual machine inside kernelArithmetic operations and tests on the packet dataFilters are injected by userspace in kernel via syscall
Extended BPFExtended virtual machine: more operators, data and function accessVarious attachment points
SocketSyscallTraffic control
Kernel and userspace shared structuresHash tablesArrays
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 25 / 43
![Page 38: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/38.jpg)
LLVM backend
From C file to eBPF codeWrite C codeUse eBPF LLVM backend (since LLVM 3.7)Use libbpf
Get ELF fileExtract and load section in kernel
BCC: BPF Compiler collectionInject eBPF into kernel from high level scripting languageTrace syscalls and kernel functionshttps://github.com/iovisor/bcc
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 26 / 43
![Page 39: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/39.jpg)
BCC tracing tools
http://www.brendangregg.com/ebpf.htmlÉ. Leblond (OISF) Suricata and XDP Nov. 29, 2018 27 / 43
![Page 40: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/40.jpg)
eBPF applied to security
AdvantagesReally extensibleKernel version independant when not intercepting functionsExtract info all system stacks
Host security monitoring at NetflixLinux Monitoring at Scale with eBPF (Brendan Gregg & Alex Maestretti)https://youtu.be/44nV6Mj11uw
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 28 / 43
![Page 41: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/41.jpg)
Example: BCC socket bind 1/2
Detect network serversGet all bind call to detect servicesOutput result to console
A BCC scriptPython codeeBPF code as C in a string
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 29 / 43
![Page 42: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/42.jpg)
Example: BCC socket bind 2/2
DemoStart sobindStart a nc command to listen to port 2233
Outputsudo python ./sobindPID COMM PROTO PORT ADDR9565 nc TCPv4 2233 0.0.0.09572 nc TCPv4 2233 127.0.0.2
Key featuresDirect hook in the system callNo /proc scanning but polling of resultsGet it there:https://gist.github.com/regit/1e591311fa3ba5cd0b8d73940348599a
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 30 / 43
![Page 43: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/43.jpg)
1 IntroductionSuricata 101Suricata on live traffic
2 ProblemReconstruction workPacket loss impactElephant flowWork less to get more
3 Suricata BypassIntroducing bypassBypass strategy
4 Extended Berkeley Packet Filter5 AF_PACKET bypass
eBPF bypassXDP support
6 Conclusion
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 30 / 43
![Page 44: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/44.jpg)
1 IntroductionSuricata 101Suricata on live traffic
2 ProblemReconstruction workPacket loss impactElephant flowWork less to get more
3 Suricata BypassIntroducing bypassBypass strategy
4 Extended Berkeley Packet Filter5 AF_PACKET bypass
eBPF bypassXDP support
6 Conclusion
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 30 / 43
![Page 45: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/45.jpg)
Suricata eBPF bypass architecture
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 31 / 43
![Page 46: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/46.jpg)
Test methodology
Test setupIntel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHzIntel Corporation 82599ES 10-Gigabit SFI/SFP+Live traffic:
Around 1Gbps to 2GbpsReal users so not reproducible
TestsOne hour long runDifferent stream depth valuesCollected Suricata statistics counters (JSON export)Graphs done via Timelion(https://www.elastic.co/blog/timelion-timeline)
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 32 / 43
![Page 47: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/47.jpg)
Results: stream bypass at 512kb
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 33 / 43
![Page 48: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/48.jpg)
A few words on graphics
Tests at 512kbWe have on big flow that kill thebandwidthCapture get almost nullEven number of closed bypassedflows is low
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 34 / 43
![Page 49: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/49.jpg)
Results
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 35 / 43
![Page 50: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/50.jpg)
1 IntroductionSuricata 101Suricata on live traffic
2 ProblemReconstruction workPacket loss impactElephant flowWork less to get more
3 Suricata BypassIntroducing bypassBypass strategy
4 Extended Berkeley Packet Filter5 AF_PACKET bypass
eBPF bypassXDP support
6 Conclusion
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 35 / 43
![Page 51: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/51.jpg)
A Linux kernel feature
Run a eBPF code the earliest possiblein the driverin the cardbefore the regular kernel path
Act on dataDrop packet (eXtreme Drop Performance)Transmit to kernelRewrite and transmit packet to kernelRedirect to another interfaceCPU load balance
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 36 / 43
![Page 52: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/52.jpg)
Implementation
Similar to eBPF filterSame logic for bypassOnly verdict logic is different
But annoying differenceeBPF code does the parsingNeed to bind to an interface
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 37 / 43
![Page 53: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/53.jpg)
Suricata XDP architecture
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 38 / 43
![Page 54: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/54.jpg)
AF_PACKET IPS mode
IPS and bypassCan’t drop packet at bypass stageNeed to forward from one iface to another
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 39 / 43
![Page 55: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/55.jpg)
XDP and IPS mode: bypass
Use TX_REDIRECTDirect copy from interface to interface
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 40 / 43
![Page 56: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/56.jpg)
Results
Direct NIC to NIC transferSkip all kernel taskWire speed copyIf eBPF code is fast enough
Obtained performanceTODO: Ask OISF marketing for some fake numbers to show
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 41 / 43
![Page 57: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/57.jpg)
1 IntroductionSuricata 101Suricata on live traffic
2 ProblemReconstruction workPacket loss impactElephant flowWork less to get more
3 Suricata BypassIntroducing bypassBypass strategy
4 Extended Berkeley Packet Filter5 AF_PACKET bypass
eBPF bypassXDP support
6 Conclusion
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 41 / 43
![Page 58: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/58.jpg)
Conclusion
Suricata, eBPF and XDPAvailable in Suricata 4.1, need Linux 4.16Network card bypass for Netronome comingAF_XDP capture is now in Linux vanilla
More informationStamus Networks: https://www.stamus-networks.com/Septun II: https://github.com/pevma/SEPTun-Mark-II/Suricata doc: http://suricata.readthedocs.io/en/latest/capture-hardware/ebpf-xdp.html
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 42 / 43
![Page 59: Suricata and XDP, Performance with a S like Security · Kernel ask userspace for decision on packets É. Leblond (OISF) Suricata and XDP Nov. 29, 20189/43. 1 Introduction Suricata](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec58621fbd9db361d0dde6e/html5/thumbnails/59.jpg)
Questions ?
Thanks toJesper Dangaard BrouerAlexei StarovoitovDaniel Borkmann
Contact [email protected]: @regiteric
Want more fun ?Come to Suricata trainings: https://suricata-ids.org/training/
Suricon: https://suricon.net/
É. Leblond (OISF) Suricata and XDP Nov. 29, 2018 43 / 43