![Page 1: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/1.jpg)
Software-Defined Traffic Measurement with OpenSketch
Lavanya JoseStanford University
Joint work with Minlan Yu and Rui Miao at USC1
1Friday, April 5, 13
![Page 2: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/2.jpg)
measure
control
Management is Control + Measurement
2
- Access Control
- DDoS- Flow Size Distribution
- Routing
2Friday, April 5, 13
![Page 3: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/3.jpg)
Questions we want to ask
3
1. Who’s sending a lot to 10.0.2.0/16? (Heavy Hitters)2. How are flow sizes distributed?3. Is someone doing a port scan?4. Is someone being DDoS-ed?5. Who’s getting traffic from blacklisted IPs?6. How many people downloaded files from 10.0.2.1?
3Friday, April 5, 13
![Page 4: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/4.jpg)
Switches are great at counting per flow bytes and packets
4
• NetFlow and sFlow sample packets
• NetFlow maintains per flow byte and packet counts
• Can find count of a particular flow, prefix or counts of heavy flows
4Friday, April 5, 13
![Page 5: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/5.jpg)
Problem: NetFlow counts can’t answer my questions
5
Is someone doing a port scan?NetFlow samples packets from heavy flows. Missed packets from small “port scanners”.- Increase sampling rate --> inefficient
5Friday, April 5, 13
![Page 6: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/6.jpg)
Streaming algorithms
6
+ Process efficiently at line rate+ Accurate answers- But each answers a specific question
6Friday, April 5, 13
![Page 7: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/7.jpg)
7
What measurement architecture can answer all my questions?
1. Who’s sending a lot to 10.0.2.0/16? (Heavy Hitters)2. How are flow sizes distributed?3. Is someone doing a port scan?4. Is someone being DDoS-ed?5. Who’s getting traffic from blacklisted IPs?6. How many people downloaded files from 10.0.2.1?
7Friday, April 5, 13
![Page 8: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/8.jpg)
SDN Model: Find Building Blocks
8
? ? ?
1. Who’s sending a lot to 10.0.2.0/16? (Heavy Hitters)2. How are flow sizes distributed?3. Is someone doing a port scan? ...
8Friday, April 5, 13
![Page 9: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/9.jpg)
Sketches as building blocks
9
• Sketch
• Data structure
• Support approx. computing some function of data
• Much smaller than actual data
• Streaming, small per-item processing cost
• Provable space-accuracy tradeoffs9Friday, April 5, 13
![Page 10: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/10.jpg)
Sketches as building blocks
10
3 2 1 23 0 4h1h2h3
22 4 9 3 2 1
2 3 0 4 22 5
process
Packet
e.g., Count Min sketchto store counts of frequent source IP addresses
(Cormode 2005)
10Friday, April 5, 13
![Page 11: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/11.jpg)
Sketches as building blocks
10
3 2 1 23 0 4h1h2h3
22 4 9 3 2 1
2 3 0 4 22 5
process
Source IPaddress : 23.43.12.1
e.g., Count Min sketchto store counts of frequent source IP addresses
(Cormode 2005)
10Friday, April 5, 13
![Page 12: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/12.jpg)
Sketches as building blocks
10
3 2 1 23 0 4h1h2h3
22 4 9 3 2 1
2 3 0 4 22 5
process
Source IPaddress : 23.43.12.1
e.g., Count Min sketchto store counts of frequent source IP addresses
(Cormode 2005)
10Friday, April 5, 13
![Page 13: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/13.jpg)
Sketches as building blocks
10
3 2 1 23 0 4h1h2h3
22 4 9 3 2 1
2 3 0 4 22 5
24
process
Source IPaddress : 23.43.12.1
e.g., Count Min sketchto store counts of frequent source IP addresses
(Cormode 2005)
10Friday, April 5, 13
![Page 14: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/14.jpg)
Sketches as building blocks
10
3 2 1 23 0 4h1h2h3
22 4 9 3 2 1
2 3 0 4 22 5
24
23
23
process
Source IPaddress : 23.43.12.1
e.g., Count Min sketchto store counts of frequent source IP addresses
(Cormode 2005)
10Friday, April 5, 13
![Page 15: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/15.jpg)
Sketches as building blockse.g., Count Min sketch
to store counts of frequent source IP addresses
11
3 2 1 24 0 4
23 4 9 3 2 1
2 3 0 4 23 5query
h1h2h3
(Cormode 2005)
11Friday, April 5, 13
![Page 16: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/16.jpg)
Sketches as building blockse.g., Count Min sketch
to store counts of frequent source IP addresses
11
3 2 1 24 0 4# packets from 23.43.12.1? 23 4 9 3 2 1
2 3 0 4 23 5query
h1h2h3
(Cormode 2005)
11Friday, April 5, 13
![Page 17: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/17.jpg)
Sketches as building blockse.g., Count Min sketch
to store counts of frequent source IP addresses
11
3 2 1 24 0 4# packets from 23.43.12.1? 23 4 9 3 2 1
2 3 0 4 23 5
24 23 23query
estimate pick min.
h1h2h3
(Cormode 2005)
11Friday, April 5, 13
![Page 18: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/18.jpg)
Sketches as building blockse.g., Count Min sketch
to store counts of frequent source IP addresses
11
3 2 1 24 0 4# packets from 23.43.12.1? 23 4 9 3 2 1
2 3 0 4 23 5
23query
estimate pick min.
h1h2h3
(Cormode 2005)
11Friday, April 5, 13
![Page 19: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/19.jpg)
Sketches as building blockse.g., Count Min sketch
to store counts of frequent source IP addresses
12
23estimatepick min.
+ Provable space-accuracy tradeoffs
(Cormode 2005)
12Friday, April 5, 13
![Page 20: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/20.jpg)
13
Identifyingheavy “keys”
Counting, storing
statistics
Picking packetsto measure
Sketches as building blocks
(Reversible Sketch: Schweller 2004)
13Friday, April 5, 13
![Page 21: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/21.jpg)
1. Who’s sending a lot to 10.0.2.0/16? (Heavy Hitters)2. How are flow sizes distributed?3. Is someone doing a port scan?4. Is someone being DDoS-ed?5. Who’s getting traffic from blacklisted IPs?6. How many people downloaded files from 10.0.2.1?
14
...answer many questions
Identifyingheavy “keys”
Counting, storing
statistics
a lotWho’s
(Reversible Sketch: Schweller 2004)
14Friday, April 5, 13
![Page 22: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/22.jpg)
estimatequeryprocess
- frequency count
estimatequeryprocess
- cardinality
But each sketch estimatesonly one function
15
estimatequeryprocess
- set membership
estimatequeryprocess
- heavy “keys”
15Friday, April 5, 13
![Page 23: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/23.jpg)
16
3-stage pipeline
estimatequery
- frequency count
estimatequery
- cardinality
estimatequery
- set membership
estimatequery
- heavy “keys”
processprocessprocessprocess
16Friday, April 5, 13
![Page 24: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/24.jpg)
16
3-stage pipeline
Hash Classify Count
estimatequery
- frequency count
estimatequery
- cardinality
estimatequery
- set membership
estimatequery
- heavy “keys”
16Friday, April 5, 13
![Page 25: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/25.jpg)
1717
3-stage pipeline
Hash Classify CountPacket
hash values
compute counter
addresses
hash values
header fields
header fields
pick fields to hash
pick field to match
header fields
17Friday, April 5, 13
![Page 26: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/26.jpg)
18
3-stage pipeline
Hash Classify Count
estimatequery
- frequency count
estimatequery
- cardinality
estimatequery
- set membership
estimatequery
- heavy “keys”
18Friday, April 5, 13
![Page 27: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/27.jpg)
18
3-stage pipeline
Identifyingheavy “keys”
Counting, storing
statistics
Picking packetsto measure
Hash Classify Count
18Friday, April 5, 13
![Page 28: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/28.jpg)
18
3-stage pipeline
Identifyingheavy “keys”
Counting, storing
statistics
Picking packetsto measure
Hash Classify Count
1. Who’s sending a lot to 10.0.2.0/16? (Heavy Hitters)2. How are flow sizes distributed?3. Is someone doing a port scan?
18Friday, April 5, 13
![Page 29: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/29.jpg)
1. Who’s sending a lot to 10.0.2.0/16? (Heavy Hitters)
2. How are flow sizes distributed?
3. Is ..
19
OpenSketch Measurement Framework
Identifyingheavy “keys”
Counting, storing
statistics
Picking packetsto measure
Hash Classify Count
Controller
Data Plane
19Friday, April 5, 13
![Page 30: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/30.jpg)
1. Who’s sending a lot to 10.0.2.0/16? (Heavy Hitters)
2. How are flow sizes distributed?
3. Is ..
19
OpenSketch Measurement Framework
Identifyingheavy “keys”
Counting, storing
statistics
Picking packetsto measure
Hash Classify Count
Measurement Library
Controller
Data Plane
19Friday, April 5, 13
![Page 31: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/31.jpg)
1. Who’s sending a lot to 10.0.2.0/16? (Heavy Hitters)
2. How are flow sizes distributed?
3. Is ..
19
OpenSketch Measurement Framework
Identifyingheavy “keys”
Counting, storing
statistics
Picking packetsto measure
Hash Classify Count
Measurement Library
Controller
Data Plane
Measurement Programs
19Friday, April 5, 13
![Page 32: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/32.jpg)
1. Who’s sending a lot to 10.0.2.0/16? (Heavy Hitters)
2. How are flow sizes distributed?
3. Is ..
19
OpenSketch Measurement Framework
Identifyingheavy “keys”
Counting, storing
statistics
Picking packetsto measure
Hash Classify Count
Measurement Library
Controller
Data Plane
Measurement Programs
#, field, range
(match, action)
# counters, size,update type,
addr. calculation
19Friday, April 5, 13
![Page 33: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/33.jpg)
1. Who’s sending a lot to 10.0.2.0/16? (Heavy Hitters)
2. How are flow sizes distributed?
3. Is ..
19
OpenSketch Measurement Framework
Identifyingheavy “keys”
Counting, storing
statistics
Picking packetsto measure
Hash Classify Count
Measurement Library
Controller
Data Plane
Measurement Programs
19Friday, April 5, 13
![Page 34: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/34.jpg)
Details
• Implementing sketches with the Pipeline
• Configuring the Pipeline
• Evaluation and NetFPGA prototype
20
20Friday, April 5, 13
![Page 35: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/35.jpg)
Count Min Sketch with the Pipelineto store counts of frequent source IP addresses
21
3 2 1 23 0 4h1h2h3
22 4 9 3 2 1
2 3 0 4 22 5
24
23
23
process
Counting, storing
statistics
Hash Count
Source IPaddress : 23.43.12.1
21Friday, April 5, 13
![Page 36: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/36.jpg)
Bitmap Sketch with the Pipelineto store number of different destination port numbers
22
h 0 0 1 0 0 1 0 0 1 0
process
Counting, storing
statistics
Hash Count
Packet
22Friday, April 5, 13
![Page 37: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/37.jpg)
Bitmap Sketch with the Pipelineto store number of different destination port numbers
22
h 0 0 1 0 0 1 0 0 1 0
process
Counting, storing
statistics
Hash Count
Destination portnumber : 5596
22Friday, April 5, 13
![Page 38: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/38.jpg)
Bitmap Sketch with the Pipelineto store number of different destination port numbers
22
h 0 0 1 0 0 1 0 0 1 01
process
Counting, storing
statistics
Hash Count
Destination portnumber : 5596
22Friday, April 5, 13
![Page 39: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/39.jpg)
Bitmap Sketch with the Pipelineto store number of different destination port numbers
23
1 0 1 0 0 1 0 0 1 0
Counting, storing
statistics
# different destination portnumbers?
query
(Whang 1990)
23Friday, April 5, 13
![Page 40: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/40.jpg)
Bitmap Sketch with the Pipelineto store number of different destination port numbers
23
1 0 1 0 0 1 0 0 1 0
Counting, storing
statistics
# different destination portnumbers?
6/10query
estimateestimate N = -10 ln(6/10)= 5
(Whang 1990)Six counters out often are 0.
23Friday, April 5, 13
![Page 41: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/41.jpg)
Other Sketches
•K-ary Sketch for heavy changes
•Bloom Filter Sketch to check set membership
•PCSA sketch to count distinct values
24
(Schweller 2004; Goel 2010; Flajolet 1985)
24Friday, April 5, 13
![Page 42: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/42.jpg)
Efficient implementation of 3-stage pipeline
25
Hash Classify Count
hashin parallel TCAM rules
cheap fast memoryMBs of SRAM
25Friday, April 5, 13
![Page 43: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/43.jpg)
Similar functions, diverse configurations
26
Hash Classify Count
?? hash functions
?? TCAM entries for
classify rules
?? MBs of SRAM
for tables of counters
26Friday, April 5, 13
![Page 44: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/44.jpg)
Similar functions, diverse configurations
27
Hash
4-8 simple hash functions per
question
- Count Min: 3
- Bloom Filters: 7-8
- Fixed size reversible sketch: 5 - Can share hash functions
27Friday, April 5, 13
![Page 45: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/45.jpg)
Similar functions, diverse configurations
28
Classify
30-40 TCAM entries per question maximum
- Match a prefix/ value: 1 rule
- Match a set of values: Bloom Filters
28Friday, April 5, 13
![Page 46: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/46.jpg)
Similar functions, diverse configurations
29
Count
up to 8MB SRAM
From simulation and worst case bounds for different tasks
29Friday, April 5, 13
![Page 47: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/47.jpg)
rela
tive
err
or (
%)
SRAM (MB)
Similar functions, diverse configurations
29
Count
up to 8MB SRAM 2MB for 1.7% err
Change detection
29Friday, April 5, 13
![Page 48: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/48.jpg)
rela
tive
err
or (
%)
SRAM (MB)
Similar functions, diverse configurations
29
Count
up to 8MB SRAM
7.5MB for 0.5% err
Change detection
29Friday, April 5, 13
![Page 49: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/49.jpg)
Measurement tasks
30
(Heavy Hitters: Cormode 2005; Flow Size Distribution: Kumar 2004; Change detection: Schweller 2004; DDoS detection: Venkataraman 2005)
1. Who’s sending a lot to 10.0.2.0/16? (Heavy Hitters)2. How are flow sizes distributed?3. Is someone doing a port scan?4. Is someone being DDoS-ed?5. Who’s getting traffic from blacklisted IPs?6. How many people downloaded files from 10.0.2.1?
30Friday, April 5, 13
![Page 50: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/50.jpg)
More efficient than NetFlow (Heavy Hitters)
0 2 4 6 8
10 12 14
200 400 600 800 1000 1200
False
pos
. (%
)
Switch memory size (KB)
NetFlow-posOpenSketch-pos
31
Efficient- needs 1/4th as much memory as
NetFlow for 4% f.p.
31Friday, April 5, 13
![Page 51: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/51.jpg)
More efficient than NetFlow (Heavy Hitters)
Accurate- with 600KB, OpenSketch has less than 0.05% f.p. NetFlow has around 3%
0 2 4 6 8
10 12 14
200 400 600 800 1000 1200
False
pos
. (%
)
Switch memory size (KB)
NetFlow-posOpenSketch-pos
31
31Friday, April 5, 13
![Page 52: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/52.jpg)
OpenSketch NetFPGA Prototype
32
• 3-stage meas. pipeline parallel to forwarding
• Full throughput 1Gbps @ 4 ports
• Measurement pipeline takes fewer cycles than forwarding
32Friday, April 5, 13
![Page 53: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/53.jpg)
Conclusion
33
• Current switches good for flow statistics
• But they don’t answer basic measurement questions
• Like identify heavy hitters, detect DDoS attacks, port scans, traffic from blacklisted IP address etc.
33Friday, April 5, 13
![Page 54: Software Defined Tra c Measurement with OpenSketch · Software-Defined Tra!c Measurement with OpenSketch Lavanya Jose Stanford University Joint work with Minlan Yu and Rui Miao](https://reader031.vdocuments.us/reader031/viewer/2022022015/5b5cce297f8b9ad21d8cecfe/html5/thumbnails/54.jpg)
Takeaway
34
• Hash, classify and count pipeline in the Data Plane
• And sketch based building blocks in the Control Plane
• Make measurement in switches efficient and easy
34Friday, April 5, 13