automatic test packet generation
TRANSCRIPT
![Page 1: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/1.jpg)
Automatic Test Packet
Generation
Tushar Jadhav
![Page 2: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/2.jpg)
Contents
Introduction
Problem Statement
Concept
Conclusion
References
![Page 3: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/3.jpg)
Introduction
![Page 4: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/4.jpg)
Outages in world
![Page 5: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/5.jpg)
Network Troubleshooting
Downtime:
25%take over an hour to resolve. (estimated $60K-110K/hour ).
Current tools:
Ping, Traceroute, SNMP.
70%asked for better tools, automatic tests
![Page 6: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/6.jpg)
Automatic Test Packet Generation
(ATPG) generates packets :
- To test the liveness of the underlying
topology .
- To test the compatibility between
data plane state and configuration
specifications.
- To test performance assertions such
as packet latency.
![Page 7: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/7.jpg)
Problem Definition
Most common causes of network
failure
1) Hardware failures
2) Software bugs
Others failure’s include reachability
failures and throughput/latency
degradation.
Our goal is to automatically detect
these types of failures.
![Page 8: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/8.jpg)
Goals
Automatically generate test packets to
test the network state, faults before
being
noticed by application.
Augment human wisdom and intuition.
Reduce the downtime.
Save money.
![Page 9: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/9.jpg)
Concept
ATPG System :
![Page 10: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/10.jpg)
Roadmap
Reachability Analysis
Test Packet Generation & Selection
Fault Localization
Implementation & Evaluation
![Page 11: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/11.jpg)
Reachability Analysis
Header Space Analysis
All-pairs reachability : Compute all
classes of packets that can flow
between every pair of ports.
![Page 12: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/12.jpg)
All-pairs reachability
![Page 13: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/13.jpg)
Test Packet Generation &
Selection
All-pairs reachability
![Page 14: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/14.jpg)
Test Packet Selection
Packets in all-pairs reachability table
are more than necessary.
Goal :
Select a minimum subset of packets
whose histories cover the whole rule
set.
![Page 15: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/15.jpg)
Min Set Cover
![Page 16: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/16.jpg)
Test Packet Selection
Min Set Cover
-Exercise all rules
-Sent out
Periodically
- “Redundant”
- Will be used in
Fault Localization
![Page 17: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/17.jpg)
Fault Localization
![Page 18: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/18.jpg)
Fault Localization
In ATPG: we can choose packets!
Step 1:
Use results from regular test packets– F (potentially broken rules) = Union from all failing packets
– P (known good rules) = Union from all passing packets
– Suspect Set = F –P
![Page 19: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/19.jpg)
Step 2 :
Use reserved test packets
- Pick packets that test only one rule in the suspect set, and send them out for testing
- Passed: eliminate
- Failed: label it as “broken”
Step 3:
Brute force
- Continue with test packets that test two or more rules in the suspect set, until the set is small enough
![Page 20: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/20.jpg)
Implementation
Putting Them all together :
![Page 21: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/21.jpg)
Cisco/Juniper Parsers
-Translate router configuration files and forwarding tables (FIB) into Header space representation.
Test Packet Generation/Selection
- Hassel: A python header space library
- Min-Set-Cover
- Python’s multiprocess module to parallelize
SDN can simplify the design
![Page 22: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/22.jpg)
ATPG for Performance
Testing
ATPG can also be used for detecting and localizing performance problems.
Intuition: generalize results of a test from success/failure to performance (e.g. latency)
Users can inject performance errors
![Page 23: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/23.jpg)
Limitations
Dynamic/Non-deterministic boxes
- e.g. NAT
“Invisible” rules
- e.g. backup rules
Transient network states
Ambiguous states (work in progress)
- e.g. ECMP
![Page 24: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/24.jpg)
Conclusion
Test’s the forwarding state by
generating minimal link, queue, rule
cover automatically.
Efficient liveness testing,reachability
testing.
Better tool than ping & traceroute.
Brings more testing and coverage of
networks
![Page 25: Automatic test packet generation](https://reader034.vdocuments.us/reader034/viewer/2022042716/55ab0aa91a28abcf718b4690/html5/thumbnails/25.jpg)
References
P. Kazemian, G. Varghese, and N. McKeown, “Header space analysis : Static checking for networks,” in Proc. NSDI, 2012, pp. 9–9.
Y. Bejerano and R. Rastogi, “Robust monitoring of link delays and faults in IP networks,” IEEE/ACM Trans. Netw., vol. 14, no. 5, pp.1092–1103, Oct. 2006
A. Dhamdhere, R. Teixeira, C. Dovrolis, and C. Diot, “Netdiagnoser:Troubleshooting network unreachabilities using end-to-end probes and routing data,” in Proc. ACM CoNEXT, 2007, pp. 18:1–18:12.