a real-world test-bed for mobile ad hoc networks:
DESCRIPTION
A Real-World Test-bed for Mobile Ad hoc Networks: Methodology, Experimentations, Simulation and Results. Per Gunningberg, Erik Nordström, Christian Rohner, Oskar Wibling Uppsala University. Background and problem. IETF is standardizing MANET (Mobile Adhoc NETwork) routing protocols: - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/1.jpg)
1Per Gunningberg©
A Real-World Test-bed for Mobile Ad hoc Networks:
Methodology, Experimentations, Simulation and Results.
Per Gunningberg, Erik Nordström, Christian Rohner, Oskar WiblingUppsala University
![Page 2: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/2.jpg)
2Per Gunningberg©
Background and problem
IETF is standardizing MANET (Mobile Adhoc NETwork) routing protocols:
– One proactive protocol - knowledge about all nodes– One reactive protocol - path on the need basis
Based on experiences from three protocols:– AODV - Adhoc On Demand Distance Vector
(reactive)– DSR - Dynamic Source Routing (reactive)– OLSR - OptiMized Link State Routing(proactive)
Problem: But majority of research done through simulations...
![Page 3: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/3.jpg)
3Per Gunningberg©
Part One
A test-bed for evaluating ad hoc routing protocols.
Close to reality
What to measure and how to analyze
Repeatable experiments
Grey Zone Phenomena
Conclusion
![Page 4: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/4.jpg)
4Per Gunningberg©
The Uppsala Ad hoc Protocol Evaluation Testbed (APE)
People carrying laptops with 802.11b
Suitable for indoor experiments that are hard to model in simulation
![Page 5: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/5.jpg)
5Per Gunningberg©
![Page 6: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/6.jpg)
6Per Gunningberg©
The Ad hoc Protocol Evaluation Testbed (APE)
Execution environment on top of existing OS.– Runs on Win and Linux
Scenarios with movement choreography.
Emphasizes easy management for scaling.
800++ downloads.
![Page 7: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/7.jpg)
7Per Gunningberg©
Laptop instructions (choreography)
node.11.action.0.msg=Test is starting... node.11.action.0.command=start_spyd node.11.action.0.duration=1 node.11.action.1.command=my_iperf c 2 t 330 node.11.action.1.msg=Stay at this location.node.11.action.1.duration=30 node.11.action.2.msg=Start moving! Go to Point A, the end of building. node.11.action.2.duration=75 node.11.action.3.msg=You should have arrived at Point A. Please stay. node.11.action.3.duration=30
![Page 8: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/8.jpg)
8Per Gunningberg©
Measurement procedures
Every node collects SNR from every other node it can hear during the test session
Every event is time stamped
Received Packets/Application results are collected at all nodes
Routing state snapshots are collected
Analysis is done after the test session.
![Page 9: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/9.jpg)
9Per Gunningberg©
Replaying a scenario
• SNR mapped to virtual distance• Each time interval corresponds to a
topological map
T5025 12510
075
150
Point A
Point D
![Page 10: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/10.jpg)
10Per Gunningberg©
APE is a Testbed for…
1. Relative protocol performance comparisons
2. Radio channel effects on ad hoc routing protocols
3. Interactions between hardware, software, protocol, mobility and radio environmentExample: Grey Zone Phenomena
4. Validation of simulation models
5. Generation of traces
![Page 11: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/11.jpg)
11Per Gunningberg©
802.11 Gray Zone Phenomena
AA
10 2
3
Broadcast
Unicast
3
![Page 12: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/12.jpg)
12Per Gunningberg©
Challenge
Results should be reproducible and comparable between tests
It follows that experiments must be repeatable...
...and therefore stochastic factors need to be dealt with
So – what can we achieve?
![Page 13: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/13.jpg)
13Per Gunningberg©
Stochastic Factors in Real World Experiments
Node mobility adds frequent changes in the network topology.– We use choreography and “measure
topology differences”
Variations in hardware and software configuration.– We use identical hardware and software.
Time varying radio environment affects link quality and error rates.
![Page 14: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/14.jpg)
14Per Gunningberg©
Topology differences - visual check
RED = Average mobilityGREEN = 25% with lowest mobilityBLUE = 25% with highest mobility
Experiment 1 Experiment 2
![Page 15: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/15.jpg)
15Per Gunningberg©
Part Two
Evaluating MANET protocols with the APE testbed, simulation and emulation.
Scenarios
UDP, Ping and TCP
Side-by-side comparison
Faulty protocol constructs
Conclusion
![Page 16: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/16.jpg)
16Per Gunningberg©
Coupling Simulation, Emulation and Real World
![Page 17: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/17.jpg)
17Per Gunningberg©
Routing protocols ability to adapt
OLSR - Proactive Link state routing. Monitors neighbors and exchange link state info.
AODV - broadcasts to set up path. HELLO or Link feedback to detect link failure.
DSR - broadcasts with source route. Listens to other traffic to find shorter route. RTT measurements and network ACKs.
React to connectivity changes
![Page 18: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/18.jpg)
18Per Gunningberg©
Emulation
•Same configuration as Real world
•Table-top emulation
•MAC filters force connectivity changes
•Reduces radio and mobility factors
•Interference reduces bandwidth
![Page 19: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/19.jpg)
19Per Gunningberg©
Simulation
•Scenarios recreated in a ns2-simulation using “default” models:
– Transmission range tuned to better match indoors– Mobility with jitter modeled after real world
measurements– Results averaged over 10 runs
•Results provide a baseline
•Can simulations using default (simple) models be used to predict routing protocol performance in complex real world environments?
![Page 20: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/20.jpg)
20Per Gunningberg©
Multidimensional Comparison
Three MANET routing protocol implementations:– OOLSR, AODV-UU, DSR-UU
Three traffic types:– UDP (20 pkts/s CBR)– Ping (20 pkts/s CBR)– TCP (File transfer)
Three mobility scenarios:– End node swap, Relay node swap, Roaming node
Three environments (dimensions):– Simulation, Emulation, Real world
3x3x3x(10 runs) = 270 runs
![Page 21: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/21.jpg)
21Per Gunningberg©
Experimental Test Environment
•Indoors with offices and corridors
•Four nodes (0, 1, 2, 3)
•Four waypoints (A, B, C, D)
•One data stream from node 3 to node 0
![Page 22: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/22.jpg)
22Per Gunningberg©
Relay Node swap
AAA B C D
0 1 2 3
![Page 23: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/23.jpg)
23Per Gunningberg©
Scenarios – Relay Node Swap
•End nodes stationary
•Intermediate nodes changes position
•Hop count never smaller than 2
![Page 24: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/24.jpg)
24Per Gunningberg©
End node swap
AAA B C D
0 1 2 3
![Page 25: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/25.jpg)
25Per Gunningberg©
Scenarios – End Node Swap
•End nodes change positions
•Intermediary nodes stationary
•Hop count changes from 3 to (2) and 1 and back
![Page 26: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/26.jpg)
26Per Gunningberg©
Roaming node
AAA B C D
0 1 2
3
![Page 27: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/27.jpg)
27Per Gunningberg©
Scenarios – Roaming Node
•Roaming node is source node
•All other nodes stationary
![Page 28: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/28.jpg)
28Per Gunningberg©
Results – Relay Node Swap
Simulation Emulation Real World
0
0.2
0.4
0.6
0.8
1
1.2
UDP (Delivery ratio)
AODV-UU DSR-UU OOLSR
Simulation Emulation Real World
0
0.2
0.4
0.6
0.8
1
1.2
Ping (Delivery ratio)
AODV-UU DSR-UU OOLSR
Simulation Emulation Real World
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
TCP (Mbps)
AODV-UU DSR-UU OOLSR
![Page 29: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/29.jpg)
29Per Gunningberg©
Results – End Node Swap
Simulation Emulation Real World
0
0.2
0.4
0.6
0.8
1
1.2
UDP (Delivery ratio)
AODV-UU DSR-UU OOLSR
Simulation Emulation Real World
0
0.5
1
1.5
2
2.5
3
TCP (Mbps)
AODV-UU DSR-UU OOLSR
![Page 30: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/30.jpg)
30Per Gunningberg©
Results – Roaming Node
Simulation Emulation Real World
0
0.2
0.4
0.6
0.8
1
1.2
UDP (Delivery ratio)
AODV-UU DSR-UU OOLSR
Simulation Emulation Real World
0
0.2
0.4
0.6
0.8
1
1.2
Ping (Delivery ratio)
AODV-UU DSR-UU OOLSR
Simulation Emulation Real World
0
0.5
1
1.5
2
2.5
3
3.5
4
TCP (Mbps)
AODV-UU DSR-UU OOLSR
![Page 31: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/31.jpg)
31Per Gunningberg©
AODV - UDP - End Node Swap
![Page 32: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/32.jpg)
32Per Gunningberg©
OLSR - UDP - End Node Swap
![Page 33: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/33.jpg)
33Per Gunningberg©
TCP - Simulation/Real World
![Page 34: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/34.jpg)
34Per Gunningberg©
Observations
Simulation and Emulation similar in absolute CBR performance but not in relative protocol ranking
Real world CBR performance is significantly lower
Discrepancy grows with traffic complexity and scenario
TCP performance is orders of magnitude lower for real world compared to simulation
periods of no-progress time in real world
![Page 35: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/35.jpg)
35Per Gunningberg©
Observations (continued)
OLSR tries less hard to re-route and therefore achieves more even performance
•Radio factors account for most of the discrepancy between simulation and real world...
•...but secondary effects, such as cross-layer interactions that are protocol specific, dominate, e.g.:
– Lost HELLOs (AODV)– Excessive buffering (DSR)
![Page 36: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/36.jpg)
36Per Gunningberg©
Protocol comparison conclusion
If one protocol performs better than another in simulation, is it possible to assume the same for
the real world?
NO
![Page 37: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/37.jpg)
37Per Gunningberg©
Latency - Ping - Relay Node
Simulation Real World
0
500
1000
1500
2000
2500
3000
3500
4000
Relay node swap (Ping) RTT std. dev.
AODV-UU DSR-UU OOLSR
![Page 38: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/38.jpg)
38Per Gunningberg©
Flip-Flop Routing DSR
Real Word
Simulation
![Page 39: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/39.jpg)
39Per Gunningberg©
Adapting to topology change
![Page 40: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/40.jpg)
40Per Gunningberg©
Routing Control Overhead
![Page 41: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/41.jpg)
41Per Gunningberg©
Conclusions
APE aims to address the lack of real world ad hoc experimental research test-beds
Repeatability addressed at a level that allows relative protocol comparisons
The value of cross-environment evaluation
Revealing of sensing problems leading to instabilities and poor performance
Not visible in simulations
![Page 42: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/42.jpg)
42Per Gunningberg©
The End
Paper:
http://www.it.uu.se/research/group/core/publications/GC_technical_report.pdf
APE testbed:http://apetestbed.sourceforge.net/
The Research group:http://www.it.uu.se/research/group/core/
![Page 43: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/43.jpg)
43Per Gunningberg©
Extra Slides
More details…
![Page 44: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/44.jpg)
44Per Gunningberg©
Self Interference Simulation
![Page 45: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/45.jpg)
45Per Gunningberg©
UDP
![Page 46: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/46.jpg)
46Per Gunningberg©
Ping
![Page 47: A Real-World Test-bed for Mobile Ad hoc Networks:](https://reader034.vdocuments.us/reader034/viewer/2022052702/568146ff550346895db43b52/html5/thumbnails/47.jpg)
47Per Gunningberg©
TCP