internet infrastructure measurement: challenges and tools
DESCRIPTION
Internet Infrastructure Measurement: Challenges and Tools. Mustafa Zali Internet Measurement Tuesday, 26 Aban 1388. Introduction. Review the physical properties of Internet Physical Properties Devices (routers, NAT boxes, firewalls, switches), Links (wired, wireless) Topology Properties - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/1.jpg)
Internet Infrastructure Measurement: Challenges and Tools
Mustafa Zali
Internet Measurement
Tuesday, 26 Aban 1388
1
![Page 2: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/2.jpg)
Introduction
Review the physical properties of Internet Physical Properties
Devices (routers, NAT boxes, firewalls, switches), Links (wired, wireless)
Topology Properties Various levels – Autonomous Systems, Points of
Presence, Routers, Interfaces Traffic Properties
Delays (Transmission, Propagation, Queuing, Processing etc.), Losses, Throughput, Jitter
2
![Page 3: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/3.jpg)
Outline
Properties
Challenges
Tools
3
![Page 4: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/4.jpg)
Properties
Review the important properties of Internet in bottom-up approach: Component Devices Topology: How devices interconnected Interaction of traffic and infrastructure
Our focus in on properties affected by physical infrastructure
4
![Page 5: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/5.jpg)
Physical Devices Properties
Internet: End Systems, Core Core: Switch, Router, Link The infrastructure that concerns us here is
core of internet.
5
![Page 6: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/6.jpg)
Link
Viewed at the IP layer propagation of data from one node to another is via links.
The details of links is hidden from IP layer (ch 2). Link properties
Propagation delay Capacity Packet delay Packet loss jitter
6
![Page 7: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/7.jpg)
Router
Routers move packets from one link to another.
Drop tail Active Queue Management
7
![Page 8: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/8.jpg)
Router
Routing Protocol Packet
Forwarding table updates
Routing Engine
Forwarding Table
Forwarding Engine
8
![Page 9: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/9.jpg)
Router
Interface
Switching Fabric
Interface Buffer-Interface
Buffer-Interface
9
![Page 10: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/10.jpg)
Wireless
The primary goal of wireless connection is to link users to wired infrastructure
Wireless technology: distance, data rate, reliability, potential interference, number of current users.
Security problem: very open nature of wireless
10
![Page 11: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/11.jpg)
Wireless- Technologies
Narrowband Wideband: allows signal to be detected easily
by receiver. Infrared: using high frequency range.
11
![Page 12: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/12.jpg)
Wireless- Standards
802.x: 802.11a, 802.11b, 802.11g 802.11b: WiFi (Wireless fidelity) Bluetooth: shorter distance, less power
consumption, cheaper WiMAX: 802.16
12
![Page 13: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/13.jpg)
Wireless
Measurements Signal strength Amount of power consumed Data bite rate Degree of coverage Session related information (duration, set-up time) Other traditional measurements
13
![Page 14: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/14.jpg)
Topology properties
Four level Autonomous systems: Independently operated and managed
network BGP protocol for routing between them.
Point of presence: Consists of one or more routers in a single location.
Router: Router graph Vertices are router and edges are links between them
Interface: Interface Graph Vertices are router intreface and edges are links one-hop connection
14
![Page 15: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/15.jpg)
Interaction of Traffic and Network Network constrains traffic:
Minimum possible delay Maximum possible throughput
15
![Page 16: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/16.jpg)
Packet Delay
Routing delay Packet processing delay Queuing delay Additional delay
Transmission delay Propagation delay
hp dGd
delayontransmissit
sizepackets
speedlinkv
lengthlinkd
16
![Page 17: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/17.jpg)
Packet Loss
In element n:
Aggregate loss:
Along pass is aggregate of hops:
n
nn C
L
i
inn ,11
hpp lGl 1log
17
![Page 18: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/18.jpg)
Throughput
Throughput
Throughput on path
T
Cn
hp tGt
18
![Page 19: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/19.jpg)
Packet Jitter
Variability of packet inter arrival times Low jitter: more predictable, more reliable
19
![Page 20: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/20.jpg)
Challenges
Poor Observability: Observability is not built into the design of Internet protocols and components.
Reasons for this: Core Simplicity Hidden Layers Hidden Pieces Administrative Barriers
20
![Page 21: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/21.jpg)
Core Simplicity
Stateless nature: Stupid network Routers is very simple. Explosive growth of Internet As network elements do not track packets
individually, interaction of traffic with the network is hard to observe
21
![Page 22: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/22.jpg)
Hidden Layers
Below IP level, packet transmission implemented in many ways.
These details are hidden from IP level. Detailed measurement can not capture these
details.
22
![Page 23: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/23.jpg)
Hidden Pieces - Middleboxes
End-to-end argument.
Firewalls – provide security Traffic Shapers – assist in traffic management Proxies – improve performance by terminating TCP
inside network. (Cache proxy) NAT boxes – utilize IP address space efficiently
Each of these impedes visibility of network components. firewalls may block active probing requests NATs hide away the no. of hosts and the structure of the
network on the other side
23
![Page 24: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/24.jpg)
Administrative Barriers
Owing to the competition-sensitive nature of the data required (topology, traffic etc.), ISPs actively seek to hide these details from outside discovery
Information that they do provide are often simplified. E.g.: Instead of publishing router-level topologies,
ISPs often publish PoP-level topologies
24
![Page 25: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/25.jpg)
Tools Classification
Active Measurement Passive Measurement Fused/Combined Measurement Bandwidth Measurement
25
![Page 26: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/26.jpg)
Active Measurement Tools
Methods that involve adding traffic to the network for the purposes of measurement
Ping: Sends ICMP ECHO_REQUEST and captures ECHO_REPLY Useful for measuring RTTs Only sender needs to be under experiment control Zing: Sends at random, exponential time
26
![Page 27: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/27.jpg)
Traceroute
Useful for determining path from a source to a destination
Uses the TTL (Time To Live) field in the IP header in a clever but distorted way
A large scale measurement system called skitter uses traceroute to discover network topology (Chapter 10)
27
![Page 28: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/28.jpg)
IP Header and the TTL field
ver length
32 bits
data (variable length,typically a TCP
or UDP segment)
16-bit identifier
Internet checksum
time tolive
32 bit source IP address
IP protocol versionnumber
header length (bytes)
max numberremaining hops
(decremented at each router)
forfragmentation/reassembly
total datagramlength (bytes)
upper layer protocolto deliver payload to
head.len
type ofservice
“type” of data flgsfragment
offsetupper layer
32 bit destination IP address
Options (if any) E.g. timestamp,record routetaken, specifylist of routers to visit.
28
![Page 29: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/29.jpg)
Traceroute Problem
Suppose the path between A and D is to be determined using traceroute
A
X Y
D
B C
29
![Page 30: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/30.jpg)
Traceroute Process
A
X Y
D
B C
Dest = D
TTL = 1
B: “time exceeded”
30
![Page 31: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/31.jpg)
Traceroute Process
A
X Y
D
B CDest = D
TTL = 2
C: “time exceeded”
31
![Page 32: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/32.jpg)
Traceroute Process
A
X Y
D
B C
Dest = D
TTL = 3
D: “echo reply”
32
![Page 33: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/33.jpg)
Traceroute issues
Path Asymmetry (Destination -> Source need not retrace Source -> Destination)
Unstable Paths and False Edges
Aliases
Measurement Load
33
![Page 34: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/34.jpg)
Unstable Paths and False EdgesInferred path: A -> B -> Y
A
X Y
D
B C
Dest = D
TTL = 1
B: “time exceeded”
Dest = D
TTL = 2
Y: “time exceeded”
34
![Page 35: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/35.jpg)
Aliases
IP addresses are for interfaces and not routers
Routers typically have many interfaces, each with its own IP address
IP addresses of all the router interfaces are aliases
Traceroute results require resolution of aliases if they are to be used for topology building
35
![Page 36: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/36.jpg)
Aliases
Alias resolution: Send packet to both interface. Close IP ID field and same TTL field. Record Route Option. (The address of interface
that is packet sent.) Guess: difference in last bits.
36
![Page 37: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/37.jpg)
Measurement Load
Traceroute inserts considerable load on network links if attempting a large-scale topology discovery
Optimizations reduce this load considerably Track interfaces visited already Assumption: Routers are stable and only one path exists. If single source is used, instead of going from source to
destination, a better approach is to retrace from destination to source.
If multiple sources and multiple destinations are used, sharing information among these would bring down load considerably (A->B->C->D, X->B)
37
![Page 38: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/38.jpg)
System Support
Injecting and capturing packets, has several security problems.
Efficient packet injection and accurate measurement of arrival and departure times are best done at kernel level
Using scriptroute, unprivileged users can inject and capture packets
Periscope’s API helps define new probing structures and inference techniques for extracting results from arrival patterns of responses
Unrestricted access to the network interface raises security concerns
38
![Page 39: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/39.jpg)
Passive Measurement
Methods that capture traffic generated by other users and applications to build the topology
39
![Page 40: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/40.jpg)
BGP
A BGP routing table is the set of paths. Each path is the sequence of ASes. Each AS advertises the routes that it knows. Routeviews repository is useful for passive
internet analysis and monitoring.
40
![Page 41: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/41.jpg)
41
![Page 42: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/42.jpg)
BGP– Advantages and Disadvantages Large set of AS-AS, router-router connections can
be learned by simply processing captured tables
However, especially using BGP views, there could be potential loss of cross-connections between ASes which are along the path
Secondly, route aggregation and filtering tends to hide some connections
Also, multiple connections between ASes will be shown as a single connection in the graph
42
![Page 43: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/43.jpg)
OSPF
Capture link state announcements within routing domain.
Announcements Topology changes External routes change availability
43
![Page 44: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/44.jpg)
Fused Measurement
Combine both active and passive measurements.
Active: large amount of traffic. One way is to using passive measurement Another way is to augment passively
obtained BGP topologies with additional inter AS connections.
44
![Page 45: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/45.jpg)
Bandwidth Measurement
Bandwidth – amount of data the network can transmit per unit time
Bandwidth measure requirements Streaming media applications Server selection Estimating the bandwidth for TCP flow control Verification of service level agreement
45
![Page 46: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/46.jpg)
Bandwidth Measurement
Bandwidth measurement is a active process Bottleneck: link with minimum bandwidth Three kinds of bandwidth:
capacity: max throughput a link can sustain, available bandwidth: capacity – used bandwidth and
bulk transfer capacity: rate that a new single long-lived TCP connection would obtain over a path
dxxutut
t
1 Ctu 1
46
![Page 47: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/47.jpg)
Bandwidth Measurement
Tight link: Link with minimum available bandwidth
Narrow link : Link with minimum capacity
delayontransmissidelayqueueingfbandwidth ,
47
![Page 48: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/48.jpg)
Bandwidth Measurement Methods These focus on observing how packet delay
(queuing and transmission) is affected by link properties
Four types: Packet-pair Methods Size-delay Methods Self-induced Congestion Bulk Transfer Capacity Measurement
48
![Page 49: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/49.jpg)
Packet-Pair Methods
Methods to measure capacity and available bandwidth Involve sending probe packets with known inter-packet
gaps and measuring the same gap downstream where C is the capacity, L is the length of probe packets,
max delta is the maximum inter-packet gap measured downstream
49
![Page 50: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/50.jpg)
Packet-Pair Methods- Capacity
iii C
L,max1
h
hhh
hhh
C
L
C
L
C
L
C
L
C
L
C
L
,...,,,maxmax
,,maxmax,max
100
111
Hiii
Hih C
L
CL
,...,0
,...,001 min
1max,max
50
![Page 51: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/51.jpg)
Packet-Pair Methods- Capacity Capacity of narrow link can be estimated:
The packets should be queued at bottleneck link:
Cross traffic: sending many probe packets
1
h
LC
C
L0
51
![Page 52: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/52.jpg)
Packet-Pair Methods- Bandwidth Assumption:
FIFO queuing Router queue is not empty between first
and second probe packet Tight link is narrow link
0
011 hCA
52
![Page 53: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/53.jpg)
Size Delay Methods
Useful for measuring link capacities on each link along a path
Based on the observation that transmission delay is affected by link capacity and packet size
The idea is to send many different sized packets and measure the difference in delays affected by packet size.
Then the capacity of each link will be a function of these differences
Method assumes there is no cross-traffic, no variation in packet size
Measurements become less accurate if the length of the path grows
53
![Page 54: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/54.jpg)
Size Delay Methods
Queuing delay Transmission delay Propagation delay
54
![Page 55: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/55.jpg)
Size Delay Methods
i
k kii C
LLT
1
i
k ki C1
1
1
1
iii BB
C
LLT iii
55
![Page 56: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/56.jpg)
Self Induced Congestion
Find the maximum probe rate that create congestion
Increase R until congestion occurs Problem: cross traffic
56
![Page 57: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/57.jpg)
Bulk Transfer Capacity Management One opens a TCP connection over the path
and sends as much as data that the path can handle
57
![Page 58: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/58.jpg)
Caveats in Bandwidth Measurements High rate links make it difficult to measure
bandwidth accurately because of small delays
Wireless links affect rate dramatically on fine timescales
FIFO order is not guaranteed in wireless links
Layer 2 devices can cause underestimation of a IP hop’s capacity by introducing additional transmission delays
58
![Page 59: Internet Infrastructure Measurement: Challenges and Tools](https://reader036.vdocuments.us/reader036/viewer/2022062309/568150bd550346895dbedbff/html5/thumbnails/59.jpg)
Conclusion
Internet Measurement is key to designing the next generation communication network
Fundamental design principles of the current internet make it harder for measuring various aspects of it
Preliminary research has resulted in a set of basic tools and methods to measure aspects like topology, traffic etc.
Accuracy of such methods is still an open question There is still a lot of ground to cover in this direction and
this is where researchers like you come into the equation!
59