pinger: actively measuring the worldwide internet’s end-to-end performance
DESCRIPTION
PingER: Actively measuring the worldwide Internet’s end-to-end performance. DRAFT. Les Cottrell SLAC Workshop at the University of Malaya in Kuala Lumpur , June 24-25, 2013. Agenda. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/1.jpg)
PingER: Actively measuring the worldwide Internet’s end-to-end performance
Les CottrellSLAC
Workshop at the University of Malaya in Kuala Lumpur , June 24-25, 2013
![Page 2: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/2.jpg)
AgendaUsing PingER measurements going back to 1998 and covering 168 countries, this talk will illustrate, Internet performance worldwide.
•Brief history
•How can the Internet help development?
•How does PingER measure Internet performance?
•What do we measure, what does it tell us?
•What do we find?
•Case studies illustrating PingER
•Managing PingER
![Page 3: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/3.jpg)
History• Story of Ping
• Early PingER
• Extension to Developing Regions
• Extension to Pakistan
• Extension to Malaysia
![Page 4: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/4.jpg)
The start• Ping tool invented by Mike Muuss
– “a little thousand-line hack” during a single evening to troubleshoot “odd behavior” on the computer network at the U.S. Army’s Ballistic Research Laboratory in Maryland.
– sent a small data packet known as an echo request to an IP address, typically a remote server or network node. If the target address was reachable, it echoed back the same data, and the program recorded the time it took for the round-trip journey.
– Reminded Muuss of the percussive sound pulse sonar systems use to detect objects underwater, he named it after that sound—ping
• Now defined by RFC 792
![Page 5: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/5.jpg)
5 Joint Techs: I2 & ESnet,Stanford
Measurement Mechanism: PingER
Internet
10 ping request packets each 30 mins
RemoteHost(typicallyweb server)
>ping remhost
Ping response packets
Measure Round Trip Time & Loss
Monitor Host
![Page 6: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/6.jpg)
6
Early PingER• As the head of networking at SLAC, I set up the
system using ping simply to test connections between the laboratory and several dozen research institutions in about a dozen countries that were collaborating on a physics experiment known as BaBar to study properties of subatomic particles.
• Over the next half-decade, as word of PingER’s value spread, I extended monitoring to hundreds more physics laboratories and science centers across the globe. But the project didn’t take a humanitarian turn until 2001.
•UNIMAS
Workshop, Sarawak, Dec 2012
![Page 7: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/7.jpg)
7
Extension to the Developing regions• In 2001 I visited ICTP in Italy.• Driven by ICTP’s goals of bringing first-class science
and technology to developing countries they wanted to know how well the networks were working.
• The simple PingER project was the perfect tool for the job. Ubiquitous ping so nothing to install at remote targets.
• They offered to help expand the project to those parts of the world that needed it most.
• Within the next year, we began establishing monitoring and target hosts in countries as diverse as Ecuador, Rwanda, Jordan, and Bhutan.
UNIMASWorkshop, Sarawak, Dec 2012
![Page 8: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/8.jpg)
8
Extension to Pakistan• In 2004 set up joint agreement with NUST in Pakistan• Soon got my first real glimpse of just how much of a
difference PingER can make. – Set up a PingER monitoring site in the country to
assess performance on the then year-old Pakistan Educational Research Network (PERN).
– The network’s providers touted its bandwidth of 155 Mbps, impressive at the time. But PingER revealed that the “last mile” links to universities were dreadful. These bottleneck connections funneled data at no more than 1 Mbps, causing long delays and high packet loss.
UNIMASWorkshop, Sarawak, Dec 2012
![Page 9: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/9.jpg)
9
Extension to Pakistan
• He clearly took PingER’s lessons to heart. When construction of PERN2 began in 2009, its plans included extending high-speed, 1-Gbps data links all the way to university data centers
UNIMASWorkshop, Sarawak, Dec 2012
• During a visit to the university, I presented our findings to the chairman of Pakistan’s higher education commission, Atta-ur-Rehman, who was preparing to fund the next major upgrade to PERN.
![Page 10: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/10.jpg)
10
Extension to Malaysia
• Have set up an official signed MoU between SLAC & U of Malaysia in Sarawak (UNIMAS)
• Idea was to replicate the NUST project
• Fortnightly meetings by Skype
• Just getting started, no students yet
• Met with Vice Chancellor (VC) at one meeting
• This workshop is a follow up.
UNIMASWorkshop, Sarawak, Dec 2012
![Page 11: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/11.jpg)
11
Why do measurements of the Internet matter
UNIMASWorkshop, Sarawak, Dec 2012
![Page 12: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/12.jpg)
Why does it matter
• African scientists isolated
• Lack critical mass
• Need network to collaborate but it is terrible
• Brain drain
• Brain gain, tap diaspora
• Blend in distance learning
• Provide leadership, train trainers
12
Internet Users 2002
Cartograms from:www.geog.qmw.ac.uk/gbhgis/conference/cartogram.html
Tertiary Education fromhttp://www.worldmapper.org/
![Page 13: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/13.jpg)
13 eGY Africa 2012Workshop, Nairobi Oct 2012
How does the Internet help• Investment in information technology plays the role of
a "facilitator" that allows other innovations to take place. http://findarticles.com/p/articles/mi_m1093/is_3_45/ai_86517828/
• World Bank / IFC report: for every 10% increase in high-speed Internet connections there is an increase in economic growth of 1.3 percentage points. April 2010. http://www.infodev.org/en/Article.522.html
• Example: Uganda 15% increase in price of maize based on improved farmer bargaining power. www.itu.int/ITU-D/.../S1-01-NG-ICT_Indicators-Tim_Kelly.pptx
![Page 14: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/14.jpg)
14
How does PingER work
• Mechanism
• Coverage
UNIMASWorkshop, Sarawak, Dec 2012
![Page 15: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/15.jpg)
15 Joint Techs: I2 & ESnet,Stanford
Measurement Mechanism: PingER
Internet
10 ping request packets each 30 mins
RemoteHost(typicallyweb server)
>ping remhost
Ping response packets
Measure Round Trip Time & Loss
On
ce a Day
Uses ubiquitous ping
Monitor Host
Repositories
NUST
![Page 16: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/16.jpg)
16
Deployment• Monitors > 90 in 23 countries, 4 in Africa
Summer Joint TechsStanford, July 2012
• Beacons monitored by most monitors (~100)• Remote sites monitored by some monitors (~750)
![Page 17: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/17.jpg)
17
Metrics Available from PingER• UnReachability
• Minimum RTT
• Average RTT
• Jitter
• Loss
• Derived throughput
• MOS
• Directness of Connection
• OthersUNIMAS
Workshop, Sarawak, Dec 2012
![Page 18: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/18.jpg)
18 eGY Africa 2012Workshop, Nairobi Oct 2012
Unreachability
Summer Joint TechsStanford, July 2012
Unreachability: e.g. N. African uprisings Jan ‘11
NARSS (Cairo)
Helwan (Cairo)
EUN (Cairo)
23:59 Jan 28
23:59 Jan 27
12:00 Jan 27
• Impact varied: start time, recovery time, after effects• Egypt University Network (EUN) down least time
– NARSS via AlterNet->Italy->Egypt, Helwan &EUN via PCCW Global
• Libya first went dark 06:00 Feb 19 for 3 days, then again on Mar 4th more permanently
• Algeria, Morocco, Tripoli not noticeable
=No pings respond
![Page 19: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/19.jpg)
19
Average Round Trip Time (RTT)• Mainly a distance related, but also congestion (i.e. at
the edge)
• For real-time multimedia (H.323) traffic RTT: 0-300ms =Good, 300-600ms=Acceptable, and > 600ms= poor.
UNIMASWorkshop, Sarawak, Dec 2012
![Page 20: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/20.jpg)
20
Minimum RTT history by region• Minimize effects of congestion and queuing
UNIMASWorkshop, Sarawak, Dec 2012
![Page 21: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/21.jpg)
21 UNIMASWorkshop, Sarawak, Dec 2012
•GEOS (Geostationary Earth Orbit Satellite)–Good coverage, but expensive in $/Mbps–& long delays min RTT >450ms easy to spot
N.b. RTTs > 250 ms bad for VoIP
Impact of GEOS vs. TerrestrialGEOSGEOS
Demo
![Page 22: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/22.jpg)
22 UNIMASWorkshop, Sarawak, Dec 2012
Conversion history by country seen by min-RTT
Demo
![Page 23: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/23.jpg)
23
Jitter• Mainly at edges, critical for real time: VoIP, gaming• Exponential improvement (factor 10 in 6 yrs.)• The optimum amount of one way latency is 11 ms for keeping time in
music. – Above that delay and they tend to slow down. – >50-70 ms performances tended to completely fall apart.
UNIMASWorkshop, Sarawak, Dec 2012
• For real time haptic control and feedback for medical operations <=80ms is needed.
N. America, Europe, E Asia & Oceania < 1msAfrica, S. Asia & S.E. Asia worst off
![Page 24: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/24.jpg)
24 Summer Joint TechsStanford, July 2012
• Low (<1%) losses are good.• Huge impact on data transfer times (due to timeouts)• Real time impact due to recovery timeouts, e.g. echoing typing• Losses are mainly at the edge, so often distance independent• Losses improving roughly exponentially, ~factor 100 in 12 years
24
Loss has Similar
behavior to thruput
• Best <0.1%: N. America, E. Asia, Europe, Australasia
• Worst> 1%:• Africa & C. Asia
Loss
![Page 25: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/25.jpg)
25 UNIMASWorkshop, Sarawak, Dec 2012
Summer Joint TechsStanford, July 2012
Derived ThroughputDerived throughput ~ 8*1460/(RTT*sqrt(loss))
Mathis et. al
Europe, E. Asia & Australasia merging
Behind Europe:
4 yrs: Russia, 7 yrs:L America,
M East, SE Asia
11 yrs.: India, C. Asia
13 yrs.: Africa
![Page 26: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/26.jpg)
26
• ITU metric, based on quality of a conversation– Originally people listen and give quality 1-5– Can derive from RTT, jitter and loss
UNIMASWorkshop, Sarawak, Dec 2012
MOSMean Opinion Score MOS)
• >=4 is good,
• 3-4 is fair,
• 2-3 is poor.
Important for VoIP
Usa
ble
From the PingER project http://www-iepm.slac.stanford.edu/pinger
![Page 27: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/27.jpg)
27 UNIMASWorkshop, Sarawak, Dec 2012
Directness of connection (Alpha)• Alpha to allow for delays in network equipment &
indirectness of actual route. D = 1 way distance• Distance=D(km)=Alpha*[RTT(msec)*100(km/msec)]– Alpha = D(km) / (min_RTT[msec] * 100 [km/msec])
• If know lat/long of monitor and remote host then know D, so with min-RTT can estimate Alpha– Max(Alpha) =1 = direct (great circle) route and no
network delays– Alpha > 1 probably identifies bad lat/long
coordinates for hosts.– Low values typically mean very indirect route, or
satellite or slow connection (e.g. wireless)– Alpha typically ~ 0.45
![Page 28: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/28.jpg)
28 UNIMASWorkshop, Sarawak, Dec 2012
S.E. Asia Directivity
UNIMASWorkshop, Sarawak, Dec 2012
UNIMAS to …W MY A~0.45
•->PH via JP A~0.22
•->ID via HK A~0.17
•->TH via SG A~0.3
•->BR via ? A~0.07
•->KH via HK-VN A~0.16
•->SG A~0.23
Need measurements from W MY + routing info
TEIN3Trans Eurasia Info Net
![Page 29: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/29.jpg)
29 UNIMASWorkshop, Sarawak, Dec 2012
Alpha worldwide
• Interest in Polar route with Global warming
Summer Joint TechsStanford, July 2012
Alpha=0.71A
lpha
=0.7
3
Alpha
=0.7
3SLAC
JP
AU
NZ
NSK.RU
TW0.180.16IN
DE
EG0.34
JP0.32
o.41
AU0.53
![Page 30: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/30.jpg)
30
• Big improvements for C Asia, S Asia & Australasia
UNIMASWorkshop, Sarawak, Dec 2012
Directivity (Alpha) from SLAC to world
Summer Joint TechsStanford, July 2012
More stable year to year as add more hosts
![Page 31: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/31.jpg)
31
Other metrics• Duplicate packets (try ping www.cern.ch, load
balancing?)
• Out of order packets (parallel paths)
• Conditional Loss Probability (non-random loss)
– one packet is lost the following packet is also lost– route change, loss of sync, spanning tree
reconfiguration
• Maximum packet loss (useful for buffer bloat?)
![Page 32: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/32.jpg)
32
S.E. Asia
• Just started mining early data
• Where does Malaysia sit
• How much variation in SE Asia
• Variation in Malaysia
• Troubles at UNIMAS
• Top and bottom 3 sites monitored in Malaysia
UNIMASWorkshop, Sarawak, Dec 2012
![Page 33: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/33.jpg)
33
Malaysia vs. Other Regions
UNIMASWorkshop, Sarawak, Dec 2012
![Page 34: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/34.jpg)
34
Variation between SE Asian countries
• Factor of 10 between Singapore and Laos
• Singapore 4x better than next countries
• Exponentially improving with time
• On its ownSingaporeapproachesE Asia.
UNIMASWorkshop, Sarawak, Dec 2012
![Page 35: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/35.jpg)
35
Avg-RTT, jitter & loss by Malay State• Need low values of all 3 metrics
• RTTs similar, big diffs in jitter & loss
• Allianze UniColl looks bad
• UTP next worst loss
UNIMASWorkshop, Sarawak, Dec 2012
Seen from SLAC, Nov 2012
![Page 36: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/36.jpg)
36
Non lossy Malaysian hosts seen from SLAC Nov 2012
UNIMASWorkshop, Sarawak, Dec 2012
UNIMAS
MIMOS
MIMOS
UNISZA
Note monitoring host (SLAC) down
![Page 37: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/37.jpg)
37
OCESB
Lo
ssy
ho
st s
een
fro
m S
LA
C N
ov
2012
UPSI
UTEM
MIU
AIU
Sabah
![Page 38: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/38.jpg)
38
Diving deeper: packet loss Nov 27-28• Allianze University College unreachable• UTEM, MIU, UPSI, OCESB, AIU, SABAH experienced loss Nov
26• No PingER loss from the rest on Nov 26: Johor, Kelantan, KL,
Nigeri Sembilan, Sarawak, Terengganu
UNIMASWorkshop, Sarawak, Dec 2012
1930-midnight MST, backup?
UTEMLosses isolated, not correlated with large RTT
![Page 39: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/39.jpg)
39
Another host, large RTTs correlate with time of day
UNIMASWorkshop, Sarawak, Dec 2012
![Page 40: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/40.jpg)
40
UNIMAS Jitter
UNIMASWorkshop, Sarawak, Dec 2012
UNIMAS to
MalaysiaJitter
AllianzeUniversity
College
Universiti TechnologiPetronas
![Page 41: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/41.jpg)
41
Why not show UNIMAS to Malaysia more
• Big changes in RTT affect throughput especially for Kuching
• UNIMAS was seeing congestion – This would be seen everywhere– Turn on shaping– Removes loss &
day-night variations
UNIMASWorkshop, Sarawak, Dec 2012
2
1RT
T m
s
Background loss colors
SLAC to UNIMAS Oct-Nov 2012
![Page 42: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/42.jpg)
42 UNIMASWorkshop, Sarawak, Dec 2012
Increase of capacity to UNIMAS from 200Mbps to 500Mps
UNIMASWorkshop, Sarawak, Dec 2012
Huge spikes removed
![Page 43: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/43.jpg)
43 UNIMASWorkshop, Sarawak, Dec 2012
UNIMASWorkshop, Sarawak, Dec 2012
Improvement
![Page 44: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/44.jpg)
44
Unreachable Malaysian hosts
• Unreachable from SLAC Nov 1-26, 2012:1. 92% Allianze University College
2. 20% www.ocesb.com.my (Speedtest)
3. 17% Universiti Teknologi Petronas, Bandar Seri Iskandar
4. 4% University Malaysia Kelantan
• 100% Reachable– MIMOS, UNIMAS, Sultan Zainal Abidin University,
USIM, University Teknologi Malaysia, Sultan Idris University of Education
UNIMASWorkshop, Sarawak, Dec 2012
![Page 45: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/45.jpg)
45
Case Study Africa
UNIMASWorkshop, Sarawak, Dec 2012
![Page 46: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/46.jpg)
46
Submarine Cables
2012
1 cable, W Coast only, No competition (340Gb/s)
World Cup S Africa 2008 led to many submarine cables connecting Africa to rest of the worldCapacity, shorter RTT, competitionMost cables are now activeFuture Cables promise connectivity:
ACE Q3 2012: France-Spain-Morocco & many African countries
SAex 2014: Brazil-Angola-SA WASACE 2014: France-US-Brazil-Nigeria-
Angola-SABRICS 2014: Brazil-SA-Mauritius-India-
China-RussiaCable capacity increase from 0.34Tb/s in
2008 to 87.5 Tb/s by 2014 (factor ~3000)http://manypossibilities.net/african-undersea-cables
/
2008
![Page 47: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/47.jpg)
47 Summer Joint TechsStanford, July 2012
Min-RTT satellite vs. fibre
2010
2009
2008
OK
to
US
![Page 48: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/48.jpg)
48
Comparison in minimum from SLAC to African Countries in 2008 and 2012.
UNIMASWorkshop, Sarawak, Dec 2012
2008 2012800
600
400
200
0
800
600
400
200
0TerrestrialTerrestrial
GEOS GEOS
Minimum RTT from SLAC to African Countries 2008 vs.. 2012
Min
RT
T (
ms)
Note the countries that have gone from GEOS to terrestrial
![Page 49: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/49.jpg)
49
RTT
Summer Joint TechsStanford, July 2012
• Angola step mid-May, more stable
• Zambia one direction reduce 720>550ms– Unstable, still
trying?• Tanzania, also
dramatic reduction in losses
• Uganda inland via Kenya, 2 step process
• Rwanda Sep 25• Many sites still
to connect
750ms750ms 450ms450ms
Aug 20Aug 20
SLAC to AngolaSLAC to Angola
SLAC to ZambiaSLAC to Zambia
SLAC to TanzaniaSLAC to Tanzania
SLAC to UgandaSLAC to Uganda
1 direction1 direction
Both directionsBoth directions
Sep 27Sep 27
1 direction1 direction Both directions?Both directions?
RTT, e.g. changes in 2009
![Page 50: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/50.jpg)
50
Intra Africa Optical Fibre Networkhttp://www.ubuntunet.net/fibre-map
Summer Joint TechsStanford, July 2012
Just as important as the submarine cables serving the coasts, are the tens if not hundreds of millions of dollars being invested in new terrestrial fiber to move this capacity inland.
![Page 51: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/51.jpg)
51
Other 3 billion (O3b)• Refers to population of world without
broadband
• Constellation of 8 Medium Earth Orbit satellites at altitude 8000km
• Min RTTs factor of 4 less than GEOS – ~125ms, similar to inter-continent land lines
• Backed by SES World Skies, HSBC, Google…
• Launch 2013
eGY Africa WorkshopNairobi Oct 2012
![Page 52: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/52.jpg)
52
Differences within Africa• Southern Africa now leads, caught North • Central is worst off• Notice improvement since 2008
– satellite=> terrestrial, competition
eGY Africa 2012Workshop, Nairobi Oct 2012
Derived throughput from SLAC to African regions
• Africa improving, catching up?
![Page 53: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/53.jpg)
53
Africa was 16 yrs. behind Europe in 2009
UNIMASWorkshop, Sarawak, Dec 2012
![Page 54: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/54.jpg)
54
Africa is now 14 yrs. behind Europe in 2012
UNIMASWorkshop, Sarawak, Dec 201214 years
![Page 55: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/55.jpg)
55
Africa might catch up with Europe in 20 years at current rate of improvement
UNIMASWorkshop, Sarawak, Dec 2012
Was falling further behind
70 times w
orse
26 years beh
ind
Eu
rop
e
![Page 56: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/56.jpg)
56
• By 2011 prices had dropped only factor of 2– Alternative fibre often owned by electricity
companies, pipelines and not allowed to sell, lease or operate – needs deregulation and is happening
– Business model: ISPs sell to large corporations, gov, edu, NGOs=>small customer base to recover costs from =>high prices
• WiFI & Mobile to the rescue, overlay 3G with fibre net
• Alex Twinomugisha originally published |Why Are African Internet Access Prices Still High? on Africa Business Source
Prices
![Page 57: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/57.jpg)
57 eGY Africa 2012Workshop, Nairobi Oct 2012
Then there is the cost
![Page 58: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/58.jpg)
58
Africa Broadband costs vs. rest of the world
UNIMASWorkshop, Sarawak, Dec 2012
![Page 59: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/59.jpg)
59
NRENs in Africa
eGY Africa WorkshopNairobi Oct. 2012
With connection to GÉANT going live end 2012, UbuntuNet will provide sub-Saharan Africa with infrastructure for global, and regional research collaboration and e-learning
N. Africa connected via EUMED to Europe. Also Arab States REN formed 1 year ago
From PingER Project
![Page 60: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/60.jpg)
60
Traceroutes from S Africa & Burkina Faso in 2009
UNIMASWorkshop, Sarawak, Dec 2012
Despite having NRENs & terrestrial fibres along both East & West coasts of Africa connecting to most maritime countries, still most inter-African routes went via Europe and N America
Not only did this add large delays, but also resulted in costly inter-continental rates
![Page 61: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/61.jpg)
61
Setting up IXPs for better connectivity
UNIMASWorkshop, Sarawak, Dec 2012
![Page 62: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/62.jpg)
62
Opportunities: Square Kilometre Array (SKA)
Summer Joint TechsStanford, July 2012
• Build in in Sub-Saharan states with cores in South Africa and Australia,
• €1.5 billion, construction start 2016, initial observations 2019
“…equivalent to ten times the global internet traffic today”
“…equivalent to ten times the global internet traffic today”
![Page 63: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/63.jpg)
63
Opportunities: Continued• Aug 30, 2012: 220 donated computer servers
from CERN, Switzerland, will start a journey to be delivered to the Kwame Nkrumah University of Science and Technology in Ghana.– This will provide a new computing center for
KNUST and boost African physics onto the international stage, helping African students, e.g. enable participation in simulations of LHC data.
• Strategic plan for a synchrotron light source in southern Africa – 2 day workshop Dec 2011, Pretoria– http://indico.saip.org.za/conferenceDisplay.py?ovw=True&confId=12
![Page 64: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/64.jpg)
64
More Information
• Case Study– https://confluence.slac.stanford.edu/display/IEPM/N
ew+E.+Coast+of+Africa+Fibre
• Telegeography submarine cable map– http://www.submarinecablemap.com/
• Africa Undersea Cables– http://manypossibilities.net/african-undersea-cables/
• Ubuntunet– http://www.ubuntunet.net/fibre-map
eGY Africa WorkshopNairobi, Oct. 2012
![Page 65: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/65.jpg)
65
Mediterranean Cable Cut
• See https://confluence.slac.stanford.edu/display/IEPM/Effects+of+Fibre+Outage+through+Mediterranean
UNIMASWorkshop, Sarawak, Dec 2012
![Page 66: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/66.jpg)
66
Mediterranean Fibre Cuts Jan 31, 2008• 2 major cables: SEAMEW4, FLAG cut off
Marseilles and Alexandria
• Traffic falls over to SEAMEW3
• SEAMEW3 congested, resulted in doubling RTTs, increased jitter & Loss
• Start between 6:47am and 7:16am GMT• Reduced bandwidth by over 50% to over 20 countries
UNIMASWorkshop, Sarawak, Dec 2012
200=>400msmsLost connectionSLAC to tanta.edu.eg800
6004002000
RT
T m
s
![Page 67: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/67.jpg)
67
Affected many countries
UNIMASWorkshop, Sarawak, Dec 2012
![Page 68: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/68.jpg)
68
Impact on loss lasted several days
UNIMASWorkshop, Sarawak, Dec 2012
Afr
ica
M E
ast
S A
sia
SE
A
sia
![Page 69: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/69.jpg)
69
World events Case studies
• Japanese Earthquake & Tsunami, 2011– https://confluence.slac.stanford.edu/display/IEPM/J
apanese+Earthquake+March+11th%2C+2011
• Chilean Earthquake 2010– https://confluence.slac.stanford.edu/display/IEPM/C
hilean+Earthquake+Feb+27th%2C+2010
• Syria Nov 29, 2012– https://confluence.slac.stanford.edu/display/IEPM/S
yria+shuts+down+its+Internet+connection UNIMAS
Workshop, Sarawak, Dec 2012
![Page 70: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/70.jpg)
70
Effect of Japanese Earthquake & Tsunami March 11, 2011
UNIMASWorkshop, Sarawak, Dec 2012
2001000
RT
T (
ms)
SLAC to KEK
Traffic rerouted Eastwards
![Page 71: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/71.jpg)
71
Chilean earthquake, Feb 27, 2010
• Telmex is an Internet service provider in Chile so it should have better connectivity than most organizations. The plot shows lack of connectivity (black), losses (colored background, see the legend below) and median RTT. The loss of connectivity following the earthquake at 6:34am UTC on February 27th is apparent. Connectivity was re-established at about 9:30pm (UTC) that evening followed by considerable instability.
UNIMASWorkshop, Sarawak, Dec 2012
![Page 72: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/72.jpg)
72
Syria shuts down Internet, Nov-29-12• Renesys reported:
– Starting at 10:26 UTC on Thursday, 29 November (12:26pm in Damascus), Syria's international Internet connectivity shut down.
• Prior to this working well
SLAC to Thawra Online
100
200
0
mse
c
12:00:0000:00:0012:00:0000:00:0012:00:00
SLAC to iNET
100
200
0
mse
c
Source: PingER
29 Nov ‘12
Damascus
iNET
![Page 73: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/73.jpg)
73
Other uses
UNIMASWorkshop, Sarawak, Dec 2012
![Page 74: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/74.jpg)
74
Trouble shooting
• Identify when RTT etc. changed (or did not)
• Has congestion increased (jitter, diurnal changes)
• Is it worse than similar paths
• Is it related to a route change?– Also keep daily traceroutes
• Identify that it is no longer a problem
• Identify problem sites to focus on
UNIMASWorkshop, Sarawak, Dec 2012
![Page 75: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/75.jpg)
75
Choosing ISP
• Choosing a service provider given a choice:– For SLAC ESnet vs. Internet2
• Got ISP to change routing to keep our business
– Choice of a DSL provider for residential coverage in Bay Area
UNIMASWorkshop, Sarawak, Dec 2012
![Page 76: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/76.jpg)
76
Comparison to UN ITU ICT Development Index
UNIMASWorkshop, Sarawak, Dec 2012
100
1000
10000
ICT Development Index from the UN International Telecommunications Union
2 4 6 810
Pin
gER
Der
ived
thr
ough
put
(kbi
ts/s
ec)
Bubble size = Population
Demo
![Page 77: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/77.jpg)
77
Malaysian and Brunei sites monitored from UNIMAS
eGY Africa 2012Workshop, Nairobi Oct 2012Demo
![Page 78: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/78.jpg)
78
UNIMAS to SE. Asia
eGY Africa 2012Workshop, Nairobi Oct 2012
CambodiaThailand
Philippines
![Page 79: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/79.jpg)
79
Viewing data
eGY Africa 2012Workshop, Nairobi Oct 2012
• Pingtable.pl– Select metric, tick, source, destination etc– Table downloadable for Excel– Details on hosts, and graphs of performance
• Table of country to country connections
• Map– See monitors, beacons, remotes– Select metric, src, dst, draw colored lines
connecting, colors = metric value– Graphs of metric and pinger hosts per country
• Maps of metric performance by Country
• Google Explorer and motion charts
![Page 80: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/80.jpg)
80
Managing• Site map
– Introductions
• Meta database– Oracle database => perl require script and
pinger.xml files
• Data gathering status
• Measurements:– Run from cronjob each 30 mins
• Spotting anomalies, hosts with same IP addr, hosts with missing information (country, lat/lon
• Finding hostseGY Africa 2012
Workshop, Nairobi Oct 2012
![Page 81: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/81.jpg)
81
Problems• Hosts change IP address
– Maybe v frequent, e.g. a cluster or slowly as host with same name is replaced with a new host with new IP address.
• Hosts not where you think they are
• No packet loss how to calculate Throughput ~ 1460(bytes)*8(bits)/(RTT*sqrt(loss))
• Throughput approx. only good for TCP Reno– OS’ such as Windows, Linux now allow other
congestion control algorithms
eGY Africa 2012Workshop, Nairobi Oct 2012
![Page 82: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/82.jpg)
82
Joining • ~ 85 active monitors worldwide
– https://confluence.slac.stanford.edu/display/IEPM/Installation+Overview gives the state
• Invitation letter for monitoring sites– http://www-iepm.slac.stanford.edu/pinger/letters/invite-monit
or.doc
• Host requirements– http://www-iepm.slac.stanford.edu/pinger/tools/mon-req.html
• Download site for application etc.– Traceroute/ping server– Pinger2.pl measuring engine– https://confluence.slac.stanford.edu/display/IEPM/Installation
+Overview
![Page 83: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/83.jpg)
83
Demo• Interactive demonstrations of the data mining
capabilities of public data sources provided by organizations such as the UN and ITU coupled with monitoring data from PingER
• http://www-iepm.slac.stanford.edu/pinger/explorer.html
Summer Joint TechsStanford, July 2012
![Page 84: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/84.jpg)
84 UNIMASWorkshop, Sarawak, Dec 2012
More Information• PingER web home page
– http://www-iepm.slac.stanford.edu/pinger/
• Tutorial on network monitoring & PingER– http://www.slac.stanford.edu/comp/net/wan-mon/tutorial.html
• PingER data Explorer– www-iepm.slac.stanford.edu/pinger/explorer.html
• PingER Project site map– http://www-iepm.slac.stanford.edu/pinger/site.html
• Invitation to join– www-iepm.slac.stanford.edu/pinger/letters/invite-
monitor.doc
![Page 85: PingER: Actively measuring the worldwide Internet’s end-to-end performance](https://reader035.vdocuments.us/reader035/viewer/2022062520/568158fa550346895dc63588/html5/thumbnails/85.jpg)
85 eGY Africa 2012Workshop, Nairobi Oct 2012