visualizing the digital divide from an internet point of view & challenges prepared by: les...
Post on 28-Dec-2015
221 Views
Preview:
TRANSCRIPT
Visualizing the Digital Divide from an Internet Point of View & Challenges
Prepared by: Les CottrellSLAC
Umar KalimNIIT,Shahryar KhanNIIT, Akbar MehdiNIIT
For COMSATS University, Islamabad, Pakistan, March 14, 2007
http://www.slac.stanford.edu/grp/scs/net/talk07/comsats-mar07.ppt
Outline• Digital Divide:
– Examples of effect of Digital Divide & why it matters– How we measure it– What we find
• A network challenge for mathematicians, statisticians
Why Does it Matter
4. Sep 05, international fibre to Pakistan fails for 12 days, satellite backup can only handle 25% traffic, call centres given priority. Research & Education sites cut off from Internet for 12 days
Heloise Emdon, Acacia Southern
AfricaUNDP Global Meeting for ICT for
Development, Ottawa 10-13 July
3. Primary health care giver, somewhere in Africa, with sonar machine, digital camera and arrangement with national academic hospital and/or international health institute to assist in diagnostics. After 10 dial-up attempts, she abandons attempts to connect
1. School in a secondary town in an East Coast country with networked computer lab spends 2/3rds of its annual budget to pay for the dial-up connection.– Disconnects
2. Telecentre in a country with fairly good connectivity has no connectivity– The telecentre resorts to generating revenue from photocopies,
PC training, CD Roms for content.
How do we measure it?
• PingER project• Arguably the world’s most extensive active end-to-
end Internet Performance Project
PingER Methodology
Internet
10 ping request packets each 30 mins
RemoteHost(typicallya server)
Monitoring host
>ping remhost
Ping response packets
Measure Round Trip Time & Loss
Data Repository @ SLAC
On
ce a Day
Uses ubiquitous ping
Architecture• Monitor hosts send 21 pings each 30 mins to Remote
Hosts and cache results• Archive hosts gather data daily, save, analyze & make
results available publicly via web
PingER Deployment• PingER project originally (1995) to measure network
performance for US, Europe and Japanese HEP community• Extended this century to measure Digital Divide:
– Collaboration with ICTP Science Dissemination Unit http://sdu.ictp.it – ICFA/SCIC: http://icfa-scic.web.cern.ch/ICFA-SCIC/
• Monitor 44 sites in S. Asia
• >120 countries (99% world’s connected population)• >30 monitor sites in 14 countries
World Measurements: Min RTT from US• Maps show increased coverage • Min RTT indicates best possible, i.e. no queuing• >600ms probably geo-stationary satellite• Between developed regions min-RTT dominated by
distance– Little improvement possible
• Only a few places still using satellite for international access, mainly Africa & Central Asia
2000 2006
Effect of Losses• Losses critical, cause multi-second timeouts
• Typically depend on a bad link, so ~distance independent
• > 4-6% video-conf irritating, non-native language speakers unable to communicate
• > 4-5% irritating for interactive telnet, X windows
• >2.5% VoIP annoying every 30 seconds or so
• Burst losses of > 1% slightly annoying for VoIP
Losses from SLAC to world
>=12%>=5% <12%
>=2.5% < 5%>=1% < 2.5%
< 1%
• # hosts monitored increased seven-fold• Increase in fraction with good loss
– Despite adding more hosts in developing world
Loss Improvement by Population
• Loss by country weighted by population of country
Unreachability • All pings of a set fail ≡ unreachable
• Shows fragility, ~ distance independent
• Developed regions US, Canada, Europe, Oceania, E Asia lead– Factor of 10 improvement in 8 years
• Africa, S. Asia followed by M East & L. America worst off
• Africa NOT improving
US & CanadaEurope
E Asia
C Asia
SE Europe
SE Asia
S AsiaOceania
Africa
L America M East
Russia
DevelopedRegions
DevelopingRegions
World thruput seen from US
Behind Europe6 Yrs: Russia, Latin America 7 Yrs: Mid-East, SE Asia10 Yrs: South Asia11 Yrs: Cent. Asia12 Yrs: Africa
South Asia, Central Asia, and
Africa are in Danger of Falling
Even Farther Behind
Throughput ~1460Bytes /(RTT*sqrt(loss))(Mathis et al)
Normalized for Details• Note step
changes• Africa v.
poor• S. Asia
improving• N. America,
Europe, E Asia, Oceania lead
Overall (Aug 06)• ~ Sorted by Average throughput• Within region performance better (black ellipses)• Europe, N. America, E. Asia generally good• M. East, Oceania, S.E. Asia, L. America acceptable• C. Asia, S. Asia poor, Africa bad (>100 times worse)
Mo
nit
ore
d C
ou
ntr
y
South Asia• Population
S Asia Bandwidth & Internet use• Note Log scale for BW• India region leader• Pakistan leads bw/pop• Nepal very poor
• Pakistan leads % users• Sri Lanka leads hosts%
%• Pakistan leads bw/pop• Nepal, Bangladesh,
Afghanistan very poor
S Asia PingER Coverage
• Monitor 44 sites in region.• 6 Monitoring hosts (3 ea in India &
Pakistan)
Loss from CERN
Min-RTT from CERN
• Divides into 2– India, Maldives, Pakistan, Sri Lanka– Bangladesh, Nepal, Bhutan, Afghanistan
• Weekend vs. weekday indicates heavy congestion
Derived thruput
Digital Access Index (DAI): Infrastructure availability, Affordability of access,
Education, Quality of ICT, & Internet usageEurope, E Asia (except China), Oceania top right
Israel & Singapore with top group
Middle East in middle, Iran poorest
Africa bottom left
S. Asia split: Bhutan, Nepal, Bangladesh with Africa
India, Pak, Sri Lanka better
Strong positive linear correlation,
C Asia
DAI & S. Asia
D.D. Conclusions• Last mile problems, and network fragility• Decreasing use of satellites, expensive, but still needed
for many remote countries in Africa and C. Asia• Africa ~ 10 years behind and falling further behind,
leads to “information famine”• E. Africa factor of 100 behind Europe
– EASSy project will bring fibre to E. Africa, hopefully better access than SAT3
• Africa big target of opportunity– Growth in # users 2000-2005 200%, Africa 625% – Need more competitive pricing
• Fibre competition, government divest for access, low cost VSAT licenses
• Consortiums to aggregate & get better pricing ($/BW reduces with BW)– Need better routing - IXPs– Need training & skills for optimal bandwidth management
• Internet performance correlates strongly with UNDP & ITU development indices– Increase coverage of monitoring to understand Internet performance
Challenge, however…• Elegant graphics are great to understand problems
BUT:– Can be thousands of graphs to look at (many site pairs,
many devices, many metrics)– Need automated problem recognition AND diagnosis
• So developing tools to reliably detect significant, persistent changes in performance– Initially using simple plateau algorithm to detect step
changes
• Some are seasonal• Others are not• Events may affect
multiple-metrics
• Misconfigured windows• New path• Very noisy
Examples of real data
• Seasonal effects– Daily & weekly
Caltech: thrulay
Nov05
Mar060
800Mbps
UToronto: miperf
Nov05
Jan060
250
Mbps
UTDallas Pathchirp
thrulay
Mar-10-06 Mar-20-06iperf0
120
Mbps
• Events can be caused by host or site congestion• Few route changes result in bandwidth changes (~20%)• Many significant events are not associated with route
changes (~50%)
Changes in network topology (BGP) can result in dramatic changes in performance
Snapshot of traceroute summary table
Samples of traceroute trees generated from the table
ABwE measurement one/minute for 24 hours Thurs Oct 9 9:00am to Fri Oct 10 9:01am
Drop in performance(From original path: SLAC-CENIC-Caltech to SLAC-Esnet-LosNettos (100Mbps) -Caltech )
Back to original path
Changes detected by IEPM-Iperf and AbWE
Esnet-LosNettos segment in the path(100 Mbits/s)
Hour
Rem
ote
host
Dynamic BW capacity (DBC)
Cross-traffic (XT)
Available BW = (DBC-XT)
Mbit
s/s
Notes:1. Caltech misrouted via Los-Nettos 100Mbps commercial net 14:00-17:002. ESnet/GEANT working on routes from 2:00 to 14:003. A previous occurrence went un-noticed for 2 months4. Next step is to auto detect and notify
Los-Nettos (100Mbps)
On the other hand• Route changes may affect the RTT (in yellow)• Yet have no noticeable effect on on available bandwidth or
throughput
Route changes
AvailableBandwidth
AchievableThroughput
Seasonal Effects on events• Change in bandwidth (drops) between 19:00 &
22:00 Pacific Time (7:00-10:00am PK time)
• Causes more anomalous events around this time
Forecasting• Over-provisioned
paths should have pretty flat time series
– Short/local term smoothing
– Long term linear trends– Seasonal smoothing
• But seasonal trends (diurnal, weekly need to be accounted for) on about 10% of our paths
• Use Holt-Winters triple exponential weighted moving averages
Econometrics• Econometrists use forecasting techniques for
predicting the behavior of economic metrics– Auto Regressive Integrated Moving Average (ARIMA &
ARMA)– Very mathematical, multiple techniques:
• Integration to make stationary• Auto-regression• Moving Averages • Determining parameters etc. can be an art
– Our (Fareena Saqib) first look at was promising• Have a long document of how far we got
– Do not currently have someone working on next steps.
Experimental Alerting• Have false positives down to reasonable level (few
per week), so sending alerts to developers
• Saved in database
• Links to traceroutes, event analysis, time-series
More information/Questions• Acknowledgements:
– Harvey Newman and ICFA/SCIC for a raison d’etre, ICTP for contacts and education on Africa, Mike Jensen for Africa information, NIIT/Pakistan for developing valuable tools, Maxim Grigoriev (FNAL), Warren Matthews (GATech) for ongoing code development for PingER, USAID MoST/Pakistan for development funding, SLAC for support for ongoing management/operations support of PingER
• PingER– www-iepm.slac.stanford.edu/pinger, sdu.ictp.it/pinger/africa.html
• Human Development– http://www.gapminder.org/
• Case Studies:
– https://confluence.slac.stanford.edu/display/IEPM/Sub-Sahara+Case+Study
– http://sdu.ictp.it/lowbandwidth/program/case-studies/index.html
Extra Slides Follow
Costs compared to West• Sites in many countries have bandwidth< US residence
– “10 Meg is Here”, www.lightreading.com/document.asp?doc_id=104415
• Africa: $5460/Mbps/m– W Africa $8K/Mbps/m– N Africa $520/Mbps/m
• Often cross-country cost dominates cf. international
1 yr of Internet access > average annual income of most Africans, Survey by Paul Budde Communnications
UNDP Human Development Index (HDI)
• A long and healthy life, as measured by life expectancy at birth
• Knowledge, as measured by the adult literacy rate (with two-thirds weight) and the combined primary, secondary and tertiary gross enrolment ratio (with one-third weight)
• A decent standard of living, as measured by GDP per capita. Africa
PingER- Strong Correlation- Non subjective- Quicker / easier to update
Med. & Africa vs HDI• N. Africa has 10 times poorer performance than Europe• Croatia has 13 times better performance than Albania• Israel has 8 times better performance than rest of M East
Med. Countries• E. Africa poor,
limited by satellite access
• W. Africa big differences, some (Senegal) can afford SAT3 fibre others use satellite
• Great diversity between & within regions
Why does it matter: Business•G8 specifically pledged support for African higher education and research by “Helping develop skilled professionals for Africa's private and public sectors, through supporting networks of excellence between African's and other countries' institutions of higher education and centres of excellence in science and technology institutions” G8 specifically pledged support for African higher education and research by “Helping develop skilled professionals for Africa's private and public sectors, through supporting networks of excellence between African's and other countries' institutions of higher education and centres of excellence in science and technology institutions”
•Saturating western markets•High growth IT markets: BRIC•NOT business as usual
– New business models– Distinct needs– Dearth of distribution channels
Traditional MNCBusiness Model >$20K per year
75 to 100 million people
Some MNCs
>$1,500 - 20K per year1.5 to 1.75 billion people
Local Firms
<$1,500 per year4 billion people
Future Opportunity?
Prahalad and Hart
Karen CoppockRDVP, Stanford
top related