henp networks and grids for global vos: from vision to reality
Post on 09-Jan-2016
34 Views
Preview:
DESCRIPTION
TRANSCRIPT
HENP HENP Networks and Grids for Networks and Grids for Global VOs: from Vision to Reality Global VOs: from Vision to Reality
Harvey B. NewmanHarvey B. Newman California Institute of TechnologyCalifornia Institute of Technology
GNEW2004GNEW2004March 15, 2004March 15, 2004
ICFA and Global Networks ICFA and Global Networks for HENPfor HENP
ICFA and Global Networks ICFA and Global Networks for HENPfor HENP
National and International Networks, with sufficient National and International Networks, with sufficient (rapidly increasing) capacity and seamless end-to-end (rapidly increasing) capacity and seamless end-to-end capability, are essential forcapability, are essential for The daily conduct of collaborative work in bothThe daily conduct of collaborative work in both
experiment and theory experiment and theory Detector development & construction Detector development & construction
on a global scale on a global scale Grid systems supporting analysis involving Grid systems supporting analysis involving
physicists in all world regions physicists in all world regions The conception, design and implementation of The conception, design and implementation of
next generation facilities as “global networks” next generation facilities as “global networks” ““Collaborations on this scale would never have been Collaborations on this scale would never have been
attempted, if they could not rely on excellent networks”attempted, if they could not rely on excellent networks”
The Challenges of Next Generation Science in the Information Age
The Challenges of Next Generation Science in the Information Age
Flagship Applications Flagship Applications High Energy & Nuclear Physics, AstroPhysics Sky Surveys:High Energy & Nuclear Physics, AstroPhysics Sky Surveys:
TByte to PByte “block” transfers at 1-10+ Gbps TByte to PByte “block” transfers at 1-10+ Gbps eVLBI:eVLBI: Many real time data streams at 1-10 Gbps Many real time data streams at 1-10 Gbps BioInformatics, Clinical Imaging:BioInformatics, Clinical Imaging: GByte images on demand GByte images on demand
HEP Data Example: HEP Data Example: From Petabytes in 2004, ~100 Petabytes by 2008, From Petabytes in 2004, ~100 Petabytes by 2008,
to ~1 Exabyte by ~2013-5. to ~1 Exabyte by ~2013-5. Provide results with rapid turnaround, coordinating Provide results with rapid turnaround, coordinating
large but limited computing and data handling resources,large but limited computing and data handling resources,over networks of varying capability in different world regionsover networks of varying capability in different world regions
Advanced integrated applications, such as Data Grids, Advanced integrated applications, such as Data Grids, rely on seamless operation of our LANs and WANsrely on seamless operation of our LANs and WANs With reliable, quantifiable high performanceWith reliable, quantifiable high performance
Petabytes of complex data explored and analyzed by Petabytes of complex data explored and analyzed by 1000s of globally dispersed scientists, in hundreds of teams1000s of globally dispersed scientists, in hundreds of teams
Four LHC Experiments: The Four LHC Experiments: The Petabyte to Exabyte Petabyte to Exabyte
ChallengeChallengeATLAS, CMS, ALICE, LHCBATLAS, CMS, ALICE, LHCB
Higgs + New particles; Quark-Gluon Plasma; CP ViolationHiggs + New particles; Quark-Gluon Plasma; CP Violation
Tens of PB 2008; To 1 EB by ~2015Tens of PB 2008; To 1 EB by ~2015 Hundreds of TFlops To PetaFlops Hundreds of TFlops To PetaFlops
6000+ Physicists & Engineers; 60+ Countries;
250 Institutions
LHC Data Grid HierarchyLHC Data Grid Hierarchy
Tier 1
Tier2 Center
Online System
CERN Center PBs of Disk;
Tape Robot
FNAL CenterIN2P3 Center INFN Center RAL Center
InstituteInstituteInstituteInstitute
Workstations
~100-1500 MBytes/sec
2.5-10 Gbps
0.1 to 10 GbpsTens of Petabytes by 2007-8.An Exabyte ~5-7 Years later.Physics data cache
~PByte/sec
~10 Gbps
Tier2 CenterTier2 CenterTier2 Center
~2.5-10 Gbps
Tier 0 +1
Tier 3
Tier 4
Tier2 Center Tier 2
Experiment
CERN/Outside Resource Ratio ~1:2Tier0/( Tier1)/( Tier2) ~1:1:1
Emerging Vision: A Richly Structured, Global Dynamic System
ICFA Standing Committee on Interregional Connectivity (SCIC)
ICFA Standing Committee on Interregional Connectivity (SCIC)
Created by ICFA in July 1998 in VancouverCreated by ICFA in July 1998 in Vancouver CHARGE: CHARGE:
Make recommendations to ICFA concerning the connectivity Make recommendations to ICFA concerning the connectivity between between the Americasthe Americas, Asia and Europe, Asia and Europe As part of the process of developing theseAs part of the process of developing these
recommendations, the committee should recommendations, the committee should Monitor traffic Monitor traffic Keep track of technology developmentsKeep track of technology developments Periodically review forecasts of future Periodically review forecasts of future
bandwidth needs, and bandwidth needs, and Provide early warning of potential problemsProvide early warning of potential problems
Representatives: Major labs, ECFA, ACFA, North Representatives: Major labs, ECFA, ACFA, North and South American Physics Community and South American Physics Community
SCIC in 2003-2004 http://cern.ch/icfa-scic SCIC in 2003-2004 http://cern.ch/icfa-scic
Strong Focus on the Digital Divide Since 2002Strong Focus on the Digital Divide Since 2002
Three 2004 Reports; Presented to ICFA Feb. 13Three 2004 Reports; Presented to ICFA Feb. 13
Main Report: “Networking for HENP”Main Report: “Networking for HENP” [H. Newman et al.] [H. Newman et al.] Includes Brief Updates on Monitoring, the Digital Divide Includes Brief Updates on Monitoring, the Digital Divide
and Advanced Technologies [*]and Advanced Technologies [*] A World Network Overview (with 27 Appendices): A World Network Overview (with 27 Appendices):
Status and Plans for the Next Few YearsStatus and Plans for the Next Few Years of National and Regional Networks, of National and Regional Networks, and Optical Network Initiatives and Optical Network Initiatives
Monitoring Working Group ReportMonitoring Working Group Report [L. Cottrell] [L. Cottrell] Digital Divide in RussiaDigital Divide in Russia [V. Ilyin] [V. Ilyin]
[*] [*] Also See the 2003 SCIC Reports of the Advanced Also See the 2003 SCIC Reports of the Advanced Technologies and Digital Divide Working GroupsTechnologies and Digital Divide Working Groups
ICFA Report: Networks for HENPICFA Report: Networks for HENPGeneral Conclusions (1)General Conclusions (1)
ICFA Report: Networks for HENPICFA Report: Networks for HENPGeneral Conclusions (1)General Conclusions (1)
Bandwidth Usage Continues to Grow by 80-100% Per YearBandwidth Usage Continues to Grow by 80-100% Per Year Current generation of 2.5-10 Gbps backbones and major Int’l links Current generation of 2.5-10 Gbps backbones and major Int’l links
used by HENP arrived in the last 2 Years [US+Europe+Japan+Korea]used by HENP arrived in the last 2 Years [US+Europe+Japan+Korea] Capability Increased from ~4 to several hundred times, Capability Increased from ~4 to several hundred times,
i.e. much faster than Moore’s Law i.e. much faster than Moore’s Law This is a direct result of the continued precipitous fall This is a direct result of the continued precipitous fall
of network prices for 2.5 or 10 Gbps links in these regions of network prices for 2.5 or 10 Gbps links in these regions Technological progress may drive BW higher, unit price lowerTechnological progress may drive BW higher, unit price lower
More wavelengths on a fiber; Cheap, widespread Gbit EthernetMore wavelengths on a fiber; Cheap, widespread Gbit Ethernet Grids may accelerate this growth, and the demand for Grids may accelerate this growth, and the demand for
seamless high performanceseamless high performance Some regions are moving to owned or leased dark fiber Some regions are moving to owned or leased dark fiber The rapid rate of progress is confined mostly to the US, Europe, The rapid rate of progress is confined mostly to the US, Europe,
Japan , Korea, and the major TransAtlantic and Pacific routes Japan , Korea, and the major TransAtlantic and Pacific routes This may worsen the problem of the This may worsen the problem of the Digital DivideDigital Divide
History of Bandwidth Usage – One Large Network; One Large Research Site
History of Bandwidth Usage – One Large Network; One Large Research Site
ESnet Monthly Accepted Traffic 1/90-1/04
0
50
100
150
200
250
300
Jan,
90Oc
t, 90
Jul,9
1Ap
r, 92
Jan,
93Oc
t,93
Jul, 9
4Ap
r, 95
Jan,
96Oc
t, 96
Jul, 9
7Ap
r, 98
Jan,
99Oc
t, 99
Jul, 0
0Ap
r, 01
Jan,
02Oc
t, 02
Jul, 0
3
TByte
/Mon
th
SLAC Traffic ~300 Mbps; ESnet Limit
Growth in Steps: ~ 10X/4 YearsProjected: ~2 Terabits/s by
~2014
ESnet Accepted Traffic 1/90 – 1/04Exponential Growth Since ’92;
Annual Rate Increased from 1.7 to 2.0X Per Year In the Last 5 Years
Internet Growth in the World At Large
Internet Growth in the World At Large
Derived throughputs in kbits/s for Aug '03August-03 AU CA EDU GOV CH DE DK HU IT UK JP NET ORG RU SU Avg
N America 234 716 9346 1993 322 344 305 329 337 405 197 3231 4121 278 148 1487Australasia 7833 72 258 224 134 132 164 279 194 96 939Balkans 149 233 276 365 1111 1544 939 4745 1093 1148 155 321 351 670 346 896Europe 148 234 316 380 1662 1714 1049 1840 1596 2538 164 390 367 720 298 895Baltics 126 183 265 331 644 1009 1693 851 896 804 149 298 313 1050 213 588Russia 129 170 220 225 508 593 912 545 587 561 321 285 244 809 768 458E Asia 142 196 219 209 143 163 157 152 147 181 3559 205 219 163 157 401M East 125 187 168 234 227 505 386 453 645 572 125 243 283 331 185 311L America 131 150 220 191 165 162 152 159 164 175 117 198 186 139 100 161S Asia 73 179 112 121Caucasus 87 87Central Asia 64 68 66Africa 40 56 27 44 32 31 38Avg 1002 205 898 418 455 754 699 1023 563 727 598 548 698 520 257 496
Amsterdam Internet Exchange Point Example
75-100% Growth Per Year
10 G
20 G
Avg
5 Minute
Max
Some Growth Spurts;Typically In Summer-
Fall
The Rate of HENP Network Usage Growth (~100% Per Year) is Similar to the World at Large
Bandwidth Growth of Int’l HENP Bandwidth Growth of Int’l HENP Networks (US-CERN Example)Networks (US-CERN Example)
Bandwidth Growth of Int’l HENP Bandwidth Growth of Int’l HENP Networks (US-CERN Example)Networks (US-CERN Example)
Rate of Progress >> Moore’s Law. Rate of Progress >> Moore’s Law. (US-CERN Example)(US-CERN Example) 9.6 kbps Analog9.6 kbps Analog (1985) (1985) 64-256 kbps Digital (1989 - 1994) [X 7 – 27]64-256 kbps Digital (1989 - 1994) [X 7 – 27] 1.5 Mbps Shared (1990-3; IBM) [X 160]1.5 Mbps Shared (1990-3; IBM) [X 160] 2 -4 Mbps2 -4 Mbps (1996-1998) [X 200-400] (1996-1998) [X 200-400] 12-20 Mbps (1999-2000) [X 1.2k-2k]12-20 Mbps (1999-2000) [X 1.2k-2k] 155-310 Mbps155-310 Mbps (2001-2) [X 16k – 32k] (2001-2) [X 16k – 32k] 622 Mbps622 Mbps (2002-3) [X 65k] (2002-3) [X 65k] 2.5 Gbps 2.5 Gbps (2003-4) [X 250k] (2003-4) [X 250k] 10 Gbps 10 Gbps (2005) [X 1M] (2005) [X 1M]
A factor of ~1M over a period of 1985-2005 A factor of ~1M over a period of 1985-2005 (a factor of ~5k during 1995-2005)(a factor of ~5k during 1995-2005)
HENP has become a leading applications driver, HENP has become a leading applications driver, and also a co-developer of global networks; and also a co-developer of global networks;
Note 10 Gbps
Connections to Poland,
Czech Republic, Hungary
Pan-European Multi-Gigabit Backbone (33 Countries)
February 2004
Planning Underway for “GEANT2” (GN2), Multi-Lambda Backbone, to Start In 2005
Core Capacity on Western European NRENs 2001-2003
15 European NRENs have made a step up to 1, 2.5 or 10 Gbps core capacity in the last 3
years
100M
1G
10G
Log Scale
TERENA Compendiumwww.terena.nl
Int’l Circuit ~ 5-10 GbpsDomestic Circuit 30 – 100 Mbps
SuperSINET 10 Gbps
SuperSINET 10 Gbps IP; Tagged VPNs Additional 1 GbE Inter-University Waves For HEP 4 X 2.5 Gb to NY; 10 GbE Peerings: to Abilene, ESnet and GEANT
SuperSINET in JAPAN: Updated Map Oct. 2003SuperSINET in JAPAN: Updated Map Oct. 2003
SURFNet5 in the Netherlands Fully Optical 10G IP NetworkSURFNet5 in the Netherlands Fully Optical 10G IP Network
Fully dual stack:Fully dual stack: IPv4 + IPv6 IPv4 + IPv6
65% of Customer 65% of Customer Base Connects Base Connects with Gigabit with Gigabit EthernetEthernet
Germany: 2003, 2004, 2005Germany: 2003, 2004, 2005
GWIN Connects 550 Universities, Labs, Other InstitutionsGWIN Connects 550 Universities, Labs, Other Institutions
GWIN: Q4/03GWIN: Q4/03 GWIN: Q4/04GWIN: Q4/04PlanPlan
XWIN: Q4/05XWIN: Q4/05((Dark Fiber Option)Dark Fiber Option)
Abilene - Upgrade Completed!Abilene - Upgrade Completed!
1660 km of Dark Fiber CWDM Links, up to 112 km.
1 to 4 Gbps (GbE)
August 2002: First NREN in Europe to establish Int’l GbE Dark Fiber Link, to Austria
April 2003 to Czech Republic.
Planning 10 Gbps Backbone; dark fiber link to Poland this year.
PROGRESS in SE Europe (Sk, Pl, Cz, Hu, …)
Romania: Improved to 34 to 155 Mbps in 2003; GEANT-Bucharest Link Improved: 155 to 622 Mbps.
Note: Inter-City Links Were Only 2 to 6 Mbps in 2002
Romania: Improved to 34 to 155 Mbps in 2003; GEANT-Bucharest Link Improved: 155 to 622 Mbps.
Note: Inter-City Links Were Only 2 to 6 Mbps in 2002
Timişoara
GEANT connection
RoEduNet2004
Plans for IntercityDark Fiber Backbone
Australia (AARnet): SXTransport Project in 2004
Australia (AARnet): SXTransport Project in 2004
Connect Major Australian Universities to 10 Gbps BackboneConnect Major Australian Universities to 10 Gbps Backbone Two 10 Gbps Research Links to the USTwo 10 Gbps Research Links to the US Aarnet/USLIC Collaboration on Net R&D Starting SoonAarnet/USLIC Collaboration on Net R&D Starting Soon
Connect Telescopesin Australia, Hawaii (Muana
Kea)
GLORIAD: Global Optical Ring (US-Russia-China)
GLORIAD: Global Optical Ring (US-Russia-China)
“ “Little Gloriad” (OC3) Launched January 12; to OC192 Little Gloriad” (OC3) Launched January 12; to OC192
Also Important for Intra-Russia Connectivity;Education and Outreach
ITER Distributed Ops.;Fusion-HEP Cooperation
National Lambda Rail (NLR)National Lambda Rail (NLR)
15808 Terminal, Regen or OADM siteFiber route
NLRComing Up Now; Initially 4 10G
WavelengthsFull Footprint
by ~4Q04 Internet2 HOPI Initiative (w/HEP) To 40 10G Waves in Future
Transition beginning now to optical, multi-wavelength Community owned or leased
fiber networks for R&E
PITPIT
PORPOR
FREFRE
RALRAL
WALWAL
NASNASPHOPHO
OLGOLGATLATL
CHICHI
CLECLE
KANKAN
OGDOGDSACSAC BOSBOSNYCNYC
WDCWDC
STRSTR
DALDAL
DENDEN
LAXLAX
SVLSVL
SEASEA
SDGSDG
JACJAC
18 State Dark Fiber Initiatives
In the U.S. (As of 3/04)
California (CALREN), Colorado (FRGP/BRAN)Connecticut Educ. Network,Florida Lambda Rail, Indiana (I-LIGHT), Illinois (I-WIRE), Md./DC/No. Virginia (MAX),Michigan, Minnesota, NY + New England (NEREN), N. Carolina (NC LambdaRail), Ohio (Third Frontier Net) Oregon, Rhode Island (OSHEAN), SURA Crossroads (SE U.S.), Texas,Utah, Wisconsin
FiberCO
The Move to Dark Fiber is Spreading
CA*net4 (Canada)Two 10G Waves Vancouver- Halifax
CA*net4 (Canada)Two 10G Waves Vancouver- Halifax
Interconnects Interconnects Regional Nets at 10GRegional Nets at 10G
Acts as Parallel Acts as Parallel Discipline-Oriented Discipline-Oriented Nets; 650 SitesNets; 650 Sites
Connects to US Nets Connects to US Nets at Seattle, Chicago at Seattle, Chicago and NYCand NYC
Third Nat’l Lambda Third Nat’l Lambda Later in 2004Later in 2004
User Controlled User Controlled Light Path SoftwareLight Path Software(UCLP) for “Lambda(UCLP) for “LambdaGrids”Grids”
Calgary
ReginaWinnipeg
OttawaMontreal
Toronto
Halifax
St. John’s
Fredericton
Charlottetown
Chicago
Seattle
New York
Edmonton
Saskatoon
Vancouver
Calgary
ReginaWinnipeg
OttawaMontreal
Toronto
Halifax
St. John’s
Fredericton
Charlottetown
Chicago
Seattle
New York
Edmonton
Saskatoon
Vancouver
SURFNet6 in the Netherlands 3000 km of Owned Dark FiberSURFNet6 in the Netherlands
3000 km of Owned Dark Fiber
40M Euro Project40M Euro ProjectScheduled Scheduled
Start Mid-2005;Start Mid-2005;Support Hybrid GridsSupport Hybrid Grids
* 9/01 105 Mbps 30 Streams: SLAC-IN2P3; 102 Mbps 1 Stream CIT-CERN9/01 105 Mbps 30 Streams: SLAC-IN2P3; 102 Mbps 1 Stream CIT-CERN 5/20/02 450-600 Mbps SLAC-Manchester on OC12 with ~100 Streams5/20/02 450-600 Mbps SLAC-Manchester on OC12 with ~100 Streams 6/1/02 290 Mbps Chicago-CERN One Stream on OC12 6/1/02 290 Mbps Chicago-CERN One Stream on OC12 9/02 850, 1350, 1900 Mbps Chicago-CERN 1,2,3 GbE Streams, 2.5G Link9/02 850, 1350, 1900 Mbps Chicago-CERN 1,2,3 GbE Streams, 2.5G Link 11/02 [LSR] 930 Mbps in 1 Stream California-CERN, and California-AMS11/02 [LSR] 930 Mbps in 1 Stream California-CERN, and California-AMS FAST TCP 9.4 Gbps in 10 Flows California-ChicagoFAST TCP 9.4 Gbps in 10 Flows California-Chicago 2/03 [LSR] 2.38 Gbps in 1 Stream California-Geneva (99% Link Use)2/03 [LSR] 2.38 Gbps in 1 Stream California-Geneva (99% Link Use) 5/03 [LSR] 0.94 Gbps IPv6 in 1 Stream Chicago- Geneva5/03 [LSR] 0.94 Gbps IPv6 in 1 Stream Chicago- Geneva TW & SC2003 [LSR]: 5.65 Gbps (IPv4), 4.0 Gbps (IPv6) GVA-PHX (11 kkm)TW & SC2003 [LSR]: 5.65 Gbps (IPv4), 4.0 Gbps (IPv6) GVA-PHX (11 kkm) 3/043/04 [LSR] 6.25 Gbps (IPv4) in 8 Streams LA-CERN [LSR] 6.25 Gbps (IPv4) in 8 Streams LA-CERN
HEP is Learning How to Use Gbps Networks Fully: Factor of ~500 Gain in Max. Sustained TCP Thruput
in 4 Years, On Some US+Transoceanic Routes
HEP is Learning How to Use Gbps Networks Fully: Factor of ~500 Gain in Max. Sustained TCP Thruput
in 4 Years, On Some US+Transoceanic Routes
Transatlantic Ultraspeed TCP TransfersTransatlantic Ultraspeed TCP TransfersThroughput Achieved: X50 in 2 yearsThroughput Achieved: X50 in 2 years
Juniper, HP
Level(3)Telehous
e
Terabyte Transfers by the Caltech-CERN Team: Across Abilene (Internet2) Chicago-LA,
Sharing with normal network traffic Oct 15: 5.64 Gbps IPv4 Palexpo-L.A. (10.9 kkm)
Peaceful Coexistence with a Joint Internet2-Telecom World VRVS Videoconference
Nov 18: 4.00 Gbps IPv6 Geneva-Phoenix (11.5 kkm)March 2004: 6.25 Gbps in 8 Streams (S2IO Interfaces)
Nov 19: 23+ Gbps TCP: Caltech, Nov 19: 23+ Gbps TCP: Caltech, SLAC, CERN, LANL, UvA, ManchesterSLAC, CERN, LANL, UvA, Manchester
HENP Major Links: Bandwidth Roadmap in Gbps
HENP Major Links: Bandwidth Roadmap in Gbps
Year Production Experimental Remarks
2001 0.155 0.622-2.5 SONET/SDH
2002 0.622 2.5 SONET/SDH DWDM; GigE Integ.
2003 2.5 10 DWDM; 1 + 10 GigE Integration
2005 10 2-4 X 10 Switch; Provisioning
2007 2-4 X 10 ~10 X 10; 40 Gbps
1st Gen. Grids
2009 ~10 X 10 or 1-2 X 40
~5 X 40 or ~20-50 X 10
40 Gbps Switching
2011 ~5 X 40 or
~20 X 10
~25 X 40 or ~100 X 10
2nd Gen Grids Terabit Networks
2013 ~Terabit ~MultiTbps ~Fill One Fiber
Continuing the Trend: ~1000 Times Bandwidth Growth Per Decade;A new DOE Science Network Roadmap: Compatible
HENP Lambda Grids:Fibers for Physics
HENP Lambda Grids:Fibers for Physics
Problem: Extract “Small” Data Subsets of 1 to 100 Terabytes Problem: Extract “Small” Data Subsets of 1 to 100 Terabytes from 1 to 1000 Petabyte Data Storesfrom 1 to 1000 Petabyte Data Stores
Survivability of the HENP Global Grid System, with Survivability of the HENP Global Grid System, with hundreds of such transactions per day (circa 2007-8)hundreds of such transactions per day (circa 2007-8)requires that each transaction be completed in a requires that each transaction be completed in a relatively short time. relatively short time.
Example: Take 800 secs to complete the transaction. ThenExample: Take 800 secs to complete the transaction. Then Transaction Size (TB)Transaction Size (TB) Net Throughput (Gbps)Net Throughput (Gbps) 1 101 10 10 10010 100 100 1000 (Capacity of 100 1000 (Capacity of
Fiber Today) Fiber Today) Summary: Providing Switching of 10 Gbps wavelengthsSummary: Providing Switching of 10 Gbps wavelengths
within ~2-4 years; and Terabit Switching within 5-8 years within ~2-4 years; and Terabit Switching within 5-8 years would enable “10G Lambda Grids with Terabyte transactions”,would enable “10G Lambda Grids with Terabyte transactions”,to fully realize the discovery potential of major HENP programs, to fully realize the discovery potential of major HENP programs, as well as other data-intensive research.as well as other data-intensive research.
ICFA Report: Networks for HENPICFA Report: Networks for HENPGeneral Conclusions (2)General Conclusions (2)
ICFA Report: Networks for HENPICFA Report: Networks for HENPGeneral Conclusions (2)General Conclusions (2)
Reliable high Reliable high End-to-end PerformanceEnd-to-end Performance of networked applications such as of networked applications such as large file transfers and Data Grids is required. Achieving this requires:large file transfers and Data Grids is required. Achieving this requires:
Removing local, last mile, and nat’l and int’l bottlenecks Removing local, last mile, and nat’l and int’l bottlenecks end-to-end, whether technical or political in origin. end-to-end, whether technical or political in origin.While National and International backbones have reached 2.5 to 10 Gbps While National and International backbones have reached 2.5 to 10 Gbps speeds in many countries, the bandwidths speeds in many countries, the bandwidths across borders, the countrysideacross borders, the countrysideor the city may be much less. or the city may be much less. This problem is very widespread in our community, with examples This problem is very widespread in our community, with examples stretching from China to South America to the Northeastern U.S. stretching from China to South America to the Northeastern U.S. Root causes for this vary, from lack of local infrastructure to Root causes for this vary, from lack of local infrastructure to unfavorable pricing policies. unfavorable pricing policies.
Upgrading campus infrastructures.Upgrading campus infrastructures. These are still not designed to support Gbps data transfers in most of HEP These are still not designed to support Gbps data transfers in most of HEP centers. One reason for the under-utilization of National and International centers. One reason for the under-utilization of National and International backbones, is the lack of bandwidth to groups of end-users inside the campus. backbones, is the lack of bandwidth to groups of end-users inside the campus.
End-to-end monitoring extending to all regions serving our community.End-to-end monitoring extending to all regions serving our community. A coherent approach to monitoring that allows physicists throughout A coherent approach to monitoring that allows physicists throughout our community to extract clear, unambiguous and inclusive information our community to extract clear, unambiguous and inclusive information is a prerequisite for this. is a prerequisite for this.
ICFA Report: Networks for HENPICFA Report: Networks for HENPGeneral Conclusions (3)General Conclusions (3)
ICFA Report: Networks for HENPICFA Report: Networks for HENPGeneral Conclusions (3)General Conclusions (3)
We must Remove Firewall Bottlenecks We must Remove Firewall Bottlenecks [Also at some Major HEP Labs][Also at some Major HEP Labs]
Firewall systems are so far behind the needsFirewall systems are so far behind the needs that they that they won’t match the data flow of Grid applications. The maximum won’t match the data flow of Grid applications. The maximum throughput measured across available products is limited to a throughput measured across available products is limited to a few X 100 Mbps !few X 100 Mbps !
It is urgent to address this issue by designing new architectures It is urgent to address this issue by designing new architectures that eliminate/alleviate the need for conventional firewallsthat eliminate/alleviate the need for conventional firewalls. For . For example, Point-to-point provisioned high-speed circuits as example, Point-to-point provisioned high-speed circuits as proposed by emerging Light Path technologies could remove proposed by emerging Light Path technologies could remove the bottleneck. the bottleneck.
With endpoint authentication [as in Grid AAA systems],With endpoint authentication [as in Grid AAA systems], for example, for example, the point-to-point paths are private, intrusion resistant circuits, the point-to-point paths are private, intrusion resistant circuits, so they should be able to bypass firewalls if the endpoint sitesso they should be able to bypass firewalls if the endpoint sites trust each other. trust each other.
The original Computational and Data Grid concepts are The original Computational and Data Grid concepts are largely stateless, open systems: known to be scalable largely stateless, open systems: known to be scalable
Analogous to the WebAnalogous to the WebThe classical Grid architecture had a number of implicit The classical Grid architecture had a number of implicit
assumptionsassumptions The ability to locate and schedule suitable resources,The ability to locate and schedule suitable resources,
within a tolerably short time (i.e. resource richness) within a tolerably short time (i.e. resource richness) Short transactions with relatively simple failure modesShort transactions with relatively simple failure modes
HENP Grids are HENP Grids are Data Intensive & Resource-ConstrainedData Intensive & Resource-Constrained 1000s of users competing for resources at dozens of sites1000s of users competing for resources at dozens of sites Resource usage governed by local and global policiesResource usage governed by local and global policies Long transactions; some long queuesLong transactions; some long queues
HENP HENP Stateful, End-to-end Monitored and Tracked ParadigmStateful, End-to-end Monitored and Tracked Paradigm Adopted in OGSA [Now WS Resource Framework]Adopted in OGSA [Now WS Resource Framework]
HENP Data Grids, and Now HENP Data Grids, and Now Services-Oriented GridsServices-Oriented Grids
The Move to OGSA and then The Move to OGSA and then Managed Integration SystemsManaged Integration Systems
Incr
ease
d f
un
ctio
nal
ity,
stan
dar
diz
atio
n
Time
Customsolutions
Open GridServices Arch
GGF: OGSI, …(+ OASIS, W3C)
Multiple implementations,including Globus Toolkit
Web services + …
Globus Toolkit
Defacto standardsGGF: GridFTP, GSI
X.509,LDAP,FTP, …
App-specificServices~Integrated Systems~Integrated Systems
Stateful; ManagedWeb Services
Resrc Framwk
Managing Global Systems: Dynamic Managing Global Systems: Dynamic Scalable Services ArchitectureScalable Services Architecture
MonALISA: http://monalisa.cacr.caltech.edu
““Station Server” Station Server” Services-enginesServices-engines at sites host many at sites host many“Dynamic Services”“Dynamic Services”Scales to Scales to
thousands of thousands of service-Instancesservice-Instances
Servers autodiscover Servers autodiscover and interconnect and interconnect dynamically to form dynamically to form a robust fabric a robust fabric
Autonomous agentsAutonomous agents
42Caltech GAE Team
Grid Analysis EnvironmentGrid Analysis Environment
Analysis Clients talk standard protocols to the “Grid Services Web Server”, a.k.a. the Clarens data/services portal, with a simple Web service API
The secure Clarens portal hides the complexity of the Grid Services from the client
Key features: Global Scheduler, Catalogs, Monitoring, and Grid-wide Execution service;Clarens servers forma Global Peer to peer Network
SchedulerCatalogs
Analysis Client
Grid ServicesWeb Server
ExecutionPriority
Manager
Grid WideExecutionService
DataManagement
Fully-ConcretePlanner
Fully-AbstractPlanner
Analysis Client
AnalysisClient
Virtual Data
ReplicaApplicationsMonitoring
Partially-AbstractPlanner
Metadata
HTTP, SOAP, XML/RPC
CLARENS: Web Services Architecture
Integrated hybrid experimental network, leveraging Transatlantic Integrated hybrid experimental network, leveraging Transatlantic R&D network partnerships; packet-switched + dynamic optical paths R&D network partnerships; packet-switched + dynamic optical paths 10 GbE across US and the Atlantic: NLR, DataTAG, TransLight, 10 GbE across US and the Atlantic: NLR, DataTAG, TransLight,
NetherLight, UKLight, etc.; Extensions to Japan, Taiwan, BrazilNetherLight, UKLight, etc.; Extensions to Japan, Taiwan, Brazil End-to-end monitoring; Realtime tracking and optimization; End-to-end monitoring; Realtime tracking and optimization;
Dynamic bandwidth provisioningDynamic bandwidth provisioning Agent-based services spanning all layers of the system, from the Agent-based services spanning all layers of the system, from the
optical cross-connects to the applications.optical cross-connects to the applications.
National Lambda RailNational Lambda Rail
UltraLight Collaboration:UltraLight Collaboration:http://ultralight.caltech.eduhttp://ultralight.caltech.edu
Caltech, UF, FIU, UMich, SLAC,FNAL,MIT/Haystack,CERN, UERJ(Rio), NLR, CENIC, UCAID,Translight, UKLight, Netherlight, UvA, UCLondon, KEK, Taiwan
Cisco, Level(3)
Flagship Applications(HENP, VLBI, Oncology, …)
Grid/Storage Management
Network Protocols &Bandwidth Management
En
d-t
o-e
nd
Mo
nit
ori
ng
Inte
llig
en
t A
ge
nts
En
d-t
o-e
nd
Mo
nit
ori
ng
Inte
llig
en
t A
ge
nts
Application Frameworks
Distributed CPU & Storage
Network Fabric
Grid Middleware
Flagship Applications(HENP, VLBI, Oncology, …)
Grid/Storage Management
Network Protocols &Bandwidth Management
En
d-t
o-e
nd
Mo
nit
ori
ng
Inte
llig
en
t A
ge
nts
En
d-t
o-e
nd
Mo
nit
ori
ng
Inte
llig
en
t A
ge
nts
Application Frameworks
Distributed CPU & Storage
Network Fabric
Grid Middleware
PITPIT
PORPOR
FREFRE
RALRAL
WALWAL
NASNASPHOPHO
OLGOLG ATLATL
CHICHICLECLE
KANKAN
OGDOGDSACSAC NYCNYC
WDCWDC
STRSTR
DALDAL
DENDEN
LAXLAX
SVLSVL
SEASEA
SDGSDG
JACJAC
GLIF: Global Lambda Integrated Facility
DWDM SURFnet
lambda service path
IP service path
10 Gbit/s
SURFnet10 Gbit/s
SURFnet10 Gbit/s
IEEAF10 Gbit/s
DwingelooASTRON/JIVE
DwingelooASTRON/JIVE
PragueCzechLight
PragueCzechLight
2.5 Gbit/s
NSF10 Gbit/s
LondonUKLightLondonUKLight
StockholmNorthernLightStockholm
NorthernLight
2.5 Gbit/s
New YorkMANLANNew YorkMANLAN
TokyoWIDETokyoWIDE
10 Gbit/s
10Gbit/s
10 Gbit/s
2x10 Gbit/s
IEEAF10 Gbit/s
2x10 Gbit/s
10 Gbit/s
2.5 Gbit/s
2.5 Gbit/s
TokyoAPANTokyoAPAN
GenevaCERN
GenevaCERN
ChicagoChicago AmsterdamAmsterdam
“GLIF is a World Scale Lambda based Lab for Application & Middleware development, where Grid applications ride on dynamically configured networks based on optical wavelengths ...
GLIF will use the Lambda network to support data transport for the most demanding e-Science applications, concurrent with the normal best effort Internet for commodity traffic.”
10 Gbps Wavelengths For R&E Network Development Are
Proliferating, Across Continents and Oceans
SCIC Report 2004 The Digital DivideSCIC Report 2004 The Digital Divide
As the pace of network advances continues to accelerate, As the pace of network advances continues to accelerate, the gap between the economically “favored” regions and the gap between the economically “favored” regions and the rest of the world is in danger of widening. the rest of the world is in danger of widening.
We must therefore workWe must therefore work to Close the Digital Divideto Close the Digital Divide To make Physicists from All World Regions Full Partners To make Physicists from All World Regions Full Partners
in Their Experiments; and in the Process of Discoveryin Their Experiments; and in the Process of Discovery This is essential for the health of our global This is essential for the health of our global
experimental collaborations, our plans for future experimental collaborations, our plans for future projects, and our field.projects, and our field.
SCIC Monitoring WG PingER (Also IEPM-BW) SCIC Monitoring WG PingER (Also IEPM-BW)
Measurements fromMeasurements from 33 monitors in 12 countries33 monitors in 12 countries 850 remote hosts in 100 850 remote hosts in 100
Countries; 3700 monitor-Countries; 3700 monitor-remote site pairsremote site pairs
Measurements go back to ‘95Measurements go back to ‘95 Reports on link reliability, Reports on link reliability,
qualityquality Aggregation in Aggregation in
“affinity groups”“affinity groups”
Countries monitoredCountries monitored Contain 78% of Contain 78% of
world populationworld population 99% of Internet users 99% of Internet users
Monitoring Sites
Affinity Groups (Countries)
Anglo America (2), Latin America (14), Europe (24), S.E. Europe (9), Africa (21), Mid East (7), Caucasus (3), Central Asia (8), Russia
includes Belarus & Ukraine (3), S. Asia (7), China (1) and Australasia (2).
SCIC Monitoring WG - Throughput Improvements 1995-2004
Bandwidth of TCP < MSS/(RTT*Sqrt(Loss)) (1)
(1) 1) Matthis et al., Computer Communication Review 27(3), July 1997Matthis et al., Computer Communication Review 27(3), July 1997
60% annual improvement
Factor ~100/10 yr
Progress: but Digital Divide is Mostly Maintained
Some Regions ~5-10 Years
Behind
SE Europe, Russia, Central
Asia May be Catching Up
(Slowly);
India Ever-Farther Behind
APAN Link Information 2004.1.05 sec@apan.netCountries Network Bandwidth(Mbps) AUP/Remark
AU-US AARNet 310 to 2 X 10 Gbps R&E + CommodityCN-HK CERNET/HARNET 2 R&ECN-HK CSTNET 155 R&ECN-JP CERNET 10 R&ECN-JP CERNET 45 Native IPv6CN-UK CERNET 45 R&ECN-US CERNET 10 Research CN-US CERNET 200 R&ECN-US CSTNET 155 R&EHK-US HARNET 45 R&EHK-TW HARNET/TANET 10 R&EJP-ID AI3(ITB) 2/1.5 R&EJP-KR APII 2Gbps R&EJP-MY AI3(USM) 1.5/0.5 R&EJP-PH AI3(ASTI) 1.5/0.5 R&EJP-PH MAFFIN 2 Research JP-SG AI3(SICU) 1.5/0.5 R&EJP-TH AI3(AIT) 1.5/0.5 R&EJP-TH SINET(ThaiSarn) 2 R&EJP-US TransPac 10 Gbps R&EJP-VN AI3(IOIT) 1.5/0.5 R&E
Inhomogeneous Bandwidth Distributioin Latin America. CAESAR Report (6/02)Inhomogeneous Bandwidth Distributioin Latin America. CAESAR Report (6/02)
Int’l Links0.071 Gbps
Used
4,236 Gbps Capacity to Latin America
Need to Pay Attentionto End-point connections
DAI: State of the WorldDAI: State of the World
DAI: State of the WorldDAI: State of the World
Digital Access Index Top Ten +Digital Access Index Top Ten +
Pakistan 0.03 0.54 0.41 0.2 0.01 0.24
Work on the Digital Dividefrom Several PerspectivesWork on the Digital Dividefrom Several Perspectives
Work on Policies and/or Pricing: pk, in, br, cn, SE Europe, …Work on Policies and/or Pricing: pk, in, br, cn, SE Europe, … Share Information: Comparative Performance and BW Pricing Share Information: Comparative Performance and BW Pricing Exploit Model Cases: e.g. Poland, Slovakia, Czech RepublicExploit Model Cases: e.g. Poland, Slovakia, Czech Republic Find Ways to work with vendors, NRENs, and/or Gov’tsFind Ways to work with vendors, NRENs, and/or Gov’ts
Inter-Regional Projects Inter-Regional Projects GLORIAD, Russia-China-US Optical RingGLORIAD, Russia-China-US Optical Ring South America: CHEPREO (US-Brazil); EU ALICE ProjectSouth America: CHEPREO (US-Brazil); EU ALICE Project Virtual SILK Highway Project (DESY): FSU satellite linksVirtual SILK Highway Project (DESY): FSU satellite links
Help with Modernizing the Infrastructure Help with Modernizing the Infrastructure Design, Commissioning, DevelopmentDesign, Commissioning, Development Provide Tools for Effective Use: Monitoring, CollaborationProvide Tools for Effective Use: Monitoring, Collaboration
Participate in Standards Development; Open ToolsParticipate in Standards Development; Open Tools Advanced TCP stacks; Grid systemsAdvanced TCP stacks; Grid systems
Workshops and Tutorials/Training SessionsWorkshops and Tutorials/Training Sessions Example: ICFA Digital Divide and HEPGrid Workshop in RioExample: ICFA Digital Divide and HEPGrid Workshop in Rio
Raise General Awareness of the Problem; Approaches to SolutionsRaise General Awareness of the Problem; Approaches to Solutions WSIS/RSIS; Trieste WorkshopWSIS/RSIS; Trieste Workshop
International Connectivity Costs in the Different European Market Segments
Market segment
Number of Countries
Cost Range
Liberal Market with transparent pricing
8 1-1.4
Liberal Market with less transparent pricing structure
7 1.8-3.3
Emerging Market without transparent pricing
3 7.5-7.8
Traditional Monopolist market
9 18-39
Dai Davies SERENATE Workshop Feb. 2003
Ratio to 114 If Include Turkey, Malta;Correlated with the Number of Competing Vendors
www.serenate.org
Virtual Silk HighwayVirtual Silk Highway Hub Earth Station at DESY with
access to the European NRENs and the Internet via GEANT Providing International
Internet access directly National Earth Station at each
Partner site Operated by DESY, providing
international access Individual uplinks, common
down-link, using DVB Currently 4 Mbps Up; 12
Down; for ~$ 350k/Year
The SILK Highway Countriesin Central Asia & the Caucasus
Note: Satellite Links are a Boon to the Region, but Unit Costs are Very High compared to Fiber.
There is a Continued Need for Fiber Infrastructure
A Series of Strategic Studies into the Future of Research and Education Networking in Europe
A significant “divide” exists in Europe – the worst countries [Macedonia, B-H, Albania, etc.] are 1000s of times worse off than the best. Also many of the 10 new EU members are ~5X worse off
than the 15 present members. If there is one single technical lesson from SERENATE it is that
transmission is moving from the electrical domain to optical. When there’s good competition users can still lease traditional
communications services (bandwidth) on an annual basis. But: Without enough competition prices go through the roof.
The more you look at underlying costs the more you see the need for users to get access to fibre.
Our best advice has to be “If you’re in a mess, you must get access to fibre”.
From Summary and Conclusions by D.O. Williams, CERN
See www.serenate.org
Dark Fiber in Eastern Europe Poland: PIONIER Network
Dark Fiber in Eastern Europe Poland: PIONIER Network
ŁÓDŹ
TORUŃ
P OZNAŃ
B YDGOS ZCZ
OLS ZTYN
B IAŁYS TOK
GDAŃS K
KOS ZALIN
S ZCZECIN
ZIELONAGÓRA
WROCŁAW
CZĘS TOCHOWA
KRAKÓW RZES ZÓW
LUB LIN
KIELCE
P UŁAWY
RADOM
KATOWICEGLIWICE
B IELS KO-B IAŁA
OP OLE
GUB IN WARS ZAWA
CIES ZYN
S IEDLCE
P IO NIE R n o d e sIn s ta lle d fib e r
F ib e rs p la n n e d in 2 0 0 4P IO NIE R n o d e s p la n n e d in 2 0 0 4
2650 km Fiber 2650 km Fiber ConnectingConnecting
16 MANs; 5200 km 16 MANs; 5200 km and 21 MANs by 2005 and 21 MANs by 2005
Support Support Computational GridsComputational Grids
Domain-Specific Domain-Specific Grids Grids
Digital LibrariesDigital Libraries Interactive TVInteractive TV
Add’l Fibers for Add’l Fibers for e-Regional Initiatives e-Regional Initiatives
CESNET Dark Fiber Case Study (Czech Republic): Within ReachCESNET Dark Fiber Case Study (Czech Republic): Within Reach
Case Study ResultCase Study ResultWavelength ServiceWavelength Service
Vs. Fiber Lease:Vs. Fiber Lease:
Cost Savings of Cost Savings of 50-70% Over 4 Years50-70% Over 4 Years
for Long 2.5G for Long 2.5G or 10G Linksor 10G Links
For Example: 4 X 10G For Example: 4 X 10G Over 300 km Over 300 km
for 14k Euro/Month for 14k Euro/Month
2513 km 2513 km Leased Fibers Leased Fibers (Since 1999) (Since 1999)
1 x 2,5G Leased 1 x 2,5G
(EURO/Month)about 150km (e.g. Ústí n.L. - Liberec) 7,000
about 300km (e.g. Praha - Brno) 8,000
***
4 x 2,5G Leased 4 x 2,5G
(EURO/Month)about 150km (e.g. Ústí n.L. - Liberec) 14,000
about 300km (e.g. Praha - Brno) 23,000
***
1 x 10G Leased 1 x 10G
(EURO/Month)about 150km (e.g. Ústí n.L. - Liberec) 14,000
about 300km (e.g. Praha - Brno) 16,000
***
4 x 10G Leased 4 x 10G
(EURO/Month)about 150km (e.g. Ústí n.L. - Liberec) 29,000
about 300km (e.g. Praha - Brno) 47,000
***
2 x booster 24dBm, 2 x DCF, DWDM 10G2 x (booster +In-line + preamplifier), 6 x DCF, DWDM 10G
Leased fibre with own equipment (EURO/Month)
5 000 *7 000 **
2 x booster 18dBm 2 x booster 27dBm + 2 x preamplifier + 6 x DCF
Leased fibre with own equipment (EURO/Month)
12 000 *14 000 **
8 000 **
2 x booster 21dBm, 2 x DCF2 x (booster 21dBm + in-line + preamplifier) + 6 x DCF
2 x booster 24dBm, DWDM 2,5G2 x (booster +In-line + preamplifier), 6 x DCF, DWDM 2,5G
Leased fibre with own equipment (EURO/Month)
5 000 *
Leased fibre with own equipment (EURO/Month)
8 000 *11 000 **
NEWS: Bulletin: ONE TWOWELCOME BULLETIN General InformationRegistrationTravel Information Hotel RegistrationParticipant List How to Get UERJ/Hotel Computer AccountsUseful Phone Numbers ProgramContact us: Secretariat Chairmen
CLAF CNPQ FAPERJ UERJ
SPONSORS
HEPGRID and Digital Divide Workshop
UERJ, Rio de Janeiro, Feb. 16-20 2004
Theme: Global Collaborations, Grids and Their Relationship to the Digital
Divide
For the past three years the SCIC has focused on understanding and seeking the means of reducing or eliminating the Digital Divide, and
proposed to ICFA that these issues, as they affect our field of High
Energy Physics, be brought to our community for discussion.
This led to ICFA’s approval, in July 2003, of the Digital Divide and H
EP Grid Workshop. http://www.uerj.br/lishep2004
TutorialsC++Grid TechnologiesGrid-Enabled
AnalysisNetworksCollaborative Systems
All Sessions andTutorials Available
Live Via VRVS
World Summit on the Information Society(WSIS): Geneva 12/2003 and Tunis in 2005 World Summit on the Information Society
(WSIS): Geneva 12/2003 and Tunis in 2005 The UN General Assembly adopted in 2001 a resolution The UN General Assembly adopted in 2001 a resolution
endorsing the organization of the endorsing the organization of the World Summit on the World Summit on the Information Society (WSIS),Information Society (WSIS), under UN Secretary-General, under UN Secretary-General, Kofi Annan, with the ITU and host governments taking Kofi Annan, with the ITU and host governments taking the lead role in its preparation. the lead role in its preparation.
GOAL: To Create an Information Society:GOAL: To Create an Information Society: A Common Definition was adoptedA Common Definition was adoptedin the “Tokyo Declaration” of January 2003:in the “Tokyo Declaration” of January 2003:“… One in which highly developed ICT networks, equitable “… One in which highly developed ICT networks, equitable and ubiquitous access to information, appropriate content and ubiquitous access to information, appropriate content in accessible formats and effective communication can in accessible formats and effective communication can help people achieve their potential”help people achieve their potential”
Kofi Annan Challenged the Scientific Community to Help (3/03) Kofi Annan Challenged the Scientific Community to Help (3/03) CERN and ICFA SCIC have been quite active in the WSIS in CERN and ICFA SCIC have been quite active in the WSIS in
Geneva (12/2003) Geneva (12/2003)
Role of Science in the Information Society. Palexpo, Geneva 2004
Role of Science in the Information Society. Palexpo, Geneva 2004
CERN SIS Forum, and CERN SIS Forum, and CERN/Caltech Online CERN/Caltech Online
StandStand Visitors:Visitors:
Kofi AnnanKofi Annan, UN Sec’y General , UN Sec’y General John H. MarburgerJohn H. Marburger, ,
Science Adviser to US PresidentScience Adviser to US PresidentIon IliescuIon Iliescu, President of , President of
Romania; Romania; and Dan Nica, Minister and Dan Nica, Minister of ICTof ICT
Jean-Paul Hubert, Ambassador Jean-Paul Hubert, Ambassador of Canada in Switzerland of Canada in Switzerland
Carlo Lamprecht, Pres. of Carlo Lamprecht, Pres. of Economic Dept. of Canton Economic Dept. of Canton
de Genevade Geneva … …
Role of Sciences in Information Society. Palexpo, Geneva 2003
Role of Sciences in Information Society. Palexpo, Geneva 2003
Demos at the CERN/Caltech Demos at the CERN/Caltech RSIS RSIS Online StandOnline Stand
Distance diagnosis and surgery using Distance diagnosis and surgery using Robots with “haptic” feedback Robots with “haptic” feedback (Geneva-Canada)(Geneva-Canada)
World Scale multisite multi-protocol World Scale multisite multi-protocol videoconference with VRVSvideoconference with VRVS
(Europe-US-Asia-South America) (Europe-US-Asia-South America) Music Grid: live performance with Music Grid: live performance with
bands at St. John’s, Canada and the bands at St. John’s, Canada and the Music Conservatory of Geneva on Music Conservatory of Geneva on stagestage
Advanced network and Grid-enabled Advanced network and Grid-enabled analysis demonstrationsanalysis demonstrations
Monitoring very large scale Grid farms Monitoring very large scale Grid farms with MonALISAwith MonALISA
VRVS on WindowsVRVS on WindowsVRVS (Version 3)VRVS (Version 3)Meeting in 8 Time Meeting in 8 Time
ZonesZones
28k registered hosts28k registered hostsUsers in 103 CountriesUsers in 103 Countries
2-3X Growth/Year2-3X Growth/Year
Networks, Grids and HENPNetworks, Grids and HENP Network backbones and major links used by HENP experimentsNetwork backbones and major links used by HENP experiments
are advancing rapidly are advancing rapidly To the 10 G range in < 2 years; much faster than Moore’s LawTo the 10 G range in < 2 years; much faster than Moore’s Law Continuing a trend: a factor ~1000 improvement per decade; Continuing a trend: a factor ~1000 improvement per decade;
a new HENP (and DOE) Roadmap a new HENP (and DOE) Roadmap HENP is learning to use long distance 10 Gbps networks effectively HENP is learning to use long distance 10 Gbps networks effectively
2003-2004 Developments: to 6 Gbps flows over 11 kkm2003-2004 Developments: to 6 Gbps flows over 11 kkm Transition to community-owned and operated R&E networks Transition to community-owned and operated R&E networks
is beginning (ca, nl, us, pl, cz, sk …) or considered (de, ro, …) is beginning (ca, nl, us, pl, cz, sk …) or considered (de, ro, …) Removing Regional, Last Mile, Local Bottlenecks and Removing Regional, Last Mile, Local Bottlenecks and
Compromises in Network Quality are nowCompromises in Network Quality are now On the critical path, in all world regionsOn the critical path, in all world regions
Digital Divide: Network improvements are especially neededDigital Divide: Network improvements are especially neededin SE Europe, Much of Asia, So. America; and Africain SE Europe, Much of Asia, So. America; and Africa
Work on These Issues in Concert with Internet2, Terena, APAN, Work on These Issues in Concert with Internet2, Terena, APAN, AMPATH; DataTAG, the Grid projects & the GGFAMPATH; DataTAG, the Grid projects & the GGF
Computing Model Progress CMS Internal Review of Software and Computing
Some Extra Slides Some Extra Slides
FollowFollow
ICFA and International Networking ICFA and International Networking
ICFA Statement on Communications in Int’l HEPICFA Statement on Communications in Int’l HEPCollaborations of October 17, 1996Collaborations of October 17, 1996 See See http://www.fnal.gov/directorate/icfa/icfa_communicaes.htmlhttp://www.fnal.gov/directorate/icfa/icfa_communicaes.html
““ICFA urges that all countries and institutions wishing ICFA urges that all countries and institutions wishing to participate even more effectively and fully in to participate even more effectively and fully in international HEP Collaborations should:international HEP Collaborations should: Review their operating methods to ensure they Review their operating methods to ensure they
are fully adapted to remote participation are fully adapted to remote participation Strive to provide the necessary communications Strive to provide the necessary communications
facilities and adequate international bandwidth” facilities and adequate international bandwidth”
NREN Core Network Size (Mbps-km):http://www.terena.nl/compendium/2002
NREN Core Network Size (Mbps-km):http://www.terena.nl/compendium/2002
Logarithmic Scale
1k
100k
100
100M
10M
1M
10kRo
It
PlGrIr
Ukr
Hu Cz
Es
Nl
Fi
Ch
Lagging
In Transition
Leading
Advanced
Network Readiness Index:How Ready to Use Modern ICTs [*]?
Network Readiness Index:How Ready to Use Modern ICTs [*]?
NetworkReadiness
Index
Environment
Readiness
Usage
Market
Infrastructure
Political/Regulatory
Individual Readiness
Gov’t Readiness
Business Readiness
Individual Usage
Gov’t Usage
Business Usage
(FI)
(FI)
(SG)
(US)(IC)
(SG)
(US)
(SG)
(US)
(FI)
(FI)
(DE)
(KR)
( ): Which Country is First
From the 2002-2003 Global InformationTechnology Report. See http://www.weforum.org
Throughput vs Net Readiness Index Throughput vs Net Readiness Index NRI from Center for Int’l Development, Harvard NRI from Center for Int’l Development, Harvard http://www.cid.harvard.edu/cr/pdf/gitrr2002_ch02.pdfhttp://www.cid.harvard.edu/cr/pdf/gitrr2002_ch02.pdf
Improved correlation (0.21 Improved correlation (0.21 0.41) by Using derived throughput ~0.41) by Using derived throughput ~ MSS / (RTT * sqrt(loss)); fit an exponential MSS / (RTT * sqrt(loss)); fit an exponential
Interesting Outliers: Slovakia, Hungary, Portugal. LithuaniaInteresting Outliers: Slovakia, Hungary, Portugal. Lithuania
Intern
et for all fo
cusA
&R
focus
NRI TopsNRI TopsFinland Finland 5.925.92US US 5.795.79SingaporeSingapore 5.745.74Sweden Sweden 5.585.58Iceland Iceland 5.515.51Canada Canada 5.445.44UKUK 5.355.35Denmark Denmark 5.335.33TaiwanTaiwan 5.315.31GermanyGermany 5.295.29Netherlnd Netherlnd 5.285.28Israel Israel 5.225.22Switz’land Switz’land 5.185.18Korea Korea 5.105.10
UltraLight UltraLight
http://ultralight.caltech.edu Serving the major LHC experiments; developments Serving the major LHC experiments; developments broadly applicable to other data-intensive programs broadly applicable to other data-intensive programs ““Hybrid” packet-switched and circuit-switched, Hybrid” packet-switched and circuit-switched,
dynamically managed optical networkdynamically managed optical network Global services for system managementGlobal services for system management Trans-US wavelength riding on NLR: LA-SNV-CHI-JAXTrans-US wavelength riding on NLR: LA-SNV-CHI-JAX Leveraging advanced research & production networksLeveraging advanced research & production networks
USLIC/DataTAG, SURFnet/NLlight, UKLight, USLIC/DataTAG, SURFnet/NLlight, UKLight, Abilene, CA*net4 Abilene, CA*net4
Dark fiber to CIT, SLAC, FNAL, UMich; Florida Light RailDark fiber to CIT, SLAC, FNAL, UMich; Florida Light Rail Intercont’l extensions: Rio de Janeiro, Tokyo, TaiwanIntercont’l extensions: Rio de Janeiro, Tokyo, Taiwan
Flagship Applications with a diverse traffic mixFlagship Applications with a diverse traffic mix HENP: TByte to PByte “block” data transfers at 1-10+ HENP: TByte to PByte “block” data transfers at 1-10+
GbpsGbps eVLBI: Real time data streams at 1 to several GbpseVLBI: Real time data streams at 1 to several Gbps
User Requirements
In ALL countries and in ALL disciplines researchers are eagerly anticipating improved networking tools. There is no divide on the demand-side. Sciences, such as particle physics, which make heavy use of advanced networking, must help to break down any divide on the supply-side, or else declare themselves elitist and irrelevant to researchers in essentially all developing countries.
Connectivity pricing and competition
In some locations the price of connectivity is (really) unreasonably high
Linked (obviously) to how competitive the market is Strong competition on routes between various key European
cities, and between major national centres Less competition effectively none as you move to countries
with de facto monopoly or simply to parts of countries where operators see little reason to invest.
While some expensive routes are where you would expect, others are much more surprising (at first sight), like Canterbury and Lancaster (UK) and parts of Brittany (F)
Understanding transmission costs and DIY solutions
Own trenching only makes sense in very special cases. Say 1-30 km. Even then look for partners.
Maybe useful (as a threat) over longer distances in countries with crazy pricing
Now possible to lease (short- or long-term) fibres on many routes in Europe [0.5 to 2 KEuro/km Typ.]
Transmission costs jump at ~200 km [below which you can operate with “Nothing In Line” (NIL), above which you need amplifiers] and ~800 km [above which you need signal regenerators]
Possibly leading to some new approaches in GEANT-2 implementation
Jensen, ICTP
Typ. 0-7 bpsPer Person
Limited by many externalsystemic
factors: Electricity;
Import Duties;Education; Trade
restrictions
Jensen, ICTP
Progress in Africa ?
DAI: State of the WorldDAI: State of the World
DAI: State of the WorldDAI: State of the World
RENATER3 in France2.5 G Backbone (Since 10/2002)
RENATER3 in France2.5 G Backbone (Since 10/2002)
650 Sites, Most Connect 650 Sites, Most Connect Through MANs Through MANs
< 50 Direct Connected< 50 Direct Connected GEANT Connection to GEANT Connection to
10G in March 2004 10G in March 2004 IN2P3 InvestigatingIN2P3 Investigating
Dark Fiber to CERNDark Fiber to CERN
APANAPANAPAN Link Map
Access PointExchange PointCurrent Status2004 (plan)
USA
Europe
AU
ID
VN
LK
KR
TH
MY
SG
TW
RU
PH
CN
HK
JP
APAN Members
ASEM Member:China, Japan, Korea, Malaysia, Philippines, Singapore, Thail and
Non-ASEM Member:
Australia, Taiwan, Hong Kong, Sri Lanka
Applying APAN Membership:
Bangladesh, Nepal
Courtesy: Prof. Kilnam Chon (APAN Chair)
AMPATH Miami-RNP Brazil
AMPATH Miami-REUNA (Chile)
Note: CMS-Tier1 (Brazil);ALMA (Chile)
Brazil: RNP in Early 2004Brazil: RNP in Early 2004
NREN Core Network Size (Mbps-km):Last Year
http://www.terena.nl/compendium/2002
NREN Core Network Size (Mbps-km):Last Year
http://www.terena.nl/compendium/2002Logarithmic Scale
1k
100k
100
100M
10M
1M
10kRo
It
PlGrIr
Ukr
Hu Cz
Es
Nl
Fi
Ch
Lagging
In Transition
Leading
Advanced
Relative Cost of Connectivity Compared with Number of Suppliers
0
5
10
15
20
25
30
35
40
45
0 2 4 6 8 10 12 14
Number of suppliers
Relative cost
Trend Line
Multipliers for Differing Circuit Speeds
0
0.2
0.4
0.6
0.8
1
1.2
050010001500200025003000
Speed In MBps
Multiplier
PingER derived throughput vs. the ITU Digital Access Index (DAI) for PingER countries
monitored from the U.S.
PingER derived throughput vs. the ITU Digital Access Index (DAI) for PingER countries
monitored from the U.S.
NRNs’ Bandwidth in Latin AmericaCountry Organization Existing REN? National connections External
CapacityNumber of
Connected SitesConnection
to USInternet2
Argentina RETINA yes 256Kbps – 34 Mbps 59 Mbps 56 yes
Bolivia BOLnet yes 64 – 128 Kbps 1.5 Mbps 18 no
Brazil RNPyes
2 – 30 Mbps (backbone up to 622 Mbps
202 Mbps369
yes
Chile REUNAyes 155 Mbps
45 Mbps via AMPATH
18yes
Colombia RedCETCol Not known Not known Not known Not known Not known
Costa Rica CRNet yes 32 – 512 Kbps Not known 34 no
Cuba RedUniv University Network
19.2 Kbps – 2 MbpsNot known
23no
Ecuador FUNDACYT In planning no
El Salvador CONACYT In planning no
Guatemala Not known Non-existent no
Honduras HONDUnet Not known no
Nicaragua Not known no
Panama PANNET/SENACYT
University/Gov. Network
256 – 512 Kbps 1.54 Mbps 11 no
Peru CONCYTEC In planning no
Uruguay RAU yes 64 Kbps to 1 Mbps 6 Mbps 46 no
Venezuela REACCIUN yes 45 Mbps via AMPATH
78 January 2003
Source:
CAESAR - Review of Develop-ments in Latin America
Undersea Optical InfrastructureSubmarine Fiber-Optic Cable
SystemTotal Bandwidth Capacity (GB)
Americas 1 .560
Americas II 2.5
South American Crossing 1,280
Columbus II .560
Columbus III 2.5
Telefonica’s Emergia 1,920
ARCOS 960
Maya-1 60
360 Americas 10
The total aggregate bandwidth capacity Latin America and Caribbean region is estimated at 4,236 GB
It is generally accepted that once a technology is perceived as having broad utilitarian value, price as a % of per capita income, is the main driver of penetration
C. Casasus, CUDI (Mexico); CANARIE
Penetration of telecommunications in low income countries is further inhibited by at least 3 factors…
Low income per capitaLess competition. Higher prices
from monopoliesFewer applications. No broad
utilitarian value
C. Casasus, CUDI (Mexico); CANARIE
Income vs. penetration given the price of a technology
Per capitaIncome
% Penetration
High income countries
Low
High
Low High
Low income countries
C. Casasus, CUDI (Mexico); CANARIE
Telecom monopolies have even higher prices in low income countries
Fewer entrants. Less competition No unbundling Price cap regulation creates cross
subsidies between costumer groups. Large customers (inelastic) subsidize
small costumers (elastic). High bandwidth services are very expensive
Inefficient ROW regulation Inefficient spectrum policies
C. Casasus, CUDI (Mexico); CANARIE
Virtual Silk Highway Project Managed by DESY and Partners
Virtual SILK Highway Project (from 11/01): Virtual SILK Highway Project (from 11/01): NATO ($ 2.5 M) and Partners ($ 1.1M) NATO ($ 2.5 M) and Partners ($ 1.1M) Satellite Links to South Caucasus Satellite Links to South Caucasus and Central Asia (8 Countries) and Central Asia (8 Countries)
In 2001-2 (pre-SILK) BW 64-512 kbpsIn 2001-2 (pre-SILK) BW 64-512 kbpsProposed VSAT to get 10-50 X BWProposed VSAT to get 10-50 X BW for same cost for same cost
See See www.silkproject.orgwww.silkproject.org[*] [*] Partners: CISCO, DESY. GEANT, Partners: CISCO, DESY. GEANT, UNDP, US State Dept., Worldbank, UNDP, US State Dept., Worldbank, UC London, Univ. Groenigen UC London, Univ. Groenigen
Note: NATO Science for Peace Program
top related