Download - OptIPuter Goal: Removing Bandwidth Barriers to e-Science ATLAS Sloan Digital Sky Survey LHC ALMA
OptIPuter Goal:Removing Bandwidth Barriers to e-Science
ATLAS
Sloan Digital Sky Survey
LHC
ALMA
Why Optical NetworksWill Become the 21st Century Driver
Scientific American, January 2001
Number of Years0 1 2 3 4 5
Pe
rfo
rma
nc
e p
er
Do
llar
Sp
en
t
Data Storage(bits per square inch)
(Doubling time 12 Months)
Optical Fiber(bits per second)
(Doubling time 9 Months)
Silicon Computer Chips(Number of Transistors)
(Doubling time 18 Months)
The OptIPuter Project – Removing Bandwidth as an Obstacle In Data Intensive Sciences
• NSF Large Information Technology Research Proposal– UCSD and UIC Lead Campuses—Larry Smarr PI– USC, UCI, SDSU, NW, TA&M Partnering Campuses
• Industrial Partners: IBM, Sun, Telcordia/SAIC, Chiaro, Calient• $13.5 Million Over Five Years• Optical IP Streams From Lab Clusters to Large Data Objects NIH Biomedical Informatics Research Network
NSF EarthScope
http://ncmir.ucsd.edu/gallery.html
siovizcenter.ucsd.edu/library/gallery/shoot1/index.shtml
Application Barrier One:Gigabyte Data Objects Need Interactive Visualization
• Montages--Hundred-Million Pixel 2-D Images– Microscopy or Telescopes
– Remote Sensing
• GigaZone 3-D Objects– Seismic or Medical Imaging
– Supercomputer Simulations
• Interactive Analysis and Visualization of Such High Resolution Data Objects Requires: – Scalable Visualization Displays– Montage and Volumetric Visualization Software
– UIC EVL’s JuxtaView and Vol-a-Tile
OptIPuter Project Goal:Scaling to 100 Million Pixels
• JuxtaView (UIC EVL) on PerspecTile LCD Wall– Digital Montage
Viewer – 8000x3600 Pixel
Resolution~30M Pixels
• Display Is Powered By – 16 PCs with
Graphics Cards– 2 Gigabit Networking
per PC
Source: Jason Leigh, EVL, UIC; USGS EROS
NCMIR – Brain Microscopy
(2800x4000 24 layers)
Application Barrier Two:Campus Grid Infrastructure is Inadequate
• Campus Infrastructure is Designed for Web Objects– Being Swamped by Sharing of Digital Multimedia Objects– Little Strategic Thinking About Needs of Data Researchers
• Challenge of Matching Storage to Bandwidth– Need To Ingest And Feed Data At Multi-Gbps– Scaling to Enormous Capacity – Use Standards-Based Commodity Clusters (Rocks)
• OptIPuter Aims at Prototyping a National Architecture– Federated National and Global Data Repositories– Lambdas on Demand– Campus Laboratories Using Clusters with TeraBuckets– Campus Eventually with a Shared PetaCache
OptIPuter 2004 @ UCSDCoupling Linux Clusters with High Resolution Visualization
10
1
4
102
ChiaroEnstara
op-nodes-ucsd-y1.5 9/26/03 -grh
SDSCJSOESDSC
Annex
CSE 2
Fiber toCRCA
Fiber to6th College
44Bonded
GigE
1
Dell 5224
8-node cluster(shared)
8-node cluster(shared)
PreussSchool
DellGeowall
IBM 9-nodeViz Cluster
SIO SOM
Dell 5224
Sun 32-nodecomputecluster
Sun 32-nodecomputecluster
IBM 48-nodeStorageCluster21TB
Dell 5224
IBM 128-nodeCompute Cluster
(shared)
100-node cluster(shared)
Dell 5224
1
Sun 32-nodeStoragecluster
Sun 128-nodecompute(shared)
8-node cluster(shared)
2
4-nodecontrol
Sun 32-nodecomputecluster
GigE Switch10GigE Uplink
GigE Switch10GigE Uplink
UCSD 6509Shared VLAN
OptIPuter is Studying the Best Application Usagefor Both Routed vs. Switched Lambdas
• OptIPuter Evaluating Both:– Routers
– Chiaro, Juniper, Cisco, Force10
– Optical Switches– Calient, Glimmerglass
– Lightpath Accelerators– BigBandWidth
• UCSD Focusing on Routing Initially• UIC Focusing on Switching Initially• Next Year Merge into Mixed Optical Fabric
ChiaroEstara
Glimmerglass
Application Barrier Three:Shared Internet Makes Interactive Gigabyte Impossible
• NASA Earth Observation System– Over 100,000 Users Pull Data from Federated Repositories
– Two Million Data Products Delivered per Year
– 10-50 Mbps (May 2003) Throughput to Campuses– Typically Over Abilene From Goddard, Langley, or EROS
• Biomedical Informatics Research Network (BIRN) – Between UCSD and Boston– Similar Story– Lots of Specialized Networking Tuning Used
– 50-80 Mbps
• Remote Interactive Megabyte is Possible • But Interactive Gigabyte is Impossible
IP over Lambdas with Alternate Protocols
Multi-Latency OptIPuter LaboratoryNational-Scale Experimental Network
Source: Tom West, CEO NLR (Booth 3409)
Chicago OptIPuter
StarLightNU, UIC
SoCalOptIPuter
USC, UCI UCSD, SDSU
2000 Miles 10 ms
=1000x Campus Latency
“National Lambda Rail” PartnershipServes Very High-End Experimental and Research Applications
4 x 10GB Wavelengths Initially Capable of 40 x 10Gb wavelengths at Buildout
An International-Scale OptIPuter is Operational over the First Set of 76 International GE TransLight Lambdas
European lambdas to US–8 GEs Amsterdam— Chicago–8 GEs London—ChicagoCanadian lambdas to US–8 GEs Chicago—Canada —NYC–8 GEs Chicago—Canada —SeattleUS lambdas to Europe–4 GEs Chicago—Amsterdam–3 GEs Chicago—CERNEuropean lambdas–8 GEs Amsterdam—CERN –2 GEs Prague—Amsterdam–2 GEs Stockholm—Amsterdam–8 GEs London—AmsterdamTransPAC lambda–1 GE Chicago—TokyoIEEAF lambdas (blue)–8 GEs NYC—Amsterdam–8 GEs Seattle—Tokyo
UKLight
CERN
NorthernLight
Source: Tom DeFanti, EVL, UIC