cenic: pacific wave and prp update big news for big data
TRANSCRIPT
Pacific Wave and PRP UpdateBig News for Big Data
David ReeseDr. Larry Smarr
CENIC ANNUAL CONFERENCE 2016
Pacific Wave• Began as first geographically distributed exchange in 2004• Pacific Wave is an open exchange supporting both
commercial and R&E peers• Currently serves 29 countries peering across the Pacific
and Western United States• With PNWGP and TransPac, announced the first 100Gbps
Trans-Pacific link from Tokyo to Seattle in 2015
R&E Exchanges within R&E• Pacific Wave (Western US)
– CENIC and PNWGP• StarLight (Chicago, IL)
– StarLight Consortium/MREN• MANLAN (New York, NY)
– NYSERnet• WIX (Washington, DC)
– University of Maryland/MAX GigaPOP• AmLight (Miami, Florida)
– Florida International University/Florida LambdaRail
National/Global Activities• NSF provides support of the R&E exchange points
through the competitive IRNC (International Research Network Connections) program with funding for backbone, infrastructure and innovation
• The Global Lambda Integrated Facility– The GLIF brings together some of the world’s premier
networking engineers who are working together to develop and international infrastructure
Pacific Wave and NSF/IRNC• Pacific Wave has been partially supported
through three separate five-year National Science Foundation grants supporting growth, connectivity and innovation
• Current award promotes 100G expansion and implementation of SDX capabilities within Pacific Wave (ACI-1451050)
Pacific Waveand WRN
• Pacific Wave and the Western Region Network provide for a 100Gbps network spanning the Western United States serving PNWGP, CENIC, FRGP, ABQGP and UH.
• Pacific Wave and NSF IRNC awardee PIREN (Univ of Hawaii) work together supporting AARNet links to California and Washington and expansion of high-speed service through the Pacific Islands Region
Pacific Wave and Collaboration• Pacific Wave and StarLight are working together
on the SDX implementation to provide seamless cross-domain support for participants of both exchanges.
• Have direct interconnectivity between the exchanges with 2 100G links between Seattle-Chicago and Denver-Chicago
Nx100G Across the Pacific• CURRENT:
– TransPac/Pacific Wave (Tokyo-Seattle)– SINGAREN/Internet2 (Singapore-Los Angeles)– SINET/SoftBank/Pacific Wave (Tokyo-Los Angeles)
• FUTURE:– AARNET/PIREN/Pacific Wave (Australia-LA)– AARNET/PIREN/Pacific Wave (Australia-SEA)– UH/PIREN/Pacific Wave (Guam-Hawaii-LA)
The Pacific Research Platform (PRP)• NSF CC-NIE and similar projects represent significant investments in campus
infrastructure including SDN, DMZ’s (~130 projects)
• But the scientists are still struggling with the complexity of using the network and interoperability between different implementations of DMZ’s
• PRP focuses on enabling the science communities to make effective use of the high performance infrastructure that is available.
• The idea was hatched in December 2014 – take advantages of the infrastructure, including a PERFSONAR grid for measurement.
• And DTN’s and common software suite to demonstrate a proof of concept for the PRP
• Demonstrated at the CENIC Spring meeting (March 2015)
CENIC/PRP Backbone Sets Stage for 2016 Wireless Expansion of HPWREN into Orange and Riverside Counties
• CENIC/PRP Will Connect UCSD and SDSU
– Data Redundancy – Disaster Recovery – High Availability
• CENIC Extension to UCI & UCR– Data Replication Sites
UCR
UCI
UCSD
SDSU
Source: Frank Vernon, Greg Hidley, UCSD
“Pacific Wave & Pacific Research Platform Update:
Big News for Big Data”
Invited Presentation with David ReeseAnnual CENIC Conference 2016
UC DavisMarch 21, 2016
Dr. Larry SmarrDirector, California Institute for Telecommunications and Information Technology
Harry E. Gruber Professor, Dept. of Computer Science and Engineering
Jacobs School of Engineering, UCSDhttp://lsmarr.calit2.net 16
For Big Data Science, One Needs Bandwidths Orders of Magnitude Higher Than the Shared Internet
Bandwidth from My Office in Calit2’s Qualcomm Institute
Bandwidth On the Pacific Research Platform:
500 Times the Bandwidth of the Shared Internet!
How Prism@UCSD Transforms Big Data Microbiome Science:Preparing for Knight/Smarr 1 Million Core-Hour Analysis
12 Cores/GPU128 GB RAM3.5 TB SSD48TB Disk
10Gbps NIC
Knight Lab
10Gbps
Gordon
Prism@UCSD
Data Oasis7.5PB,
200GB/s
Knight 1024 ClusterIn SDSC Co-Lo
CHERuB100Gbps
Emperor & Other Vis Tools
64Mpixel Data Analysis Wall
120Gbps
40Gbps
1.3Tbps
Next Step: The Pacific Research Platform Creates a Regional End-to-End Science-Driven “Big Data Freeway System”
NSF CC*DNI Grant$5M 10/2015-10/2020
PI: Larry Smarr, UC San Diego Calit2Co-Pis:• Camille Crittenden, UC Berkeley CITRIS, • Tom DeFanti, UC San Diego Calit2, • Philip Papadopoulos, UC San Diego SDSC, • Frank Wuerthwein, UC San Diego Physics and
SDSC
Dan Cayan USGS Water Resources Discipline
Scripps Institution of Oceanography, UC San Diego
much support from Mary Tyree, Mike Dettinger, Guido Franco and other colleagues
NCAR Upgrading to 10Gbps Link from Wyoming and Boulder to CENIC/PRP
Sponsors: California Energy Commission NOAA RISA program California DWR, DOE, NSF
Planning for climate change in California substantial shifts on top of already high climate variability
UCSD Campus Climate Researchers Need to Download Results from NCAR Remote Supercomputer Simulations
to Make Regional Climate Change Forecasts
average summer afternoon temperatureaverage summer afternoon temperature
Downscaling Supercomputer Climate SimulationsTo Provide High Res Predictions for California Over Next 50 Years
21
Source: Hugo Hidalgo, Tapash Das, Mike Dettinger
Cancer Genomics Hub (UCSC) is Housed in SDSC:Large Data Flows to End Users at UCSC, UCB, UCSF, …
1G
8G
Data Source: David Haussler, Brad Smith, UCSC
15GJan 2016
Cancer Genomics Hub Users are Downloading 30,000 TB per Year
GE’s Industrial Internet Generates 10,000 TB per Day! 30,000 TBPer Year
Two Automated Telescope SurveysCreating Huge Datasets Will Drive PRP
300 images per night. 100MB per raw image
30GB per night
120GB per night
250 images per night. 530MB per raw image
150 GB per night
800GB per nightWhen processed
at NERSC Increased by 4x
Source: Peter Nugent, Division Deputy for Scientific Engagement, LBLProfessor of Astronomy, UC Berkeley
Precursors to LSST and NCSA
PRP Allows Researchersto Bring Datasets from NERSC
to Their Local Clusters for In-Depth Science Analysis
OSG Federates Clusters in 40/50 States:Creating a Scientific Compute and Storage “Cloud”
Source: Miron Livny, Frank Wuerthwein, OSG
We are Experimenting with the PRP for Large Hadron Collider Data Analysis Using The West Coast Open Science Grid on 10-100Gbps Optical Networks
Crossed 100 Million
Core-Hours/MonthIn Dec 2015
Over 1 Billion Data Transfers
Moved200 Petabytes
In 2015
Supported Over200 Million Jobs
In 2015
Source: Miron Livny, Frank Wuerthwein, OSG
ATLAS
CMS
PRP Will Support the Computation and Data Analysisin the Search for Sources of Gravitational Radiation
Connecting Caltech at 10Gb/s to SDSC Comet PetaFLOP supercomputer,
enabling LIGO computations to enter via the same PRP “job cache” as for LHC.
800,000 core-hours LIGO data analysis7 million core-hours computing waveforms
Forty Years of Computing Gravitational Waves From Colliding Black Holes
1977
L. Smarr and K. EppleyGravitational Radiation Computed from
an Axisymmetric Black Hole Collision 40 Years
2016
LIGO ConsortiumSpiral Black Hole Collision
MegaFLOPSPetaFLOPS
40G FIONAs
20x40G PRP-connected
WAVE@UC San Diego
PRP LinksCreates Distributed Virtual Reality
PRP
CAVE@UC Merced
UCD
UCSF
Stanford
NASAAMES/NREN
UCSC
UCSB
Caltech
USC UCLA
UCIUCSD SDSU
UCR
EsnetDoE Labs
UW/PNWGPSeattle
Berkeley
UCM
Los Nettos
Internet2
Internet2Seattle
Note: This diagram represents a subset of sites and connections.
* Institutions withActive Archaeology
Programs
“In an ideal world –Extremely high bandwidth to
move large cultural heritage datasets around the PRP cloud for
processing & viewing in CAVEs around PRP with Unlimited
Storage for permanent archiving.”-Tom Levy, UCSD
PRP is NOT Just for Big Data Science and Engineering:Linking Cultural Heritage and Archaeology Datasets
Building on CENIC’s ExpansionTo Libraries, Museums,
and Cultural Sites
Next Step: Global Research PlatformBuilding on CENIC/Pacific Wave and GLIF
Current InternationalGRP Partners
Learn More About PRP on Wednesday
PRP Science Driver PI Workshop Description
[Flyers will be available at the CENIC registration desk]
Date and Time: Wed., March 23 from 9 - 11 am
Location: Executive Meeting Room, Hyatt Place UC Davis
Description: Panel discussion on capacity and promise of the Pacific Research Platform, a high-speed, expanded cyberinfrastructure that will move data 1,000 times faster than today's inter-campus shared Internet.