International Research Computing “The convergence of next generation computing
and next generation networking”
Bill St. Arnaud
CANARIE Inc – www.canarie.ca
Who is CANARIE?
> Mandate to develop Canada’s next generation Internet> Private, not for profit corporation funded by government of
Canada> Partners include major telephone companies, Cisco, Nortel,
Alcatel, Ericsson, IBM, universities, research centers etc> We build and operate Canada’s national research and
education network – CA*net 4
Bringing CANARIE South to California
New 72 channel x 40 Gbps ROADM Networks
Boston
San Diego
Amsterdam
10 Gbps Wave from CENIC
CA*net 4 Network details
> CA*net 4 is NOT a single homogenous network> CA*net 4 is made up of many virtual networks (APNs) dedicated
to different communities and applications on a common substrate
– High energy physics network– Network for government research labs– Network for distributed computer backplane– Virtual networks for network research
> One of the parallel networks is a general purpose IP network– This is the only network that carries IPv6– All other networks only carry IPv4
> New ROADM will allow us offer 10G wavelengths for $25k per year
Carleton University
School of Architecture
PhysicsDepartment
Engineering Telecom
Firewall
Main campusNetwork
BorderRouter
University
Internet
Eucalyptus*Design Network
GlobalPhysicsNetwork
ResearchTestbed
NREN
Campus CWDM
10G
1G
1G
1G
10G
*Will connect to UCLA Architecture and Optiputer
University of British Columbia
Research Hospital
TRIUMF
Engineering Telecom
Firewall
Main campusNetwork
BorderRouter
University
Internet
Health Network
GlobalPhysicsNetwork
3D HDTV toMcGill
BCnet
Campus DWDM
5G
1G
1G
1G
3G
1GTier 1
Tier 2
CERN
Convergence of Next Generation Computing and Next Generation Networks
> Moving from channels to platforms– The old Internet and telecom was focused on setting up
communication channels to distribute data – routers, switches, paths, etc
– But data should not have locality
> Convergence of cyber-infrastructure and next generation Internet
> Integration of Web 2.0, SOA, P2P, Enterprise 2.0 and NGI
> E.g. Van Jacobson content centric networking> Grids
Enterprise 2.0
> “… a defining moment in business history. We are on the threshold of a dramatic shift in the way that firms are organized, innovate and create value. Information technology and new networked business structures are removing the sources of friction in our economy.
> .. A new breed of open, networked organization-the Enterprise 2.0-is emerging. ... Collaboration is the new foundation [of this new paradigm]...
> Normally the term collaboration conjures up images of office workers interacting effectively together. But the concept is changing. By "collaboration" we mean the increasing richness of means by which objects (things, people and firms) can work together enhanced by the medium of the Internet. We have described this as the fundamental transition of the Internet from being a communications platform to a computation platform.”
Modern HPC System Architecture
Computations Simulations
64-bit or 32-bitSMP or Cluster
Data Storage & Management
DB server, SAN, NAS, Tape Lib.
SAN
Lab or Field Data collection
PC, Laptop, PDA, .... etc.
Interpretation- visualization
Graphic WkstnRender Engine
Interpretation - data mining
Content Manager,AI Tools, ... etc.
Collaboration
GroupwareVirtualization
Web Portal
Remote Viz.
Computations Simulations
64-bit or 32-bitSMP or Cluster
Typical Large system today
Sensor Sensor Instrument Instrument Sensor
Layer 2 switch
Layer 3 switch/router
SONET/DWDM
ProcessProcess
Process Process Process
SONET/DWDM
HPC
Firewall
Internet
VPN
USER
Instrument Pod
Service Oriented Architectures
Sensor Sensor Instrument Instrument Sensor
Layer 2/3 switch
LAN
LAN
Data Management System
CA*net 4
VPN
USER
Instrument Pod
WS*
WS*WS
CA*net 4Lightpath
Process
ProcessWS**
WS*
Process
ProcessWS**
WS*
Process
ProcessWS
WS
Web serviceInterface
*CANARIE UCLP
**New web services
HPC
Workflow Screen Shot
Integrated Grid-Lightpath Workflow
Future Internet Research and Experimentation
ICT unit F4 New Infrastructure Paradigms & Experimental FacilitiesDG Information Society and Media, European Commission, Brussels
ICT unit F4 New Infrastructure Paradigms & Experimental FacilitiesDG Information Society and Media, European Commission, Brussels
The European FIRE Initiative
Per Blixt, Max Lemke, Fabrizio Sestini
http://cordis.europa.eu/fp7/ict/fire
The European FIRE Initiative
Per Blixt, Max Lemke, Fabrizio Sestini
http://cordis.europa.eu/fp7/ict/fire
New Architectures
Situated Services
BIONETSBIONETSAutonomic service evolution
HAGGLEHAGGLEOpportunistic networking (cross-layer)
Common research issues:Security, resilience, self-* (organisation,
evolution,healing, …) interaction of new
paradigms with society
ANAANA Beyond IP
self-org.
Autonomic communication elements
CASCADASCASCADAS
FIRE-baselineFIRE-baseline
GENI Facility Conceptual Design
Slicing, Virtualization, Programmability
Mobile Wireless Network
Sensor Network
Edge Site
Source: Peter Freeman NSF
A New Science ParadigmComputers – Networks - International
> Thousand years ago: Experimental Science - description of natural phenomena
> Last few hundred years: Theoretical Science - Newton’s Laws, Maxwell’s Equations …
> Last few decades: Computational Science - simulation of complex phenomena
> Today: e-Science or Data-centric Science - unify theory, experiment, and simulation - using data exploration and data mining• Data captured by instruments • Data generated by simulations• Data generated by sensor networks– Scientist analyzes databases/files
(With thanks to Jim Gray)
2
22.
3
4
a
cG
a
a
2
22.
3
4
a
cG
a
a
Source: Tony Hey Microsoft
Bill Gate’s vision of Science in 2020: Project NeptunePresented at SC’06
Source: Tony Hey Microsoft
Science user perspective
Sensor/InstrumentWS**
LANWS*Science Pod
LANWS*
Log Archive Process 1WS**
Log Archive Process 2WS**
ONS15454WS* NLR or CA*net 4
WS* CANARIE UCLP
WS* New Web service
DMAS
WS** New development
LightpathWS*
WS AAA process
WS**
User defined WSFL bindings
WS HPC Process
WS**
USER with WSFL binding software
UDDI orWSIL service registry
Undersea Sensor
Network
Connected & Controllable
Over the Internet
Sensors and instruments controlled over the Internet using cyber-infrastructure workflow
Source: Tony Hey Microsoft
The Means: NEPTUNE -CANARIE Project
> Embrace open standards to further innovation, collaboration and knowledge sharing (Open Source, Linux, OASIS)
> Migration from proprietary or legacy instrument control schemes (RS-232, RS-485, LECIS, SCADA)
> Abstract and service-enable system end-points (instruments, PC control stations, compute clusters, data management or analytical applications)
> Introduce a loosely-coupled integration fabric which places the burden on the “service bus” rather than the end-points (SOA, ESB, SOAP/XML)
> Exploit reliable asynchronous messaging or synchronous messaging as a means for instrument control or data interchange
> Research driven process and service choreography (BPEL, BPEL4WS)
> Adopt self describing data packets that can be inspected or transformed in transit (XML)
> Provide dynamic and ever-evolving experimental design or conditional handling support via content based routing for alerting, workflow, and event management
To adopt and further refine CANARIE to realize a generic web-service enabled telemetry and control system for VENUS and NEPTUNE
The Reality: Canada and California delivering Bill Gate’s vision today
Integrate Instruments & Sensors into a LambdaGrid Computing Environment With Web Services Interfaces and workflow
Two projects funded by CANARIE: McGill underwater HDTV
Neptune Canada Cyber-InfrastructureClose collaboration with SDSC/Scripps “LOOKING” and NSF ORION
Several Canadian Industry Partners especially SMEs
Eucalyptus Participatory Design Studio Grid – using UCLP and SOA
> Carleton Immersive Media Studio (CIMS), Carleton University, Canada
The Participatory Design Studio will allow architects and industrial designers at multiple locations to collaborate in real time by sharing computational resources, geometry datasets, and multimedia content. The expected result is the development and field testing of a Service Oriented Architecture utilizing User Controlled Light Paths (UCLPv2) on CA*net 4 that provides university architecture staff and students in Ottawa and Montreal with on-demand simultaneous shared access to visualization, modeling, and visual communication tools. The project is innovative because commercially available architectural tools not originally intended for long-distance use will become easy-to-use powerful enablers of long-distance design participation.
Source: Maxine Brown
Winter Simulation Conference 2006 • Monterey, California • December 2006
CIMS-Ottawa - home
CIMS-La Jolla - remote
Eucalyptus: Collaborative Architecture Design between California and Canada
Source: Gabriel Wainer
Eucalyptus Network (APN)
San Diego State this spring
Source: Gabriel Wainer
Eucalyptus SOA
Source: Gabriel Wainer
Eucalyptus APN
Source: Gabriel Wainer
The OptIPuter Project – Creating High Resolution Portals Over Dedicated Optical Channels to Global Science Data
> Establishing an OptIPuter Node at CRC will Enable the BADLABTM to Develop Collaborative Visualization Environments Using Lightpath Services across CAnet 4 to Calit2
Source: Larry Smarr CAL-It2
CineGrid™ Initiative
CineGrid Initiative
> UCSD/Calit2, USA> Keio/DMC, Japan> USC/CNTV, USA> UIC/EVL, USA> UW/Research Channel, USA> UCI/Calit2, USA> UIUC/NCSA, USA> Tohoku University, Japan> TUT/CTL, Japan> SFSU/INGI, USA> Ryerson University, Canada> NTT Labs, Japan> Cisco, USA> City University of Hong Kong, School
Creative Media, Hong Kong> Italian National Cinema School, Italy> USC/Entertainment Technology Center,
USA> LSU/Center for Computation and
Technology, USA> Digital Cinema Technology Forum, Japan> Connecticut State Library, USA> Canadian Film Center, Canada> University of Amsterdam, Netherlands> De Waag (Netherlands), Netherlands
CineGrid is an initiative to provide media professionals access to global cyberinfrastructure capable of carrying ultra-high-performance digital media using photonic networks, middleware, transport protocols and collaboration tools originally developed for scientific research, visualization, and Grid computing.
Photo: Harry Ammons
Tokyo
Keio/DMC
Seattle Chicago
San Diego
UCSD/Calit2
Abilene
JGN II
PNWGP
Pacific WaveCENIC
CAVEwave
StarLight
Otemachi
GEMnet2/NTT
CineGridTM International Real-Time Streaming 4K Digital Cinema
TorontoRyerson
Source: Maxine Brown
UCLP for international science
> Hyugen’s Cassinni probe landing on Titan
> First time VBLI data ever transferred over networks
> AARNet’s was connected to a CANARIE (CA) switch and a User Controlled LightPath (UCLP) set up at 1GbE to the Joint Institution for VLBI in Europe (JIVE, NL)
Data Reservoir Project
> Goal to create a global grid infrastructure to enable distributed data sharing and high-speed computing for data analysis and numerical simulations
> Online 2-PFLOPS system (part of the GRAPE-DR project), to be operational in 2008
• University of Tokyo, WIDE Project, JGN2 network, APAN, Fujitsu Computer Technologies, NTT Communications, Japan
• Chelsio Communications• StarLight, PNWGP, IEEAF,
USA• CANARIE, Canada• SURFnet, SARA and
University of Amsterdam, The Netherlands
Won April 26, 2006 Internet2 Land Speed Records (I2-LSR) in theIPv4 and IPv6 single and multi-stream categories. For IPv4, created a network path over 30,000 kilometers crossing eight international networks and exchange points, and transferred data at a rate of 8.80Gbps, or 264,147 terabit-meters per second(Tb-mps). For IPv6: created a path over 30,000 kilometers, crossing five international networks, and transferred data at a rate of 6.96 Gbps, or 208,800 Tb-mps.
http://data-reservoir.adm.s.u-tokyo.ac.jp
Source: Maxine Brown
GridJam: A Networked 3D Immersive Performance
Gridjam is an art and research project to study real-time, interactive, low-latency, partly improvised, 3D visualized, musical performances. The Virtual Color Organ (VCO) is a 3D immersive environment in which music is visually realized in colored and image-textured shapes as it is heard. The VCO visually illustrates information in a music’s score, the composer’s instructions to the musicians, and the musicians’ contributions to the score as they improvise in reaction to one another’s performances and to the immersive visual experiences. The VCO displays the emergent properties within the meaning of music, both as information and as art.
> Fine Arts Department, ARTSLab and Center for High Performance Computing, University of New Mexico
> Mills College, CA> Calit2, UCSD, CA> University of Alberta, Canada> De Waag, NL (tentative)> V2_, Institute for the Unstable Media, NL (tentative)
http://jackox.net/pages/gridjampages/Gridjam1.htmlSource: Maxine Brown
Phosphorus: Lambda User Controlled Infrastructure For European Research
> European Union (EU) Research Networking Testbeds IST program
– 30-month project, to begin October 2006
> An alliance of European and Global partners to develop advanced solutions of application-level middleware and underlying management and control plane technologies
> Project Vision and Mission
– To address key technical challenges in enabling on-demand end-to-end network services across multiple domains
– To treat the underlying network as a first-class Grid resource– To demonstrate solutions and functionalities across a testbed involving European
NRENs, GÉANT2, Cross Border Dark Fibre and GLIF connectivity infrastructuresSource: Maxine Brown
Global Lambda Visualization Facility (GLVF)
www.evl.uic.edu/cavern/glvf
– Problem: Optical networks and LambdaGrids enable large-scale global science collaborations − but interoperable visualization and collaboration tools are missing!
– Solution: Launched in September 2005, a group of iGrid 2005 Workshop participants who were designing and developing complementary, distributed visualization and collaboration technologies decided to pool expertise, build on each other’s successes, and integrate their work into a coherent whole, providing a unique model for international partnerships.
− Jason Leigh, Electronic Visualization Laboratory, University of Illinois at Chicago (organizer)
Source: Maxine Brown
NCMIRSIO
UIC
USGS EDC
SARA
NCSA & TRECC
Calit2
AIST
SFU KISTI
U MichiganNortel
www.optiputer.netwww.evl.uic.edu/cavern/glvf
GLVF Visualization Technologies
Source: Maxine Brown
Next: San Diego Interactive Imaging of High Resolution Brain Slices from McGill University
Source: Mark Ellisman, UCSD, Calit2
There are 7407 Slices at 20 µmEach Image has 8513 x 12,472 pixels
CA*net4
Capacity Capability Vector Major Data StorageSMP
Canada’s National Platform for HPC
Future? - Interconnection of Compute Canada and NSF Petascale and TeraGrid
Net Neutrality – CALEA- etc
> Universities and Telephone companies face the same problem of small number of heavy users consuming expensive Internet bandwidth
> University solution is to cap bandwidth from dormitories and/or block types of traffic
> Telecom solution is to build a two tiered Internet or doing volume capping– A high speed un-congested channel for the telco traffic particularly
aimed at carrying video> Universities can play a leadership role in piloting new last mile
(hundred feet) architectures that address problems of dormitories – May serve as possible model for telcos
> University students are ideal early adopters and were instrumental in diffusion of the Internet throughout larger community
One possible solution
> Following is example of one possible solution– There may be others- this is not intended to be definitive or exclusive
> Work with a few universities on a small number of pilots where interested students can lease or control dedicated fiber/copper to university colo point
> They can directly peer with other students in the dormitory across a “white light” switch or user controlled VLAN switch; and/or
> Connect to service providers of their and/or setup point to point user controlled VLANs to other students across NLR, CA*net 4, GLORIAD, GLIF, SURFnet, i2Cat, KREOnet, etc
> Primary application would be collaborative video such as Inuk, YouTube and/or CineGrid
Advantages for student
> One time very small cost for UNLIMITED bandwidth forever to university colo
> Cross connect to service provider of their choice or research network(s)
> NO or very low monthly Internet service fees for connection to content providers or connection across research networks
> Participate in new global collaborative models such as Inuk, YouTube.com, MySpace or CineGrid
> Direct connection to content and application providers> Student installs transceiver or simple media hub at their
computer– Media hub with CWDM for about $200 which includes laser, Gbe
transceiver etc
Inuk Networks www.inuknetworks.com
> Offering free triple play to university dormitories in the UK
> Free cable TV, telephony> Over the air channels from around the world> Also deliver university content to cable systems and
other institutions around the world> Make money by selling eyeballs> Joost has same strategy
Ottawa backbone fiber
Background material
> http://www.multichannel.com/article/CA6332098.html> it only takes about 10 BitTorrent users bartering files
on a node (of around 500) to double the delays experienced by everybody else.
> http://www.news.utoronto.ca/bin6/060222-2074.asp > University students played critical role in diffusion of
the Internet to the global community