Download - Ilya Baldin [email protected]@renci.org. 2
2
ExoGENI Overview
3
14 GPO-funded racks built by IBM◦ Partnership between RENCI, Duke and IBM◦ IBM x3650 M4 servers (X-series 2U)
1x146GB 10K SAS hard drive +1x500GB secondary drive 48G RAM 1333Mhz Dual-socket 8-core CPU Dual 1Gbps adapter (management network) 10G dual-port Chelseo adapter (dataplane)
◦ BNT 8264 10G/40G OpenFlow switch◦ DS3512 6TB sliverable storage
iSCSI interface for head node image storage as well as experimenter slivering
◦ Cisco(UCS-B) and Dell configuration also exist Each rack is a small networked cloud
◦ OpenStack-based with NEuca extensions◦ xCAT for baremetal node provisioning
http://wiki.exogeni.net
Testbed
4
ExoGENI is a collection of off-the shelf institutional clouds◦ With a GENI federation on top◦ xCAT – IBM product◦ OpenStack- RedHat product
Operators decide how much capacity to delegate to GENI and how much to retain for yourself
Familiar industry-standard interfaces (EC2) GENI Interface
◦ Mostly does what GENI experimenters expect
ExoGENI
ExoGENI at a glance
6
Rack Software Stack
Deployment structure
An ExoGENI cloud “rack site”
11
ExoGENI racks are separate aggregates but also act as a single aggregate◦ Transparent stitching of resources from multiple
racks ExoGENI is designed to bridge distributed
experimentation, computational sciences and Big Data◦ Already running HPC workflows linked to OSG
and national supercomputers◦ Newly introduced support for storage slivering◦ Strong performance isolation is one of key goals
ExoGENI unique features
12
GENI tools: Flack, GENI Portal, omni◦ Give access to common GENI capabilities◦ Also mostly compatible with
ExoGENI native stitching ExoGENI automated resource binding
ExoGENI-specific tools: Flukes◦ Accepts GENI credentials◦ Access to ExoGENI-specific features
Elastic Cluster slices Storage provisioning Stitching to campus infrastructure
ExoGENI experimenter tools
Presentation title goes here 13
ExoGENI activities
14
Compute nodes◦ Up to 100 VMs in each full rack◦ A few (2) bare-metal nodes◦ BYOI (Bring Your Own Image)
True Layer 2 slice topologies can be created ◦ Within individual racks ◦ Between racks◦ With automatic and user-specified resource binding and
slice topology embedding◦ Stitching across I2, ESnet, NLR, regional providers.
Dynamic wherever possible OpenFlow experimentation
◦ Within racks◦ Between racks◦ Include OpenFlow overlays in NLR (and I2)◦ On-ramp to campus OpenFlow network (if available)
Experimenters are allowed and encouraged to use their own virtual appliance images
Since Dec 2012◦ 2500+ slices
Experimentation
Virtual network exchange
Virtual colocampus net to circuit fabric
Multi-homedcloud hosts
with network control
Topology embedding and stitching
Computed embedding
Workflows, services,
etc.
Slice half-way around the world ExoGENI rack in Sydney, Australia Multiple VLAN tags on a pinned path from Sydney to
LA Internet2/OSCARS ORCA-provisioned dynamic circuit
◦ LA, Chicago NSI statically pinned segment with multiple VLAN
tags◦ Chicago, NY, Amsterdam◦ Planning to add dynamic NSI interface
ExoGENI rack in Amsterdam◦ ~14000 miles◦ 120ms delay
17
Strong isolation is the goal Compute instances are
KVM based and get a dedicated number of cores
VLANs are the basis of connectivity◦ VLANs can be best effort or
bandwidth-provisioned (within and between racks)
ExoGENI slice isolation
Scientific Workflows
Workflow Management Systems◦ Pegasus, Custom scripts, etc.
Lack of tools to integrate with dynamic infrastructures◦ Orchestrate the infrastructure in response to application◦ Integrate data movement with workflows for optimized
performance◦ Manage application in response to infrastructure
Scenarios◦ Computational with varying demands◦ Data-driven with large static data-set(s)◦ Data-driven with large amount of input/output data
Scientific Workflows
Dynamic Workflow Steps (Computational)
21
Workflow slices• 462,969 condor jobs since using the on-ramp to engage-submit3 (OSG)
Dynamic Workflow Steps (On-Ramp)
Dynamic Workflow Steps (Data-driven)
Hardware-in-the loop slices•Hardware-in-the-Loop Facility Using RTDS & PMUs (FREEDM Center, NCSU)
RENCI Data Center
Experimental PMU Data from RTDS
0 500 1000 1500 2000 2500 3000 35000.98
0.99
1
1.01
1.02
1.03
1.04
Time in seconds (sec.) with origin at: 12:38:00 EST
Voltage M
agnitude (
pu)
PMU Voltage Magnitudes in AEP - Richview Event
JF KR MF RK OR OS
60 65 70 75 80 85 90
-0.02
-0.015
-0.01
-0.005
0
0.005
0.01
0.015
0.02
Time (sec)
Fa
st O
scill
atio
ns (
pu
)
Bus 1Bus 2Midpoint
800 810 820 830 840 850 860 8709.1
9.2
9.3
9.4
9.5
9.6
9.7
9.8
Angle
diffe
rence (
deg)
Time in seconds starting at:04-Aug-2007 16:30:00
tva mac
PMU Measurements
1 #
112
11
ModeInterarea
aa
aa
ss
s
2 #
222
22
ModeInterarea
aa
aa
ss
s
3 #
332
33
ModeInterarea
aa
aa
ss
s
Zero/First Order Hold
2 #
222
22
ModeInterarea
i iaia
iaia
ss
s
Intra-clusterVirtualization
PoP at UNC Chapel Hill
PoP at Duke University
Cluster 1
Cluster 2Cluster 3
NC State PoP
Distributed Execution of Time-critical Synchrophasor Applications
Fiber optic network for PMU data communication using IEEE C37.118
New GENI-WAMS Testbed
Latency & Processing Delays
Packet Loss
Network Jitters
Cyber-security Man-in-middle attacks
Aranya Chakrabortty, Aaron Hurz (NCSU)Yufeng Xin (RENCI/UNC)
28
http://www.exogeni.net◦ ExoBlog http://www.exogeni.net/category/exoblog/
http://wiki.exogeni.net
Thank you!