experimenting with virtual sdxs using chameleon and … · sdsn geni phys dmz dtn ip egress ben...

www. chameleoncloud.org

AP RIL 6 , 2 0 1 8 1

EXPERIMENTING WITH VIRTUAL SDXS USING CHAMELEON AND EXOGENI

Paul RuthRENCI – University of North Carolina

[email protected]


OUTLINE

� Background� ExoGENI testbed (wide footprint edge cloud)

� NSF Cloud Chameleon testbed (mid-scale cloud)

� Experiments Spanning Testbeds � Inter-slice stitching

� Campus stitching

� Inter-testbed stitching

� Software Defined eXchange (SDX) Experiments� SAFE SDX (RENCI, DUKE, US DOE/Esnet)

� SciDAS (Clemson, RENCI, Washington State University)


NSF GLOBAL ENVIRONMENT FOR NETWORK INNOVATIONS (GENI)

Virtual laboratory for networking and distributed systems research and education


GENI FEDERATION� Federated identity

� InCommon

� X.509 identity certificates

� Common APIs� Aggregate Manager

� Clearinghouse

� Agreed upon resource description language� RSpec

� ExoGENI translates relevant portions from NDL-OWL to RSpec and back as needed

� Several major portions� ExoGENI, InstaGENI, WiMax, Internet2 AL2S, ESnet

� Federation with EU FIRE effort


Cloud Providers

Virtual Compute and Storage Infrastructure

Network Transit Providers

Cloud APIs (Amazon EC2 ..) Network Provisioning APIs (DOE ESNetOSCARS, Internet2, OESS, OGF NSI …)

Virtual Network Infrastructure

EXOGENI


Mutually Isolated Virtual Networks

VM VM

VM VM

VM VM

VM VM

VM VM

VM VM

VM VM

VM VM

VM VM

VM VM

VM VM

VM VM

Edge Providers(Compute Clouds and Network Providers)

Mutually Isolated Slicesof Virtual Resources

Workflows

EXOGENI


EXOGENI

� Relationship to GENI� One of two computational testbeds built for GENI

� Implements GENI API

� Accepts GENI users

� Notable features:� Wide scale footprint (20 sites)

� Edge clouds (OpenStack)

� Dynamic layer 2 circuits between sites

� Stitchports: layer 2 connections to external resources

� Limitations� Small scale computational sites

� No core network control


EXOGENITOPOLOGY


EXOGENITOOLS


EXOGENI: STITCHING


OUTLINE









CHAMELEON PHASE 1 IN A NUTSHELL� Deeply reconfigurable: “As close as possible to having it in your lab”

� Deep reconfigurability (bare metal) and isolation

� Power on/off, reboot from custom kernel, serial console access, etc.

� But also – modest KVM cloud for ease of use

� Large-scale: “Big Data, Big Compute research”

� ~650 nodes (~15,000 cores), 5 PB of storage distributed over 2 sites connected with 100G network…

� …and diverse: ARMs, Atoms, FPGAs, GPUs, etc.

� Blueprint for a sustainable production testbed: “cost-effective to deploy, operate, and enhance”

� Powered by OpenStack with bare metal reconfiguration (Ironic)

� Open production testbed for Computer Science Research

� Project started in 10/2014, testbed available since 07/2015

� Currently 1,600+ users, 300+ projects


CHAMELEON: PHASE 1 HARDWARE

SCUs connect tocoreandfullyconnected toeachother

HeterogeneousCloudUnits

ARMs,Atoms,lowpowerXeions, FPGAs,GPUs,SSDs, etc.

SwitchStandardCloudUnit42compute4storagex10

Chicago

To UTSA, GENI, Future Partners

AustinChameleonCoreNetwork

100Gbps uplink publicnetwork(eachsite)

CoreServices3.6PBCentralFileSystems, FrontEndandDataMovers

CoreServicesFrontEndandData

MoverNodes 504x86ComputeServers48Dist.StorageServers102HeterogeneousServers16Mgt andStorageNodes

SwitchStandardCloudUnit42compute4storagex2


NEW HARDWARE� 4 new Standard Cloud Units (32 node racks in 2U chassis)

� 3x Intel Xeon “Sky Lake” racks (2x @UC, 1x @TACC)

� 1x future Intel Xeon rack (@TACC) in Y2

� Corsa DP2000 series switches� 2x DP2400 with 100Gbps uplinks (@UC)

� 1x DP2200 with 100Gbps uplink (@TACC)

� Each switch will have a 10 Gbps connection to nodes in the SCU

� Optional Ethernet connection in both racks

� More storage configurations� Global store @UC: 5 servers with 12x10TB disks each

� Additional storage @TACC: 150 TB of NVMes

� Accelerators: 16 nodes with 2 Volta GPUs (8@UC, 8@TACC)

� Maintenance, support and reserve


CORSA DP2000 SERIES SWITCHES� Hardware Network Isolation

� Sliceable Network Hardware

� Tenant controlled Virtual Forwarding Contexts (VFC)

� Software Defined Networking (SDN)� OpenFlow v1.3

� User defined controllers

� Performance� 10 Gbps within a site

� 100 Gbps between UC/TACC (Aggregated)


StandardCloudUnit

NETWORK HARDWARE

Chicago

Internet 2 AL2S, GENI, Future Partners

Austin

ChameleonCoreNetwork100Gbps uplink publicnetwork

(eachsite)

StandardCloudUnit

Corsa DP2400Corsa DP2400

StackedSwitches(LogicallyOne)

StandardCloudUnit

Corsa DP2200

100Gbps(Aggregate)

100Gbps(Aggregate)


ISOLATED VIRTUAL SDN SWITCH� Isolated Tenant Networks

� BYOC– Bring your own controller: isolated user controlled virtual OpenFlowswitches (coming soon)

StandardCloudUnit

Corsa Switch

ComputeNode

(TenantA)

ComputeNode

(TenantA)

ComputeNode

(TenantB)

ComputeNode

(TenantB)

VFC(TenantA)

VFC(TenantB)

OpenFlowController(TenantB)

OpenFlowController(TenantA)

Ryu


StandardCloudUnit

CHAMELEON: SDN EXPERIMENTS

� Chameleon Networking

� RENCI added to the team� Hardware Network Isolation

� Corsa DP2000 series

� OpenFlow v1.3 � Sliceable Network Hardware� Tenant controlled Virtual Forwarding

Contexts (VFC)

� Isolated Tenant Networks� BYOC – Bring your own controller

� Wide-area Stitching

� Between Chameleon Sites (100 Gbps)� ExoGENI� Campus networks (ScienceDMZs)

CorsaDP2400Switch

Internet 2 AL2S, GENI, Future Partners

Chicago

Austin

ComputeNode

(TenantA)

OpenFlowController(TenantB)

OpenFlowController(TenantA)

Ryu

VFC(TenantA)

ComputeNode

(TenantA)

ChameleonCoreNetwork100Gbps uplink publicnetwork

ComputeNode

(TenantB)

ComputeNode

(TenantB)

VFC(Tenantb)


OUTLINE









EXOGENI: INTER-SLICE STITCHING


Public Internet




Starlight



Starlight

Service Slice Client Slice


OUTLINE









EXOGENITO CAMPUS STITCHING

IPcore(L3) Circuit fabric

providers

SDSN

GENI

Phys

DMZDTN

IPegress

BEN

I2/A2LS

ESnet

OtherGENIsitesOthercampusesOtherfacilities

ControlplaneAPIs

Dukecampusboundary

L2egress

e.g.GENI-APIe.g.OSCARSe.g.Plexuse.g.ORCA

Duke University Software Defined Science Network (SDSN)Science DMZ


EXOGENITO CAMPUS STITCHING

Stitchport: Named meeting point linking a layer 2 circuit between ExoGENI and

external resources.

Stitchport Duke SDSN


MULTI-TESTBED EXPERIMENTS

Starlight

Client SliceService Slice

Client Campus


OUTLINE









CHAMELEON TO EXOGENI STITCHING

• Dynamic VLANs• Connectivity to

ExoGENI Stitchport

• ExoGENI slice• Dynamic Chameleon

Stitchport

Stitched L2 path


CHAMELEON TO EXOGENI STITCHING

StitchPort

Stitching between ExoGENI and Chameleon nodes


INTER-TESTBED EXPERIMENTS

Starlight

Client SliceService Slice

Client Campus



Starlight

Service Slice Client Slice

Client CampusClient Slice



Starlight

Virtual SDXService Slice Client Slice

Client CampusClient Slice


OUTLINE









CICI SAFE PROJECT

“Creating Dynamic Superfacilities the SAFE Way”Paul Ruth, Cong Wang, Mert Cevik, RENCI

Jeff Chase, YuanjunYao, Qiang Cao, Victor Orlikowski. Charley Kneifel, Duke Univeristy

Nick Buraglio, ESnet

NSF CICI Award #1642142


SUPERFACILITY

� Definition� Two or more existing facilities (e.g. instruments, compute resources, data repositories) using

high-performance networks and data management software in order to increase scientific output.

� Currently manually created � Superfacilities are purpose-built manually for a specific scientific application or community.

� Trust: “handshake model”

� Ideally automated� Advanced Science DMZs and federated Infrastructure-as-a-Service provide the technical

building blocks to construct dynamic superfacilities on demand.


SUPERFACILITY

� Definition� Two or more existing facilities (e.g. instruments, compute resources, data repositories) using

high-performance networks and data management software in order to increase scientific output.

� Currently manually created� Superfacilities are purpose-built manually for a specific scientific application or community.

� Trust: “handshake model”

� Ideally automated� Advanced Science DMZs and federated Infrastructure-as-a-Service provide the technical

building blocks to construct dynamic superfacilities on demand.

Trust also needs to be automated


SUPERFACILITIES THE SAFE WAYDuke Science DMZ Other Campus

ExoGENI Slice

IDS IDS IDS IDS

DTN

Virtual SDX

• Automating Superfacilites– Multiple domains– Friction free L2 paths

• Naked L2 paths are not secure– Handshake model of trust is not possible

• Virtual SDX (vSDX)– Distributed– Enforces SDX connectivity policy– Enforces client’s forwarding policy

(security, BGP, etc.)– Intrusion Detection System (Bro)

• SAFE: Secure Authorization for Federated Environments– Isolates applications from logic concerns

• Certificate discovery (DAGs)• Logic inference• Cryptography

– Logic scripting language• Slang (SAFE Language)• Based on Datalog

– Shared certificate repository• Stores statements and DAGs


SCIDAS

1PBStge/FIONA 1PBStge./FIONA 1PBStge./FIONA

Cost-AwareOptimize

iRODSShim (aaS)

API

PerfSONARShim (aaS)

API PerfSONARmapping

Requester

Orchestrator

Network


SCIDAS

Automated vSDX superfacility


THANK [email protected]

experimenting with virtual sdxs using chameleon and … · sdsn geni phys dmz dtn ip egress ben...

Documents