the massachusetts open cloud: an open cloud exchange · 2018-02-06 · bdaas @ moc cloud dataverse...
TRANSCRIPT
![Page 1: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/1.jpg)
The Massachusetts Open Cloud: an Open Cloud eXchange
1
![Page 2: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/2.jpg)
CLOUD COMPUTING IS HAVING A DRAMATIC IMPACT
• On-demand access
• Economies of scale
All compute/storage will move to the cloud?
2
![Page 3: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/3.jpg)
Today’s IaaS clouds
• One company responsible for implementing and operating the cloud
• Typically highly secretive about operational practices
• Exposes limited information to enable optimizations
3
![Page 4: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/4.jpg)
What’s the problem
• Lots of innovation above the IaaS level… but
• consider EnterpriseDB, or Akamai
• Lots of different providers… but
• bandwidth between providers limited
• offerings incompatible; switching a problem
• price challenges to moving
• No visibility/auditing internal processes
• Where is your data!
• Price is terrible for computers run 24x7x365
We are in the equivalent of the pre-Internet world,
where AOL and CompuServe dominated
on-line access
We are in the equivalent of the pre-Internet world,
where AOL and CompuServe dominated
on-line access
![Page 5: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/5.jpg)
Is a different model possible?An “Open Cloud eXchange
(OCX)”
C3DDB
HPC
Big Data
Web
5
![Page 6: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/6.jpg)
BIG BOX STORE SHOPPING MALL
6
![Page 7: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/7.jpg)
CATHEDRAL BAZAAR
7
![Page 8: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/8.jpg)
Why is this important
• Anyone can add a new service and compete in a level playing field
• History tells us the opening up to rich community/marketplace competition results in innovation/efficiency:
• “The Cathedral and the Bazaar” by Eric Steven Raymond
• “The Master Switch: The Rise and Fall of Information Empires” by Tim Wu
• This could fundamentally change systems research:
• access to real data
• access to real users
• access to scale
![Page 9: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/9.jpg)
Without that…solving the spherical horse problem…
![Page 10: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/10.jpg)
This isn’t crazy… really
• Current clouds are incredibly expensive…
• Much of industry locked out of current clouds
• lots of great open source software
• lots of great niche markets; markets important to us…
• lots of users concerned by vendor lock in…
• this doesn’t need to be AWS scale to be worth it
• “Past a certain scale; little advantage to economy of scale” — John Goodhue
![Page 11: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/11.jpg)
MGHPCC
15 MW, 90,000 square feet + can grow
10s of thousand HPC users, potentially many more cloud users
11
![Page 12: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/12.jpg)
The Massachusetts Open Cloud
12
![Page 13: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/13.jpg)
THE MASSACHUSETTS COLLABORATORS
13
![Page 14: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/14.jpg)
Operating Systems, Power,Security, Marketplace…
Cloud Technology
University Research IT PartnersBU, HU, NU, UMass, MIT,MGHPCC
PartnersBrocade, CISCO, Intel, Lenovo, Red Hat, Two Sigma, USAF, Dell, Fujitsu, Mellanox, Cambridge Computer…
Users/applicationsBigData, HPC, Life Sciences, …
Core TeamOCX model, HIL, Billing,Intermediaries…
DataBU, HU, NU, MIT, UMass, Foundations, Govt…
Education and WorkforceStudents, industry
14
MOC Ecosystem
![Page 15: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/15.jpg)
It’s real…• Available now: Production OpenStack services…
• Small scale, but growing (couple of hundred servers, 550 TB storage), 200+ users
• VMs, on-demand Big Data (Hadoop, SPARK...),
• What’s coming:
–Simple GUI for end users
–OpenShift – Red Hat
–Federation across universities
–Rapid/secure Hardware as a Service
–20+ PB NESE DataLake
–Cloud Dataverse
• Platform for enormous range of research projects across BU, NEU, MIT & Harvard
15
![Page 16: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/16.jpg)
Red Hat Collaboratory
• Mix & Match
• HIL & BMI (and QUADS integration)
• Big Data Analytics and Cloud Dataverse
• Datacenter-scale Data Delivery Network (D3N)
• Monitoring, Tracing, Analytics …
• OpenShift on the MOC
• Accelerator Testbed
End-to-end POC: Radiology in the cloud targeting OpenShift with accelerators
16
![Page 17: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/17.jpg)
Mix & Match:Resource Federation
17
![Page 18: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/18.jpg)
Keystone
Neutron GlanceNova Cinder
Keystone
OPENSTACK FOR AN OCX
• OpenStack is a natural starting point
• Mix & Match federation
Keystone
Neutron GlanceNova Cinder
18
![Page 19: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/19.jpg)
Mix and Match (Resource Federation)• Solution
• Proxy between OpenStack services
• Status of the project• Hosted upstream by the OpenStack infrastructure
• https://github.com/openstack/mixmatch
• Production deployment planned for Q1 2017
• Team:• Core Team: Kristi Nikolla, Eric Juma, Jeremy
Freudberg
• Contributors: Adam Young (Red Hat), George Silvis, Wjdan Alharthi, Minying Lu, Kyle Liberti
• More information:• https://info.massopencloud.org/blog/mixmatch-fe
deration/
Boston University
Northeastern University
mixmatch
Nova
Keystone
CinderKeystone
19
![Page 21: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/21.jpg)
Why Bare Metal?
Useful for different workloads:
• Staging, testing, production
• HPC + Cloud
• Max / predictable performance
• Run VMs
• Non-virtualizable hardware
• Increased Security
• Less trust in the provider
21
![Page 22: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/22.jpg)
MOC Projects in Bare Metal
• Bringing software configuration advantages of virtualization to bare metal:
a) Hardware Isolation Layer• Allocate & configure nodes and networks
b) Bare Metal Imager• Image management: fast provisioning, cloning and snapshotting of disks
c) Secure Cloud• Checks that each machine is in pristine / untampered
![Page 23: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/23.jpg)
HIL: Hardware isolation layer
![Page 24: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/24.jpg)
HPC
Datacenter has isolated silos
25
![Page 25: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/25.jpg)
Hardware isolation layer
Allocate physical nodesAllocate networks Connect nodes and networks26
![Page 26: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/26.jpg)
Status
• In production at the Mass Open Cloud: production OpenStack environment, staging area(s), OS research, Big Data on-demand
• Supports variety of provisioning systems: Foreman, MaaS, Ironic, home brewed research (EbbRT)
![Page 27: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/27.jpg)
HIL and QUADS integration
28
Goal: Extend Red Hat’s QUADS (Quick and Dirty Scheduler) to be able to use the MOC’s HIL to manage hardware isolation
Reasons/Motivation: Enhances QUADS portability, and endows HIL the ability to dynamically schedule
Before After
![Page 28: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/28.jpg)
BMI: Bare Metal Imager
![Page 29: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/29.jpg)
Bare Metal Imager:VM-like disk image management
iSCSI-based Able to provision + boot in < 5 min
Turk, A., Gudimetla, R. S., Kaynar, E. U., Hennessey, J., Tikale, S., Desnoyers, P., & Krieger, O. (2016). An Experiment on Bare-Metal BigData Provisioning. In 8th USENIX
Workshop on Hot Topics in Cloud Computing (HotCloud 16).30
![Page 30: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/30.jpg)
Secure Cloud
![Page 31: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/31.jpg)
Secure Cloud
• Goals• Increase confidence in the firmware
• Minimal amount of provider-trusted changes
• Transition nodes quickly
• Firmware integrity (system & peripheral):• Measure
• Protect
• Replace
• Maintain/Audit
• Vendor survey
• Hardware specification
![Page 32: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/32.jpg)
![Page 33: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/33.jpg)
People/contacts:• HIL (Hardware Isolation Layer) github.com/CCI-MOC/hil
• Contact: [email protected]
• Core team: Jason Hennessey (BU), Sahil Tikale (BU), Ian Denhardt (BU), Peter Desnoyers (NEU), Orran Krieger (BU), Jethro Sun (BU), Kristi Nikolla (BU), Nicholas Matsuura (USAF), Naved Ansari (BU), Kyle Hogan (BU), Mengyuan Sun (MIT), Gwen Faline Edgar (MIT)
• Contributors (some were past affiliations): George Silvis III (BU), Yue Zhang (BU), Apoorve Mohan (NEU), Ravisantosh Gudimetla (NEU), Minying Lu (BU), Zhaoliang Liu (NEU), Ryan Abouzahra (USAF), Jonathan Bell (BU), Jonathan Bernard (BU), Rohan Garg (NEU), Andrew Mohn (BU), Abhishek Raju (NEU), Ritesh Singh (NEU), Ron Unrau and Valerie Young (BU)
• BMI (Bare Metal Imager) github.com/CCI-MOC/ims• Contact: Gene Cooperman <[email protected]>
• Core team: Gene Cooperman (NEU), Naved Ansari (BU), Apoorve Mohan (NEU), Pranay Surana (NEU), Ravi Santosh Gudimetla (Redhat, formerly NEU), Sourabh Bollapragada (NEU)
• Contributors: Jason Hennessey (BU), Ata Turk (BU), Ugur Kaynar (BU), Sahil Tikale (BU), Orran Krieger (BU), Peter Desnoyers (NEU)
• Secure Cloud• Contact: Jason Hennessey <[email protected]>
• Core team: Jason Hennessey (BU), Nabil Schear (MIT LL), Trammell Hudson (Two Sigma), Orran Krieger (BU) Gerardo Ravago (BU), Kyle Hogan (BU), Ravi S. Gudimetla (NEU), Larry Rudolph (Two Sigma), Mayank Varia (BU)
![Page 34: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/34.jpg)
Cloud Dataverse
43
![Page 35: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/35.jpg)
Dataverse
44
• Dataverse is an open-source software
platform for building data repositories
• It provides an incentive to share data
• Gives credit through data citation
• Provides mechanisms for control over
data access
• Builds a community:• To foster new research in data sharing
• To define new standards and best practices
• Installed in 20 repositories world wide
• Hosting dataverses from > 500 institutions
![Page 36: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/36.jpg)
Cloud Dataverse
Cloud Dataverse
45
• A dataset repository solution for cloud
• Extends Dataverse
• Store datasets in Object Store (Swift)
• Harvest datasets from all Dataverses
• Compute button that enables on-cloud
computation
• No need for download
![Page 37: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/37.jpg)
BDaaS @ MOC
Cloud DataverseDataset Repository BigData clusters spinned via
OpenStack Sahara to compute over datasets
46
![Page 38: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/38.jpg)
BDaaS @ MOC
Cloud DataverseDataset Repository BigData clusters spinned via
OpenStack Sahara to compute over datasets
47
if not enough resources / highly utilized
![Page 39: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/39.jpg)
BDaaS @ MOC
Cloud DataverseDataset Repository BigData clusters spinned via
OpenStack Sahara to compute over datasets
48
if not enough resources / highly utilized
Expand OpenStack cluster using HIL and BMI
![Page 40: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/40.jpg)
BDaaS @ MOC
Cloud DataverseDataset Repository BigData clusters spinned via
OpenStack Sahara to compute over datasets
49
while cluster is being deployed
BigData Caching:cache datasetsfor the BigData cluster
prefetch
![Page 41: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/41.jpg)
Datacenter-scale Data Delivery Network
(D3N)MOC, Red Hat, Intel, Brocade, Lenovo, 2Sigma
![Page 42: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/42.jpg)
Data Lake in a typical DC
North Eastern Storage Exchange (NESE): 20+PB Harvard, NEU, MIT, BU, UMass 51
![Page 43: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/43.jpg)
Simple deployment:
• Dedicated cache servers per rack
• L1 : Rack Local
– reduce inter rack traffic
• L2 : Cluster Local
– reduce clusters and back-end storage traffic
• Implemented by modifying CEPH Rados Gateway
Datacenter scale Data Delivery Network (D3N)
52
![Page 44: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/44.jpg)
D3N Results
1 2 3 4 5 6 7 8Number of Hadoop Nodes
00.5
11.5
22.5
33.5
44.5
55.5
Agg
rega
te T
hrou
ghpu
t (G
B/s
)
RGWD3N L1 Hit
Maximum SSD Bandwidth
1 2 3 4 5 6 7 8Number of Curl Nodes
0.51
1.52
2.53
3.54
4.55
Agg
rega
te T
hrou
ghpu
t (G
B/s
)
RGWD3N L1 Hit
Maximum SSD Bandwidth
• Exceeds maximum bandwidth Hadoop
• Demonstrates makes sense to share expensive SSDs – faster than local disk
• With extreme benchmark can saturate SSD & 40 Gb NIC
• Will be of enormous value with NESE data lake
53
![Page 45: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/45.jpg)
Red Hat Collaboratory
• Mix & Match
• HIL & BMI (and QUADS integration)
• Big Data Analytics and Cloud Dataverse
• Datacenter-scale Data Delivery Network (D3N)
• Monitoring, Tracing, Analytics …
• OpenShift on the MOC
• Accelerator Testbed
End-to-end POC: Radiology in the cloud targeting OpenShift with accelerators
54
![Page 46: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/46.jpg)
Monitoring, Tracing, Analytics
• Problem
– Complexity; distributed systems
• Solved … with data
– We need data … good data, to help us deal with complex, distributed systems
– We need help distilling that data for human consumption
56
![Page 47: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/47.jpg)
57
Collect & Consolidate- Datacenter, - Physical, - Network, - Cloud Management Layerto enable analysis cross layers
MOC Monitoring Infrastructure
![Page 48: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/48.jpg)
Approach
• Working on data collection
– Sprinkle TLC (Telemetry, Logs, Configuration)• Fluentd, rsyslog, collectd,
prometheus, etc. (See pbench)
– Data Model, Observability and automation lightning talks
– OpenShift Aggregated Logging and RHV, OpenStack to follow
• Analytics
– See OpenShift and the insightful application development life cycle
58
![Page 49: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/49.jpg)
Workflow-centric tracingin OpenStack
• Raja Sambasivan Ata Turk, Joe Talerico, Peter Portante, Orran Krieger
59
![Page 50: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/50.jpg)
Today’s distributed systems
E.g.,TwitterE.g., Netflix
Machine-centric tools insufficient {GDB,
gprof,strace,linux perf. countersNetflix “death star”: http://www.slideshare.net/adriancockcroft/fast-delivery-devops-israel
Twitter “death star”: https://twitter.com/adrianco/status/441883572618948608
![Page 51: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/51.jpg)
Workflow-centric tracing
Provides the needed coherent view
App Server Distributed FSTable store
Client
Server
Trace point (e.g., at functions)
Get
27 ms
25 ms17 µs�
�
�
�
��
� Metadata (e.g., IDs)
![Page 52: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/52.jpg)
Explore tracing’s potential in OpenStack
Implement tracing in OpenStack
Explore new tools for new classes of problems and different operational tasks
Explore applicability of existing diagnosis tools (e.g., Spectroscope [NSDI’11])
Use OSProfiler as a starting point
![Page 53: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/53.jpg)
Red Hat Collaboratory
• Mix & Match
• HIL & BMI (and QUADS integration)
• Big Data Analytics and Cloud Dataverse
• Datacenter-scale Data Delivery Network (D3N)
• Monitoring, Tracing, Analytics …
• OpenShift on the MOC
• Accelerator Testbed
End-to-end POC: Radiology in the cloud targeting OpenShift with accelerators
63
![Page 54: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/54.jpg)
Exemplar
PC BG1K BG4k BG16k0
2250
4500
6750
9000
11250
seco
nds
synthetic1024x1024200 slices
Fetal Image Reconstruction
~2.4hrs
~24s
resized,cropped96x9650 slices
APP
IRTK24hrs
encapsulates packets with
Radiology in the cloud workflow
64
![Page 55: The Massachusetts Open Cloud: an Open Cloud eXchange · 2018-02-06 · BDaaS @ MOC Cloud Dataverse Dataset Repository BigData clusters spinned via OpenStack Sahara to compute over](https://reader033.vdocuments.us/reader033/viewer/2022050305/5f6deea84e8baf35ca521514/html5/thumbnails/55.jpg)
Concluding remarks
• MOC a functioning small scale cloud for region today:
–http://info.massopencloud.org/blog/user-account-request-form/
• Key driver is the OCX Model:
–Key enablers going on in OpenStack
–Could become important component of clouds
–Major research challenge & opportunities: presented a small sampling
–Enabling research to co-exists with production:
• real data, real users, real scale
• Combining innovation open source, research, cloud (CI/CD)