secure and scalable infrastructures for cloud operations ... · cloud3 cloud2 felix eberhardt, fgt...
TRANSCRIPT
![Page 1: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/1.jpg)
Secure and Scalable Infrastructures for Cloud Operations (SSICLOPS) Resource Management in federated OpenStack cloud environments Felix Eberhardt
Stefan Klauck Max Plauth 28.09.2017
![Page 2: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/2.jpg)
■ Management of federated private cloud infrastructures
■ Network communication improvements
■ Workload scheduling across datacenters
■ Security- and privacy-aware storage and processing
SSICLOPS: Research Areas
Chart 2
02.2015 – 02.2018
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
![Page 3: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/3.jpg)
1. In-Memory Databases in the cloud (HYRISELab)
2. High-Energy Physics Workload
3. Network Function Virtualization in a NGPoP
4. Content Distribution and Caching
SSICLOPS: Use Cases
Chart 3
02.2015 – 02.2018
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
![Page 4: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/4.jpg)
SSICLOPS: Partners
Chart 4
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
![Page 5: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/5.jpg)
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
SSICLOPS: Big Picture
Chart 5
![Page 6: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/6.jpg)
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
SSICLOPS: System Level
Chart 6
![Page 7: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/7.jpg)
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
SSICLOPS: Rack Level
Chart 7
![Page 8: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/8.jpg)
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
SSICLOPS: Intra Cloud Level
Chart 8
![Page 9: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/9.jpg)
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
SSICLOPS: Inter Cloud Level
Chart 9
![Page 10: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/10.jpg)
SSICLOPS: Collaboration
WP3WP1
WP2
WP4
Hydrant FOSS Instrumented 40G NIC
Scenario 1:In-MemoryDatabasesin the Cloud
Scenario 3:Network Function
Virtualizationin a Next-Gen PoP
NetmapScenario 2:
High-Energy Physics
mSwitch Software Switch
StackMap Accelerated Socket API
PASTE Network NVMM Integration
Netmap VM Passthrough
Kernel Accelerated Packet Processing
Scenario 4:Content Distribution
and Caching
Unikernel Performance Monitor
Prism I/O Granularity Proxy
Compact Privacy Policy Language
Federated OpenStack Testbed
Multipath TCP Proxy
Secure Multipath TCP
TLS Session Sharing
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 Chart 10
![Page 11: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/11.jpg)
SSICLOPS: Collaboration
WP3WP1
WP2
WP4
Hydrant FOSS Instrumented 40G NIC
Scenario 1:In-MemoryDatabasesin the Cloud
Scenario 3:Network Function
Virtualizationin a Next-Gen PoP
NetmapScenario 2:
High-Energy Physics
mSwitch Software Switch
StackMap Accelerated Socket API
PASTE Network NVMM Integration
Netmap VM Passthrough
Kernel Accelerated Packet Processing
Scenario 4:Content Distribution
and Caching
Unikernel Performance Monitor
Prism I/O Granularity Proxy
Compact Privacy Policy Language
Federated OpenStack Testbed
Multipath TCP Proxy
Secure Multipath TCP
TLS Session Sharing
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 Chart 11
![Page 12: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/12.jpg)
SSICLOPS: Policy Language
Chart 12
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
offers annotation
Policy.definition
Userexpectations
Providerofferings
transform
Provider
Load.balancing.and.adherence
![Page 13: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/13.jpg)
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
SSICLOPS: Policy Support in OpenStack
Chart 13
![Page 14: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/14.jpg)
Experiments: Federated Cloud Replication
Chart 14
Mid
dlew
are
#1
keys
tone
mid
dlew
are
polic
ymid
dlew
are
Mid
dlew
are
#n
GET /servers HTTP/1.1X-Auth-Token: 809d57a
GET /servers HTTP/1.1X-Auth-Token: 809d57aX_USER_ID: 3f729X_USER_NAME: admin
GET /servers HTTP/1.1X-Auth-Token: 809d57aX_USER_ID: 3f729X_USER_NAME: adminX_POLICY: e30=
GET /servers HTTP/1.1X-Auth-Token: 1nv4l1d
Serv
ice
RequestValid
RequestInvalid
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
![Page 15: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/15.jpg)
Experiments: Federated Cloud Replication
Chart 15
PolicyExtensionPolicyExtension
func_pathsMethod f
Original Function f
OpenStack API Class
Function g Function g
Original Function f
Method fPolicyExtensions ladload PolicyExtensions
OpenStack API Class
Replacement Function f
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
![Page 16: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/16.jpg)
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
SSICLOPS: Testbed
Chart 16
![Page 17: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/17.jpg)
SSICLOPS Testbed: Federated Clouds
Chart 17
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
![Page 18: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/18.jpg)
SSICLOPS Testbed: Federated Clouds – MPTCP Proxy
Chart 18
Network(node
Network(node
Network(node
Layer 2(meshedtunnels
Cloud 1 Cloud 3
Cloud 2Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
![Page 19: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/19.jpg)
SSICLOPS Testbed: Federated Clouds – MPTCP Proxy
Chart 19
Network(node
Network(node
Network(node
Layer 2(meshedtunnels
Multicastand(unicastforwarder
Link failure resilience
Cloud 1 Cloud 3
Cloud 2Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
![Page 20: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/20.jpg)
Dispatcher
HyriseMaster Node
HyriseReplica Node i
HTTP
HTTPHTTP
TCP/IP
SSICLOPS: Use Case In Memory Database
Single node in-memory database
Open source database
No policy support
No scale-out support
Multi node in-memory database
Compact Privacy Policy Language (WP2)
Elasticity: OpenStack (WP3)
Scale-out: StackMap (WP1)
Availability: MPTCP (WP1)
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 Chart 20
Hyrise Node
HTTP
https://github.com/hyrise
![Page 21: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/21.jpg)
SSICLOPS: Use Case In Memory Database
Chart 21
Dispatcher
Cluster
HyrisePrimary Node
ClusterInterface
RequestHandler
DataStorage Logger
HyriseReplica Node i
ClusterInterface
RequestHandler
DataStorage Logger
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
Cluster interface sends (dictionary encoded) log information to replicas Frequency is configurable and based on • Number of calls • Exceeding buffer size • Time since last transmission
Ongoing Cooperation with SSICLOPS partner NetApp (WP1).
![Page 22: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/22.jpg)
Use Case In Memory Database Elasticity
Chart 22
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
Elasticity is the capability to shrink and extend the database cluster depending on the current workload. ■ Hyrise-R dispatcher monitors workload
■ Hyrise instances can join and leave the cluster
![Page 23: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/23.jpg)
Use Case In Memory Database Policy Language Integration
Chart 23
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
![Page 24: Secure and Scalable Infrastructures for Cloud Operations ... · Cloud3 Cloud2 Felix Eberhardt, FGT Betriebssysteme, 28.09.2017 . Dispatcher Hyrise Master Node Hyrise Replica Node](https://reader033.vdocuments.us/reader033/viewer/2022052015/602dd51f4cad8425110bf087/html5/thumbnails/24.jpg)
Felix Eberhardt, FGT Betriebssysteme, 28.09.2017
SSICLOPS
Questions
Chart 24