1

CS-580K/480K Advanced Topics in Cloud Computing

Miscellaneous Topics

2

Recall: Micro-Services

“service-orientedarchitecture composed of loosely coupledelements that have bounded contexts”

Web Servers +

BussinessLogic +

Database Servers

s1

s3

s2

s4

s5

Design

Pattern

Micro-services

Each time do one thing and do it well

Monolithic Application

Break-down into microservices

Make each micro service HA

Protect against regional outages

Region A Region B

Explosion in number ofcontainers / processes:

Increase of infrastructure cost footprint

Increase of operationalmanagement cost andcomplexity

Platform Manageability and Cost Challenge

A cloud-native platform for

short-running, stateless computationand

event-driven applicationswhich

scales up and down instantly and automaticallyand

charges for actual usage at a millisecond granularity

Serverless Computing?

Serverless deployment & operations model

No serversJust code

Server-less means no servers?Or worry-less about servers? User provides just code

While the server provisioning and code execution is taken care of by serverless

platforms

Current Platforms for Serverless

Azure Functions

AWS Lambda

Kubernetes

Google Functions

Red-Hat

IBM Cloud Functions

Why is Serverless Attractive?

• Makes app development & ops dramatically faster, cheaper, easier

• Drives infrastructure cost savings

• Avoids vendor lock-in, that you can use other provider’s services

Source: Jason McGee, IBM; Serverless Conference 2017.

Key factors for infrastructure cost savings

PyWren: a massive data framework for Lambda

• Open Source MapReduce framework using Lambda• Word count job on 83M items is only 17% slower than PySpark

running on dedicated servers.• Sort 1TB data in 3.4 minutes (Spark 100TB in 23 min)

Occupy the Cloud: Distributed Computing for the 99%, Eric Jonas, Qifan Pu, Shivaram Venkataraman, Ion Stoica, Benjamin Recht, https://arxiv.org/abs/1702.04024

https://github.com/pywren/pywrenhttp://pywren.io/

You could investigate this system!

Microservices

Mobile Backends

IoT

Modest Stream Processing

Bots, ML Inferencing

Serverless is good forshort-running

stateless

event-driven

Serverless is not good forlong-running

stateful

number crunching

Databases

Deep Learning Training

Heavy-Duty Stream Analytics

Numerical Simulationf(x)

What is Serverless good for?

Service integration

Video Streaming

Sources1. Scalable Microservices at Netflix. Challenges and Tools of the Trade

2. Introduction to Microservices: https://www.slideshare.net/AmazonWebServices/introduction-to-microservices-66320469/15

3. Learn Kubernetes Basics: https://kubernetes.io/docs/tutorials/kubernetes-basics/

12

13

Shift in Paradigm

• Compute has been predominantly cloud dominated▪ Centralization allows scaling

▪ Avoid CapEx for infrastructure

▪ IoT and mass device adoption causes cloud overuse/bottlenecks

• Shifting compute closer to devices/sensors using cloudlets, micro datacenters, or fog nodes

14

What is Edge?

• Edge computing is computing that’s done at or near the source of the data, instead of relying on the cloud at one of a dozen data centers to do all the work.

15

Motivation

• Latency• Face recognition takes 370–620 ms, speech

recognition takes 300–450 ms for short phrases, only 4ms to tell that a sound is a human voice, VR applications that use headtracked systems require latencies of less than 16 ms to achieve perceptual stability

• Cloud cannot guarantee low latency which is important for certain types of applications.

• Exponential growth of IoT devices worsens the latency issue

• We need an in-between solution – between IoT devices and Cloud

16

Proximity - “location, location, and location”

• Proximity of Edge helps in 4 ways:• Responsiveness - physical proximity helps with end to end latency,

bandwidth, and low jitter

• Scalability - cumulative ingress of many end devices causes high bandwidth, edge offloading helps lower this bandwidth

• Privacy policy enforcement - the edge can act as a privacy filter to impose the policy prior to releasing the data to the cloud

• Masking outages - edge serves as fallbacks in cloud outages

17

Future of Edge

• Technical challenges

• Distributed control SW mechanism and algorithms

• Management of distributed edge clusters - have to provide cloud advantage of lower management with centralized infrastructure

• Weaker perimeter security of edge compared to cloud datacenters

• Weak reliability

• Non-technical challenges

• Lack of unique applications and services that leverage edge computing

18

Future of Edge

• Three important trends that coincide with edge compute• SDN - software defined networking & NFV - network function

virtualization

• Ultra low latency wireless networks (e.g., 5G)

• Compute capabilities of wearables, smartphones, and other “extreme edge” devices

19

Sources

• What is edge computing and why it matters: https://www.networkworld.com/article/3224893/what-is-edge-computing-and-how-it-s-changing-the-network.html

• Demystifying Edge Computing -- Device Edge vs. Cloud Edge：https://www.forbes.com/sites/janakirammsv/2017/09/15/demystifying-edge-computing-device-edge-vs-cloud-edge/#1038e0613633

• N. Mor et al., "Toward a Global Data Infrastructure," in IEEE Internet Computing, vol. 20, no. 3, pp. 54-62, May-June 2016.

• Build a blockchain network for trusted IoT：https://developer.ibm.com/technologies/iot/

• What is Edge Computing? https://www.ge.com/digital/blog/what-edge-computing

20

21

Network Function Virtualization

• It is hard to bring new services into today’s networks due to the complexity of networks

• NFV leverages virtualization technology to separate software instance from hardware platform

22

Benefits

• Reduce capital investment and energy consumption

• Decrease the time to market of a new service

• Introduce targeted and tailored services based on customer need.

23

NFV vs. SDN

• NFV refers to the virtualization of network components/functions.

• SDN refers to a network architecture that injects automation and programmability into the network by decoupling network control and forwarding functions.

24

Challenges of NFV

• Performance

• Transition from traditional network functions to NFV

• Dynamically instantiate and place virtual appliances.

• Reliability and stability

• Security

25

In Summary

• NFV decouples network functions from the hardware

• Those network functions are called virtual network functions (VNFs)

• VNFs run in virtual machines on commercial off-the-shelf (COTS) hardware

• COTS hardware is typically x86 servers

• NFV is used by software-defined networks

• NFV allows for scaling of VMs to handle changes in data center traffic

• NFV theoretically simplifies network operation, but not always in practice

• SDN and NFV are related but independent of each other

26

Sources

• Network Function Virtualization: Challenges and Opportunities for Innovations: http://www.ttcenter.ir/ArticleFiles/ENARTICLE/3431.pdf

• ETSI, “NFV - Update White Paper,” Oct 2013, http://portal.etsi.org/NFV/NFV_White_Paper2.pdf

• ETSI, “Network Function Virtualization,” http://www.etsi.org/technologiesclusters/technologies/nfv

• ETSI, “Architectural Framework,” Oct 2013, http://www.etsi.org/deliver/etsi_gs/NFV/001_099/002/01.01.01_60/gs_NFV 002v010101p.pdf

• ETSI, “NFV Terminology for Main Concepts in NFV,” Oct 2013, http://www.etsi.org/deliver/etsi_gs/NFV/001_099/003/01.01.01_60/gs_NFV 003v010101p.pdf

• ETSI, “NFV Use Cases,” http://www.etsi.org/deliver/etsi_gs/NFV/001_099/001/01.01.01_60/gs_NFV 001v010101p.pdf

• ETSI, “NFV Virtualization Requirements,”, Oct 2013, 17 pp. http://www.etsi.org/deliver/etsi_gs/NFV/001_099/004/01.01.01_60/gs_NFV 004v010101p.pdf

• M. Cohn, “NFV, An Insider’s Perspective: Part 1: Goals, History, and Promise,” Sep 2013, http://www.sdncentral.com/education/nfv-insidersperspective-part-1-goals-history-promise/2013/09/

• M. Cohn, “NFV Insider’s Perspective, Part 2: There’s a Network in NFV – The Business Case for SDN,” Sep 2013, http://www.sdncentral.com/education/nfv-insiders-perspective-part-2- theres-network-nfv-business-case-sdn/2013/09/

27

28

Monolithic Kernel & Microkernel

Application

logic

File

server

Network

stack

Microkernel

Monolithic Kernel

Application

logic

Memory

Management

Memory

ManagementFile

server

Network

stack

• Microkernel: excluding system services from kernel space results in much smaller code base, and is a promising way to make the system stable and secure.

• Pragmatic

Pragmatic Challenges

• Performance – system services move to user space

• Difficult to implement applications – rewriting everything from scratch?

• Cannot support existing legacy applications.

30

Application

logic

File

server

Network

stack

Microkernel

Memory

Management

State of the Art – VM Virtualization

VFS, System calls

IPC, File systems

Scheduler, Virtual

memory

Device drivers

Applications

Microkernel

Virtual Machine Monitor

Virtual Machine

• Performance overhead

• Security concerns

31

A Light-Weight Alternative?

VFS, System calls

IPC, File systems

Scheduler, Virtual

memory

Device drivers

Applications

Microkernel

Virtual Machine Monitor

Virtual Machine

• A tailored kernel with only required drivers and the basic support routines for these drivers to function

32

A Light-Weight Alternative?

System

calls

Memory

Allocation

TCP/IP

Applications

Microkernel

Virtual Machine Monitor

Virtual Machine

• A tailored kernel with only required drivers and the basic support routines for these drivers to function

• A minimal “kernel”

• No user/kernel space separation neededPCI/NIC driver

33

A Light-Weight Alternative?

Procedure

calls

Memory

Allocation

TCP/IP

Applications

Microkernel

Virtual Machine Monitor

Virtual Machine

• A tailored kernel with only required drivers and the basic support routines for these drivers to function

• A minimal “kernel”

• No user/kernel space separation needed

• Fast access to kernel from user applications

PCI/NIC driver

34

A Light-Weight Alternative?

Procedure

calls

Memory

Allocation

TCP/IP

Applications

Microkernel

Virtual Machine Monitor

Virtual Machine

• A tailored kernel with only required drivers and the basic support routines for these drivers to function

• A minimal “kernel”

• No user/kernel space separation needed

• Fast access to kernel from user applications

• A thin, platform-specific software layer to access underlying resources

PCI/NIC driver

35

A Light-Weight Alternative?

Procedure

calls

Memory

Allocation

TCP/IP

Applications

Microkernel

Virtual Machine

• A tailored kernel with only required drivers and the basic support routines for these drivers to function

• A minimal “kernel”

• No user/kernel space separation needed

• Fast access to kernel from user applications

• A thin, platform-specific software layer to access underlying resources

PCI/NIC driver

Hypervisor

Unikernel

36

Unikernel Container with Security

Bins/Libs

Applications

Host OS

Docker Container

• Run legacy applications as native unikernelthreads in a self-contained environment, including

• application code

• dependencies (libraries)

• required system services (e.g., file systems and network devices)

• and hypervisor

37

Docker

Customized, tailored

kernel

Applications

Microkernel

Unikernel Container

Hypervisor

Bins/Libs

Conclusions

• “Small” kernel space -> “Big” user space

• More design exploration in the user space is needed

• Unikernel containers could serve as one option to support legacy applications in a light-weight manner

38

39