Real-Time Cloud Computing

ChenyangLuCyber-PhysicalSystemsLaboratoryDepartmentofComputerScienceandEngineeringh<p://www.cse.wustl.edu/~lu/

Internet of Things

Ø  Convergence of q  Miniaturized hardware: integrate processor, sensors and radios.q  Low-power wireless: connect millions of devices to the Internet.q  Data analytics: make sense of sensor data.q  Rea-time cloud: scalable real-time computing.

Ø  Enable real-time control of physical environments

R.Dor,G.Hackmann,Z.Yang,C.Lu,Y.Chen,M.KollefandT.C.Bailey,ExperienceswithanEnd-To-EndWirelessClinicalMonitoringSystem,ConferenceonWirelessHealth(WH'12),October2012.

Real-TimeCloud

Ø  Internet of Things à large-scale sensing and controlq  Real-time data analytics (aka Big Data)

q  Smart manufacturing, smart transportation, smart grid…

Ø  Example: Intelligent Transportationq  Data center collects data from cameras and roadside detectors.

q  Control traffic signals and message signs in real-time.

q  Transportation information feed to drivers.

q  SCATS @ Sydney: controlling 3,400 signals at 1s round-trip latency.

Ø  Latency-sensitive applications, e.g., cloud gamingq  Xbox One: cloud offloading computation of environmental elementsq  Sony acquired Gaikai, an open cloud gaming platform.

3

EmbeddedSystemVirtualiza7on

Ø Consolidate 100 ECUs à ~10 multicore processors.Ø  Integrate multiple systems on a common platform.

q  Infotainment on Linux or Android

q  Safety-critical control on AUTOSARq  Virtualization: COQOS, Integrity Multivisor, Xen automotive

Ø Must preserve real-time performance on a virtualized platform!

10/14/16 4

Source:h<p://www.edn.com/design/automoYve/4399434/MulYcore-and-virtualizaYon-in-automoYve-environments

Ø  Existing hypervisors provide no guarantee on latencyq  Xen: credit scheduler, [credit, cap]

q  VMware ESXi: [reservation, share, limitation]

q  Microsoft Hyper-V: [reserve, weight, limit]

Ø Clouds lack service level agreement on latencyq  EC2, Compute Engine, Azure: #VCPUs

5

Cloudisreal-7metoday

Current clouds provision resources, not latency!

TowardsReal-TimeCloudsØ  Support real-time applications in the cloud.

q  Latency guarantees for tasks running in virtual machines (VMs).

q  Real-time performance isolation between VMs.

q  Resource sharing between real-time and non-real-time VMs.

Ø Multi-level real-time performance provisioning.q  RT-Xen à real-time VM scheduling in a virtualized host.

q  VATC à real-time network I/O in a virtualized host.q  RT-OpenStack à real-time cloud resource management.

6

VATC: Real-Time Communication

RT-O

penS

tack

XenVirtualiza7onArchitecture

Ø Xen: type-1, baremetal hypervisor q  Domain-0: drivers, tool stack to control VMs.

q  Guest Domain: para-virtualized or fully virtualized OS.

Ø  Scheduling hierarchyq  Xen schedules VCPUs on PCPUs.

q  Guest OS schedules threads on VCPUs.

q  Xen credit scheduler: round-robin with proportional share.

7

PCPUs

OS Sched

Xen Scheduler

OS Sched OS Sched

VCPUReal-TimeTask

RT-Xen

Ø  Real-time schedulers in the Xen hypervisor.Ø  Provide real-time guarantees to tasks in VMs.

Ø  Incorporated in Xen 4.5 as the real-time scheduler.

8

RT-Xen

h<ps://sites.google.com/site/realYmexen/

S. Xi, M. Xu, C. Lu, L. Phan, C. Gill, O. Sokolsky and I. Lee, Real-Time Multi-Core Virtual Machine Scheduling in Xen, ACM International Conference on Embedded Software (EMSOFT'14), October 2014.

Composi7onalScheduling

Ø Analytical real-time guarantees to tasks running in VMs.Ø  VM resource interfaces

q  A set of VCPUs each with resource demand <period, budget >

q  Hides task-specific informationq  Computed based on compositional scheduling analysis

9

Resource Interface Resource Interface

Resource Interface

Hypervisor

Virtual Machines

Workload Workload

Scheduler

Scheduler

Scheduler

Real-TimeSchedulerDesignØ  Global scheduling

q  Allow VCPU migration across cores

q  Work conserving – utilize any available cores

q  Migration overhead and cache penalty

Ø  Partitioned schedulingq  Assign and bind VCPUs to cores

q  Cores may idle when others have work pending

q  No migration overhead or cache penalty

Ø  Enforce resource interface through budget managementq  Periodic server vs. deferrable server

Ø  Priority: Earliest Deadline First vs. Deadline Monotonic

10

RT-Xenvs.Xen

11

•  Xen misses deadlines at 22% of CPU capacity.

•  RT-Xen delivers real-time performance at 78% of CPU capacity.

VirtualizedNetworkI/O

Ø Xen handles all network traffic through Dom0Ø  Real-time and non-real-time traffic share Dom0

q  CPU and network contention

Ø  Long delays for real-time traffic in virtualized hosts

XenHypervisor

NetworkComponents

Non Real-Time

App

Dom2

CPUMemory Storage

Real-Time App

Dom1Dom0

NIC

12

NetworkI/OinVirtualizedHostsØ  Linux Queueing Discipline

q  Rate-limit and shape flows

q  Prioritization or fair packet scheduling

Ø  Priority inversion in virtualization componentsq  between transmissionsq  between transmission and reception

Ø  VATC: Virtualization-Aware Traffic Controlq  Process packets in prioritized kernel threadsq  Dedicated packet queues per priority NIC

QueueingDiscipline

Dom0

Real-Time App

Dom1Non- Real-Time App

Dom2

Virtualiza7onComponents

13

C. Li, S. Xi, C. Lu, C. Gill and R. Guerin, Prioritizing Soft Real-Time Network Traffic in Virtualized Hosts Based on Xen, IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS'15), April 2015.

Real-TimeTrafficLatency

VATC reduces priority inversion à lower latency for real-time traffic.

14

10 16 32 64 128 256 512 1024 Dyn Cons Dyn0

0.5

1

1.5

2

2.5

3

3.5

4

Roun

d−tri

p La

tenc

y ( m

s)

Interrupt Interval (µs)

Prio, Dom0−3.18FQ_CoDel, Dom0−3.18VATC

•  Medianround-triplatencyofreal-Ymetraffic.•  CPUcontenYonfromtwosmall-packetinterferingstreams.

VirtualizedHostàCloud

Ø  Provide real-time performance to real-time VMsØ Achieve high resource utilization

15

OpenStackLimita7ons

Ø  Popular open-source cloud management system

Ø  VM resource interfaceq  Number of VCPUsq  Not real-time

Ø  VM-to-host mappingq  Filtering (admission control)

•  VCPU-to-PCPU ratio (16:1), max VMs per host (50)

•  Coarse-grained admission control for CPU resources

q  Ranking (VM allocation)

•  Balance memory usage

•  No consideration of CPU resources

16

Manager

Host Host Host

VM VM VM VM VM

RT-OpenStack

Ø Co-hosting real-time VMs with non-real-time VMs

Ø Deliver real-time performanceq  Support RT-Xen resource interfaceq  Real-time-aware VM-to-host mapping

Ø Achieve high resource utilizationq  Co-locate non-real-time VMs with real-time VMsq  Non-real-time VMs consume remaining resources without affecting the

real-time performance of real-time VMs

17

S. Xi, C. Li, C. Lu, C. Gill, M. Xu, L. Phan, I. Lee, O. Sokolsky, RT-OpenStack: CPU Resource Management for Real-Time Cloud Computing, IEEE International Conference on Cloud Computing (CLOUD'15), June 2015.

RT-OpenStack:VM-to-HostMapping

Ø Admission control: RT-Filterq  Accept real-time VMs based on real-time schedulability and memory

q  Consider only accepted real-time VMs

Ø  VM allocation: RT-Weigherq  Balance CPU utilization

q  Consider only accepted real-time VMs

18

ResourceInterface AdmissionControl VMAlloca7on

Real-TimeVMs {<period,budget>} Schedulability+Memory CPUUYlizaYon

Non-Real-TimeVMs BestEffort Memory Memory

OpenStack

19

13%

36%

31% 61% 37%

75%30%29%

47% 32%

73%HadoopfinishYme:314seconds

Ø  Overload four hosts with real-time VMs à deadline misses. Ø  Two hosts running non-real-time VMs only.

Ø  Unbalanced distribution of real-time domains.

RT-OpenStack

20

0%

0%0%

0%

0% 0%

0%

0%0% 0%

0%

HadoopfinishYme:435seconds

Ø  Schedulability guarantees for real-time VMs à no deadline miss.Ø  Distribute real-time VMs across hosts.

Ø  Hadoop makes progress using remaining CPU resources.

ConclusionsØ New applications demand real-time cloud services.

q  Internet-scale monitoring and control.

q  Latency-sensitive cloud applications.

Ø  Towards real-time cloudØ  RT-Xen: real-time VM scheduling in virtualized hosts.Ø  VATC: real-time network I/O in virtualized hosts.

Ø  RT-OpenStack: real-time guarantees at high CPU utilization.

Ø  RTM: real-time messaging.

21

VATC: Real-Time Communication

RT-O

penS

tack