![Page 1: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/1.jpg)
Refactoring the OS around Explicit Resource Containers with Continuous Adaptation
Operating Systems Research in the Par Lab
Presented by Juan A. Colmenares, Gage Eads and Sarah Bird at the End of the Par Lab Symposium
May 30, 2013 Berkeley, CA
The OS Group Par Lab, UC Berkeley
http://tessellation.cs.berkeley.edu
![Page 2: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/2.jpg)
Acknowledgment
• Research supported by
– Intel (Award #024894)
– Microsoft (Award #024263)
– U.C. Discovery funding (Award #DIG07-102270)
• Additional support from Par Lab affiliates
– National Instruments, NEC, Nokia, NVIDIA, Samsung, Sun Microsystems
Disclaimer No part of this presentation necessarily represents the views
and opinions of the aforementioned sponsors
![Page 3: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/3.jpg)
• Hilfi Alkaff (UCB) • Krste Asanović (UCB Faculty) • Rimas Avižienis (UCB) • Davide Bartolini (Politecnico di
Milano / UCB) • Eric Battenberg (UCB) • Sarah Bird (UCB) • David Chou (UCB) • Juan Colmenares (UCB/Samsung) • Henry Cook (UCB) • Gage Eads (UCB) • Brian Gluzman (UCB) • Ben Hindman (UCB) • Steven Hofmeyr (LBL) • Eduardo Huerta (ICSI) • Costin Iancu (LBL) • Israel Jacques (UCB)
• John Kubiatowicz (Lead, UCB Faculty) • Albert Kim (UCB) • Kevin Klues (UCB) • Akihito Kohiga (NEC) • Eric Love (UCB) • Rose Liu (MIT) • Miquel Moretó (UCB/UPC) • Nitesh Mor (UCB) • Paul Pearce (UCB) • Heidi Pan (MIT/Intel) • Nils Peters (UCB/Qualcomm) • Barret Rhoden (UCB) • Eric Roman (LBL) • Ian Saxton (UCB) • John Shalf (LBL) • Burton Smith (MSR) • Andrew Waterman (USB) • David Wessel (UCB Faculty) • David Zhu (UCB)
Team and Collaborators
![Page 4: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/4.jpg)
At The Beginning: A.D. 2008
Many-core trend
Mixed workloads on client devices
High throughput parallel apps
Interactive apps
Real-time apps
BIP
S
Cores
Users’ expectations increase with core count You got more, then
![Page 5: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/5.jpg)
Common OS Anecdotes
• Screen freezes when running a heavy compile job
• Video chat becomes choppy when a local app starts
• Impossible for me to watch a video with a scientific simulation in the background
• Hey OS, what can I do? Sometimes it goes …
![Page 6: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/6.jpg)
Can We Solve Those Problems? Can We Reinvent the OS to … ?
• Take advantage of many-core platforms
• Properly serve simultaneous applications of different types and with conflicting requirements
• Meet users’ expectations about performance
and started Tessellation OS, Lithe, and PACORA
![Page 7: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/7.jpg)
Goals in Tessellation OS
• Support a dynamic mix of high-throughput parallel, interactive, and real-time applications
• Allow applications to deliver guaranteed or at least consistent performance
• Enable adaptation to changes in the application mix and resource availability
![Page 8: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/8.jpg)
Focus on Resources to Provide Performance Guarantees
• What do we want to guarantee?
– Throughput (e.g., requests/sec)
– Latency to response (e.g., service time)
– Others: energy/power budget
• What type of guarantees?
– Probabilistic with high confidence
• The “impedance-mismatch” problem
– Service Level Agreements (SLAs) indicate properties that programmer/user wants
– The resources required to satisfy the SLA are not things that programmer/user really understands
![Page 9: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/9.jpg)
Application1
QoS-aware Scheduler
Block Service
QoS-aware Scheduler
Network Service
QoS-aware Scheduler
GUI Service
Channel
Running System
(Data Plane)
Application2
Channel
Performance Reports
Resource Assignments
Resource Allocation (Control Plane)
Partitioning
and
Distribution
Observation
and
Modeling
Adaptive Resource-Centric Computing (ARCC) [DAC’13]
Cell Cell
Cell
![Page 10: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/10.jpg)
Application1
QoS-aware Scheduler
Block Service
QoS-aware Scheduler
Network Service
QoS-aware Scheduler
GUI Service
Channel
Running System
(Data Plane)
Application2
Channel
Performance Reports
Resource Assignments
Resource Allocation (Control Plane)
Partitioning
and
Distribution
Observation
and
Modeling
Adaptive Resource-Centric Computing (ARCC)
Cell Cell
Cell
Cells: Performance-Isolated Resource Containers • Provide guaranteed access to assigned resources • Give full user-level control of the resources
![Page 11: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/11.jpg)
Application1
QoS-aware Scheduler
Block Service
QoS-aware Scheduler
Network Service
QoS-aware Scheduler
GUI Service
Channel
Running System
(Data Plane)
Application2
Channel
Performance Reports
Resource Assignments
Resource Allocation (Control Plane)
Partitioning
and
Distribution
Observation
and
Modeling
Adaptive Resource-Centric Computing (ARCC)
Cell Cell
Cell
Customizable User-level Runtimes in Cells • To best meet applications’ needs
![Page 12: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/12.jpg)
Application1
QoS-aware Scheduler
Block Service
QoS-aware Scheduler
Network Service
QoS-aware Scheduler
GUI Service
Channel
Running System
(Data Plane)
Application2
Channel
Performance Reports
Resource Assignments
Resource Allocation (Control Plane)
Partitioning
and
Distribution
Observation
and
Modeling
Adaptive Resource-Centric Computing (ARCC)
Cell Cell
Cell
OS Services with QoS Guarantees • Reside in dedicated cells, have exclusive control
over devices, and arbitrate access to them
![Page 13: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/13.jpg)
Application1
QoS-aware Scheduler
Block Service
QoS-aware Scheduler
Network Service
QoS-aware Scheduler
GUI Service
Channel
Running System
(Data Plane)
Application2
Channel
Performance Reports
Resource Assignments
Resource Allocation (Control Plane)
Partitioning
and
Distribution
Observation
and
Modeling
Adaptive Resource-Centric Computing (ARCC)
Cell Cell
Cell
Adaptive Resource Allocation • Automatically discovers the mix of resource
assignments that maximizes overall system utility
![Page 14: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/14.jpg)
Two-Level Scheduling
Level 1 Coarse-grained Resource Allocation and Distribution
Level 2 Fine-grained Application-specific Scheduling
• Chunks of resources distributed to applications (Global Decisions)
• Apps use their resources in any way they see fit (Local Decisions)
Split
Monolithic CPU and Resource
Scheduling
into two pieces
![Page 15: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/15.jpg)
Space-Time Partitioning
Time
Spac
e
Yellow partition grows due to adaptation
Spatial Partition •Key for performance isolation
Spatial partitioning is not static and may vary over time •Partitions can be time multiplexed; resources are gang-scheduled
•Partitioning adapts to system’s needs
• Each partition receives a vector of basic resources • A partition may also receive
– Exclusive access to other resources (e.g., a device) – Guaranteed fractional services from other partitions
![Page 16: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/16.jpg)
The Cell: Our Partitioning Abstraction User-level Software Container
with Guaranteed Access to Resources
2nd-level Scheduling
2nd-level Mem Mgmt
Address Space A
Address Space B
Cell A
Task
Time
Spac
e
Cell B
• Full control over resources it owns when mapped to hardware
• Resources exported to user-level • Adaptive user-level runtimes • Efficient inter-cell communication
channels
Yellow partition grows due to adaptation
![Page 17: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/17.jpg)
Basis of a Component-based Model with Composable Performance
• Applications = Set of interacting components deployed on different cells
– Applications split into performance-incompatible and mutually distrusting cells with controlled communication
– OS Services are independent servers that provide QoS
Application Component
Device Drivers
File Service
Real-time Cell
Core Application
Parallel Library
Channel
Channel
Storage Device
![Page 18: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/18.jpg)
Customizable User-Level Runtimes Lithe: A framework for hierarchical cooperative
user-level schedulers [PLDI’10]
• Non-preemptive scheduling • Key abstraction
– Hardware threads (harts) – No oversubscription!
• Enables efficient composition of parallel libraries
• http://lithe.eecs.berkeley.edu
harts
Tessellation Kernel (Partition Support)
Application
Library A
Library B
Sched 1 Sched 2 Sched 3
Lithe Runtime
Cell
Hardware cores
[PLDI’10] H. Pan, B. Hindman, K. Asanovic. Composing parallel software efficiently with Lithe.
![Page 19: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/19.jpg)
• Available preemptive schedulers – Round-robin (and pthreads) – EDF and Fixed Priority – Multiprocessor Constant Bandwidth
Server (M-CBS) [ECRTS’04]
– Juggle: A load balancer for SPMD applications [CLUSTER’12]
• Able to handle cell resizing Tessellation Kernel (Partition Support)
Application
Cell
[ECRTS’04] S. Baruah et al. Executing aperiodic jobs in a multiprocessor constant-bandwidth server implementation. ECRTS'04. [CLUSTER’12] S. Hofmeyr, J. Colmenares et al. Juggle: Addressing extrinsic load imbalances in SPMD applications on multicore computers. Cluster Computing Journal.
PULSE Framework
Scheduler X
Hardware cores
Timer interrupts
Customizable User-Level Runtimes PULSE: A framework for
Preemptive User-Level SchEdulers
![Page 20: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/20.jpg)
GUI Service [CATA’ 12] An OS Service with QoS Guarantees
• Exploits task parallelism for improved service times • Provides differentiated service to applications and
soft service-time guarantees
[CATA’12] A. Kim, J. Colmenares, et al. A soft real-time, parallel GUI service in Tessellation many-core OS. [Best Paper Award]
![Page 21: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/21.jpg)
Nano-X vs. GUI Service Service times for 4 30-fps video players and 4 60-fps video players,
each sending 1000 expensive requests
Missed Deadlines
Each bar represents 4 video clients.
Above each bar is the total number of
deadlines missed for the group.
Max
Mean
Min
(#): Allocated hardware threads
![Page 22: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/22.jpg)
• Supports reservations and proportional share of bandwidth
– Using mClock scheduling algorithm [OSDI’10] (on top of PULSE)
• NIC driver is entirely contained in user-space – No system calls when transmitting and receiving buffers
[DAC’13] J.A. Colmenares, G. Eads, et al. Tessellation: Refactoring the OS around explicit resource containers with continuous adaptation. [JAES’13] J.A. Colmenares, G. Eads, et al. A multi-core operating system with QoS-guarantees for network audio applications. [OSDI’10] A. Gulati et al. mClock: handling throughput variability for hypervisor IO scheduling.
Network Service [DAC’13, JAES’13] An OS Service with QoS Guarantees
(Avg. throughput = 125.2 KB/s)
![Page 23: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/23.jpg)
Adaptive Resource Allocation in Tessellation OS
[DAC’13] J.A. Colmenares, G. Eads et al. Tessellation: Refactoring the OS around explicit resource containers with continuous adaptation.
![Page 24: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/24.jpg)
Syst
em P
enal
ty
Continuously minimize the penalty of the system
(subject to restrictions on the total amount of resources)
Response Time1
Pen
alty
1
Response Time2 ((0,2), …, (n-1,2)) Response Time2
Response Timei
Penalty Function
Pen
alty
2
Pen
alty
i
Set
of
Ru
nn
ing
Ap
plic
atio
ns
Resource Allocation using Convex Optimization with Online Application Performance Models
Response Time Function
Response Time1((0,1), …, (n-1,1))
Speech Recognition
Stencil
Graph Traversal
Response Timei((0,i), …, (n-1,i))
PACORA
[HOTPAR’11] S. Bird and B. Smith. PACORA: Performance aware convex optimization for resource allocation.
![Page 25: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/25.jpg)
Known Facts and Lessons Learned
• [KF] Implementing an OS from scratch is challenging
• [KF] Supporting IO devices is very important
• [LL] Tessellation’s structuring redistributes complexity
– Lot of complexity moved from the kernel to user-level runtimes
– Contending factor: overhead
• [LL] Having a simple kernel is very beneficial
– Easier to reason about it, especially when providing performance guarantees
• [LL] Coordination between kernel and cell’s user-level runtime is tricky (e.g., during cell resizing)
– Not many LOCs, but very subtle issues and difficult to debug
![Page 26: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/26.jpg)
Summary • Challenge: Reinventing the OS for many-core platforms
– Properly serve simultaneous applications of different types – Meet users’ performance expectations
• Approach: Adaptive Resource-Centric Computing – Focuses on Space-Time Partitioning and Two-Level
Scheduling – Includes
• Cells: Resource containers • Customizable user-level runtimes • OS services with QoS guarantees • Adaptive resource allocation
• Implementation: Tess OS, Lithe, PULSE, and PACORA • Effectiveness: Demonstrated in publications and
demos!
![Page 27: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/27.jpg)
Demos on Tessellation OS
• Adaptive Resource Centric Computing
– Entirely on Tessellation
• Live Musical Performance – A synthesizer performing parallel
real-time audio processing and controlled via the SLABS multi-touch interface
• Million Song Recommendation (Pardora) System
– Specialized code on top of TBB/Lithe, plus python code
• Virtual Instrument – One of the backends
![Page 28: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/28.jpg)
Next
• Demonstration of Adaptive Resource Centric Computing
– Gage Eads and Sarah Bird, UC Berkeley
• Testimonial
– Dave Probert, Microsoft
![Page 29: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/29.jpg)
Adaptive Resource Centric Computing Demonstration
![Page 30: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/30.jpg)
Network
Cores
Adaptive Resource Centric Computing Demonstration
Bandwidth Hog
Video Players (1 vid per thread) psearchy
• TCP client application
• Single-threaded • Goal: consume
maximum bandwidth
• Receives video stream from TCP connection
• 2 video sizes • Performs H.264
decoding • Goal: 30 FPS
• Parallel file indexing benchmark
• Scalable work-queue based parallelism
• Goal: maximize indexing throughput
(QoS from network service) (QoS from kernel)
![Page 31: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/31.jpg)
THANKS
Come and see our demos!
Questions?
![Page 32: Refactoring the OS around Explicit Resource Containers with …parlab.eecs.berkeley.edu/sites/all/parlab/files... · 2013-06-07 · Adaptive Resource Centric Computing Demonstration](https://reader034.vdocuments.us/reader034/viewer/2022043019/5f3b392747290722bf147395/html5/thumbnails/32.jpg)
Gang Scheduling in Tessellation
• No need of inter-core communication (in the common case) due to use of synchronized clocks
• Different time-multiplexing policies for cells
Identical or Consistent Schedules
Multiplexer
Core 2 Core 1
Gang-Scheduling Algorithm
Core 3 Core 0
Multiplexer
Gang-Scheduling Algorithm
Multiplexer Gang-
Scheduling Algorithm
Multiplexer
Gang-Scheduling Algorithm
No
Mu
xed