The Role of Virtualization in Exascale Production Systems

Jack LangeAssistant Professor

University of Pittsburgh

Where are exascale OSes headed?• Linux is accepted on Petascale

– More than 80% of Top500– Significant push to run Linux on production systems

• Pros– Complex systems are easier to manage– Existing codebase– Familiar and portable environment

• Cons– Not as responsive to application requirements– Exascale is not a primary design goal

• Linux is great for Ops, but only OK for Apps

Lightweight Kernels (LWKs)• What do Apps need?

– Simple memory management with little to no overhead– Low noise characteristics– Ability to perform large “bulk” I/O– …

• Traditional strengths of Lightweight Kernel architectures– Application driven resource management

• Lightweight kernels still have utility– But probably not as the primary system OS– App selectable runtime environment

• Challenge– Can we provide a lightweight environment on a heavyweight OS?

Virtualization on Linux• Bypass Linux management layers

• Palacios selectively takes over resource management– Memory, devices, CPUs– Repurpose existing mechanisms

• Allocate large chunks of resources and manage them internally

• Kernel module– Does not require kernel modifications – Implements lightweight interface– Compatible with CNL (~2.6.32)

Available in Palacios 1.3 (Nov. 2011)

Virtualized Dual Software Stack

Hardware

Palacios VMM

Lightweight Kernel

ManagementProcesses

+System Daemons

Linux Module Interface

Linux derived Compute Node

OS

HPC Application

Palacios ResourceManagers

• LWKs manage large resource allocations from VMM• Apps can choose which allocators/management layers to deploy on• LWKs can be co-designed with Apps and deployed on production systems

Performance isolationMemory Performance (GB/s)

Noise Comparison between native Linux and virtual LWK

Thank you• Jack Lange

– jacklange@cs.pitt.edu– http://www.cs.pitt.edu/~jacklange

• V3Vee Project – http://www.v3vee.org