evaluating gpu passthrough in xen for high performance cloud computing

Evaluating GPU Passthrough in Xen for High Performance Cloud

Computing Andrew J. Younge1, John Paul Walters2, Stephen P. Crago2, and Geoffrey C. Fox1

1 Indiana University

2 USC / Information Sciences Institute

Where are we in the Cloud?

• Cloud computing spans may areas of expertise

• Today, focus only on IaaS and the underlying hardware

• Things we do here effect the entire pyramid!

http://futuregrid.org 2

Motivation

• Need for GPUs on Clouds– GPUs are becoming commonplace in scientific

computing– Great performance-per-watt

• Different competing methods for virtualizing GPUs– Remote API for CUDA calls– Direct GPU usage within VM

• Advantages and disadvantages to both solutions

3

Front-end GPU API • Translate all CUDA calls into remote method

invocations• Users share GPUs across a node or cluster• Can run within a VM, as no hardware is needed,

only a remote API• Many implementations for CUDA

– rCUDA, gVirtus, vCUDA, GViM, etc..

• Many desktop virtualization technologies do the same for OpenGL & DirectX

http://futuregrid.org 4

Front-end GPU API

http://futuregrid.org 5

Front-end API Limitations

• Can use remote GPUs, but all data goes over the network– Can be very inefficient for applications with non-

trivial memory movement

• Usually doesn’t support CUDA extensions in C– Have to separate CPU and GPU code– Requires special decouple mechanism

• Cannot directly drop in solution with existing solutions.

http://futuregrid.org 6

Direct GPU Passthrough

• Allow VMs to directly access GPU hardware• Enables CUDA and OpenCL code• Utilizes PCI-passthrough of device to guest VM

– Uses hardware directed I/O virt (VT-d or IOMMU)– Provides direct isolation and security of device– Removes host overhead entirely

• Similar to what Amazon EC2 uses

http://futuregrid.org 7

Direct GPU Passthrough

http://futuregrid.org 8

9

Hardware SetupSandy Bridge + Kepler Westmere + Fermi

CPU (cores) 2x E5-2670 (16) 2x X5660 (12)

Clock Speed 2.6 GHz 2.6 GHz

RAM 48 GB 192 GB

NUMA Nodes 2 2

GPU 1x Nvidia Tesla K20m 2x Nvidia Tesla C2075

Type Linux Kernel Linux DistroNative Host 2.6.32-279 CentOS 6.4

Xen Dom0 4.2.22 3.4.53-8 CentOS 6.4

DomU Guest VM 2.6.32-279 CentOS 6.4

SHOC Benchmark Suite

• Developed by Future Technologies Group @ Oak Ridge National Laboratory• Provides 70 benchmarks

– Synthetic micro benchmarks– 3rd party applications– OpenCL and CUDA implementations

• Represents well-rounded view for GPU performance

http://futuregrid.org 10

http://futuregrid.org 11

http://futuregrid.org 12

http://futuregrid.org 13

http://futuregrid.org 14

Initial Thoughts

• Raw GPU computational abilities impacted less than 1% in VMs compared to base system– Excellent sign for supporting GPUs in the Cloud

• However, overhead occurs during large transfers between CPU & GPU– Much higher overhead for Westmere/Fermi test

architecture– Around 15% overhead in worst-case benchmark– Sandy-bridge/Kepler overhead lower

http://futuregrid.org 15

http://futuregrid.org 16

http://futuregrid.org 17

Discussion• GPU Passthrough possible in Xen!

– Results show high performance GPU computation a reality with Xen

• Overhead is minimal for GPU computation – Sandy-Bridge/Kepler has < 1.2% overall overhead– Westmere/Fermi has < 1% computational overhead, 7-25%

PCIE overhead

• PCIE overhead not likely due to VT-d mechanisms– NUMA configuration in Westmere CPU architecture

• GPU PCI Passthrough performs better than other front-end remote API solutions

http://futuregrid.org 18

Future Work

• Support PCI Passthrough in Cloud IaaS Framework – OpenStack Nova– Work for both GPUs and other PCI devices– Show performance better than EC2

• Resolve NUMA issues with Westmere architecture and Fermi GPUs

• Evaluate other hypervisor GPU possibilities • Support large scale distributed CPU+GPU

computation in the Cloudhttp://futuregrid.org 19

Conclusion

• GPUs are here to stay in scientific computing– Many Petascale systems use GPUs– Expected GPU Exascale machine (2020-ish)

• Providing HPC in the Cloud is key to the viability of scientific cloud computing.

• OpenStack provides an ideal architecture to enable HPC in clouds.

http://futuregrid.org 20

Thanks!

Acknowledgements:• NSF FutureGrid project

– GPU cluster hardware– FutureGrid team @ IU

• USC/ISI APEX research group

• Persistent Systems Graduate Fellowship

• Xen open source community

About Me:Andrew J. Younge

Ph.D CandidateIndiana University Bloomington, IN USAEmail – ajyounge@indiana.eduWebsite – http://ajyounge.com

http://portal.futuregrid.org

http://futuregrid.org 21

Extra Slides

http://futuregrid.org 22

FutureGrid: a Distributed Testbed

PrivatePublic FG Network

NID: Network Impairment Device

http://futuregrid.org 24

OpenStack GPU Cloud Prototype

http://futuregrid.org 25

~ 1.25%

26

~.64%

~3.62%

27

Overhead in Bandwidth

28