trillium: the code is the irhyu/talk/virt19-07-18-slides.pdf · svga2: tgsi as ir and visa native...

Trillium: The Code is the IR

Amogh Akshintala, Hangchen Yu,

Arthur M. Peters, Christopher J. Rossbach

#2A. Akshintala, H. Yu, A. M. Peters and C. J. Rossbach, Trillium, Virt’19

GPGPU Virtualization

End-to-end comparison of prior approaches

Lessons learnt:

● Virtual ISA unnecessary

● Para-virtual API remoting only feasible option

Brief Overview


General purpose computing on GPUs

↑↑↑

GPUCPU

Performance Gain

Scientific Computing

Machine Learning


The need to virtualize GPUsP

erf

orm

ance

Cost

P3.2xlarge1x V100, $2,200/mo

P2.xlarge

1x K80, $650/mo

g3s.xlarge1x M60, $540/mo

Credits to BitFusion Inc.


The need to virtualize GPUsP

erf

orm

ance

Cost

P3.2xlarge1x V100, $2,200/mo

P2.xlarge

1x K80, $650/mo

g3s.xlarge1x M60, $540/mo

Virtual V1000.75x V100, $1,500

Virtual V1000.5x V100, $1,000

Credits to BitFusion Inc.


Existing techniques are impractical

Sharing Interposition Isolation Compatibility Slowdown

Pass Through ❌ 1x

Full-virtualization

Everybody sacrifices somethingUser-mode API Remoting

Para-virtualization


Hypervisor

Virtual GPU

Full-virtualization

Native stack

Vendor Library

Vendor Driver

GPU

VM

Application

Vendor Library

Vendor Driver


✔️ ✔️ ✔️ ❌ 100x


User-mode API Remoting

Native stack

Hypervisor

Custom API Server

Vendor Library

Vendor Driver

GPU

VM

Application

Custom User-mode Library


✔️ ❌ ✔️/❌ ❌ 1.5x


Para-virtualization

Native stack

Hypervisor

Vendor Library

Vendor Driver

GPU

VM

Application

Custom User-mode Library

Interface Translator

Custom Guest Driver

Custom Virtual GPU


✔️ ✔️ ✔️ ❌ 10x


End-to-end performance comparison

Y. Suzuki, S. Kato, H. Yamada, K. Kono, “GPUvm: Why Not Virtualizing GPUs at the Hypervisor?”, ATC’14


More details in the paper...


Everybody sacrifices something


Full-virtualization ✔️ ✔️ ✔️ ❌ 100x

User-mode API Remoting ✔️ ❌ ✔️/❌ ❌ 1.5x

Para-virtualization ✔️ ✔️ ✔️ ❌ 10x


Para-virtual API-remoting...


Full-virtualization ✔️ ✔️ ✔️ ❌ 100x



Para-virtual API remoting ✔️ ✔️ ✔️ 1.5x


SVGA2: TGSI as IR and vISA

Native stack

Hypervisor

Vendor Library

Vendor Driver

GPU

VMApplication

Translator

OpenGL/DX11 Libraries

vmwgfx.ko

SVGA Device

TGSI, Tungsten Graphics Shader Infrastructure, is an intermediate language for describing shaders.

GLSL Code

TGSI

Native ISA


Xen-SVGA: computing support

Native stack

Hypervisor

Vendor Library

GPU

VMApplication

Mesa3D OpenCL Library

Nouveau

OpenCL Code

TGSI

Native ISA

TGSI LLVM Compiler

Nouveau

TGSI

Xen-SVGA Device


Xen-SVGA: computing support

Native stack

Hypervisor

Vendor Library

GPU

VMApplication


Nouveau

OpenCL Code

TGSI

Native ISA

TGSI LLVM Compiler

Nouveau

TGSI

Xen-SVGA Device


vISA - Boon or bane?

PTX, a low-level parallel thread execution virtual machine and instruction set architecture (ISA).


Trillium: Eliminates vISA

Native stack

Hypervisor

Vendor Library

Nouveau

GPU

VMApplication


Nouveau

OpenCL Code

Native ISA

Trillium Device


So are we done?


Checkpoint

✓ Virtual ISA is unnecessary

✓ Para-virtual API remoting system performs better




Trillium ✔️ ✔️ ✔️ ❌ 2.4x





Para-virtualization ✔️ ✔️ ✔️ ❌ 1-10x

Trillium ✔️ ✔️ ✔️ ❌ 2.4x

✕ Interposing too low in the stack

✕ API-specific





Para-virtualization ✔️ ✔️ ✔️ ❌ 1-10x

Trillium ✔️ ✔️ ✔️ ❌ 2.4x

AvA ✔️ ✔️ ✔️ ✔️ <1.5x

✕ Interposing too low in the stack

✕ API-specific


Para-virtual

API Stack

Native stack

Automatic virtualization of acceleratorsC

L.h

+ A

nn

ota

tio

ns

Hypervisor

Generated API Server

Vendor Library

Vendor Driver

Accelerators

VMApplication

AvA Guest Driver

AvA Virtual Device

H. Yu, A. M. Peters, A. Akshintala, C. J. Rossbach, Automatic Virtualization of Accelerators, HotOS’19

Generated User-mode Library


Preliminary development effort

Type APIs LoC Time Difficulty

GPUvm Full-virt 1 20 000 Years ★★★★

SVGA2 Para-virt 2 MANY! Years ★★★★

AvA

Automatic

Para-virtual

API Remoting

9 OpenCL: 835 Days ★


Conclusion

Lessons

● Virtual ISA is unnecessary○ Decouple device virtualization from GPU ISA virtualization

● Para-virtual API remoting can lead to better performance and properties

New para-virtual API remoting system: AvA

● “No IR” enables interposition at user-mode APIs

● Compatibility is compensated

trillium: the code is the irhyu/talk/virt19-07-18-slides.pdf · svga2: tgsi as ir and visa native...

Documents