A Source-to-Source OpenACC compiler for CUDA
Akihiro Tabuchi†1
Masahiro Nakao†2
Mitsuhisa Sato†1
†1. Graduate School of Systems and Information Engineering, University of Tsukuba
†2. Center for Computational Sciences, University of Tsukuba
Outline
• Background• OpenACC• Compiler Implementation• Performance Evaluation• Conclusion & Future Work
Background
• Accelerator programming model– CUDA (for NVIDIA GPU)– OpenCL (for various accelerators)
• Accelerator programming is complex– memory management, kernel function, …– low productivity & low portability
• OpenACC is proposed to solve these problems
OpenACC
• The directive-based programming model for accelerators– support C, C++ and Fortran
• Offloading model– offload a part of code to an accelerator
• High productivity– only adding directives
• High portability– run on any accelerators as long as the compiler supports it
Example of OpenACC
int main(){int i;int a[N], b[N], c[N];/* initialize array ‘a’ and ‘b’ */
#pragma acc parallel loop copyin(a,b) copyout(c)
for(i = 0; i < N; i++){c[i] = a[i] + b[i];
}}
This directive specifies data transfers and loop offloading and parallelization
Purpose of Research
• Designing and implementing an open source OpenACC compiler– Target language : C– Target accelerator : NVIDIA GPU– Source-to-source approach• C + OpenACC → C + CUDA API• This approach enables to leave detailed machine-
specific code optimization to the mature CUDA compiler by NVIDIA
– The result of compilation is a executable file
Related Work
• Commercial compiler– PGI Accelerator compiler– CAPS HMPP– Cray compiler
• Open source compiler– accULL
• developed at University of La Laguna in Spain• Source-to-source translation• Backend is CUDA and OpenCL• Output is codes and a Makefile
OpenACC directives
• parallel• kernels• loop• data• host_data• update• wait
• cache• declare• parallel loop• kernels loop
(OpenACC specification 1.0)
data constructint a[4];#pragma acc data copy(a){
/* some codes using ‘a’ */
}
host memory device memory
computationon device
Data management on AcceleratorIf an array is specified in “copy” clause …1. Device memory allocation2. Data transfer from host to device3. Data transfer from device to host4. Device memory release
at the beginning of region
at the end of region
Translation of data constructint a[4];#pragma acc data copy(a){
/* some codes using ‘a’ */ }
int a[4];{
void *_ACC_DEVICE_ADDR_a,*_ACC_HOST_DESC_a;_ACC_gpu_init_data(&_ACC_HOST_DESC_a, &_ACC_DEVICE_ADDR_a, a,
4*sizeof(int)); _ACC_gpu_copy_data(_ACC_HOST_DESC_a, 400); { /* some codes using ‘a’ */}_ACC_gpu_copy_data(_ACC_HOST_DESC_a, 401);
_ACC_gpu_finalize_data(_ACC_HOST_DESC_b);}
allocate ‘a’ on GPU
copy ‘a’ to GPU from host
free ‘a’ on GPU
copy ‘a’ to host from GPU
• host address• device address• size• ….
• Codes in parallel region are executed on device• Three levels of parallelism
– gang– worker– vector
parallel construct#pragma acc parallel num_gangs(1) vector_length(128){
/* codes in parallel region */ }
OpenACC CUDA
gang thread block
worker (warp)
vector thread
• The number of gang or worker or vector length can be specified by clauses
Translation of parallel construct#pragma acc parallel num_gangs(1) vector_length(128){
/* codes in parallel region */ }
__global__ static void _ACC_GPU_FUNC_0_DEVICE( ... ){
/* codes in parallel region */}extern "C” void _ACC_GPU_FUNC_0( … ){
dim3 _ACC_block(1, 1, 1), _ACC_thread(128, 1, 1);
_ACC_GPU_FUNC_0_DEVICE<<<_ACC_block,_ACC_thread>>>( ... );_ACC_GPU_M_BARRIER_KERNEL();
}
GPU kernel function
kernellaunchfunction
loop construct/* inside parallel region */ #pragma acc loop vectorfor(i = 0; i < 256; i++){
a[i]++;}
• Loop construct describes parallelism of loop– Distribute loop iteration among gang, worker or vector– Two or more parallelisms can be specified for a loop
• Loops with no loop directive in parallel region is basically executed serially.
8 9 10
11
12
13
14
15
Translation of loop construct (1/3)
/* inner parallel region */ #pragma acc loop vectorfor(i = 0; i < N; i++){
a[i]++;}
1. A virtual index which is the same length as loop iteration is prepared
2. The virtual index is divided and distributed among blocks and/or threads
0 1 2 3 4 5 6 7
8 9 10
11
12
13
14
15
0 1 2 3 4 5 6 7
3. Each thread calculates the value of loop variable from the virtual index and executes loop body
Translation of loop construct (2/3)/* inner parallel region */ #pragma acc loop vectorfor(i = 0; i < N; i++){
a[i]++;}
/* inner gpu kernel code */ int i, _ACC_idx; int _ACC_init, _ACC_cond, _ACC_step;_ACC_gpu_init_thread_x_iter(&_ACC_init, &_ACC_cond, &_ACC_step, 0, N, 1); for(_ACC_idx = _ACC_init; _ACC_idx < _ACC_cond; _ACC_idx += _ACC_step){
_ACC_gpu_calc_idx(_ACC_idx, &i, 0, N, 1); a[i]++; } calculate ‘i’ from virtual index
virtual index: _ACC_idxvirtual index range : _ACC_init, cond, step
calculate the range of virtual index
virtual indexrange variables
loop body
Translation of loop construct(3/3)
• Our compiler supports 2D blocking for nested loops– Nested loops are distributed among the 2D blocks
in the 2D grid in CUDA (default block size is 16x16)– But it’s not allowed in OpenACC 2.0 and “tile”
clause is provided instead#pragma acc loop gang vectorfor( i = 0; i < N; i++)#pragma acc loop gang vector for(j = 0; j < N; j++) /* … */
distribute
2D Grid
2D Block
Compiler Implementation
• Our compiler translates C with OpenACC directives to C with CUDA API– read C code with directives and output translated code– using Omni compiler infrastructure
• Omni compiler infrastructure– a set of programs for a source-to-source compiler with
code analysis and transformation – supports C and Fortran95
Flow of Compilation
Omni compiler infrastructure
sample.gpu.o
acc runtime
sample_tmp.o
Omni Frontend
OpenACCtranslator
C compiler
nvcc
a.out
sample.c sample.xml
sample_tmp.c
sample.cu
XcodeML
C with ACC API
CUDA
C with OpenACCdirectives
Performance Evaluation
• Benchmark– Matrix multiplication– N-body problem– NAS Parallel Benchmarks – CG
• Evaluation environment– 1 node of Cray XK6m-200• CPU : AMD Opteron Processor 6272 (2.1GHz)• GPU : NVIDIA X2090 (MatMul, N-body)
: NVIDIA K20 (NPB CG)
Performance Comparison
• Cray compiler• Our compiler• Hand written CUDA– The code is written in CUDA and compiled by NVCC– The code doesn’t use shared memory of GPU
• Our compiler (2D-blocking)– The code uses 2D blocking and is compiled by our
compiler– This is applied to only matrix multiplication
Matrix multiplication
1K 2K 4K 8K0
1
2
3
4
5
6
Cray compilerHand-written CUDAOur compilerOur compiler, 2D-Block-ing
Matrix size
Rela
tive
perf
orm
ance
aga
inst
CP
U
4.6x
5.5x
1.5x
1.4x
The performance of our compiler using 2D-blocking and hand-written CUDA are slightly lower
Matrix multiplication
• Our compiler achieves better performance than that of Cray compiler– The PTX code directly generated by Cray compiler has more operations
in the innermost loop– Our compiler outputs CUDA code, and NVCC generates more
optimized PTX code
• 2D-blocking is lower performance– default 2D block size (16x16) is not adequate to this program– the best block size was 512x2– Hand-written CUDA code also uses 16x16 block
N-body
1K 2K 4K 8K 16K 32K0
5
10
15
20
25
30
35
Cray compilerHand-written CUDAOur compiler
The number of particles
Rela
tive
perf
orm
ance
aga
inst
CP
U
5.4x
31x
0.95x
1.2x
At the small problem size, the performance of our compiler is lower than that of Cray compiler
N-body
• At small problem size, the performance became worse– Decline in the utilization of Streaming Multiprocessors(SMs)
• A kernel is executed by SMs per thread block– If the number of blocks is smaller than that of SMs, the
performance of the kernel becomes low.• Default block size
– Cray compiler : 128 threads / block– Our compiler : 256 threads / block
NPB CG
S(1400)
W(7000)
A(14000)
B(75000)
C(150000)0
2
4
6
8
10
12
Cray compilerOur compiler
Class(Matrix size)
Rela
tive
perf
oman
ce a
gain
st
CPU
the performance is lower than that of CPU and Cray compiler
0.66x
9.7x
0.74x
2.1x
NPB CG
• At class S, the performance of GPU is lower than that of CPU– Overheads are larger compared with kernel execution time
• launching kernel functions• synchronization with device• data allocation / release / transfer
• The overhead is larger than that of Cray compiler
– large overhead of reduction• The performance of GPU kernels are better than that of Cray compiler
Conclusion
• We implemented a source-to-source OpenACC compiler for CUDA– C with OpenACC directives → C with CUDA API– Using Omni compiler infrastructure
• In most case, the performance of GPU code by our compiler is higher than that of CPU single core– Speedup of up to 31 times at N-body
• Our compiler makes use of CUDA backend successfully by source-to-source approach– the performance is often better than that of Cray compiler
• There is room for performance improvement– using suitable grid size and block size– reducing overhead of synchronization and reduction