to gpu synchronize or not gpu synchronize? wu-chun feng and shucai xiao department of computer...
TRANSCRIPT
To GPU Synchronize or Not GPU
Synchronize?Wu-chun Feng and Shucai
Xiao
Department of Computer Science, Department of Electrical and Computer Engineering, Virginia
Tech
ISCAS 2010
Outline
• Introduction• Preliminaries• Related Works• Proposed GPU-based
Synchronization• Problems, Experiments, and
Analysis• Conclusions
Introduction
• Multi(many)-core era has come.• General purpose GPU (GPGPU)
allows massively parallel computation with low cost.
• GPUs typically map well only to data-parallel or task-parallel applications – Due to the lack of support for
communication between streaming multiprocessors (SMs).
Introduction (cont.)
• Communication could be done via global memory.– Need barrier synchronization.
• CPU barrier synchronization– By (inefficiently) implementing the
barrier synchronization via the host CPU.
– Slow.
Introduction (cont.)
• GPU barrier synchronization– Improve performance by 10~40%.– Theoretically run the risk that
barrier may release earlier.
• CUDA 2.2 support new function _threadfence() to solve this problem.– The correctness could be guarantee.
Introduction (cont.)
• Unfortunately, _threadfence() incurs so much overhead in the proposed GPU barrier synchronization.– That is, CPU barrier synchronization
performs as well as or better than the GPU barrier synchronizations in many cases.
• “Whether to GPU synchronize or not GPU synchronize?”
Preliminaries: CUDA
• Compute Unified Device Architecture• Developed by nVIDIA.• The CPU code does the sequential part.• Highly parallelized part usually implement
in the GPU code, called kernel.• Calling GPU function in CPU code is called
kernel launch.• In a kernel, threads are grouped as a grid of
thread blocks, and each thread block contains a number of threads.– Multiple blocks can be executed on the same SM,
but one block cannot be executed across different SMs.
Preliminaries: GPU architecture
Preliminaries: synchronization
• Synchronization in parallel programming.– Making sure that each thread get the right data
for computation.
• CUDA provides a data communication mechanism for threads within a single block via the barrier function syncthreads().– Intra-SM communication.
• However, there is no explicit software or hardware support for data communication of threads across different blocks.– Inter-SM communication.
Related Works
• When multiple GPU thread blocks are scheduled to execute on a single SM simultaneously, deadlock might occurs.– In multi-core environment, a process can
yield its execution to other processes, but CUDA blocks do not.
• [17] assign only one block per SM to address this problem.
Related Works (cont.)
• When a barrier synchronization is needed across different blocks, programmers traditionally use a kernel launch as a way to implicitly barrier synchronize [4], [7].
• [14] propose a protocol for data communication across multiple GPUs.– Data needs to be transferred to the host
memory first and then copied back to the device memory, and hence poor performance in different SMs on a single GPU.
Proposed GPU-based Synchronization
• GPU synchronization– Lock-based synchronization.
• Single mutex variable for all thread blocks.• Once a block finishes its computation on
an SM, it atomically increments the mutex variable.
– Lock-free synchronization.• One distinct variable to control each block,
thus eliminating the need for different blocks to contend for the single mutex variable.
• The need for atomic addition is removed.
Experiments
• Environment– GeForce GTX 280: 30SMs, 8 cores each,
running at 1.3GHz (shader clock).– CUDA 2.2 SDK.– Details are omitted.
• Two experiments.– Dynamic programming (DP) for genomic
sequence alignment (specifically the Smith-Waterman algorithm).
– Bitonic sort (BS).
Performance comparisons
Problems, Experiments, and Analysis
• In order to eliminate the infinitesimal risk that barrier may release earlier when the proposed synchronization run, _threadfence() is used and hence incurred the overhead.
• The same experiments with modified barrier using _threadfence().
Performance comparisons
Problems, Experiments, and Analysis (cont.)
• Analyze GPU lock-based synchronization as an example, whose operation set is a superset of that are used in the other one.
• Synchronization overhead– ta is the overhead of atomic add– tc is the mutex variable checking time– ts is the time consumed by syncthreads()– tf is the threadfence() execution time.
Problems, Experiments, and Analysis (cont.)
• Unfortunately, the execution times for these component operations cannot be measured directly on the GPU.
• An indirect approach.– A kernel’s execution time can be
expressed as
– Measuring the kernel execution time both with and without a specific operation and taking the time difference as the overhead of that operation.
Execution time profiling
• Use micro benchmark to test.– Calculates the average of two floats over
10,000 iterations.• CPU synchronization
– Each kernel calculates the average once, and the kernel is launched 10,000 times.
• GPU synchronization– The kernel is launched only once, and
there is a 10,000-iteration for loop used in the kernel with the GPU barrier function called in each loop.
Results
• For 10,000 times of execution, ts = 0.541, ta = 2.300 × n, tc = 5.564, and tf = 0.333 × n + 7.267, where n is the number of blocks in the kernel, and the units are in milliseconds.
t5t4
t3t1t2
ts=t3-t1
ta=t2
tc=t4-t3-t2
tf=t5-t4
Conclusions
• Demonstrate the efficiency of inter-SM communication using GPU-based barrier synchronization.
• To eliminate the risk of asynchronous, threadfence() is used though it incurs high overhead.
• Grudgingly conclude that one should GPU synchronize (with or without threadfence()) on the current generation, but more definitive ’yes’ for the next generation GPUs.