nvidia’s fermi: the first complete gpu computing architecture a white paper by peter n. glaskowsky...
TRANSCRIPT
NVIDIA’S FERMI:THE FIRST COMPLETE GPU COMPUTING ARCHITECTUREA WHITE PAPER BY PETER N. GLASKOWSKY
Presented by: Presented by: Ahmad HammadCourse: Course: CSE 661 - Fall 2011
Outline
Introduction What is GPU Computing? Fermi
The Programming Model The Streaming Multiprocessor The Cache and Memory Hierarchy
Conclusion
Introduction
Traditional microprocessor technology see diminishing returns. Improvement in clock speeds and
architectural sophistication is slowing Focus has shifted to multicore designs.
These too are reaching practical limits for personal computing.
Introduction (2)
CPUs are optimized for applications where work done by limited number of threads Threads exhibit high data locality Mix of different operations High percentage of conditional branches.
CPUs are inefficient for high-performance computing applications Integer and floating-point execution units is
small Most of the CPU space, complexity and heat it
generates, devoted to: Caches, instruction decoders, branch predictors, and
other features to enhance single-threaded performance.
Introduction (3)
GPU design aims applications with multiple threads dominated by long sequences of computational instructions.
GPUs are much better at thread handling, data caching, virtual memory management, flow control, and other CPU-like features.
CPUs will never go away, but GPUs deliver more cost-effective and
energy-efficient performance.
Introduction (4)
The key GPU design goal is to maximize floating-point throughput.
Most of the circuitry within each core is dedicated to computation, rather than speculative features
power consumed by GPUs goes into the application’s actual algorithmic work.
What is GPU Computing?
Use of a graphics processing unit to do general purpose scientific and engineering computing.
GPU computing not a replacement for CPU computing. Each approach has advantages for certain kinds of
software. CPU and GPU work together in a heterogeneous
co-processing computing model. The sequential part of the application runs on the
CPU the computationally-intensive part is accelerated by
the GPU.
What is GPU Computing?
From the user’s perspective, the application just runs faster because of using the GPU to boost performance.
Fermi
Code name for NVIDIA’s next-generation CUDA architecture
consists of 16 streaming multiprocessors (SMs)
each consisting of 32 cores each can execute one floating-point or integer
instruction per clock.
The SMs are supported by a second-level cache Host interface GigaThread scheduler Multiple DRAM interfaces.
Fermi
Code name for NVIDIA’s next-generation CUDA architecture
The Programming Model
Complexity of the Fermi architecture is managed by multi-level programming model allows software developers to focus on
algorithm design No need to know details about mapping
algorithm to the hardware improve productivity
The Programming Model Kernels In NVIDIA’s CUDA software platform the
computational elements of algorithms called kernels Kernels can be written in the C language extended with additional keywords to
express parallelism directly Once compiled, kernels consist of many
threads that execute the same program in parallel
The Programming Model Thread Blocks
The Programming Model Warps Thread blocks are divided into warps of 32
threads. The warp is the fundamental unit of
dispatch within a single SM. Two warps from different thread blocks
can be issued and executed concurrently increase hardware utilization and energy
efficiency. Thread blocks are grouped into grids
each executes a unique kernel
The Programming Model
At any one time, the entire Fermi device is dedicated to a single application. an application may include multiple kernels. Fermi supports simultaneous execution of
multiple kernels from the same application Each kernel distributed to one or more SMs
This capability increases the utilization of the device
The Programming Model GigaThread Switching from one application to
another needs 25µsec Short enough to maintain high utilization
even when running multiple applications This switching is managed by
GigaThread (hardware thread scheduler) Manages 1,536 simultaneously active
threads for each streaming multiprocessor across 16 kernels.
The Programming Model Languages Fermi support
C-language FORTRAN (with independent solutions from
The Portland Group and NOAA) Java, Matlab, and Python
Fermi brings new instruction level support for C++ previously unsupported on GPUs will make GPU computing more widely
available than ever.
Supported software platforms Supported software platforms
NVIDIA’s own CUDA development environment
The OpenCL standard managed by the Khronos Group
Microsoft’s Direct Compute API.
The Streaming Multiprocessor Comprise 32 cores each:
can perform floating-point and integer operations
16 load-store units for memory operations
four special-function units 64K of local SRAM split between cache
and local memory.
The Streaming Multiprocessor core Floating-point operations follow the IEEE 754-
2008 floating-point standard. Each core can perform
one single-precision fused multiply-add operation in each clock period
one double-precision fused multiply-add FMA in two clock periods. no rounding off in the intermediate result
Fermi performs more than 8× as many double-precision operations per clock than previous GPU generations
The Streaming Multiprocessor core FMA support increases the accuracy and
performance of other mathematical operations division and square root extended-precision arithmetic interval arithmetic Linear algebra.
The integer ALU supports the usual mathematical and logical operations including multiplication, on both 32-bit and
64-bit values.
The Streaming Multiprocessor Memory operations Handled by a set of 16 load-store units in
each SM. load/store instructions refer to memory in
terms of two-dimensional arrays providing addresses in terms of x and y
values. Data can be converted from one format
to another as it passes between DRAM and the core registers at the full rate. examples of optimizations unique to GPUs
The Streaming Multiprocessor four Special Function Units handle special operations such as sin,
cos and exp Four of these operations can be issued
per cycle in each SM.
The Streaming Multiprocessor execution blocks Within Fermi SM has four execution blocks
Cores are divided into two execution blocks:16 cores each.
One block offer 16 load-store units One block of the four SFUs,
In each cycle, 32 instructions can be dispatched from one or two warps to these blocks. Two cycles to execute the 32 instructions on the cores or
load/store units. 32 special-function instructions can issued in single
cycle takes eight cycles to complete on the four SFUs. (32/4 =
8)
This figure shows how instructions are issued to the execution blocks.
The Cache and Memory Hierarchy L1 Fermi architecture provides local
memory in each SM, can be split Shared memory First-level (L1) cache for global memory
references. The local memory is 64K in size
Split 16K/48K or 48K/16K between L1 cache and shared memory. Depends on How much shared memory is needed, how predictable the kernel’s accesses to global
memory are likely to be.
The Cache and Memory Hierarchy L2 Fermi come with an L2 cache
768KB in size for a 512-core chip. Covers GPU local DRAM as well as system
memory. The L2 cache subsystem implements:
Set of memory read-modify-write atomic operations Managing access to data shared across thread blocks
or kernels. atomic operations are 5× to 20× faster than on
previous GPUs using conventional synchronization methods.
The Cache and Memory Hierarchy DRAM The final stage of the local memory
hierarchy. Fermi provides six 64-bit DRAM channels
that support SDDR3 and GDDR5 DRAMs. Up to 6GB of GDDR5 DRAM can be
connected to the chip.
Error Correcting Code ECC
Fermi is the first GPU to provide ECC protects DRAM, register files, shared memories, L1 and L2
caches. The level of protection is known as SECDED:
single (bit) error correction, double error detection. Instead of each 64-bit memory channel
carrying eight extra bits for ECC information NVIDIA has a secrete solution for packing the ECC
bits into reserved lines of memory.
Conclusion
CPUs is the best for dynamic workloads with short sequences of computational operations and unpredictable control flow.
workloads dominated by computational work performed within a simpler control flow need GPU