1 programming multicore processors aamir shafi high performance computing lab
TRANSCRIPT
1
Programming Multicore Processors
Aamir Shafi
High Performance Computing Lab
http://hpc.seecs.nust.edu.pk
2
Serial Computation • Traditionally, software has been written for
serial computation:• To be run on a single computer having a single
Central Processing Unit (CPU)• A problem is broken into a discrete series of
instructions
Parallel Computation• Parallel computing is the simultaneous use of
multiple compute resources to solve a computational problem:• Also known as High Performance Computing (HPC)
• The prime focus of HPC is performance—the ability to solve biggest possible problems in the least possible time
3
Traditional Usage of Parallel Computing--Scientific Computing • Traditionally parallel computing is used to solve
challenging scientific problems by doing simulations: • For this reason, it is also called “Scientific Computing”:
• Computational science
4
Emergence of Multi-core Processors
• In the last decade, performance of processors is not enhanced by increasing clock speed:• Increasing clock speed directly increases power
consumption• Power is dissipated as heat, not practical to cool down
processors• Intel canceled a project to produce 4 GHz processor!
• This led to the emergence of multi-core processors:• Performance is increased by increasing processing cores
that run on lower clock speed:• Implies better power usage
5Disruptive Technology!
6
Moore’s Law is Alive and Well
7
Power Wall
Why Multi-core Processors Consume Lesser Power
• Dynamic power is proportional to V2fC• Increasing frequency (f) also increases supply voltage
(V): more than linear effect • Increasing cores increases capacitance (C) but has only
a linear effect
8
9
Software in the Multi-core Era• The challenge has been thrown to the software
industry:• Parallelism is perhaps the answer
• The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software:• http://www.gotw.ca/publications/concurrency-ddj.htm
• Some excerpts:• The biggest sea change in software development since the
OO revolution is knocking at the door, and its name is Concurrency
• This essentially means every software programmer will be a parallel programmer:• The main motivation behind conducting this “Programming
Multicore Processors” workshop
10
About the “Programming Multicore Processors” Workshop
12
Course Contents …
A little background on Parallel Computing Approaches
Parallel Hardware• Three main classifications:
• Shared Memory Multi-processors: • Symmetric Multi-Processors (SMP)• Multi-core Processors
• Distributed Memory Multi-processors• Massively Parallel Processors (MPP)• Clusters:
• Commodity and custom clusters
• Hybrid Multi-processors: • Mixture of shared and distributed memory technologies
13
14
First Type: Shared Memory Multi-processors
• All processors have access to shared memory: • Notion of “Global Address Space”
15
Symmetric Multi-Processors (SMP)• A SMP is a parallel processing system with a shared-
everything approach:• The term signifies that each processor shares the main
memory and possibly the cache
• Typically a SMP can have 2 to 256 processors• Also called Uniform Memory Access (UMA)• Examples include AMD Athlon, AMD Opteron 200 and
2000 series, Intel XEON etc
16
Multi-core Processors
17
Second Type: Distributed Memory
• Each processor has its own local memory• Processors communicate with each other by
message passing on an interconnect
18
Cluster Computers• A group of PCs or workstations or Macs (called
nodes) connected to each other via a fast (and private) interconnect: • Each node is an independent computer
• Each cluster has one head-node and multiple compute-nodes:• Users logon to head-node and start parallel jobs on
compute-nodes
• Two popular cluster classifications: • Beowulf Clusters (http://www.beowulf.org)• Rocks Clusters (http://www.rocksclusters.org)
19
Proc 6
Proc 0
Proc 1
Proc 3
Proc 2
Proc 4
Proc 5
Proc 7
Cluster Computer
20
Third Type: Hybrid• Modern clusters have hybrid architecture:
• Distributed memory for inter-node (between nodes) communications
• Shared memory for intra-node (within a node) communications
21
SMP and Multi-core clusters• Most modern commodity clusters have SMP and/or
multi-core nodes: • Processors not only communicate via interconnect, but
shared memory programming is also required
• This trend is likely to continue: • Even a new name “constellations” has been proposed
Classification of Parallel Computers
22
Parallel HardwareParallel Hardware
Shared Memory HardwareShared Memory Hardware Distributed Memory HardwareDistributed Memory Hardware
SMPsSMPs Multicore ProcessorsMulticore Processors ClustersClustersMPPsMPPs
In this workshop, we will learn how to program shared memory parallel hardware …
Parallel Hardware Shared Memory Hardware *
Writing Parallel Software• There are mainly two approaches for writing
parallel software• The first approach is to use libraries (packages)
written in already existing languages:• Economical
• The second and more radical approach is to provide new languages: • Parallel Computing has a history of novel parallel
languages• These languages provide high level parallelism
constructs:
23
24
Shared Memory Languages and Libraries
• Designed to support parallel programming on shared memory platforms:• OpenMP:
• Consists of a set of compiler directives, library routines, and environment variables
• The runtime uses fork-join model of parallel execution
• Cilk++:• A design goal was to support asynchronous parallelism
• A set of keywords:• cilk_for, cilk_spawn, cilk_sync …
• POSIX Threads (PThreads)• Threads Building Blocks (TBB)
Distributed Memory Languages and Libraries
• Libraries: • Message Passing Interface (MPI)—defacto standard• PVM
• Languages:• High Performance Fortran (HPF): • Fortran M:• HPJava:
25
26
Our Focus• Shared Memory and Multi-core Processors
Machines:• Using POSIX Threads• Using OpenMP• Using Cilk++ (covered briefly)
• Disruptive Technology: • Using Graphics Processing Units (GPUs) by NVIDIA for
general-purpose computing
Day One
27
Timings Topic Presenter10:00 to 10:30 Introduction to multicore
computingAamir Shafi
10:30 to 11:30 Background discussion—review of processes, threads, and architecture. Speedup analysis
Akbar Mehdi
11:30 to 11:45 Break11:45 to 12:55P Introduction to POSIX
ThreadsAkbar Mehdi
12:55P to 1:25P Prayers break1:25P to 2:30P Practical Session—Run
hello world PThreads program, introduce Linux, top, Solaris. Also introduce the first coding assignment
Akbar Mehdi
Day Two
28
Timings Topic Presenter
10:00 to 11:00 POSIX Threads continued…
Akbar Mehdi
11:00 to 12:55P Introduction to OpenMP
Mohsan Jameel
12:55P to 1:25P Prayer Break
1:25P to 2:30P OpenMP continued… + Lab session
Mohsan Jameel
Day Three
29
Timings Topic Presenter10:00 to 12:00 Parallelizing the Image
Processing Application using PThreads and OpenMP—Practical Session
Akbar Mehdi and Mohsan Jameel
12:00 to 12:55P Introduction to Intel Cilk++ Aamir Shafi
12:55 to 1:25P Prayer Break
1:25P to 2:30P Introduction to NVIDIA CUDA
Akbar Mehdi
2:30P to 2:35P Concluding Remarks Aamir Shafi
Learning Objectives• To become aware of the multicore revolution and
its impact on the computer software industry• To program multicore processors using POSIX
Threads• To program multicore processors using OpenMP
and Cilk++• To program Graphics Processing Units (GPUs) for
general purpose computation (using NVIDIA CUDA API)
30
You may download the tentative agenda from http://hpc.seecs.nust.edu.pk/~aamir/res/mc_agenda.pdf
Next Session
• Review of important and relevant Operating Systems and Computer Architecture concepts by Akbar Mehdi ….
31