chapter 1 pc
Post on 12-Aug-2015
197 Views
Preview:
TRANSCRIPT
Chapter 1Chapter 1
Introduction to Parallel Introduction to Parallel Computing Computing
Dr. Muhammad Hanif Durad
Department of Computer and Information Sciences
Pakistan Institute Engineering and Applied Sciences
hanif@pieas.edu.pk
Dr. Hanif Durad 3
Lecture Outline
Motivating Parallelism Scope of Parallel Computing Applications Organization and Contents of the Course Texts and References
Parallel computing
Parallel Processing is a term used to denote a large class of techniques that are used to provide simultaneous data processing tasks for the purpose of Save time and/or money Solve larger problems
Parallel computing is the simultaneous use of multiple compute resources to solve a computational problem 4
parallelcomputing-120214042541-phpapp02-130408023815-phpapp01.ppt
The Universe is Parallel
5
Galaxy formation Planetary movement Weather and ocean patterns Tectonic plate drift Rush hour traffic Automobile assembly line Building a jet Ordering a hamburger
at the drive through.
Areas of Parallel Computing
Physics – applied, nuclear, particle, condensed matter, high pressure, fusion, photonics
Bioscience, Biotechnology, Genetics Chemistry, Molecular Sciences Geology, Seismology Mechanical Engineering - from prosthetics to spacecraft Electrical Engineering, Circuit Design, Microelectronics Computer Science, Mathematics
Dr. Hanif Durad 6
Why Use Parallel Computing? Save time and/or money: In theory, throwing more resources
at a task will shorten its time to completion, with potential cost savings. Parallel computers can be built from cheap, commodity components.
Solve larger problems: Many problems are so large and/or complex that it is impractical or impossible to solve them on a single computer, especially given limited computer memory.
Better response times: As the computing tasks are engaged by a group of processors, the tasks are completed in a smaller amount of time
7
Ways to Classify Parallel Computers
One of the more widely used classifications, in use since 1966, is called Flynn's Taxonomy
The 4 possible classifications according to Flynn’s are :
Single Instruction, Single Data (SISD) Single Instruction, Multiple Data (SIMD) Multiple Instruction, Single Data (MISD) Multiple Instruction, Multiple Data (MIMD)
Dr. Hanif Durad 9
Technology Trends: Moore’s Law
Dr. Hanif Durad 11
Slide from Maurice Herlihy
Clock speed
flattening sharply
Transistor count still
rising
6963_L1.ppt
Techology Trends: Power Issues
Dr. Hanif Durad 12
From www.electronics-cooling.com/.../jan00_a2f2.jpg 6963_L1.ppt
Power Perspective
Dr. Hanif Durad 13
Gig
aFlo
p/s
Meg
aWat
ts
0.001
0.01
0.1
1
10
100
1000
10000
100000
1000000
10000000
100000000
1000000000
1960 1970 1980 1990 2000 2010 2020
Performance (Gflops)
Power
Slide source: Bob Lucas
6963_L1.ppt
Increase speed processor
Greater no. of transistors Operation can be done in fewer clock cycles
Increased clock speed More operations per unit time
Example 8088/8086 : 5 Mhz, 29000 transistors E6700 Core 2 Duo: 2.66 GHz, 291 million
transistor
lecture1-100131132656-phpapp02.pptx
The Multi-Core Paradigm Shift
Key ideas: Movement away from increasingly complex
processor design and faster clocks Replicated functionality (i.e., parallel) is simpler to
design Resources more efficiently utilized Huge power management advantages
Dr. Hanif Durad 15
6963_L1.ppt
Multicore
A multi-core processor is one processor that contains two or more complete functional units. Such chips are now the focus of Intel and AMD. A multi-core chip is a form of SMP
lecture1-100131132656-phpapp02.pptx
parallelprogramming-130823023925-phpapp01.pptx
Concept of GPGPU (General-Purpose Computing on GPUs)
Idea: Potential for very high performance at low cost Architecture well suited for certain kinds of parallel applications (data
parallel) Demonstrations of 30-100X speedup over CPU
Early challenges: Architectures very customized to graphics problems (e.g., vertex and
fragment processors) Programmed using graphics-specific programming models or libraries
Recent trends: Some convergence between commodity and GPUs and their associated
parallel programming modelsDr. Hanif Durad 17
6963_L1.ppt
Parallel Programming Model Parallel programming models in common use:
Shared Memory (without threads) Threads Distributed Memory / Message Passing Data Parallel Hybrid Single Program Multiple Data (SPMD) Multiple Program Multiple Data (MPMD)
These models are NOT specific to a particular type of machine or memory architecture
Any of these models can be implemented on any underlying hardware18
parallelprogramming-130823023925-phpapp01.pptx, p-60/114
Classes of Parallel Computers
Cluster computing A cluster is a group of loosely coupled computers that
work together closely, so that in some respects they can be regarded as a single computer
Massive parallel processing A massively parallel processor (MPP) is a single
computer with many networked processors. MPPs have many of the same characteristics as
clusters, but MPPs have specialized interconnect networks Dr. Hanif Durad 19
parallelprogramming-130823023925-phpapp01.pptx, p-99/114
Classes of Parallel Computers
Grid computing Compared to clusters, grids tend to be more loosely
coupled, heterogeneous, and geographically dispersed
Dr. Hanif Durad 20
Cloud Computing A cloud provider has 100s of thousands of nodes (aka
servers). Cloud computing is massively-parallel computing with
multi-processors (i.e. many multi-core processors) In principle, your application may run on one, two, …
thousands of servers (i.e. processors) For your application to run on one, two, … thousands of
servers, your application code or data must be parallelized. i.e. Split up into independent or relatively independent parts.
Dr. Hanif Durad 21
02Distributed_Computing.pptx
Dr. Hanif Durad 22
Organization and Contents of this Course
1. Fundamentals: This part of the class covers basic parallel platforms, principles of algorithm design, group communication primitives, and analytical modeling techniques.
2. Parallel Programming: This part of the class deals with programming using message passing libraries and threads.
3. Parallel Algorithms: This part of the class covers basic algorithms for matrix computations, graphs, sorting, discrete optimization, and dynamic programming.
1+2=CIS-546 PC3=CIS-645 PA
Summary Introduction to Parallel Computing Technology Trends
Multicore GPGPU (General-Purpose Computing on GPUs) Parallel Programming Model
Classes of Parallel Computers Cluster computing Massive parallel processing (MPP) Grid computing Cloud Computing
Dr. Hanif Durad 23
Dr. Hanif Durad 24
References Grama A., Gupta A., Karypis G., and Kumar V., Introduction
to Parallel Computing, 2nd ed., Addison-Wesley, 2003. (Main Text Book)
Thomas Rauber, Gudula Rünger, Parallel Programming: For Multicore and Cluster Systems, Springer, 2010.
Peter Pacheco, An Introduction to Parallel Programming, Elsevier, 2011.
Michael J. Quinn , Parallel Programming in C with MPI and OpenMP, McGraw-Hill, 2004.
References Mostafa Abd-El-Barr and Hesham El-Rewini, Advanced
Computer Architecture and Parallel Processing, John Wiley and Sons, 2005.
Culler D. E., Singh J. P., and Gupta A., Parallel Computer Architecture: A Hardware/Software Approach, Morgan Kaufmann Publisher, 1999 .
Foster I., Designing and Building Parallel Programs, Addision-Wesley, 1995.
Dr. Hanif Durad 25
top related