computer architecture parallel processing ola flygt växjö university ola.flygt@msi.vxu.se +46 470...
Post on 25-Dec-2015
213 Views
Preview:
TRANSCRIPT
Computer Architecture
Parallel ProcessingOla Flygt
Växjö Universityhttp://w3.msi.vxu.se/users/ofl/
Ola.Flygt@msi.vxu.se+46 470 70 86 49
Outline
Basic conceptsTypes and levels of parallelismClassification of parallel
architectureBasic parallel techniquesRelationships between languages
and parallel architecture
CH03
Basic concepts
The concept of programordered set of instructions
(programmer’s view)executable file (operating system’s
view)
The concept of process
OS view, process relates to executionProcess creation
setting up the process descriptionallocating an address spaceloading the program into the allocated address
space, andpassing the process description to the scheduler
process statesready to runrunningwait
Process spawning (independent processes)
A
CB
D E
The concept of thread
smaller chunks of code (lightweight)threads are created within and
belong to processfor parallel thread processing,
scheduling is performed on a per-thread basis
finer-grain, less overhead on switching from thread to thread
Single-thread process or multi-thread (dependent)Thread tree
Process
Threads
Three basic methods for creating and terminating
threads1. unsynchronized creation and
unsynchronized terminationcalling library functions: CREATE_THREAD,
START_THREAD
2. unsynchronized creation and synchronized terminationFORK and JOIN
3. synchronized creation and synchronized terminationCOBEGIN and COEND
Processes and threads in languages
Black box view: T: thread
T2 T0 T1 T1 T2 T0 . . . Tn
FORK
JOIN
FORK
JOIN
COBEGIN
COEND
. . .
The concepts of concurrent execution (N-
client 1-server)
Non pre-emptive Pre-emptive
Time-shared Prioritized
PrioritySever SeverSever
Client ClientClient
Parallel execution
N-client N-server modelSynchronous or Asynchronous
SeverClient
Concurrent and Parallel Programming Languages
Classification of programming languages
Types and levels of parallelism
Available and utilized parallelismavailable: in program or in the problem
solutionsutilized: during execution
Types of available parallelismfunctional
arises from the logic of a problem solution
dataarises from data structures
Available levels Utilized levels
User (program) level User level
Procedure level Process level
Loop level Thread level
Instruction level Instruction level
2
1
1.Exploited by architectures2.Exploited by means of operating systems
Available and utilized levels of functional
parallelism
Utilization of functional parallelism
Available parallelism can be utilized by architecture,
instruction-level parallel architectures compilers
parallel optimizing compileroperating system
multitasking
Concurrent execution models
User level --- Multiprogramming, time sharing
Process level --- MultitaskingThread level --- Multi-threading
Utilization of data parallelism
By using data-parallel architecture
Convert into functional parallelism (i.e. loop constructs)
Classification of parallel architectures
Flynn’s classificationSISDSIMDMISD (Multiple Instruction Single
Date)MIMD
Part III Part II Part IV
Data-parallel architectures Function-parallel architectures
Instruction-level
PAs
Thread-level
PAs
Process-levelPAs
ILPS MIMDs
Vectorarchitecture
Associative
architecturearchitectureand neural
SIMDs Systolic Pipelinedprocessors processorsVLIWs Superscalar Distributed
memory
(multi-computer)
Sharedmemory(multi-MIMD
DPs
Parallel architectures
processors)
Basic parallel technique
Pipelining (time)a number of functional units are employed in
sequence to perform a single computationa number of steps for each computation
Replication (space)a number of functional units perform multiply computation simultaneously
more processorsmore memorymore I/O
more computers
Relation between basic techniques and
architectures
Relationships between languages and parallel
architecture SPMD (Single Procedure Multiple data)
Loop: split into N threads that works on different invocations of the same loop
threads can execute the same code at different speeds synchronize the parallel threads at the end of the loop
barrier synchronization use MIMD
Data-parallel languages DAP Fortran
C = A + B (A, B and C are arrays) use SIMD
Other types like Vector machines or Systolic arrays does not require any specific language support.
Synchronization mechanismsTest_and_set
Semaphore
ConditionalCritical region
Monitor
Rendezvous
Send/receivemessage
Broadcast Shiftnet_receive
(processor form)
Remote procedure calls
Net_send
Using Semaphores to handle mutual execution
P1
Criticalregion
V(S)
S Semaphore
Shareddata
structureV(S)
P(S)
P2
Criticalregion
P(S)
BusyWait
Parallel distributed computing
Ada used rendezvous concepts which combines
feature of RPC and monitorsPVM (Parallel Virtual Machine)
to support workstation clustersMPI (Message-Passing Interface)
programming interface for parallel computers
Summary of forms of
parallelism
top related