slide02 parallel computers

8/6/2019 Slide02 Parallel Computers

http://slidepdf.com/reader/full/slide02-parallel-computers 1/44

Parallel ComputerArchitecture

The End of the Road

Advantages of

Multi rocessors• Able to create powerful computers by

simply connecting multiple processors

• More cost-effective than building a high-performance single processor

• Obtain fault-tolerance to carry on thetasks, albeit with degraded performance

4 Decades of Computing

Batch Era (1960s)

• IBM System/360 mainframedominated the corporate

computer centers (10 MBdisk, 1 MB magnetic corememory)

•Typical batch processing

machine

• No connection beyond thecomputer room

Time-Sharing Era (1970s)

• Advancing in ss-memory &ICs spawned the

minicomputer era

• Small, fast, and inexpensiveenough to be spreadthroughout the company at

the divisional level

• Still too expensive anddifficult to use to hand overto end-users

• Time-sharing computing

•Existing 2 kinds:

• centralized data processingmainframes

• time-sharing minicomputers

Desktop Era (1980s)

• PCs were introduced in1977

• Many players (Altairs, Tandy,

Commondore, Apple, IBM,and etc)

• Became pervasive and

change the face of computing

• Along came networkedcomputers (LAN & WAN)

Network Era (1990s)

• Advance network technologiesled to network computingparadigm

• Transition from a processor-centric view of computing to anetwork-centric view

•A number of commercialparallel computers withmultiple processors:

• Shared memory systems

• Distributed memory systems

Four Decades of ComputingFeature Batch Time-Sharing Desktop Network

Decade 1960s 1970s 1980s 1990s

Location Computer Room Terminal Room Desktop Mobile

Users Experts Specialists Individuals Groups

Data Alphanumeric Text, numbers Fonts, graphs Multimedia

Objective Calculate Access Present Communicate

Interface Punched card Kbd & CRT See & point Ask & tell

Operation Process Edit Layout Orchestrate

Connectivity None Peripheral cable LAN Internet

OwnersCorporate

computer centersDivisional IS

shopsDepartmental

end-usersEveryone

Current Trends

• The substitution of expensive and specialized parallelmachines by the more cost-effective clusters of workstations

• A cluster is a collection of stand-alone computersconnected using some interconnection network

• A pervasiveness of the Internet created interest innetwork computing and more recently in grid

computing

• Grids are geographically distributed platforms of computation - dependable, consistent, pervasive, andless expensive access to HPC facilities

Flynn’s Taxonomy of

Com uter Architecture• Based on the notion of a stream of

information

• instruction

•data

Memory

fetch execute(manipulate data as

programmed)

SingleInstruction

MultipleInstruction

SingleData

MultipleData

SIMD Architecture

Single Instruction,

Multi le Data SIMD

t i m e

P1 P2 Pn

MIMD Architecture

Multiple Instruction,

Multi le Data MIMD

t i m e

P1 P2 Pn

SIMD Architecture Model

• Consists of two parts:

• a front-end

computer

• a processor array

• each element in the processor array is identical toone another and performs operation on different datain sync

•front-end can access PE’s memory via the bus

SIMD Architecture Model

• lock-step

synchronization

• Processors either donothing or exactly thesame ops simultaneously

• In SIMD, parallelism is exploited by applyingsimultaneous operations across large sets of data

SIMD Configurations

Each PE has its own localmemory

PEs and memory modulescommunicate via the IN

MIMD Architecture

Interconnection Network

MM M M

Shared Memory MIMD Architecture

P P P P

MM M M

Message Passing MIMD Architecture

information exchangethrough central shared

memory

information exchangethrough network in

message passing systems

MIMD Architecture

MM M M

• using bus/cachearchitecture

• called SMP (symmetricmultiprocessor) since

• equal chance to read/

write memory

• equal access speed

MIMD Architecture

P P P P

MM M M

• also known asdistributed memory

• no global memory• using message passing to

move data from one toanother (Send/Recieve

pair of commands)

• this architecture giveway to Internet

connected systems

MIMD Architecture

MM M M

P P P P

MM M M

programming is easier provided scalability

DSM (distributed-shared memory) is

the hybrid between the two

• memory is physically distributed [messagepassing]

• memory can be addressed as one (logicallyshared) address space [shared memory]

• programming-wise, the architecture looks

and behaves like a shared memorymachine, but a message passing architecturelives underneath the software

SGI Origin2000

• access control - which process accesses arepossible to which resources

• synchronization - constraints limit the timeof accesses from sharing processes toshared resources

• protection - a system feature that preventsprocesses from making arbitrary access toresources belonging to other processes

• nodes are typically able to simultaneously

• store messages in buffers

• perform send/receive operations

• scalable - the number of processors can beincreased without significant decrease in

efficiency of operation

InterconnectionNetworks

Interconnection

Networks INs• Can be classified based on

• mode of operation

• control strategy

• switching techniques

• topology

Mode of Operation

• Accordingly, INs are classified as:

• Synchronous

•a single global clock used by all

• operating in a lock-step manner

• Asynchronous

•does not require a global clock

• handshaking signals are used

• Sync tends to be slower than async, sync is raceand hazard-free, however.

Control Strategy

• Accordingly, INs are classified as

• Centralized

• a single central CU is used to overseeand control the operation

•Decentralized

• the control function is distributedamong different components

Control Strategy

• The function and reliability of the central

control unit can become the bottleneck ina centralized control system

• While the crossbar is a centralized system,

the multistage interconnection networksare decentralized

Switching Techniques

• INs can be classified as:

• circuit switching

• a complete path has to be established and remain

existence during the whole communication

• packet switching

• communication takes place via messages that are dividedinto smaller entities (packets)

• packets travel in a store-and-forward manner

• While packet s/w tends to use resources more efficiently, itsuffers from variable packet delays

Topology

• Topology describes how to connectprocessors and memories to other

processors and memories

Shared Memory INs

bus-based switch-based

Message Passing INs

• Static interconnection network

• Dynamic interconnection network

Static INs

Dynamic INs

• Establish a connection between two ormore nodes on the fly as messages are

routed along the links

• The number of hops in a path from sourceto destination node is equal to the number

of point-to-point links a message musttraverse to reach its destination

Single-stage

Crossbar switch

slide02 parallel computers

Documents

lecture 2 interconnection networks for parallel computers

2.5 classiﬁcation of parallel computers

super computers ---parallel computers prof. sin-min lee...

parallel computingparallelcomp.github.io/lecture1.pdf ·...

slide02 2 functions

architecture of computers and parallel systems part 9...

super computers parallel processing

super computers ---parallel computers

understanding parallel computers parallel processing ee 613

parallel computers today

parallel computers

a shaped-charge parallel computers -...

cs267 applications of parallel computers

cs 591x – cluster computing and programming parallel...

architecture of computers and parallel systems part 4...

architecture of computers and parallel systems...

scalable parallel computers

parallel computers - ime-uspsong/mac5705/slides1.pdfslides...

architectures of parallel computers

slide02 - statistika deskriptif (eksplorasi data)