1

Bulk Synchronous Parallel Computing

Trevor SchaubJim Sellers

This presentation was prepared for Professor Stefan Dobrev in partial fulfillment of the requirements for the course CSI 4140

2

Outline Model description Why it is interesting/important Relationship to real life Some basic techniques, simple

sample program to demonstrate how the approach to algorithm design differs from PRAM and MPP

3

Why do we need change? Diverse parallel computers and models Shared mem. vs. message passing, no

clear winner Use network locality vs. don’t

Conclusion: parallel code is often computer specific. Need unifying model

4

Why BSP? Provides consistent, general framework

to develop scaleable software Similar to OO – a way of thinking and writing

programs (paradigm) Not system specific Proceed in “supersteps”

Decouples communication and synchronization

The deadlock problem is minimized

5

What is BSP? BSP is a process of dividing a task

into steps called supersteps Each process performs the task by

doing the supersteps in order The fundamental concept is the

superstep itself, and it’s structure

6

What’s a superstep?

1. Processes perform as much as possible using local data

2. Communication and synchronization

3. Move on to next superstepsuperstep

…

7

Dissection of a superstep

Local computation

Global CommunicationSynchronization

Threads/Processes

T

I

M

E

8

Model description What are the benefits of a superstep?

Easier debugging Can look at supersteps in isolation

Removes problem of deadlock Separate computation from communication

Allows reasoning of the correctness of the code nearly as easy as sequential code.

Easier shift for programmers used to sequential programming

9

Relationship to real life Is BSP in use today?

Yes, sometimes without even realizing it.

When should you use BSP? Becomes more useful with more

communication Benefits vary with the problem

10

Basic techniques Identify the sections that can be that

need communication or synchronization Modify program so that sync. happens

after communication Separate ones are grouped where possible

Before each communication group there is a global barrier

The code between the end of each communication group becomes a superstep

11

Example: See code example

12

References http://approximity.com/papers/ptfopt/node31.html http://www.cs.ucf.edu/csdept/faculty/goudreau/

cop5937_fall96/COP5937.html http://web.comlab.ox.ac.uk/oucl/research/highlights/

bsp_computing.html http://www.math.uu.nl/people/bisselin/pas1_prev.html http://www.math.ruu.nl/people/bisselin/software.html http://web.comlab.ox.ac.uk/oucl/work/bill.mccoll/

oparl.html http://www.cs.hmc.edu/courses/2001/spring/cs156/

html12/slides12.pdf http://www.cag.lcs.mit.edu/bayanihan/papers/javapdc99/

html/node2.html http://www.byte.com/art/9611/sec5/art5.htm

13