comp 322: principles of parallel programming lecture 11 ...vs3/pdf/comp322-lec11-f09-v1.pdf · comp...
TRANSCRIPT
COMP 322 Lecture 11 29 September 2009
COMP 322: Principles of Parallel Programming
Lecture 11: Parallel Programming Issues (Chapter 6)
Fall 2009
Vivek Sarkar Department of Computer Science
Rice University [email protected]
COMP 322, Fall 2009 (V.Sarkar)!2
Summary of Previous Lecture!
•! Example of Pipeline Parallelism with HJ Phasers
•! Overview of POSIX Threads (Chapter 6)
•! Question for you to think about: —!Which POSIX Threads examples in the lecture or book cannot be
expressed using HJ constructs?
COMP 322, Fall 2009 (V.Sarkar)!3
Acknowledgments for Today"s Lecture!•! Course text: “Principles of Parallel Programming”, Calvin Lin & Lawrence
Snyder
—!Includes resources available at http://www.pearsonhighered.com/educator/academic/product/0,3110,0321487907,00.html
•! “Phasers: a Unified Deadlock-Free Construct for Collective and Point-to-point Synchronization”, Jun Shirako, David M. Peixotto, Vivek Sarkar, William N. Scherer III
—!http://www.cs.rice.edu/~vs3/PDF/SPSS08-phasers.pdf
•! Companion slides for “The Art of Multiprocessor Programming” by Maurice Herlihy & Nir Shavit
—! http://www.cs.brown.edu/courses/cs176/ch01.ppt
COMP 322, Fall 2009 (V.Sarkar)!4
Pthreads Issues: Fairness!
•! Consider three POSIX threads (0, 1, 2) that repeatedly execute the following loop: while (true) {
pthread_mutex_lock(&lock);
printf(“Hello from thread %d\n”, pthread_self());
pthread_mutex_unlock(&lock);
}
•! What is the relative number of print statements that you might expect from threads 0, 1, 2?
•! What is the minimum number of print statements that might be guaranteed from threads 0, 1, 2?
•! No guarantee of fairness in the POSIX threads standard
•! Order in which locks are acquired is not guaranteed to match the order in which the threads attempt to acquire the locks
COMP 322, Fall 2009 (V.Sarkar)!5
Pthreads Issues: Serializability !
•! A concurrent execution is serializable if the execution is guaranteed to correspond to some serial execution of those threads.
Consider
/* threads compete to update global variable best_cost */
if (my_cost < best_cost) !
best_cost = my_cost; !
—!two threads
—!initial value of best_cost is 100
—!values of my_cost are 50 and 75 for threads t1 and t2
•! After execution, best_cost could be 50 or 75
•! 75 does not correspond to any serialization of the threads
! Use mutex to make the parallel program serializable
COMP 322, Fall 2009 (V.Sarkar)!6
Pthreads Issues: Deadlock!
Figure 6.7 Deadlock example. Threads T1 and T2 hold
locks L1 and L2, respectively, and each thread attempts
to acquire the other lock, which cannot be granted."
COMP 322, Fall 2009 (V.Sarkar)!7
Lock Hierarchies!
•! A simple way to prevent deadlocks is to prevent cycles in the resource allocation graph.
•! Impose an order on the locks, and require that all threads acquire their locks in the same order !!Lock hierarchy!
•! What if a thread doesn’t know a priori which locks it needs to acquire? 1.# On learning of a new lock, it can release all its existing locks and
then reacquire all locks in the proper order
2.# Use pthread_mutex_trylock() on the new lock. If that fails then revert to option 1. above.
COMP 322, Fall 2009 (V.Sarkar)!8
Counter Implementation!
public class Counter { private long value;
public long getAndIncrement() { return value++; } }
COMP 322, Fall 2009 (V.Sarkar)!9
Counter Implementation!
public class Counter { private long value;
public long getAndIncrement() { return value++; } } OK fo
r single t
hread,
not for co
ncurrent t
hreads
COMP 322, Fall 2009 (V.Sarkar)!10
What It Means!
public class Counter { private long value;
public long getAndIncrement() { return value++; } }
COMP 322, Fall 2009 (V.Sarkar)!11
What It Means!
public class Counter { private long value;
public long getAndIncrement() { return value++; } }
temp = value;
value = temp + 1;
return temp;
COMP 322, Fall 2009 (V.Sarkar)!12
time
Not so good…!
Value… 1
read 1
read 1
write 2
read 2
write 3
write 2
2 3 2
COMP 322, Fall 2009 (V.Sarkar)!13
Is this problem inherent?!
If we could only glue reads and writes together…
read
write read
write
!! !!
COMP 322, Fall 2009 (V.Sarkar)!14
Challenge!
public class Counter { private long value;
public long getAndIncrement() { temp = value; value = temp + 1; return temp; } }
COMP 322, Fall 2009 (V.Sarkar)!15
Challenge!
public class Counter { private long value;
public long getAndIncrement() { temp = value; value = temp + 1; return temp; } }
Make these steps atomic (isolated)
COMP 322, Fall 2009 (V.Sarkar)!16
Hardware Solution!
public class Counter { private long value;
public long getAndIncrement() { temp = value; value = temp + 1; return temp; } } ReadModifyWrite()
instruction
COMP 322, Fall 2009 (V.Sarkar)!17
HJ Solution!
public class Counter { private long value;
public long getAndIncrement() { isolated { temp = value; value = temp + 1; } return temp; } }
COMP 322, Fall 2009 (V.Sarkar)!18
Mutual Exclusion or “Alice & Bob share a pond”!
A B
COMP 322, Fall 2009 (V.Sarkar)!19
Alice has a pet!
A B
COMP 322, Fall 2009 (V.Sarkar)!20
Bob has a pet!
A B
COMP 322, Fall 2009 (V.Sarkar)!21
The Problem!
A B
The pets don’t get along
COMP 322, Fall 2009 (V.Sarkar)!22
Formalizing the Problem!
•! Two types of formal properties in asynchronous computation:
•! Safety Properties —!Nothing bad happens ever
•! Liveness Properties —!Something good happens eventually
COMP 322, Fall 2009 (V.Sarkar)!23
Formalizing our Problem!
•! Mutual Exclusion —!Both pets never in pond simultaneously
—!This is a safety property
•! No Deadlock —!if only one wants in, it gets in
—!if both want in, one gets in.
—!This is a liveness property
COMP 322, Fall 2009 (V.Sarkar)!24
Simple Protocol!
•! Idea —!Just look at the pond
•! Gotcha —!Not atomic
—!Trees obscure the view
COMP 322, Fall 2009 (V.Sarkar)!25
Interpretation!
•! Threads can’t “see” what other threads are doing
•! Explicit communication required for coordination
COMP 322, Fall 2009 (V.Sarkar)!26
Cell Phone Protocol!
•! Idea —!Bob calls Alice (or vice-versa)
•! Gotcha —!Bob takes shower
—!Alice recharges battery
—!Bob out shopping for pet food …
COMP 322, Fall 2009 (V.Sarkar)!27
Interpretation!
•! Message-passing doesn’t work
•! Recipient might not be —!Listening
—!There at all
•! Communication must be —!Persistent (like writing)
—!Not transient (like speaking)
COMP 322, Fall 2009 (V.Sarkar)!28
Can Protocol!
cola
cola
COMP 322, Fall 2009 (V.Sarkar)!29
Bob conveys a bit!
A B
cola
COMP 322, Fall 2009 (V.Sarkar)!30
Bob conveys a bit!
A B
cola
COMP 322, Fall 2009 (V.Sarkar)!31
Can Protocol!
•! Idea —!Cans on Alice’s windowsill
—!Strings lead to Bob’s house
—!Bob pulls strings, knocks over cans
•! Gotcha —!Cans cannot be reused
—!Bob runs out of cans
COMP 322, Fall 2009 (V.Sarkar)!32
Interpretation!
•! Cannot solve mutual exclusion with interrupts —!Sender sets fixed bit in receiver’s space
—!Receiver resets bit when ready
—!Requires unbounded number of interrupt bits
COMP 322, Fall 2009 (V.Sarkar)!33
Flag Protocol!
A B
COMP 322, Fall 2009 (V.Sarkar)!34
Alice"s Protocol (sort of)!
A B
COMP 322, Fall 2009 (V.Sarkar)!35
Bob"s Protocol (sort of)!
A B
COMP 322, Fall 2009 (V.Sarkar)!36 Art of Multiprocessor Programming
36
Alice"s Protocol!
•! Raise flag
•! Wait until Bob’s flag is down
•! Unleash pet
•! Lower flag when pet returns
COMP 322, Fall 2009 (V.Sarkar)!37
Bob"s Protocol!
•! Raise flag
•! Wait until Alice’s flag is down
•! Unleash pet
•! Lower flag when pet returns
COMP 322, Fall 2009 (V.Sarkar)!38
Bob"s Protocol (2nd try)!
•! Raise flag
•! While Alice’s flag is up —!Lower flag
—!Wait for Alice’s flag to go down
—!Raise flag
•! Unleash pet
•! Lower flag when pet returns
COMP 322, Fall 2009 (V.Sarkar)!39
Remarks!
•! Protocol is unfair —!Bob’s pet might never get in
•! Protocol uses waiting —!If Bob is eaten by his pet, Alice’s pet might never get in
COMP 322, Fall 2009 (V.Sarkar)!40
Moral of Story!
•!Mutual Exclusion cannot be solved by
—!transient communication (cell phones)
—!interrupts (cans)
•! It can be solved by
—! one-bit shared variables
—! that can be read or written
COMP 322, Fall 2009 (V.Sarkar)!41
The Fable Continues!
•! Alice and Bob fall in love & marry
COMP 322, Fall 2009 (V.Sarkar)!42
The Fable Continues!
•! Alice and Bob fall in love & marry
•! Then they fall out of love & divorce —!She gets the pets
—!He has to feed them
COMP 322, Fall 2009 (V.Sarkar)!43
The Fable Continues!
•! Alice and Bob fall in love & marry
•! Then they fall out of love & divorce —!She gets the pets
—!He has to feed them
•! Leading to a new coordination problem: Producer-Consumer
COMP 322, Fall 2009 (V.Sarkar)!44
Bob Puts Food in the Pond!
A
COMP 322, Fall 2009 (V.Sarkar)!45
mmm…
Alice releases her pets to Feed!
B mmm…
COMP 322, Fall 2009 (V.Sarkar)!46
Producer/Consumer!
•! Alice and Bob can’t meet —!Each has restraining order on other
—!So he puts food in the pond
—!And later, she releases the pets
•! Avoid —!Releasing pets when there’s no food
—!Putting out food if uneaten food remains
COMP 322, Fall 2009 (V.Sarkar)!47
Producer/Consumer!
•! Need a mechanism so that —!Bob lets Alice know when food has been put out
—!Alice lets Bob know when to put out more food
COMP 322, Fall 2009 (V.Sarkar)!48
Surprise Solution!
A B
cola
COMP 322, Fall 2009 (V.Sarkar)!49
Bob puts food in Pond!
A B
cola
COMP 322, Fall 2009 (V.Sarkar)!50
Bob knocks over Can!
A B
cola
COMP 322, Fall 2009 (V.Sarkar)!51
Alice Releases Pets!
A B
cola
yum… B yum…
COMP 322, Fall 2009 (V.Sarkar)!52
Alice Resets Can when Pets are Fed!
A B
cola
COMP 322, Fall 2009 (V.Sarkar)!53 Art of Multiprocessor Programming
53
Pseudocode!
while (true) { while (can.isUp()){}; pet.release(); pet.recapture(); can.reset(); }
Alice’s code
COMP 322, Fall 2009 (V.Sarkar)!54
Pseudocode!
while (true) { while (can.isUp()){}; pet.release(); pet.recapture(); can.reset(); }
Alice’s code
while (true) { while (can.isDown()){}; pond.stockWithFood(); can.knockOver(); }
Bob’s code
COMP 322, Fall 2009 (V.Sarkar)!55
Correctness!
•! Mutual Exclusion —!Pets and Bob never together in pond
COMP 322, Fall 2009 (V.Sarkar)!56
Correctness!•! Mutual Exclusion
—!Pets and Bob never together in pond
•! No Starvation if Bob always willing to feed, and pets always famished, then
pets eat infinitely often.
COMP 322, Fall 2009 (V.Sarkar)!57
Correctness!•! Mutual Exclusion
—!Pets and Bob never together in pond
•! No Starvation if Bob always willing to feed, and pets always famished, then
pets eat infinitely often.
•! Producer/Consumer The pets never enter pond unless there is food, and Bob never
provides food if there is unconsumed food.
safety
liveness
safety
COMP 322, Fall 2009 (V.Sarkar)!58
Could Also Solve Using Flags!
A B
COMP 322, Fall 2009 (V.Sarkar)!59
Figure 6.1!
A bounded buffer with producers and consumers. The Put and Get cursors indicate where the producers will insert the next item and where the consumers will remove its next item.
COMP 322, Fall 2009 (V.Sarkar)!60
Figure 6.3 Bounded buffer example using condition variables nonempty and nonfull.!
COMP 322, Fall 2009 (V.Sarkar)!61
Bound option in phasers!•! Constructor
—!phaser(mode m, int bound_size);
•! next operation —!A task registered in SIG mode will block if it is >= bound_size
phases past the current phase
COMP 322, Fall 2009 (V.Sarkar)!62
Single-Producer Single-Consumer Bounded Buffer!
finish {!
final phaser ph = new phaser(<SIG_WAIT>, bound_size);!
async phased (ph<SIG>) !
while (…) { insert(); next; } // producer!
async phased (ph<WAIT>)!
while (…) { next; remove(); } // consumer!
}!
COMP 322, Fall 2009 (V.Sarkar)!63
Summary of Today"s Lecture!
•! POSIX Threads as an illustration of parallel programming issues: —!Fairness
—!Serializability
—!Deadlock
—!Safety
—!Liveness
—!Bounded buffer example
•! Many of these issues arise in other parallel programming languages, but constructs in higher level languages such as HJ can help (async, finish, isolated, phasers)