Concurrent Queues and Stacks
Companion slides for
The Art of Multiprocessor Programming
by Maurice Herlihy & Nir Shavit
Art of Multiprocessor Programming 2 2
The Five-Fold Path
• Coarse-grained locking
• Fine-grained locking
• Optimistic synchronization
• Lazy synchronization
• Lock-free synchronization
Art of Multiprocessor Programming 3 3
Another Fundamental Problem
• We told you about
– Sets implemented by linked lists
• Next: queues
• Next: stacks
Art of Multiprocessor Programming 4 4
Queues & Stacks
• pool of items
Art of Multiprocessor Programming 5 5
Queues
deq()/ enq( )
Total order
First in
First out
Art of Multiprocessor Programming 6 6
Stacks
pop()/
push( )
Total order
Last in
First out
Art of Multiprocessor Programming 7 7
Bounded
• Fixed capacity
• Good when resources an issue
Art of Multiprocessor Programming 8 8
Unbounded
• Unlimited capacity
• Often more convenient
Art of Multiprocessor Programming 9
Blocking
zzz …
Block on attempt to remove from empty stack or queue
Art of Multiprocessor Programming 10
Blocking
zzz …
Block on attempt to add to full bounded stack or queue
Art of Multiprocessor Programming 11
Non-Blocking
Ouch!
Throw exception on attempt to remove from empty stack or queue
Art of Multiprocessor Programming 12 12
This Lecture
• Queue
– Bounded, blocking, lock-based
– Unbounded, non-blocking, lock-free
• Stack
– Unbounded, non-blocking lock-free
– Elimination-backoff algorithm
Art of Multiprocessor Programming 13 13
Queue: Concurrency
enq(x) y=deq()
enq() and deq()
work at different
ends of the object
tail head
Art of Multiprocessor Programming 14 14
Concurrency
enq(x)
Challenge: what if
the queue is empty
or full?
y=deq()
Art of Multiprocessor Programming 15 15
Bounded Queue
Sentinel
head
tail
Art of Multiprocessor Programming 16 16
Bounded Queue
head
tail
First actual item
Art of Multiprocessor Programming 17 17
Bounded Queue
head
tail
Lock out other
deq() calls
deqLock
Art of Multiprocessor Programming 18 18
Bounded Queue
head
tail
Lock out other
enq() calls
deqLock
enqLock
Art of Multiprocessor Programming 19 19
Not Done Yet
head
tail
deqLock
enqLock
Need to tell whether
queue is full or
empty
Art of Multiprocessor Programming 20 20
Not Done Yet
head
tail
deqLock
enqLock
Max size is 8 items
size
1
Art of Multiprocessor Programming 21 21
Not Done Yet
head
tail
deqLock
enqLock
Incremented by enq()
Decremented by deq()
size
1
Art of Multiprocessor Programming 22 22
Enqueuer
head
tail
deqLock
enqLock
size
1
Lock enqLock
Art of Multiprocessor Programming 23 23
Enqueuer
head
tail
deqLock
enqLock
size
1
Read size
OK
Art of Multiprocessor Programming 24 24
Enqueuer
head
tail
deqLock
enqLock
size
1
No need to
lock tail
Art of Multiprocessor Programming 25 25
Enqueuer
head
tail
deqLock
enqLock
size
1
Enqueue Node
Art of Multiprocessor Programming 26 26
Enqueuer
head
tail
deqLock
enqLock
size
1 2
getAndincrement()
Art of Multiprocessor Programming 27 27
Enqueuer
head
tail
deqLock
enqLock
size
8 Release lock
2
Art of Multiprocessor Programming 28 28
Enqueuer
head
tail
deqLock
enqLock
size
2
If queue was empty,
notify waiting dequeuers
Art of Multiprocessor Programming 29 29
Unsuccesful Enqueuer
head
tail
deqLock
enqLock
size
8
Uh-oh
Read size
…
Art of Multiprocessor Programming 30 30
Dequeuer
head
tail
deqLock
enqLock
size
2
Lock deqLock
Art of Multiprocessor Programming 31 31
Dequeuer
head
tail
deqLock
enqLock
size
2
Read sentinel’s
next field
OK
Art of Multiprocessor Programming 32 32
Dequeuer
head
tail
deqLock
enqLock
size
2
Read value
Art of Multiprocessor Programming 33 33
Dequeuer
head
tail
deqLock
enqLock
size
2
Make first Node
new sentinel
Art of Multiprocessor Programming 34 34
Dequeuer
head
tail
deqLock
enqLock
size
1
Decrement
size
Art of Multiprocessor Programming 35 35
Dequeuer
head
tail
deqLock
enqLock
size
1 Release
deqLock
Art of Multiprocessor Programming 36 36
Unsuccesful Dequeuer
head
tail
deqLock
enqLock
size
0
Read sentinel’s
next field
uh-oh
Art of Multiprocessor Programming 37 37
Digression: Monitor Locks
• Java synchronized objects and ReentrantLocks are monitors
• Allow blocking on a condition rather than spinning
• Threads: – acquire and release lock
– wait on a condition
Art of Multiprocessor Programming 38 38
public interface Lock {
void lock();
void lockInterruptibly() throws InterruptedException;
boolean tryLock();
boolean tryLock(long time, TimeUnit unit);
Condition newCondition();
void unlock;
}
The Java Lock Interface
Acquire lock
Art of Multiprocessor Programming 39 39
public interface Lock {
void lock();
void lockInterruptibly() throws InterruptedException;
boolean tryLock();
boolean tryLock(long time, TimeUnit unit);
Condition newCondition();
void unlock;
}
The Java Lock Interface
Release lock
Art of Multiprocessor Programming 40 40
public interface Lock {
void lock();
void lockInterruptibly() throws InterruptedException;
boolean tryLock();
boolean tryLock(long time, TimeUnit unit);
Condition newCondition();
void unlock;
}
The Java Lock Interface
Try for lock, but not too hard
Art of Multiprocessor Programming 41 41
public interface Lock {
void lock();
void lockInterruptibly() throws InterruptedException;
boolean tryLock();
boolean tryLock(long time, TimeUnit unit);
Condition newCondition();
void unlock;
}
The Java Lock Interface
Create condition to wait on
Art of Multiprocessor Programming 42 42
The Java Lock Interface
public interface Lock {
void lock();
void lockInterruptibly() throws InterruptedException;
boolean tryLock();
boolean tryLock(long time, TimeUnit unit);
Condition newCondition();
void unlock;
}
Never mind what this method does
Art of Multiprocessor Programming 43 43
Lock Conditions
public interface Condition {
void await();
boolean await(long time, TimeUnit unit);
…
void signal();
void signalAll();
}
Art of Multiprocessor Programming 44 44
public interface Condition {
void await();
boolean await(long time, TimeUnit unit);
…
void signal();
void signalAll();
}
Lock Conditions
Release lock and
wait on condition
Art of Multiprocessor Programming 45 45
public interface Condition {
void await();
boolean await(long time, TimeUnit unit);
…
void signal();
void signalAll();
}
Lock Conditions
Wake up one waiting thread
Art of Multiprocessor Programming 46 46
public interface Condition {
void await();
boolean await(long time, TimeUnit unit);
…
void signal();
void signalAll();
}
Lock Conditions
Wake up all waiting threads
Art of Multiprocessor Programming 47 47
Await
• Releases lock associated with q
• Sleeps (gives up processor)
• Awakens (resumes running)
• Reacquires lock & returns
q.await()
Art of Multiprocessor Programming 48 48
Signal
• Awakens one waiting thread
– Which will reacquire lock
q.signal();
Art of Multiprocessor Programming 49 49
Signal All
• Awakens all waiting threads
– Which will each reacquire lock
q.signalAll();
Art of Multiprocessor Programming 50 50
A Monitor Lock
Cri
tical S
ecti
on
waiting room
lock()
unlock()
Art of Multiprocessor Programming 51 51
Unsuccessful Deq
Cri
tical S
ecti
on
waiting room
lock()
await()
deq()
Oh no,
empty!
Art of Multiprocessor Programming 52 52
Another One
Cri
tical S
ecti
on
waiting room
lock()
await()
deq()
Oh no,
empty!
Art of Multiprocessor Programming 53 53
Enqueuer to the Rescue
Cri
tical S
ecti
on
waiting room
lock()
signalAll() enq( )
unlock()
Yawn! Yawn!
Art of Multiprocessor Programming 54 54
Yawn!
Monitor Signalling
Cri
tical S
ecti
on
waiting room
Yawn!
Awakened thread
might still lose lock to
outside contender…
Art of Multiprocessor Programming 55 55
Dequeuers Signalled
Cri
tical S
ecti
on
waiting room
Found it
Yawn!
Art of Multiprocessor Programming 56 56
Yawn!
Dequeuers Signaled
Cri
tical S
ecti
on
waiting room
Still empty!
Art of Multiprocessor Programming 57 57
Dollar Short + Day Late
Cri
tical S
ecti
on
waiting room
Art of Multiprocessor Programming 58 58
public class Queue<T> {
int head = 0, tail = 0;
T[QSIZE] items;
public synchronized T deq() {
while (tail – head == 0)
wait();
T result = items[head % QSIZE]; head++;
notifyAll();
return result;
}
…
}}
Java Synchronized Methods
Art of Multiprocessor Programming 59 59
public class Queue<T> {
int head = 0, tail = 0;
T[QSIZE] items;
public synchronized T deq() {
while (tail – head == 0)
wait();
T result = items[head % QSIZE]; head++;
notifyAll();
return result;
}
…
}}
Java Synchronized Methods
Each object has an implicit
lock with an implicit condition
Art of Multiprocessor Programming 60 60
public class Queue<T> {
int head = 0, tail = 0;
T[QSIZE] items;
public synchronized T deq() {
while (tail – head == 0)
wait();
T result = items[head % QSIZE]; head++;
notifyAll();
return result;
}
…
}}
Java Synchronized Methods
Lock on entry,
unlock on return
Art of Multiprocessor Programming 61 61
public class Queue<T> {
int head = 0, tail = 0;
T[QSIZE] items;
public synchronized T deq() {
while (tail – head == 0)
wait();
T result = items[head % QSIZE]; head++;
this.notifyAll();
return result;
}
…
}}
Java Synchronized Methods
Wait on implicit
condition
Art of Multiprocessor Programming 62 62
public class Queue<T> {
int head = 0, tail = 0;
T[QSIZE] items;
public synchronized T deq() {
while (tail – head == 0)
this.wait();
T result = items[head % QSIZE]; head++;
notifyAll();
return result;
}
…
}}
Java Synchronized Methods
Signal all threads waiting
on condition
Art of Multiprocessor Programming 63 63
(Pop!) The Bounded Queue
public class BoundedQueue<T> {
ReentrantLock enqLock, deqLock;
Condition notEmptyCondition, notFullCondition;
AtomicInteger size;
Node head;
Node tail;
int capacity;
enqLock = new ReentrantLock();
notFullCondition = enqLock.newCondition();
deqLock = new ReentrantLock();
notEmptyCondition = deqLock.newCondition();
}
Art of Multiprocessor Programming 64 64
Beware Lost Wake-Ups
Cri
tical S
ecti
on
waiting room
lock()
Queue empty
so signal ()
enq( )
unlock()
Yawn!
Art of Multiprocessor Programming 65 65
Lost Wake-Up
Cri
tical S
ecti
on
waiting room
lock()
enq( )
unlock()
Yawn!
Queue not
empty so no
need to signal
Art of Multiprocessor Programming 66 66
Lost Wake-Up
Cri
tical S
ecti
on
waiting room
Yawn!
Art of Multiprocessor Programming 67 67
Lost Wake-Up
Cri
tical S
ecti
on
waiting room
Found it
Art of Multiprocessor Programming 68 68
What’s Wrong Here?
Cri
tical S
ecti
on
waiting room
Still waiting ….!
Art of Multiprocessor Programming 69
Solution to Lost Wakeup
• Always use
– signalAll() and notifyAll()
• Not
– signal() and notify()
69
Art of Multiprocessor Programming 70 70
The enq() & deq() Methods
• Share no locks
– That’s good
• But do share an atomic counter
– Accessed on every method call
– That’s not so good
• Can we alleviate this bottleneck?
Art of Multiprocessor Programming 71 71
Split the Counter
• The enq() method
– Increments only
– Cares only if value is capacity
• The deq() method
– Decrements only
– Cares only if value is zero
Art of Multiprocessor Programming 72 72
Split Counter
• Enqueuer increments enqSize
• Dequeuer decrements deqSize
• When enqueuer runs out
– Locks deqLock
– computes size = enqSize - DeqSize
• Intermittent synchronization
– Not with each method call
– Need both locks! (careful …)
Art of Multiprocessor Programming 73 73
A Lock-Free Queue
Sentinel
head
tail
Art of Multiprocessor Programming 74 74
Compare and Set
CAS
Art of Multiprocessor Programming 75 75
Enqueue
head
tail
enq( )
Art of Multiprocessor Programming 76 76
Enqueue
head
tail
Art of Multiprocessor Programming 77 77
Logical Enqueue
head
tail
CAS
Art of Multiprocessor Programming 78 78
Physical Enqueue
head
tail CAS
Art of Multiprocessor Programming 79 79
Enqueue
• These two steps are not atomic
• The tail field refers to either
– Actual last Node (good)
– Penultimate Node (not so good)
• Be prepared!
Art of Multiprocessor Programming 80 80
Enqueue
• What do you do if you find
– A trailing tail?
• Stop and help fix it
– If tail node has non-null next field
– CAS the queue’s tail field to tail.next
• As in the universal construction
Art of Multiprocessor Programming 81 81
When CASs Fail
• During logical enqueue
– Abandon hope, restart
– Still lock-free (why?)
• During physical enqueue
– Ignore it (why?)
Art of Multiprocessor Programming 82 82
Dequeuer
head
tail
Read value
Art of Multiprocessor Programming 83 83
Dequeuer
head
tail
Make first Node
new sentinel
CAS
Art of Multiprocessor Programming 84 84
Memory Reuse?
• What do we do with nodes after we
dequeue them?
• Java: let garbage collector deal?
• Suppose there is no GC, or we prefer
not to use it?
Art of Multiprocessor Programming 85 85
Dequeuer
head
tail
CAS
Can recycle
Art of Multiprocessor Programming 86 86
Simple Solution
• Each thread has a free list of unused
queue nodes
• Allocate node: pop from list
• Free node: push onto list
• Deal with underflow somehow …
Art of Multiprocessor Programming 87 87
Why Recycling is Hard
Free pool
head tail
Want to
redirect
head from
gray to red
zzz…
Art of Multiprocessor Programming 88 88
Both Nodes Reclaimed
Free pool
zzz
head tail
Art of Multiprocessor Programming 89 89
One Node Recycled
Free pool
Yawn!
head tail
Art of Multiprocessor Programming 90 90
Why Recycling is Hard
Free pool
CAS
head tail
OK, here
I go!
Art of Multiprocessor Programming 91 91
Recycle FAIL
Free pool
zOMG what went wrong?
head tail
Art of Multiprocessor Programming 92 92
The Dreaded ABA Problem head tail
Head reference has value A
Thread reads value A
Art of Multiprocessor Programming 93 93
Dreaded ABA continued
zzz
head tail
Head reference has value B
Node A freed
Art of Multiprocessor Programming 94 94
Dreaded ABA continued
Yawn!
head tail
Head reference has value A again
Node A recycled and reinitialized
Art of Multiprocessor Programming 95 95
Dreaded ABA continued
CAS
head tail
CAS succeeds because references match,
even though reference’s meaning has changed
Art of Multiprocessor Programming 96 96
The Dreaded ABA FAIL
• Is a result of CAS() semantics
– I blame Sun, Intel, AMD, …
• Not with Load-Locked/Store-Conditional
– Good for IBM?
Art of Multiprocessor Programming 97 97
Dreaded ABA – A Solution
• Tag each pointer with a counter
• Unique over lifetime of node
• Pointer size vs word size issues
• Overflow?
– Don’t worry be happy?
– Bounded tags?
• AtomicStampedReference class
Art of Multiprocessor Programming 98 98
Atomic Stamped Reference
• AtomicStampedReference class
– Java.util.concurrent.atomic package
address S
Stamp
Reference
Can get reference & stamp atomically
Art of Multiprocessor Programming 99 99
Concurrent Stack
• Methods
– push(x)
– pop()
• Last-in, First-out (LIFO) order
• Lock-Free!
Art of Multiprocessor Programming 100 100
Empty Stack
Top
Art of Multiprocessor Programming 101 101
Push
Top
Art of Multiprocessor Programming 102 102
Push
Top CAS
Art of Multiprocessor Programming 103 103
Push
Top
Art of Multiprocessor Programming 104 104
Push
Top
Art of Multiprocessor Programming 105 105
Push
Top
Art of Multiprocessor Programming 106 106
Push
Top CAS
Art of Multiprocessor Programming 107 107
Push
Top
Art of Multiprocessor Programming 108 108
Pop
Top
Art of Multiprocessor Programming 109 109
Pop
Top CAS
Art of Multiprocessor Programming 110 110
Pop
Top CAS
mine!
Art of Multiprocessor Programming 111 111
Pop
Top CAS
Art of Multiprocessor Programming 112 112
Pop
Top
Art of Multiprocessor Programming 113 113
Lock-free Stack
• Good
– No locking
• Bad
– Without GC, fear ABA
– Without backoff, huge contention at top
– In any case, no parallelism
Art of Multiprocessor Programming 114 114
Big Question
• Are stacks inherently sequential?
• Reasons why
– Every pop() call fights for top item
• Reasons why not
– Stay tuned …
Art of Multiprocessor Programming 115 115
Elimination-Backoff Stack
• How to
– “turn contention into parallelism”
• Replace familiar
– exponential backoff
• With alternative
– elimination-backoff
Art of Multiprocessor Programming 116 116
Observation
Push( )
Pop()
linearizable stack
After an equal number
of pushes and pops,
stack stays the same Yes!
Art of Multiprocessor Programming 117 117
Idea: Elimination Array
Push( )
Pop()
stack
Pick at
random
Pick at
random Elimination
Array
Art of Multiprocessor Programming 118 118
Push Collides With Pop
Push( )
Pop()
stack
continue
continue
No need to
access stack Yes!
Art of Multiprocessor Programming 119 119
No Collision
Push( )
Pop()
stack
If no collision,
access stack
If pushes collide or
pops collide
access stack
Art of Multiprocessor Programming 120 120
Elimination-Backoff Stack
• Lock-free stack + elimination array
• Access Lock-free stack,
– If uncontended, apply operation
– if contended, back off to elimination array
and attempt elimination
Art of Multiprocessor Programming 121 121
Elimination-Backoff Stack
Push( )
Pop() Top CAS
If CAS fails, back off
Art of Multiprocessor Programming 122 122
Dynamic Range and Delay
Push( )
Pick random range and
max waiting time based
on level of contention
encountered
Art of Multiprocessor Programming 123 123
Linearizability
• Un-eliminated calls
– linearized as before
• Eliminated calls:
– linearize pop() immediately after matching
push()
• Combination is a linearizable stack
Art of Multiprocessor Programming 124 124
Un-Eliminated Linearizability
push(v1)
time time
push(v1)
pop(v1) pop(v1)
Art of Multiprocessor Programming 125 125
Eliminated Linearizability
pop(v2) push(v1)
push(v2)
time time
push(v2)
pop(v2) push(v1)
pop(v1)
Collision
Point
Red calls are
eliminated
pop(v1)
Art of Multiprocessor Programming 126 126
Backoff Has Dual Effect
• Elimination introduces parallelism
• Backoff to array cuts contention on lock-
free stack
• Elimination in array cuts down number
of threads accessing lock-free stack
Art of Multiprocessor Programming 127 127
public class EliminationArray {
private static final int duration = ...;
private static final int timeUnit = ...;
Exchanger<T>[] exchanger;
public EliminationArray(int capacity) {
exchanger = new Exchanger[capacity];
for (int i = 0; i < capacity; i++)
exchanger[i] = new Exchanger<T>();
…
}
…
}
Elimination Array
Art of Multiprocessor Programming 128 128
public class EliminationArray {
private static final int duration = ...;
private static final int timeUnit = ...;
Exchanger<T>[] exchanger;
public EliminationArray(int capacity) {
exchanger = new Exchanger[capacity];
for (int i = 0; i < capacity; i++)
exchanger[i] = new Exchanger<T>();
…
}
…
}
Elimination Array
An array of Exchangers
Art of Multiprocessor Programming 129 129
public class Exchanger<T> {
AtomicStampedReference<T> slot
= new AtomicStampedReference<T>(null, 0);
Digression: A Lock-Free
Exchanger
Art of Multiprocessor Programming 130 130
public class Exchanger<T> {
AtomicStampedReference<T> slot
= new AtomicStampedReference<T>(null, 0);
A Lock-Free Exchanger
Atomically modifiable
reference + status
Art of Multiprocessor Programming 131 131
Atomic Stamped Reference
• AtomicStampedReference class
– Java.util.concurrent.atomic package
• In C or C++:
address S reference
stamp
Art of Multiprocessor Programming 132 132
Extracting Reference & Stamp
public T get(int[] stampHolder);
Art of Multiprocessor Programming 133 133
Extracting Reference & Stamp
public T get(int[] stampHolder);
Returns reference to
object of type T
Returns stamp at
array index 0!
Art of Multiprocessor Programming 134 134
Exchanger Status
enum Status {EMPTY, WAITING, BUSY};
Art of Multiprocessor Programming 135 135
Exchanger Status
enum Status {EMPTY, WAITING, BUSY};
Nothing yet
enum Status {EMPTY, WAITING, BUSY};
Art of Multiprocessor Programming 136 136
Exchange Status
Nothing yet
One thread is waiting
for rendez-vous
Art of Multiprocessor Programming 137 137
Exchange Status
enum Status {EMPTY, WAITING, BUSY};
Nothing yet
One thread is waiting
for rendez-vous
Other threads busy
with rendez-vous
Art of Multiprocessor Programming 138 138
public T Exchange(T myItem, long nanos)
throws TimeoutException {
long timeBound = System.nanoTime() + nanos;
int[] stampHolder = {EMPTY};
while (true) {
if (System.nanoTime() > timeBound)
throw new TimeoutException();
T herItem = slot.get(stampHolder);
int stamp = stampHolder[0];
switch(stamp) {
case EMPTY: … // slot is free
case WAITING: … // someone waiting for me
case BUSY: … // others exchanging
}
}
The Exchange
Art of Multiprocessor Programming 139 139
public T Exchange(T myItem, long nanos)
throws TimeoutException {
long timeBound = System.nanoTime() + nanos;
int[] stampHolder = {EMPTY};
while (true) {
if (System.nanoTime() > timeBound)
throw new TimeoutException();
T herItem = slot.get(stampHolder);
int stamp = stampHolder[0];
switch(stamp) {
case EMPTY: … // slot is free
case WAITING: … // someone waiting for me
case BUSY: … // others exchanging
}
}
The Exchange
Item and timeout
Art of Multiprocessor Programming 140 140
public T Exchange(T myItem, long nanos)
throws TimeoutException {
long timeBound = System.nanoTime() + nanos;
int[] stampHolder = {EMPTY};
while (true) {
if (System.nanoTime() > timeBound)
throw new TimeoutException();
T herItem = slot.get(stampHolder);
int stamp = stampHolder[0];
switch(stamp) {
case EMPTY: … // slot is free
case WAITING: … // someone waiting for me
case BUSY: … // others exchanging
}
}
The Exchange
Array holds status
Art of Multiprocessor Programming 141 141
public T Exchange(T myItem, long nanos) throws
TimeoutException {
long timeBound = System.nanoTime() + nanos;
int[] stampHolder = {0};
while (true) {
if (System.nanoTime() > timeBound)
throw new TimeoutException();
T herItem = slot.get(stampHolder);
int stamp = stampHolder[0];
switch(stamp) {
case EMPTY: // slot is free
case WAITING: // someone waiting for me
case BUSY: // others exchanging
}
}}
The Exchange
Loop until timeout
Art of Multiprocessor Programming 142 142
public T Exchange(T myItem, long nanos) throws
TimeoutException {
long timeBound = System.nanoTime() + nanos;
int[] stampHolder = {0};
while (true) {
if (System.nanoTime() > timeBound)
throw new TimeoutException();
T herItem = slot.get(stampHolder);
int stamp = stampHolder[0];
switch(stamp) {
case EMPTY: // slot is free
case WAITING: // someone waiting for me
case BUSY: // others exchanging
}
}}
The Exchange
Get other’s item and status
Art of Multiprocessor Programming 143 143
public T Exchange(T myItem, long nanos) throws
TimeoutException {
long timeBound = System.nanoTime() + nanos;
int[] stampHolder = {0};
while (true) {
if (System.nanoTime() > timeBound)
throw new TimeoutException();
T herItem = slot.get(stampHolder);
int stamp = stampHolder[0];
switch(stamp) {
case EMPTY: … // slot is free
case WAITING: … // someone waiting for me
case BUSY: … // others exchanging
}
}}
The Exchange
An Exchanger has three possible states
Art of Multiprocessor Programming 144 144
Lock-free Exchanger
EMPTY
Art of Multiprocessor Programming 145 145
EMPTY
Lock-free Exchanger
CAS
Art of Multiprocessor Programming 146 146
WAITING
Lock-free Exchanger
Art of Multiprocessor Programming 147 147
Lock-free Exchanger
In search of
partner …
WAITING
Art of Multiprocessor Programming 148 148
WAITING
Lock-free Exchanger
Slot
Still waiting …
Try to exchange
item and set
status to BUSY
CAS
Art of Multiprocessor Programming 149 149
BUSY
Lock-free Exchanger
Slot
Partner showed
up, take item and
reset to EMPTY
item status
Art of Multiprocessor Programming 150 150
EMPTY BUSY
Lock-free Exchanger
Slot
item status
Partner showed
up, take item and
reset to EMPTY
Art of Multiprocessor Programming 151 151
case EMPTY: // slot is free
if (slot.CAS(herItem, myItem, EMPTY, WAITING)) {
while (System.nanoTime() < timeBound){
herItem = slot.get(stampHolder);
if (stampHolder[0] == BUSY) {
slot.set(null, EMPTY);
return herItem;
}}
if (slot.CAS(myItem, null, WAITING, EMPTY)){
throw new TimeoutException();
} else {
herItem = slot.get(stampHolder);
slot.set(null, EMPTY);
return herItem;
}
} break;
Exchanger State EMPTY
Art of Multiprocessor Programming 152 152
case EMPTY: // slot is free
if (slot.CAS(herItem, myItem, EMPTY, WAITING)) {
while (System.nanoTime() < timeBound){
herItem = slot.get(stampHolder);
if (stampHolder[0] == BUSY) {
slot.set(null, EMPTY);
return herItem;
}}
if (slot.CAS(myItem, null, WAITING, EMPTY)){
throw new TimeoutException();
} else {
herItem = slot.get(stampHolder);
slot.set(null, EMPTY);
return herItem;
}
} break;
Exchanger State EMPTY
Try to insert myItem and
change state to WAITING
Art of Multiprocessor Programming 153 153
case EMPTY: // slot is free
if (slot.CAS(herItem, myItem, EMPTY, WAITING)) {
while (System.nanoTime() < timeBound){
herItem = slot.get(stampHolder);
if (stampHolder[0] == BUSY) {
slot.set(null, EMPTY);
return herItem;
}}
if (slot.CAS(myItem, null, WAITING, EMPTY)){
throw new TimeoutException();
} else {
herItem = slot.get(stampHolder);
slot.set(null, EMPTY);
return herItem;
}
} break;
Exchanger State EMPTY
Spin until either
myItem is taken or timeout
Art of Multiprocessor Programming 154 154
case EMPTY: // slot is free
if (slot.CAS(herItem, myItem, EMPTY, WAITING)) {
while (System.nanoTime() < timeBound){
herItem = slot.get(stampHolder);
if (stampHolder[0] == BUSY) {
slot.set(null, EMPTY);
return herItem;
}}
if (slot.CAS(myItem, null, WAITING, EMPTY)){
throw new TimeoutException();
} else {
herItem = slot.get(stampHolder);
slot.set(null, EMPTY);
return herItem;
}
} break;
Exchanger State EMPTY
myItem was taken,
so return herItem
that was put in its place
Art of Multiprocessor Programming 155 155
case EMPTY: // slot is free
if (slot.CAS(herItem, myItem, EMPTY, WAITING)) {
while (System.nanoTime() < timeBound){
herItem = slot.get(stampHolder);
if (stampHolder[0] == BUSY) {
slot.set(null, EMPTY);
return herItem;
}}
if (slot.CAS(myItem, null, WAITING, EMPTY)){
throw new TimeoutException();
} else {
herItem = slot.get(stampHolder);
slot.set(null, EMPTY);
return herItem;
}
} break;
Exchanger State EMPTY
Otherwise we ran out of time,
try to reset status to EMPTY
and time out
Art of Multiprocessor Programming 156 Art of Multiprocessor
Programming© Herlihy-Shavit
2007
156
case EMPTY: // slot is free
if (slot.compareAndSet(herItem, myItem, WAITING,
BUSY)) {
while (System.nanoTime() < timeBound){
herItem = slot.get(stampHolder);
if (stampHolder[0] == BUSY) {
slot.set(null, EMPTY);
return herItem;
}}
if (slot.compareAndSet(myItem, null, WAITING,
EMPTY)){throw new TimeoutException();
} else {
herItem = slot.get(stampHolder);
slot.set(null, EMPTY);
return herItem;
}
} break;
Exchanger State EMPTY
If reset failed,
someone showed up after all,
so take that item
Art of Multiprocessor Programming 157 157
case EMPTY: // slot is free
if (slot.CAS(herItem, myItem, EMPTY, WAITING)) {
while (System.nanoTime() < timeBound){
herItem = slot.get(stampHolder);
if (stampHolder[0] == BUSY) {
slot.set(null, EMPTY);
return herItem;
}}
if (slot.CAS(myItem, null, WAITING, EMPTY)){
throw new TimeoutException();
} else {
herItem = slot.get(stampHolder);
slot.set(null, EMPTY);
return herItem;
}
} break;
Exchanger State EMPTY
Clear slot and take that item
Art of Multiprocessor Programming 158 158
case EMPTY: // slot is free
if (slot.CAS(herItem, myItem, EMPTY, WAITING)) {
while (System.nanoTime() < timeBound){
herItem = slot.get(stampHolder);
if (stampHolder[0] == BUSY) {
slot.set(null, EMPTY);
return herItem;
}}
if (slot.CAS(myItem, null, WAITING, EMPTY)){
throw new TimeoutException();
} else {
herItem = slot.get(stampHolder);
slot.set(null, EMPTY);
return herItem;
}
} break;
Exchanger State EMPTY
If initial CAS failed,
then someone else changed status
from EMPTY to WAITING,
so retry from start
Art of Multiprocessor Programming 159 159
case WAITING: // someone waiting for me
if (slot.CAS(herItem, myItem, WAITING, BUSY))
return herItem;
break;
case BUSY: // others in middle of exchanging
break;
default: // impossible
break;
}
}
}
}
States WAITING and BUSY
Art of Multiprocessor Programming 160 160
case WAITING: // someone waiting for me
if (slot.CAS(herItem, myItem, WAITING, BUSY))
return herItem;
break;
case BUSY: // others in middle of exchanging
break;
default: // impossible
break;
}
}
}
}
States WAITING and BUSY
someone is waiting to exchange,
so try to CAS my item in
and change state to BUSY
Art of Multiprocessor Programming 161 161
case WAITING: // someone waiting for me
if (slot.CAS(herItem, myItem, WAITING, BUSY))
return herItem;
break;
case BUSY: // others in middle of exchanging
break;
default: // impossible
break;
}
}
}
}
States WAITING and BUSY
If successful, return other’s item,
otherwise someone else took it,
so try again from start
Art of Multiprocessor Programming 162 162
case WAITING: // someone waiting for me
if (slot.CAS(herItem, myItem, WAITING, BUSY))
return herItem;
break;
case BUSY: // others in middle of exchanging
break;
default: // impossible
break;
}
}
}
}
States WAITING and BUSY
If BUSY,
other threads exchanging,
so start again
Art of Multiprocessor Programming 163 163
The Exchanger Slot
• Exchanger is lock-free
• Because the only way an exchange can
fail is if others repeatedly succeeded or
no-one showed up
• The slot we need does not require
symmetric exchange
Art of Multiprocessor Programming 164 164
public class EliminationArray {
…
public T visit(T value, int range)
throws TimeoutException {
int slot = random.nextInt(range);
int nanodur = convertToNanos(duration, timeUnit));
return (exchanger[slot].exchange(value, nanodur)
}}
Back to the Stack: the
Elimination Array
Art of Multiprocessor Programming 165 165
public class EliminationArray {
…
public T visit(T value, int range)
throws TimeoutException {
int slot = random.nextInt(range);
int nanodur = convertToNanos(duration, timeUnit));
return (exchanger[slot].exchange(value, nanodur)
}}
Elimination Array
visit the elimination array
with fixed value and range
Art of Multiprocessor Programming 166 166
public class EliminationArray {
…
public T visit(T value, int range)
throws TimeoutException {
int slot = random.nextInt(range);
int nanodur = convertToNanos(duration, timeUnit));
return (exchanger[slot].exchange(value, nanodur)
}}
Elimination Array
Pick a random array entry
Art of Multiprocessor Programming 167 167
public class EliminationArray {
…
public T visit(T value, int range)
throws TimeoutException {
int slot = random.nextInt(range);
int nanodur = convertToNanos(duration, timeUnit));
return (exchanger[slot].exchange(value, nanodur)
}}
Elimination Array
Exchange value or time out
Art of Multiprocessor Programming 168 168
public void push(T value) {
...
while (true) {
if (tryPush(node)) {
return;
} else try {
T otherValue =
eliminationArray.visit(value,policy.range);
if (otherValue == null) {
return;
}
}
Elimination Stack Push
Art of Multiprocessor Programming 169 169
public void push(T value) {
...
while (true) {
if (tryPush(node)) {
return;
} else try {
T otherValue =
eliminationArray.visit(value,policy.range);
if (otherValue == null) {
return;
}
}
Elimination Stack Push
First, try to push
Art of Multiprocessor Programming 170 170
public void push(T value) {
...
while (true) {
if (tryPush(node)) {
return;
} else try {
T otherValue =
eliminationArray.visit(value,policy.range);
if (otherValue == null) {
return;
}
}
Elimination Stack Push
If I failed, backoff & try to eliminate
Art of Multiprocessor Programming 171 171
public void push(T value) {
...
while (true) {
if (tryPush(node)) {
return;
} else try {
T otherValue =
eliminationArray.visit(value,policy.range);
if (otherValue == null) {
return;
}
}
Elimination Stack Push
Value pushed and range to try
Art of Multiprocessor Programming 172 172
public void push(T value) {
...
while (true) {
if (tryPush(node)) {
return;
} else try {
T otherValue =
eliminationArray.visit(value,policy.range);
if (otherValue == null) {
return;
}
}
Elimination Stack Push
Only pop() leaves null,
so elimination was successful
Art of Multiprocessor Programming 173 173
public void push(T value) {
...
while (true) {
if (tryPush(node)) {
return;
} else try {
T otherValue =
eliminationArray.visit(value,policy.range);
if (otherValue == null) {
return;
}
}
Elimination Stack Push
Otherwise, retry push() on lock-free stack
Art of Multiprocessor Programming 174 174
public T pop() {
...
while (true) {
if (tryPop()) {
return returnNode.value;
} else
try {
T otherValue =
eliminationArray.visit(null,policy.range;
if (otherValue != null) {
return otherValue;
}
}
}}
Elimination Stack Pop
Art of Multiprocessor Programming 175 175
public T pop() {
...
while (true) {
if (tryPop()) {
return returnNode.value;
} else
try {
T otherValue =
eliminationArray.visit(null,policy.range;
if ( otherValue != null) {
return otherValue;
}
}
}}
Elimination Stack Pop
If value not null, other thread is a push(),
so elimination succeeded
Art of Multiprocessor Programming 176 176
Summary
• We saw both lock-based and lock-free implementations of
• queues and stacks
• Don’t be quick to declare a data structure inherently sequential
– Linearizable stack is not inherently sequential (though it is in worst case)
• ABA is a real problem, pay attention
Art of Multiprocessor Programming 177 177
This work is licensed under a Creative Commons Attribution-
ShareAlike 2.5 License.
• :
– — to copy, distribute and transmit the work
– — to adapt the work
• :
– . You must attribute the work to “The Art of
Multiprocessor Programming” (but not in any way that
suggests that the authors endorse you or your use of the
work).
– . If you alter, transform, or build upon this work,
you may distribute the resulting work only under the same,
similar or a compatible license.
• For any reuse or distribution, you must make clear to others the
license terms of this work. The best way to do this is with a link
to
– http://creativecommons.org/licenses/by-sa/3.0/.
• Any of the above conditions can be waived if you get permission
from the copyright holder.
• Nothing in this license impairs or restricts the author's moral
rights.
Art of Multiprocessor Programming 178