lecture 6-2 : concurrent queues and stacks companion slides for the art of multiprocessor...

Lecture 6-2 :Concurrent Queues and

Stacks

Companion slides forThe Art of Multiprocessor

Programmingby Maurice Herlihy & Nir Shavit

• Data Structure similar to Set– Does not necessarily provide contains()

method– Allows the same item to appear more than

once– get() and set()

Art of Multiprocessor Programming© Herlihy-Shavit 2007

public interface Pool<T> {

void put(T item);

T get();

Queues & Stacks

• Both: pool of items• Queue

– enq() & deq()– First-in-first-out (FIFO) order

• Stack– push() & pop()– Last-in-first-out (LIFO) order

Bounded vs Unbounded

• Bounded– Fixed capacity– Good when resources an issue

• Unbounded– Holds any number of objects

Blocking vs Non-Blocking

• Problem cases:– Removing from empty pool– Adding to full (bounded) pool

• Blocking– Caller waits until state changes

• Non-Blocking– Method throws exception

Queue: Concurrency

enq(x) y=deq()

enq() and deq() work at

different ends of the object

tail head

Concurrency

enq(x)

Challenge: what if the queue is empty or full?

y=deq()

tailhead

A Bounded Lock-Based Queue

public class BoundedQueue<T> {

ReentrantLock enqLock, deqLock;

Condition notEmptyCondition, notFullCondition;

AtomicInteger size;

Node head;

Node tail;

int capacity;

public BoundedQueue(int capacity) {

this.capacity = capacity;

this.head = new Node(null);

this.tail = head;

this.size = new AtomicInteger(0);

this.enqLock = new ReentrantLock();

this.notFullCondition = enqLock.newCondition();

this.deqLock = new ReentrantLock();

this.notEmptyCondition = deqLock.newCondition();

protected class Node {

public T value;

public Node next;

public Node(T x) {

value = x;

next = null;

public void enq(T x) {

boolean mustWakeDequeuers = false;

enqLock.lock();

while (size.get() == capacity)

notFullCondition.await();

Node e = new Node(x);

tail.next = e;

tail = e;

if (size.getAndIncrement() == 0) {

mustWakeDequeuers = true;

} finally {

enqLock.unlock();

if (mustWakeDequeuers) {

deqLock.lock();

notEmptyCondition.signalAll();

} finally {

deqLock.unlock();

public T deq() {

T result;

boolean mustWakeEnqueuers = false;

deqLock.lock();

while (size.get() == 0) {

notEmptyCondition.await();

result = head.next.value;

head = head.next;

if (size.getAndDecrement() == capacity) {

mustWakeEnqueuers = true;

} finally {

deqLock.unlock();

if (mustWakeEnqueuers) {

enqLock.lock();

notFullCondition.signalAll();

} finally {

enqLock.unlock();

return result;

• enqLock/deqLock– At most one enqueuer/dequeuer at a time can

manipulate the queue’s fields

• Two locks– Enqueuer does not lock out dequeuer – vice versa

• Association– enqLock associated with notFullCondition– deqLock associated with notEmptyCondition

enqueue1. Acquires enqLock2. Reads the size field3. If full, enqueuer must wait until dequeuer makes room4. enqueuer waits on notFullCondition field, releasing

enqLock temporarily, and blocking until that condition is signaled.

5. Each time the thread awakens, it checks whether there is a room, and if not, goes back to sleep

6. Insert new item into tail7. Release enqLock8. If queue was empty, notify/signal waiting dequeuers

dequeue1. Acquires deqLock2. Reads the size field3. If empty, dequeuer must wait until item is enqueued4. dequeuer waits on notEmptyCondition field, releasing

deqLock temporarily, and blocking until that condition is signaled.

5. Each time the thread awakens, it checks whether item was enqueued, and if not, goes back to sleep

6. Assigne the value of head’s next node to “result” and reset head to head’s next node

7. Release deqLock8. If queue was full, notify/signal waiting enqueuers9. Return “result”

Art of Multiprocessor Programming 1313

Bounded Queue

Sentinel

Bounded Queue

First actual item

Bounded Queue

Lock out other deq()

deqLock

Bounded Queue

Lock out other enq()

deqLock

enqLock

Not Done Yet

deqLock

enqLock

Need to tell whether queue is full or

Not Done Yet

deqLock

enqLock

Max size is 8 items

Not Done Yet

deqLock

enqLock

Incremented by enq()Decremented by deq()

Enqueuer

deqLock

enqLock

Lock enqLock

Enqueuer

deqLock

enqLock

Read size

Enqueuer

deqLock

enqLock

No need to lock tail

Enqueuer

deqLock

enqLock

Enqueue Node

Enqueuer

deqLock

enqLock

getAndincrement()

Enqueuer

deqLock

enqLock

8 Release lock2

Enqueuer

deqLock

enqLock

If queue was empty, notify

waiting dequeuers

Unsuccesful Enqueuer

deqLock

enqLock

Read size

Dequeuer

deqLock

enqLock

Lock deqLock

Dequeuer

deqLock

enqLock

Read sentinel’s next field

Dequeuer

deqLock

enqLock

Read value

Dequeuer

deqLock

enqLock

Make first Node new sentinel

Dequeuer

deqLock

enqLock

Decrement size

Dequeuer

deqLock

enqLock

1Release deqLock

Monitor Locks

• Java ReentrantLocks are monitors• Allow blocking on a condition

rather than spinning• Threads:

– acquire and release lock– wait on a condition

public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock;}

The Java Lock Interface

Create condition to wait on

Lock Conditions

public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); }

Lock Conditions

Release lock and wait on condition

Lock Conditions

Wake up one waiting thread

Lock Conditions

Wake up all waiting threads

• Releases lock associated with q• Sleeps (gives up processor)• Awakens (resumes running)• Reacquires lock & returns

q.await()

Signal

• Awakens one waiting thread– Which will reacquire lock

q.signal();

Signal All

• Awakens all waiting threads– Which will each reacquire lock

q.signalAll();

Unbounded Lock-Free Queue (Nonblocking)

• Unbounded– No need to count the number of items

• Lock-free– Use AtomicReference<V>

• An object reference that may be updated atomically.

– boolean compareAndSet(V expect, V update)• Atomically sets the value to the given updated value if

the current value == the expected value.

• Nonblocking– No need to provide conditions on which to wait

A Lock-Free Queue

Sentinel

Enqueue

Enq( )

Enqueue

Logical Enqueue

Physical Enqueue

tailCAS

Enqueue

• These two steps are not atomic• The tail field refers to either

– Actual last Node (good)– Penultimate Node (not so good)

• Be prepared!

Enqueue

• What do you do if you find– A trailing tail?

• Stop and help fix it– If tail node has non-null next field– CAS the queue’s tail field to tail.next

• As in the universal construction

When CASs Fail

• During logical enqueue– Abandon hope, restart– Still lock-free (why?)

• During physical enqueue– Ignore it (why?)

Dequeuer

Read value

Dequeuer

Make first Node new sentinel

Unbounded Lock-Free Queue(Nonblocking)

public class LockFreeQueue<T> {

private AtomicReference<Node> head;

private AtomicReference<Node> tail;

public LockFreeQueue() {

this.head = new AtomicReference<Node>(null);

this.tail = new AtomicReference<Node>(null);

public class Node {

public T value;

public AtomicReference<Node> next;

public Node(T value) {

this.value = value;

this.next = new AtomicReference<Node>(null);

public void enq(T item) {

Node node = new Node(item);

while (true) {

Node last = tail.get();

Node next = last.next.get();

if (last == tail.get()) {

if (next == null) {

if (last.next.compareAndSet(next, node)) {

tail.compareAndSet(last, node);

return;

} else {

tail.compareAndSet(last, next);

public T deq() throws EmptyException {

while (true) {

Node first = head.get();

Node last = tail.get();

Node next = first.next.get();

if (first == head.get()) {

if (first == last) {

if (next = null) {

throw new EmptyException();

tail.compareAndSet(last, next);

} else {

T value = next.value;

if (head.compareAndSet(first,next))

return value;

Unbounded Lock-Free Queue(Nonblocking)

Concurrent Stack

• Methods– push(x) – pop()

• Last-in, First-out (LIFO) order• Lock-Free!

Empty Stack

TopCAS

public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff(); }}

Lock-free Stack

public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public Boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) }public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff()}}

Lock-free Stack

tryPush attempts to push a node

public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff()}}

Lock-free Stack

Read top value

Lock-free Stack

current top will be new node’s successor

Lock-free Stack

Try to swing top, return success or failure

Lock-free Stack

Push calls tryPush

Lock-free Stack

Create new node

Lock-free Stack

If tryPush() fails,back off before retrying

protected boolean tryPush(Node node)

Node oldTop = top.get();

node.next = oldTop;

return (top.compareAndSet(oldTop, node));

public void push( T value )

Node node = new Node( value );

while (true) {

if (tryPush(node)) { return; }

else { backoff.backoff( ); }

protected Node tryPop( ) throws EmptyException

Node oldTop = top.get();

if ( oldTop == null ) {

throw new EmptyException( );

Node newTop = oldTop.next;

if ( top.compareAndSet( oldTop, newTop ) ) {

return oldTop;

} else { return null; }

public T pop() throws EmptyException {

while (true) {

Node returnNode = tryPop( );

if ( returnNode != null ) {

return returnNode.value;

} else { backoff.backoff( ); }

Unbounded Lock-Free Stack

Lock-free Stack

• Good– No locking

• Bad– Without GC, fear ABA– Without backoff, huge contention at

top – In any case, no parallelism

Question

• Are stacks inherently sequential?• Reasons why

– Every pop() call fights for top item

• Reasons why not– Think about it!

lecture 6-2 : concurrent queues and stacks companion slides for the art of multiprocessor...

Documents

barrier synchronization companion slides for the art of...

priority queues dan dvorin based on ‘the art of...

monitors and blocking synchronization dalia cohn alperovich...

spin locks and contention companion slides for the art of...

1 based on: the art of multiprocessor programming maurice...

skiplists and balanced search the art of multiprocessor...

foundations of shared memory companion slides for the art of...

multiprocessor architecture basics companion slides for the...

companion slides for the art of multiprocessor programming...

linked lists: lazy and non-blocking synchronization based on...

programming language basics companion slides for the art of...

linked lists: locking, lock-free, and beyond … companion...

mutual exclusion companion slides for the art of...

programming paradigms for concurrency … paradigms for...

concurrent objects companion slides for the art of...

shared counters and parallelism companion slides for the art...

concurrent queues companion slides for the art of...

multiprocessor architecture basics companion slides for the...

concurrency &...

art of multiprocessor programming 1 transactional memory...