introduction to lock-free data-structures and algorithms micah j best may 14/09

Introduction to Lock-freeData-structures and algorithms

Micah J BestMay 14/09

Introduction to the Introduction

• Some problems with locks:– Deadlock: The condition where there are at

least two processes A and B such that A holds a lock on a resource required by B to complete and B holds a lock on a resource required by A to complete and they wait indefinitely.

– In general Deadlock avoidance is hard if not NP-Complete.

(Some) Problems with Locks

• Priority Inversion: A low-priority process holds a lock on a resource desired by a high priority process.

• Not very granular, a small percentage of the operations performed in the critical section may actually modify shared memory, hurting performance

If not locks than what?

• Without hardware support Lock-free algorithms are possible, but not really practical (exceptions do exist: see Lamport’s Bakery Algorithm)

• Hardware support can be generalized as atomic operations.

What is Lock-free? (Definitions)• Atomic: From the Greek ἄτομος (atomos)

meaning indivisible or uncuttable. • Atomic Operations: A set of operations that

execute as if they were a single simple operation.

What is Lock-free? (Definitions)

• Lock-free: Algorithms/Data Structures that can be invoked or accessed in a parallel context accessing shared memory without a mechanism to protect their critical section (such as a mutex)

• Critical Section: A set of instructions necessary to complete a single complete operation on a shared memory resource. (Ex: Pop operation on a stack)

Compare and Swap (CAS)

• For integers n and n' and a memory location a

• CAS( n, a, n' )if the value at address a is n

write the value of n' to address a return true

otherwise return false


• Executes atomically, often the values of n and n' represent memory addresses

• Supported in some form as far back as the IBM 370 and available on almost all modern general purpose microprocessors (Pentium, Power PC, etc)


• Often used to implement 'busy waiting’

... success <- false Do success <- CAS( n, a, n') Until success ...


• Example: Pushing an item onto a lock-free stack Let Top be address of the top of the stack

Let Next(I) where I is a valid address of a stack item be the address where the address of the next item after I is stored

Let New be the address of the item for the stack


• Example: Pushing an item onto a lock-free stack

Push ( New )Success <- falseDo

T' <- TopNext(New) <- T’Success <- CAS( T', Top, New )

Until Success

Compare and Swap (CAS)• Example: Pushing an item onto a lock-free stack Push ( New )Success <- false

DoT' <- TopNext(New) <- T’Success <- CAS( T', Top, New )

Until Success

5 3

Head



Until Success

5 3

Top

We create a new item

7



Until Success

5 3

Top

Read the value of Top

7

T’



Until Success

5 3

Top

Set the new item’s Next to T’

7

T’



Until Success

5 3

Top

Suppose another thread adds an item before the assignment and the CAS

7

T’

3



Until Success

5 3

Top

T’ != Top so the CAS will fail – we must start again

7

T’

3



Until Success

5 3

Top

7

T’

3



Until Success

5 3

Top

7

T’

3

This time the CAS succeeds and we are done

ABA problem

• Where the value read(and stored) by a process (A) is changed by another process (B) to the same value as read by A, but causes some other state change that causes A to produce an incorrect result

ABA problem

• Consider the stack and two threads A and B and the following (incorrect) pop operation:Let Value(I), where I is a valid address of a stack item, be the value associated with I

Pop ()Do

T' <- Top’N' <- Next(T')V <- Value(T') Success <- CAS( T', Top, N' )

Until Success Return V

ABA problemA B

5 3

Top

72

ABA problemA

Do

T' <- TopN' <- Next(T’)V <- Value(T')

B

5 3

Top

72

V = 7

T’

N’

ABA problemA

Do

T' <- Top’N' <- Next(T’)V <- Value(T')

B

Pop()

5 3 2

V = 7

T’

N’

7 is returned

Top

ABA problemA

Do


B

Pop()

Push( 42 )

5 3 2

V = 7

T’42

The new item is created in

the old memory location

N’Top

ABA problemA

Do


Success <- CAS( T', Top, N’ )

B

Pop()

Push( 42 )

5 3 2

V = 7

T’

N’Top

42

The CAS willSucceed

ABA problemA

Do


Success <- CAS( T', Top, N’ )Until Success Return V

B

Pop()

Push( 42 )

5 3 2

V = 7

T’

N’Top

42

7 will be returned

ABA problemA

Do


Success <- CAS( T', Top, N’ )Until Success Return V

B

Pop()

Push( 42 )

5 3 2

V = 7

T’

N’Top

42

7 will be returned- again.

What is Wait-free?

• Wait free: Algorithms/Data Structures that can be invoked or accessed in a parallel context accessing shared memory such that execution time is guaranteed and predictable.

• Almost always lock-free.• Not all lock-free constructions are wait-free

(most use some form of 'busy waiting' which is inherently unpredictable)

Multi-Producer/Consumer Circular Queues

(The Problem)

• Circular queue: An array of fixed size, to be accessed in a FIFO manner

• Multi-producer: More than one process may write to the queue at once

• Multi-consumer: More than one process may read from the queue at once


(The Problem)

• The advantages of lock free:– After a cell in the array has been 'reserved' for

a particular process - writing or reading will cause no contention.

– Only maintenance of queue parameters need made safe


(The Algorithm)

• The advantages of lock free:– After a cell in the array has been 'reserved' for

a particular process - writing or reading will cause no contention.

– Only maintenance of queue parameters need made safe


(The Algorithm)• Sequential Version:

Let A(i) be the element of the array at position i (starting at 0) Let Size be the number of cells in the arrayLet Head be the index of the first free cell for writing Let Tail be the index of the first occupied cell for writingLet Write(i) represent the work to write the data in question to array position i


(The Algorithm)• Sequential Version:

Enqueue()

If not head = tail

Write( Head )

Head <- ( Head + 1 ) mod Size

else

Fail


(The Algorithm)• Obvious not correct in a parallel contextConsider:

A B



A BEnqueue()



A BEnqueue()

Enqueue()



A BEnqueue()

Enqueue()

Write( Head )



A BEnqueue()

Enqueue()

Write( Head )

Write( Head )



A BEnqueue()

Enqueue()

Write( Head )

Write( Head )

Head<-( Head + 1 ) mod Size



A BEnqueue()

Enqueue()

Write( Head )

Write( Head )



Values will be written to the same place


(The Algorithm)

• Need additional information:Let NumWriters and NumReaders be the maximum amount of writers and readers allowed in the queue respectively

Initially NumWriters = Size and NumReaders = 0


(The Algorithm)• First solution: Advance the head pointer

atomicallyLet *Head be the memory address of the value HeadLet *NR be the address of NumReadersEnqueue() ( NumWriters is decremented )

Success <- falseDo

H' <- HeadSuccess <- CAS( H', *Head, (H' + 1) mod Size )

Until SuccessWrite( H' )Increase NumReaders


(The Algorithm)

• Just when you thought is was (thread) safe:Consider the following series of events:

Queue is Empty

HeadNumReaders = 0


(The Algorithm)• Just when you thought is was (thread) safe:

Consider the following series of events: Queue is EmptyThread A succeeds in advancing Head pointer

HeadNumReaders = 0



Consider the following series of events: Queue is EmptyThread A succeeds in advancing Head pointerThread A begins writing

HeadNumReaders = 0

23423234



Consider the following series of events: Queue is EmptyThread A succeeds in advancing Head pointerThread A begins writing Thread B succeeds in advancing Head pointer

HeadNumReaders = 0

234232343



Consider the following series of events: Queue is EmptyThread A succeeds in advancing Head pointerThread A begins writing Thread B succeeds in advancing Head pointerThread B begins writing

HeadNumReaders = 0

23458334

2342323431



Consider the following series of events: Queue is EmptyThread A succeeds in advancing Head pointerThread A begins writing Thread B succeeds in advancing Head pointerThread B begins writingThread B finishes writing

HeadNumReaders = 0

23423234319

234583349839274



Consider the following series of events: Queue is EmptyThread A succeeds in advancing Head pointerThread A begins writing Thread B succeeds in advancing Head pointerThread B begins writingThread B finishes writingThread B succeeds in increasing NumReaders

HeadNumReaders = 1

234232343195

234583349839274



Consider the following series of events: Queue is EmptyThread A succeeds in advancing Head pointerThread A begins writing Thread B succeeds in advancing Head pointerThread B begins writingThread B finishes writingThread B succeeds in increasing NumReaders

HeadNumReaders = 1

234232343195

234583349839274

This can now be read – before

writing has finished!


(The Algorithm)• A 'busy waiting' solution

Let WriteMarker be equal to Head when Queue is created

Enqueue() ( NumWriters is decremented )Success <- falseDo


Until SuccessWrite( H' )Do

W <- WriteMarker Until W = H’WriteMarker <- ( WriteMarker + 1 ) mod SizeIncrease NumReaders


(The Algorithm)Enqueue() ( NumWriters is decremented )

Success <- falseDo




Head

242342342342343

2423423

We begin as another thread is already writing to the queue

H’

WriteMarker



Success <- falseDo




Head

242342342342343

24234234

Copy the head index

H’

WriteMarker



Success <- falseDo




Head

242342342342343

242342348

Advance head(assume we succeed)

H’

WriteMarker



Success <- falseDo




Head

242342342342343

2423423485

Write to the buffer

H’

WriteMarker

97234



Success <- falseDo




Head

242342342342343

24234234858

Write to the buffer

H’

WriteMarker

9723498734



Success <- falseDo




Head

242342342342343

242342348586

Write to the buffer

H’

WriteMarker

972349873402393



Success <- falseDo




Head

242342342342343

2423423485867

Keeping looping until the other thread advances WriteMarker

H’

WriteMarker

972349873402393



Success <- falseDo




Head

242342342342343

24234234858678


H’

WriteMarker

972349873402393



Success <- falseDo




Head

242342342342343

242342348586784


H’

WriteMarker

972349873402393



Success <- falseDo




Head

242342342342343

242342348586784

Now we advance WriteMarker ourselves.

(Why don’t we need to CAS this?)

H’

WriteMarker

972349873402393



Success <- falseDo




Head

242342342342343

242342348586784

Finally we increase NumReaders now that the data

is safe to be read

H’

WriteMarker

972349873402393


(The Algorithm)• Everything works better when we all cooperateLet Done(i) be a boolean variable associated with each cell in the Array, Initially

falseLet *WM be the address of WriteMarkersEnqueue() ( NumWriters is decremented )

Success <- falseDo

H' <- HeadSuccess <- CAS( H', *H, (H' + 1) mod Size )

Until SuccessWrite( H' )


(The Algorithm)• Everything works better when we all cooperate(con’t)Done(H') <- true

Do Success <- falseIf not WriteMarker = Head and Done( WriteMarker )

W <- WriteMarker Success <- CAS( W, *WM, (W+1) mod Size )if Success

Done( W ) <- falseIncrease NumReaders

While Success


(The Algorithm)Done(H') <- true




While Success

Head

2423423

8653869

WriteMarker

972349873402393

Done: false false false falsefalse true false

The story so far:






While Success

Head

2423423

8653869

WriteMarker

972349873402393


The story so far:There are two cells currently

being written






While Success

Head

2423423

8653869

WriteMarker

972349873402393


The story so far:This is ‘us’






While Success

Head

2423423

8653869

WriteMarker

972349873402393


Writing continues






While Success

Head

24234237

86538694

WriteMarker

972349873402393


Writing continues






While Success

Head

2423423745

8653869445

WriteMarker

972349873402393


Writing continues






While Success

Head

24234237458

86538694454

WriteMarker

972349873402393


Writing continues






While Success

Head

2423423745893

8653869445489

WriteMarker

972349873402393


Writing continues






While Success

Head

242342374589334

865386944548973

WriteMarker

972349873402393


Writing finishes






While Success

Head

242342374589334

865386944548973

WriteMarker

972349873402393

Done: false false false falsetrue true true

Both threads update Done






While Success

Head

242342374589334

865386944548973

WriteMarker

972349873402393


Both threads try to move the write pointer






While Success

Head

242342374589334

865386944548973

WriteMarker

972349873402393


We succeed. This means that the other thread has

failed. It will finish knowing that we will ‘clean things up’






While Success

Head

242342374589334

865386944548973

WriteMarker

972349873402393

Done: false false false falsefalse true true

Update the Done flag






While Success

Head

242342374589334

865386944548973

WriteMarker

972349873402393


Let another reader in – now that we know it’s safe.






While Success

Head

242342374589334

865386944548973

WriteMarker

972349873402393


Repeat until all everything is in a proper state or

somebody else takes over.






While Success

Head

242342374589334

865386944548973

WriteMarker

972349873402393

Done: false false false falsefalse false true








While Success

Head

242342374589334

865386944548973

WriteMarker

972349873402393

Done: false false false falsefalse false false








While Success

Head

242342374589334

865386944548973

WriteMarker

972349873402393

Done: false false false falsefalse false false

Exit with the knowledge of a job well done


(Proof - Sketch)• Everything works better when we all cooperate

• Criteria for correctness: – 1) At all times: ( WriteMarker - Tail ) mod Size >=

NumReaders – 2) Enqueue always terminates– 3) At any point in execution if no Enqueue operation is

in progress then WriteMarker = Head– 4) Once an item is enqueued in cell i no other

enqueue operation will write to i until it is dequeued (follows from 1 and case analysis)

Some Performance Results

introduction to lock-free data-structures and algorithms micah j best may 14/09

Documents

value of n

definitions lockfree

stack slide

false slide

performance slide

integers n

values of n

shared memory resource