non-blocking synchronization — what is it and why we (don't?) need it

AlexeyFyodorovOdnoklassniki@23derevo

Non-blockingsynchronizationwhatisitandwhywe(don't?)needit

Whatisthistalkabout?

3

Concurrency

Whoisthistalkfor?

5

For concurrency beginners

SorryPlease go to another room

6

For concurrency beginners

SorryPlease go to another room

For non-blocking programming beginners

A short introduction

For advanced concurrent programmers

We will talk about CAS/atomics implementation details!

For hipsters We will cover internet hype!Immutable vs. Mutable

Intro.Locking

8

9

Main Models

Shared Memory Messaging

write + read send + onReceive

Similar to how we program it Similar to how a real hardware works

Distributed ProgrammingConcurrent Programming

10

Advantages of Parallelism

Resource utilization

Async handling

Simplicity

Utilization of several cores/CPUs

aka PERFORMANCE

Complexity goes to magic frameworks• ArrayBlockingQueue• ConcurrentHashMap• Akka

Responsible services, Responsible UI

11Lock lock = new ReentrantLock()

Lock lock = new ReentrantLock(true)

12Fairness

Lock lock = new ReentrantLock(true)

13

Locking in Java

Old Schoolwait()notify()notifyAll()

synchronized {doSomething();

}

public synchronized foo() {doSomethingElse();

} Lock lock = new ReentrantLock();try {

lock.lock();doSomething();

} finally {lock.unlock();

} Since Java 5

IntheLanguage IntheStandardLibrary(JDK)

java.util.concurrent.*java.util.concurrent.atomic.*

14

Counter

public interface Counter {

long get();

void increment();

}

15

Simple Counter

public interface Counter {

long get();

void increment();

}

public class SimpleCounter implements Counter {

long value = 0;

public long get() {return value;

}

public void increment() {value++;

}

}

16

Volatile Counter

public class VolatileCounter implements Counter {

volatile long value = 0;

public long get() {return value;

}

public void increment() {value++;

}

}

public interface Counter {

long get();

void increment();

}

17

Synchronized Counter

public class SynchronizedCounter implements Counter {

long value = 0;

public synchronized long get() {return value;

}

public synchronized void increment() {value++;

}

}

public interface Counter {

long get();

void increment();

}

18

Disadvantages of Locking

• Deadlocks• Priority Inversion• Reliability - What will happen if lock owner die?

• Performance- Scheduler can push lock owner out- No parallelism inside a critical section!

19

Amdahl’s Law

α non-parallelizable part of the computation

1-α parallelizable part of the computation

p number of threads

Sp="

α#%&α'

Weneedsomethingelse!

WelcometoNon-blockingSynchronization

22

If-Modify-Write

volatile int value = 0;

if (value == 0) {value = 42;

}

NoAtomicityintermsofJMM

23

Compare and Swap

int value = 0;

synchronized (...) {if (value == 0) {

value = 42;}

}

24

Compare and Swap

int value = 0;

i.compareAndSet(0, 42);

25

CAS Semantics

public class PseudoCAS {

private long value;

public synchronized long get() {return value;

}

public synchronized long compareAndSwap(long expectedValue, long newValue) {long oldValue = value;if (oldValue == expectedValue) {

value = newValue;}return oldValue;

}

public synchronized boolean compareAndSet(long expectedValue, long newValue) {return expectedValue == compareAndSwap(expectedValue, newValue);

}

}

26

Pseudo-CAS Counter

public class PseudoCasLoopCounter implements Counter {

private PseudoCAS value = new PseudoCAS();

public long get() {return value.get();

}

public void increment() {long v;do {

v = value.get();} while (value.compareAndSet(v, v + 1));

}

}

public interface Counter {

long get();

void increment();

}

27

Pseudo-CAS Counter

public class PseudoCasLoopCounter implements Counter {

private PseudoCAS value = new PseudoCAS();

public long get() {return value.get();

}

public void increment() {long v;do {

v = value.get();} while (value.compareAndSet(v, v + 1));

}

}

public interface Counter {

long get();

void increment();

}

CASin Java

29

CAS in Java

Since Java 5, JSR166

java.util.concurrent.atomic

• Scalars• Field updaters• Arrays• Compound variables• Accumulators/Adders

since Java 8

30

Scalars

• AtomicBoolean

• AtomicInteger

• AtomicLong

• AtomicReference

31

Scalars

• AtomicBoolean

• AtomicInteger

• AtomicLong

• AtomicReference

32

• boolean compareAndSet(long expect, long update)

• long addAndGet(long delta)

• long getAndAdd(long delta)

• long getAndDecrement()• long getAndIncrement()• long incrementAndGet()• …

AtomicLong

33

• boolean compareAndSet(long expect, long update)

• long addAndGet(long delta)

• long getAndAdd(long delta)

• long getAndDecrement()• long getAndIncrement()• long incrementAndGet()• …

AtomicLong

34

CAS Counter

public interface Counter {

long get();

void increment();

}

public class CasLoopCounter implements Counter {

private AtomicLong value = new AtomicLong();

public long get() {return value.get();

}

public void increment() {long v;do {

v = value.get();} while (value.compareAndSet(v, v + 1));

}

}

35

Compare and Swap — Hardware Support

compare-and-swapCAS

load-link / store-conditionalLL/SC

cmpxchg

ldrex/strex lwarx/stwcx

36

CAS Disadvantages

ContendedCAS —> tonsofuseless CPUcycles

do {v = value.get();

} while (value.compareAndSet(v, v + 1));

Writing fastandcorrectCASalgorithmsrequiresanexpertise

37

• boolean compareAndSet(long expect, long update)

• long addAndGet(long delta)

• long getAndAdd(long delta)

• long getAndDecrement()• long getAndIncrement()• long incrementAndGet()• …

AtomicLong

38

Get-and-Add Counter

public interface Counter {

long get();

void increment();

}

public class CasLoopCounter implements Counter {

private AtomicLong value = new AtomicLong();

public long get() {return value.get();

}

public void increment() {value.getAndAdd(1);

}

}

3939

atomicLong.getAndAdd(5)

JDK7u95 JDK8u74

83

46

15 11

132105

45 43

1 2 3 4

ops/μs

threads

4040

atomicLong.getAndAdd(5)

JDK7u95 JDK8u74

83

46

15 11

132105

45 43

1 2 3 4

ops/μs

threads

4141

atomicLong.getAndAdd(5)

lock addq $0x5,0x10(%rbp))loop:mov 0x10(%rbx),%raxmov %rax,%r11add $0x5,%r11lock cmpxchg %r11,0x10(%rbx)sete %r11bmovzbl %r11b,%r11dtest %r10d,%r10dje loop

JDK7u95-XX:+PrintAssembly JDK8u74-XX:+PrintAssembly

83

46

15 11

132105

45 43

1 2 3 4

ops/μs

threads

42

AtomicLong.getAndAdd() — JDK 7

cmpxchg

43

AtomicLong.getAndAdd() — JDK 8

lock addqJVMIntrinsic

44

Multivariable Invariant

45

Multivariable Invariant

46

Field Updaters

• AtomicIntegerFieldUpdater- Reflection-based updater for volatile int

• AtomicLongFieldUpdater- Reflection-based updater for volatile long

• AtomicReferenceFieldUpdater- Reflection-based updater for volatile object

47

AtomicLongFieldUpdater

long addAndGet(T obj, long delta)

boolean compareAndSet(T obj, long exp, long upd)

long getAndAdd(T obj, long delta)

long incrementAndGet(T obj)

48

Volatile Counter

public class VolatileCounter implements Counter {

volatile long value = 0;

public long get() {return value;

}

public void increment() {value++;

}

}

public interface Counter {

long get();

void increment();

}

49

AtomicLongFieldUpdater-based Counter

public class AFUCounter implements Counter {

private final VolatileCounter counter = new VolatileCounter();

AtomicLongFieldUpdater<VolatileCounter> updater= AtomicLongFieldUpdater.newUpdater(VolatileCounter.class, "value");

public AFUCounter() throws NoSuchFieldException {Field field = VolatileCounter.class.getDeclaredField("value");field.setAccessible(true);

}

public long get() {return updater.get(counter);

}

public void increment() {updater.addAndGet(counter, 1);

}

}

50

AtomicLongFieldUpdater

51

AtomicLongFieldUpdater

52

AtomicArrays

AtomicIntegerArray

AtomicLongArray

AtomicReferenceArray

• long addAndGet(int i, long delta)

• long getAndAdd(int i, long delta)

• boolean compareAndSet(int i, long exp, long upd)

• long incrementAndGet(int i)

• …

53

Compound Variables

AtomicMarkableReferenceV сompareAndSet( V expectedRef, V newRef, boolean expectedMark, boolean newMark)

AtomicStampedReferenceboolean compareAndSet( V expectedRef, V newRef, int expectedStamp, int newStamp)

54

Accumulators

• DoubleAccumulator

• DoubleAdder

• LongAccumulator

• LongAdder

• (Striped64)

• void accumulate(long x)

• long get()

• long getThenReset()

• void reset()

55

Non-blocking Guarantees

Wait-Free Per-thread progress is guaranteed

Lock-Free Overall progress is guaranteed

Obstruction-Free Overall progress is guaranteed if threads don’t interfere with each other

56

AtomicLong-based Counter

public interface Counter {

long get();

void increment();

}

public class CasCounter implements Counter {

private AtomicLong value = new AtomicLong();

public long get() {return value.get();

}

public void increment() {long v;do {

v = value.get();} while (value.compareAndSet(v, v + 1));

}

}

Anyguarantees?A. Wait-FreeB. Lock-FreeC. Obstruction-FreeD. Noguarantees

57

AtomicLong-based Counter

public interface Counter {

long get();

void increment();

}

public class CasCounter implements Counter {

private AtomicLong value = new AtomicLong();

public long get() {return value.get();

}

public void increment() {long v;do {

v = value.get();} while (value.compareAndSet(v, v + 1));

}

}

Anyguarantees?A. Wait-FreeB. Lock-FreeC. Obstruction-FreeD. Noguarantees

58

AtomicLong-based Counter

public interface Counter {

long get();

void increment();

}

public class CasCounter implements Counter {

private AtomicLong value = new AtomicLong();

public long get() {return value.get();

}

public void increment() {long v;do {

v = value.get();} while (value.compareAndSet(v, v + 1));

}

}

Anyguaranteesonx64?A. Wait-FreeB. Lock-FreeC. Obstruction-FreeD. Noguarantees

59

AtomicLong-based Counter

public interface Counter {

long get();

void increment();

}

public class CasCounter implements Counter {

private AtomicLong value = new AtomicLong();

public long get() {return value.get();

}

public void increment() {long v;do {

v = value.get();} while (value.compareAndSet(v, v + 1));

}

}

Anyguaranteesonx64?A. Wait-FreeB. Lock-FreeC. Obstruction-FreeD. Noguarantees

Non-blockingDataStructures

61

Non-blocking Stack

62

Non-blocking queue

Michael and Scott, 1996https://www.research.ibm.com/people/m/michael/podc-1996.pdf

Threads help each other

References

64

Books

65

Concurrency-interest

http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest

Doug Lee and Co

66

Q & A

67

Thank you!