Max Registers, Counters, and Monotone Circuits

Keren Censor, Technion

Joint work with:James Aspnes, Yale UniversityHagit Attiya, Technion

(Slides 18-21 were added to Keren’s original presentation. DH.)January 2010

Counting

Counter: reading and incrementing operations

In sequential algorithms – one variable Counting is critical for some programs

in multiprocessing systems

Example: Algorithms for randomized consensus

2

012

A single register

Does not work even for 2 processes and 2 increments An increment is not an atomic operation

s1s1 s2

s2

Register

Register

3

read

write(100)

read

write(1)

...

Model of Shared Memory

System of n processes Communicating through Read/Write

registers

4

s1s1 s2

s2

R1R1

snsn

R2R2 Rm

Rm…

…

read v write(v) ok

Model of Distributed Computation Crash failures: a failed process stops

taking steps Wait-free computation: an operation of a process

must work even if all other processes fail Asynchronous system: no timing

assumptions No clocks, no bound on the time between steps Schedule (and failures) controlled by an

adversary Cannot tell whether a process is slow or

failed

5

Model of Distributed Computation Complexity measures:

Number of steps per operation Amount of space

Local computation has no cost

6

Counter

Can be implemented using snapshots in linear time (in n)

7

CounterCounter

incrementok

readCounterv

+1

Snapshot-based counter

One single-writer register for each process

Increment by updating your register Read by reading all the registers Non trivial since we require

linearizability

8

R1R1 R2

R2 RnRn…

Goal and related work

Required: Counters with sub-linear (in the number of processes n) step complexity per operation

Lower bound of Ω(n) for time complexity by Jayanti, Tan, and Toueg [PODC 1996] and similar lower bounds by Ellen, Hendler, and Shavit [FOCS 2005]

Motivated work on approximate counting[Aspnes and C, SODA 2009]

9

Exact counting

Give up on sub-linear exact counting? Or inspect lower bound more carefully:

Based on executions with many increments

But some applications use a small number of increments

We show an implementation of a bounded counter where each operation takes sub-linear time

10

long operation

A tree-based counter

s1s1 s2

s2 s3s3 sn

sns4s4 …

s1+s2s1+s2 s3+s4

s3+s4

s1+...+s4

s1+...+s4

∑si∑si

… …

sn-

1+sn

sn-

1+sn

ReadCounter: return value at root

Increment: recursively increment from leaf to root

+1+1p1 increments

update

update

updateupdate

updateupdate pk reads

11

O(log n) steps to increment

O(1) steps to read counter

Seems nice, but…

If each node is a multi-writer register, then even for 2 processes and 2 increments this does not work

s1s1 s2

s2

s1+s2s1+s2

+1+1p1 increments

p2 increments

+1+1+1+1+1+1

update 1

update 1 update 2update 2

Counter is incorrect

Counter is incorrect

12

Max register

Replace multi-writer registers with Max Registers

In this case the tree-based counter works If max registers are linearizable then so is

counter

13

Max Register

Max Register

WriteMax(v)ok

ReadMaxv

Maximal value

previously written

A tree-based counter

s1s1 s2

s2 s3s3 sn

sns4s4 …

s1+s2s1+s2 s3+s4

s3+s4

s1+...+s4

s1+...+s4

∑si∑si

… …

sn-

1+sn

sn-

1+sn

ReadCounter:return value at root

Increment: recursively increment from leaf to root

14

Max register – recursive construction MaxReg0: Max register that supports only

the value 0 WriteMax is a no-op, and ReadMax returns 0

MaxReg1 supports values in 0,1 Built from two MaxReg0 objects and one additional multi-writer register

“switch”

MaxReg0MaxReg0

15

MaxReg0MaxReg0 MaxReg0

MaxReg0

switchswitchWriteMax 0011 =1=1

ReadMax

switch=0 : return 0

switch=1 : return 1

Max register – recursive construction MaxRegk supports values in 0,…,2k-1

Built from two MaxRegk-1 objects with values in 0,…,2k-1-1

and one additional multi-writer register “switch”

MaxRegk-1MaxRegk-1 MaxRegk-1

MaxRegk-1

MaxRegk

switchswitchWriteMax tttt < 2k-1 ?tt t-2k-1t-2k-1

ReadMax

=1=1

= ?

tt ttswitch=0 : return t

switch=1 : return t+2k-1

16

MaxRegk unfolded

switchswitch

MaxReg0

MaxReg0

…

switchswitch

switchswitch switchswitch

MaxReg0

MaxReg0

MaxReg0

MaxReg0

MaxReg0

MaxReg0

MaxReg0

MaxReg0

……

Complexity does not depend on n:

WriteMax and ReadMax in O(k) steps

MaxRegk

17

Pseudocode: WriteMax(r,t)18

Pseudocode: ReadMax(r)19

Why is this algorithm linearizable?

20

Three operation types:

Operations on left branch: ReadMax(r) ops that read 0 from r.switch and WriteMax(r,t) with t<m that read 0 from r.switch

Operations on right branch: ReadMax(r) ops that read 1 from r.switch and WriteMax(r,t) with tm

Writes with no effect: WriteMax(r,t) ops with t<m that read 1 from r.switch

21

Ordering operations:

Operations on left branch ordered before those on right branch

A Write with no effect linearized at latest possible time within its execution interval

Operations on left and right branch retain their linearization points (and are legal by induction hypothesis).

Why is this algorithm linearizable? (cont'd)

A tree-based counter

s1s1 s2

s2 s3s3 sn

sns4s4 …

s1+s2s1+s2 s3+s4

s3+s4

s1+...+s4

s1+...+s4

∑si∑si

… …

sn-

1+sn

sn-

1+sn

ReadCounter:return value at root

Increment: recursively increment from leaf to root

22

m-valued counter:ReadCounter: O(log m) stepsIncrement: O(log n log m)

steps

Analysis

Inductive linearizability proof No contradiction with lower bound of

JTT because of bounded size of max register and counter

Extension to unbounded max registers (and counters) with complexity according to value written or read Both WriteMax and ReadMax of value v take

O(min(log v, n)) steps

23

Unbalanced tree

MaxReg0

MaxReg0

…

switchswitch

switchswitch

switchswitchswitchswitch

MaxReg0

MaxReg0

MaxReg0

MaxReg0

24

Bentley and Yao [1976]

switchswitch

switchswitchswitchswitch

MaxReg0

MaxReg0

MaxReg0

MaxReg0

MaxReg0

MaxReg0

MaxReg0

MaxReg0

Leaf i is at depth

O(log i)

Unbounded max register

MaxReg0

MaxReg0

…

switchswitch

switchswitch

Snapshot-based

counter

Snapshot-based

counter

switchswitchswitchswitch

MaxReg0

MaxReg0

MaxReg0

MaxReg0

WriteMax and ReadMax of vin O(min(log v, n)) steps

25

Lower bound of min(log m, n-1) Sm = executions with WriteMax

operations up to value m by p1…,pn-1, followed by one ReadMax operation by pn

T(m,n) = worst case cost of ReadMax in Sm

26

pn reads

Lower bound of min(log m, n-1) No process takes steps after pn so pn does not

write

Reads a fixed register R. Did anyone write to R? k = minimal such that there is a write to R in Sk

No one in Sk-1 writes to R so T(m,n)≥T(k-1,n)+1

27

pn reads

pn readsR

Lower bound of min(log m, n-1) In addition, consider a run in Sk that writes

to R

28

write to R by pi

Finish writes except by pi

Execution with writes in k,

…,m

write to R by pi

T(m,n) ≥ T(m-k+1,n-1)+1

pn returns maximal value from k,…,m

pn reads

Solve recurrence: T(m,n) ≥ 1+ mink max(T(k-1,n), T(m-k+1,n-1))

, we had T(m,n)≥T(k-1,n)+1

R

Rpn reads

Summary

Implementation of max registers with O(min(log v, n)) steps per operation writing or reading the value v

Sub-linear implementation of counters

Extension of counters to any monotone circuit with monotone consistency instead of linearizability

29

Summary

Lower bounds An alternative proof for JTT Tight lower bound for max registers Randomized lower bound

Further research: randomized algorithm?

Take-home message: Lower bounds do not always have the final

say

30