cafÉ: scalable task pool with adjustable fairness and contention dmitry basin, rui fan, idit...
TRANSCRIPT
![Page 1: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/1.jpg)
CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention
Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman
Technion, Israel Institute of Technology
![Page 2: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/2.jpg)
Task Pools
Exist in most server applications:Web Servers, e.g., building block of SEDA ArchitectureHandling asynchronous requests
Ubiquitous programming pattern for parallel programs
Scalability is essential!
Task PoolProducer
sConsumers
Shared Memory
![Page 3: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/3.jpg)
Typical Implementation: FIFO Queue
Has inherent scalability problem
Do we really need FIFO ?In many cases no!We would like to:
Relax the requirementControl the degree of relaxation
contention pointsShared Memory
![Page 4: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/4.jpg)
CAFÉ: Contention and Fairness ExplorerOrdered list of scalable bounded non-FIFO pools
TreeContainer size controls contention-fairness trade-off
garbage collected
TreeContainer TreeContainer TreeContainerJava VM
Less fairnessLess contention
More fairnessMore contention
Tree height
0
Pure FIFO
![Page 5: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/5.jpg)
TreeContainer (TC) SpecificationBounded containerA put operation can fail if no free space
foundA get operation returns a task or null if TC
is emptyRandomized algorithms for put and get
operations
![Page 6: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/6.jpg)
TreeContainer Data StructureComplete binary
tree
Free node
Used node without a
task
Occupied node containing a
task
Right sub-tree doesn’t
contain tasks
Left sub-tree has tasks
![Page 7: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/7.jpg)
Get/Put OperationsStep 1: find target node
By navigating the treeStep 2: perform put/get on that node
Need to handle racesStep 3: update routes
Update meta-data on path to node – tricky!
![Page 8: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/8.jpg)
Task
Get() OperationGet(): Start from the root
TreeContainer
CAS
Close to the root updates
are rare
Step3: Update routes up to
the root
Step1: Navigate to a
task by random walk on arrows
graph
Step 2: Extract the task.
![Page 9: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/9.jpg)
Put() Operation
Level 0
Level 1
Level 2
Level 3
TreeContainer
Random node
occupied
Random node
occupied
Random node
√ free
Taskput():
Every level of the tree implemented by array of nodes
occupied
Random node
![Page 10: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/10.jpg)
Put() OperationTaskput():
Go to the highest free predecessor
CA
S op
eration
TreeContainer
Level 0
Level 1
Level 2
Level 3Random node
√ free
Finished Step 1: found free node
Step 2: occupy the free node
![Page 11: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/11.jpg)
Put() Operationput(): Close to the root, updates
are raretrue
TreeContainer
Upd
ate
rout
es
![Page 12: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/12.jpg)
RacesConcurrency issues are not trivial :)Challenge:
guarantee linearizabilityavoid updating all the metadata up to the
root upon each operationSee the paper
![Page 13: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/13.jpg)
TreeContainer propertiesPut/Get operations are
linearizablewait-free
Under the worst-case thread scheduling:Good step complexity of puts
When N nodes occupied - O(log2N) whpDoes not depend on TC size
Good tree density (arbitrarily close to 2h whp)
TreeContainer
![Page 14: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/14.jpg)
CAFÉ Data Structures
TC TC
GT
TC
PT
![Page 15: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/15.jpg)
CAFÉ Data Structures
TC TC TC
PT
TC
TC.Put(task) false
Allocate and connect new TC
TC.Put(task) true
GT
![Page 16: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/16.jpg)
TC
CAFÉ:get() from Empty TC
TC TC TC
PT
TC
TC.Getnull TC.Gettask
Garbage collected
GT
![Page 17: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/17.jpg)
CAFÉ: Races
TC TC TC
PT
TC
Suspended producer thread
The task is lost for consumers
GT
![Page 18: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/18.jpg)
CAFÉ: Handling Races – Try 1
TC TC TC
PT
TC
Move GT back
Check if GT bypassed TC GT
![Page 19: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/19.jpg)
CAFÉ: Races (2)
TC TC TC
PT
TC
TC.Get null
Consumer thread
Producer thread
Going to
move GT
forward
TC.Put(task) true
Consumers can access the task
can terminateTask is lost
GT
![Page 20: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/20.jpg)
CAFÉ: Handling Races – Try 2
TC TC TC
PT
TC
Read prev,If empty read curr
GT
cur>
<prev
Lock-Free. To make it wait-free we do additional tricks.
![Page 21: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/21.jpg)
CAFÉ: PropertiesSafety:
Put()/Get() operations are linearizableWait-freedom:
Get() operations are deterministically wait-free
Put() operations are wait-free with probability 1
Fairness:Preserves order among trees
![Page 22: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/22.jpg)
Evaluation SetupCompared pools:
LBQ: Java 6 FIFO blocking queueCLQ: Java 6 FIFO non-blocking queue (M&S)EDQ: non-FIFO Elimination-Diffraction Tree
Queue
Evaluation server:8 AMD Opteron quad-cores total 32 cores
![Page 23: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/23.jpg)
CAFÉ evaluationThroughput
CAFÉ-13: CAFÉ with tree height 13
LBQ: Java 6 FIFO blocking queue CLQ: Java 6 FIFO non-blocking
queue (M&S) EDQ: non-FIFO Elimination-
Diffraction Tree Queue Throughput as a function of thread number
factor of 30over lock-free implementations
![Page 24: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/24.jpg)
CAFÉ evaluationThroughput
CAFÉ: CAFÉ queue LBQ: Java 6 FIFO blocking queue CLQ: Java 6 FIFO non-blocking
queue (M&S) EDQ: non-FIFO Elimination-
Diffraction Tree Queue .
CAFÉ throughput as a function of TreeContainer height
![Page 25: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/25.jpg)
CAFÉ evaluationCAS-failures CAS failures per operation as a function of TreeContainer height
![Page 26: CAFÉ: Scalable Task Pool with Adjustable Fairness and Contention Dmitry Basin, Rui Fan, Idit Keidar, Ofer Kiselov, Dmitri Perelman Technion, Israel Institute](https://reader030.vdocuments.us/reader030/viewer/2022032516/56649c4d5503460f948f1f67/html5/thumbnails/26.jpg)
SummaryCAFÉ:
EfficientWait-FreeWith adjustable fairness and contention
Thank you