![Page 1: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/1.jpg)
E81 CSE 532S: Advanced Multi-Paradigm Software Development
Chris Gill and Venkita SubramonianDepartment of Computer Science and Engineering
Washington University, St. [email protected]
A Concurrency and Synchronization Pattern Language
![Page 2: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/2.jpg)
Main Themes of this Talk
• Review of concurrency and synchronization
• Review of patterns and pattern languages
• Design scenario case study– A peer-to-peer client-server application– Increasing liveness with concurrency
patterns– Maintaining safety with synchronization
patterns
![Page 3: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/3.jpg)
Concurrency• Forms of concurrency
– Logical (single processor) parallelism– Physical (multi-processor) parallelism
• Goal: Increase Liveness– Progress is made: deadlock is avoided
• Also, avoid long blocking intervals if possible
– Progress is also optimized where possible• Independent activities logically parallel
– Ideally, physically parallel if you can afford additional resources
• Full utilization of resources: something is always running
– E.g., both I/O bound and compute bound threads in a process
• The most eligible task is always running– E.g., the one with the highest priority among those enabled
![Page 4: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/4.jpg)
Concurrency, Continued• Benefits
– Performance• Still make progress if one thread blocks (e.g., for I/O)
– Preemption• Higher priority threads preempt lower-priority ones
• Drawbacks– Object state corruption due to race conditions– Resource contention
• Need isolation of inter-dependent operations– For concurrency, synchronization patterns do this
• At a cost of reducing concurrency somewhat• And at a greater risk of deadlock
![Page 5: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/5.jpg)
Synchronization• Forms of synchronization
– Thread (lightweight) vs. process (heavy) locks• Why is process locking more expensive?
– Binary (mutex) vs. counted (semaphore) locks– What is being locked
• Scopes: scoped locking• Objects: thread-safe interface • Role-based: readers-writer locks• “It depends”: strategized locking
• Goal: Safety– Threads do not corrupt objects or resources– More generally, bad inter-leavings are avoided
• Atomic: runs to completion without being preempted• Granularity at which operations are atomic matters
![Page 6: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/6.jpg)
Synchronization, Continued• Benefits
– Safety • Prevent threads from interfering in bad ways
– Control over (and reproducibility of) concurrency• Restrict the set of possible inter-leavings of threads
• Drawbacks– Reduces concurrency (extreme form: deadlock)– Adds overhead, increases complexity of code
• Need to apply locking carefully, with a purpose– Only lock where necessary
• Based on design forces of the specific application• Allow locks to be changed, or even removed
![Page 7: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/7.jpg)
Concurrency & Synchronization• Concurrency hazard: race conditions
– Two or more threads access an object/resource– The interleaving of their statements matters– Some inter-leavings have bad consequences
• Synchronization hazard: deadlock– One or more threads access an object/resource– Access to the resource is serialized– Chain of accesses leads to mutual blocking
• Clearly, there is a design tension– Must be resolved for each distinct application– Applying patterns helps if guided by design forces
![Page 8: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/8.jpg)
Review: What is a Pattern Language?
• A narrative that composes patterns– Not just a catalog or listing of the patterns– Reconciles design forces between patterns– Provides an outline for design steps
• A generator for a complete design–Patterns may leave consequences–Other patterns can resolve them–Generative designs resolve all forces
•Internal tensions don’t “pull design apart”
![Page 9: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/9.jpg)
Towards a Pattern Language• Key insights
– Each pattern solves a particular set of issues– When applied, it may leave others open
• Depends on the scale of the context• E.g., ACT does not itself address synchronization
– Although it may help in combination with say the TSS pattern
– It may also raise new issues• For example, synchronization raises deadlock issues
• None of this matters without a context– Patterns are only useful when applied to a
design– A design stems from the context
(problem+forces) of something we’re trying to build
• So, let’s imagine we’re building something
![Page 10: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/10.jpg)
Motivating Scenario• Peer-to-peer client server applications
– Each process sends and receives requests– Communicates with many other endsystems at once
• Distributed data and analysis components – Spread more or less evenly throughout network– Each has a particular function or kind of information– Endsystems can look up locations of specific
components• Directory, naming, and trading services are available
• Heterogeneous requests– Long-running independent file retrieval requests
• “Send me the news data feed from 04/21/03”• “Send me the stock market ticker for 04/21/03”
– Short-duration interdependent analysis requests• “Compute the change (+ or -) for each stock sector”• “Then find the largest change in a stock sector”• “Then find 5 news events with highest correlation to that
change”
![Page 11: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/11.jpg)
Motivating Scenario, Continued• Given
– A high-quality LAN-scale analysis server …– designed to manage tens of components …– and interact with tens of other similar servers …– all connected via one company’s private network
• Assume – Written in C++, leverages C++11/STL features
extensively– Design uses Wrapper Façade, other basic patterns– Reactive implementation (initially single-threaded)
• We’ll discuss other alternatives in the next part of the course• E.g., proactive approaches using interceptor to mark/notify
progress
• Goal– Scale this to a business-to-business VPN environment …– … with hundreds instead of tens of nodes locally– … each with hundreds instead of tens of collaborators
![Page 12: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/12.jpg)
Review of Concurrency Patterns
• Active Object– Methods hand off request to internal worker thread
• Monitor Object– Threads allowed to access object state one-at-a-time
• Half-Sync/Half-Async– Process asynchronously arriving work in synchronous
thread
• Leader/Followers– HS/HA optimization: leader delegates until its work
arrives
• Thread Specific Storage – Separate storage for each thread, to avoid contention
• These complement the synchronization patterns
![Page 13: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/13.jpg)
Review of Synchronization Patterns
• Scoped Locking– Ensures a lock is acquired/released in a scope
• Strategized Locking– Customize locks for safety, liveness, optimization
• Thread-Safe Interface– Reduce internal locking overhead– Avoid self-deadlock
• These complement the concurrency patterns
![Page 14: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/14.jpg)
How Do We Get Started?• Simply apply patterns in order listed?
– Sounds ad hoc, no reason to think this would work
• A better idea is to let the context guide us– Compare current design forces to a pattern
context
• Let’s start with the most obvious issue– Long-running tasks block other tasks– Long-running tasks need to be made concurrent
• Leads to selection of our first pattern– Concurrent handlers for long-running requests– I.e., apply the Active Object pattern
![Page 15: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/15.jpg)
Active Object Context• Pattern’s use in this design
– Worker thread in handler– Input thread deposits requests
for it– Queue decouples
request/execution between threads
• LRRH::handle_event– An enqueue method for AO
that puts a command object into the queue
– Allows AOs to be registered with an asynchronously triggered dispatcher, e.g., an input (or reactor) thread
Long RunningRequest Handlers (LRRH)
enqueuerequests worker
thread
input thread
![Page 16: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/16.jpg)
Active Object Consequences• Clear benefits for LRRH
– Method invocation & execution are decoupled
– Helps prevent deadlock– Reduces blocking
• Drawbacks– Queuing overhead– Context-switch overhead– Implementation complexity
• Costs well amortized so far– But only for LRRH– So, we can’t apply as is to
short-running requests
• Next design refinement– Apply Half-Sync/Half-Async
Long RunningRequest Handlers (LRRH)
enqueuerequests worker
thread
input thread
![Page 17: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/17.jpg)
Half-Sync/Half-Async Context • Still want long-running activities
– Less overhead when related data are processed synchronously
• Also want short-lived enqueue and dequeue operations– Requests arrive asynchronously– Keep reactor and handlers responsive
• How to get both?– Make queueing layer smarter– Chain requests by their dependencies– E.g., HS/HA (via pipes & filters)
• How this is done is crucial– Blocking input (reactor) thread is bad– So, worker thread chains them before
it works on a chain!
Mixed Duration Request Handlers (MDRH)
enqueuerequests worker
thread
input thread
worker chains related requests
chainrequests
![Page 18: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/18.jpg)
HS/HA Consequences, part I• Problem: race condition
– Reactor thread enqueues while active object thread dequeues
• BTW, this has been an issue– Since we applied Active Object– But we pushed first on concurrency– Could have taken the other branch
in our design process first, instead– Emphasizes iterative/rapid design!
• Need a mutex – Serialize access to the queue data
structure itself
• But then need to avoid deadlock– What if an exception is thrown?
(E.g., a call to new fails)
• Solution: apply scoped locking
enqueuerequests worker
thread
input thread
worker chains related requests
Mixed Duration Request Handlers (MDRH)
![Page 19: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/19.jpg)
Scoped Locking Context
• Define/identify scopes around critical sections– I.e., the active object enqueue and dequeue methods
• Provide a local guard object on the stack– as the first element within the scope of a critical
section
• Guard ensures a thread automatically acquires the lock – Guard constructor locks upon entering a critical section
• Guard ensures thread automatically releases the lock – Guard destructor unlocks upon exiting the local scope
• Guarded locks will thus always be released– even if exceptions are thrown from within the critical
section
![Page 20: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/20.jpg)
Scoped Locking Consequences
• Benefit: increased robustness with exceptions
• Liability: deadlock potential when used recursively– Self deadlock could occur if
the lock is not recursive– E.g., if worker thread
generates requests for itself
• Pattern language solution– Apply another pattern!– I.e., strategized locking
![Page 21: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/21.jpg)
Strategized Locking Context & Consequences
• Parameterizes synchronization mechanisms – That protect critical sections from concurrent access
• Different handlers may need different kinds of synchronization– Null locks (i.e., if we’d ever add passive handlers)– Recursive and non-recursive forms of mutex locks– Readers/writer locks
• Decouple handlers and locks– Can change one without changing the other– Enhancements, bug fixes should be straight forward
• Flexibility vs. complexity, time vs. space
![Page 22: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/22.jpg)
HSHA Consequences, part II• Pop back up to HS/HA
– Now that we’ve addressed queue locking issues
• A new concurrency issue– Chains that are ready, may
wait– While thread works on others
• Could increase # of MDRH’s– Also increase context
switches
• A better idea– Notice the chaining of related
requests?– Looks like “sorting the mail”– I.e., apply Leader/Followers
enqueuerequests worker
thread
input thread
worker chains related requests
Mixed Duration Request Handlers (MDRH)
![Page 23: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/23.jpg)
• Leader when elected picks an incomplete chain
• Appends requests from reactor to growing chains
• When a chain is complete– If its for the leader
• Leader takes request chain• A new leader is elected• Old leader becomes follower• Old leader works on the chain
– If it’s for someone else• Leader hands off to others• Continues to build chains
Leader/Followers Context
enqueuerequests leader
thread
input thread
handoff
chains
followerthreads
Mixed Duration Request Handlers (MDRH)
![Page 24: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/24.jpg)
• Key benefits– Greater cache affinity: thread locality of reference– Context switch only at some events
• Liabilities– Some increase in implementation complexity
• I.e., to elect leaders, identify whose chain is whose• We’ll just live with this for the purposes here
– Some additional overhead • But we already separated reactor from handlers• So impact on lowest asynchronous level is reduced• We’ll go a couple steps further in reducing this
• Overall, benefits of applying LF here outweigh costs
Leader/Followers Consequences
![Page 25: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/25.jpg)
TSS Context & Consequences• Must lock thread hand-off
– But access w/in a chain is single-threaded and thus ok
• Can save results locally– Put request result in TSS– Retrieve as input for next
request in that chain
• Benefits– Avoids locking overhead– Increases concurrency
• Liabilities– Slightly more complex to
store and fetch results– Portability of TSS (may
need to emulate in code)
enqueuerequests leader
thread
input thread
handoff
chains
followerthreads
Mixed Duration Request Handlers (MDRH)
![Page 26: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/26.jpg)
A Few Remaining Issues
• Shared services across threads (naming, lookup)– Threads need safe
access
• Design trade-off– Between simplicity and
potential concurrency improvement
• Alternative patterns– Monitor Object
• E.g., for new services
– Thread Safe Interface• E.g., for legacy services
enqueuerequests leader
thread
input thread
handoff
chains
followerthreads
Mixed Duration Request Handlers (MDRH)
![Page 27: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/27.jpg)
Monitor Object Context & Consequences
• Threads use common services– E.g., Naming, Directory– Need thread-safe service access
• Single copy of each service– Inefficient to duplicate services– Threads access them sporadically
• Could just make services active– More complex if even possible
• Solution: Monitor Object– Synchronize access to the service
• Consequences– If worried about performance, can
yield on condition variables– Assume that’s not an issue here
(reasonable for Naming, etc.)
followerthreads
commonservice
![Page 28: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/28.jpg)
Thread-Safe Interface Context and Consequences
• Since synchronous service performance is fine– Go for simplicity, low-overhead – Avoids self-deadlock from intra-
component method calls– Avoid complexity of condition
based thread yield– Minimizes locking overhead
• Separates Concerns – Interface methods deal with
synchronization– Implementation methods deal
with functionality
• Only option with some single-threaded legacy services– Can’t modify, so wrap instead
followerthreads
serviceimplementation
thread-safeservice
interface
![Page 29: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/29.jpg)
Pattern Language Review
• We first applied Active Object– Made long-running handlers concurrent
• Then we applied Half-Sync/Half-Async– For chains of short-running requests
• Next were scoped & strategized locking– To make thread hand-off safe, efficient
• Then we applied Leader/Followers– To reduce thread context switches
![Page 30: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/30.jpg)
Pattern Language Review, Continued
• We applied TSS next– Efficient access to separate
information for each thread can help reduce contention
• Finally we applied Monitor Object and/or Thread-Safe Interface– For synchronized access to common
(possibly legacy) services
![Page 31: E81 CSE 532S: Advanced Multi-Paradigm Software Development Chris Gill and Venkita Subramonian Department of Computer Science and Engineering Washington](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ed15503460f94be0c21/html5/thumbnails/31.jpg)
Concluding Remarks• Design mastery is often not based on novelty
– Though occasionally it can help– It is mostly a matter of careful observation
• of the current context as the design unfolds …• and how consequences of each decision shape later ones
• Study patterns and pattern languages– Benefit from prior experience with patterns
• Including published experience of other “design masters”– Practice working through design examples like this– Build intuition about how patterns fit together
• Mastery is a journey not a destination– Learn from every design, yours and others