windows application development

5-1JMH Associates © 2004, All rights reserved

Windows Application DevelopmentWindows Application DevelopmentWindows Application DevelopmentWindows Application Development

Chapter 5

Threading Models for Reliability


OBJECTIVESOBJECTIVESOBJECTIVESOBJECTIVES

Upon completion of this Chapter, you will be able to: Solve more complex threading problems Use standard threading models in application design Combine mutexes and events reliably and usefully Build reliable, maintainable applications Understand and avoid many common pitfalls


ContentsContentsContentsContents

1. Multithreading Models

2. COM Model Comparison

3. Events and Mutexes Together

4. The Condition Variable Model

5. SignalObjectAndWait

6. Using the Condition Variable Model

7. Some Performance Tradeoffs

8. Avoiding Incorrect Code - Hints

10. Lab Exercise 5


1. Multithreading Models1. Multithreading Models1. Multithreading Models1. Multithreading Models

Advantages of using thread models: Expedite design and development Models are well understood and well tested

Avoid potential multithreading problems Models match structure of most programming problems

Very simple cases, plus three “classic” models Boss/worker Pipeline Client/server

Plus, combined models

“Programming in the large”


Boss/Worker ModelBoss/Worker ModelBoss/Worker ModelBoss/Worker Model

Program ResourcesInput(Stream)

Files

Databases

Workers

TaskN

Task2

Task1

main ()

Boss

...

Computations


Pipeline ModelPipeline ModelPipeline ModelPipeline Model

Q Q

Transmitter Receiver

Consumers

Q

Q

M1

M2

.

.

.M5

P1

Producers

PN

Message

TOFROM

DATA

Log

C1

CN


Client/Server ModelClient/Server ModelClient/Server ModelClient/Server Model

TLS

TLS

CodeShared DataConstants

Thread-ThreadCommunication

& Synchronization

Arg

Thread 1Specific Data

ServerThread 1

Stack 1

Client 1

Arg

ServerThread N Client N

Thread NSpecific Data

StatusBroadcastand Monitor

TLS

Stack

Arg

Statistics

Stack


CombinationsCombinationsCombinationsCombinations

Large systems rarely a pure instance of a single model Pipeline stages often implemented as boss/ worker

systems A server may perform client requests in a pipeline

sequence Always use good implementation documenting individual

modules in terms of these models Easier to maintain, understand and troubleshoot


2. COM2. COM2. COM2. COM

Microsoft’s Component Object Model Object-oriented technology Single threaded object

Only one thread can access it Apartment model threading

Unique thread assigned to each instance of an object Free threaded object

A thread from a thread pool is assigned, or created, when a request is made


COM – Useful TerminologyCOM – Useful TerminologyCOM – Useful TerminologyCOM – Useful Terminology

Thread pool Collection of available threads

Symmetric thread model A group of threads performs the same task using exactly

the same thread functions

Asymmetric thread model Different for separate thread functions

COM only Don’t confuse with “Classical Models”


3. Events and Mutexes Together3. Events and Mutexes Together3. Events and Mutexes Together3. Events and Mutexes Together

Example: Pipeline producer/consumer Producer creates a checksummed message

A mutex protects the message Producer signals the consumer that a message is ready

The producer waits on an event The mutex defines the critical code section for accessing

the message data structure object Assures the object’s “invariant” properties

The event is used to signal that there is a new message Signals that the object (message queue) is in a specified

state


Events and Mutexes TogetherEvents and Mutexes TogetherEvents and Mutexes TogetherEvents and Mutexes Together

One thread (producer) locks the data structure Changes the object’s state by creating a new message Sets or pulses the event – new message

One or more threads (consumers) wait on the event for the object to reach the desired state

The wait must occur outside the critical code section A consumer thread can also lock the mutex

And test the object’s state The state will not changer while being tested


4. The Condition Variable Model4. The Condition Variable Model4. The Condition Variable Model4. The Condition Variable Model

“Programming in the small”

Concepts are taken from Pthreads (Supplem’try Section)

Several key elements: Data structure of type STATE_TYPE

Contains all the data such as messages, checksums, etc. A mutex and one or more associated events

The mutex protects the data structure The events signal “interesting” data structure state changes

Boolean functions evaluate “condition variable predicates” For example, “a new message is ready” An event is associated with each condition variable predicate The event is signaled with the cvp becomes true


Condition Variable Model: DataCondition Variable Model: DataCondition Variable Model: DataCondition Variable Model: Data

Shared Data Structure:

typedef struct _state_t {

HANDLE Guard; /* Mutex to protect the object */

HANDLE cvpSet; /* Autoreset Event */

. . . other condition variables

/* State structure with counts, etc. */

struct STATE_VAR_TYPE StateVar;

} STATE_TYPE State;

. . .

/* Initialize State, creating mutex & event(s) */

. . .


Condition Variable Model: ProducerCondition Variable Model: ProducerCondition Variable Model: ProducerCondition Variable Model: Producer

/* PRODUCER thread that modifies State */

WaitForSingleObject (State.Guard, INFINITE);

/* Change state so that the CV predicate holds */

. . . State.StateVar.xyz = . . . ;

SetEvent (State.cvpSet);

ReleaseMutex (State.Guard);

/* End of the interesting part of the producer */

. . .


Condition Variable Model: ConsumerCondition Variable Model: ConsumerCondition Variable Model: ConsumerCondition Variable Model: Consumer

/* CONSUMER thread waits for a particular state */


while (!cvp(&State)) {


WaitForSingleObject (State.CvpSet, TimeOut);


}

. . .


/* End of the interesting part of the consumer */


Condition Variable Model CommentsCondition Variable Model CommentsCondition Variable Model CommentsCondition Variable Model Comments

You need to repeat the loop and test The timeout is finite – a tunable parameter

CVM avoids missed signals and other problems

This is the “broadcast” version Multiple consumers released by one producer SetEvent() Subsequent behavior depends on cvp() and

produce/consume semantics Uses an AR event and SetEvent()

“Signal” version uses MR event and PulseEvent() A single consumer is released


5. SignalObjectAndWait5. SignalObjectAndWait5. SignalObjectAndWait5. SignalObjectAndWait

Three essential steps of the loop in the consumer Unlock the mutex Wait on the event Lock the mutex again

The first two are not atomic The producer can signal before the consumer waits Risk: A missed signal – hence the timeout

SignalObjectAndWait() combines the first two steps One atomic function Pthreads combines all three steps >= NT 4.0 (not on Win 9x, Me)


SignalObjectAndWaitSignalObjectAndWaitSignalObjectAndWaitSignalObjectAndWait

DWORD SignalObjectAndWait(

HANDLE hObjectToSignal, // Mutex

HANDLE hObjectToWaitOn, // Event

DWORD dwMilliseconds, // time-out in ms

BOOL bAlertable // Use FALSE – until S7;

SOAW is general purpose Just the mutex-event usage is shown

Timeout can usually be infinite

Be sure to:

#define _WIN32_WINNT 0x400 // WINBASE.H


6. Using the CV Model6. Using the CV Model6. Using the CV Model6. Using the CV Model

Producer locks the mutex, changes state, sets the event and unlocks the mutex

Event should be set with the mutex locked Consumer tests the CV predicate with the mutex locked

If the predicate does not hold, the consumer must unlock the mutex before waiting on the event

Event wait must have a timeout – to avoid missed signals Unless you use SOAW!

Consumer always retests the predicate after the event wait Consumer always owns the mutex when it leaves the loop

Whether loop body was executed or not


CV Model VariationCV Model VariationCV Model VariationCV Model Variation

In producer/consumer code, multiple threads released Auto-reset event SetEvent

Or, there may be only one message available and multiple consuming threads

Event should be manual-reset Producer should call PulseEvent

Assure exactly one thread is released


When Interested in the When Interested in the NextNext Event EventWhen Interested in the When Interested in the NextNext Event Event

/* CONSUMER thread waits for NEXT state change */WaitForSingleObject (State.Guard, INFINITE);do { SignalObjectAndWait (State.Guard, State.cvpSet, TimeOut, FALSE); WaitForSingleObject (State.Guard, INFINITE);

} while (!cvp(&State));/* Thread now owns the mutex and cvp(&State) holds *//* Take appropriate action, perhaps modifying State */ . . .ReleaseMutex (State.Guard);/* End of the interesting part of the consumer */


7. Some Performance Tradeoffs7. Some Performance Tradeoffs7. Some Performance Tradeoffs7. Some Performance Tradeoffs


8. Avoiding Incorrect Code - Hints8. Avoiding Incorrect Code - Hints8. Avoiding Incorrect Code - Hints8. Avoiding Incorrect Code - Hints

Pay attention to design, implementation, and use of familiar programming models

Best debugging technique: Don’t create bugs in the first place

Many serious defects will elude the most extensive and expensive testing

Debuggers change timing behavior Masking the race conditions you wish to expose

Most of these hints are from Programming with POSIX Threads by David Butenhof, Addison-Wesley, 1997.


Avoiding Incorrect Code - HintsAvoiding Incorrect Code - HintsAvoiding Incorrect Code - HintsAvoiding Incorrect Code - Hints

Avoid relying on “thread inertia” Never bet on a thread race Scheduling is not the same as synchronization Sequence races can occur

Even when you use mutexes to protect shared data Cooperate to avoid deadlocks Never share events between predicates


Avoiding Incorrect Code - HintsAvoiding Incorrect Code - HintsAvoiding Incorrect Code - HintsAvoiding Incorrect Code - Hints

Beware of sharing stacks and related memory corrupters Beware: No warning of thread stack overflow! Be sure to use the volatile storage modifier Use the condition variable model properly Understand your invariants and condition variable

predicates Keep it simple Test on multiple systems (single and multiprocessor) Testing is necessary but not sufficient Be humble Be prepared for unpleasant surprises


Thread Blocking NotesThread Blocking Notes

At any point any thread may go to sleep for an unbounded period of time

No ordering exists between threads unless you cause ordering

Thread scheduling behavior can vary widely Various operating system implementations Different models of the same system Systems with different processor speeds

Explicit synchronization is required

There is no timeout from a pthread_join() call


Word TearingWord TearingWord TearingWord Tearing

A single mutex should be used in every critical section associated with a shared resource

a.k.a. “fighting over cache lines” - SMP Example: A processor may access quadword memory units

64 bits, since a word is 16 bits 16-bit words are not separate resources When different threads access distinct values that lie in the

same quadword, we have word tearing The entire quadword should be protected with one mutex

Demonstration program included in exercise files Ask the instructor for more information if interested!


Word Tearing (2 of 2)Word Tearing (2 of 2)Word Tearing (2 of 2)Word Tearing (2 of 2)Memory

4 16- Bit Words

Thread B Value Wins;Next Time; Thread A Might Win

Thread A Thread B

44332251

44332211

44332211

48332211

44332211

48332211

Writes to First Byte

Terminates

Writes to Last Byte

Terminates

Read

Read


Deadlock Avoidance, Lock HierarchiesDeadlock Avoidance, Lock HierarchiesDeadlock Avoidance, Lock HierarchiesDeadlock Avoidance, Lock Hierarchies

Avoid deadlocks by using a lock hierarchy whereby mutexes are always locked in the same sequence

Release locks in the reverse sequence

Additional deadlock notes During development and troubleshooting

Assure that multiple locks are always acquired in the same order

Document the lock hierarchy Lock hierarchies are provably deadlock free


Lock (&ListB.guard)

No Deadlocks with Hierarchical LocksNo Deadlocks with Hierarchical LocksNo Deadlocks with Hierarchical LocksNo Deadlocks with Hierarchical LocksAddSharedElement

ThreadDeleteSharedElement

Thread

RunningReady

OSSCHEDULER

Lock (&ListA.guard)Preempt

Ready

Run

Running

Unock (&ListB.guard)

Unlock (&ListA.guard)

Preempt

Ready

Running

Blocked

Run

Lock (&ListA.guard)Blocked

UnblockReady

Run

Lock (&ListB.guard)

Unock (&ListB.guard)

Unlock (&ListA.guard)

Running


9. Lab Exercise 10-19. Lab Exercise 10-19. Lab Exercise 10-19. Lab Exercise 10-1

ThreeStage.c implements the illustrated pipeline model See the Pipeline slide It was used to generate the performance results It is implemented with SignalObjectAndWait

Fix QueueObjX.c, and/or

Determine performance of different implementations on your system(s)

windows application development

Documents