CIS 620 Advanced Operating Systems

Lecture 9 – Synchronization

Prof. Timothy Arndt

BU 331

Synchronization

• We will be looking at synchronization in two kinds of systems:– Shared memory systems

• Mutual exclusion• Semaphores

– Message passing systems• Time in distributed systems• Mutual exclusion• Election in distributed systems

Shared Memory Coordination

• We will be looking at process coordination using shared memory and busy waiting.– So we don't send messages but read and write

shared variables.– When we need to wait, we loop and don't

context switch.– This can be wasteful of resources if we must

wait a long time.

Shared Memory Coordination

– Context switching primitives normally use busy waiting in their implementation.

• Mutual Exclusion– Consider adding one to a shared variable V.– When compiled onto many machines get three

instructions• load r1 V• add r1 r1+1• store r1 V

Mutual Exclusion

– Assume V is initially 10 and one process begins the 3 instruction sequence.• After the first instruction context switch to another

process.– Registers are of course saved.

• New process does all three instructions.• Context switch back.

– Registers are of course restored.• First process finishes.• V has been incremented twice but has only reached

11.

Mutual Exclusion

– The problem is that the 3 instruction sequence must be atomic, i.e. cannot be interleaved with another execution of these instructions

– That is, one execution excludes the possibility of another. So they must exclude each other, i.e. we must have mutual exclusion.

– This was a race condition.• Hard bugs to find since non-deterministic.

– Can in general involve more than two processes

Mutual Exclusion

– The portion of code that requires mutual exclusion is often called a critical section.

• One approach is to prevent context switching.– We can do this for the kernel of a uniprocessor.

• Mask interrupts.

– Not feasible for user mode processes.– Not feasible for multiprocessors.

Mutual Exclusion

– Critical Section Problem is to implement:looptrying-partcritical-sectionreleasing-partnon-critical section

end loop

– So that when many processes execute this you never have more than one in the critical section.

– That is you must write the trying-part and the releasing-part.

Mutual Exclusion

– Trivial solution.• Let releasing-part be simply "halt”.

– This shows we need to specify the problem better.

– Additional requirement:• Assume that if a process begins execution of its

critical section and no other process enters the critical section, then the first process will eventually exit the critical section.

Mutual Exclusion

• Then the requirement is "If a process is executing its trying part, then some process will eventually enter the critical section".

• Software-only solutions to CS problem.– We assume the existence of atomic loads and

stores.• Only up to word length (i.e. not a whole page).

– We start with the case of two processes.– Easy if we want tasks to alternate in CS and

you know which one goes first in CS.

Mutual Exclusion

shared int turn = 1

loop loop

while (turn=2) while (turn=1)

CS CS

turn=2 turn=1

NCS NCS

Mutual Exclusion

– But always alternating does not satisfy the additional requirement above.

– Let NCS for process 1 be an infinite loop (or a halt). • We will get to a point when process 2 is in its trying

part but turn=1 and turn will not change. • So some process enters its trying part but neither

process will enter the CS.

Mutual Exclusion

• The first solution that worked was discovered by a mathematician named Dekker. – Now we will use turn only to resolve disputes.

Dekker’s Algorithm

/* Variables are global and shared */for (; ;) { // process 1 - an infinite loop to show it enters

// CS more than once. Turn is initially 1.p1wants = 1;while (p2wants == 1) {

if (turn == 2) {p1wants = 0;while (turn == 2) {/* Empty loop */}p1wants = 1;

}}critical_section();turn = 2;p1wants = 0;noncritical_section();

}

Dekker’s Algorithm

/* Variables are global and shared */for (; ;) { // process 2 - an infinite loop to show it enters

// CS more than once. Turn is initially 1.p2wants = 1;while (p1wants == 1) {

if (turn == 1) {p2wants = 0;while (turn == 1) {/* Empty loop */}p2wants = 1;

}}critical_section();turn = 1;p2wants = 0;noncritical_section();

}

Mutual Exclusion

– The winner-to-be just loops waiting for the loser to give up and then goes into the CS.

– The loser-to-be:• Gives up.• Waits to see that the winner has finished.• Starts over (knowing it will win).

– Dijkstra extended Dekker's solution for > 2 processes.

– Others improved the fairness of Dijkstra's algorithm.

Mutual Exclusion

– These complicated methods remained the simplest known until 1981 when Peterson found a much simpler method.

– Keep Dekker's idea of using turn only to resolve disputes, but drop the complicated then body of the if.

Peterson’s Algorithm

/* Variables are global and shared */for (; ;) { // process 1 - an infinite loop to show it enters

// CS more than once.p1wants = 1;turn = 2;while (p2wants && turn == 2) {/* empty loop */}critical_section();p1wants = 0;noncritical_section();

}

Peterson’s Algorithm

/* Variables are global and shared */for (; ;) { // process 2 - an infinite loop to show it enters

// CS more than once.p2wants = 1;turn = 1;while (p1wants && turn == 1) {/* empty loop */}critical_section();p2wants = 0;noncritical_section();

}

Semaphores

– Trying and release often called entry and exit, or wait and signal, or down and up, or P and V (the latter are from Dutch words since Dijkstra is Dutch).

– Let’s try to formalize the entry and exit parts.– To get mutual exclusion we need to ensure that

no more than one task can pass through P until a V has occurred. The idea is to keep trying to walk through the gate and when you succeed atomically close the gate behind you so that no one else can enter.

Semaphores

– Definition (not an implementation):• Let S be an enumerated type with values closed and

open (like a gate).

P(S) is

while S = closed

S closed• The failed test and the assignment are a single

atomic action.

Semaphores

P(S) is label: {[ --begin atomic part if S = open S closed else }] --end atomic part goto label

V(S) is S open

Semaphores

– Note that this P and V (not yet implemented) can be used to solve the critical section problem very easily.• The entry part is P(S).• The exit part is V(S).

– Note that Dekker and Peterson do not give us a P and V since each process has a unique entry and a unique exit.

– S is called a (binary) semaphore.

Semaphores

– To implement binary semaphores we need some help from our hardware friends.

Boolean in out X

TestAndSet(X) is

oldx X X true

return oldx

– Note that the name is a good one. This function tests the value of X and sets it (i.e. sets it true; reset is to set false).

Semaphores

– Now P/V for binary semaphores is trivial.• S is Boolean variable (false is open, true is closed).

P(S) is

while (TestAndSet(S))

V(S) is

S false

– This works fine no matter how many processes are involved.

Counting Semaphores

– Now want to consider permitting a bounded number of processors into what might be called a semi-critical section.

loop P(S) SCS // at most k processes can be here

// simultaneously V(S) NCS

– A semaphore S with this property is called a counting semaphore.

Counting Semaphores

– If k=1, we get a binary semaphore so counting semaphore generalizes to binary semaphore.

– How can we implement a counting semaphore given binary semaphores?• S is a nonnegative integer.• Initialize S to k, the max number allowed in SCS.• Use k=1 to get binary semaphore (hence the name

binary).• We only ask for:

– Limit of k in SCS (analogue of mutual exclusion).– Progress: If process enters P and < k in SCS, a process

will enter the SCS.

Counting Semaphores

• We do not ask for fairness, and don't assume it (for the binary semaphore) either.

binary semaphore q // initially open

binary semaphore r // initially closed

integer NS; // might be negative, keeps value of S

P(S) is V(S) isP(q) P(q)

NS-- NS++if NS < 0 if NS <= 0

V(q) V(r)P(r)

else

V(q) V(q)

Distributed System Coordination

– No shared memory so:• No semaphores• No test-and-set• No fetch-and-add

– Basic "rules of the game"• The relevant information is scattered around the

system• Each process makes decisions based on local

information• Try to avoid a single point of failure• No common clock supplied by hardware

Distributed System Coordination

– Our first goal is to deal with the last rule.• Some common time is needed (e.g. make).• Hardware doesn't supply it.• So software must.

– For isolated uniprocessor all that really matters is that the clock advances monotonically (never goes backward).• This says that if one time is greater than another the

second happened later than the first.

Distributed System Coordination

– For isolated multiple processor system need the clocks to agree so if one processor marks one file as later than a second processor marks a second file, the second marking really did come after the first.

– Processor has a timer that interrupts periodically. The interrupt is called a clock tick.

– Software keeps a count of # of ticks since a known time.

Distributed System Coordination

• Initialized at boot time– Sadly the hardware used to generate clock ticks

isn't perfect and some run slightly faster than others. This causes clock skew.

– Clocks that agree with each other (i.e. are consistent, see below) are called logical clocks.

– If the system must interact with the real world (say a human), it is important that the system time is at least close to the real time.• Clocks that agree with real time are called physical

clocks.

Logical Clocks

– What does it mean for clocks to agree with each other?

– For logical clocks we only care that if one event happens before another, the first occurs at an earlier time on the local clock.

– We say a happens before b, written ab if one of the following holds:• Events a and b are in the same process, and a occurs

before b.• Event a is the sending of message M and b is the

receiving of M.

Logical Clocks

• Transitivity, i.e. a c and c b.

– We say a happens after b if b happens before a.

– We say a and b are concurrent if neither a b nor b a.

– We require for logical clocks that a b implies C(a) < C(b), where C(x) is the time at which x occurs. That is C(x) is the value of the local clock when event x occurs.

Logical Clocks

– How do we ensure this rule (a b implies C(a) < C(b))? We start by using the local clock value, but need some fixes.

– Condition 1 is almost satisfied. If a process stays on a processor, the clock value will never decrease. So we need two fixes.• If a process moves from one processor to another,

must save the clock value on the first and make sure that the second is no earlier (indeed it should probably be at lease one tick later).

Logical Clocks

• If two events occur rapidly (say two message sends), make sure that there is at lease one clock tick between.

– Condition 2 is not satisfied. The clocks on different processors can definitely differ a little so a message might arrive before it was sent.

– The fix is to record in the message the time of sending and when it arrives, set the local clock to max(local time, message send time + 1)

– The above is Lamport's algorithm for synchronizing logical clocks.

Example: Totally Ordered Multicasting

• Updating a replicated database and leaving it in an inconsistent state.

Vector Clocks

• Concurrent message transmission using logical clocks.

Vector Clocks

• Vector clocks are constructed by letting each process Pi maintain a vector VCi with the following two properties:

1. VCi [ i ] is the number of events that have occurred so far at Pi. In other words, VCi [ i ] is the local logical clock at process Pi .

2. If VCi [ j ] = k then Pi knows that k events have occurred at Pj. It is thus Pi’s knowledge of the local time at Pj .

Vector Clocks

• Steps carried out to accomplish property 2 of previous slide:

1. Before executing an event Pi executes VCi [ i ] ← VCi [i ] + 1.

2. When process Pi sends a message m to Pj, it sets m’s (vector) timestamp ts (m) equal to VCi after having executed the previous step.

3. Upon the receipt of a message m, process Pj adjusts its own vector by setting VCj [k ] ← max{VCj [k ], ts (m)[k ]} for each k, after which it executes the first step and delivers the message to the application.

Enforcing Causal Communication

• Enforcing causal communication.

Physical Clocks

– What is time in the physical world.• Earth revolves about the sun once per year• Earth rotates on its axis once per day• 1/24 day is hour, etc. for minute and second• But earth "day" isn't exactly the same all the time so

now we use atomic clocks to define one second to be the time for a cesium 133 atom to make 9,192,631,770 "transitions”.

– We don't care about any of this really. For us the right time is that broadcast by the NIST on WWV (short wave radio) or via GEOS satellite.

Physical Clocks

– How do we get the times on machines to agree with NIST?• Send a message to to NIST (or a surrogate) asking

for the right time and change yours to that one.• How often?• If you know:

– the max drift rate D (each machine drifts D from reality)– the max error you are willing to tolerate E

• can calculate how long L you can wait between updates.

• L = E / 2D

Physical Clocks

• If only one machine drifts (other is NIST) L = E/D.

– How should we change the time?• It is bad to have big jumps and bad to go backwards.• So make an adjustment in how much you add at

each clock tick.• How do you send message and get reply in zero (or

even just "known in advance") time?• You can't. See how long it took for reply to come

back, subtract service time at NIST and divide by 2. Add this to the time NIST returns.

Physical Clocks

– What if you can't reach a machine known to be perfect (or within D)?• Ask "around" for time.• Take average.• Broadcast result.• Eliminate outliers.• Try to contact nodes who can contact NIST or nodes

who can contact nodes who can contact NIST.

Global Positioning System

• Computing a position in a two-dimensional space.

Global Positioning System

• Real world facts that complicate GPS

1.It takes a while before data on a satellite’s position reaches the receiver.

2.The receiver’s clock is generally not in synch with that of a satellite.

Mutual Exclusion

– Try to do mutual exclusion without shared memory.

– Centralized approach• Pick a process as a coordinator (mutual-exclusion-

server)• To get access to Critical Section send message to

coordinator and await reply.• When you leave CS send message to coordinator.• When coordinator gets a message requesting CS it

– Replies if the CS is free– Enter requesters name into waiting queue

Mutual Exclusion

• When coordinator gets a message announcing departure from CS

– Removes head entry from list of waiters and replies to it

• The simplest solution and perhaps the best

– Distributed solution• When you want to get into CS • Send request message to everyone (except yourself)

– Include timestamp (logical clock!)

• Wait until you receive OK from everyone• When receive request ...

Mutual Exclusion

• If you are not in CS and don't want to be, say OK • If you are in CS, put requester's name on list• If you are not in CS but want to be,

– If your TS is lower, put name on list– If your TS is higher, send OK

• When you leave CS, send OK to all on your list

• Token Passing solution• Form logical ring• Pass token around ring• When you have the token you can enter CS (hold on

to the token until you exit)

Mutual Exclusion

• Comparison– Centralized is best– Distributed of theoretical interest– Token passing good if hardware is ring based

(e.g. a token ring LAN)

Election

– How do you get the coordinator for centralized algorithm?

– Bully Algorithm• Used initially and in general when any process

notices that the process it thinks is the coordinator is not responding.

• Processes are numbered (e.g. by their addresses or names or whatever).

• The algorithm below determines the highest numbered process, which then becomes the elected member.

Election

– Called the bully algorithm because the biggest wins.

– When a process wants to hold an election• It sends an election message to all higher numbered

processes.• If any respond, original process gives up.• If none respond, it has won and announces election

result to all processors (called coordinator message in book).

– When you receive an election message:• Send OK.

Election

• Start an election yourself.– When a process comes back up, it starts an

election.• Ring algorithm (again biggest number

wins).– Form a (logical) ring.– When a process wants a new leader:

• Send election message to next process (in ring) include its process # in message.

• If message not ack'ed, send to next in ring.

Election

• When message returns:– Election done.– Send result message around the ring.

– When request message arrives:• Add your # to message and send to next process.

– If concurrent elections occur, same result for all.

Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All

rights reserved. 0-13-239227-5

Elections in Wireless Environments (1)

• Figure 6-22. Election algorithm in a wireless network, with node a as the source. (a) Initial network. (b)–(e) The build-tree phase

Elections in Wireless Environments

• Election algorithm in a wireless network, with node a as the source. (a) Initial network. (b)–(e) The build-tree phase

Elections in Wireless Environments

• (e) The build-tree phase. (f) Reporting of best node to source.

Elections in Large-Scale Systems

• Requirements for superpeer selection:

1. Normal nodes should have low-latency access to superpeers.

2. Superpeers should be evenly distributed across the overlay network.

3. There should be a predefined portion of superpeers relative to the total number of nodes in the overlay network.

4. Each superpeer should not need to serve more than a fixed number of normal nodes.

Elections in Large-Scale Systems

• Moving tokens in a two-dimensional space using repulsion forces.