1

Sharing Data Safely: Sharing Data Safely: Interprocess Synchronization and Interprocess Synchronization and

CommunicationCommunication

2In These Notes . . . In These Notes . . . • Sharing data safely

– When multiple threads/processes interact in a system, new species of bugs arise1. Compiler tries to save time by not reloading values which it doesn’t

realize may have changed

2. Switching between threads can lead to trying to operate upon partially updated variables/data structures

– We must design the system to prevent or avoid them

3Volatile DataVolatile Data• Compilers assume that variables in memory do not change

spontaneously, and optimize based on that belief– Don’t reload a variable from memory if you haven’t stored something there

since the last time you read from it

– Read variable from memory into register (faster access)

– Write back to memory at end of the procedure, or before a procedure call

• This optimization can fail– Example: reading from input port, polling for key press

• while (SW_0) ; will read from SW_0 once and reuse that value

• Will generate an infinite loop triggered by SW_0 being true

• Variables for which it fails– Memory-mapped peripheral register – register changes on its own

– Global variables modified by an ISR – ISR changes the variable

– Global variables in a multithreaded application – another thread or ISR changes the variable

4The Volatile DirectiveThe Volatile Directive• Need to tell compiler which variables may change outside

of their control– Use volatile keyword to force compiler to reload these vars from

memory for each usevolatile unsigned int num_ints;

– Pointer to a volatile intvolatile int * var; // orint volatile * var;

– Now each C source read of a variable (e.g. status register) will result in a assembly language move instruction

– Good explanation in Nigel Jones’ “Volatile,” Embedded Systems Programming July 2001

5Cooperation and Sharing InformationCooperation and Sharing Information• Program consists of one or more threads/processes• Any two threads/processes are either independent or

cooperating• Cooperation enables

– Improved performance by overlapping activities or working in parallel

– Better program structure (easier to develop and debug)

– Easy sharing of information

• Two methods to share information– Shared memory

– Message passing

6Shared MemoryShared Memory• Is practical when communication cost is low• Low-end embedded systems have no memory protection

support– Threads can access the data directly – e.g. global variables

– (Who needs seatbelts or airbags!)

• UNIX and high-end embedded systems have memory protection support– Impossible to see other processes’ memory space by default

• E.g. virtual memory

– Establish a mapping between process’s address space to a named memory object which can be shared across processes

– POSIX Threads (pthreads) API is a standard for workstation programming

7Message PassingMessage Passing• Most useful when communication cost is high

– Often used for distributed systems

• Producer process generates message, consumer process receives it

• Each process must be able to name other process• Consumer is assumed to have an infinite receive queue

– Bounded queue complicates the programming

• OS manages messages• Mailbox is a queue with only one entry

8The Shared Data ProblemThe Shared Data Problem• Often we want to split work between

ISR and the task code• Some variables must be shared to

transfer information• Problem results from task code using

shared data non-atomically– An atomic part of a program is

non-interruptible– A critical section (group of instructions)

in a program must be executed atomically for correct program behavior

– “If, while non-atomically reading (writing) a shared data object, a task can be preempted by code which writes (reads or writes) that shared data object, then a race condition exists”

• get_ticks() returns a long, formed by concatenating variable tchi and register tc– If an interrupt occurs in get_ticks,

we may get old value of tchi and new value of tc

– This is a race condition

volatile unsigned int tchi=0;#pragma INTERRUPT tc_isrvoid tc_isr(void) { tchi++;}

unsigned long get_ticks(){ unsigned long temp; temp = tchi; temp <<= 16; temp += tc; return temp;}

12

3

45

Step temp tchi tc

1 0x00001234 0x1234 0xfffe

2 0x12340000 0x1234 0xffff

3 0x12340000 0x1235 0x0000

4 0x12340001 0x1235 0x0001

9Critical Sections Lead to Race ConditionsCritical Sections Lead to Race Conditions• Critical section: A non-re-entrant piece of code that can only be

executed by one process at a time. Some synchronization mechanism is required at the entry and exit of the critical section to ensure exclusive use.

• Re-entrant Code: Code which can have multiple simultaneous, interleaved, or nested invocations which will not interfere with each other. This is important for parallel processing, recursive functions or subroutines, and interrupt handling.

– If invocations must share data, the code is non-reentrant. (e.g. using global variable, not restoring all relevant processor state (e.g. flags))

– If each invocation has its own data, the code is reentrant. (e.g. using own stack frame and restoring all relevant processor state)

• Race condition: Anomalous behavior due to unexpected critical dependence on the relative timing of events. Result of get_ticks() example depends on the relative timing of the read and write operations.

10Long IntegerLong Integer

• What if f2() starts running after the f1’s add.w (resulting in a carry) but before the adcf.w?

• Race condition due to non-atomic operation– Data structures– Large variables

; void f1()

add.w #0001H,_ct

adcf.w _ct+2

rts

; void f2()

cmp.w #0,_ct

jnz unequal

cmp.w #1,_ct+2

jnz unequal

; equal code here

jmp cont

unequal:

; unequal code here

cont:

long int ct;

void f1() {

ct++;

}

void f2() {

if (ct==0x00010000)

/* … */

}

11Is Queue Access Atomic for Serial Example?Is Queue Access Atomic for Serial Example?• Size field is modified by both

enqueue and dequeue functions

• Does compiler generate code which is atomic?

• The original unoptimized code is very inefficient, but the optimized version is good

; Enqueue; q->Size++;

mov.w -2[FB],A0 ; q mov.w -2[FB],A1 ; q mov.w 0024H[A0],0024H[A1]add.w #0001H,0024H[A1]

; Dequeue; q->Size--;

mov.w -3[FB],A0 ; q mov.w -3[FB],A1 ; q mov.w 0024H[A0],0024H[A1]sub.w #0001H,0024H[A1]

; Optimized Enqueue; q->Size++;

mov.w -2[FB],A1 ; q add.w #0001H,0024H[A1]

; Optimized Dequeue; q->Size--;

mov.w -3[FB],A1 ; q sub.w #0001H,0024H[A1]

12Solution 1 – Disable InterruptsSolution 1 – Disable Interrupts• Disable interrupts during critical

section– Renesas syntax ->

• Problems– You must determine where the critical

sections are, not the compiler (it’s not smart enough)

– Disabling interrupts increases the response time for other interrupts

– What if interrupts were already disabled when we called get_ticks?

– Need to restore the interrupt masking to previous value

#define ENABLE_INTS {_asm(" FSET I");}

#define DISABLE_INTS{_asm(" FCLR I");}

unsigned long get_ticks(){ unsigned long temp; DISABLE_INTS; temp = tchi; temp <<= 16; temp += tc;

ENABLE_INTS; return temp;}

13Are Interrupts Currently Enabled?Are Interrupts Currently Enabled?• FLG’s I flag (bit 6)

– Enables/disables interrupts– Section 1.4 of ESM

• Need to examine flag register, but how?– Not memory-mapped– Can’t access with BTST

• Solution– STC: Store from control register

(ESM)– Use a macro (CLPM98) to copy

the flag bit into a variable iflg in our code (we copy the whole register, then mask out the other bits) – nifty feature!

– Later use that variable iflg to determine whether to re-enable interrupts

#define I_MASK (0x0040)#define GET_INT_STATUS(x) {_asm(" STC

FLG,$$[FB]",x); x &= I_MASK;}

#define ENABLE_INTS {_asm(" FSET I");}#define DISABLE_INTS {_asm(" FCLR I");}

unsigned long get_ticks(){ unsigned long temp, iflg; GET_INT_STATUS(iflg); DISABLE_INTS; temp = tchi; temp <<= 16; temp += tc; if (iflg) ENABLE_INTS; return temp;}

14Solution 2 – Disable the SchedulerSolution 2 – Disable the Scheduler• If no ISR shares this data with the thread, can disable

scheduler, keeping it from switching to another thread• Interrupts are still enabled• Counter-productive

– We added the scheduler to provide efficient processor sharing

– This defeats the purpose of the scheduler!

15Solution 3 – Use Semaphore to Control AccessSolution 3 – Use Semaphore to Control Access• Operating system typically offers mutual exclusion

support through semaphores– Provide mutually exclusive access to a shared resource– Signal occurrence of events– Link resumption of threads to semaphore events– Allow tasks to synchronize their activities

• Behavior– Thread requests semaphore to enter critical section– If semaphore available (non-zero), thread enters critical section

and OS updates semaphore state (sets to zero or decrements)– If semaphore unavailable (zero), OS moves thread to waiting

queue– When a semaphore becomes available, OS moves the thread

waiting on it to the ready queue– After critical section, thread releases semaphore

16Semaphore Operations Provided by OSSemaphore Operations Provided by OS• Creation/initialization• Take/Wait/Pend/P

– Often includes time-out parameter. Wait returns error code, allowing calling task to decide how to deal with lack of semaphore.

• Release/Signal/Post/V– If no task is waiting on semaphore, increment its value– If any tasks are waiting on this semaphore, move the highest

priority (or longest-waiting) task to the Ready queue• Two types of Semaphores

– Binary (0 and 1)• Only one thread can access shared resource at a time

– Counting (0 through N)• Up to N devices can access shared resource at a time

17Using SemaphoresUsing Semaphores• Rules and Overview

– We create a semaphore to guard a shared resource to maintain data integrity

– We must get permission to access the resource– We must release that permission when done

• Semaphore operations– Take (P) the semaphore before (down, pend)– Release (V) it after (up, post)

• Value of semaphore indicates number of units of resource available for use– Use a binary semaphore (1 or 0) to control access to

a specific resource

• P: wait until semaphore is free, then take it (down)– If semaphore is free, take it and continue executing– Otherwise put calling thread into waiting state

• V: release the semaphore (up)– If a task is waiting for this semaphore, move that

task to the ready queue

long int counter;void f1() { Take(counter_sem); counter++; Release(counter_sem);}void f2() { Take(counter_sem); counter++; Release(counter_sem);}

18Implementing SemaphoresImplementing Semaphores• Semaphores are implemented with OS function or macro• Fundamental issue: need to access guard variable atomically• Option 1: disable interrupts before accessing semaphore

– Easy to implement– Needs no special support– Delays processor unnecessarily– Becomes a large problem for multiprocessors

• Option 2: use special read-modify-write assembler instruction– Test-and-set

• Read a memory location and, if the value is 0, set it to 1 and return true. Otherwise, return false

• M16C: BTSTS dest (Bit test and set)– Z <= 1 if dest == 0 (“return value is Z flag”), else Z <= 0– C <= 1 if dest != 0, else C <= 0– dest <= 1

• BTSTC: Bit test and clear– Fetch-and-increment

• Return the current value of a memory location and increment the value in memory by 1– Compare-and-swap

• Compare the value of a memory location with an old value, and if not the same, replace with a new value

19Solutions to Shared Data ProblemSolutions to Shared Data Problem• Disable interrupts

– Only method if ISR and task share data

– Fast – single instruction, typically

– Greedy – slows down response time for all other threads

• Disable scheduler• Poor performance if no kernel used

• Use OS-provided semaphore– Some slowdown, but only significantly affects threads using

them

– Need OS

20uC/OS-II Intertask Comm. and SynchronizationuC/OS-II Intertask Comm. and Synchronization• OS provides three mechanisms

– Semaphores– Message Mailboxes– Message Queues

• Common points– Only tasks (not ISRs) can wait on events– Multiple tasks can wait on the same event

• When event occurs, highest-priority task runs first

– Can specify a time-out period– Implemented with Event Control Blocks (stored in an

OS_EVENT type struct)• OSEventType: semaphore, mailbox, or queue• OSEventPtr: pointer, used only for mailbox or queue• OSEventCnt: counter used for semaphore (up to 65535)• OSEventTbl/OSEventGrp: indicate tasks waiting on event

21SemaphoresSemaphores• Uses

– Signal when something has happened– Provide mutually exclusive access to a shared resource– Control access to a set of identical counted resources

• Enable this service by defining OS_SEM_EN to 1 in os_cfg.h

• Student exercise: Are these semaphores implemented by disabling interrupts or using test-and-set instructions?

• Creating a semaphore before its first use– OSSemCreate() in os_sem.c

• Argument: initial value– Binary semaphore: set to 1– Counting semaphore for N shared resources: set to N

• Return value: pointer to ECB – Is null if no ECBs left– Use this pointer to refer to semaphore (as “handle”)

– Cannot delete or free up ECB for semaphore!

22Using SemaphoresUsing Semaphores• Taking a semaphore

– OSSemPend() in os_sem.c• Arguments:

– pointer to semaphore ECB,

– time-out value (16 bits)

» Time-out value of 0 indicates task will never time out!

– error code pointer

» Points to place to put error code if something fails

– Doesn’t return (task blocks) until message available, or time out expires

– So, your code should check error code pointer to determine if it timed out

• Signaling a semaphore– OSSemPost() in os_sem.c

• Arguments: pointer to semaphore ECB

• Highest priority task waiting on semaphore will be added to ready list (and perhaps run immediately)

23Example: uC/OSII DemoExample: uC/OSII Demo• Tasks

– Task 1• Flashes red LED• Displays count of loop iterations on LCD top line

– Task 2• Flashes green LED

– Task 3• Flashes yellow LED• Displays count of loop iterations on LCD bottom line

• LCD problem– Tasks 1 and 3 use the LCD– Task 1 is higher priority than Task 3, so it can preempt task 3– This preemption could occur within Task 3’s access to the LCD

• If this happens, displayed text will be garbled (or controller IC may even hang) – Fix this by sequentializing access (enforcing mutual exclusion)

• itoa() problem– itoa() function converts an integer to a text string (char array)– Space for only one string is allocated, so itoa is non-reentrant– Various fixes possible.

• Here, just protecting it with LCD semaphore to ensure mutual exclusion• Could also change itoa() so that calling function must allocate space.

24Semaphore SolutionSemaphore Solution// define semaphore variable as globalOS_EVENT * LCD_Sem;

// create semaphore in main(), check for failureLCD_Sem = OSSemCreate(1);if (!LCD_Sem) {

DisplayString(LCD_LINE1, "Sem Fail");while (1);

}

// Task 1OSSemPend(LCD_Sem, 10000, &err);if (err == OS_NO_ERR) {

DisplayString(LCD_LINE1, p);err = OSSemPost(LCD_Sem);

} else … // failure code

// Task 3OSSemPend(LCD_Sem, 10000, &err);if (err == OS_NO_ERR) {

DisplayString(LCD_LINE2, p);err = OSSemPost(LCD_Sem);

} else … // failure code

25More On SemaphoresMore On Semaphores• Checking the status of a semaphore

– OSSemQuery() in os_sem.c• Copies the contents of the ECB into your own data structure

• Useful for debugging

26MailboxesMailboxes• Use

– Allows task or ISR to send one pointer-size variable to another task• Only one item – no additional buffering

– Usually pass a pointer to data structure with actual information• Enable this service by defining OS_MBOX_EN to 1 in os_cfg.h• Must create mailbox before using it

– OSMboxCreate()• Argument: optional pointer to message (if Mbox must be full after creation), or

else NULL• Send a message to a mailbox

– OSMboxPost()• Arguments

– Pointer to mailbox ECB– Pointer to message (or, a pointer-sized variable)

• Wait for message to arrive at mailbox– OSMboxPend()

• Arguments– Pointer to mailbox ECB– Time-out value (16 bits)– Pointer to space for writing error code

– Doesn’t return until message available, or time out expires– Be sure to check error code!

27More on MailboxesMore on Mailboxes• ISR should get message from mailbox without blocking

– OSMboxAccept()• Argument: pointer to mailbox ECB

– Returns mailbox contents (pointer) if not empty, void if mailbox was empty

• OSMboxQuery()• Copies the contents of the ECB into your own data structure

• Useful for debugging

28Mailbox ExampleMailbox Example• Enhance demonstration• Task 2 will detect when switch S3 is pressed and tell Task

4 via an mbox• Add Task 4

– Task 4 blocks on the mbox (runs only when something posts a message in the mbox)

– When it gets the message, will change a global direction variable

– Task 3 uses this direction variable for counting

29Mailbox SolutionMailbox Solution// define mailbox variable as globalOS_EVENT * Switch3_Mbox;int T3CountDir = 1; // task 3 counts up initially#define SWITCH3_PRESSED (12345)

main() {… // create mailbox, check for failure

Switch3_Mbox = OSMboxCreate(NULL);if (!Switch3_Mbox) {

DisplayString(LCD_LINE1, "MboxFail");while (1);

} // create task 4

OSTaskCreate(Task4,NULL,TOS(Task4Stk), 8); } // end of main

Task3() {…

counter += T3CountDir; // count in specified direction …}Task4() {…

message = (INT16U) OSMboxPend(Switch3_Mbox, 0, &err); // never time out if (err == OS_NO_ERR) { // got a message

if (message == SWITCH3_PRESSED) {T3CountDir *= -1;

} …}

30Message QueuesMessage Queues• Provides a queue of pointers• Allows tasks or ISRs to send pointer-sized variables to

another task with buffering (several can accumulate)– Enable this service by defining OS_Q_EN to 1 in os_cfg.h– Define number of queues as OS_MAX_QS in os_cfg.h– Queue is implemented as circular buffer

• Must create queue first– OSQCreate()

• Arguments• start: array of pointers

– size: number of pointers in array start

– User must define array pointed to by start– Returns pointer to ECB– Cannot delete the queue

31More on Message QueuesMore on Message Queues• Waiting on a message queue

– OSQPend()• Arguments

– Pointer to queue ECB

– Time-out value

– Pointer to error code

• Returns the message, or NULL

• Doesn’t return until message available, or time out expires

• Be sure to check error code after function returns

• Posting a message in a queue– OSQPost

• Arguments– Pointer to queue ECB

– Pointer to enqueue

• Posts to tail of queue (FIFO)

– OSQPostFront• Like OSQPost, but bypasses rest of queue and posts to head (stack)

32Even More on Message QueuesEven More on Message Queues• Getting a message without blocking

– OSQAccept()• Argument: pointer to queue ECB

– Returns NULL if queue empty, or pointer if not empty

– Also dequeues returnd pointer value

33Message Queue ExampleMessage Queue Example• Enhance demonstration further• Task 2 will

– Detect when switch S2 is pressed,

– Sample the ADC several times

– Send those readings to task 1 via a message queue

• Task 1 will display those readings on the LCD rather than count– Task 1 will wait for a message in the ADC_Q

– If it gets one, it will display it on the LCD and try again (with a slight delay, to make it visible to humans)

– If it doesn’t before its period expires, it will display the counter value instead

• See code for details

34

• A needs resources X and Y• B needs resources X and Y• Sequence leading to deadlock

– A requests and gets (locks) X– context switch– B locks Y– B requests X, doesn’t get it,

leading to…– context switch– A can’t get Y– B can’t get X

DeadlockDeadlockA() {

lock(X);lock(Y);….unlock(Y);unlock(X);

}

B() {lock(Y);lock(X);….unlock(X);unlock(Y);

}

ContextSwitch

B never gets X

35Deadlock (ContDeadlock (Contd)d)• Deadlock: A situation where two or more processes are

unable to proceed because each is waiting for one of the others to do something.

• Livelock: When two or more processes continuously change their state in response to changes in the other process(es) without doing any useful work. This is similar to deadlock in that no progress is made but differs in that neither process is blocked or waiting for anything.

• Deadlock can occur whenever multiple parties are competing for exclusive access to multiple resources what can be done?

– Deadlock prevention– Deadlock avoidance– Deadlock detection and recovery

36Deadlock PreventionDeadlock Prevention• Deny one of the four necessary conditions

– Make resources sharable• No mutual exclusion

– Processes MUST request ALL resources at the same time.• Either all at start or release all before requesting more• “Hold and wait for” not allowed• Poor resource utilization and possible starvation

– If process requests a resource which is unavailable • It must release all resources it currently holds and try again later • Allow preemption • Leads loss of work

– Impose an ordering on resource types. • Process requests resources in a pre-defined order • No circular wait• This can be too restrictive

37More Deadlock StrategiesMore Deadlock Strategies• Avoidance

– Allow necessary conditions to occur, but use algorithms to predict deadlock and refuse resource requests which could lead to deadlock – Called Banker’s Algorithm

– Running this algorithm on all resource requests eats up compute time

• Detection and Recovery– Check for circular wait periodically. If detected, terminate all

deadlocked processes (extreme solution but very common)

– Checking for circular wait is expensive

– Terminating all deadlocked processes might not be appropriate