(c) 2011 dave stewart - inhand electronics - systems conference boston, september 20111 real-time...

(c) 2011 Dave Stewart - InHand Electronics - www.inhand.com Embedded Systems Conference Boston, September 20111

Real-Time Design Guidelinesand Rules of Thumb

Embedded Systems Conference Boston 2011ESC-309

Dave Stewart, PhDDirector of Research and Systems IntegrationInHand [email protected]


Class Objective

To provide some guidelines and rules of thumb to quickly make the right decisions when designing and implementing real-time systems, even if you don’t have an in-depth theoretical or practical real-time systems experience.

Guideline: Generally a good practice to follow

Rule of Thumb: quick decision or approximation

Caution: Common Pitfall to Avoid


I. Choosing the Right Environment

II. Software Decomposition

III. Practical Real-Time Scheduling

IV. Interprocess Communication

V. Implementation Solutions

Contents


Hard vs. soft real-time Predictable systems Preemption Operating systems vs. executives



Terminology

Hard Real-Time Timing requirements must be met precisely

Failure to meet requirements leads to significant failure catastrophic (system is destroyed) damage to environment or injury to people economic (product fails in the market place)

Soft Real-Time There is some flexibility in the timing constraints


Hard or Soft?

Are these applications Hard or Soft real-time? Flight Control System Bottling Plant Production Line Anti-Lock Braking System DVD Playback Airline Reservation System Internet Video Cellular Phone RF Reception


Hard or Soft?

Are these applications Hard or Soft real-time? Flight Control System Soft Bottling Plant Production Line Hard Anti-Lock Braking System Soft DVD Playback Hard Airline Reservation System Soft Internet Video Soft Cellular Phone RF Reception Hard

My OpinionAny Surprises?


Soft Real-Time as a Spectrum

Almost all applications can be soft real-time Sometimes, one or more timing constraints are hard

Right-most end of spectrum Rarely are all timing constraints hard Few applications are non-real-time

Left-most end of spectrum

HardNot RT

Soft

ComputerSimulation

UserInterface

ElectronicEngine

WirelessCommunication

InternetVideo

TemperatureControl

FlightControl


Real vs. Artificial Constraints

Real There is no flexibility in the timing constraint. If it is changed or

relaxed, the system will fail. Eg. 150 msec window for filling bottle on conveyor belt.

Artificial Timing constraint was determined somewhat arbitrarily. E.g. 40 Hz rate for cruise control.

Important to distinguish between Real and Artificial: Artificial constraints can be refined, possibly improving other

design factors (like cost & functionality) Real constraints must be met at any cost.


Terminology

Predictable The system operates within known bounds. Exact order of operations is not always known.

Deterministic A special (extreme) case of predictable. The system operates with exact order of operations

known at all times.

A Real-Time System’s timing behavior needs to be Predictable, but seldom Deterministic


Example

Dice: Predictable or Deterministic?


How do we make a System Predictable?

Maximize periodic activities Minimize aperiodic (random) activities

Avoid unnecessary preemption Choose appropriate scheduling strategies Eliminate interrupts Use deferrable or sporadic server for aperiodic tasks Use only “guaranteed” soft real-time tasks Measure execution time to monitor resource usage Implement proper timing error detection and handling


Terminology

Periodic Code that executes at regular intervals Rate refers to how often task executes:

E.g. 50 Hz (50 times per second) Period = 1/Rate.

E.g. 1/(50Hz) = 20 msec.

Aperiodic Code that executes irregularly or randomly. Minimum inter-arrival time is shortest amount of time

between two successive aperiodic events.


Periodic

In Real-Life, is it easier to schedule? 1hr meeting at the same time each day? 1hr meeting at the same time each week? 4hr meeting at the same time each week? 1hr meeting twice a week, but time changes?

Real-Time is like real-life. It is easier to schedule: Tasks that are periodic instead of irregular Less frequent tasks Shorter tasks


KISS Principle

From the Jargon Dictionary: KISS Principle /kis' prin'si-pl/ n.

“Keep it short and simple”

Periodic Tasks help KISS. Execute each task as slow as possible Execute each task for as little time as possible Anything else adds complexity


Preemption

Do you need it? Is it helping or hurting your application?


Preemption: When it Helps

Wide variety of periods and execution times If freq of fastest task < 100 * freq of slowest task,

then the task set is likely schedulable without preemption Multiple applications sharing the processor RT and non-RT tasks sharing a processor Hard and soft RT tasks sharing the processor More tolerant to design-time mistakes

But this leads to laziness at design time


Preemption: What it Costs

Reduces predictability Increases overhead (scheduling, context switch) Possible race conditions Introduces possibility of priority inversion Significant increase in overhead to avoid race

conditions and priority inversion Reduces schedulable bound if blocking is allowed More difficult to test and debug Potential for deadlock if blocking occurs Estimate at least 25% increase in CPU usage for a

preemptive system, compared to non-preemptive.


Preemption: Do You Need It?

Use KISS to decide: If your application needs it, preemption can be a

tremendous benefit to reduce complexity. If you don’t need it but use it, your application will be

much more complex than necessary.

The best compromise might be limited preemption.

more later


Real-Time Operating Systems (RTOS)

Every RTOS provides preemption. Why don’t any provide the option of

non-preemption?


Non-Preemptive RTOS

Called “Real-Time Executive” (RTE) There are no COTS RTEs because they can be

implemented in 5 to 10 lines of code. You can’t sell 10 lines of code for $3000.

I don’t need preemption, so I don’t need an RTOS? Maybe

I need preemption, so I need an RTOS? Yes

COTS: Commercial Off-The-Shelf


COTS RTOS

When do I use an RTOS? You determine preemption is necessary Lots of COTS input/output devices, like VGA monitor,

USB serial ports, touchscreen, ethernet, etc. You are using advanced CASE tools to design and

implement software

CASE: Computer-Aided Software Engineering


COTS RTOS

Roll my own RTOS vs. COTS RTOS If you don’t have a degree in RTOS design, always use COTS Roll your own only if you are an expert in the field,

and there are extenuating circumstances requiring it. E.g. Using RT mechanisms not available in commercial RTOS Domain-specific certifications require 100% control of source code If you roll your own, use a “free” RTOS as a starting point (eg

uCOS, RTLinux) You are in the business of selling RTOS technology

Licensing costs: Must compare cost of license(s) to cost of labor. More often than not, cost of labor is substantially higher


Real-Time Executive

When do I use a Real-Time Executive? You determine preemption is not necessary Most or all of the I/O is custom You need 100% control of all code on the processor

for software certification Extreme processor resource constraints

RAM or ROM < 10 KBytes CPU speed < 10 MHz


Real-Time Executive

Roll my own? Yes, because there are no commercial ones Use a Commercial or Free IDE that provides the

compiler, standard libraries, and debugging tools Do NOT hard-code the process flow.

That’s not an RTE, its just hacked code. An expert should design the core RTE engine, so that

user tasks simply “plug in” to the framework.

IDE: Integrated Development Environment



Circular Dependencies are BAD Use a Data Flow Architecture Model of a Real-Time Task


Circular Dependencies are BAD

Circular Dependencies are the primary cause of “spaghetti code”

Use Dependency Trees to spot and remove circular dependencies

Eliminating circular dependencies provides the following benefits: Component reusability Fault confinement Incremental testing Modularized performance and timing analysis


abc

def ghi

jkl stu

mno uvwpqr

xyz

jkljkl

uvwpqr

xyz

Dependency Tree Example

A circular dependency changes the tree into a graph.Avoid circular dependencies at (almost) all costs.

abc

def ghi

jkl stu

mno uvwpqr

xyz

ghi

jkl stu

mno uvwpqr

xyz

abc

def ghi

jkl stu

mno uvwpqr

xyz

abc

def ghi

jkl stu

mno uvwpqr

xyz


Avoiding circular dependencies

Never have a single “globals.h” or equivalent file with all header information.

Instead, one .h file per “module” Only put exported declarations in the .h file. Data structures that are private should be defined in

the .c file, not in the .h file.

If a module is dependent on another, only then should it include the .h file of the other module.


Breaking Circular Dependencies

ghi

jkl

ghi jkl

shared

replace with

This is not always easy.It will force careful reconsideration of the design.

Re-designing to satisfy dependency requirements will result in softwarethat is substantially easier to maintain.

Additional details is beyond the scope of this class.


Use a Data-Flow Architecture

Every task is independent Use the dependency tree to verify this

Only use explicit Interprocess Communication to exchange data between tasks

task A

ipc

task A

task B

task Btask B

task A

task A

task B

Bad Good


Use a Data-Flow Architecture

Independent tasks enable use of real-time scheduling

Without real-time scheduling, timing constraints cannot be guaranteed


Data Flow Architecture Example(Cruise Control System)

engine

force to

brake

brake pedal

accel

accelerator pedal

ccui

cruise controluser interface

wheels

force to

dashboard

speedometer

position

wheel position resolver odometer

dx/dt

control

split-pm

fra

frb

xz

+–=

+

–

fd

fde

fdw

wheels

engine

vr

Ur

vz

Start with outputsthen produce diagramright to left.


Model of a Real-Time Task

: IPC Outputy1

ym

r1 rp

real-timetask

:

:IPC Inputx1

xm

ConfigurationConstants

Sensor Inputs and Control Outputs

k1 kp

:


Model of a Real-Time Task

Each task has a main loop that does the following:

Read Inputs/Events

Do Processing

Write Outputs

Wait for EventFor periodic tasks, event is time-based.

For other tasks, event could be an interrupt, message arrival, semaphore wakeup, or any other signal.


Proper Implementation of Periodic Tasks

DO (use a framework and “wait until” to ensure no clock skew) nextstart = clock(); while (1) { nextstart += period; wait until period // wait for timer event call task->cycle() } DON’T (implement timing inside task, and use sleep that causes clock skew) taskMain() { laststart = clock(); while (1) { delta = period – (clock() – lasttime); sleep(delta); // wait for timer event do stuff } } Additional examples when real-time scheduling examples are presented.


Proper Implementation of Aperiodic Tasks

Use Aperiodic Servers More details near end of this talk.


Your Life is a Real-Time System Cyclic executives Rate monotonic algorithm Dynamic scheduling



Practical Real-Time Scheduling

Now: Hard Real-Time Periodic Tasks Non-Preemptive Preemptive Fixed Priority Dynamic Priority

Later: Interrupts Aperiodic servers Soft real-time tasks Limited preemption


Your Life IS a Real-Time System


Real-Life Scheduling

If you have 10 tasks to do, what order to you do them in? If you go to the bank, in what order are you served? If you need to catch a flight at 5pm, how does that affect

your schedule? If it is lunch time and you have a report due at 3pm, do

you stop for lunch? If the report is due at 7pm, do you stop for lunch? These same kinds of decisions must be made for each

task in a real-time system. That’s real-time scheduling.


Basic Cyclic Executive

Everything executes at same rate Example: 3 Tasks Each task executes every 20 msec

(TA=TB=TC=20) Note: Ti=Period of Task i

Execution time constraints: SCi < TA

Note: Ci=Worst Case Execution Time of Task i

Suitable for simple control loops.

A

B

C

A B C

0 20 40 60 time (msec)

A B C A B C A B C


Basic Cyclic Executive Implementation

typedef void (*funcptr)(void);void A(void) { // B and C defined similarly //code goes here}

funcptr cyclic[] = { A, B, C};const int cycliclength = sizeof(cyclic)/sizeof(funcptr);

cyclicexec(funcptr cyclic[],int cycliclength, int cycletime) { int i; intrSetPeriod(20msec); while (1) { intrWait(); for (i=0;i<cycliclength;++i) cyclic[i](); }}

A B C

0 20 40 60 time (msec)

A B C A B C A B C

See paper fordetails onimplementingthe timer.

A

B

C


Multi-Rate Cyclic Executive

Tasks execute at different rates Example: 3 Tasks A executes every 20 msec (TA=20)

B and C execute every 40 msec (TB=TC=40) Execution time constraints:

CA+CB < TA && CA+CC < TA

time (msec)0 20 40 60

A B A C A B A C


Multi-Rate Cyclic Executive Implementation

typedef void (*funcptr)(void);void A(void) { // B and C defined similarly //code goes here}funcptr cyclic[] = { A, B, NULL, A, C, NULL};const int cycliclength = sizeof(cyclic)/sizeof(funcptr);

cyclicexec(funcptr cyclic[],int cycliclength, int cycletime) { int i; intrSetPeriod(20msec); while (1) { for (i=0;i<cycliclength;++i) { if (cyclic[i]==NULL) intrWait(); else cyclic[i](); }}

time (msec)0 20 40 60

A B A C A B A C


Multi-Rate Cyclic Executive

Rates for each task are constrained: Periods of each task must be harmonic To setup schedule, place fastest task in schedule first Then place next fastest, and so on.

Execution time is constrained: SCi for tasks executing each cycle must be less than

period of interrupt.


Harmonic Task Set

Period of Task K is multiple of every faster task "k>1, Tk=nk*Tk-1, k, nk are integers

Examples of Harmonic: 10, 20, 40, 80, 160 10, 30, 120, 480 20, 100, 2000

Examples of Non-Harmonic: 10, 25, 50, 75 10, 30, 100, 500 20, 120, 2200


Dynamic Multi-Rate Cyclic Executive

Relaxes constraints on period and execution time Uses Earliest-Deadline First (EDF) Algorithm

Task with earliest deadline executes next Deadline of current cycle is time task is ready for next cycle

Rate/Period constraints: none

Execution Time constraints: (S Ui) <1 (i.e. CPU not overloaded) Note: Ui=Ci/Ti=Utilization of Task I Ci for each task is < T1/2 i.e. period of fastest task dictates max execution time for every other task

Cost of using this method: Increased RTE complexity Requires more thorough analysis to guarantee scheduling feasibility


Implementation ofDynamic Multi-Rate Cyclic Executive

while (1) {do {

etimeClock(&now);// Move newly awoken tasks from PauseQ to ReadyQ

while (!taskEmptyQ(pauseQ) && etimeCompare(taskSchedTime(pauseQ),now) <= 0) {

taskptr=taskDequeue(&pauseQ); etimeAdd(taskptr>schedtime, taskptr->period); taskEnqueue(&readyQ,taskptr);

}} while (readyQ.next == NULL);

// Head of readyQ has earliest-deadline tasktaskDequeue(&readyQ, taskptr);

taskptr->func(args);

taskEnqueue(&pauseQ, taskptr);}

(note, some detail has been omitted)


Preemptive Scheduling

Always execute task with highest priority, even if it means preempting a task with lower priority.


How are Priorities Assigned?

Fixed Priority Priorities assigned prior to execution Priorities are Constant E.g. Rate Monotonic Algorithm (RMA)

Dynamic Priority Priorities change as a function of time E.g. Earliest-Deadline-First Algorithm (EDF) E.g. Minimum-Laxity-First Algorithm (MLF)

Mixed Priority Priorities include both a fixed part and dynamic part E.g. Maximum-Urgency-First Algorithm (MUF)


Terminology

Transient Overload: The state of a real-time system when at least one task will fail to

meet its timing constraints because of a lack of resources. Guaranteed Task:

A task is said to be guaranteed if it will always meet its deadlines, even in a transient overload situation. Hard real-time tasks must be guaranteed tasks.

Task Set: The set of all tasks sharing a processor.

Critical Task Set: The set of guaranteed tasks (a subset of the task set).

Schedulable Task Set All tasks in the task set are guaranteed tasks. No task ever misses a deadline.


Preemptive Scheduling Example

Consider a very simple system with two tasks: P1: T1= 50msec, C1=25msec, U1=50%. P2: T2=100msec, C2=40msec, U2=40%. U1+U2=90%. Since utilization is less than 100%, we would expect that both

tasks can be scheduled without either missing deadlines.

Using Fixed-Priority Scheduling, only two scenarios: Case 1: Priority P1 > Priority P2

Case 2: Priority P1 < Priority P2

missed deadline: a timing error where a task does not complete its execution in time.


Case 1

Priority P1 > Priority P2 P1: T1= 50msec, C1=25msec, U1=50%. P2: T2=100msec, C2=40msec, U2=40%.

P1

P2

0 10 20 30 40 50 60 70 80 90 100 110

P1 meets1st deadline

1st deadline P1

P1 meets2nd deadline


2nd deadline P11st deadline P2


Case 2

Priority P1 < Priority P2 P1: T1= 50msec, C1=25msec, U1=50%. P2: T2=100msec, C2=40msec, U2=40%.

P1

P2

0 10 20 30 40 50 60 70 80 90 100 110

1st deadline P1

P1 meets 2nd deadline


2nd deadline P11st deadline P2


P1 misses1st deadlineP1 restarts


Rate Monotonic Algorithm (RMA)

An algorithm for assigning fixed priorities The higher the frequency (or shorter the period)

of a task, the higher its priority.

Case 1 was an example of using RMA assignments. Notice how Case 1 resulted in all tasks meeting their

deadlines. Case 2 resulted in some one of the two tasks missing

deadlines.



Advantages Optimal fixed priority scheduling algorithm

If a task set cannot be scheduled using RMA, then it cannot be scheduled using any other fixed priority assignment.

If there is a transient overload, it is guaranteed that the lowest priority tasks not in the critical task set are the ones that can fail.

Disadvantages Schedulable bound often less than 100% if task set is not

harmonic Schedulable bound is the maximum utilization of the critical task

set. Example: See Case 3


Case 3: Like Case 1but C2 & T2 Change; U2 Unchanged

Priority P1 > Priority P2 P1: T1= 50msec, C1=25msec, U1=50%. P2: T2= 75msec, C2=35msec, U2=40%.

P1

P2

P1 meets P1 meets

2nd deadline P1

0 10 20 30 40 50 60 70 80 90 100 110

1st deadline

1st deadline P1

1st deadline P2

2nd deadline

P2 misses1st deadline



Even though Utilization only 90%RMA cannot schedule this task set

RMA Rules of thumb: To achieve 100% schedulable utilization using RMA, task set

must be harmonic. If task set is not harmonic, schedulable utilization can be as low

as 70%.

To Achieve 100% Utilization with non-harmonic task sets, must use Dynamic Scheduling For example, see Case 4.


Deadline Monotonic Algorithm

A Variation of RMA, except that deadline of a task can be earlier than start of next period. Designed to reduce jitter of executing tasks.

Problems with Deadline Monotonic Algorithm: Schedulable bound lower than RMA In my experience 99% of the time, jitter is not an

issue. Don’t use it!


Case 4: Like Case 3but use EDF Scheduling

Priority P1, P2 = F(deadline) P1: T1= 50msec, C1=25msec, U1=50%. P2: T2= 75msec, C2=35msec, U2=40%.

P1

P2

10 20 30 40 50 60 70 80 90 100 110


1st deadline


P1 1st deadline P2


2nd deadline P1



Earliest Deadline First

Advantages Schedulable bound is always 100%.

All tasks are in critical set if utilization < 100% Optimal dynamic priority scheduling algorithm

If a task set cannot be scheduled using EDF, then it cannot be scheduled using any other algorithm.

Disadvantages If there is a transient overload, there is no control

over which task fails. None of the tasks are in critical set if utilization >= 100%


Mixed Priority Scheduling

Combines the Advantages of RMA and EDF Schedulable bound of 100% for critical task set In case of transient overload, critical task set can still be

guaranteed

Various Options to Mixed-Priority Scheduling E.g. Maximum-Urgency-First (MUF) E.g. Limited Preemption Rate Groups

Mixed-priority scheduling is a more advanced topic;more discussion later.


Baseline SchedulingWhat Method Should You Use?

Non-Preemption Basic cyclic executive Multi-rate cyclic executive Dynamic cyclic executive

Preemption Rate monotonic algorithm Earliest-deadline-first algorithm More complex mixed-priority algorithm

Answer: KISS PrinciplePick the simplest one that meets your needs.


Other Considerations

Other design needs may alter the choice of a basic scheduling method. For example: Interprocess communication Aperiodic events Soft real-time support Interaction with non-real-time systems Dynamic system configuration changes Overloaded resources I/O requirements

These and other issues discussed next.



Race conditions Semaphores Priority inversion Real-time IPC mechanisms


Interprocess Communication

Real-Time Applications are built as collections of tasks executing at one or more rates.

Tasks exchange data through IPC.

If done incorrectly, IPC is one of the primary sources of problems in RT Systems.


Non-Preemptive Environments

Use Shared Memory between tasks. Fastest form of IPC No RTE overhead to access shared data No concern of race conditions since tasks accessing

the shared data cannot be preempted.

If there are interrupts, they can preempt non-preemptive tasks. Must be treated as a preemptive environment from

an IPC perspective, if the interrupt handler shares memory or resources with any of the tasks.


IPC in Preemptive Environments

Shared Memory and Resources Cause Problems.

This includes: Global variables Static local variables Shared memory segments I/O device registers Library functions that are not “Multi-Tasking Safe”


Race Conditions

The main problem with sharing in preemptive environments is ensuring data does not get corrupted due to a race condition.

The One Lane Bridge is Shared. It is an analogy for a shared variable.If Trains do not take special precautions, an accident will occur.If software doesn’t take precautions, data corruption will occur.

This is a Race Condition.


Semaphores

A “Semaphore” is a traffic light

The One Lane Bridge is Shared. It is an analogy for a shared variable.If Trains do not take special precautions, an accident will occur.If software doesn’t take precautions, data corruption will occur.

This is a Race Condition.


Semaphores

Semaphores are a common solution in operating systems to deal with race conditions.

In real-time systems they lead to priority inversion.

In fact, any IPC mechanism that involves blocking, including monitors, condition variables, and message passing, can lead to priority inversion.


Priority Inversion

Blocking of a high priority task due to a lower priority task locking a shared resource.


Priority InversionShown by Example

Assume 3 Tasks: PH: A high priority task PM: A medium priority task PL: A low priority task


Priority InversionShown by Example

holding R1

PH

PM

PL

PLstarts

PL

PHstarts

gets R1PHneeds R1

PLstarts

PH is blocked, while alower priority task PM is notusing resource R1 is running. Hence, we havea priority inversion.

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

suspendedready to runrunningblocked, wait ing for R1

state of task

^^^^^


Solutions to Priority Inversion

There are two fundamental ways to deal with race conditions while avoiding priority inversion: Use a mechanism that adjusts priorities of low-priority

tasks holding resources. Use a non-blocking mechanism.

WARNING! Many RTOS provide blocking IPC mechanisms that do not adjust priorities.

Do NOT use these mechanisms in Real-Time Systems!


Adjusting Priorities:Priority Inheritance Protocol

Warning: For demonstration only. Not a good protocol.


Adjusting Priorities:Priority Inheritance Protocol

PLstarts

PLgets R1

PHstarts

PHneeds R1

PMstarts

PLreleasesR1

PHreleasesR1

PHends PM

endsPLends

PH

PM

PL

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

due to priority inversionPL assumes priority of PH

PM does not execute, because priorityof PL is temporarily higher

holding R1

suspendedready to runrunningblocked, waiting for R1

state of task

^^^^^


Adjusting PrioritiesPriority Ceiling Protocol (PCP)

Highest Locking Protocol (HLP) is very similar. Similar to Priority Inheritance, except that each resource is

given a priority ceiling, which is the priority of the highest-priority task that may lock the resource.

When a resource is locked, task immediately inherits the priority ceiling.

Advantage: It Works

Disadvantages: Significant RTOS overhead Significant reduction in schedulable bound due to blocking Many RTOS do not provide this mechanism


Non-Blocking RT IPC

The alternative to PCP is to use a non-blocking form of communication. Some examples: State variable mechanism Non-blocking message passing Triple-buffer mechanism FIFOs


State Variable (SVAR) Communication

Different versions for different environments Used in state-based design to store “states.”

E.g. “velocity” of a vehicle can be a state variable in a cruise control system, because only the current speed is important.

Key attribute of states: Only the most recent data is needed. Allows tasks executing at different rates to communicate with each

other: If a producer is faster than a consumer, then the consumer may miss some

variable changes. If a producer is slower than a consumer, then the consumer may read the

same value multiple times. State data is “small”, so that a task can lock out interrupts (thus

preventing preemption) for a few microseconds to update the state. Each module reads the state and makes a local copy before executing

each cycle, then updates the state after completion of its cycle. It does not access the state variables at any other time.


SVAR ArchitectureNon-Preemptive Environment

Global Table

IntrA1

taskA2

taskAk


SVAR ArchitectureNon-Preemptive with Interrupts

Global Table

localtable

IntrA1

taskA2

taskAk


SVAR ArchitecturePreemptive Single Processor

Global Table

localtable

localtable

localtable

taskA1

taskA2

taskAk


ExampleConsider following Data-Flow

qm

xd

qr

xm

vr

xr

gfwdkinxm

ginvkin rmmscinterptball

ndof

dh

ndof

dh

wm

fromsensors

toactuators

fromtrackball


SVAR Global and Local TablesContain Data as Follows

xd

tball cinterp rmmsgfwdkinginvkin

xr .xd xm xr xr NDOF qr qrqmDH NDOFwmqm NDOF DH xmDH

ND OFDHxmxr vr xd qr qm wmGlobal Table:

Local Tables:

PBOs:


SVAR ArchitectureMultiprocessor Shared Memory Environment

Global Table

localtable

taskA 1

TaskA j

Processor A

localtable

localtable

TaskK1

Taskkj

Processor L

localtable

memcpy()or DMA


SVAR ArchitectureNetwork Environment

Global Table

localtable

TaskK1

Taskk j

Processor L

localtable

network messages

Processor X

data server

data client

localtable

TaskK1

Taskk j

Processor A

localtable

data client


Non-Blocking Message Passing

Just Like Regular Message Passing, Except: Sending: if message queue is full, abort operation

and continue. Receiving: if message queue is empty, return

immediately without data.

In such cases, it is important to design the system to properly handle the above cases.


Messages vs. States

In most cases, messages can be converted to states, and vice versa.

Messages are best used for real-time client/server architectures.

States are best used in feedback control applications.


SEND (“apply brake ”) . . .

control module

“apply brake”

“apply brake”

brake = ON . . .

IF brake == ON activate brake

ELSE

brake 2

brake

ON

global table (shared memory)

brake

ON

brake

ON

local table local table

...

IF brake == ON activate brake

ELSE

brake 1

brake

ON

local table

...

Using Messages, “Apply Brake”:

Using States, “The Brakes Should be ON”:

control module

RECEIVE (&message)IF message == “apply brake” activate brakeELSE : brake 2

RECEIVE (&message)IF message == “apply brake” activate brakeELSE : brake 1


Triple Buffer Mechanism

Used for point-to-point state communication Does not require any hardware or RTOS support Three (or more) copies of the data exist. No two tasks ever read and write from the same copy

at the same time. When a writer finishes, it marks the buffer as ‘new’,

and moves on to a different buffer. Always at least one buffer not being used by other

tasks. This eliminates the need to tightly synchronize readers and

writers.


Triple Buffer Mechanism

acquire anEMPTY buffer

anybuffers

EMPTY?

acquire oldestunused buffer

fill bufferwith new data

mark bufferas FULL

wait for start ofcycle signal

reuse buffer fromprevious cycle

anybuffersFULL?

acquire newestFULL buffer

copy data outof buffer

mark bufferas EMPTY

wait for start ofcycle signal

sender receiver


FIFOs (First-In-First-Out)

Similar to triple-buffer mechanism, except that it guarantees all data is available.

Similar to message passing, but with less overhead and suitable for large volume of data.

On average, consumer must be at least as fast as producer, otherwise data loss can occur.

Can be used for communication between real-time and non-real-time tasks.


Other RT IPC

There are hundreds (or thousands) of real-time IPC mechanisms described in the literature.

Most RTOS only implement zero to one or two of them. You may find you need to implement your own.

Consider IPC in conjunction with scheduling. If scheduling can prevent race conditions, then you don’t need complex IPC. Instead, you can just use simple shared memory.


V. Implementation Solutions

Timing failure detection and handling Soft real-time scheduling Interrupts and aperiodic events Dynamic system configuration changes Timing anomalies


Timing Failure Detection and Handling

Examples of Timing Errors: Missed deadline Using too much execution time End-to-end response time too long Incorrect sampling or control rate Clock skew Too much jitter Inaccurate system or real-time clock


Online Timing Error Detection and Handling

An RTOS, RT Executive, or component framework may include built-in mechanisms to detect some timing errors. Example to follow considers missed deadline and

using too much execution time. Used as the basis for implementing guaranteed

soft real-time tasks and aperiodic servers.


Timing Failure Handler (TFH) Implementation

Requires a Spare Timer Each time a new task is being swapped in, scheduler

programs timer with lesser of the following two values: task’s deadline; set type to DEADLINE estimated remaining execution time; set type to MAXEXEC

Interrupt handler for this timer is defined as follows: (see next slide)


TFH Implementation

tfhhandler(int type){

switch (type){case DEADLINE: /* missed deadline */

set_priority(Ph);nextstart += Tds;set_deadline(nextstart,Cds,Tds);return;

case MAXEXEC: /* execution time used up */set_priority(Pl);return;

} /* end switch */} D.B. Stewart and P.K. Khosla, “Mechanisms for Detecting and Handling Timing

Errors,” Communications of the ACM, v.40, n.1, pp. 87-94, January 1997. http://www.embedded-zone.com/bib/mags/cacm97.shtml

http://www.embedded-zone.com/bib/mags/cacm97.shtml


TFH Implementation

If DEADLINE error Ensure task has its normal priority. This task will continue executing in time allotted for its next period.

Thus it now has a new deadline at end of next period. Task will be forced to skip its next period to reclaim the time,

resulting in a missed cycle. If MAXEXEC error

The amount of reserved execution time by this task has been used up.

Lower priority of task, so that other high priority tasks can execute. If there is leftover CPU time, then this task will execute. When it is

done, its priority gets raised back to normal on next cycle. If there is insufficient CPU time, a DEADLINE error will follow.


Timing Failure Handler Uses

Guaranteed Soft Real-Time Tasks Use the TFH to detect missed timing constraints,

then update the tasks accordingly

Aperiodic Servers Use the TFH to detect aperiodic events that use

more than budgeted capacity, and use that to manage priorities and schedule

Above items are described next.


Soft Real-Time Scheduling

Soft real-time systems are always one of the following two: Accidental Soft Real-Time Guaranteed Soft Real-Time

Most systems claimed to be soft real-time are accidental soft real-time


Accidental Soft Real-Time

These are systems that are claimed to be soft real-time, but in reality they do not provide any real-time guarantees.

Typical “features” of such systems: System can usually keep up to real-time. If the system fails to meet its timing requirement, the

system does not crash, but the result still appears to meet requirements.

Tried to get hard real-time, but failed. Possible starvation of the soft real-time tasks.


Guaranteed Soft Real-Time

It is acceptable for the primary timing constraints to not always be met.

There is a secondary set of timing requirements that are always met.

Guarantee of no starvation of any soft real-time task.

Actively modify scheduling to provide degraded performance during transient overloads.


Using TFH to Implement GuaranteedSoft Real-Time Tasks

Suppose a task has the following execution profile:

100%

50%

0%0 exec time10 20 26 30 40

WCET=40msec. 80% of the time, task uses 26 msec or less.

80%


Using TFH to Implement GuaranteedSoft Real-Time Tasks

Suppose a task has the execution profile on previous slide: If task is hard real-time, then 40 msec must be reserved. If task has a period of 60 msec, then utilization for the task

U=40/60=66.7%. If task is soft real-time, then set its capacity to 26 msec. Then

U=26/60=43%. If execution takes less than 26 msec (as happens 80% of the time),

then task will meet its deadline. If execution takes more than 26 msec, MAXEXEC error is generated.

Task gets lower priority for any execution time greater than 26 msec until its deadline time is reached.

If task misses deadline, then in worst case it has 14 more msec to execute, during its next period.

It is guaranteed to meet its next cycle, and thus will never miss two deadlines in a row.

The net effect of such guaranteed soft real-time tasks is that in the worst-case, when the CPU is fully loaded, the task will execute at half rate.


Interrupts

Since interrupts cannot be scheduled, they always have highest priority. This could lead to priority inversion.

Periodic interrupts (like a regular timer interrupt) can be treated in RT analysis as the highest priority periodic tasks. WCET must be known. To maximize schedulable utilization, the period must be less than the period of the fastest task in the system.

Randomly arriving interrupts can create undesired effects. If they are necessary, keep execution as small as possible.

For longer execution, use an aperiodic server, with the interrupt handler sending a signal to trigger execution of the server.

“Well-behaved” interrupts only come occasionally, with a known MIAT . In analysis, randomly arriving interrupts are defined to have period

T=minimum inter-arrival time. WCET must be known. A predictable system cannot be built if interrupts do not have a

minimum inter-arrival time. Must use aperiodic servers instead.

MIAT = minimal inter-arrival time


Aperiodic Servers

Aperiodic servers offer the ability to predictably deal with randomly arriving events.

Predictability is achieved by reserving some CPU time to handle aperiodic events.

TFH is used to managed that reserved CPU time. A Deferrable Server uses the same TFH Handler as soft real-time

tasks. Sporadic Server is more complex to implement. Not much of an

improvement over Deferrable Server.

D.B. Stewart and P.K. Khosla, “Mechanisms for Detecting and Handling Timing Errors,” Communications of the ACM, v.40, n.1, pp. 87-94, January 1997.




Aperiodic Server Implementation

An aperiodic server can be implemented just like a soft real-time periodic task. Period T = “Replenishment Time” WCET C = “Capacity” Worst-case utilization U=C/T.

Depending on the objectives of the aperiodic server, there are different ways to define T and C.


Aperiodic Servers Types

Hard Real-Time requiring fastest response time possible. MIAT and WCET must be known. Period Taperiodic =MIAT/K; C=WCET/K.

K is selected such that Taperiodic < Tfastest_task. Disadvantage: U=C/T could be significant, as this task

will have highest priority in the system. If possible, design system to relax the ‘fastest

response time possible.’



Hard real-time, with deadline. Immediate response is not needed, but execution

must complete by a given deadline. Set Taperiodic =MIAT, C=WCET.

U=C/ Taperiodic is same as previous case and could still be significant, but this server might have lower priority than some other hard real-time tasks thus improving overall schedulability.



Soft real-time. If MIAT not known or WCET not known, then aperiodic server

must be soft real-time. Decide what percentage of CPU to reserve for processing the

aperiodic event. Thus U defined first. Set Taperiodic =desired response time.

Set C=U* Taperiodic. If event arrives, then aperiodic server will attempt to process it

within the desired response time. If unable, it will continue processing as soon as possible, but not using more than the reserved amount of CPU time (unless no other real-time tasks with reserved execution time are ready to run).



Soft real-time. Priority of the aperiodic server can be controlled:

To increase priority: decrease both T and C by a factor of K (K>1).

To decrease priority: increase both T and C by a factor of K (K>1).

If desired response time rarely being achieved, then increase U, the reserved CPU utilization for the server.


Dynamic System Configuration Changes

Many real-time systems are implemented with multiple modes of operation

Each mode might execute a different set of tasks Create a finite state machine (FSM) to define modes as states, and

mode changes as transitions. Only change set of executing tasks when mode changes. Thus

each state can be defined to have a specific task set. Analyze RT performance for each state. Changing the period of a task is equivalent to a mode change. Use dynamically reconfigurable software to change states in real-

time. As one example, see D.B. Stewart, “Software Components for Real-

Time Systems,” Embedded Systems Programming, December 2000. http://www.embedded-zone.com/bib/mags/esp2000.shtml

http://www.embedded-zone.com/bib/mags/esp2000.shtml


Summary - Objective

To provide some guidelines and rules of thumb to quickly make the right decisions when designing and implementing real-time systems, even if you don’t have an in-depth theoretical or practical real-time systems experience.

Guideline: Generally a good practice to follow

Rule of Thumb: quick decision or approximation

Caution: Common Pitfall to Avoid


Summary - Content





V. Common Problems and Solutions


Additional Resources

Publications on Embedded Research Solutions Website http://www.embedded-zone.com/publications.shtml

Embedded Systems Conference Proceedings http://www.esconline.com/papers.htm

Embedded Systems Programming Magazine Archive http://www.embedded.com/archive.htm

Real-Time Systems Symposium (RTSS) [for theory] http://www.computer.org/proceedings/proceed_r-z.htm#r Requires IEEE Computer Society membership for access

EG3 Embedded Systems Portal http://www.eg3.com/

InHand Electronics http://www.inhand.com/

http://www.embedded-zone.com/publications.shtml

http://www.esconline.com/papers.htm

http://www.embedded.com/archive.htm

http://www.computer.org/proceedings/proceed_r-z.htm#r

http://www.eg3.com/

http://www.inhand.com/

(c) 2011 dave stewart - inhand electronics - systems conference boston, september 20111 real-time...

Documents

applications hard

hard right

market place soft realtime

right environment slide

realtime design guidelines

timing constraints

artificial constraints

right decisions