Verification of Real Time Operating Verification of Real Time Operating System with Model CheckingSystem with Model Checking

Toshiaki AokiToshiaki AokiJapan Advanced Institute of Science and TechnologyJapan Advanced Institute of Science and Technology

BackgroundBackground RTOS (Real-Time Operating Systems) are used for RTOS (Real-Time Operating Systems) are used for

embedded systems.embedded systems. Those RTOS are provided for various platforms by various Those RTOS are provided for various platforms by various

companies.companies. Standards of RTOS are proposed.Standards of RTOS are proposed.

OSEK/VDX(AUTOSAR OS), μITRON, etc.OSEK/VDX(AUTOSAR OS), μITRON, etc. Standards are defined as documents described in Standards are defined as documents described in

natural language.natural language. The documents are likely misunderstood.The documents are likely misunderstood. The documents themselves tend to be ambiguous. The documents themselves tend to be ambiguous.

Those may lead to implement RTOS which does not Those may lead to implement RTOS which does not conform to the standards.conform to the standards. Such RTOS makes embedded systems unreliable. Such RTOS makes embedded systems unreliable.

AimAim We are proposing a method to ensure that RTOS We are proposing a method to ensure that RTOS

conforms to the standards.conforms to the standards. Our target is OSEK/VDX RTOS.Our target is OSEK/VDX RTOS.

It is very hard to ensure the conformance after the It is very hard to ensure the conformance after the implementation by testing.implementation by testing. Test cases to ensure the conformance depend on the inside of Test cases to ensure the conformance depend on the inside of

the implementation.the implementation. Describing exhaustive test cases is very hard.Describing exhaustive test cases is very hard.

We ensure the conformance in the design phase of We ensure the conformance in the design phase of RTOS development.RTOS development. We apply model checking to the design of RTOS to exhaustively We apply model checking to the design of RTOS to exhaustively

check it.check it. We are using Spin model checker.We are using Spin model checker.

ApproachApproach We directly describe RTOS design in Promela.We directly describe RTOS design in Promela.

Promela is the specification language of Spin.Promela is the specification language of Spin. We need the other Promela descriptions for checking the design.We need the other Promela descriptions for checking the design.

How to use RTOS and expected results are described in them.How to use RTOS and expected results are described in them. We obtain such Promela descriptions by modeling the possible usage of We obtain such Promela descriptions by modeling the possible usage of

OSEK/VDX RTOS and expected results based on the specification.OSEK/VDX RTOS and expected results based on the specification.

OSEK/VDX Specification

RTOS design

directly describe RTOS design in Promela.

Promela descriptionfor checking the design

Modeling the possible usage of RTOS and expected results.

model checking by Spin

straightforwardtarget

OSEK/VDX RTOSOSEK/VDX RTOS OSEK/VDX RTOS is used for automotive systems.OSEK/VDX RTOS is used for automotive systems.

Currently, its standard is being developed as AUTOSAR OS.Currently, its standard is being developed as AUTOSAR OS. The most of the OSEK/VDX standard specification is inherited to The most of the OSEK/VDX standard specification is inherited to

AUTOSAR OS.AUTOSAR OS. OSEK/VDX RTOS deal with preemptive fixed-priority OSEK/VDX RTOS deal with preemptive fixed-priority

multi-tasks, resources, interrupts, and so on.multi-tasks, resources, interrupts, and so on. The resources are similar to lock/unlock primitives with a priority The resources are similar to lock/unlock primitives with a priority

ceiling protocol.ceiling protocol. Fixed priorities are assigned to the interrupts.Fixed priorities are assigned to the interrupts.

Task A

Task B

time

preempt

Task A

Task B

time

terminate

terminateactivate

Task A < Task B

activate

Task B < Task A

Design of OSEK/VDX RTOSDesign of OSEK/VDX RTOS We focus on the scheduler of RTOS.We focus on the scheduler of RTOS.

The most of the specification defines how to schedule the tasks.The most of the specification defines how to schedule the tasks. It is easy to describe the scheduler in Promela.It is easy to describe the scheduler in Promela.

The scheduler is realized by task queues and tables to keep their The scheduler is realized by task queues and tables to keep their information.information.

Those are straightforwardly described in Promela.Those are straightforwardly described in Promela. You can download the Promela description of μITRON RTOS which is You can download the Promela description of μITRON RTOS which is

similar to OSEK/VDX RTOS in the following URL.similar to OSEK/VDX RTOS in the following URL. http://aoki-www.jaist.ac.jp/~toshiaki/modules/tinyd0/index.php?id=10http://aoki-www.jaist.ac.jp/~toshiaki/modules/tinyd0/index.php?id=10

task queues

TCB

tables for resource and interrupts management

The scheduler of OSEK/VDX RTOS

Design VerificationDesign Verification RTOS is an open system.RTOS is an open system.

RTOS does scheduling of the tasks if it gets stimulus RTOS does scheduling of the tasks if it gets stimulus such as system call invocations.such as system call invocations.

RTOS does not do anything if it does not get any RTOS does not do anything if it does not get any stimulus.stimulus.

We need the description of the outside of RTOS We need the description of the outside of RTOS to verify the RTOS design.to verify the RTOS design. The outside consists of an application which invokes The outside consists of an application which invokes

the system calls and hardware which causes the the system calls and hardware which causes the interrupts.interrupts.

The outside of the verification target is called an The outside of the verification target is called an environmentenvironment..

EnvironmentEnvironment The description of an environment consists of The description of an environment consists of

invocations of the system calls of RTOS.invocations of the system calls of RTOS. represented as state transition models with the system calls.represented as state transition models with the system calls.

expected results of those invocations. expected results of those invocations. represented as assertions assigned to those states.represented as assertions assigned to those states.

Model checking the RTOS design in combination with Model checking the RTOS design in combination with the description of the environment by Spin.the description of the environment by Spin.

(SUS,SUS)

(RDY,RUN)

(RUN,SUS) (RUN,SUS)

1:TerminateTask()

2:TerminateTask()

ActivateTask(1)

ActivateTask(2)

RTOS design Environment (2 Tasks, Task 1<Task 2)

invokes

info

Approaches to Describe Approaches to Describe EnvironmentsEnvironments

Automatic generation by static analysis on programs.Automatic generation by static analysis on programs. BEG (Bandera Environment Generator) [O.Tkachuk ’03]BEG (Bandera Environment Generator) [O.Tkachuk ’03] Effective if the programs corresponding to the environment Effective if the programs corresponding to the environment

available.available. Usually not the case in the design stage.Usually not the case in the design stage.

Invoke all the system calls non-deterministicallyInvoke all the system calls non-deterministically Universal environment [J.Penix ’00]Universal environment [J.Penix ’00] Able to check all the execution sequences exhaustively.Able to check all the execution sequences exhaustively. Description of properties becomes complex. State explosion.Description of properties becomes complex. State explosion.

Our approach: Modeling execution sequences in a specific range Our approach: Modeling execution sequences in a specific range depending on the properties to check.depending on the properties to check. Ex.) Normal execution sequences, abnormal execution Ex.) Normal execution sequences, abnormal execution

sequences, and interrupt handling ...sequences, and interrupt handling ... Description of properties becomes simple. Possible to avoid Description of properties becomes simple. Possible to avoid

state explosion.state explosion.

Environment ModelingEnvironment Modeling Various environments can be considered.Various environments can be considered.

The number of tasks, the variation of priorities, number of the resources, The number of tasks, the variation of priorities, number of the resources, invocation relations of the system calls, and so on.invocation relations of the system calls, and so on.

It is impossible to make all the descriptions of the environments by It is impossible to make all the descriptions of the environments by hand.hand.

If all the environments are realized by one Promela description, that If all the environments are realized by one Promela description, that may cause state explosion problem.may cause state explosion problem.

We model the variations of the environments, then automatically We model the variations of the environments, then automatically generate the Promela descriptiongenerate the Promela description of each environment.of each environment.

Environment Model

TaskA(pr:5)

ResA(pr:8)

Get/Release

TaskA(pr:5)

TaskB(pr:3)

Activate

TaskA(pr:5)

TaskB(pr:5)

Activate

TaskA(pr:5)

TaskB(pr:5)

Activate ResA(pr:8)

Get/Release

• 2 tasks with different priorities.• 1 task and 1 resource.• 2 tasks and 1 resource.• 1 task and 2 resource.....

Possible Environments Modeling

Promela Descriptionsof Environments

Automatic Generation

RTOS Design

Model Checking

Environment ModelingEnvironment Modeling We model possible structures of the environments using We model possible structures of the environments using

class diagram.class diagram. The variations of the environments are represented as The variations of the environments are represented as

multiplicities of associations among classes.multiplicities of associations among classes. Constraints on attribute values and multiplicities are described in Constraints on attribute values and multiplicities are described in

OCL (Object Constraint Language).OCL (Object Constraint Language).

• 2 tasks with different priorities.• 1 task and 1 resource.• 2 tasks and 1 resource.• 1 task and 2 resource.....

Possible Environments

invself.res->forall(r|pr=<r.pr)

assertself@Rdy implies GetTaskState(self.id)==GetReady()self@Run implies GetTurn()==self.id

Taskpr : {1..P}states {Sus,Rdy,Run}

Resource

pr : {1..Q}states {Fre,Occ(n:1..M)}

res:(0..N)

tsk:(1..M)

task:(0..M)

Environment ModelingEnvironment Modeling The possible behavior of the tasks and resources are described The possible behavior of the tasks and resources are described

using statechart diagram with some extension.using statechart diagram with some extension. Introducing a Introducing a derived transitionderived transition.. The derived transition causes the state transition of the other instances.The derived transition causes the state transition of the other instances.

| S1 -> S2 {ins} causes the state transition of the instance 'ins' from 'S1' to | S1 -> S2 {ins} causes the state transition of the instance 'ins' from 'S1' to 'S2'.'S2'.

The derived transition makes the statechart model simple.The derived transition makes the statechart model simple.

[tid==self.id & ExRun() & GetPr(GetRun())<self.pr]ActivateTask(tid)| Run->Rdy { GetRun() }

Sus

Run

A transition of task behavior

1. Currently, a task is in the suspended state, and there is another task which is in the run state.

EX) Task 1: Sus, Task 2: Run, ...2. If the task having a priority which is higher than the

running task moves to the run state, the running task moves to the ready state simultaneously.

EX) Task 1: Run, Task 2: Rdy

Environment ModelingEnvironment Modeling The possible behavior of the tasks and resources are The possible behavior of the tasks and resources are

described using statechart diagram with some extension.described using statechart diagram with some extension. We assign an assertion to each state.We assign an assertion to each state. The assertion checks whether an observed state of RTOS is the The assertion checks whether an observed state of RTOS is the

expected one or not. expected one or not.

Statechart Model for Task Behavior

SusRdy Run

[tid==self.id&!ExRun()]ActivateTask(tid)

[tid==self.id&ExRun()& GetPr(GetRun())<self.pr]ActivateTask(tid)| Run->Rdy {GetRun()}

[tid==self.id&ExRun()& (GetPr(GetRun())>=self.pr]ActivateTask(tid)

[tid==self.id&!ExRdy() &!OccRes(self)]TerminateTask(tid)

[tid==self.id&ExRdy()& !OccRes(self)]TerminateTask(tid)| Rdy->Run {GetRdyMax()}

DeclareTask(self.pr,self.id)

Generation of EnvironmentsGeneration of Environments We automatically generate environments from the environment We automatically generate environments from the environment

model.model. A Promela description of an environment is generated for each of the A Promela description of an environment is generated for each of the

variations described in the environment model.variations described in the environment model. Procedure of the generation.Procedure of the generation.

1.1. Generate all the object graphs in a given bound from the class model Generate all the object graphs in a given bound from the class model of the environment model.of the environment model.

2.2. Make the state transition models of each of the object graphs based Make the state transition models of each of the object graphs based on the statechart model of the environment model.on the statechart model of the environment model.

3.3. Translate the state transition models into Promela descriptions.Translate the state transition models into Promela descriptions.

Environment Model

... ...

... ...

Object Graphs

1.Instantiate

... ...

... ...

state transition modelsfor generated object graphs

2.Composing behavior with the evaluation of the constraints

Promela descriptions

...

...

...

...

...

...

...

...

3.Translation

Generation of EnvironmentsGeneration of Environments

1.1. Generate all the object graphs in a given bound Generate all the object graphs in a given bound from the class model of the environment model.from the class model of the environment model.

T1:Taskpr=1

T2:Taskpr=3

R1:Resourcepr=2

T1:Taskpr=1

T2:Taskpr=1

R1:Resourcepr=2

T1:Taskpr=1

T2:Taskpr=3

R1:Resourcepr=3

T1:Taskpr=1

T2:Taskpr=1

R1:Resourcepr=3

T1:Taskpr=1

T2:Taskpr=3

R1:Resourcepr=2

T1:Taskpr=3

T2:Taskpr=1

R1:Resourcepr=3

Taskpr : {1..3}states {Sus,Rdy,Run}

Resource

pr : {2..4}states {Fre,Occ(n:1..2)}

res:(0..3)

task:(0..2)

Environment Model

Instantiate

Object Graphs

...

...

...

Generation of EnvironmentsGeneration of Environments2.2. Make the state transition models of each of the object graphs based Make the state transition models of each of the object graphs based

on the statechart model of the environment model.on the statechart model of the environment model.

h(r)

f(pr)

g(tid)

T1:Taskpr=1

T2:Taskpr=3

R1:Resourcepr=2

Environment Model

Task

p(rid)

Resource

An object graph

q(pr)h(R1)

f(1)

g(T1)

f(2)

g(T2)

T1:Task

T2:Task

p(R1)q(2)

R1:Resouce

Instantiate state transition models based on the environment model.

EnvironmentCompose the state transition models with the evaluation of the constraints.

3.We can directly translate the environment to the Promela description.

Environment GeneratorEnvironment Generator We have implemented an environment generator.We have implemented an environment generator.

Implemented in SML/NJ.Implemented in SML/NJ. The environment model is described in text.The environment model is described in text. Automatically generates Promela descriptions of all the environments.Automatically generates Promela descriptions of all the environments.

Environment model （ rtos.env ）

Promela description （ case1.spin ）

Generation ResultsGeneration Results We generate various environments for the verification of the We generate various environments for the verification of the

RTOS design by our environment generator.RTOS design by our environment generator. It takes very long time to check the RTOS design against all It takes very long time to check the RTOS design against all

the descriptions.the descriptions. Ex) It take 12 hours to check about 1000 descriptions by the Ex) It take 12 hours to check about 1000 descriptions by the

following machine.following machine. CPU: Core2DUO 2.13 GHz, Memory 2Gbyte.CPU: Core2DUO 2.13 GHz, Memory 2Gbyte.

It may take 12*50 = 600h= 25 days (about 1 month).It may take 12*50 = 600h= 25 days (about 1 month).

R/TR/T 11 22 33 44

00 4 (0.0s)4 (0.0s) 20 (0.0s)20 (0.0s) 140 (0.5s)140 (0.5s) 1540 (81.3s)1540 (81.3s)

11 8 (0.0s)8 (0.0s) 104 (0.2s)104 (0.2s) 1496 (31.6s)1496 (31.6s) 30664 (9.4h)30664 (9.4h)

22 12 (0.1s)12 (0.1s) 468 (2.6s)468 (2.6s) 15132 (56.1m)15132 (56.1m) N/AN/A

33 16 (0.1s)16 (0.1s) 1840 (61.4s)1840 (61.4s) N/AN/A N/AN/A

The number of generated Promela descriptions

N/A: It takes too long..

Verification by Computer ClusterVerification by Computer Cluster

We are doing model checking by computer cluster.We are doing model checking by computer cluster. Check a large number of the Promela descriptions by many workstations.Check a large number of the Promela descriptions by many workstations.

Currently, we are using collaborative facilities for verification, named Currently, we are using collaborative facilities for verification, named 'SATSUKI' in AIST.'SATSUKI' in AIST. We did some experiments.We did some experiments. We checked all the description by the following cluster.We checked all the description by the following cluster.

The number of nodes: 70The number of nodes: 70 Sun Fire X4150, Xeon X5260 3.3GHz Dual Core, 8Gbyte Memory.Sun Fire X4150, Xeon X5260 3.3GHz Dual Core, 8Gbyte Memory.

About 6 hours.About 6 hours.

Environment Modeling Display the result

Computer Cluster

Environment Generator/Distributing descriptions to cluster.

Accumulate results and analyze them.

Verification ResultsVerification Results We found several bugs of the RTOS design by this approach.We found several bugs of the RTOS design by this approach.

We first checked the RTOS design by some environments and remove We first checked the RTOS design by some environments and remove found bugs.found bugs.

Then, we verified it by this approach.Then, we verified it by this approach. A found bug.A found bug.

Structure of tasks and resources.Structure of tasks and resources. Tasks: T1 with the priority 3, T2 with the priority 1.Tasks: T1 with the priority 3, T2 with the priority 1. Resources: R1 with the priority 4, R2 with the priority 2.Resources: R1 with the priority 4, R2 with the priority 2.

R1 is used by T1 and T2. R2 is used by only T2.R1 is used by T1 and T2. R2 is used by only T2. This structure is somewhat special.This structure is somewhat special.

In this case, the priority ceiling protocol was not correctly realized in the In this case, the priority ceiling protocol was not correctly realized in the RTOS design.RTOS design.

TCB was not updated when a task returns a resource for a specific TCB was not updated when a task returns a resource for a specific execution sequence.execution sequence.

We made a mistake in condition branches .We made a mistake in condition branches .

DiscussionDiscussion Our approach can be regarded as bounded model Our approach can be regarded as bounded model

checking.checking. Variations are described in the class diagram.Variations are described in the class diagram. We instantiate a part of them by bounding the multiplicities We instantiate a part of them by bounding the multiplicities

and attribute values.and attribute values.

→ → It is useful to find bugs in the RTOS design.It is useful to find bugs in the RTOS design. Comparing the model checking results with each other Comparing the model checking results with each other

help us to find the bugs.help us to find the bugs. We need theorem proving to ensure that the RTOS We need theorem proving to ensure that the RTOS

design is correct for all the variations.design is correct for all the variations. If we focus on a specific RTOS, the variations may finite.If we focus on a specific RTOS, the variations may finite. However, state explosion problem may occur because However, state explosion problem may occur because

RTOS usually deals with hundreds of tasks.RTOS usually deals with hundreds of tasks.

DiscussionDiscussion Our approach makes it possible to do model checking in parallel.Our approach makes it possible to do model checking in parallel.

We statically divide state space to be checked according to the We statically divide state space to be checked according to the variations of the environments.variations of the environments.

We have advantage to searching larger state space of the RTOS We have advantage to searching larger state space of the RTOS design.design. Parts of the state spaces checked by the generated environments are Parts of the state spaces checked by the generated environments are

overlapped.overlapped. We still have an advantage in the state space.We still have an advantage in the state space.

Many results are obtained.Many results are obtained. It is very hard to check all of them precisely.It is very hard to check all of them precisely.

100,000 checks may return 100,000 counter examples.100,000 checks may return 100,000 counter examples. Conversely, we obtain much information about the checks.Conversely, we obtain much information about the checks.

Applying statistic methods such as cluster analysis and machine learning for Applying statistic methods such as cluster analysis and machine learning for analyzing the results is meaningful.analyzing the results is meaningful.

Visualizing the results allows us to intuitively understand what happens in the Visualizing the results allows us to intuitively understand what happens in the RTOS design.RTOS design.

DiscussionDiscussion We carefully made the environment model based on the OSEK/VDX

specification document. Extracting descriptions of the tasks scheduling from the document

Those descriptions are scattered in the document. We make pre/post conditions of each system calls from those descriptions.

The conditions are described in natural language mixed with logical formulas. We construct the statechart model based on the pre/post conditions.

Formal treatments are needed. Some parts of the document are ambiguous. Removing mistakes made in the transformations of the document.

OSEK/VDXspecificationdocument

Extract descriptionof task scheduling

descriptionof task scheduling

descriptionof task scheduling

ActivateTaskpre: ...post: ....

ChainTaskpre: ...post: ....

TerminateTaskpre: ...post: ....

pre/post

pre/post

pre/post

ActivateTask

ChainTask

TerminateTask

Ongoing WorksOngoing Works Model checking a huge number of descriptions using

computer cluster. How to assign the descriptions to machines of the computer

cluster. Achieve the load balance of the machines.

How to analyze results obtained by model checking. Visualization of the results. statistical analysis of the results.

Formalization of the OSEK/VDX specification document. We are describing the document in VDM.

Identifying what problems are to formalize standard documents. We will start to study about desirable styles of standard

documents in a formal specification language. Working with Prof.Kokichi Futatsugi's group using Cafe/OBJ.

Ongoing WorksOngoing Works We are verifying a RTOS which will be embedded in a

new series of cars. Collaborating with DENSO and NEC(NEC Electronics and NEC

Microsystems) Conforming to ISO61508/26262 standards. Improving the quality of RTOS.

Mainly dealing with the design and testing of RTOS.

RTOS design

UML diagrams

Promela descriptions

Review and Model Checking

RTOS

Models for testing

Test case generator

Test Cases

Test Cases

Testing

CConclusiononclusion WWe introduced our approach to verifying a RTOS design e introduced our approach to verifying a RTOS design

with model checking.with model checking. MModeling environments of RTOS.odeling environments of RTOS. AAutomatic genenration of Promela descriptions for model utomatic genenration of Promela descriptions for model

checking the RTOS design.checking the RTOS design. MModel checking is performed in computer cluster.odel checking is performed in computer cluster.

OOur environment modeling is applicable not only to the ur environment modeling is applicable not only to the OSEK/VDX RTOS design but also to the other ones.OSEK/VDX RTOS design but also to the other ones. TThat will be effective to systems which have various hat will be effective to systems which have various

environments such as OS, middleware and libraries.environments such as OS, middleware and libraries. We are developing tools which perform model checking in We are developing tools which perform model checking in

computer cluster, then analyze and visualize those results.computer cluster, then analyze and visualize those results. We are applying our approach to practical OSEK/VDX We are applying our approach to practical OSEK/VDX

RTOS with some automotive companies. RTOS with some automotive companies.

Testing specification called MODISTARC Testing specification called MODISTARC is provided to ensure the conformance of is provided to ensure the conformance of RTOS to the OSEK/VDX standard.RTOS to the OSEK/VDX standard.MODISTARC does not provide test cases but MODISTARC does not provide test cases but

functions to be tested.functions to be tested.We need define test cases based on We need define test cases based on

MODISTARC.MODISTARC.

It is very hard to define exhaustive test cases It is very hard to define exhaustive test cases based on MODISTARC.based on MODISTARC.The test cases depend on RTOS products.The test cases depend on RTOS products.