real-time systems specification and test ofpeople.cs.aau.dk/~bnielsen/testudbud/defense.pdf · 1....

1

Specification and Test of Real-Time Systems

Ph.D. Lecture by

Brian Nielsen

2

What is a Real-Time System?• Correctness depends on absolute timing of events

• ”when the temperature reaches 90º, the valve to the cooling water must open within 1 sec”

• Communication with external physical environment• Inherently Complex• How to develop real-time systems correctly?

– Requirements must be clear and concise (formal)– Tools to deal with complexity

Coolingwater

Computersystem

temperature

Open/close valve

tank

3

Why Improve Testing?• Costly (30% of development time)• Significant errors not found• The validation activity used by

industry today

4

What is Testing?•Testing: execution of an implementation with the

purpose of finding errors•Conformance testing: does the behavior of a system comply to the

specification•Black-box testing: only external behavior is visible/relevant

A Specification Test cases

Click!Wait 0.1click!DBLClick?

(pass)


(pass)

click?x:=0

click?x<2

x>=2

DBLclick! Click!Wait 1.5click!DBLClick?

(pass)


(pass)

Click!Wait 5click!DBLClick?

(fail)

Click!Wait 5click!DBLClick?

(fail)

5

How do we cope with real-life specs?

isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662




in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0

up!up!up!up!up!up!up!up!up!up!up!up!up!up!up!up!up!

in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662

dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!

up!up!up!up!up!up!up!up!up!up!up!up!up!up!up!up!up!Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662



in0?in0?in0?in0?in0?in0?in0?in0?in0?in0?in0?in0?in0?in0?in0?in0?in0?Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324






dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662


in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324

empty?empty?empty?empty?empty?empty?empty?empty?empty?empty?empty?empty?empty?empty?empty?empty?empty?Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662


dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1

dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1


empty?empty?empty?empty?empty?empty?empty?empty?empty?empty?empty?empty?empty?empty?empty?empty?empty?Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324Xdn<=9324



isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880


coll!coll!coll!coll!coll!coll!coll!coll!coll!coll!coll!coll!coll!coll!coll!coll!coll!

isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1

s3s3s3s3s3s3s3s3s3s3s3s3s3s3s3s3s3s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1

s4s4s4s4s4s4s4s4s4s4s4s4s4s4s4s4s4

s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s8s8s8s8s8s8s8s8s8s8s8s8s8s8s8s8s8


s0to1s0to1s0to1s0to1s0to1s0to1s0to1s0to1s0to1s0to1s0to1s0to1s0to1s0to1s0to1s0to1s0to1



s0s0s0s0s0s0s0s0s0s0s0s0s0s0s0s0s0 s1s1s1s1s1s1s1s1s1s1s1s1s1s1s1s1s1






configconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigsystem sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;

sendersendersendersendersendersendersendersendersendersendersendersendersendersendersendersendersender

Philips Sender with collision detection

6

Automated Testing• Automatic generation of test cases from

(formal) specifications• Automatic execution of test cases• Commercial tools becoming available• Cases:

– International Space Station [Peleska et al. ’99]

– SSCOP protocol [Jard et al. ’00]– Satelite power and thermal control system [Schlingloff ’97]

• Little support for real-time constraints

7

Analysis of Real-Time Systems• Model Checking aims at automatically

proving real-time properties of an abstract model of a system

• Efficient symbolic techniques for reach-ability analysis [Dill’89, Larsen et al. ’96, Larsen et al. ’97]– Constraint solving (Zones, DBM, CDD)

• UppAal• Numerous successful industrial applications

– Philips audio protocol [Larsen et al. ’96],

– B&O power controller [Skou et al. ’99],

– …

8

My Thesis• The symbolic reachability techniques

developed for model checking can also be used for automatic generation of tests.

• Can systematically generate tests• Can handle cases of realistic size

9

Contributions• Basic algorithms for test generation from CSM

specifications• Framework for selection of real-time test cases• Timed Test generation algorithms

– Direct interpretation of TA specifications– From a restricted class of TA, a guaranteed covering

test suite wrt. a selection criterion• Tool implementation RTCAT• Applications and evaluation• Language support for modular and reusable

specifications– Functional components – Real-time and synchronization constraint patterns

10

Outline1. Introduction2. Overview of untimed techniques3. Timed

1. Timed testing setup2. Systematic algorithm (ERA)

4. Evaluation5. Conclusions + Future work

11

Test Suite

Ingredients in Automated Testing

Testing Theory Selection StrategyTools

-Verdicts-Measured Coverage-Diagnostic info

-Data structures-Algorithms

-Correctness criterion-Model of Tests -Relevant tests

-Test purposes-Impl/spec coverage-Fault models

-Abstract tests-Executable tests

Specifications-Required behavior-Environment assumptions

Test Execution

12

Test Suite

Work On Untimed Testing

Testing Theory Selection StrategyTool

-TestGen-Succes Graph

-Hennessy -(Coverage of SG)

Specification-Com. State Machines-(LTS)

13

• Shared variables, enabling conditions, assignments• CCS-style parallel composition with synchronous

communication• Semantics Given as Labeled Transition System

coin? coin?

cof!tea!s0

s1 s2

LTS for Non-deterministic Coffee Machine

Specifications

Communicating State Machines

14

Tests and Test Execution• Test: LTS with state based verdicts

– Pass, fail, (Inconclusive)• Test Execution:

– Parallel composition of tester and implementation– Synchronous communication on complementary

actions

a!

b!

c!

a?

b?

c?

a!

b?

c!

(fail)

(fail)

(fail)

(pass)

ImplementationTester

Testing Theory

15

Implementation Relations

• Testing Preorder [Hennessy ’84]

• Conf [Brinksma ’88], Ioconf [Tretmans ’96]

•

1. S must T iff ∀Σ ∈ Comp(T ‖ S). Σ is successful.

2. S may T iff ∃Σ ∈ Comp(T ‖ S). Σ is successful.

3. S �must I iff ∀ T ∈ Ltlts. S must T implies I must T

4. S �may I iff ∀ T ∈ Ltlts. S may T implies I may T

5. S �te I iff S �must I and S �may I

S �te I iff I conf S ∧ Tr(I) ⊆ Tr(S) ∧ Tr(S) ⊆ Tr(I)

Testing Theory

16

Hennessy Testers

1. Find a trace σ in spec2. Compute B=S after σ3. Find a set of actions A st. B must A

}{ .. 11 nn aabb mm mustafter

1b

nb

na1a

nbbb ... 21 mcan

1b

nb

∅=

....

1

1

mustaftercannot

n

n

bbbb

m

m

1b

nb

…Generate ”all” such test passed by the specification

Testing Theory

17

Direct Interpretation

coin? coin?

cof!tea!s0

s1 s2

coin!{s0} M={ {coin} }, C={coin}, R={tea,cof}

M={ {tea,cof} }, C={tea,cof}, R={coin}{s1,s2}cof?tea?

…

coin!{s0} M={ {coin} }, C={coin}, R={tea,cof}

{s1,s2}cof?tea?

{s0}

M={ {tea,cof} }, C={tea,cof}, R={coin}

M={ {coin} }, C={coin}, R= {tea,cof}tea? cof?

•Scales well (no state explosion)•Systematic?

Algorithms

18

• A success graph is an LTS– τ-reduced, Determinized, Trace Equivalent – States labeled with: Must, Can, Refusal sets

• Eases systematic generation– Mark tests already generated– Measuring coverage– (Is a good selection criterion)

• State explosion Problem!

Success Graph

coin? coin?

cof!tea!s0

s1 s2

coin?

cof!tea!{s0}

{s1,s2}

M={ {coin} }, C={coin},R={tea,cof}

M={ tea,cof} },C={tea,cof},R={coin}

Algorithms

19

Test Suite

About Time …

Testing Theory Selection StrategyTest Generation Tool

-RTCAT-Timed Hennessy Tests -Equivalence classes-Guaranteed Coverage

-Abstract tests (TA)

Specification-Timed Automata-ERA

20

Timed Automata

},,,{~,~and~ >≥≤<∈− cxcxx ijji

s1coin?,x:=0

2<=x<=4, give?,x:=0

x>=4,give?,x:=0

x>=2, cof! x>=1,thinCof!

s2

s3 s4

s5 s6

Permitted Guards:

Specifications

21

Our Timed Hennessy Tests}{ )(.).(..).( 1111 nnnn aadbdbd mm mustafter +εεε

0:,, 11 == xdxb

0:,, == xdxb nn

0:

,, 1

=

= +

xdxa nn

0:

,, 11

=

= +

xdxa n

nn bdbd ).(..).( 11 εε mcan ∅=

).(..).().(..).(

11

11

mustaftercannot

nn

nn

bdbdbdbd

εεεε

m

m

0:,, 11 == xdxb

0:,, == xdxb nn

0:,, 11 == xdxb

0:,, == xdxb nn

Tester and implementation communicatesusing urgent synchronous actions

Testing Theory

22

A Direct Interpretation• Similar to the untimed case, except

for choice of delay• Which delay to choose ?

– All– Random– Periodic

• The sets after ε(d) and after a may be infinite– Must be computed symbolically

Algorithms

23

Zones and Timed Automata• A zone is a conjunction of linear in-equations

of the form:

• Forms convex polyhedra

• Semantic state: – l is a location– u is a vector of current clock values

• Symbolic State: – l is a location – Z is a zone (solution set of clock vectors)

},,,{~,~and~ >≥≤<∈− cxcxx ijji

),( ul

],[ Zl

Algorithms

24

X

Y

Symbolic Execution

X

z

67

56

45

34

),(),(

zzzxzfreez

xzborderzzz

∧==== ↓

l1 l2 l3

a, x:=0

Y

X

z7z 6z

Backwardstep

Forwardstep

a, x:=0

Y

3z

4z5z

X

Y

b

x

gzzzz

zz

∧==

=↑

=

23

12

0:1

1z

2z

3zbg

],[ 1 Zlbgb ,0:, =xa

1l

1l 2l

2l

Algorithms

25

A Timed Success Graph• Goal

– Make it easy to generate timed Hennessy tests• Finding traces• Deadlock properties• Verdicts

– Ease systematic generation of tests wrt a coverage criterion

Algorithms

26

Bizarre Results from TA-theory

• Non-deterministic TAs are more expressive than deterministic ones– A non-deterministic TA cannot be determinized– Language inclusion is undecidable

• TAs with internal (τ-actions) are more expressive than TAs without– τ-edges with clock resets lying on a directed cycle cannot

be removed– other τ-edges can be removed, but may cause a blow-up

in the number of locations

In contrast to finite state automata:

[Diekert et al. ´97]

[Alur et al. ´94]

Simple clean subset of TA wanted!

Specifications

27

Event Recording Automata (ERA)• For each action a, Xa is the event clock

for a– Xa is reset when a occurs– Xa records the amount of time since last

occurrence of a – No other assignments permitted– No internal actions/synchronization

• Determinizable, but strict subclass of TA

• Clean model: event clocks

[Alur, Fix, Henzinger ‘94]

Specifications

28

Coffee VendingMachine ERA

{s3,s4}

{s2}

{s1}

{s3} {s4}

{s5} {s6}

coin?

give?2<=Xcoin<4

give?Xcoin>4

give?Xcoin=4

cof!Xgive>=2

cof!Xgive>=2

thinCof!Xgive>=1

thinCof!Xgive>=1

s1

Coin?

2<=Xcoin<=4give?

Xcoin>=4give?

XGive>=2cof!

Xgive>=1,thinCof!

s2

s3 s4

s5 s6

DeterminizedERA

Specifications

29

Domain Testing

• Partition input space into equivalence classes

• Apply uniformity hypothesis• Use extreme values to support

uniformity hypothesis

Selection Strategy

int abs(int x);

0,0,{ <xif

otherwizex

expected Input output

5 5-5 00 01 1-1 0

30

Stable Edge Set Partitioning{s3,s4}

{s5} {s6}

cof!Xgive>=2

thinCof!Xgive>=1

•When enabledness of edges changes, implementation should match this change

•Specification is used as implementation guide => implementation can be expected to be stable

•Equivalent wrt. Must, Can, and Refusal sets

Xgive

cannot coincannot givecannot cofcannot tea

1 2 3 4

cannot coincannot givemust{tea,cof}

cannot coincannot cofcannot givecan tea

1

2

3

Xcoin{s3,s4}

Can teaCan cof

1Q

2Q

3Q

x

xx

Selection Strategy

31

Selection Criterion

• A complete Covering Test Suite 1. For each (reachable) equivalence class Q2. For one point in Q via some trace σ3. Makes all observation M, C, R in S after σ

Xgive

cannot coincannot givecannot cofcannot tea

1 2 3 4

cannot coincannot givemust{tea,cof}

cannot coincannot cofcannot givecan tea

1

2

3

Xcoin{s3,s4}

Can teaCan cof

x)}0,{( 0l

σ

Selection Strategy

32

Equivalence Class Graph (CofM)Algorithms

[{s1}, tt]

[{s2}, Xcoin=4]

[{s2}, 2<=Xcoin<4]

[{s2}, Xcoin>4]

[{s2}, Xcoin<2]

[{s3,s4}, Xgive>=2]

[{s3,s4}, 1<=Xgive<2]

[{s3,s4}, 1<=Xgive<1]

[{s4}, Xgive>=1]

[{s4}, Xgive<1]

[{s3}, Xgive>=2]

[{s3}, Xgive<2]

[{s6}, tt]

[{s5}, tt]

coin

give thinCof

thinCof

thinCof

cof

cof

give

give

give

give

give

give

coin

coin

coin

33

Reachability GraphAlgorithms

[{s1}, tt]

[{s2}, Xcoin=4]

[{s2}, 2<=Xcoin<4]

[{s2}, Xcoin>4]

[{s2}, Xcoin<2]

[{s3,s4}, Xgive>=2]

[{s3,s4}, 1<=Xgive<2]

[{s3,s4}, 1<=Xgive<1]

[{s4}, Xgive>=1]

[{s4}, Xgive<1]

[{s3}, Xgive>=2]

[{s3}, Xgive<2]

[{s6}, tt]

[{s5}, tt]

coin

give thinCof

thinCof

thinCof

cof

cof

give

give

give

give

give

give

coin

coin

coin

34

Algorithms

[{s1}, tt]

[{s2}, Xcoin=4]

[{s2}, 2<=Xcoin<4]

[{s2}, Xcoin>4]

[{s2}, Xcoin<2]

[{s3,s4}, Xgive>=2]

[{s3,s4}, 1<=Xgive<2]

[{s3,s4}, 1<=Xgive<1]

[{s4}, Xgive>=1]

[{s4}, Xgive<1]

[{s3}, Xgive>=2]

[{s3}, Xgive<2]

[{s6}, tt]

[{s5}, tt]

coin

give thinCof

thinCof

thinCof

cof

cof

give

give

give

give

give

give

coin

coin

coin

Back-Propagate

35

Select DelaysAlgorithms

[{s1}, tt]

[{s2}, Xcoin=4]

[{s2}, 2<=Xcoin<4]

[{s2}, Xcoin>4]

[{s2}, Xcoin<2]

[{s3,s4}, Xgive>=2]

[{s3,s4}, 1<=Xgive<2]

[{s3,s4}, 1<=Xgive<1]

[{s4}, Xgive>=1]

[{s4}, Xgive<1]

[{s3}, Xgive>=2]

[{s3}, Xgive<2]

[{s6}, tt]

[{s5}, tt]

coin

give thinCof

thinCof

thinCof

cof

cof

give

give

give

give

give

give

coin

coin

coin

36

Generated Tests for CofM• 16 test cases

Xcoin=100coin!

Xcoin=2give!

XthinCof=100Act

Xgive=101thinCof!

Xcoin=100coin!

Xcoin=105give!

Xcof=100Act

Xgive=102cof!

Xcoin=100coin!

Xcoin=4give!

Xgive=1thinCof!

Xcoin=100coin!

Xcoin=4give!

Xgive=102thinCof!

Xgive=102cof!

Test Suite

37

Evaluation• Specifications

– Philips Audio Protocol – Pedestrian Crossing– Token-Bus

• Event Recording Automata as Specification Language

• Testing Preorder, test language• Implementability• Tool Performance

– Number of tests– Traversal order– Memory+time usage

38

Tool Developed• Networks of Event Recording Automata

sharing clocks and discrete variables• Traversal order

– depth- and breadth-first construction of EQC-graph and reachability graph

• Composed or Individually generated Hennessey Tests

• Partial search methods– limit trace depth– use bit-state hashing on equivalence class and

reachability graph• 22 Kloc in C++

39

The Philips Audio Protocol• A bus based protocol for exchanging

control messages between audio components– Collisions– Tolerance on timing events

01 1 10 0 0Bit streamManchester encoding

TX RX

TX RX TX RX

40

Philips Sender

withcollision

detectionisUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn>=4018Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662




in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000Xdn>=80000sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0sent:=0




up!up!up!up!up!up!up!up!up!up!up!up!up!up!up!up!up!Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn>=4218Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662Xdn<=4662









dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662


in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?in1?Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=8436Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324Xdn>=9324



dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1

dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!dn!Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup>=4218Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662Xup<=4662sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1sent:=1





isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6440Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880Xdn<=6880


coll!coll!coll!coll!coll!coll!coll!coll!coll!coll!coll!coll!coll!coll!coll!coll!coll!

isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?isUp?sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1sent==1

s3s3s3s3s3s3s3s3s3s3s3s3s3s3s3s3s3s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1s1to1


s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s1to0s8s8s8s8s8s8s8s8s8s8s8s8s8s8s8s8s8





s0s0s0s0s0s0s0s0s0s0s0s0s0s0s0s0s0 s1s1s1s1s1s1s1s1s1s1s1s1s1s1s1s1s1






configconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigsystem sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;system sender;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;int sent;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;observable in1, in0, up, dn, empty, isUp, coll;

sendersendersendersendersendersendersendersendersendersendersendersendersendersendersendersendersender

41

Philips Receiver




L0L0L0L0L0L0L0L0L0L0L0L0L0L0L0L0L0


end!end!end!end!end!end!end!end!end!end!end!end!end!end!end!end!end!

end!end!end!end!end!end!end!end!end!end!end!end!end!end!end!end!end!odd==1odd==1odd==1odd==1odd==1odd==1odd==1odd==1odd==1odd==1odd==1odd==1odd==1odd==1odd==1odd==1odd==1XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979

out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!odd==0odd==0odd==0odd==0odd==0odd==0odd==0odd==0odd==0odd==0odd==0odd==0odd==0odd==0odd==0odd==0odd==0XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP>=18981XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979XVUP<=20979

out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP<=16317XVUP<=16317XVUP<=16317XVUP<=16317XVUP<=16317XVUP<=16317XVUP<=16317XVUP<=16317XVUP<=16317XVUP<=16317XVUP<=16317XVUP<=16317XVUP<=16317XVUP<=16317XVUP<=16317XVUP<=16317XVUP<=16317

VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP>=14736XVUP<=18981XVUP<=18981XVUP<=18981XVUP<=18981XVUP<=18981XVUP<=18981XVUP<=18981XVUP<=18981XVUP<=18981XVUP<=18981XVUP<=18981XVUP<=18981XVUP<=18981XVUP<=18981XVUP<=18981XVUP<=18981XVUP<=18981





out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!

out1!out1!out1!out1!out1!out1!out1!out1!out1!out1!out1!out1!out1!out1!out1!out1!out1!

out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!out0!odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1

out1!out1!out1!out1!out1!out1!out1!out1!out1!out1!out1!out1!out1!out1!out1!out1!out1!odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1odd:=-odd+1

VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?VUP?odd:=0odd:=0odd:=0odd:=0odd:=0odd:=0odd:=0odd:=0odd:=0odd:=0odd:=0odd:=0odd:=0odd:=0odd:=0odd:=0odd:=0

L1L1L1L1L1L1L1L1L1L1L1L1L1L1L1L1L1

s1s1s1s1s1s1s1s1s1s1s1s1s1s1s1s1s1idleidleidleidleidleidleidleidleidleidleidleidleidleidleidleidleidle

system receiver;system receiver;system receiver;system receiver;system receiver;system receiver;system receiver;system receiver;system receiver;system receiver;system receiver;system receiver;system receiver;system receiver;system receiver;system receiver;system receiver;int odd;int odd;int odd;int odd;int odd;int odd;int odd;int odd;int odd;int odd;int odd;int odd;int odd;int odd;int odd;int odd;int odd;observable VUP, out1, out0, end; observable VUP, out1, out0, end; observable VUP, out1, out0, end; observable VUP, out1, out0, end; observable VUP, out1, out0, end; observable VUP, out1, out0, end; observable VUP, out1, out0, end; observable VUP, out1, out0, end; observable VUP, out1, out0, end; observable VUP, out1, out0, end; observable VUP, out1, out0, end; observable VUP, out1, out0, end; observable VUP, out1, out0, end; observable VUP, out1, out0, end; observable VUP, out1, out0, end; observable VUP, out1, out0, end; observable VUP, out1, out0, end;

receiverreceiverreceiverreceiverreceiverreceiverreceiverreceiverreceiverreceiverreceiverreceiverreceiverreceiverreceiverreceiverreceiver

configconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfig

42

Token Passing3• A station may send only when it holds a token• It may hold the token for at most 100 time units• Token must be passed on to next station in the ring

I am a token

43

Token Passing3

th=1th=1th=1th=1th=1th=1th=1th=1th=1th=1th=1th=1th=1th=1th=1th=1th=1GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2

RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!

send1!send1!send1!send1!send1!send1!send1!send1!send1!send1!send1!send1!send1!send1!send1!send1!send1!XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100 send1!send1!send1!send1!send1!send1!send1!send1!send1!send1!send1!send1!send1!send1!send1!send1!send1!

XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100XGT1<100

XGT1=100XGT1=100XGT1=100XGT1=100XGT1=100XGT1=100XGT1=100XGT1=100XGT1=100XGT1=100XGT1=100XGT1=100XGT1=100XGT1=100XGT1=100XGT1=100XGT1=100RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!RT1!th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2th:=2XGT0=100XGT0=100XGT0=100XGT0=100XGT0=100XGT0=100XGT0=100XGT0=100XGT0=100XGT0=100XGT0=100XGT0=100XGT0=100XGT0=100XGT0=100XGT0=100XGT0=100

RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1

th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1th:=1RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!RT0!

send0!send0!send0!send0!send0!send0!send0!send0!send0!send0!send0!send0!send0!send0!send0!send0!send0!XGT0<100XGT0<100XGT0<100XGT0<100XGT0<100XGT0<100XGT0<100XGT0<100XGT0<100XGT0<100XGT0<100XGT0<100XGT0<100XGT0<100XGT0<100XGT0<100XGT0<100


th=0th=0th=0th=0th=0th=0th=0th=0th=0th=0th=0th=0th=0th=0th=0th=0th=0GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0? XGT2=100XGT2=100XGT2=100XGT2=100XGT2=100XGT2=100XGT2=100XGT2=100XGT2=100XGT2=100XGT2=100XGT2=100XGT2=100XGT2=100XGT2=100XGT2=100XGT2=100

RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0



th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0th:=0RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!RT2!

th=2th=2th=2th=2th=2th=2th=2th=2th=2th=2th=2th=2th=2th=2th=2th=2th=2GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?

th=0th=0th=0th=0th=0th=0th=0th=0th=0th=0th=0th=0th=0th=0th=0th=0th=0GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0?GT0? th=1th=1th=1th=1th=1th=1th=1th=1th=1th=1th=1th=1th=1th=1th=1th=1th=1

GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?GT1?th=2th=2th=2th=2th=2th=2th=2th=2th=2th=2th=2th=2th=2th=2th=2th=2th=2GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?GT2?


s3s3s3s3s3s3s3s3s3s3s3s3s3s3s3s3s3 s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s3s3s3s3s3s3s3s3s3s3s3s3s3s3s3s3s3


s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s2s3s3s3s3s3s3s3s3s3s3s3s3s3s3s3s3s3

s1s1s1s1s1s1s1s1s1s1s1s1s1s1s1s1s1station2station2station2station2station2station2station2station2station2station2station2station2station2station2station2station2station2station1station1station1station1station1station1station1station1station1station1station1station1station1station1station1station1station1

configconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigconfigsystem station0, station1, station2;system station0, station1, station2;system station0, station1, station2;system station0, station1, station2;system station0, station1, station2;system station0, station1, station2;system station0, station1, station2;system station0, station1, station2;system station0, station1, station2;system station0, station1, station2;system station0, station1, station2;system station0, station1, station2;system station0, station1, station2;system station0, station1, station2;system station0, station1, station2;system station0, station1, station2;system station0, station1, station2;observable GT0, RT0, send0, GT1, RT1, send1, GT2, RT2, send2;observable GT0, RT0, send0, GT1, RT1, send1, GT2, RT2, send2;observable GT0, RT0, send0, GT1, RT1, send1, GT2, RT2, send2;observable GT0, RT0, send0, GT1, RT1, send1, GT2, RT2, send2;observable GT0, RT0, send0, GT1, RT1, send1, GT2, RT2, send2;observable GT0, RT0, send0, GT1, RT1, send1, GT2, RT2, send2;observable GT0, RT0, send0, GT1, RT1, send1, GT2, RT2, send2;observable GT0, RT0, send0, GT1, RT1, send1, GT2, RT2, send2;observable GT0, RT0, send0, GT1, RT1, send1, GT2, RT2, send2;observable GT0, RT0, send0, GT1, RT1, send1, GT2, RT2, send2;observable GT0, RT0, send0, GT1, RT1, send1, GT2, RT2, send2;observable GT0, RT0, send0, GT1, RT1, send1, GT2, RT2, send2;observable GT0, RT0, send0, GT1, RT1, send1, GT2, RT2, send2;observable GT0, RT0, send0, GT1, RT1, send1, GT2, RT2, send2;observable GT0, RT0, send0, GT1, RT1, send1, GT2, RT2, send2;observable GT0, RT0, send0, GT1, RT1, send1, GT2, RT2, send2;observable GT0, RT0, send0, GT1, RT1, send1, GT2, RT2, send2;int th;int th;int th;int th;int th;int th;int th;int th;int th;int th;int th;int th;int th;int th;int th;int th;int th;

station0station0station0station0station0station0station0station0station0station0station0station0station0station0station0station0station0

44

Experimental resultsCofM PedX Philips (R) Philips (S) Token7

Equivalence Classes 14 19 60 59 42Breadth First

Symbolic States 17 77 71 120 15427Time (s) 1 2 1 2 541Memory (MB) 5 5 5 5 40C-Number of Tests 16 30 97 99 71C-Total Length 45 151 527 548 574I-Number of Tests 22 39 118 116 84I-Total Length 58 188 614 622 665

Depth FirstSymbolic States 17 247 85 119 7283Time (s) 1 4 2 3 158Memory (MB) 5 5 5 5 24C-Number of Tests 16 26 86 95 60C-Total Length 45 595 1619 1154 5290I-Number of Tests 22 39 118 116 84I-Total Length 58 676 2103 1250 6321

45

Number of Generated Tests• Findings

– The number of generated tests in all configurations is very reasonable

– The number of equivalence classes is very reasonable

– Test composition (only) reduces test suite size by 10-25%

• Conclusions– Selection strategy feasible for specs of similiar

size– Plenty room for more tests

• Extreme value selection• Longer traces• Larger specifications

46

Traversal Order• Findings

– Depth-first: fewer but much larger test suites– Breadth-first: more tests, but much shorter test

suites– (Depth-first some times more efficient than BF)

• Conclusions– Use BF when an economic covering test suite is

the goal– Use DF when check of behavior after long

sequences is the goal

47

Space and Time usage• Findings

– Except for Token Passing(n) the memory and time usage is insignificant

– Token Passing is problematic– A huge number of symbolic states needed to

terminate reachability analysis– Number of clocks has severe impact

• Conclusions– Symbolic reachability analysis (and EQC graph

construction) is a bottleneck

48

• Specification language related– More active clocks compared to TA spec– More locations in ERAs compared to TA

• Implementation related– Reachability performed on EQC-graph– Representation of concave sets as several zones– Inefficient zone implementation compared to

uppaal2k (>100*)

Possible explanations

49

Conclusions• Yes, the symbolic methods can be used for

test generation– Proposed a systematic technique– Guarantees coverage wrt. coverage criterion

• Reasonable sized, practically relevant systems could be handled– Applicable ”as is” for strictly timed controllers

• Current tools can be improved• Scalability to very large specifications

could not be demonstrated.

50

Short Term Future Work• Further Case Studies + test execution• Add Timing Uncertainty + symbolic test cases• Distinguish Input and Output actions• Add modeling of environment assumptions + test

purposes• Loosen ERA restrictions• New Engine for constraint solving • Moderate effort will enable an even

larger application domain

51

Long Term Future Work• Hybrid Systems• Performance Testing

52

A Continuous Time Testing-Preorder?

• Timed Hennessy tests does not fully characterize testing power of arbitrary timed automata

• Proposal for discrete Timed Process Language [Hennessy & Regan ’95]– ”extending the theory of tests for a language where time

is not discrete will be a major challenge.”• Conjecture:

(x<=C)

ττττ, x>=Ca_1, x<C

a_n, x<C

T(a_n,x)T(a_1,x)

…

T(¬A,x)

53

Pedestrian crossing• Green time is at least 30 sec for cars and 20 for

pedestrians • 5 seconds red time in each direction at each switch• Pedestrians requests passage• Pedestrians may extend their green time with 20 sec at

most 3 times in succession

54

Pedestrian Crossing Controller

req?req?req?req?req?req?req?req?req?req?req?req?req?req?req?req?req?rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1

car_rd!car_rd!car_rd!car_rd!car_rd!car_rd!car_rd!car_rd!car_rd!car_rd!car_rd!car_rd!car_rd!car_rd!car_rd!car_rd!car_rd!Xcar_gr>=30Xcar_gr>=30Xcar_gr>=30Xcar_gr>=30Xcar_gr>=30Xcar_gr>=30Xcar_gr>=30Xcar_gr>=30Xcar_gr>=30Xcar_gr>=30Xcar_gr>=30Xcar_gr>=30Xcar_gr>=30Xcar_gr>=30Xcar_gr>=30Xcar_gr>=30Xcar_gr>=30rq=1rq=1rq=1rq=1rq=1rq=1rq=1rq=1rq=1rq=1rq=1rq=1rq=1rq=1rq=1rq=1rq=1

req?req?req?req?req?req?req?req?req?req?req?req?req?req?req?req?req?rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1rq:=1

req?req?req?req?req?req?req?req?req?req?req?req?req?req?req?req?req?rq<=3rq<=3rq<=3rq<=3rq<=3rq<=3rq<=3rq<=3rq<=3rq<=3rq<=3rq<=3rq<=3rq<=3rq<=3rq<=3rq<=3Xreq<20Xreq<20Xreq<20Xreq<20Xreq<20Xreq<20Xreq<20Xreq<20Xreq<20Xreq<20Xreq<20Xreq<20Xreq<20Xreq<20Xreq<20Xreq<20Xreq<20rq:=rq+1rq:=rq+1rq:=rq+1rq:=rq+1rq:=rq+1rq:=rq+1rq:=rq+1rq:=rq+1rq:=rq+1rq:=rq+1rq:=rq+1rq:=rq+1rq:=rq+1rq:=rq+1rq:=rq+1rq:=rq+1rq:=rq+1

req?req?req?req?req?req?req?req?req?req?req?req?req?req?req?req?req?Xped_gr<20Xped_gr<20Xped_gr<20Xped_gr<20Xped_gr<20Xped_gr<20Xped_gr<20Xped_gr<20Xped_gr<20Xped_gr<20Xped_gr<20Xped_gr<20Xped_gr<20Xped_gr<20Xped_gr<20Xped_gr<20Xped_gr<20

ped_gr!ped_gr!ped_gr!ped_gr!ped_gr!ped_gr!ped_gr!ped_gr!ped_gr!ped_gr!ped_gr!ped_gr!ped_gr!ped_gr!ped_gr!ped_gr!ped_gr!Xcar_rd>=5Xcar_rd>=5Xcar_rd>=5Xcar_rd>=5Xcar_rd>=5Xcar_rd>=5Xcar_rd>=5Xcar_rd>=5Xcar_rd>=5Xcar_rd>=5Xcar_rd>=5Xcar_rd>=5Xcar_rd>=5Xcar_rd>=5Xcar_rd>=5Xcar_rd>=5Xcar_rd>=5

car_gr!car_gr!car_gr!car_gr!car_gr!car_gr!car_gr!car_gr!car_gr!car_gr!car_gr!car_gr!car_gr!car_gr!car_gr!car_gr!car_gr!Xped_rd>=5Xped_rd>=5Xped_rd>=5Xped_rd>=5Xped_rd>=5Xped_rd>=5Xped_rd>=5Xped_rd>=5Xped_rd>=5Xped_rd>=5Xped_rd>=5Xped_rd>=5Xped_rd>=5Xped_rd>=5Xped_rd>=5Xped_rd>=5Xped_rd>=5

ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!Xped_gr>=20Xped_gr>=20Xped_gr>=20Xped_gr>=20Xped_gr>=20Xped_gr>=20Xped_gr>=20Xped_gr>=20Xped_gr>=20Xped_gr>=20Xped_gr>=20Xped_gr>=20Xped_gr>=20Xped_gr>=20Xped_gr>=20Xped_gr>=20Xped_gr>=20rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0

ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!ped_rd!Xreq>=20Xreq>=20Xreq>=20Xreq>=20Xreq>=20Xreq>=20Xreq>=20Xreq>=20Xreq>=20Xreq>=20Xreq>=20Xreq>=20Xreq>=20Xreq>=20Xreq>=20Xreq>=20Xreq>=20rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0rq:=0

grRqgrRqgrRqgrRqgrRqgrRqgrRqgrRqgrRqgrRqgrRqgrRqgrRqgrRqgrRqgrRqgrRq

grNotRqgrNotRqgrNotRqgrNotRqgrNotRqgrNotRqgrNotRqgrNotRqgrNotRqgrNotRqgrNotRqgrNotRqgrNotRqgrNotRqgrNotRqgrNotRqgrNotRq

redredredredredredredredredredredredredredredredred

g1g1g1g1g1g1g1g1g1g1g1g1g1g1g1g1g1

r1r1r1r1r1r1r1r1r1r1r1r1r1r1r1r1r1

ConfigConfigConfigConfigConfigConfigConfigConfigConfigConfigConfigConfigConfigConfigConfigConfigConfigobservable req, car_gr, car_rd, ped_rd, ped_gr;observable req, car_gr, car_rd, ped_rd, ped_gr;observable req, car_gr, car_rd, ped_rd, ped_gr;observable req, car_gr, car_rd, ped_rd, ped_gr;observable req, car_gr, car_rd, ped_rd, ped_gr;observable req, car_gr, car_rd, ped_rd, ped_gr;observable req, car_gr, car_rd, ped_rd, ped_gr;observable req, car_gr, car_rd, ped_rd, ped_gr;observable req, car_gr, car_rd, ped_rd, ped_gr;observable req, car_gr, car_rd, ped_rd, ped_gr;observable req, car_gr, car_rd, ped_rd, ped_gr;observable req, car_gr, car_rd, ped_rd, ped_gr;observable req, car_gr, car_rd, ped_rd, ped_gr;observable req, car_gr, car_rd, ped_rd, ped_gr;observable req, car_gr, car_rd, ped_rd, ped_gr;observable req, car_gr, car_rd, ped_rd, ped_gr;observable req, car_gr, car_rd, ped_rd, ped_gr;int rq;int rq;int rq;int rq;int rq;int rq;int rq;int rq;int rq;int rq;int rq;int rq;int rq;int rq;int rq;int rq;int rq;system pedXing;system pedXing;system pedXing;system pedXing;system pedXing;system pedXing;system pedXing;system pedXing;system pedXing;system pedXing;system pedXing;system pedXing;system pedXing;system pedXing;system pedXing;system pedXing;system pedXing;

pedXingpedXingpedXingpedXingpedXingpedXingpedXingpedXingpedXingpedXingpedXingpedXingpedXingpedXingpedXingpedXingpedXing

55

Selection Strategy: Comparison of partitioningRequired

#tests

Grain sizeLargeSmall

Weak Strong

Regions

Stable Edge-set

Edges

Action

Strength of Uniformity hypothesis

•Region: Two (super) states are equivalent if they enable the same (set of) regions

•Stable Edge set: Two (super) states are equivalent if they enable the same (set of) locations and edges. Equivalent wrt. single action Hennessy tests.

•Edges: Two (super) states are equivalent if they enable the same edge.

•Action: Two (super) states are equivalent if they enable the same action.

real-time systems specification and test ofpeople.cs.aau.dk/~bnielsen/testudbud/defense.pdf · 1....

Documents