the uml/marte veri er
TRANSCRIPT
ETR course
The UML/MARTE VerifierA Property Driven toolchain for model checking real time systems
Marc Pantel (based on Ning Ge and Faiez Zalila work)
Universite de Toulouse, IRIT-CNRS, ACADIE
August 27, 2015
Work funded byFUI TOPCASED, ITEA OPEES, FUI Projet P, ITEA openETCS, IRT Saint Exupery
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 1 / 59
Outline
1 Introduction
2 Method to integrate formal verification for DSMLs
3 Property-Driven Approach
4 Semantic Mapping from UML-MARTE to TPN
5 Real-Time Property Specification
6 Observer-Based Property Verification
7 Property Specific State Space Reduction
8 Feedback Analysis Proposal
9 Synthesis
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 2 / 59
Introduction
Outline
1 Introduction
2 Method to integrate formal verification for DSMLs
3 Property-Driven Approach
4 Semantic Mapping from UML-MARTE to TPN
5 Real-Time Property Specification
6 Observer-Based Property Verification
7 Property Specific State Space Reduction
8 Feedback Analysis Proposal
9 Synthesis
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 3 / 59
Introduction
Safety Critical Real-Time Embedded Systems
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 4 / 59
Introduction
Safety Critical Real-Time Embedded Systems
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 4 / 59
Introduction
Real-Time Requirements
!"#$%&'(")!"*+',"("-./
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 5 / 59
Introduction
Real-Time Requirements
!"#$%&'(")!"*+',"("-./
012'3#$)4("),"*+',"("-./
567/'3#$)4("),"*+',"("-./
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 5 / 59
Introduction
Real-Time Requirements
!"#$%&'(")!"*+',"("-./
012'3#$)4("),"*+',"("-./
567/'3#$)4("),"*+',"("-./
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 5 / 59
Introduction
Real-Time Requirements
!"#$%&'(")!"*+',"("-./
012'3#$)4("),"*+',"("-./
!"#$%&'($)*$'*+($!,(*-$-*"*./.+0$-*1$,-2%$(3*$4.52(❚
567/'3#$)4("),"*+',"("-./
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 5 / 59
Introduction
Real-Time Requirements
!"#$%&'(")!"*+',"("-./
012'3#$)4("),"*+',"("-./
567/'3#$)4("),"*+',"("-./
1 5 10 15 20
!"#$%&'($)*$'*+($!,(*-$-*"*./.+0$-*1$,-2%$(3*$4.52(❚
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 5 / 59
Introduction
Real-Time Requirements
!"#$%&'(")!"*+',"("-./
012'3#$)4("),"*+',"("-./
567/'3#$)4("),"*+',"("-./
1 5 10 15 20
!"#$%&'($)*$'*+($!,(*-$-*"*./.+0$-*1$,-2%$(3*$4.52(
!"#$%&'($)*$'*+($!,(*-$-*"*./.+0$-*1$,-2%$(3*$4.52($.+$6789$:8;%'$.+$*!"3$4*-.2<❚
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 5 / 59
Introduction
Real-Time Requirements
!"#$%&'(")!"*+',"("-./
012'3#$)4("),"*+',"("-./
567/'3#$)4("),"*+',"("-./
1 5 10 15 20
!"#$%&'($)*$'*+($!,(*-$-*"*./.+0$-*1$,-2%$(3*$4.52(
!"#$%&'($)*$'*+($!,(*-$-*"*./.+0$-*1$,-2%$(3*$4.52($.+$6789$:8;%'$.+$*!"3$4*-.2<
=3*$*+0.+*$#**4'$'(!55.+0$,2-$!($5*!'($>$'*"❚
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 5 / 59
Introduction
Real-Time Requirements
!"#$%&'(")!"*+',"("-./
012'3#$)4("),"*+',"("-./
567/'3#$)4("),"*+',"("-./
1 5 10 15 20
!"#$%&'($)*$'*+($!,(*-$-*"*./.+0$-*1$,-2%$(3*$4.52(
!"#$%&'($)*$'*+($!,(*-$-*"*./.+0$-*1$,-2%$(3*$4.52($.+$6789$:8;%'$.+$*!"3$4*-.2<=3*$*+0.+*$#**4'$'(!55.+0$,2-$!($5*!'($>$'*"❚
!"*+',"893'"-.)
:",';3#41-
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 5 / 59
Introduction
Model Driven Engineering & Formal Methods
!"#$%&'()*$+&,+-)+$$()+-
."(/0%&!$12"#3
4
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 6 / 59
Introduction
Model Driven Engineering & Formal Methods
!"#$%&"'"()
*
!"#$%&"'"()*
+,+
-&./%)".)$&
"0
1"*%2(
-&./%)".)
$&"0
1"*%2(0
+,+1")3%4"
50
1"*%2(
1")3%4"50
1"*%2(0+,+
675"
8"("&3
97(
675"0
8"("&397
(
+,+
!"#$%&"'"()*
+,+
1")3%4"50
1"*%2(0
+,+
-&./%)".)$&
"0
1"*%2(0
+,+
-&./%)".)$&"0
1"*%2(0+,+
!"#$%&"'"()
*
+,+
!"#$%&"'"(
)*
+,+
:75"401&%;"(0<(2%(""&%(2
=7&'340:")/75*
,
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 6 / 59
Introduction
Model Driven Engineering & Formal Methods
!"#$%&"'"()
*
!"#$%&"'"()*
+,+
-&./%)".)$&
"0
1"*%2(
-&./%)".)
$&"0
1"*%2(0
+,+1")3%4"
50
1"*%2(
1")3%4"50
1"*%2(0+,+
675"
8"("&3
97(
675"0
8"("&397
(
+,+
!"#$%&"'"()*
+,+
1")3%4"50
1"*%2(0
+,+
-&./%)".)$&
"0
1"*%2(0
+,+
-&./%)".)$&"0
1"*%2(0+,+
!"#$%&"'"()
*
+,+
!"#$%&"'"(
)*
+,+
M3_str
[0,60000]
M1_str
[0,50000]KU1_devitf MFD1_devitf
KU1_offset
[0,0]
MFD1_offset
[25000,25000]
FM1_offset
[0,0]
NDB_input
NDB_data
[0,0] NDB_execp
NDB_exectr
[0,20000]
FM1a_devitf
MFD1_hold
MFD1_data
[0,0] MFD1_execp
KU1_data
[0,0]KU1_execp
KU1_exectr
[0,25000]KU1_output
FM1_data
[0,0]
MFD1_input
SP_inittr
KU1_input SP_initp
KU1_holdMFD1_waitp
KU1_null
[0,0]KU1_waitp
KU1_waittr
[50000,50000]
MFD1_waittr
[50000,50000]
MFD1_null
[0,0]
FM1_holdFM1_waitp
FM1_null
[0,0]
FM1_waittr
[60000,60000]
FM1_inputFM1_execp
FM1_exectr
[0,30000]FM1_output
FM1a_offset
[0,0]
FM1_devitf
FM1a_hold
FM1a_input
M7_sp NDB_devitf
NDB_offset
[0,0] NDB_hold
NDB_null
[0,0] NDB_waitp
NDB_waittr
[100000,100000]
FM1a_MFD1_comm
[310,490]
KU1_FM1_comm
[298,444]
M3_sp
FM1_NDB_comm
[268,310]
M7_str
[0,100000]
NDB_output
M1_sp
NDB_FM1a_comm
[400,508] NDB_bag
NDB_FM1a_bag
[0,64000]
FM1a_exectr
[0,30000]
FM1a_data
[0,0]
FM1a_null
[0,0]
FM1a_waittr
[60000,60000]
FM1a_waitp
FM1a_execp FM1a_output
MFD1_output
MFD1_exectr
[0,25000]
:75"401&%;"(0<(2%(""&%(2
,
=7&'340:")/75*
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 6 / 59
Introduction
V & V in MDE
!"#$%&"'"()* !"
#$%&"'"()*
+,+
-&./%)".)$&"0
1"*%2( -
&./%)".)$&"0
1"*%2(0+,+
1")3%4"50
1"*%2( 1
")3%4"50
1"*%2(0+,+
675"
8"("&397(
675"08"("&397(
+,+
!"#$%&"'"()*
+,+
1")3%4"50
1"*%2(0+,+
-&./%)".)$&"0
1"*%2(0+,+
-&./%)".)$&"0
1"*%2(0+,+
!"#$%&"'"()*
+,+
!"#$%&"'"()*
+,+
:%'"0;%("
Note: from MeMVaTEx methodology
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 7 / 59
Proposed method
Outline
1 Introduction
2 Method to integrate formal verification for DSMLs
3 Property-Driven Approach
4 Semantic Mapping from UML-MARTE to TPN
5 Real-Time Property Specification
6 Observer-Based Property Verification
7 Property Specific State Space Reduction
8 Feedback Analysis Proposal
9 Synthesis
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 8 / 59
Proposed method
Domain-Specific Modeling Languages (DSMLs)
model
model
model
represented by
represented by
represented by
conforms to
conforms to
conforms to
Model-Driven Engineering
editors
simulators
User
verifiers
generators
DSML
editors
simulators
User
verifiers
generators
DSML
editors
simulators
User
verifiers
generators
DSML
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 9 / 59
Proposed method
Domain-Specific Modeling Languages (DSMLs)
model
model
model
represented by
represented by
represented by
conforms to
conforms to
conforms to
Model-Driven EngineeringLanguage Engineering
editorsLanguage
expert
Domain expert
simulators
User
verifiers
generators
DSML
editors
simulators
User
verifiers
generators
DSML
editors
simulators
User
verifiers
generators
DSML
Language expert
Domain expert
Language expert
Domain expert
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 9 / 59
Proposed method
Verification and Validation (V&V) activities
model
model
model
represented by
represented by
represented by
conforms to
conforms to
conforms to
Model-Driven EngineeringLanguage Engineering
Formal verification
editorsLanguage
expert
Domain expert
simulators
User
verifiers
generators
DSML
editors
simulators
User
verifiers
generators
DSML
editors
simulators
User
verifiers
generators
DSML
Language expert
Domain expert
Language expert
Domain expert
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 9 / 59
Proposed method
Formal verification technique
User requirements:
Ease of useAutomationEfficiencySoundnessCompleteness
Candidate:
Automated theorem proving (SAT/SMT solvers) (logic based, userprovided dedicated abstractions)Abstract interpretation (state based, automated generic abstractions)Model checking (state based, user provided dedicated abstractions)
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 10 / 59
Proposed method
Model checking based formal verification architecture
model-checkingtools
DSMLmodel
Formalmodel
Formalproperties
Formalverification
results
DSMLverification
resultsDSML
end-user
defines
obtains
defines/uses
DSMLbehavioral properties
Formal verificationDSML Verifier
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 11 / 59
Proposed method
Model checking based formal verification architecture
model-checkingtools
DSMLmodel
Formalmodel
Formalproperties
Formalverification
results
DSMLverification
resultsDSML
end-user
defines
obtains
defines/uses
DSMLbehavioral properties
Formal verification
Interpretation approach (Operational semantics)
Translational approach (Translational semantics)
DSML Verifier
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 11 / 59
Proposed method
Translational approach
model-checkingtools
DSMLmodel
Formalmodel
Formalproperties
Formalverification
results
DSMLverification
resultsDSML
end-user
defines
obtains
defines/uses
DSMLbehavioral properties
Formal verification
Translational approach (Translational semantics)
DSML Verifier
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 11 / 59
Proposed method
DSML Verifier: Reuse formal tools
model-checkingtools
DSMLmodel
Formalmodel
Formalproperties
Formalverification
results
DSMLverification
resultsDSML
end-user
defines
obtains
defines/uses
DSMLbehavioral properties
model-checkingtools
Formalmodel
Formalproperties
Formalverification
results
DSML Verifier
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 11 / 59
Proposed method
Defining a translational semantics
model-checkingtools
DSMLmodel
Formalmodel
Formalproperties
Formalverification
results
DSMLverification
resultsDSML
end-user
defines
obtains
defines/uses
DSMLbehavioral properties
model-checkingtools
Formalmodel
Formalproperties
Formalverification
results
Translational semantics
Domain expert
Language expert
specifies implementsDSML Verifier
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 11 / 59
Proposed method
Completing the integration
model-checkingtools
DSMLmodel
Formalmodel
Formalproperties
Formalverification
results
DSMLverification
resultsDSML
end-user
defines
obtains
defines/uses
DSMLbehavioral properties
model-checkingtools
Formalmodel
Formalproperties
Formalverification
results
Translational semantics
Domain expert
Language expert
specifies implements
Properties generation
Feedbackverification
results
DSML Verifier
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 11 / 59
Proposed method
Use case driven method
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 12 / 59
Proposed method
Use case driven method
Ad-hoc solutions
Analyse results
Suggest generic
solutions
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 12 / 59
Proposed method
Use case driven method
Ad-hoc solutions
Analyse results
Suggest generic
solutions Capitalize know-how
and expertise
Apply on use-case
Validateproposed solutions
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 12 / 59
Proposed method
Use case driven method
Ad-hoc solutions
Analyse results
Suggest generic
solutions Capitalize know-how
and expertise
Apply on use-case
Validateproposed solutions
Apply on use-case
Validate proposedsolutions
Capitalize know-how
and expertise
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 12 / 59
Proposed method
Use case driven method
Ad-hoc solutions
Analyse results
Suggest generic
solutions Capitalize know-how
and expertise Package our contributions
Collect applications feedbacks
Synthesize our contributions
Apply on use-case
Validateproposed solutions
Apply on use-case
Validate proposedsolutions
Capitalize know-how
and expertise
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 12 / 59
Proposed method
Case Study: Flight Management System (FMS)
Rely on Integrated Modular Avionics (IMA) principles
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 13 / 59
Proposed method
FMS Architecture Model by Boniol and Lauer
Module1KU1
MFD1
Module2KU2
MFD2
Module3FM1
Module4FM2
Module5ADIRU1
Module6ADIRU2
Module7NDB
S1
S4 S5
S3S2
RDC1 RDC2
sensor1 sensor2
keyboard1 display2 keyboard2display1
!"#$%"&'()*+&,-'.#)$
!"/+.$0%'.#)$
123
functions, AFDX network
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 14 / 59
Proposed method
FMS Architecture Model by Boniol and Lauer
Module1KU1
MFD1
Module2KU2
MFD2
Module3FM1
Module4FM2
Module5ADIRU1
Module6ADIRU2
Module7NDB
S1
S4 S5
S3S2
RDC1 RDC2
sensor1 sensor2
keyboard1 display2 keyboard2display1
!"#$%"&'()*+&,-'.#)$
!"/+.$0%'.#)$
123
%04
functions, AFDX network
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 14 / 59
Proposed method
FMS Architecture Model by Boniol and Lauer
Module1KU1
MFD1
Module2KU2
MFD2
Module3FM1
Module4FM2
Module5ADIRU1
Module6ADIRU2
Module7NDB
S1
S4 S5
S3S2
RDC1 RDC2
sensor1 sensor2
keyboard1 display2 keyboard2display1
!"#$%"&'()*+&,-'.#)$
!"/+.$0%'.#)$
123
4+1(5 4+1(6
functions, AFDX network
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 14 / 59
Proposed method
FMS Architecture Model by Boniol and Lauer
Module1KU1
MFD1
Module2KU2
MFD2
Module3FM1
Module4FM2
Module5ADIRU1
Module6ADIRU2
Module7NDB
S1
S4 S5
S3S2
RDC1 RDC2
sensor1 sensor2
keyboard1 display2 keyboard2display1
!"#$%"&'()*+&,-'.#)$
!"/+.$0%'.#)$
123
4.0%-5 4.0%-6
functions, AFDX network
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 14 / 59
Proposed method
FMS Architecture Model by Boniol and Lauer
Module1KU1
MFD1
Module2KU2
MFD2
Module3FM1
Module4FM2
Module5ADIRU1
Module6ADIRU2
Module7NDB
S1
S4 S5
S3S2
RDC1 RDC2
sensor1 sensor2
keyboard1 display2 keyboard2display1
!"#$%"&'()*+&,-'.#)$
!"/+.$0%'.#)$
123
,#*40%5 ,#*40%6
functions, AFDX network
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 14 / 59
Proposed method
FMS Architecture Model by Boniol and Lauer
Module1KU1
MFD1
Module2KU2
MFD2
Module3FM1
Module4FM2
Module5ADIRU1
Module6ADIRU2
Module7NDB
S1
S4 S5
S3S2
RDC1 RDC2
sensor1 sensor2
keyboard1 display2 keyboard2display1
!"#$%"&'()*+&,-'.#)$
!"/+.$0%'.#)$
123
4+1#5"6 4+1#5"7
functions, AFDX network
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 14 / 59
Proposed method
FMS Architecture Model by Boniol and Lauer
Module1KU1
MFD1
Module2KU2
MFD2
Module3FM1
Module4FM2
Module5ADIRU1
Module6ADIRU2
Module7NDB
S1
S4 S5
S3S2
RDC1 RDC2
sensor1 sensor2
keyboard1 display2 keyboard2display1
!"#$%"&'()*+&,-'.#)$
!"/+.$0%'.#)$
123
()*+4 ()*+5
functions, AFDX network
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 14 / 59
Proposed method
Latency Real-Time Requirements
In the pilot request functional chain, the time between req1 and the firstoccurrence of disp1 depending on req1 must be in time range [bct,wct].
KU1 MFD1 KU1 MFD1 KU1 MFD1 KU1 MFD1 KU1 MFD1 KU1 MFD1
FM1 FM1 FM1 FM1 FM1
NDB NDB NDB
M1
M3
M7
req1[1]
0
15 45
25 50 225
75 195
25 45 125
disp1[5] disp1[6]
l10 240
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 15 / 59
Proposed method
Verification of FMS Case Study
Proposal of Boniol and Lauer
Abstraction based on trajectory approach for the AFDX network
Formal modeling using tagged signal model
Transformed in Integer Linear Programming (ILP) problems
Model Checking?
Modeling and Analysis using timed automata & UPPALL
State space combinatorial explosion issue
Further Study on Model Checking
Methods for minimizing verification semantics to reduce the state space.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 16 / 59
Proposed method
Phase in the development process
!"#$%&"'"()* !"
#$%&"'"()*
+,+
-&./%)".)$&"0
1"*%2( -
&./%)".)$&"0
1"*%2(0+,+
1")3%4"50
1"*%2( 1
")3%4"50
1"*%2(0+,+
675"
8"("&397(
675"08"("&397(
+,+
!"#$%&"'"()*
+,+
1")3%4"50
1"*%2(0+,+
-&./%)".)$&"0
1"*%2(0+,+
-&./%)".)$&"0
1"*%2(0+,+
!"#$%&"'"()*
+,+
!"#$%&"'"()*
+,+
:%'"0;%("
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 17 / 59
Proposed method
Phase in the development process
!"#$%&"'"()* !"
#$%&"'"()*
+,+
-&./%)".)$&"0
1"*%2( -&
./%)".)$&"0
1"*%2(0+,+
1")3%4"50
1"*%2( 1
")3%4"50
1"*%2(0+,+
675"
8"("&397(
675"08"("&397(
+,+
!"#$%&"'"()*
+,+
1")3%4"50
1"*%2(0+,+
-&./%)".)$&"0
1"*%2(0+,+
-&./%)".)$&"0
1"*%2(0+,+
!"#$%&"'"()*
+,+
!"#$%&"'"()*
+,+
:%'"0;%("
<=;>=-!:?
:%'"0@")&%0A")*0B0:CA-
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 17 / 59
Property-Driven Approach
Outline
1 Introduction
2 Method to integrate formal verification for DSMLs
3 Property-Driven Approach
4 Semantic Mapping from UML-MARTE to TPN
5 Real-Time Property Specification
6 Observer-Based Property Verification
7 Property Specific State Space Reduction
8 Feedback Analysis Proposal
9 Synthesis
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 18 / 59
Property-Driven Approach
Property-Driven Approach
Principle
The formal activities in the development process are based on the purpose ofproperty-verification-ease.
Experiments by B. Combemale
Verification of structural and temporal properties for Development Process models.
Requires more scalable methods to verify quantitative properties.
Proposed method
1 Characterize expected properties.
2 Characterize mandatory observable states and events to assess these properties.
3 Express real-time properties using elementary property patterns.
4 Define translational semantics to Time Petri Net (TPN) with observers and reachabilityassertions.
5 Reduce state space: property-specific reduction for TPN.
6 Validate model and feedback: automated failure analysis.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 19 / 59
Property-Driven Approach
Time Petri Net
[0,0] [3,10]
2
[11,15]
[19,27]Pinit Tfork
Ptask1
Ptask2
Texe1
Texe2
Pjoin Texit Pexit
(10, ∞]
2
Trestart
TINA toolset
Analyze µ-calculus, LTL, CTL properties for TPN.
Integrate state space abstraction techniques (preserving different kinds of properties),on-the-fly model checking.
Data manipulation (tts): variables used in transition guards and actions.
Proposal
Rely on observers and reachability assertions.
Transform quantitative problem into reachability problem.
Minimize semantics for observation based on state space preserving markings.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 20 / 59
Property-Driven Approach
Challenge & Property-Driven Verification Framework
TPN
Reduced Observer TPN
ReachabilityAssertions
TPN Model CheckingTag Property Pattern Result
Architecture/Behavior Mapping
Observer TPN Generation
IterationTag
Property Pattern Result
Real-Time PropertySpecification
Verification Result
Computation
Real-Time PropertyVerification Result
Feedback Generation
System ModelReal-Time
RequirementArchitecture
ModelBehavior
Model
UML Real-Time Software Model
Timing Property Pattern
Timing Property Pattern
Real-Time Property Patterns
1
5
3
2
3
3
Observer TPN
Tag Property Pattern Result Interpretation
3
TPN Reduction
4
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 21 / 59
Property-Driven Approach
Challenge & Property-Driven Verification Framework
TPN
Reduced Observer TPN
Reachability Assertions
TPN Model CheckingTag Property Pattern Result
Architecture/Behavior Mapping
Observer TPN Generation
IterationTag
Property Pattern Result
Real-Time PropertySpecification
Verification Result
Computation
Real-Time PropertyVerification Result
Feedback Generation
System ModelReal-Time
RequirementArchitecture
ModelBehavior
Model
UML Real-Time Software Model
Timing Property Pattern
Timing Property Pattern
Real-Time Property Patterns
1
5
3
2
3
3
Observer TPN
Tag Property Pattern Result Interpretation
3
TPN Reduction
4
[0,0] [3,10]
2
[11,15]
[19,27]Pinit Tfork
Ptask1
Ptask2
Texe1
Texe2
Pjoin Texit Pexit
(10, ∞]
2
Trestart
TINA toolset
Assess µ-calculus, LTL, CTL requirements for TPN.
Integrate state space abstraction techniques (preserving different types of properties),on-the-fly model checking.
Provide data manipulation (tts): variables used in transition guards and actions.
Proposal
Rely on observers and reachability assertions.
Transform quantitative problems into reachability problems.
Minimize semantics for observation based on state space preserving markings.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 21 / 59
Property-Driven Approach
Challenge & Property-Driven Verification Framework
TPN
Reduced Observer TPN
ReachabilityAssertions
TPN Model CheckingTag Property Pattern Result
Architecture/Behavior Mapping
Observer TPN Generation
Property Pattern Result
Real-Time PropertySpecification
Verification Result
Computation
Real-Time PropertyVerification Result
Feedback Generation
System ModelReal-Time
RequirementArchitecture
ModelBehavior
Model
UML Real-Time Software Model
Timing Property Pattern
Timing Property Pattern
Real-Time Property Patterns
1
5
3
2
3
3
Observer TPN
Tag Property Pattern Result Interpretation
3
TPN Reduction
4
IterationTag
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 21 / 59
Property-Driven Approach
Challenge & Property-Driven Verification Framework
TPN
Reduced Observer TPN
ReachabilityAssertions
TPN Model CheckingTag Property Pattern Result
Architecture/Behavior Mapping
Observer TPN Generation
Property Pattern Result
Real-Time PropertySpecification
Verification Result
Computation
Real-Time PropertyVerification Result
Feedback Generation
System ModelReal-Time
RequirementArchitecture
ModelBehavior
Model
UML Real-Time Software Model
Timing Property Pattern
Timing Property Pattern
Real-Time Property Patterns
1
5
3
2
3
3
Observer TPN
Tag Property Pattern Result Interpretation
3
TPN Reduction
4
IterationTag
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 21 / 59
Property-Driven Approach
Challenge & Property-Driven Verification Framework
TPN
Reduced Observer TPN
ReachabilityAssertions
TPN Model CheckingTag Property Pattern Result
Architecture/Behavior Mapping
Observer TPN Generation
Property Pattern Result
Real-Time PropertySpecification
Verification Result
Computation
Real-Time PropertyVerification Result
Feedback Generation
System ModelReal-Time
RequirementArchitecture
ModelBehavior
Model
UML Real-Time Software Model
Timing Property Pattern
Timing Property Pattern
Real-Time Property Patterns
1
5
3
2
3
3
Observer TPN
Tag Property Pattern Result Interpretation
3
IterationTag
TPN Reduction
4
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 21 / 59
Property-Driven Approach
Challenge & Property-Driven Verification Framework
TPN
Reduced Observer TPN
ReachabilityAssertions
TPN Model CheckingTag Property Pattern Result
Architecture/Behavior Mapping
Observer TPN Generation
Property Pattern Result
Real-Time PropertySpecification
Verification Result
Computation
Real-Time PropertyVerification Result
Feedback Generation
System ModelReal-Time
RequirementArchitecture
ModelBehavior
Model
UML Real-Time Software Model
Timing Property Pattern
Timing Property Pattern
Real-Time Property Patterns
1
5
3
2
3
3
Observer TPN
Tag Property Pattern Result Interpretation
3
TPN Reduction
4
IterationTag
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 21 / 59
Property-Driven Approach
Challenge & Property-Driven Verification Framework
TPN
Reduced Observer TPN
ReachabilityAssertions
TPN Model CheckingTag Property Pattern Result
Architecture/Behavior Mapping
Observer TPN Generation
IterationTag
Property Pattern Result
Real-Time PropertySpecification
Verification Result
Computation
Real-Time PropertyVerification Result
Feedback Generation
System ModelReal-Time
RequirementArchitecture
ModelBehavior
Model
UML Real-Time Software Model
Timing Property Pattern
Timing Property Pattern
Real-Time Property Patterns
1
5
3
2
3
3
Observer TPN
Tag Property Pattern Result Interpretation
3
TPN Reduction
4
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 21 / 59
Property-Driven Approach
Challenge & Property-Driven Verification Framework
TPN
Reduced Observer TPN
ReachabilityAssertions
TPN Model CheckingTag Property Pattern Result
Architecture/Behavior Mapping
Observer TPN Generation
IterationTag
Property Pattern Result
Real-Time PropertySpecification
Verification Result
Computation
Real-Time PropertyVerification Result
Feedback Generation
System ModelReal-Time
RequirementArchitecture
ModelBehavior
Model
UML Real-Time Software Model
Timing Property Pattern
Timing Property Pattern
Real-Time Property Patterns
1
5
3
2
3
3
Observer TPN
Tag Property Pattern Result Interpretation
3
TPN Reduction
4
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 21 / 59
Semantic Mapping from UML-MARTE to TPN
Outline
1 Introduction
2 Method to integrate formal verification for DSMLs
3 Property-Driven Approach
4 Semantic Mapping from UML-MARTE to TPN
5 Real-Time Property Specification
6 Observer-Based Property Verification
7 Property Specific State Space Reduction
8 Feedback Analysis Proposal
9 Synthesis
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 22 / 59
Semantic Mapping from UML-MARTE to TPN
Modeling Context
Real-Time Software Systems
Clocks: single & multiple clocks (rate, drift, offset)
Communication: synchronous & asynchronous
Object Value
Ignored in the architecture design
Cyclic execution
Event-trigger: activated by the data and control flow
Time-trigger: also activated by the rising edge of time cycle
MARTE
Simplification on the use of MARTE
Resource scheduling
A generic scheduling algorithm with preemption option is provided
Real-Time Software Systems
Clocks: single & multiple clocks (rate, drift, offset)
Communication: synchronous & asynchronous
Object Value
Ignored in the architecture design phase
Cyclic execution
Event-trigger: activated by the data and control flow
Time-trigger: also activated by the rising edge of time cycle
MARTE
Simplification on the use of MARTE
Resource scheduling
A generic scheduling algorithm with preemption option is provided
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 23 / 59
Semantic Mapping from UML-MARTE to TPN
Modeling Context
Real-Time Software Systems
Clocks: single & multiple clocks (rate, drift, offset)
Communication: synchronous & asynchronous
Object Value
Ignored in the architecture design phase
Cyclic execution
Event-trigger: activated by the data and control flow
Time-trigger: also activated by the rising edge of time cycle
MARTE
Simplification on the use of MARTE
Resource scheduling
A generic scheduling algorithm with preemption option is provided
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 23 / 59
Semantic Mapping from UML-MARTE to TPN
Modeling Context
Real-Time Software Systems
Clocks: single & multiple clocks (rate, drift, offset)
Communication: synchronous & asynchronous
Object Value
Ignored in the architecture design phase
Cyclic execution
Event-trigger: activated by the data and control flow
Time-trigger: also activated by the rising edge of time cycle
MARTE
Simplification on the use of MARTE
Resource scheduling
A generic scheduling algorithm with preemption option is provided
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 23 / 59
Semantic Mapping from UML-MARTE to TPN
Modeling Context
Real-Time Software Systems
Clocks: single & multiple clocks (rate, drift, offset)
Communication: synchronous & asynchronous
Object Value
Ignored in the architecture design phase
Cyclic execution
Event-trigger: activated by the data and control flow
Time-trigger: also activated by the rising edge of time cycle
MARTE
Simplification on the use of MARTE
Resource scheduling
A generic scheduling algorithm with preemption option is provided
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 23 / 59
Semantic Mapping from UML-MARTE to TPN
Modeling Context
Real-Time Software Systems
Clocks: single & multiple clocks (rate, drift, offset)
Communication: synchronous & asynchronous
Object Value
Ignored in the architecture design phase
Cyclic execution
Event-trigger: activated by the data and control flow
Time-trigger: also activated by the rising edge of time cycle
MARTE
Simplification on the use of MARTE
Resource scheduling
A generic scheduling algorithm with preemption option is provided
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 23 / 59
Semantic Mapping from UML-MARTE to TPN
Modeling Context
Real-Time Software Systems
Clocks: single & multiple clocks (rate, drift, offset)
Communication: synchronous & asynchronous
Object Value
Ignored in the architecture design
Cyclic execution
Event-trigger: activated by the data and control flow
Time-trigger: also activated by the rising edge of time cycle
MARTE
Simplification on the use of MARTE
Resource scheduling
A generic scheduling algorithm with preemption option is provided
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 23 / 59
Semantic Mapping from UML-MARTE to TPN
Defining Mapping Semantics from UML-MARTE to TPN
Semantic Mapping Objectives
1 Conforming to the semantics in UML Specification 2.4.1, explicit semantics forvariation points
2 Property specific semantic mapping, preserving minimal set of property-relevantsemantics as possible
3 Standardized mapping for some untimed UML elements
4 Verification-ease, guarantee efficiency of model checking
5 Facilitate the assembly of mapping results
UML-MARTE diagrams
Composite structure diagram
Activity diagram
State machine diagram
Covers a large scope of modeling elements
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 24 / 59
Semantic Mapping from UML-MARTE to TPN
FMS: Modeling for Latency Requirement
Functional chain on IMA
FM1
FM2 NDB
KU1
FM2
MFD1FM1NDB
MFD2
req1
disp2
disp1
wpInfo2
wpInfo1
answer2
answer1
query2
query1wpId1
wpId2
Architecture
M1:KU_MFD_Module
<<Allocated>> req
<<Allocated>> wpId
<<Allocated>> disp
<<Allocated>> wpInfo
M3:FM_Module
<<Allocated>> wpId
<<Allocated>> query
<<Allocated>> wpInfo
<<Allocated>> anwser
M7:NDB_Module
<<Allocated>> query
<<Allocated>> anwser
<<CommunicationMedia>>
<<CommunicationMedia>>
<<CommunicationMedia>>
<<CommunicationMedia>>
Behavior of FM module
<<RtSpecification>>occKind = PeriodicPattern
(period=[60000,60000]; phase=[0,60000]; occurrences=-1)
FM1
<<Allocated>> wpId1
<<TimeProcessing>>
<<Allocated>> query1
FM1a
<<Allocated>> answer1
<<TimeProcessing>>
<<Allocated>> wpInfo1
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 25 / 59
Semantic Mapping from UML-MARTE to TPN
FMS: TPN Mapping Result
!"#
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 26 / 59
Semantic Mapping from UML-MARTE to TPN
FMS: TPN Mapping Result
!"#$%&'$()*+,
!" $%&
-,.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 26 / 59
Semantic Mapping from UML-MARTE to TPN
FMS: TPN Mapping Result
!"#$%&'$()*+,
!" $%&
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 26 / 59
Semantic Mapping from UML-MARTE to TPN
FMS: TPN Mapping Result
!"#"$%&'(
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 26 / 59
Semantic Mapping from UML-MARTE to TPN
FMS: TPN Mapping Result
!"#"$%&'(
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 26 / 59
Semantic Mapping from UML-MARTE to TPN
FMS: TPN Mapping Result
!"#"$%&'(
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 26 / 59
Semantic Mapping from UML-MARTE to TPN
FMS: TPN Mapping Result
!"#"$%&'(
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 26 / 59
Semantic Mapping from UML-MARTE to TPN
FMS: TPN Mapping Result
!"#$%&'()*
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 26 / 59
Semantic Mapping from UML-MARTE to TPN
FMS: TPN Mapping Result
!"#$%&'$()*+,
!" $%&
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 26 / 59
Semantic Mapping from UML-MARTE to TPN
FMS: TPN Mapping Result
!"#$%&'$()*+,
!" $%&
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 26 / 59
Semantic Mapping from UML-MARTE to TPN
FMS: TPN Mapping Result
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 26 / 59
Real-Time Property Specification
Outline
1 Introduction
2 Method to integrate formal verification for DSMLs
3 Property-Driven Approach
4 Semantic Mapping from UML-MARTE to TPN
5 Real-Time Property Specification
6 Observer-Based Property Verification
7 Property Specific State Space Reduction
8 Feedback Analysis Proposal
9 Synthesis
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 27 / 59
Real-Time Property Specification
Property Pattern Approach
Research Background
Qualitative patterns proposed by Dwyer cover 90% temporal requirements.
Extension to quantitative patterns by Konrad.
Speci&cation
Qualitative Quantitative
Occurrence Order Duration PeriodicQuantitative
Order
Absence Existence PrecedenceChain
Precedence
ResponseUniversalityBounded
Existence
Chain
Response
Maximum
Duration
Bounded
Recurrence
Bounded
Response
Minimum
DurationBounded
Invariance
Type
Catalog
PaCern
Classi&cation
by Dwyer
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 28 / 59
Real-Time Property Specification
Real-Time Property Patterns
Problem
Specification-orientation, semantically not atomic.
Proposal
A set of verification-ease elementary time property patterns.
Works as a bridge between specification patterns and formalverification.
Transform Dwyer and Konrad specification patterns and mostMARTE CCSL (Clock Constraint Specification Language) constraints.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 29 / 59
Real-Time Property Specification
Real-Time Property Patterns
Atomic Pa*ern
Event Modi2erState
Scope Modi2erBasic PredicateOccurrence Modi2er
Real-Time Property
Composite Pa*ern
Real-Time
Property Pa0ern
Exist A After B Within [bct, wct]
Operator Occurrence Basic predicate Scope
Absent B global
or Exist A ∧ B between (B + bct) and (B + wct)
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 30 / 59
Real-Time Property Specification
FMS: Latency Specification
Atomic Pa*ern
Event Modi2erState
Scope Modi2erBasic PredicateOccurrence Modi2er
Real-Time Property
Composite Pa*ern
Real-Time
Property Pa0ern
FMS latency property:time between pilot’s request and first disp depending on request must be in [bct,wct]
Operator Occurrence Basic predicate Scope
always T (req, disp) ≥ bct global
and always T (req, disp) ≤ wct global
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 31 / 59
Observer-Based Property Verification
Outline
1 Introduction
2 Method to integrate formal verification for DSMLs
3 Property-Driven Approach
4 Semantic Mapping from UML-MARTE to TPN
5 Real-Time Property Specification
6 Observer-Based Property Verification
7 Property Specific State Space Reduction
8 Feedback Analysis Proposal
9 Synthesis
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 32 / 59
Observer-Based Property Verification
Verification of Real-Time Property
Proposal
Observer-based model checking approach.
Executed concurrently with the model under assessment.
Define a set of elementary observers for the property patterns.
TPN observers for event based property.tts observers for state based property.
Error feedback provides all failure scenarios (that invalide theobserver)
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 33 / 59
Observer-Based Property Verification
Design of Observers
Soundness Requirement
Time divergence
No side-effect on the system’s original behavior.
Ensured by construction (structure of the patterns).
TPN Structure TPN Structure
TPN Structure
Component A TPN Component B TPN
TPN Observer
[0,0] [0,0]
TA TB
ptester
Efficiency Requirement
State Abstraction: abstraction preserving markingsRelated work: Abid (PhD thesis, 2013), tts observers with priority arc, state abstraction
Relatively optimal (minimizes states and transition numbers – not proved)
Independent checking: allows parallel computation
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 34 / 59
Observer-Based Property Verification
Catalog of Observers
Event modifier observers:
E E'
TPN
Structure
Observer
Predicate observers:
EM
TPN
Structure
!!"#$%%&'()*+%
Observer
Scope modifier observers
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 35 / 59
Observer-Based Property Verification
Occurrence Modifier
Assume in the state class graph
P: set of states that match the predicate,
S : set of states that match the scope,
P ∧ S : set of states that match both the predicate and the scope.
Occurrence
Exist Predicate in Scope:
{P ∧ S 6= ∅ if S 6= ∅;True if S = ∅.
Absent Predicate in Scope: P ∧ S = ∅Always Predicate in Scope: P ∧ S = S
!"#$%&'%()"*+% !"#$%&'%()"*+%
!"#$%
&'%()"*+%
!"#$% &'$()% &*+,-$
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 36 / 59
Observer-Based Property Verification
Computing Bound Value of Property
Requirement
When performing model checking, an observer can give an answersuch as Yes or No for the satisfaction of the given property.
For quantitative properties, however, users usually expect to knowwhat is the bound [bct,wct] of that property instead of whether theproperty is bounded by [bct,wct]?
Solution
An iterative method that will gradually approach the bound value byintegrating the observers into a binary (k-ary) search engine.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 37 / 59
Observer-Based Property Verification
FMS: Verification of Latency Property
[tmin,tmin] [0,0]
[tmax,tmax]
[0,0]
[0,25000]
MFD_exectrSP_ini6r
[0,0]
MFD_exectrSP_ini6r
[0,0] [0,25000]
BCT Observer WCT Observer
TPN System
... ...
TPN System
... ...
(a) Best Case (a) Worst Case
TesterA
[0,0] [0,0]
Over>owA Over>owB
TesterA
Over>ow2 2
Property Property Value (ms) State/Transition Number Execution Time (s)
LatencySystem N/A 9378/23250 N/A
wct 450.4 67105/145024 278.313bct 75.2 11162/28922 43.781
Same results as Boniol and LauerMarc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 38 / 59
Observer-Based Property Verification
FMS: Verification of Latency Property
TPN
Reduced Observer TPN
ReachabilityAssertions
TPN Model CheckingTag Property Pattern Result
Architecture/Behavior Mapping
Observer TPN Generation
Property Pattern Result
Real-Time PropertySpecification
Verification Result
Computation
Real-Time PropertyVerification Result
Feedback Generation
System ModelReal-Time
RequirementArchitecture
ModelBehavior
Model
UML Real-Time Software Model
Timing Property Pattern
Timing Property Pattern
Real-Time Property Patterns
1
5
3
2
3
3
Observer TPN
Tag Property Pattern Result Interpretation
3
TPN Reduction
4
IterationTag
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 39 / 59
Property Specific State Space Reduction
Outline
1 Introduction
2 Method to integrate formal verification for DSMLs
3 Property-Driven Approach
4 Semantic Mapping from UML-MARTE to TPN
5 Real-Time Property Specification
6 Observer-Based Property Verification
7 Property Specific State Space Reduction
8 Feedback Analysis Proposal
9 Synthesis
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 40 / 59
Property Specific State Space Reduction
State Space Reduction for TPN
Minimizing verification semantics
Modeling abstraction
Mapping abstraction
State space abstraction provided by TINA
On-the-fly model checking provided by TINA
Existing reduction techniques in model checking
Focus on universal properties
Property specific reduction methods are needed
Solution
1 Remove property irrelevant semantics
2 Combine property relevant semantics by replacing original sub-nets by behavioralequivalent ones
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 41 / 59
Property Specific State Space Reduction
Removal of Property-Irrelevant Semantics
Idea: analyze causality in the state class graph to remove transitions and statesirrelevant to the observed transitions and states.
Paradox: if the state class graph can be generated and analyzed, the reduction isnot needed.
Solution: use dependence analysis as an over-approximation.
Algorithm: search for and remove TPN places and transitions that the targetproperty does not depend on.
A
B
C
D
ETPN Model
F
Obs
Obs
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 42 / 59
Property Specific State Space Reduction
Regular Real-Time Property Specific Behavior
A
Occurrence Time [tmini , tmax
i ] Time Diff [tmini − tmin
i−1, tmaxi − tmax
i−1 ]
0 [0, 0] -1 [5, 10] [5, 10]2 [22, 79] [17, 69]3 [39, 148] [17, 69]... ... ...n [5+17(n-1), 10+ 69(n-1)] [17, 69]
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 43 / 59
Property Specific State Space Reduction
Regular Real-Time Property Specific Behavior
t4
[5,10] p1
t1
[17,69] p2p0
t5
[0,0]p5
A
Before Reduction
177 states /365 transitions
After Reduction
3 states / 3 transitions
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 44 / 59
Property Specific State Space Reduction
Regular Real-Time Property Specific Behavior
Observation
Regular behaviors occur in property related elements.
What are real-time property related elements?
Firing occurrence times of the observed transitions.
The time range of each occurrence of the observed outgoing transitions.
Proposal
Identify potential regular behaviors.
Detect sub-nets that may exhibit these behaviors.
Construct simpler substitute sub-nets that exhibit the same behaviors.
Verify the behavioral equivalence between the original sub-net and the substitute.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 45 / 59
Property Specific State Space Reduction
Regular Real-Time Property Specific Behaviors
Principle
After replacing the target sub-net int the system, this one exhibits exactly the sameproperty specific behavior as before.
Regular behaviors
Occurrence times, firing time range of the outgoing transition
Finite firing occurrence : sequential sectionInfinite firing occurrence: (sequential section) + loop section
B'
A
C
TA
C
[t1,t2]
[t3,t4]
….
[tm,tn]
B
[ti,tj]
[tp,tq]
….
[tx,ty]
A'
TB
(a) (b)
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 46 / 59
Property Specific State Space Reduction
Divide and Conquer Reduction Approach
System
A
A'
B C
B' C'
3 steps:
1 Identification: some reducible sub-nets like A, B, and C are identified.
One-way-out pattern: single portal outgoing transitionGeneric pattern: single portal incoming and outgoing transition.
2 Reduction: search for the regularity of real-time behavior, construct reducedsub-nets (A′,B ′, and C ′), relying on observers.
3 Refinement: verify the correctness (behavioral equivalence) of the reducedsub-nets, relying on observers by model checking.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 47 / 59
Property Specific State Space Reduction
What is the benefit of this method?
Benefit
make a trade-off between computation time and space
turns the combination problem of O(N ·M) into a divide-and-conquerproblem of O(n · N + M · δ), where
N is the state unfolding complexity of the target sub-net,M is the complexity of the other parts of the TPN,n is unfolding times of target sub-net by the reduction and refinement,δ is the complexity introduced by the substitute sub-net;it is expected (and often the case according the early test results) that1 ≤ δ � N.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 48 / 59
Property Specific State Space Reduction
FMS: Scalability Test (Boniol and Lauer)
The latency functional chain is enlarged by increasing the number of NDB. Each latencyfunctional chain traverses P NDB, i.e. 2P + 3 functions. P = 1...11.
L1 =req1−−−→ KU1
wpId1−−−−→ FM1query1−−−−→ NDB1
query2−−−−→ ...queryP−1−−−−−−→ NDBP−1
queryP−−−−→ NDBP
answerP−−−−−→ NDBP−1
answerP−1−−−−−−−→ ...answer2−−−−−→ NDB1
answer1−−−−−→ FM1wpInfo1−−−−−→ MFD1
disp1−−−→(1)
NDB/Fun.Prop. Val. (ms) S/T (after R.) Reduction Time
(s)Analysis Time (s) Solving Time (s)
wct bct wct bct wct bct wct bct
1/7 75.2 450.4 9/10 8/9 38.049 2.484 1.860 40.533 39,9092/8 125.2 750.4 9/10 8/9 57.876 2.656 1.883 60.532 59,7593/9 275.2 1050.4 9/10 6/5 79.813 2.812 2.079 82.625 81,892
4/10 375.2 1350.4 9/10 6/5 102.500 2.906 2.079 105.406 104,5795/11 425.2 1650.4 9/10 6/5 124.987 3.015 2.102 128.002 127,0896/12 575.2 1950.4 9/10 6/5 149.359 2.891 2.196 152.250 151,5557/13 675.2 2250.4 9/10 6/5 169.607 2.953 2.227 172.560 171,8348/14 725.2 2550.4 9/10 6/5 193.329 3.031 2.250 196.360 195,5799/15 875.2 2850.4 9/10 6/5 216.239 3.000 2.211 219.239 218,45
10/16 975.2 3150.4 9/10 6/5 239.953 3.047 2.195 243.000 242,14811/17 1025.2 3450.4 9/10 6/5 263.049 3.188 2.195 266.237 265,244
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 49 / 59
Property Specific State Space Reduction
FMS: Scalability Test
0
50
100
150
200
250
300
1 2 3 4 5 6 7 8 9 10 11
Solving Time (s)
NDB Number
Latency for L1
WCT
BCT
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 50 / 59
Feedback Analysis Proposal
Outline
1 Introduction
2 Method to integrate formal verification for DSMLs
3 Property-Driven Approach
4 Semantic Mapping from UML-MARTE to TPN
5 Real-Time Property Specification
6 Observer-Based Property Verification
7 Property Specific State Space Reduction
8 Feedback Analysis Proposal
9 Synthesis
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 51 / 59
Feedback Analysis Proposal
Model Verification Feedback
State of the art
Counterexamples in state-class graph are difficult to analyze
Existing approach provide a set of suspicious component withoutparticular ranking factor
Or animate the error trace in the design model.
Abstraction Issue
Abstraction in design model at early phases.
Abstraction in the mapping from design model to verification model.
Abstraction in state class graph.
Proposal
Rank suspicious components using a suspiciousness factor, when asafety property is not satisfied
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 52 / 59
Feedback Analysis Proposal
Fault Contribution & Error Trace
Definition (Fault Contribution)
Fault Contribution CF (t) is a suspiciousness factor to measure the suspicion level of a transitiont. It is used to rank the suspiciousness of transitions on the error traces.
Definition (Error Trace)
For all the states {si} on the path from an initial state s0 to a violation state sv in thereachability graph, all the outgoing transitions of si are considered as error trace π.
0 41 2 3Svt1
t0 t2t3S0
5
8
7
9
t2 t4
t5t4
6t1
π = {t0, t1, t2, t1, t5, t4, t2, t3, t4}
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 53 / 59
Feedback Analysis Proposal
FMS: Failure Analysis for Latency Property
The bct for latency is 75.2 ms. If we want to check that it is 75.201 ms,the analysis gives the following results:
FunctionFaulty contribution Rank
Rank Var Rank Var %r0 r3 r5 r7 r0 r3 r5 r7
FM1 10,04 9,14 1,46 0,32 1 1 3 2 0,6875 0,0859375MFD1 5,64 5,00 4,91 1,13 2 3 1 1 0,6875 0,0859375KU1 4,98 5,00 4,06 0,16 4 2 2 3 0,6875 0,0859375NDB 5,45 0,58 0,25 0,16 3 6 5 3 1,6875 0,2109375
KU1 FM1 comm 1,03 0,99 0,05 0,03 5 5 6 5 0,1875 0,0234375NDB FM1a comm 1,03 0,12 0,05 0,03 6 7 6 5 0,5 0,0625FM1 MFD1 comm 1,00 1,00 0,99 0,03 8 4 4 5 2,6875 0,3359375FM1 NDB comm 1,01 0,12 0,05 0,03 7 7 6 5 0,6875 0,0859375
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 54 / 59
Synthesis
Outline
1 Introduction
2 Method to integrate formal verification for DSMLs
3 Property-Driven Approach
4 Semantic Mapping from UML-MARTE to TPN
5 Real-Time Property Specification
6 Observer-Based Property Verification
7 Property Specific State Space Reduction
8 Feedback Analysis Proposal
9 Synthesis
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 55 / 59
Synthesis
Synthesis
Property-driven proposal
Minimizing verification semantics by
Semantic mapping from UML-MARTE to TPN.
Specification of real-time requirements by property patterns.
Verification and computation of real-time property by observers.
Property-specific reduction of state space.
Feedback analysis proposal
Ranking suspicious faulty elements based on data mining of failure scenarios.
Prototype toolset
Development of toolset prototype (30264 lines of Java code using Eclipse ModelingFramework).
Experiment
Application to FMS case study and test of scalability.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 56 / 59
Synthesis
Synthesis
Property-driven proposal
Minimizing verification semantics by
Semantic mapping from UML-MARTE to TPN.
Specification of real-time requirements by property patterns.
Verification and computation of real-time property by observers.
Property-specific reduction of state space.
Feedback analysis proposal
Ranking suspicious faulty elements based on data mining of failure scenarios.
Prototype toolset
Development of toolset prototype (30264 lines of Java code using Eclipse ModelingFramework).
Experiment
Application to FMS case study and test of scalability.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 56 / 59
Synthesis
Synthesis
Property-driven proposal
Minimizing verification semantics by
Semantic mapping from UML-MARTE to TPN.
Specification of real-time requirements by property patterns.
Verification and computation of real-time property by observers.
Property-specific reduction of state space.
Feedback analysis proposal
Ranking suspicious faulty elements based on data mining of failure scenarios.
Prototype toolset
Development of toolset prototype (30264 lines of Java code using Eclipse ModelingFramework).
Experiment
Application to FMS case study and test of scalability.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 56 / 59
Synthesis
Synthesis
Property-driven proposal
Minimizing verification semantics by
Semantic mapping from UML-MARTE to TPN.
Specification of real-time requirements by property patterns.
Verification and computation of real-time property by observers.
Property-specific reduction of state space.
Feedback analysis proposal
Ranking suspicious faulty elements based on data mining of failure scenarios.
Toolset prototype
Development of toolset prototype (30264 lines of Java code using Eclipse ModelingFramework).
Experiment
Application to FMS case study and test of scalability.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 56 / 59
Synthesis
Synthesis
Property-driven proposal
Minimizing verification semantics by
Semantic mapping from UML-MARTE to TPN.
Specification of real-time requirements by property patterns.
Verification and computation of real-time property by observers.
Property-specific reduction of state space.
Feedback analysis proposal
Ranking suspicious faulty elements based on data mining of failure scenarios.
Toolset prototype
Development of toolset prototype (30264 lines of Java code using Eclipse ModelingFramework).
Experiment
Application to FMS case study and test of scalability.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 56 / 59
Synthesis
Perspective: Applications
Short term activities
Specify verification-ease property pattern with MARTE CCSL.
Other industrial case studies should be experimented and used to furthervalidate our proposal.
The automated feedback approach can be further experimented andcompared with the existing approaches.
Application to other modeling language
Apply the property-driven and feedback approaches to other end-usermodeling language such as AADL, EAST-ADL or to intermediate languageslike FIACRE.
Redefine semantic mapping.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 57 / 59
Synthesis
Perspective: Mapping Semantics
Resource scheduling semantics mapping
Specify scheduling policies such as Earliest Deadline First, FIFO, Fixed Priority, LeastLaxity First, Round Robin, Time Table Driven, etc.
Analysis of schedulability with specific policy.
Proof of correctness of a provided schedule.
Complete the toolset.
Verification of model transformation
A concern with the semantic mapping approach is whether the model transformation iscorrect.
Ideally, map to different formal models and verify if they converge into the same formalsemantics. Lost between semi-formal and formal semantics cannot be proved, onlyassessed using testing and human proof reading.
Verify some important intended behavioral properties conform to the execution semantics,such as RTC in state machine.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 58 / 59
Synthesis
Thanks for your attention!
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 59 / 59
Semantic mapping
Semantic Mapping for State Machine Diagram
Event pool, vent processing and run-to-completion semantics
Flattening semantics:
Converts a nested state machine model to an unnested model to easethe mapping afterwards.Handles regions, states (composite state and submachine state),external transitions, nested pseudostates (entry/exit point,shallow/deep history, and fork/join).Target model only contains simple states, final states, transitions (localand internal), unnested pseudostates.
Mapping semantics
Maps unnested vertices to TPN
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 60 / 59
Semantic mapping
Semantic Mapping for Composite Structure Diagram
Composite Structure Diagram specifies the internal structure of a class, includingits interaction points to other parts of the system, and the architecture of allparts managed by this class. It is used to explore run-time instances ofinterconnected instances collaborating over communications links.
Coverage Library: UML-MARTE Composite Structure DiagramNode Group Node Type TPN Mapping Coverage
Object
Part√
RoleInterface
√
Port√
CollaborationUse
ConnectionsConnector
√
InterfaceRealizationRole Binding
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 61 / 59
Semantic mapping
Semantic Mapping for Activity Diagram
Activity modeling emphasizes the sequence and conditions for coordinatinglower-level behaviors.
Coverage Library: UML-MARTE Activity DiagramNode Group Node Type TPN Mapping Coverage
Common Activity Partition
Control
Initial Node√
Decision Node√
Merge Node√
Fork Node√
Join Node√
Activity Final√
Flow Final√
Expansion RegionStructured Activity NodeConditional NodeInterruptible Activity RegionLoop NodeSequence Node
Actions Action√
Object
Activity ParameterCentral Buffer
√
DataStore√
ExpansionInput Pin
√
Output Pin√
ConnectionsControl Flow
√
Object Flow√
Exception Handler
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 62 / 59
Semantic mapping
Semantic Mapping for State Machine Diagram
The State Machine package defines a set of concepts that can be used formodeling discrete behavior through finite state- transition systems.
Coverage Library: UML-MARTE State Machine DiagramNode Group Node Type TPN Mapping Coverage
Object
Region√
State√
Composite State√
Submachine State√
ConnectionPointReferenceFinalState
√
Pseudostates
Initial√
Deep History√
Shallow History√
Join√
Fork√
Junction√
Choice√
Entry Point√
Exit Point√
Terminate√
ConnectionsExternal Transition
√
Local Transition√
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 63 / 59
Property specification
Real-Time Property Patterns MetaModel
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 64 / 59
Property specification
Verification of Real-Time Properties
[tmin,tmin] [0,0]
[tmax,tmax]
[0,0]
[0,25000]
MFD_exectrSP_ini6r
[0,0]
MFD_exectrSP_ini6r
[0,0] [0,25000]
BCT Observer WCT Observer
TPN System
... ...
TPN System
... ...
(a) Best Case of Latency Property (a) Worst Case of Latency Property
TesterA
[0,0] [0,0]
Over>owA Over>owB
TesterA
Over>ow2 2
[tmin,tmin] [0,0]
[tmax,tmax]
[0,0]
[0,25000]
MFD_output_kp1_postRDC_take_k_pre
[0,0] [0,0] [100,25200]
BCT Observer WCT Observer
TPN System
... ...
TPN System
... ...
(a) Best Case of Freshness Property (a) Worst Case of Freshness Property
TesterA
[0,0] [0,0]
Over>owA Over>owB
TesterA
Over>ow2 2
MFD_output_kp1_postRDC_take_k_pre
Property Property Value (ms)State/Transition Number Execution Time (s)
Before Reduc. After Reduc. Before Reduc. After Reduc.
LatencySystem N/A 9378/23250 N/A N/A N/AWCT 450.4 67105/145024 9/10 278.313 2.484BCT 75.2 11162/28922 8/9 43.781 3.719
FreshnessSystem N/A 53/85 N/A N/A N/AWCT 316429 259/446 34/44 7.578 3.688BCT 1012 125/202 54/79 7.360 2.125
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 65 / 59
Property specification
Freshness Real-Time Requirements
Allows to ensure that a system variable depending on another variable is fresh enough.The time interval between an event at the end of a functional chain and the earliestprevious event of the dependent event at the beginning of the chain.
Example (Freshness Requirement Example)
On the functional chain:pres1−−−→ RDC1
pres1−−−→ ADIRU1speed1−−−−→ FM1
ETA1−−−→ MFD1disp1−−−→, the worst
case of displaying ETA on the screen by MFD must not be superior to 400 ms.
RDC1
M5
M3
pres1[1]
0 25.010
25
ADIRU1 ADIRU1 ADIRU1
FM1
40 100
FM1 FM1 FM1
160 220
5
295
ETA1[1] ETA1[3]
pres1[1]pres1[4]
speed1[2]speed1[3]
KU2 MFD2 KU2 MFD2 KU2 MFD2 KU2 MFD2 KU2 MFD2 KU2 MFD2
5 280M2
speed1[1]
5
ETA1[2]
disp1[4]disp1[1] disp1[2] disp1[3]
pres1[4]
f1
f2
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 66 / 59
Property specification
Reduction by Equivalent of Sub-nets
Topology-implicit semantic equivalence (not specific)
Redundant zero-time pattern
Sequential patternIndirect initialization patternShorten cycle pattern
Sequential encapsulation pattern
Behavioral equivalence
Cannot anymore reduce using topology-implicit pattern
Necessary to propose reduction method based on property-specificbehavior
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 67 / 59
Property specification
Property Specific Reduction
Algorithms:
Identification of one-way-out pattern, relying on dependency analysis
Identification of generic pattern, relying on dependency analysis
Reduction function, relying on infinity, WCET, BCET observers
Sequential sectionLoop section: search for the loop starting firing occurrence and theloop length
Refinement function, relying on time interval observers
assess the soundness for the sequential and loop section
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 68 / 59
Property specification
Catalog of Observers
Event modifier observers:
E E'
TPN
Structure
Observer
E i : i th Occurrence of E
E−k : kth Occurrence Delay of E
E /k : k Times Slower Sub-occurrence of E
I + t: Time Passed Since System Initialization
E + t: Time Passed Since E
SS&SE : Entering and Exiting Events of a State
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 69 / 59
Property specification
Catalog of Observers
Predicate observers:
EM
TPN
Structure
!!"#$%%&'()*+%
Observer
O(E i ) = true: E i has occurred
isFinite(E ) = True: Bounded Occurrence of E
Freq(EA) · NA = Freq(EB) · NB : Equivalent Occurrence between EA
and EB
T (EA,EB) > t: Minimum Time Interval between Events
T (EA,EB) < t: Maximum Time Interval between Events
D(s) ≥ t: Minimum Time Duration of State
D(s) < t: Maximum Time Duration of State
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 70 / 59
Property specification
Catalog of Observers
Scope modifier observers:
Global
All states, denoted as ABefore E i & After E i
Between EA and EB
Occurrence modifier: Assume that in the state class graph, N(P) is thenumber of states that match the predicate P, N(S) is the number ofstates that match the scope S , and N(P ∧ S) is the number of states thatmatch both the predicate and the scope.
Exist Predicate in Scope:
{N(P ∧ S) ≥ 1 if N(S) > 0;True if N(S) = 0.
Absent Predicate in Scope: N(P ∧ S) = 0
Always Predicate in Scope: N(P ∧ S) = N(S)
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 71 / 59
Property specification
FMS: Scalability Test
Same parameters as the work of Boniol and Lauer
The depth of the case study is extended by increasing P.
KU1
MFD1
KUN
MFDN
ADIRU11
FM1
ADIRUNP
FMN
NDB1
X
X X
XX
RDC2
sensor2
keyboard1 displayN keyboardNdisplay1
X
RDC1
sensor1
ADIRU1P
NDBP...
...............
......
...........
...........
.........
ADIRUN1
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 72 / 59
Property specification
KL-Divergence
Definition (KL-Divergence)
Kullback-Lerbler (KL) Divergence is a measure that quantifies in bits how close aprobability distribution P = {pi} is to a model (or candidate) distribution Q ={qi}. The KL-divergence of Q from P over a discrete random variable is defined as
DKL(P ‖ Q) =∑i
P(i) lnP(i)
Q(i)(2)
Example
A textual document d is a discrete distribution of |d | random variables, where |d |is the number of terms in the document. Let d1 and d2 be two documents whosesimilarity we want to compute. This is done using DKL(d1 ‖ d2) and DKL(d2 ‖ d1).
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 73 / 59
Property specification
TF-IDF
Definition (TF-IDF)
TF-IDF (Term Frequency - Inverse Document Frequency): a numerical statisticwhich reflects how important a term is for a given document in a corpus(collection) of documents. It is often used as a weighting factor in informationretrieval and text mining.
Documents
Error Traces Violation States
Keyword SemanticsTerms
Transitions
Semantic
Contribution
Fault
Contribution
Definition (TC-ITC)
TC-ITC (Transition Contribution - Inverse Trace Contribution),CF (t) = TC (t) · ITC (t).
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 74 / 59
Property specification
Testbed
For a given TPN system S(P,R,M)
P are the processes which run infinitely and need a resource beforethe next task (a task is represented by a transition);
R are resource which are shared by all the processes, but onlyaccessible in an exclusive way;
M is a matrix to decide whether process Pi will need to accessresource Rj .
Create deadlock:
Randomly let some processes during some tasks forget to release aused resource.
These tasks are then considered as the error source of system’sdeadlock.
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 75 / 59
Property specification
Evaluation of Efficiency
Develop a testbed to randomly generate systems (process andresource 5-20) with deadlocks
Test thousands of case with different number of fault (1-8)
Efficiency:
System EvaluationFault Num. Test Num. Av. State/Transition Average Time (s)
1 400 4949 / 15440 2.90922 517 2428 / 7130 1.12443 500 9884 / 31237 3.35334 402 8811 / 26663 2.59985 303 6756 / 18247 1.21966 504 27094 / 75808 5.0647 757 104857 / 304741 15.00728 100 112306 / 283004 15.0289
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 76 / 59
Property specification
Evaluation of Effectiveness
Effectiveness:
EXAM Score: the percentage of statements that have to be examineduntil the first statement containing the fault is reached.
FN.Best Cases Worst Cases Average
EXAM Var. Rank Rank V. EXAM Var Rank Rank V. EXAM Rank
1 0,13335 0,00134 3,25 1,79 0,18603 0,00244 4,33 1,63 0,15969 3,792 0,04229 0,00219 1,1 1,75 0,09574 0,00213 2,11 1,75 0,069015 1,6053 0,02108 0,00106 0,75 1,52 0,05892 0,0009 1,75 1,52 0,04 1,254 0,00722 0,0004 0,26 0,49 0,039 0,00042 1,26 0,49 0,02311 0, 765 0,02044 0,0017 0,83 2,95 0,0478 0,00162 1,83 2,95 0,03412 1,336 0,05369 0,00336 2,46 7,36 0,0766 0,0033 3,46 7,36 0,065145 2,967 0,08857 0,00372 4,61 10,9 0,10822 0,0037 5,61 10,9 0,098395 5,118 0,13091 0,00099 7,3 3,95 0,14905 0,001 8,3 3,95 0,13998 7, 8
Best case EXAM 1% - 13%
Worst case EXAM 4% - 18%
Average EXAM 3% - 16% (rank: 1 - 8)
Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 77 / 59