testing model transformations
TRANSCRIPT
PRUEBAS EN EL DESARROLLO DE SOFTWARE DIRIGIDO POR
MODELOS
Juan de Lara
(trabajo conjunto con Esther Guerra)
modelling&software engineeringresearch grouphttp://www.miso.es
PRUEBAS DE TRANSFORMACIONES DE
MODELOS
Juan de Lara
(trabajo conjunto con Esther Guerra)
modelling&software engineeringresearch grouphttp://www.miso.es
Model-Driven Engineering (MDE).• Meta-Modelling and DSMLs.• Model Transformations.
Testing model-to-model transformations
Conclusions and Open lines
AGENDA
3
MODEL DRIVEN ENGINEERINGIncrease the level of abstraction in software development.
Models are actively used to:• Describe the problem…• Simulate/verify/test…• Generate code for the final application.
Move from the solution space (code-centric) to the problem space:
• Higher levels of productivity and quality.• Less accidental details.
4
MODEL DRIVEN ENGINEERINGFor specific domains:
• Avoid coding the same solutions over and over.
• Families of applications.• Domain-Specific Modelling
Languages.
CodeGenerator
• Modelling, validation and automatic generation of telephony services. 5
MODEL DRIVEN ENGINEERINGFor specific domains:
• Avoid coding the same solutions over and over.
• Families of applications.• Domain-Specific Modelling
Languages.
• Code generation from State-Machines for Railway Signaling Systems. 6
Code generator
MetaEdit+
DOMAIN SPECIFIC MODELLING LANGUAGES
Describe the structure of the domain• Relevant primitives and abstractions.• Relations, features.• Explicit expert knowledge.
7
DOMAIN SPECIFIC MODELLING LANGUAGESDSMLs need not be graphical…
xText 8
MODELS AND META-MODELSThe abstract syntax of DSMLs is defined through a meta-model
• Classes, • Attributes, • Relations.
9
Factory meta-model
Machine
Part
Conveyor
Generator Assembler
inps
outs*
*
* parts
Terminator
1..*
1..*
MODELS AND META-MODELSThe abstract syntax of DSMLs is defined through a meta-model
• Classes, • Attributes, • Relations.
10
«conforms to»
c1:Conveyor
g:Generator
a:Assemblerc2:Conveyor
c3:Conveyor t:Terminator
p2: Partp2: Partouts
outs
inps
inps
outs inps
Factory meta-model
Machine
Part
Conveyor
Generator Assembler
inps
outs*
*
* parts
Terminator
1..*
1..*
OCL CONSTRAINTS
11
Object Constraint Language
Well-formedness rules, which every model should satisfy.
Based on First-Order Logic.
g:Generator
«conforms to»
c:Conveyor
Factory meta-model
Machine
Part
Conveyor
Generator Assembler
inps
outs*
*
* parts
Terminator
1..*
1..*
Factory meta-model
Machine
Part
Conveyor
Generator Assembler
inps
outs*
*
* parts
Terminator
1..*
1..*
context Generator inv: self.inps->isEmpty() and self.outs->size()>0
context Generator inv: self.inps->isEmpty() and self.outs->size()>0context Terminator inv: self.outs->isEmpty() and self.inps->size()>0context Assembler inv: self.inps->size()>0 and self.outs.size()>0…
inps
MODELS AND META-MODELSModels are represented using concrete syntax
• Visual • Textual
No need for a 1-1 correspondence between abstract and concrete syntax elements.
12
asse
MODEL TRANSFORMATIONSModels need to be manipulated for:
• Simulation.• Optimization/refactoring.• Generating another model.• Generating code.
in-place transformations
13…
MODEL TRANSFORMATIONSModels need to be manipulated for:
• Simulation.• Optimization/refactoring.• Generating another model.• Generating code.
SourceTarget
14
model-to-modeltransformations
MODEL TRANSFORMATIONSModels need to be manipulated for:
• Simulation.• Optimization/refactoring.• Generating another model.• Generating code.
15
MMsrc MMtar
Msrc Mtar
Transformationdefinition
from to
«conforms to» «conforms to»
Transformationexecution
Transformationdeveloper
Final user
model-to-modeltransformations
MODEL TRANSFORMATIONSModels need to be manipulated for:
• Simulation.• Optimization/refactoring.• Generating another model.• Generating code.
16
Template languages
query andmodel navigation
17
SPECIFYING MODEL TRANSFORMATIONSModel transformations are not normally specified using general purpose languages
Specialized languages for transformation• Declarative/imperative• Visual/textual• Formal/semi-formal• …
LHS RHS
G H
work
18
SPECIFYING MODEL TRANSFORMATIONSModel transformations are not normally specified using general purpose languages
Specialized languages for transformation• Declarative/imperative• Visual/textual• Formal/semi-formal• …
L R
G H
K
D
l r
m k m*
f d
P.O. P.O.
SPECIFYING MODEL TRANSFORMATIONS
19
Model transformations are not normally specified using general purpose languages
Specialized languages for transformation• Declarative/imperative• Visual/textual• Formal/semi-formal• …
operation main() { // Epsilon Object Language …}operation Machine work(){ for (c in self.inps) { var p : Part := c.parts.random(); c.parts.remove(p); delete p; } for (c in self.outs) c.parts.add(new Part); }http://www.eclipse.org/epsilon/
rule Conveyor2Place { from c : FAC!Conveyor to p : PN!Place (tokens <- c.parts->size())}
Model transformations are not normally specified using general purpose languages
Specialized languages for transformation• Declarative/imperative• Visual/textual• Formal/semi-formal• …
SPECIFYING MODEL TRANSFORMATIONS
20
rule Conveyor2Place { from c : FAC!Conveyor to p : PN!Place (tokens <- c.parts->size())}
Model transformations are not normally specified using general purpose languages
Specialized languages for transformation• Declarative/imperative• Visual/textual• Formal/semi-formal• …
SPECIFYING MODEL TRANSFORMATIONS
21
SourceTarget
Model-Driven Engineering (MDE).
Testing model-to-model transformations• Challenges• PaMoMo: A specification language for MTs• Specification-based MT testing
Conclusions and open lines
AGENDA
22
TRANSFORMATION TESTINGCHALLENGES
23
Urgently needed• we have found 1000+ issues in a repository of ~100 ATL model
transformations (http://www.eclipse.org/atl/atlTransformations/)
Generation of input models• (very) costly if done manually• are we testing interesting cases?
Oracle function• is the result correct?
• syntactically (model conformant to target meta-model)• semantically (produces the intended model)
TRANSFORMATION TESTING APPROACHES
24
Black box• Meta-model based:
• Input models derived covering the input meta-model. • SAT solving/model finding
• Oracle? Random testing: just check for crashes/conformance• Do the models allow testing all properties of interest?
• Specification (contract) based:• Input models derived from a high level specification• The specification serves as oracle• Specifications for model transformations are still rarely built (OCL?)
White box• Look at the transformation logic to generate the tests• Oracle?
A SPECIFICATION LANGUAGE FOR MODEL TRANSFORMATIONSPaMoMo: A formal specification language based on algebraic graph transformation
• Pattern-based, declarative• Category of graphs and morphisms• Compilation into OCL for practical purposes
Pre-conditions: (+/-) conditions on input modelsPost-conditions: (+/-) conditions on output modelsInvariants: relations between input/output models
• Constructive (+)• Constraining (-)
25
CONSTRAINT GRAPHS (1)Encoding of (meta-)models as (typed, attributed) graphs
26
p0: Placename=“inp”tokens=2
t: Transition
in
p1: Placename=“outp”tokens=0
name=“tr”out
p0
t
p1
“inp”
2
“tr”
“outp”
0
in
out
tokens
name
name
name
tokens
Place Transitionin
out
intString
tokensname name
type
CONSTRAINT GRAPHS (2)Encoding of (meta-)models as (typed, attributed) graphs
• G= (V, EV, D, ED,
sV: EV V,
tV: EV V,
sD: ED V,
tD: ED D)• Plus algebras for data types
Typing as graph homomorphism• Four commuting functions
27
p0
t
p1
“inp”
2
“tr”
“outp”
0
in
out
tokens
name
name
name
tokens
Place Transitionin
out
intString
tokensname name
type
CONSTRAINT GRAPHS (3)To express conditions on input/output models, we need:
• Two graphs typed w.r.t source/target meta-models• Attributes of graphs are variables• A formula relating variables of both• C=Gs, Gt,
28
p0: Placename=Y
m: Machinename=X
X=Y
SATISFACTION OF CONSTRAINT GRAPHSSatisfaction of constraint graphs is checked by pattern matching
• Finding graph homomorphisms
29
p: Place
p0: Placename=“inp”tokens=2
t: Transitionin p1: Placename=“outp”tokens=0
name=“tr”
out
m1={(p, p0)} m2={(p, p1)}
SATISFACTION OF CONSTRAINT GRAPHSHow to relate constraint graphs and graphs?
30
p: Place
p0: Placename=“inp”tokens=2
t: Transitionin p1: Placename=“outp”tokens=0
name=“tr”
out
m1={(p, p0)}
tokens=XX>0
SATISFACTION OF CONSTRAINT GRAPHS
31
p: Place
p0: Placename=N1tokens=T1
t: Transitionin p1: Placename=N3tokens=T2
name=N2
out
m1={(p, p0)}
tokens=X
1 X>0
2 N1=“inp” T1=2 N2=“tr” N3=“outp” T2=0
A graph can be seen as a constraint graph• Constraint graph homomorphism = graph morphism + (reverse) formula implication (in the
example )2[X/N1]1
Invariants have a bidirectional nature• Forwards/Backwards satisfaction
32
SATISFACTION OF INVARIANTS
p0: Placename=Y
m: Machinename=X
X=Y
Invariants have a bidirectional nature• Forwards satisfaction
33
SATISFACTION OF INVARIANTS
m: Machinename=X
p0: Placename=Y
m: Machinename=X
X=Y
m: Machinename=“m1”
p0: Placename=“m1”
p1: Placename=“m2”
Invariants have a bidirectional nature• Backwards satisfaction
34
SATISFACTION OF INVARIANTS
p0: Placename=Y
m: Machinename=X
X=Y
m: Machinename=“m1”
p0: Placename=“m1”
p1: Placename=“m2”
p0: Placename=Y
EXAMPLES
35
EXAMPLES
36
EXAMPLES
37
EXAMPLES
38
WHAT CAN WE DO WITH AN SPECIFICATION?Generate “reference” implementations [OGLE09]
• Based on graph transformation• Terminating, correct, complete• …but not very efficient
Testing: • Generation of oracles based on QVT [GL13], OCL [GS15]• Generation of input models, combining constraints [GS15]
SAT-based refinement checking w.r.t:• Other specifications, implementations in ATL [BEGL13]
39
[BEGL13] Büttner, Egea, Guerra, de Lara. “Checking Model Transformation Refinement”. ICMT 2013: 158-173[GL13] Guerra, de Lara, et al. “Automated verification of model transformations based on visual contracts”. Autom. Softw. Eng. 20(1): 5-46 (2013)[GS15] Guerra, Soeken. “Specification-driven model transformation testing”. SoSyM 14(2): 623-644 (2015)[OGLE09] Orejas, Guerra, de Lara, Ehrig. “Correctness, Completeness and Termination of Pattern-Based Model-to-Model Transformation”. CALCO 2009: 383-397
40
SPECIFICATION-BASED TESTING
report
selectspecification
coverage criterion
mtunit script
oracle(assertions)
input test models
.xmi
SAT solver model transformation
transformationspecification
.pamomo
transformationimplementation
.etl, .atl…
refers totester mtunit
engine
12
3
designer
developer
Given a specification, generation of:•input models.•OCL assertions. •Test script
FROM BPMN TO PETRI NETS
41
OUR SPECIFICATION LANGUAGE
42
Features of Pamomo:
•Formal, pattern-based, declarative
•Positive and negative requirements
•Compilation into OCL
P(OneStartEvent)
e.outgoing.size()=1
e
Preconditions Postconditions Invariants(conditions on (conditions on (relations between the elementsinput models) output models) in the input and output models)
N(UnconnectedPlaces)
pl.inarcs.size()=0 andpl.outarcs.size()=0
pl
N
t2pl2
+
pl1
t1.name = pl1.name and t2.name = pl2.name
N(ParallelGateway3)
t1
g
t2
+g
43
1. Translation of properties into OCL
GENERATION OF INPUT MODELS
N
t2pl2
+
pl1
t1.name = pl1.name and t2.name = pl2.name
N(ParallelGateway3)
t1
g
t2
+g
Task::allInstances()->exists(t1 | Task::allInstances()->exists(t2 | ParallelGateway::allInstances()->exists(g | t1.outgoing->includes(g) and t1<>t2 and not t2.outgoing->includes(g) )))
pre-1 inv-1
1pos-1
ocli-1oclp-1
44
1. Translation of properties into OCL
2. Select coverage criteria (e.g. generate models that allow testing one property at a time, two properties, no property, etc.)
=> the selected coverage induces a different set of ocl expressions, e.g., property1 and property2, not property 1 and property2, etc.
GENERATION OF INPUT MODELS
pre-1 inv-1
coverage criteria
1
2
pos-1
ocli-1oclp-1
ocli-1 and ocl-j
45
1. Translation of properties into OCL
2. Select coverage criteria (e.g. generate models that allow testing one property at a time, two properties, no property, etc.)
3. Use constraint solver to generate models conforming to the meta-model and satisfying the OCL expressions
GENERATION OF INPUT MODELS
pre-1 inv-1
coverage criteria
1
2
pos-1
ocli-1oclp-1
ocli-1 and ocl-j
constraint solver3
model-1
sourcemeta-model
46
• A test suite is generated from the specification
• It contains a test case for each invariant and postcondition, defining:
• assertion for the invariant or postcondition• input models to test (all but those generated from
expressions negating the invariant, as they satisfy the invariant vacuously)
GENERATION OF ORACLE
pre-1 inv-1
coverage criteria
1
2
pos-1
ocli-1oclp-1
ocli-1 and ocl-j
constraint solver3
model-1
sourcemeta-model
mtUnittest script(+ oracle)
4
transformation
report
TOOL SUPPORT
47
generation
PAMOMO•EMF-based tool.•Textual/visual syntax for patterns.•Generation of input models and test scripts. •Model generation by using the USE solver.
mtUnit•EMF-based tool.•Automates transformation testing.
Model-Driven Engineering (MDE).
Testing model-to-model transformations
Conclusions
AGENDA
48
CONCLUSIONSMDE: increase the level of abstraction in software development by the use of (domain-specific) models.
V&V of the different MDE artefacts (meta-models, transformations, code generators)
Model transformation testing:• Generation of input models• Oracle function
A specification language like PaMoMo can be used for both
49
CURRENT LINESStatic analysis of (ATL) model transformations
• Type inference• Quick fixes
DSLs for model mutation• Primitives to generate mutants of models conformant to
arbitrary meta-models• Transformations are also models
Model transformation reuse• Transformations used with different meta-models they were
initially built
50