barriers to systematic model transformation testingchechik/courses18/csc2125/week8/... ·...
TRANSCRIPT
![Page 1: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/1.jpg)
BARRIERS TO SYSTEMATIC MODEL
TRANSFORMATION TESTING
Presented By: Lobna AbuSerrieh
Benoit Baudry, Sudipto Ghosh, Franck Fleurey,
Robert France, Yves Le Traon, Jean-Marie Mottu
1
![Page 2: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/2.jpg)
Contents
2
Model Driven Engineering
Model Transformation Testing
Example
Characteristics / barriers of Model Transformation testing
Approaches to overcome these barriers
![Page 3: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/3.jpg)
Introduction
Model Driven Engineering (MDE) :
• Models constitutes the basic units of the development.
• Automated Model transformation plays critical role in
MDE.
• Airbus uses automatic code generation from SCADE
models for embedded controllers in Airbus A380.
• Objecteering: UML and MDA CASE tool which supports
MDE.
3
![Page 4: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/4.jpg)
Model Transformation/ Example
4
Flattening a state machine
MT
a. Hierarchical state machine b. Flattened state machine
![Page 5: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/5.jpg)
Model Transformation/ Metamodel
5
A Hierarchical State Machine Metamodel
![Page 6: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/6.jpg)
Model Transformation/ OCL
6
It is usually necessary to define constraints more
precisely; And it must be added to the metamodel.
OCL is commonly used to define additional constraints.
![Page 7: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/7.jpg)
Model Transformation Testing
7
The correctness of transformation is essential to the
success of MDE.
A fault in transformation can introduce a fault in the
resulted transformation model.
Since model transformations are meant to be reused,
faults present in them may result in many faulty models.
![Page 8: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/8.jpg)
8
Testing :
Prepare the Input – Run – evaluate the Output
Testing Model Transformation:
1. Generate test data
2. Define test adequacy criteria
3. Construct an Oracle
Model Transformation Testing/ Activities
![Page 9: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/9.jpg)
MTT/ Generate test data
9
Test Model for the Flattening Transformation
![Page 10: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/10.jpg)
Characteristics of Model transformation
10
Model transformation has some unique characteristics
which make testing it challenging:
1. Transformation Input and output Complexity.
2. Model management tools.
3. Various Transformation languages.
![Page 11: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/11.jpg)
Characteristics of Model transformation
11
Model transformation has some unique characteristics
which make testing it challenging:
1. Transformation Input and output Complexity.
2. Model management tools.
3. Heterogeneity of Transformation languages.
![Page 12: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/12.jpg)
Complexity of Input and Output data
12
Models are often large.
The metamodels can themselves be large & complex.
Additional constraints using OCL increases the metamodel
complexity.
OCL is a rich language with which it is possible to define
complex constraints relating a large number of elements
in the metamodel.
![Page 13: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/13.jpg)
Complexity of Input and Output data/ Input Data
13
This complexity affects the generation of test models.
Manual test data generation is error-prone.
Automatic test data generation is a complex constraint
solving problem;
It is possible to define a large number of test adequacy
criteria;
However, lack of historical data makes it difficult to
determine the effectiveness of these criteria and the fault
models they can target.
![Page 14: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/14.jpg)
Complexity of Input and Output data/ Output Data
14
Output Complexity complicates the oracle problem.
1. When the expected output model is available, the oracle
needs to compare two models. Then the oracle problem
complexity is NP-complete.
2. If the oracle is specified by listing expected properties of the
output model, then building this oracle is complicated by
the complexity of the output metamodel that describes the
output model.
![Page 15: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/15.jpg)
Characteristics of Model transformation
15
Model transformation has some unique characteristics
which make testing it challenging:
1. Transformation Input and output Complexity.
2. Model management tools.
3. Heterogeneity of Transformation languages.
![Page 16: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/16.jpg)
H/Model Management Environment
16
The construction of models involves either:
1. writing a program that builds the metamodel instances. Or
2. using model editors to manually build the instances, e.g. EMF.
Visualizing output models is difficult because graphical editors
often do not provide adequate support for layout of diagrams
that are produced by a transformation.
A confusing layout complicates manual analysis and the
comparison of two graphical representations. Specially with
regression test.
![Page 17: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/17.jpg)
Characteristics of Model transformation
17
Model transformation has some unique characteristics
which make testing it challenging:
1. Transformation Input and output Complexity.
2. Model management tools.
3. Heterogeneity of Transformation languages.
![Page 18: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/18.jpg)
Heterogeneity of Transformation Languages
18
A Large number of model transformation languages and
techniques exist.
Transformations can be implemented with general purpose
programming languages; or languages dedicated to model
transformations (e.g. QVT). In addition to tool-specific
transformation languages, e.g. Objecteering, MetaEdit+.
Testing techniques need to take this diversity into account.
![Page 19: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/19.jpg)
Promising Approaches/ Input Complexity
19
A constructive approach where models are built first and the
constraints are checked afterwards
Generating objects and assemble them according to
specific criteria in order to build complete models.
Limitation: Large number of of generated models do not
satisfy complete set of constraints
Use SAT solvers to deal with a larger amount of constraints and
generate instances that satisfy the constraints.
![Page 20: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/20.jpg)
Promising Approaches/ Output Complexity
20
Dealing with the Oracle Complexity:
1. When generating a model; test the output model directly.
2. Using partial oracle that checks only specific properties of
the output.
3. Using patterns to express pre- and post-conditions for the
transformation.
4. Using “Design by Contract” when building a model
transformation.
![Page 21: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/21.jpg)
Promising Approaches/ Model Management Environments
21
Model differencing: compares the model produced after
execution of a test case with an expected model.
EMFCompare tool is available in the Eclipse framework.
Versioning of models can benefit testing.
CVS Model is an open source initiative that proposes a tool
for versioning of models.
![Page 22: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/22.jpg)
Promising Approaches/ Heterogeneity of Transformation Languages
22
Dealing with this issue can be tackled by:
1. Specific criteria and associated test generation techniques
for each particular language.
2. Black box techniques that ignore the actual language used
for the transformation.
3. A white-box approach generates test models based on the
structure of the rules used to implement the transformation.
![Page 23: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/23.jpg)
Conclusion
23
Some of the major challenges are identified.
Solutions to some of the testing problems exist, but need more
improvement and work.
A benchmark of realistic models and model transformations
for validation and comparison purposes is needed.
![Page 24: Barriers to systematic Model Transformation Testingchechik/courses18/csc2125/week8/... · 2018-03-06 · H/Model Management Environment 16 The construction of models involves either:](https://reader034.vdocuments.us/reader034/viewer/2022042221/5ec73213730fe931f16175eb/html5/thumbnails/24.jpg)
Questions and Discussion
24