analysing the concept of quality in model-driven engineering literature: a systematic review
TRANSCRIPT
![Page 1: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/1.jpg)
Analysing the concept of quality in
model-driven engineering literature:
a systematic review
Fáber D. Giraldo, Sergio España and Oscar Pastor
RCIS 2014
Marrakesh, Morocco, June 2014
![Page 2: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/2.jpg)
Hi, I am Fáber
Giraldo
![Page 3: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/3.jpg)
Agenda
Introduction & Motivation
Systematic review design
Results achieved
Discussion
![Page 4: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/4.jpg)
![Page 5: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/5.jpg)
![Page 6: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/6.jpg)
![Page 7: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/7.jpg)
![Page 8: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/8.jpg)
Framework by Lindland, Sindre and Sølvberg [1994]
LANGUAGE
AUDIENCE
INTERPRETATION
DOMAINMODEL
PRAGMATICS
SYNTAX
SEMANTICS
completeness=77% completeness=85%
![Page 9: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/9.jpg)
UC 7
PUBLISHING HOUSE
RETURNS SIGNED REPORT
DELIVERY NOTE
UC 13
SALESMAN ISSUES
PUBLISHING HOUSE
INVOICES
SALESMAN
PUBLISHING
HOUSE
UC 13. BASIC FLOW OF ACTIONS:
1. REVIEW DELIVERY NOTES
2. RETRIEVE PUB.HOUSE DATA
3. CALCULATE TOTAL AMOUNT
4. SEND INVOICE TO PUB.HOUSE
PHO 7
PUBLISHING HOUSE RETURNS SIGNED
REPORT DELIVERY NOTESALESMAN
PHO 13
SALESMAN ISSUES PUBLISHING HOUSE
INVOICESSALESMAN
SALESMANPUBLISHING
HOUSE
PUBLISHING
HOUSE
INVOICE
SIGNED D.N.
![Page 10: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/10.jpg)
UC 7
PUBLISHING HOUSE
RETURNS SIGNED REPORT
DELIVERY NOTE
UC 13
SALESMAN ISSUES
PUBLISHING HOUSE
INVOICES
SALESMAN
PUBLISHING
HOUSE
UC 13. BASIC FLOW OF ACTIONS:
1. REVIEW DELIVERY NOTES
2. RETRIEVE PUB.HOUSE DATA
3. CALCULATE TOTAL AMOUNT
4. SEND INVOICE TO PUB.HOUSE
PHO 7
PUBLISHING HOUSE RETURNS SIGNED
REPORT DELIVERY NOTESALESMAN
PHO 13
SALESMAN ISSUES PUBLISHING HOUSE
INVOICESSALESMAN
SALESMANPUBLISHING
HOUSE
PUBLISHING
HOUSE
INVOICE
SIGNED D.N.
completeness=76% completeness=81%
N=35
![Page 11: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/11.jpg)
![Page 13: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/13.jpg)
![Page 14: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/14.jpg)
MOF
ATL
METAMODEL
CA2UML
TRANSFORMATION
COMMUNICATION
ANALYSIS
METAMODEL
UML2 CLASS
DIAGRAM
METAMODEL
REQUIREMENTS
MODEL
UML
CLASS
DIAGRAM
Conforms to
Conforms to Conforms to
Based on Conforms toConforms to Based on
input
executed
output
Conforms to
![Page 15: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/15.jpg)
Introduction & Motivation (I)
Many works on model quality
– Moody and ISO 9126
– SEQUAL (Krosgtie, Lindland, Solverg)
– Good practices in modelling (Mohagheghi)
– UML defects and guidelines
Most proposals are very theoretical (scarce validation).
Lack of conceptual consensus (what is model quality?).
It is unclear whether model-driven engineering principlesare considered.– Complementary modelling languages
– Model transformation
– Code generation
![Page 16: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/16.jpg)
Introduction & Motivation (II)
Problematic phenomena:
– Existing MDE standards do not address quality
– No standard quality framework for MDE
Research goal:
– Review literature on model / modelling language quality
– Structure proposals
– Find out their applicability to MDE
– Discover trends
![Page 17: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/17.jpg)
Agenda
Introduction & Motivation
Systematic review design
Results achieved
Discussion
![Page 18: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/18.jpg)
Systematic review design (I)
Kitchenham’s guidelines for systematic review
Research questions:
– (RQ1) What does quality mean in the context of MDEliterature?
– (RQ2) What does it mean to say that an artifactconforms to the principles of MDE?
– (RQ3) Are current quality-related methods within MDEable to deal with a set of modeling languages?
![Page 19: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/19.jpg)
Systematic review design (II)
Search
– Scientific databases (ACM Digital Lib., IEEExplore, Springer, Scopus, etc.)
– Indexing services (Google Scholar, DBLP )
– Conference Proceedings (CAISE, ER, RCIS, RE, etc.)
– Industrial repositories (OMG)
The process of reading and selecting was performed by a PhD student
under the supervision of two senior researchers
![Page 20: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/20.jpg)
Results achieved (I)
Selection
– Inclusion criteria
– Exclusion criteria
First results Depuration
![Page 21: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/21.jpg)
Systematic review design (III)
Data extraction
– Classification schemas to analyse retrieved studies include:
1. Kinds of artefacts whose quality is measured (e.g. models, modellinglanguages, sets of modelling languages)
2. Kinds of artefacts proposed to measure quality (e.g. methods and tools)
3. Degree of validation and operationalisation of 2.
![Page 22: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/22.jpg)
Agenda
Introduction & Motivation
Systematic review design
Results achieved
Discussion
![Page 23: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/23.jpg)
Results achieved (II)
![Page 24: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/24.jpg)
Results achieved (III)
We identified 11 trends on the studies
– Grouping of studies depending on the theory on which they build.
0 0 0 0 0 0 0 0 0 0 0 0
3
4
5
2
11
0 0 0 0 0
1
0
1 1
0
1
0 0
1 1 1
0 0 0 0 0 0
1
3
0
2
3
4 4
2
1
0 00 0 0 0 0 0 0 0 0 0 0 0
1
0 0 0 00 0 0 0 0 0 0 0 0 0
1
2
0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0
1
3
1
0 00 0 0 0 0 0 0 0
1 1 1
0 0 0 0
1 1
0 0 0 0 0 0
1 1
3
4
3 3
1 1
0 0 00 0 0 0 0 0 0 0 0 0
1 1
2
0
1
0 00 0 0 0 0 0 0 0
1
2
1 1
0 0 0 0 00 0 0 0 0 0
1
0 0 0 0 0 0 0 0 0 0
0
1
2
3
4
5
6
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
Num
ber
of
identifie
d stu
die
s
Year
Studies published by trends and years
MDWE SEQUAL 6C UML guid. MoSMe Tranform.
Empir. UML Underst. UML OPM2SysML Harel&Rumpe Structural
![Page 25: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/25.jpg)
Results achieved (III)
We studied the influence on some authors on others
– We applied a sampling technique to identify influential studies
![Page 26: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/26.jpg)
Agenda
Introduction & Motivation
Systematic review design
Results achieved
Discussion
![Page 27: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/27.jpg)
Discussion (II)
(RQ1) What does quality mean in the context of MDEliterature?
– There are many studies on model/modelling language quality• Most focus on UML
• There is no agreement on the definition of quality (ill-defined)
– MDE community has paid little attention to the topic so far
– The definitions of quality in the trends do not converge
– Many quality evaluation methods and metrics are ad-hoc andspecific for assessing a method created by the same authors• To prove the benefits of their method (comparative experiments)
• To provide a suitable quality evaluation method
– Few quality evaluation methods are supported by tools• Arendt and Taentze complain about the difficulty
![Page 28: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/28.jpg)
Discussion (II)
(RQ2)What does it mean to say that an artifact conformsto the principles of MDE?
– This remains unclear.
– The retrieved definitions about quality in models are a strongbasis to start the discussion
– However, most of the MDE core features are not considered indepth• suitability of languages to engineering goal
• suitability of a set of languages used in combination
• conformity to MDE principles
• management of abstraction levels
• model granularity
![Page 29: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/29.jpg)
Discussion (II)
(RQ3) Are current quality-related methods within theMDE able to deal with a set of modeling languages?
– Complex information systems require multiple viewpointssupported by conceptual models (different modellinglanguages)
– Practically all studies deal with a single language
– Those who consider a set of languages do not provide asystematic, repeatable and generalisable method to evaluateany set of languages
– The evaluation of a set of modelling languages used incombination remains an open research question.
![Page 30: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/30.jpg)
Discussion (IV)
Interesting future works
– Operationalise model quality proposals using technologicalframeworks such as EMF, MetaEdit...
– Compare model quality proposals when applied to specificMDE contexts• Laboratory research (controlled experiment)
• Field research (action-research, case study)
– Design a method for evaluating quality in MDE contexts(building on top of existing proposals)• Conceptual framework
• Methodological guidance
• Tool support
![Page 32: Analysing the concept of quality in model-driven engineering literature: a systematic review](https://reader034.vdocuments.us/reader034/viewer/2022042614/555be3e0d8b42a38078b4d0f/html5/thumbnails/32.jpg)
Analysing the concept of quality in model-driven
engineering literature: a systematic review
Fáber D. Giraldo, Sergio España and Oscar Pastor
RCIS 2014
Marrakesh, Morocco, June 2014