Download - Analysing the concept of quality in model-driven engineering literature: a systematic review
Analysing the concept of quality in
model-driven engineering literature:
a systematic review
Fáber D. Giraldo, Sergio España and Oscar Pastor
RCIS 2014
Marrakesh, Morocco, June 2014
Hi, I am Fáber
Giraldo
Agenda
Introduction & Motivation
Systematic review design
Results achieved
Discussion
Framework by Lindland, Sindre and Sølvberg [1994]
LANGUAGE
AUDIENCE
INTERPRETATION
DOMAINMODEL
PRAGMATICS
SYNTAX
SEMANTICS
completeness=77% completeness=85%
UC 7
PUBLISHING HOUSE
RETURNS SIGNED REPORT
DELIVERY NOTE
UC 13
SALESMAN ISSUES
PUBLISHING HOUSE
INVOICES
SALESMAN
PUBLISHING
HOUSE
UC 13. BASIC FLOW OF ACTIONS:
1. REVIEW DELIVERY NOTES
2. RETRIEVE PUB.HOUSE DATA
3. CALCULATE TOTAL AMOUNT
4. SEND INVOICE TO PUB.HOUSE
PHO 7
PUBLISHING HOUSE RETURNS SIGNED
REPORT DELIVERY NOTESALESMAN
PHO 13
SALESMAN ISSUES PUBLISHING HOUSE
INVOICESSALESMAN
SALESMANPUBLISHING
HOUSE
PUBLISHING
HOUSE
INVOICE
SIGNED D.N.
UC 7
PUBLISHING HOUSE
RETURNS SIGNED REPORT
DELIVERY NOTE
UC 13
SALESMAN ISSUES
PUBLISHING HOUSE
INVOICES
SALESMAN
PUBLISHING
HOUSE
UC 13. BASIC FLOW OF ACTIONS:
1. REVIEW DELIVERY NOTES
2. RETRIEVE PUB.HOUSE DATA
3. CALCULATE TOTAL AMOUNT
4. SEND INVOICE TO PUB.HOUSE
PHO 7
PUBLISHING HOUSE RETURNS SIGNED
REPORT DELIVERY NOTESALESMAN
PHO 13
SALESMAN ISSUES PUBLISHING HOUSE
INVOICESSALESMAN
SALESMANPUBLISHING
HOUSE
PUBLISHING
HOUSE
INVOICE
SIGNED D.N.
completeness=76% completeness=81%
N=35
MOF
ATL
METAMODEL
CA2UML
TRANSFORMATION
COMMUNICATION
ANALYSIS
METAMODEL
UML2 CLASS
DIAGRAM
METAMODEL
REQUIREMENTS
MODEL
UML
CLASS
DIAGRAM
Conforms to
Conforms to Conforms to
Based on Conforms toConforms to Based on
input
executed
output
Conforms to
Introduction & Motivation (I)
Many works on model quality
– Moody and ISO 9126
– SEQUAL (Krosgtie, Lindland, Solverg)
– Good practices in modelling (Mohagheghi)
– UML defects and guidelines
Most proposals are very theoretical (scarce validation).
Lack of conceptual consensus (what is model quality?).
It is unclear whether model-driven engineering principlesare considered.– Complementary modelling languages
– Model transformation
– Code generation
Introduction & Motivation (II)
Problematic phenomena:
– Existing MDE standards do not address quality
– No standard quality framework for MDE
Research goal:
– Review literature on model / modelling language quality
– Structure proposals
– Find out their applicability to MDE
– Discover trends
Agenda
Introduction & Motivation
Systematic review design
Results achieved
Discussion
Systematic review design (I)
Kitchenham’s guidelines for systematic review
Research questions:
– (RQ1) What does quality mean in the context of MDEliterature?
– (RQ2) What does it mean to say that an artifactconforms to the principles of MDE?
– (RQ3) Are current quality-related methods within MDEable to deal with a set of modeling languages?
Systematic review design (II)
Search
– Scientific databases (ACM Digital Lib., IEEExplore, Springer, Scopus, etc.)
– Indexing services (Google Scholar, DBLP )
– Conference Proceedings (CAISE, ER, RCIS, RE, etc.)
– Industrial repositories (OMG)
The process of reading and selecting was performed by a PhD student
under the supervision of two senior researchers
Results achieved (I)
Selection
– Inclusion criteria
– Exclusion criteria
First results Depuration
Systematic review design (III)
Data extraction
– Classification schemas to analyse retrieved studies include:
1. Kinds of artefacts whose quality is measured (e.g. models, modellinglanguages, sets of modelling languages)
2. Kinds of artefacts proposed to measure quality (e.g. methods and tools)
3. Degree of validation and operationalisation of 2.
Agenda
Introduction & Motivation
Systematic review design
Results achieved
Discussion
Results achieved (II)
Results achieved (III)
We identified 11 trends on the studies
– Grouping of studies depending on the theory on which they build.
0 0 0 0 0 0 0 0 0 0 0 0
3
4
5
2
11
0 0 0 0 0
1
0
1 1
0
1
0 0
1 1 1
0 0 0 0 0 0
1
3
0
2
3
4 4
2
1
0 00 0 0 0 0 0 0 0 0 0 0 0
1
0 0 0 00 0 0 0 0 0 0 0 0 0
1
2
0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0
1
3
1
0 00 0 0 0 0 0 0 0
1 1 1
0 0 0 0
1 1
0 0 0 0 0 0
1 1
3
4
3 3
1 1
0 0 00 0 0 0 0 0 0 0 0 0
1 1
2
0
1
0 00 0 0 0 0 0 0 0
1
2
1 1
0 0 0 0 00 0 0 0 0 0
1
0 0 0 0 0 0 0 0 0 0
0
1
2
3
4
5
6
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
Num
ber
of
identifie
d stu
die
s
Year
Studies published by trends and years
MDWE SEQUAL 6C UML guid. MoSMe Tranform.
Empir. UML Underst. UML OPM2SysML Harel&Rumpe Structural
Results achieved (III)
We studied the influence on some authors on others
– We applied a sampling technique to identify influential studies
Agenda
Introduction & Motivation
Systematic review design
Results achieved
Discussion
Discussion (II)
(RQ1) What does quality mean in the context of MDEliterature?
– There are many studies on model/modelling language quality• Most focus on UML
• There is no agreement on the definition of quality (ill-defined)
– MDE community has paid little attention to the topic so far
– The definitions of quality in the trends do not converge
– Many quality evaluation methods and metrics are ad-hoc andspecific for assessing a method created by the same authors• To prove the benefits of their method (comparative experiments)
• To provide a suitable quality evaluation method
– Few quality evaluation methods are supported by tools• Arendt and Taentze complain about the difficulty
Discussion (II)
(RQ2)What does it mean to say that an artifact conformsto the principles of MDE?
– This remains unclear.
– The retrieved definitions about quality in models are a strongbasis to start the discussion
– However, most of the MDE core features are not considered indepth• suitability of languages to engineering goal
• suitability of a set of languages used in combination
• conformity to MDE principles
• management of abstraction levels
• model granularity
Discussion (II)
(RQ3) Are current quality-related methods within theMDE able to deal with a set of modeling languages?
– Complex information systems require multiple viewpointssupported by conceptual models (different modellinglanguages)
– Practically all studies deal with a single language
– Those who consider a set of languages do not provide asystematic, repeatable and generalisable method to evaluateany set of languages
– The evaluation of a set of modelling languages used incombination remains an open research question.
Discussion (IV)
Interesting future works
– Operationalise model quality proposals using technologicalframeworks such as EMF, MetaEdit...
– Compare model quality proposals when applied to specificMDE contexts• Laboratory research (controlled experiment)
• Field research (action-research, case study)
– Design a method for evaluating quality in MDE contexts(building on top of existing proposals)• Conceptual framework
• Methodological guidance
• Tool support
Analysing the concept of quality in model-driven
engineering literature: a systematic review
Fáber D. Giraldo, Sergio España and Oscar Pastor
RCIS 2014
Marrakesh, Morocco, June 2014