1

Search-Based Model-Driven EngineeringJames R. Williams, Simon Poulding,

Richard F. Paige, Dimitrios S. Kolovos, Fiona A. C. PolackDepartment of Computer Science, University of York,

Deramore Lane, York, YO10 5GH, UK.{jw,smp,dkolovos,paige,fiona}@cs.york.ac.uk

Abstract—Model-Driven Engineering (MDE) is a software engineering approach that treats models as the primary developmentartefacts. Models are manipulated, using model management operations (MMO), in order to perform engineering tasks such ascomparison, validation, and code generation. We propose that many of the challenges faced in MDE could be addressed usingSearch-Based Software Engineering (SBSE) techniques: the two approaches, MDE and SBSE, are natural partners. However,a significant barrier preventing widespread application of SBSE to MDE is a suitable representation of MDE models that isamenable to metaheuristic search algorithms while also applicable across a wide range of MDE problem domains. This paperpresents such a representation and describes an implementation using a widely used MDE toolset. We illustrate the utility of therepresentation by applying it to the white-box analysis of three MMOs. Furthermore, we discuss other areas of MDE which couldbenefit from the application of SBSE techniques, and highlight research opportunities for combining the two fields.

F

1 INTRODUCTION

Model-Driven Engineering (MDE) [31], [9] is a softwareengineering practice which uses disparate models ofsystems and software to (often automatically) producean end product. Models are treated as the primarydevelopment artefacts and are manipulated by modelmanagement operations (MMO), such as model com-parison, model merging, model validation, and perhapsmost commonly, model transformation. Since MDE aimsto raise the level of abstraction at which develop-ers work, it faces its own unique set of challengesalongside many of those found in traditional softwareengineering.

Search-based software engineering (SBSE) [14], [6], [13]is based on the observation that it is often easierto check that a candidate solution solves a problemthan it is to construct a solution to that problem.In this situation, metaheuristic search algorithms maybe used to efficiently locate (near) optimal solutionsby assessing how close candidate solutions are tosolving the problem and using this information toguide the search over the the space of all possiblesolutions. Applications of SBSE have been growing inscope and sophistication over the last 10 years, withresearchers demonstrating success in applying search-based optimisation techniques to a wide variety ofsoftware engineering problems [14].

In this paper we motivate the case for applyingSBSE techniques to challenges faced in MDE. Since thesolution to many of these challenges takes the formof a model, we propose a search-amenable representa-tion that encodes the important features of the modelspace and therefore permits the search algorithmstypically used in SBSE to be applied to these MDEchallenges. The paper is structured as follows. Section

2 overviews MDE and SBSE, and illustrates the areasof MDE that could benefit from SBSE techniques. Sec-tion 3 presents our proposed representation of MDEmodels. Section 4 discusses our implementation of therepresentation for a widely used MDE toolset, andillustrates its utility by using it to perform coverageanalysis on three MMOs. Section 5 outlines manyopportunities where SBSE techniques can providemany benefits to MDE. Finally, section 6 concludesby inspecting the limitations of the representation.

2 SEARCH-BASED AND MODEL-DRIVEN

This section overviews the key concepts in MDE andSBSE before illustrating how SBSE can help to addresssome of the challenges faced in MDE.

2.1 Model-Driven Engineering

MDE aims to improve important aspects of softwareengineering, such as productivity, maintainability, andinteroperability. Hutchinson et al. empirically evaluatethe use of MDE in industry and provide evidencefor some of these perceived benefits, whilst notingthe need for appropriate supporting processes andtechniques [17]. MDE prescribes models to be firstclass citizens in the software development process[31], [9]. The goal of MDE is to model systems at thelevel of the application domain, and apply a series ofautomatic transformations to produce code. Raisingthe level of abstraction can reduce the complexity ofthe problem, whilst also allowing domain experts tobecome the primary developers of systems; softwareengineers produce the modelling languages and in-frastructures that support the domain experts [31],[32].

2

Fig. 1: A metamodel for a simple language whichallows zookeepers to assign animals to cages.

Fig. 2: A example model which conforms to themetamodel in figure 1.

Models in MDE are said to conform to a metamodel– another model that describes the concepts that amodel can use, and any constraints over those con-cepts. Metamodels can be viewed as the grammar, orabstract syntax, for modelling languages. The concretesyntax of a model is commonly implemented graphi-cally, but can be expressed textually as well, and thereare many existing tools that aid in the developmentof concrete syntaxes (e.g. GMF1 and Graphiti2 forgraphical, and Xtext3 for textual syntaxes).

To clarify these concepts, we present an illustrativeexample of a metamodel and model. Figure 1 showsa metamodel that describes a simple language formodelling the placement of animals in a zoo, andFigure 2 shows an example model conforming to thismetamodel, represented as an object diagram.

Disparate models have different uses, which canbe broadly categorised into two classes: developmentmodels and runtime models [9]. Development modelsmay be used for abstractly modelling the implementa-tion before generating code via a series of automatedtransformations. Alternatively, they may simply beused for problem illustration or analysis. Runtimemodels capture information about an executing sys-tem, enabling software to be dynamically adapted.

There are many kinds of model transformations;the two common classes being model-to-model and

1. www.eclipse.org/modeling/gmp2. www.eclipse.org/graphiti3. www.eclipse.org/Xtext

model-to-text transformations [7]. Other kinds of modelmanagement operation include: comparison, e.g. todiscover inconsistencies between two versions of amodel; merging, e.g. to combine different views ofa system into one model; validation, e.g. to ensure amodel satisfies specific domain constraints. It is thecombination of these operations into intricate work-flows that enables large and complex systems to bedeveloped more effectively.

2.2 Search-Based Software EngineeringIn SBSE, software engineering problems are couchedas optimisation problems, where the search space isthe set of all possible solutions to the problem. Therationale behind SBSE is the observation that it is ofteneasier to determine whether one solution is betterthan another, than to construct an optimal solutionto a problem [14], [6], [13]. In order to successfullyreformulate a software engineering problem as an op-timisation problem, three tasks need to be considered:

1) Define a characterisation, or representation, of so-lutions that is amenable to search techniques(applicable where the native format of solutions isparticularly complex) and can be easily mappedto the native format. The representation (com-monly strings of bits or integers) is known as thegenotype, and the native format is known as thephenotype.

2) Define a set of search operators – ways in which so-lutions are varied in order to effectively traversethe solution space.

3) Define an objective function to calculate the “fit-ness” of candidate solutions – a measure of howclose the solution is to solving the desired prob-lem.

The choice of representation and search operatorsare key to the efficacy of the search [29].

Metaheuristic search algorithms focus on identify-ing (near) optimal solutions to a problem by searchingover the solution space – the set of possible solutionspresented for consideration [6]. Metaheuristic searchalgorithms can be categorised into two kinds [6]: localsearch, which focuses on improving a single solution;and population-based search, which focuses on improv-ing the fitness of a population of solutions.

2.2.1 Local SearchThe most basic local metaheuristic search algorithmis hill-climbing: the starting candidate solution is cho-sen at random, and its fitness is compared to eachof its neighbours (those solutions deemed “close” tothe current solution though application of a singlesearch operator). If a fitter neighbour is found, itbecomes the current candidate solution, and each ofits neighbours is examined. In this way the search“climbs” the fitness landscape until no adjacent solu-tion is fitter than the current candidate. Hill-climbing

3

works well in simple fitness landscapes, but cannotescape from a local optimum in the case of a morerugged fitness landscape. This means that the selectedsolution may not be the global optimum. Extensionsof the algorithm, such as random restart hill climbing,or techniques such as simulated annealing overcome theproblem of local optima by allowing less-fit solutionsto be adopted, with some probability, to allow thesearch to move away from local optima [6], [22].

2.2.2 Population-Based Search

Other metaheuristic search techniques focus on im-proving a population of solutions, as opposed to a sin-gle solution, whilst attempting to identify a globallyoptimal solution. The most widely known population-based algorithm is the genetic algorithm (GA) [6], [10],a type of evolutionary algorithm. Inspired by biologi-cal evolution, GAs select a population of candidatesolutions which are combined and mutated to de-velop a new population. A solution is referred to asan individual, and is typically represented as a bit-string or a sequence of integers. Each componentbit or integer is called a gene. Each gene encodesone or more aspects of the solution’s native form,the phenotype. When seeking an optimal solution, aGA is executed for many generations, each of whichproduces a new population. The fittest chromosomesfrom the previous generation are selected and, inorder to maintain the diversity necessary to avoidlocal optima, are mutated and combined with otherfit solutions. Often the fittest solutions, the elite, arekept unaltered in the new population in an attemptto increase the population’s overall fitness. There area variety of possible termination conditions: the GAmay run for a set number of generations, or until adesired fitness is attained, or until solutions convergeand no further improvement in fitness is detected.

2.3 Connecting SBSE with the Challenges Facedin MDE

The observation that premises SBSE is inherent inMDE. Developing an optimal model of a domain isextremely difficult, however it can be quite straight-forward to determine whether a model accuratelycaptures the domain. It is unrealistic to think thatsearch could solve all problems associated with mod-elling, however the solutions to many of the chal-lenges and tasks faced in MDE take the form ofmodels, indicating that utilising SBSE methods couldprove fruitful.

France and Rumpe [9] enumerate three categoriesof challenges faced in MDE: modelling language chal-lenges; separation of concerns challenges; and, modelmanipulation and management challenges. We feelthat SBSE techniques can be a positive influence intackling, in particular, challenges in the latter category.

To present an overview of MDE issues and tasks re-lated to model manipulation and management whereSBSE techniques may help, we separate them intotwo groups and describe them here. Existing workthat utilises SBSE techniques to address the problemsbelow, will be presented in section 5.

2.3.1 Discovering and Validating ModelsModelling of any kind is subject to different lev-els of uncertainty, such as errors of measurementor interpretation, incomplete information, and pooror partial understanding of the domain [30]. WhenMMOs are applied to a model, uncertainty can lead tounexpected behaviour, or a small change in a modelmight result in a large change in the output of theMMOs. There are many cases where we wish to findan optimal model of a certain domain. For instance,taking the example in Figure 2, we might want tofind an optimal allocation of animals to cages suchthat no animal shares a cage with one that eats its –this is a variation of the bin packing problem. Modeland metamodel discovery and validation, therefore,are key challenges in MDE.

2.3.2 Discovering and Validating MMOsThe uncertainty involved with developing models isalso prevalent in the definition of MMOs. For ex-ample, one of the challenges found in MDE is thatof metamodel evolution [15]. When a metamodel isupdated to a new version, existing models may nolonger conform to the metamodel – these models needto be migrated. For example, we may make the Animalclass in Figure 1 abstract, and define two concretesubclasses, Mammal and Reptile. The model in Figure2 no longer conforms to the metamodel, as abstractclasses cannot be instantiated. We migrate models byapplying a number of transformation rules, called amigration strategy, but it may not be obvious what thebest strategy is. One solution may be to simply deletethe Animal objects – this would result in a model thatconforms to the new version of the metamodel, buthas lost information. A better solution would be toconvert the two Animal objects into instances of theMammal class. Furthermore, there may exist numer-ous valid, semantic-preserving, migration strategies,some of which may be more efficient than others withrespect to non-functional properties.

Validating a migration strategy (or any form ofMMO) is a challenging problem. Where viewed fromthe perspective of testing traditional software, it canbe seen that the test inputs (i.e., models) are extremelycomplex and this increases the difficulty of test dataselection and the definition of an oracle [1], [2]. Webelieve that the body of knowledge from the search-based software testing field could be very useful inaddressing this problem, as much work has lookedat ways to improve traditional software testing [23],[24].

4

2.4 Summary and MotivationThis section has given an overview of MDE andsome of the challenges faced by practitioners. Fora comprehensive review of the state of MDE andits challenges, see [9], [2], [34]. If these challengeswere couched as search problems, many of them(e.g. testing MMOs, discovering/optimising models)would share the same phenotypical form – models.Therefore, if we had a generic representation formodels, we could make use of the wealth of existingsearch practices to help solve MDE problems. Thenext section introduces a generic, search-amenablerepresentation for MDE models.

3 A CANONICAL REPRESENTATION FORMODELS

Clark et al. identify the process of reformulating asoftware engineering problem as a search problem as(a) finding a characterisation that enables solutions tothe problem to be represented in a simple mannerwhich is amenable to search; and (b), defining anobjective function that measures how close a candidatesolution is to solving the problem under scrutiny [6].The objective function is normally specific to a singleproblem domain. However, there is an opportunity todefine a canonical representation for MDE models thatis suitable for use across a range of problem domainsand is amenable to many of the search algorithmstypically used in SBSE.

This section introduces our generic representationfor MDE models, describing the structure of an indi-vidual, the genotype-phenotype mapping, the searchoperators and properties specific to the representation.

3.1 A Linear Representation for ModelsRothlauf [29] defines desirable characteristics of a rep-resentation for genetic and evolutionary algorithms,including: not encoding infeasible solutions; havinglow redundancy – reducing the number of elementswhich are unused in the mapping to a solution; andaiming for high locality – meaning that a small changeto part of an individual will also result in a smallchange to the solution it represents.

Models in MDE are represented as object graphs.A direct representation, where the search space is thesolution space (or equivalently using GA terminology,the genotype is the phenotype), would ensure an effi-cient encoding. However, significant work would berequired to define custom search techniques and de-vise custom evolutionary operators to be used by thesearch algorithm. Furthermore, models can get verylarge, which could make search impractical due to thevast memory requirements in cases of large modelsand large populations. To alleviate these problemswe propose using a linear representation of models.A linear genotype enables the use of many linear

genotype-based search algorithms, such as geneticalgorithms, evolutionary strategies, simulated anneal-ing and hill climbing. This allows MDE practitionersto determine whether different algorithms performbetter for different problems, and make use of thewealth of existing research in this area.

This section presents our linear representation ofMDE models. Firstly, we introduce the finitisationmodel, which is used to declare the data available inthe mapping process. We then explain the structureof an individual and describe the mapping frominteger string to model. Finally, we discuss some ofthe interesting characteristics of the representation.

3.1.1 Metamodel Data FinitisationWhen designing metamodels, it is common to useunbounded data types for attributes, or to assignone-to-many multiplicities to references instead ofenforcing an upper bound. This results in an infinitelylarge model space: there are an infinite number ofmodels that can conform to the metamodel. In orderto make the model space amenable to search, werequire the user to finitise the data in their metamodel,similar to the creation of the terminal set used inGenetic Programming (GP) [27]. For instance, in ourzoo example, we may wish to restrict the sizes of cagesto be integers between 5 and 50, and not the entirerange of integers. We have defined a metamodel (notshown for brevity) that allows practitioners to cap-ture finitisation information in a model. Metamodelattributes can be finitised with a list or a range ofpossible values, and both meta-classes and referencescan have a maximum scope defined. Scoping a meta-class restricts the number of its instances that canappear in a model, and scoping a reference enforcesan upper bound on the number of objects assignedto that reference. Furthermore, it is also possible tofinitise entire data types with a list or range of possiblevalues.

One requirement of the finitisation model, is thatthe user needs to define a root object type – the con-tainer object in which all model elements are situated.In MDE models, it is usual for the model to have atop-level container object (for example, a package in aUML class diagram), and it is not always possible toautomatically infer from the metamodel which classthis root object should be an instance of.

Finally, a user can choose to ignore certain metaelements (such as specific classes or attributes) if theyare not of interest to the task at hand. As with theterminal set in GP, a user may specify different meta-model data finitisations depending on the problembeing addressed.

3.1.2 The GenotypeThe representation we have defined is linear; moreprecisely, an individual in our representation is astructured string of integers. As shown in figure 3,

5

1 0 2 1 1 2 0 1 0 2 1 1 2 0 1 0 2 1 1 2 0

individual

1 0 2 1 1 2 0

segment

(a) (b)

feature pair

class node feature nodes

Fig. 3: The structure of an individual in the representation. Segments are the building block for representingmodels as integers. (a) is the feature selector node, and (b) is the feature value node.

this string is divided into a number of segments, witheach segment representing a single object in the modelbeing represented. The first node in a segment, theclass node, identifies which meta-class is instantiatedby the object being represented. Successive nodes, thefeature nodes, define the values of features from thatmeta-class. Feature nodes are grouped into pairs: thefirst member of the pair, the feature selector node, iden-tifies a feature of the meta-class; the second memberof the pair, the feature value node, specifies the valuethat should be assigned to that feature in the objectbeing represented.

3.2 Genotype-Phenotype MappingThe steps taken to transform an individual into amodel (the mapping from genotype to phenotype)are presented below, and illustrated with a simpleexample shown in figure 4.

The first two steps are performed by the user toconfigure the search:

1) Assign identifiers to features in the metamodel:In order to reference meta-elements (i.e. elementsin the metamodel), we assign identifiers to them(see figure 4a). Each instantiable meta-class isgiven an identifier, and each of its meta-featuresis also assigned an identifier, unique to that meta-class. Meta-elements that cannot be instantiated,such as enumeration types, are not assigned aclass-level identifier, however the enumerationliterals are assigned feature identifiers. Further-more, we declare one class to be the root class.All objects created during the mapping will be di-rectly or indirectly contained by a single instanceof the root class.

2) Finitise any infinite data: In order to representthe value(s) of a feature, we define a finitisationmodel for the metamodel (see figure 4b). We alsoassign identifiers to each of the data values inorder to reference them in the mapping.

The final two steps take place for every individualevaluated by the search algorithm. These steps map

(a) Step 1: Assigning identifiers to the zoo metamodel.

(b) Step 2: Finitisation information for the zoometamodel. (Simplified to make annotation legi-ble.)

(c) Step 3: Mapping a segment to anAnimal object.

Fig. 4: The steps taken to instantiate an object from ourinteger-based representation. Meta-element identifiersare displayed in grey circles.

6

an individual into a candidate solution which can thenbe analysed by the relevant fitness function.

3) Map each segment to a model object: A concretemodel object is created by mapping a segmentusing the definition in figure 3, and the meta-object identifiers defined in figures 4a and 4b.Figure 4c shows how we resolve the segment’sclass node (1) to the Animal class; so we create anAnimal object. Next, we resolve each of the featurepairs. In this example, we assign the Animal objectthe name “Tiger”, set its spaceRequired attributeto 5, and its quantity attribute to 2. Each of thevalues in the segment is interpreted modulo thenumber of valid values (i.e. the number of meta-classes for a class bit, or the number of featuresin a certain meta-class for a feature bit).

4) Construct references: When the feature selectornode maps to a reference feature, we take noteand move to the next feature pair. This is becausenot all objects will have been instantiated, andso we are unable to assign the references. Onceall segments (ignoring references) have beenmapped to objects, and those objects instantiated,we can then assign the references. To assign areference, we select the set of objects whose typeis the same as the reference target, and then takethe feature value modulo the size of the list toselect one of those objects. If the reference’s upperbound has already been reached, we ignore thatfeature pair.

Insisting that models have a single root containerobject (Step 1) requires us to make a decision as towhich one of those objects should become the rootobject. The mapping process may create multiple ob-jects of the root’s type and the reference constructionphase may distribute objects evenly amongst theseroot objects – selecting one of the root objects to keepwould mean removing the other roots and all objectsthat they contain, creating a lot of redundancy inthe individual. To address this, our transformationalways creates a single object as the root, irrespectiveof the individual. Furthermore, the source metamodelis automatically analysed to determine whether theroot object is allowed (directly or indirectly) to containobjects of its own type. If not, the root meta-classis not given an identifier and therefore cannot beinstantiated. One drawback of this solution, is that, asthe root object doesn’t have a corresponding segment,we cannot assign values to any attributes it maypossess.

3.3 Search Properties and Genetic OperatorsHere we briefly describe the search-specific character-istics of the representation – the genetic operators andmethod of initialising the search population, includinga discussion about the size of individuals. As men-tioned, due to being based on a linear genotype, our

representation can be used by many linear genotype-based search algorithms. Therefore, each search tech-nique would make use of the appropriate initialisationmethod or specific genetic operators required.

3.3.1 InitialisationIn the general case, the initial search populationof a population-based algorithm (such as a geneticalgorithm) may be generated by assigning randomvalues to each gene. Certain applications, however,may require a different method. For example, it maybe beneficial to seed the population with variants onan existing model should the goal be to optimise aspecific model. Similarly, for a local search algorithm,a random starting point can be chosen or an existingmodel could be mapped down to its genotype andused.

The length of an individual (|I|) plays an importantrole in determining the model space that can berepresented. It is defined as follows:

|I| =n∑

i=1

|Si|

where |Si| is the length of the ith segment and seg-ment length is defined as:

|S| = 1 + (2 ∗#fp)

where #fp is the number of feature pairs in thatsegment.

The segment length limits the number of structuralfeatures an object can instantiate, and the number ofsegments determines the overall size of the model.To allow for a diverse population, the number offeature pairs can differ between segments. An alter-native would be to use a fixed number of featurepairs, however this would introduce a high level ofredundancy in segments which represent meta-classeswith few features.

Currently, the number of feature pairs and numberof segments per individual are specified by the useras a range. The initialisation process randomly selectsa value from these ranges for each segment and eachindividual created. In the future these values couldbe inferred by analysing the metamodel. Individualsare not required to be the same length, to allowfor a much more diverse population and to morerealistically represent the solution space. Future workis planned to examine the effect of individual lengthon the results of different problems.

3.3.2 Standard Genetic Operators, AdaptedThe linear representation permits standard geneticoperators, such a single-point crossover, to be used.However, the structure of chromosomes in our rep-resentation means that crossover can be particularlydestructive if it is allowed to occur at any point in

7

1 0 0 1 0 0 0 1 1 0

0 0 0 1 0 1 1 3 2 5crossover point

name = "Tiger" spaceRequired = 5

:Animal spaceAvailable = 10

:Cage

animals

spaceRequired = 20 quantity = 4

:Animal spaceAvailable = 5

:Cage

animals

Individual 1

Individual 2

0 0 0 1 0 0 0 1 1 0

1 0 0 1 0 1 1 3 2 5crossover point

spaceAvailable = 5

:Cage spaceAvailable = 10

:Cage

spaceRequired = 20 quantity = 4

:Animal name = "Tiger" spaceRequired = 5

:Animal

animals animals

Fig. 5: Illustration of the crossover operator and its effect on the phenotype.

the individual. If crossover is too destructive, it can bedetrimental to the search as it can introduces too muchvariation into the population. Therefore, in order toreduce this, we allow crossover to occur only betweensegments. This allows entire objects to be copied overto the child being created – only the feature pairs thatrepresents associations will be affected. Figure 5 illus-trates the crossover operator. The animals referencesin the Cage objects are reassigned to other Animalobjects in the model (not shown). If there were noAnimal objects in the model, then those feature pairswould be redundant.

The mutation operator also differs slightly from theusual mutation operator used with linear genotypes.Figure 6 demonstrates how mutating different typesof genes causes different amounts of variation in thephenotype. Mutating the class node has the mostsignificant effect on the phenotype, whereas mutatingeither the feature selector node or feature value nodehas a lesser effect. Therefore, in our representation,each gene type has a different probability of beingmutated which is specified by the user. In the future,we plan to investigate whether using different muta-tion probabilities for the different gene types wouldimprove the effectiveness of the search algorithm.

3.3.3 Custom Genetic Operators

Initialising a population with variable-length individ-uals using variable-length segments aims to producea diverse, representative population. However, it doesnot guarantee that the individuals in the populationwill be able to represent the desired solution. There isa need for the ability to lengthen and shorten individ-uals, and so we include two new operators to do so:(i) Segment Creator inserts a randomly created segmentinto a random position in an individual; (ii) SegmentDestroyer deletes a randomly selected segment froman individual;

As with the standard genetic operators, we plan

1 0 2 1 1

0 0 2 1 1

1 1 2 1 1

1 0 1 1 1

name = "Hippo" spaceRequired = 10

:Animal

spaceAvailable = 15

:Cage

spaceRequired = 10

:Animal

name = "Pig" spaceRequired = 10

:Animal

(1)

(2)

(3)

Class node mutation

Feature selector node mutation

Feature selector node mutation

animals

Fig. 6: Illustration of the mutation operator and itseffect on the phenotype.

to thoroughly examine the effects of each of theoperators on a number of case studies.

3.4 Redundancy

During the mapping process, we introduced the no-tion of ignoring feature pairs (when references couldnot be satisfied) and discussed how we would beforced to ignore parts of the model if multiple rootobjects were allowed. Any part of the genotype thatdoes not appear in the phenotype is said to be non-coding or redundant. Too much redundancy createsan inefficient representation, which cannot encode asmuch information as its potential [29], so we aim tominimise this. There is the belief, however, that re-dundancy is a good thing, as mutations to the integerstring will activate new areas of the solution space[29], [25]. For example, changing the value of a classnode may activate a previously redundant feature pairin the production of that model object. This sectionlooks at the two structural areas where redundancymanifests – entire segments and individual featurepairs.

8

3.4.1 Non-Coding SegmentsDuring the reference construction phase of the trans-formation, we may be left with a number of objectsthat are not contained by the root object. To partiallyaddress this issue, we force the root object to containevery object it possibly can before we assign the un-handled references. The reference construction phaseis allowed to then break an object’s direct containmentwith the root. Once the reference handling phase hascompleted, there may remain a number of objects thatare not directly or indirectly contained by the rootobject. These are called islands and are automaticallyremoved from the model. The segments that producedthe objects in the islands are therefore non-codingsegments, and introduce redundancy into the repre-sentation.

3.4.2 Non-Coding Feature PairsWhere a feature pair relates to a reference, it canbecome non-coding if no referenceable objects havebeen instantiated, or if the reference has reached itsupper bound. With respect to attributes, a segmentmay contain multiple feature pairs that select thesame attribute. To deal with this, we could do one ofthree things. Firstly, we could ignore any subsequentattempts to assign an attribute value. Secondly, wecould allow reassignment, keeping the last-assignedvalue. Or finally, we could repair the feature pair toselect a different feature. The latter solution mightintroduce more clashes with other feature pairs inthe segment. The choice between the first and sec-ond solution should not cause any problems with ametaheuristic search technique, providing the choiceis consistent. Any feature pairs that are ignored aresaid to be non-coding feature pairs. Our current im-plementation (section 4) uses the second solution.

SummaryThis section presented a generic approach to encodingmodels for use with metaheuristic search techniques.We devised a linear genotype to encode models,meaning that it can be used with many well-knownmetaheuristic search algorithms, and we extended thestandard set of linear-genotype-related genetic oper-ators with model-specific operators. We now presentour implementation of the representation for a widelyused modelling platform.

4 IMPLEMENTATION FOR ECORE

We have implemented the representation and fini-tisation metamodel for Ecore – the metamodellinglanguage of the Eclipse Modeling Framework [33]. Ecoreautomatically assigns each meta-element an identifier,which could be used in the mapping process. How-ever, Ecore even assigns identifiers to non-instantiableclasses, such as abstract classes, enumerations and

Population

Individualfitness : Doublemutated : Booleanelite : Boolean

SegmentclassBit : Integer

FeaturePairfeatureSelectorBit : IntegerfeatureValueBit : Integer

individuals*

segments*

0..2parents

featurePairs*

Search

generations*

Fig. 7: The metamodel for the search population.

data types. To use Ecore’s identification scheme wouldintroduce a lot of redundancy into the representa-tion as all segments whose class bits reference non-instantiable classes would have to be ignored. Toovercome this, we take the set of instantiable classes,order them by their Ecore identifier, and assign thema new identifier for use with the mapping based ontheir position in the ordered list.

Our implementation includes the reverse transfor-mation – a phenotype-to-genotype mapper – to allowfor population seeding as previously discussed. Thesearch framework and mapping transformations arewritten in the Epsilon Object Language (EOL) [19] –a general purpose model management language. Abespoke implementation was chosen to provide aflexible basis for exploration with the representation,and also because it allows us to treat all key artefactsare models. Figure 7 shows the metamodel for thesearch population, and illustrates how we have cap-tured the genotype defined in the previous sectionas a metamodel. Our genotype-phenotype mappingtransformations are, therefore, written as model-to-model transformations.

To utilise the representation and search framework,one is simply required to implement a fitness func-tion to evaluate candidate solutions, and specify themetamodel and associated finitisation model for usein the search. Fitness functions are written in eitherEOL or Java. This allows the user to utilise the mostappropriate means to evaluate candidate solutions.For example, users might wish pass the model intoan external program for analysis.

We now illustrate the use of the representation in atypical application – performing coverage analysis of

9

MMOs.

4.1 White-Box Testing IllustrationTo demonstrate the utility of the representation andour specific implementation for Ecore, we presentproof-of-concept results for discovering models thatprovide statement coverage analysis of three MMOs4,all written in EOL. It is not our aim to provide athorough examination of a specific MDE problem.Our goal here, is to show that our representation canexpress the complex inputs needed to test MMOs, andto demonstrate the practicality of our approach on arepresentative MDE problem.

The first MMO, flowchart, animates a model of aflowchart diagram. The second MMO, triangle, is anMDE version of the triangle classification problem,which uses a very simple metamodel for triangles.The third MMO, EuGENia [20], is a complicated,real world model transformation. EuGENia takes ametamodel defined in Ecore and automatically createsthe models required to generate a graphical modeleditor in GMF, which are usually created by hand.For each of the three metamodels, we have definedappropriate finitisation models. EuGENia, in partic-ular, is a challenging coverage problem as it requiresthat models contain a number of very specific objects5.

To illustrate that our representation enables the useof multiple metaheuristic search algorithms, we com-pare three: a GA, hill climbing and random search.The GA has population of 10, the hill climbing selectsthe best neighbour from a random sample of 4 mu-tations, and random search is implemented as a GAwith the mutation probability set to 1.0 (two elitesare kept in each generation, as with the GA, to notpunish random search). The GA and random searchare executed for 10 generations, and hill climbing for25, so that all three algorithm make an equivalentnumber of evaluations6. As this experiment is justfor proof-of-concept to illustrate the feasibility andpracticality of the representation, no tuning was madeto encourage the search to achieve full coverage.Furthermore, our population size is conservative toensure that we do not brute force the solutions.

The fitness function is defined as the number ofpreviously uncovered statements minus the numberof newly covered statements, and we attempt to min-imise this value, taking the fittest individual at thetermination of the algorithm. We repeat this processup to 4 more times, attempting to discover modelsthat provide coverage of previously uncovered areas.Variable-length individuals are used, allowing models

4. The MMOs, their associated metamodels and finitisation mod-els, are available at: www.jamesrobertwilliams.co.uk/ycs2012

5. Specifics at www.eclipse.org/epsilon/doc/articles/eugenia-gmf-tutorial.

6. The full set of parameters for each algorithm, and detailsof the search framework itself, are available at: http://www.jamesrobertwilliams.co.uk/ycs2012.

to contain anywhere between 3 and 100 objects todemonstrate that our representation can handle scale.Table 1 shows the mean results over ten repetitionsof the algorithm. The number of metamodel classesgiven in the table is presented as “X (Y)”, where X isthe total number of classes, and Y is the number ofconcrete classes.

We do not attempt a detailed analysis of the results,but note only that it was straightforward to apply asimple generic search framework, designed for lineargenotypes, to a complicated MDE problem (discover-ing models) through the use of our proposed repre-sentation and its associated mapping from genotypeto phenotype. Without this representation, a customsearch framework operating directly on the modelsthemselves might have been necessary, as would havebeen the use of domain-specific genetic operators.

The next section discusses future extensions to therepresentation and highlights interesting research op-portunities for combining SBSE and MDE.

5 RESEARCH OPPORTUNITIES

This section further motivates the case for utilisingSBSE methods in MDE. Firstly, section 5.1 discusssome future work related to the representation pre-sented in the previous section. Sections 5.2, 5.3 and5.4 examine different areas in MDE that could benefitfrom SBSE techniques, and present existing work thatattempts to do so.

5.1 Exploring the Representation FurtherThe representation we presented in the previous sec-tion has a number of areas that demand furtherinvestigation. A thorough investigation of the effectsof the search operators is required. Analysing eachoperators’ effect, in relation to their probability ofapplication, over a number of distinct MDE problemswould provide guidance to users, helping them un-derstand how to best tailor the representation to theirproblem. We also plan on exploring the relationshipbetween the size of a metamodel and the length ofindividuals used in the search. The aim for manyapplications would be to use the smallest individualpossible to achieve to the search goal, as this willreduce the size of the search space.

Finally, there are some issues with our represen-tation that need to be addressed. For instance, cur-rently we do not enforce lower bounds on referencesor enforce the assignment of required fields. Thiscould be achieved by injecting segments or featurepairs into individuals, however this may negativelyimpact the effectiveness of the search method beingused. Another concern is that, as the generation ofthe root object is external to the genotype, we areunable to assign any values to its attributes. This is,however, a task that can currently be performed bythe appropriate fitness functions, or manually by the

10

Problem Metamodel MMO GA HC Random#Classes #Features L.O.C Coverage Coverage Coverage

Flowchart 6 (5) 8 19 66.5% 89.7% 76.6%Triangle 2 (2) 4 21 96% 96% 95%EuGENia 20 (15) 81 631 10.8% 12.1% 9.6%

TABLE 1: Naıve coverage analysis of three MMOs to illustrate the practicality of our representation.

user. Furthermore, we do not ensure that the encodedmodels are valid with respect to constraints definedexternally to the metamodel. Metamodels can onlydefine structural constraints and use other languages(such as the Object Contraint Language [12]) to defineextra constraints, such as class invariants. Users arecurrently required to perform this validation duringthe fitness function and assign fitnesses accordingly.

5.2 Discovering and Validating Models

As mentioned in section 2.3, SBSE methods couldhelp to address some of the problems associatedwith developing and validating models. France andRumpe [9] propose that the utilisation of models atruntime is an important research area, and search-based optimisation techniques could provide a wayto optimise models at runtime, as has been attemptedin the field of online evolution of robot controllers [3].

Previously, we have used grammatical evolution tosearch over the model space to discover interestingtextual models [35]. (The representation used in [35]suffered many flaws which we have addressed in therepresentation presented in this paper, including theability to specify data using a finitisation model, andto cross-reference other objects in the model.) Goldsbyand Cheng [11] also use search to discover models ofinterest. They use a GA to generate resilient systembehaviour models (in the form of state diagrams)based on a user-defined set of possible behaviours. Anindividual in their search space is a sequence of in-structions mapping to possible pre-defined transitionsand guards. To evaluate individuals, they translatethe behaviour models into Promela [16] and performmodel checking with respect to a set of temporalproperties.

Harman [14] cites sensitivity analysis as being animportant topic for future work in the SBSE domain.We have successfully used our representation of MDEmodels to apply sensitivity analysis to the results ofa complex, multi-objective decision making tool [36],which itself uses search to discover solutions and hasbeen applied to the Next Release Problem [4].

Recent work by Cadavid et al. [5] demonstratesa user-driven approach to validating the boundaries(relationships and multiplicity constraints) of a meta-model to ensure that a metamodel is neither under-constrained (allows invalid models) over-constrained(disallows valid models) with respect to the domain.They search over a model space specified by a setof user-defined model fragments – ranges of values

for certain properties defined in the metamodel – toobtain a set of models which are dissimilar and coverthe set of model fragments. Domain experts can thenexamine the set of models to detect any faults in themetamodel.

5.3 Discovering and Validating MMOs

The proof-of-concept experiments illustrated in Table1 show that we did not manage to gain much coverageof the EuGENia transformation. This is not a flaw ofthe representation – it is capable of expressing therequired model information – instead it is due to thevastness of the search space and the algorithm usedto explore it. An interesting side effect of performingcoverage analysis on EuGENia was that it highlightedthe need for robustness testing of MMOs – an area inwhich Harman claims SBSE can help [14]. EuGENiaexpects that its input models satisfy certain, unspec-ified, preconditions and any failure to meet thesepreconditions causes EuGENia to fail gracelessly. Ap-plying robustness testing to EuGENia, or any MMO,would allow transformation developers to discoverthe implicit preconditions of their transformation, andguard against them by creating explicit preconditions,or better failure responses.

There is much work in the field of search-basedsoftware testing [23], [24] that could be reused inMDE. There have, however, been few attempts todate that use this wealth of knowledge. Fleurey etal. [8] use an evolutionary algorithm to optimise anexisting set of test models, with respect to metamodelcoverage.

With respect to the discovery of MMOs, Kessentiniet al. [18] present a search based approach that usesexample source and target models to learn modeltransformations. Their approach applies metaheuris-tic search to “transformation fragments” – mappingsbetween elements in example source and target mod-els – in an attempt to match the best fragment toeach construct in a given source model. The set ofmatched fragments then creates the target model. Thisapproach relies heavily on the quality of the trans-formation fragments. If the fragments do not give agood representation of source models, the approachwill not be able to find good matches, and thus willnot be able to create the target model. Furthermore,the transformation fragments are verbose and have tobe written manually.

Genetic Programming may prove very useful forevolving model transformations (and other MMOs).

11

For example, the transformation language constructscould make up GP’s function set, and the associ-ated metamodel(s) would provide the terminal set(as transformations are generally written at the meta-model level, explicit information about model datais not required). Alternatively, as transformations arecommonly a sequence of rule applications, a tech-nique such as N-Gram GP [26] could be applied.

5.4 Other Application Areas and ChallengesNon-functional properties of models and MMOs, suchas performance, human understandability, and thelevel of coupling and cohesion, should also be inves-tigated. Existing work in the SBSE field has lookedat analysing coupling and cohesion of class diagrams[28], however these works do not address domain-specific models, in which traditional software metricsmay not directly apply. Li et al. [21] show howevolutionary algorithms can be used to suggest alter-native software architectures based on non-functionalrequirements.

An open challenge in the SBSE world, which appliesto MDE, is the issue of landscape visualisation [14].The complex nature of models makes the landscapeincredibly difficult to visualise. Furthermore, to allowthe union of MDE and SBSE to progress, we willrequire a number of benchmark problems for which tocompare solutions against. Research into evolutionaryalgorithms has been around for decades and providesmany benchmarks for SBSE methods to tackle. Notall of these benchmarks will map naturally to MDE,and so we need to develop a useful repository ofbenchmark problems for this field.

6 CONCLUSION

MDE provides an effective way to develop large-scale, complex systems thanks to its advocation forabstraction and decomposition. The complexity ofMDE models, however, introduces many challengesfor MDE practitioners. The solution to a large numberof MDE problems can be represented as models, andso in this paper we have proposed a generic linearrepresentation of models which is amenable to SBSEtechniques, and described an implementation of therepresentation for Ecore. Furthermore, we have high-lighted other research opportunities for utilising SBSEmethods to aid the MDE domain.

Acknowledgements. This research was supportedby the EPSRC, through the Large-Scale Complex ITSystems project, EP/F001096/1.

REFERENCES[1] B. Baudry, T. Dinh-Trong, J-M. Mottu, D. Simmonds, R. France,

S. Ghosh, F. Fleurey, and Y. La Traon. Model transformationtesting challenges. In Proc. IMDT workshop at ECMDA’06, 2006.

[2] B. Baudry, S. Ghosh, et al. Barriers to systematic modeltransformation testing. Communications of the ACM, 53(6):139–143, 2009.

[3] N. Bredeche, E. Haasdijk, and A. Eiben. On-line, on-boardevolution of robot controllers. In Artifical Evolution, volume5975 of LNCS, pages 110–121. Springer Berlin / Heidelberg,2010.

[4] Frank Burton, Richard Paige, Louis Rose, Dimitrios Kolovos,Simon Poulding, and Simon Smith. Solving acquisition prob-lems using model-driven engineering. In Antonio Vallecillo,Juha-Pekka Tolvanen, Ekkart Kindler, Harald Storrle, andDimitris Kolovos, editors, Modelling Foundations and Applica-tions, volume 7349 of Lecture Notes in Computer Science, pages428–443. Springer Berlin / Heidelberg, 2012.

[5] J. Cadavid, B. Baudry, and H. Sahraoui. Searching the bound-aries of a modeling space to test metamodels. In Proc. ICST’12,pages 131–140, 2012.

[6] J. Clark, J. J. Dolado, et al. Reformulating software engineeringas a search problem. In IEEE Software, volume 150, pages 161–175, 2003.

[7] K. Czarnecki and S. Helsen. Feature-based survey of modeltransformation approaches. IBM Systems Journal, 45(3):621–645, 2006.

[8] F. Fleurey, J. Steel, and B. Baudry. Validation in model-driven engineering: testing model transformations. In Proc.MoDeVa’04, pages 29–40, 2004.

[9] R. France and B. Rumpe. Model-driven development ofcomplex software: A research roadmap. In Proc. FOSE ’07,pages 37–54, 2007.

[10] David E. Goldberg. The Design of Innovation: Lessons from andfor competant genetic algorithms. Kluwer Academic Publishers,2002.

[11] H. J. Goldsby and B. H.C. Cheng. Avida-MDE: a digitalevolution approach to generating models of adaptive softwarebehavior. Proc. GECCO’08, pages 1751–1758, 2008.

[12] Object Management Group. Uml 2.0 ocl specification. http://www.omg.org/docs/ptc/03-10-14.pdf.

[13] M. Harman and B. F. Jones. Search based software engineering.Information and Software Technology, 43(14):833–839, December2001.

[14] Mark Harman. The current state and future of search basedsoftware engineering. In Proc. FOSE ’07, pages 342–357, 2007.

[15] M. Herrmannsdoerfer, S. Benz, and E. Juergens. COPE -automating coupled evolution of metamodels and models. InProc. 23rd ECOOP, pages 52–76. Springer-Verlag, 2009.

[16] Gerard J. Holzmann. The SPIN Model Checker. Number ISBN0-321-22862-6. Addison-Wesley, September 2003.

[17] J. Hutchinson, J. Whittle, M. Rouncefield, and S. Kristoffersen.Empirical assessment of MDE in industry. In Proc. ICSE’11,pages 471–480, 2011.

[18] M. Kessentini, H. Sahraoui, M. Boukadoum, and O. Omar.Search-based model transformation by example. Software andSystems Modeling, pages 1–18, 2010.

[19] D. S. Kolovos, R. F. Paige, and F. A. C. Polack. The EpsilonObject Language (EOL). LNCS, 4066:128–142, 2006.

[20] D. S. Kolovos, L. M. Rose, S. Abid, R. F. Paige, F. A. C. Polack,and G. Botterweck. Taming EMF and GMF using modeltransformation. LNCS, 6394:211–225, 2010.

[21] R. Li, M. R. V. Chaudron, and R. C. Ladan. Towards automatedsoftware architectures design using model transformationsand evolutionary algorithms. In Proc. GECCO’10, pages 1333–1340, 2010.

[22] Sean Luke. Essentials of Metaheuristics. Lulu, 2009. Availablefor free at http://cs.gmu.edu/∼sean/book/metaheuristics/.

[23] P. McMinn. Search-based software test data generation: asurvey. Software Testing, Verification and Reliability, 14(2):105–156, June 2004.

[24] P. McMinn. Search-based software testing: Past, present andfuture. In ICSTW’11, pages 153–163, March 2011.

[25] J.F. Miller and S.L. Smith. Redundancy and computationalefficiency in cartesian genetic programming. IEEE Transactionson Evolutionary Computation, 10(2):167–174, April 2006.

[26] R. Poli and N. F. McPhee. A linear estimation-of-distributiongp system. In Proc. 11th EuroGP, pages 206–217, Berlin,Heidelberg, 2008. Springer-Verlag.

[27] Riccardo Poli, William B. Langdon, and Nicholas FreitagMcPhee. A field guide to genetic programming. Publishedvia http://lulu.com and freely available at http://www.gp-field-guide.org.uk, 2008.

12

[28] Outi Raiha. A survey on search-based software design. Com-puter Science Review, 4(4):203–249, 2010.

[29] F. Rothlauf. Represenatations for Evolutionary and Genetic Algo-rithms. Springer Berlin Heidelberg New York, second edition,2006.

[30] A. Saltelli, K. Chan, and E. M. Scott, editors. SensitivityAnalysis. Probability and Statistics. Wiley, 2000.

[31] B. Selic. The pragmatics of model-driven development. IEEESoftware, 20(5):19–25, 2003.

[32] S. Sendall and W. Kozaczynski. Model transformation: Theheart and soul of model-driven software development. IEEESoftware, pages 42–45, 2003.

[33] D. Steinberg, F. Budinsky, M. Paternostro, and E. Merks. EMFEclipse Modeling Framework. The Eclipse Series. Addison-Wesley, second edition, 2009.

[34] R. Van Der Straeten, T. Mens, and S. Van Baelen. Challengesin model-driven software engineering. In Models in SoftwareEngineering, volume 5421 of LNCS, pages 35–47. SpringerBerlin / Heidelberg, 2009.

[35] J. R. Williams, S. Poulding, L. M. Rose, R. F. Paige, andF. A. C. Polack. Identifying desirable game character be-haviours through the application of evolutionary algorithms tomodel-driven engineering metamodels. LNCS, 6956:112–126,2011.

[36] James R. Williams, Frank R. Burton, Richard F. Paige, andFiona A. C. Polack. Sensitivity analysis in Model-DrivenEngineering. In Model Driven Engineering Languages and Sys-tems, volume 7590 of LNCS, pages 743–758. Springer Berlin /Heidelberg, 2012.