dams: distributed adaptive metaheuristic selection
TRANSCRIPT
arX
iv:1
207.
4448
v1 [
cs.N
E]
18 J
ul 2
012
DAMS: Distributed Adaptive Metaheuristic Selection
Bilel DerbelUniversité Lille 1
LIFL – CNRS – INRIA [email protected]
Sébastien VerelUniv. Nice Sophia-Antipolis
INRIA [email protected]
ABSTRACTWe present a distributed algorithm, Select Best and Mutate (SBM),in the Distributed Adaptive Metaheuristic Selection (DAMS) frame-work. DAMS is dedicated to adaptive optimization in distributedenvironments. Given a set of metaheuristics, the goal of DAMS isto coordinate their local execution on distributed nodes inorder tooptimize the global performance of the distributed system.DAMSis based on three-layer architecture allowing nodes to decide dis-tributively what local information to communicate, and what meta-heuristic to apply while the optimization process is in progress.SBM is a simple, yet efficient, adaptive distributed algorithm us-ing an exploitation component allowing nodes to select the meta-heuristic with the best locally observed performance, and an explo-ration component allowing nodes to detect the metaheuristic withthe actual best performance. SBM features are analyzed frombotha parallel and an adaptive point of view, and its efficiency isdemon-strated through experimentations and comparisons with other adap-tive strategies (sequential and distributed).
Categories and Subject DescriptorsI.2.8 [Artificial Intelligence ]: Problem Solving, Control Methods,and Search—Heuristic methods
General TermsAlgorithms
Keywordsmetaheurististics, distributed algorithms, adaptative algorithms, pa-rameter control
1. INTRODUCTIONMotivation: Evolutionary algorithms or metaheuristics are effi-
cient stochastic methods for solving a wide range of optimizationproblems. Their performances are often subject to a correctset-ting of their parameters including the representation of solution, thestochastic operators such as mutation, or crossover, the selection
Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.GECCO’11,July 12–16, 2011, Dublin, Ireland.Copyright 2011 ACM 978-1-4503-0557-0/11/07 ...$10.00.
operators, the rate of application of those operators, the stoppingcriterium, or the population size, etc. From the metaheuristics-userpoint of view, it could be a real challenge to choose the parameters,and understand this choice. Moreover, when the algorithms evolvein a distributed environment, comes additional possible designs, orparameters such as the communication between distributed nodes,the migration policy, the distribution of the metaheuristics execu-tion, or when different metaheuristics are used together, their dis-tribution over the environement, etc. The aim of this paper is theparameter setting in a fully distributed environment.
Background and related work: Following the taxonomy ofEiben et al. [4], we distinguish two types of parameter setting: thefirst one isoff-line, before the actual run, often calledparametertuning, and the second one ison-line, during the run, calledparam-eter control. Usually, parameter tuning is done by testing all or asubset of possible parameters, and select the combination of param-eters which gives the best performances. Obviously, this method istime consuming. Besides, a parameters setting at the beginning ofthe optimization process could be inappropriate at the end.Sev-eral approaches are used for the parameters control. Deterministones choose ana-priori policy of modification. But, the choice ofthis policy gets more complicated compared to the static setting asthe complexity of design increases. Self-adaptive techniques en-code the parameters into the solution and evolve them together.This approach is successfully applied in continuous optimization[6]. Nevertheless, in combinatorial optimization it oftencreatesa larger search space leading to efficiency loss. Finally, adaptivemethods use the information from the history of the search tomod-ify the parameters. The hyperheuristics are one example [2]. Theyare heuristics that adaptively control (select, combine, generate, oradapt) other heuristics. Another example is the Adaptive OperatorSelection (AOS) which controls the application of the variation op-erator, using probability matching, adaptive pursuit [12], or multi-armed bandit [3] techniques. In this work, we focus on parameterscontrol methods in a distributed setting.
In fact, the increasing number of CPU cores, parallel machineslike GPGPU, grids, etc. makes more and more the distributed en-vironments the natural framework to design effective optimizationalgorithms. Parallel Evolutionary Algorithms (EA) are well suitedto distributed environments, and have a long history with many suc-cess [13]. In parallel EA, the population is structured: theindividu-als interact only with their neighbor individuals. Two mainparallelEA models can be identified. In the island (or multi-population)one, the whole population is divided into several ones. In the cel-lular model, the population is embedded on a regular toroidal grid.Of course, between this coarse grained to fined grained, manyvari-ant exist. This field of research received more and more attention.Recent theoretical and experimental works study the influence of
parallelism [9, 10], or study the fined grained population structureon performances [8].
Previous works control the specific parameters of parallel EA, orothers parameters. In [1], the architecture of workers witha globalcontroller is used to self-adapt the population size in eachisland.Population size, and number of crossover points is controlled us-ing the average value in the whole population in [11]. Those twoexamples of works use a parallel environment where a global in-formation can be shared. In the work of Tongchim et al. [14], adistributed environment is considered for the parameters control.Each island embeds two parameters settings. Each parameters isevaluated on half the population, and the best parameter setting iscommunicated to the other islands which is used to produce newparameters. So, parameters are controlled in a self-adaptive wayusing local comparison between two settings only. Laredo etal. [7]propose the gossiping protocol newcast for P2P distributedpopula-tion EA. The communication between individuals evolves towardssmall-world networks. Thus, the selection pressure behaves like inpanmictic population, but population diversity and systemscalabil-ity outperform the fully connected population structure.
In summary, four main issues has been studied in parameter con-trol in parallel EA: static strategies where the parameterscan be dif-ferent for each island but does not change during the search;strate-gies that use a global controller; self-adaptive strategies where eachnode compares possible parameter settings; and distributed strate-gies that evolve the communication between nodes. In this work,we propose a new and intuitive way to control the parameters indistributed environment: during the search, from a set of meta-heuristics which correspond to possible parameter settings, eachnode select one metaheuristic to execute according to localinfor-mation (e.g.a performance measure) given by the other nodes.
Contributions and outline: In this paper, we define a new adap-tive parameters control method called distributed adaptive meta-heuristic selection (DAMS) dedicated to distributed environments.DAMS is designed in the manner of heterogeneous island modelEAs where different metaheuristics live together. In this context,the distributed strategy (Select Best and Mutate strategy)which se-lects either the best metaheuristic identified locally, or one randommetaheuristic is defined and studied. For more simplicity, but alsoto enlighten the main features of this strategy, the SBM is givenin the framework of DAMS. Generally speaking, from an exter-nal observer, a distributed environment can be viewed as a uniqueglobal system which can be optimized in order to perform effi-ciently. However, the existence of such a global observer isnotpossible in practice nor mandatory. From DAMS point of view,theoptimization process in such a distributed environment is though ina very local manner using only local information to coordinate theglobal distributed search and guide the optimization process. Theissue of metaheuristic selection in distributed environments is intro-duced in Section 2. From that issue, the DAMS framework is givenin Section 3 bringing out three levels of design. The select Bestand Mutate strategy (SBM) is then addressed in Section 4. Dif-ferent SBM properties are studied in Section 5 through extensiveexperimentations using the oneMax problem. Finally, in Section 6we conclude with some open issues raised by DAMS.
2. SELECTION OF METAHEURISTIC IN DIS-TRIBUTED ENVIRONMENTS
2.1 PositionLet us assume that to solve a given optimization problem, we
can use a setM of atomic functionthat we can apply in anit-
erativeway on a solution or a population of solutions. By atomicfunction we mean a black-box having well defined input and outputspecifications corresponding to the problem being solved. In otherwords, we do not have any control on an atomic function once itsis executed on some input population of solutions. To give a con-crete example, an atomic function could be a simple (determinis-tic or stochastic) move according to a well defined neighborhood.It could also consist in applying the execution of a metaheuristicfor fixed number of iterations. Hence, an atomic function canbeviewed from a very general perspective as a undividable, fixed andfully defined metaheuristic. So, in the following we call an atomicmetaheuristic, or shorter a metaheuristic, the atomic functions.
In this context, where the atomic metaheuristics can be usedin asequential way, designing a strategy to solve an optimization prob-lem turns out to design an iterative algorithm that carefully com-bines the atomic functions (metaheuristics) in a specific order. Inthis paper, we additionally assume that we are given a set of com-putational resources that can be used to run the atomic metaheuris-tics. These computational resources could be for instance aset ofphysical machines distributed over a network and exchanging mes-sages, or some parallel processors having some shared memory tocommunicate.
Having such a distributed environment and the set of atomicmetaheuristic at hand, the question we are trying to study inthispaper is as following:how can we design an efficient distributedstrategy to solve the optimization problem?Obviously, the per-formance of the designed strategy depends on how thedistributedcombination of the atomic operations is done. In particular, onehave toadaptthe search at runtime and decidedistributivelywhichatomic function should be applied at which time by which com-putational entity. The concept of DAMS introduced in this paperaims at defining in a simple way a general algorithmic frameworkallowing us to tackle the latter question.
2.2 Example of an optimal strategyTo give an algorithm following the DAMS framework, let us con-
sider the following simple example. Assume we have a networkof4 nodes denoted by:{n1, . . . , n4}. Any two nodes, but nodesn1
andn4 (see Fig 1), can communicate together by sending and re-ceiving messages throughout the network. Assume that the atomicoperations we are given are actually composed of four population-solution based metaheuristics denoted by:{n1, . . . , n4}, i.e., eachmetaheuristic accepts as input a population of solutions and outputsa new one. Then, starting with a randomly generated population,our goal is to design a distributed strategy that decides howto dis-tribute the execution of our metaheuristics on the network and whatkind of information should be exchanged by computational nodesto obtain the best possible solution. To make it simple, assumethat the best strategy, which can be considered as a distributed or-acle strategy, is actually given by distributed algorithm 1(See alsoFig. 1 for an illustration). In other words, any other distributedstrategy cannot outperforms Algorithm1.
Having this example in mind, the question we are trying to an-swer is then: How can we design a distributed strategy which iscompetitive compared to the oracle of algorithm 1? Finding such astrategy is obviously a difficult task. In fact, finding the best map-ping of our metaheuristics into the distributed environment is initself an optimization problem which could be even harder than theinitial optimization problem we are trying to solve. In thiscon-text, one possible solution is to map the metaheuristics into thedistributed environment in an adaptive way in order to guidethedistributed search and operate as close as possible of the oraclestrategy (Algorithm 1 in our example). This is exactly the ultimate
Algorithm 1 : Oracle strategy of distributed adaptive meta-heuristic selectionGenerate a random solution on each nodeni withi ∈ {1, . . . , 4};RunM3 in parallel on eachni with i ∈ {1, . . . , 4};Exchange best found solution;RunM1 in parallel on eachni with i ∈ {1, 2};RunM2 in parallel on eachni with i ∈ {3, 4};Exchange best found solution;RunM4 in parallel on eachni with i ∈ {1, . . . , 4};Return best found solution;
M3
M3 M3
M3
M2 M4M1
M4 M4
M4
M1 M2
Figure 1: Illustration of the distributed oracle strategy of algo-rithm 1. At the beginning all nodes{n1, . . . , n4} run M3, thenM1 and M2 are used, and finallyM4 concludes the distributedsearch.
goal of the adaptive metaheuristic selection in distributed environ-ments. In the following section, we introduce a general algorithm,DAMS, and discuss its different components.
3. DAMS: A GENERIC FRAMEWORKGenerally speaking, a Distributed Adaptive MetaheuristicSe-
lection (DAMS) is an adaptive strategy that allows computationalnodes to coordinate their actions distributively by exchanging lo-cal information. Based only on a local information, computationalnodes should be able to make efficient local decisions that allowsthem to guide the global search efficiently (i.e., to be as efficient aspossible compared to a distributed oracle strategy).
Before going into further details and for the sake of simplicity,we shall abstract away the nature of the distributed environmentunder which a DAMS will be effectively implemented. For thatpurpose, we use a simple unweighted graphG = (V,E) to modelthe distributed environment. A nodev ∈ V models a computa-tional node (e.g., a machine, a processor, a process, etc). An edge(u, v) ∈ E models a bidirectional direct link allowing neighboringnodesu andv to communicate together (e.g., by sending a messagethrough a physical network, by writing in a shared local/distantmemory, etc).
A high level overview of DAMS is given in distributed algo-rithm 2. Notice that the high level code given in algorithm 2 isexecuted (in parallel) by each nodev ∈ V . The input of each nodev is a setM of atomic metaheuristics.
Each nodev participating into the computations has threelocalvariables: A populationP which encodes a set of individuals. AsetS which encodes the local state of nodev. The local stateS ofnodev is updated at each computation step and allows nodev tomake local decisions. Finally, each node has a current metaheuris-tic denoted byM .
As depicted in algorithm 2, a DAMS operates in many roundsuntil some stopping condition is satisfied. Each round is organizedin three levels.
The Distributed Level: This level allows nodes to communicate
Algorithm 2 : DAMS: high level code for nodev ∈ V
Input : A set of metaheuristicsMM ← INIT_META();P ← INIT_POP();S ← INIT_STATE();I ← {M,P,S}; /∗v’s initial local
information∗/
repeat/∗∗
Distributed Level∗∗/
c← LOCAL_COMMUNICATION(I);P ← UPDATE_POPULATION(I,c);S ← UPDATE_LOCAL_STATE(I,c);/∗∗
Metaheuristic Selection Level∗∗/
M ← SELECT_META(I);/∗∗
Atomic Low Level∗∗/
(P,S)← APPLY_META(M,P );until STOPPING_CONDITION(I) ;
together according to some distributed scheme and to updatetheirlocal information. This step is guided by the current local infor-mationI = {M,P,S} which encodes the experience of nodevduring the distributed search. Typically, a node can sends and/orreceives a message to one or many of its neighbors to share someinformation about the ongoing optimization process. According toits own local information (variableI) and the local information ex-changed with neighbors (variablec), nodev can both update itslocal stateS and its local populationP . Updating local stateS isintended to allow nodev to record the experience of its neighborsfor future rounds and make local decisions accordingly. Updatinglocal populationP aims typically at allowing neighboring nodes toshare some representative solutions found so far during thesearch,e.g. best individuals. This is the standard migration stageof parallelEA. Finally, we remark that since at each round local informationI of nodev is updated with the experience of its neighbors, thenafter some rounds of execution, we may end with a local informa-tion I that reflects the search experience ofnot only v’s neighborsbut of other nodes being farther away in the network graphG, e.g.,the best found solution or the best metaheuristic could be spreadthrough the network with a broadcast communication process.
The Metaheuristic Selection Level: This level allows a node tolocally select a specific metaheuristicM to apply from setM ofavailable metaheuristics. First, we remark that this levelmay notbe independent of the distributed level. In fact, the decision of se-lecting a metaheuristic is guided by local informationI which isupdated at the distributed level. Although the metaheuristic choiceis executed locally and independently by each nodev, it may notbe independent of other nodes choices. In fact, the local commu-nication step shall allow nodes to coordinate their decisions anddefine distributively a cooperative strategy. Second, since at thedistributed level, local informationI of a a given nodev could storeinformation about the experience of other nodes in the network, themetaheuristic choice can clearly be adapted accordingly. There-fore, the main challenge when designing an efficient DAMS is tocoordinate the Metaheuristic Selection level and the DistributedLevel in order to define the best distributed optimization strategy.
The Atomic Low Level: At this level a node simply executesthe selected metaheuristicM using its populationP as input. No-tice that ’atomic’ refers to the fact that a node cannot control thesequential metaheuristic execution in any way. However, after ap-plying a given metaheuristic, the population and the local state ofeach node may be updated. For instance, one can decide to moveto
the new population following a given criterion or to simply recordsome information about the quality of the outputted population us-ing local stateS .
4. SELECT BEST AND MUTATE STRATEGYThe originality of DAMS framework is to introduce a metaheuris-
tic selection level which adaptively select a metaheuristic accordingto local information shared by the other neighboring nodes.In thissection, we give a metaheuristic selection strategy, the Select Bestand Mutate strategy (SBM), which specify the DAMS.
For the sake of clarity, we consider the classical message passingmodel: We consider ann-node simple graphG = (V,E) to modela distributed network. Two nodesu andv, such that(u, v) ∈ E,can communicate together by sending and receiving messagesto-ward edge(u, v). A high level description of SBM is given inalgorithm 3 and is discussed in next paragraphs. Note that thecode of algorihtm 3 is to be executed (in parallel) on every nodev ∈ V . The algorithm accepts as input a finite set of metaheuristicsM = ∪m
k=1{Mk} and a probability parameterpmut.
Algorithm 3 : SBM code for every nodev ∈ V
Inputs: A set of metaheuristicsM = ∪m
k=1{Mk};A metaheuristic mutation ratepmut ∈ [0, 1] ;
r ← 0;k ← INIT_META(M);P ← INIT_POP();repeat
/∗Distributed Level
∗/
SendMsg(r, k, P ) to each neighbor;P ← {P}; S ← {(r, k)};for each neighborw do
ReceiveMsg(r′, k′, P ′) from w;P ← P ∪ P ′;S ← S ∪ {(r′, k′)};
P ← UPDATE_POPULATION(P);/∗Metaheuristic Selection Level
∗/
kbest ← SELECT_BEST_META(S);if RND(0, 1) < pmut then
k ← RND(M\Mkbest)
else k ← kbest;/∗Atomic Low Level
∗/
Find a new populationPnew by applying metaheuristicMk with P as an initial population;r ← REWARD(P, Pnew);P ← Pnew;
until Stopping condition is satisfied;
Clearly, SBM follows the general scheme of a DAMS definedpreviously in Algorithm 2. One can distinguish the three basic lev-els of a DAMS and remark their inter-dependency.
SBM Distributed level: The local information defined by a DAMSis encoded implicitly in SBM using variablesr, k andP . For everynodev, variablek refers to metaheuristicMk considered currentlyby v. Variabler (reward) refers to the quality of metaheuristicMk.VariableP denotes the current population of nodev. SBM oper-ates in many rounds until a stopping condition is satisfied. At eachround, the distributed level consist in sending a message contain-ing triple (r, k, P ) to neighbors and symmetrically receiving therespective triples(r′, k′, P ′) sent by neighbors. Then after, nodev constructs two setsP andS . SetP contains information about
neighbors populations. Nodev can then update its current popula-tion P according to setP . For instance, one may selects the bestreceived individuals, or even apply some adaptive strategytakinginto account the search history. As for setS , it gathers informationabout the quality of other metaheuristics considered by neighbors.This set is used at the next DAMS level to decide on the new cur-rent metaheuristic to be chosen by nodev.
SBM Metaheuristic Selection Level: choosing a new meta-heuristic is guided by two ingredients. Firstly, using setS , nodev selects a metaheuristic according to neighbors information. Onecan imagine many strategies for selecting a metaheuristic possiblydepending not only onS but also on received populationsP . InSBM, we simply select metaheuristickbest corresponding to thebest received rewardr. Notice that defining what is the best ob-served metaheuristic could be guided by different policies. Froman exploitation point of view, function SELECT_BEST_META hasthe effect of pushing nodes to execute the metaheuristic with thebest observed performance during the search process. Obviously, ametaheuristic with good performance at some round could quicklybecomes inefficient as the search progresses. To control this is-sue, we introduce aexplorationcomponent in the selection level.In SBM, this component is simply guided by a metaheuristic mu-tation operator. In fact, every node decides to select at randomanother metaheuristic different fromMkbest
with ratepmut. Intu-itively, these two ingredients in SBM selection level shallallow dis-tributed nodes to obtain a good tradeoff between exploitation andexploration of metaheuristics, and adapt their decisions accordingto the search.
SBM Atomic Low Level: Once a new metaheuristicMk is se-lected, it is used to compute a new population. To evaluate theperformance of metaheuristicMk, SBM simply compares the pre-vious populationP with the new populationPnew using a genericREWARD function. For instance, a simple evaluation strategy couldbe by comparing the best individual fitness, the average populationfitness, or even more sophisticated adaptive strategies taking intoaccount the performances observed in previous rounds.
5. EXPERIMENTAL STUDY OF SBMIn this section, we study a specific SBM-DAMS by fully speci-
fying its different levels. We report the results we have obtained byconducting experimental campaigns using the oneMax problem.
5.1 SBM settingTo conduct experimental study of the SBM-DAMS, we test the
algorithm on a classical problem in EA, theOneMaxproblem. Thisproblem was used in recent works of Fialho et al. [5] which proposea (sequential) Adaptive Operator Selection (AOS) method based onthe Multi-Armed Bandits, and a credit assignment method whichuses the fitness comparison. The oneMax problem, the "drosophila"of evolutionary computation, is a unimodal problem defined on bi-nary strings of sizel. The fitness is the number of "1" in the bit-string. Within the AOS framework, the authors in [5] considered(1+λ)−EA and four mutation operators to validate their approach:The standard1/l bit-flip operator (every bit is flipped with the bi-nomial distribution of parameter1/l wherel is the length of thebit-strings), and the1−bit, 3−bit, and5−bit mutation operators(the b−bit mutation flips exactlyb bits, uniformly selected in theparents). In the rest of the paper, we shall also consider thesamefour atomic metaheuristics to study DAMS. Each atomic functionis one iteration of(1 + λ)-EA using one of the four mutation op-erators. Unless stated explicitly, parameterλ is set to50 as in [5].
Notice also by studying SBM-DAMS with these well-understoodoperators does not undergo any major weakness of our approachsince these operators mainly exhibits different exploration degreesthat one can encounter in other settings when using other operators.
In all reported experiments, the length of the bit strings isset tol = 104 as in [5]. PopulationP of SBM-DAMS is reduced to asingle solutionx. The initial solution is set to(0, . . . , 0). The re-wardsr is the fitness gain between parent and offspring solutions:f(xnew) − f(x). The migration policy (UPDATE_POPULATION)is elitist, it replaces the current solution by one of the best receivedsolutions if its fitness is strictly higher. The algorithm stops whenthe maximal fitnessl is reached. This allows us to compare the per-formances of SBM-DAMS according to the number of evaluationsas used in sequential algorithms; but also in terms of rounds(totalnumber of migration exchange) as used in parallel frameworks, andin terms of messages cost that is the total number of local commu-nications made through the distributed environment.
In the remainder, theoneMaxexperimental protocol is used tofirst compare the efficiency of SBM-DAMS to some oracle andnaive strategies, then the parallel properties are analyzed with acomparison to the sequential AOS method.
5.2 Experimental setupThree network topologies are studied: complete, grid, and cy-
cle. In the complete one, the graph is a clique, i.e., every node islinked to all other nodes. In the grid topology, every node can com-municate with four other neighbors except at the edge of the grid(non-toroidal grid). In the circle topology, nodes are linked to twoothers nodes to form a circle.
Since there exist no previousdistributedapproaches addressingthe same issues than DAMS, we shall compare SBM-DAMS to thestate-of-the-art sequential adaptive approaches (Section 5.4) andalso to two simple distributed strategies (Section 5.3) called rnd-DAMS and seqOracle-DAMS. (i) In rnd-DAMS, each node selectsat random (independently of node states) at each round a meta-heuristic to be executed. It allows us to evaluate the efficiency ofSBM selection method based on the best instant rewards. (ii)InseqOracle-DAMS, each node executes the sequential oracle whichgives the operator with the maximum fitness gain according tothefitness of node current solution. The sequential oracle is takenfrom the paper [3]. It is used independently by each node with-out taking into account rewards, or observed performances of oth-ers nodes. Notice that that seqOracle-DAMS outperforms a staticstrategy which use only one metaheuristic, but it is different from apure distributed oracle which could lead to better performances forthe whole distributed system, by taking take into account all nodesto select the metaheuristic of each node.
For the latter two based-line strategies, we use the same elitismmigration policy as SBM-DAMS and evaluate their performancesusing the same three topologies. For each topology, networksize isn ∈ {4, 8, 16, 36, 64}, and the performance measures are reportedover20 independent runs. Notice that for this particular study, thenumber of evaluations isλ.n times the number of rounds. Thenumber of exchanged messages is|E| times the number of roundswhere|E| is the number of communication links in the consideredtopology.
5.3 Adaptation propertiesTo tune off-line, the metaheuristics mutation rate of SBM-DAMS,
we perform a design of experiments campain withpmut ∈ {0.0005,0.005, 0.001, 0.01, 0.05, 0.1, 0.2, 0.3}. Fig. 2 shows the averagenumber of rounds to reach the optimum according to mutation ratepmut for different topologies and network sizes. Except for small
sizen = 4, the range of performances is small according to themutation rate.
2500
3000
3500
4000
4500
0.0001 0.001 0.01 0.1 1
Rou
nds
Mutation Rate
n=4n=8
n=16n=36n=64
2500
3000
3500
4000
4500
0.0001 0.001 0.01 0.1 1
Rou
nds
Mutation Rate
n=4n=8
n=16n=36n=64
2500
3000
3500
4000
4500
0.0001 0.001 0.01 0.1 1
Rou
nds
Mutation Rate
n=4n=8
n=16n=36n=64
Figure 2: Average number of rounds according to metaheuris-tic mutation rate for different sizes n and different topologies(from top to bottom: complete, grid, and circle).
Fig. 3 shows the average number of rounds to reach the optimumfor different DAMS as a function of network size and according tothe three different topologies. The metaheuristic mutation rate is setto pmut = 10−3 for SBM-DAMS. SBM-DAMS outperforms thebased-line strategies regardless to topology, except for small net-works with only4 nodes. The difference is statistically significantaccording to the non-parametric test of Mann-Whitney at1% levelof confidence. This shows that the selection best-mutation schemeof SBM-DAMS is efficient against other naive strategies.
From Fig 3, it could be surprising, at a first look, that SBM-DAMS outperforms seqOracle-DAMS, or even more the randomstrategy rnd-DAMS for complete topology with network size over16. To explain this result, Fig. 4 shows the frequency of the numberof nodes selecting each metaheuristic all along SBM-DAMS execu-tion. Left figures display particular runs which obtain the medianperformances for different network sizes. Right figures displayscorresponding average frequencies.
In left figures, we observe that nearly all nodes execute the samemetaheuristic at the same time. One metaheuristic always floodsthe network. Only few tries of other metaheuristics appear with therate of the metaheuristic mutation rate. Nevertheless, thepopula-tion of nodes is able to switch very quickly from one operatortoanother one. In addition, we observe that as the number of nodes
2500
3000
3500
4000
4500
0 10 20 30 40 50 60 70
Rou
nd
Size
SBM-DAMSrnd-DAMS
seqOracle-DAMS
2500
3000
3500
4000
4500
0 10 20 30 40 50 60 70
Rou
nd
Size
SBM-DAMSrnd-DAMS
seqOracle-DAMS
2500
3000
3500
4000
4500
0 10 20 30 40 50 60 70
Rou
nd
Size
SBM-DAMSrnd-DAMS
seqOracle-DAMS
Figure 3: Average number of rounds according to size of thenetwork for different topologies (from top to bottom: complete,grid, and circle)
grows, the1/l bit-flip mutation dominates the5 bits mutation atthe very beginning of the runs. This is confirmed by the averagefrequencies as a function of rounds (right figures) where SBM-DAMS selects bit-flip operator more frequently than5 bits in firstrounds. At the opposite, the sequential oracle seqOracle-DAMSalways chooses the5 bits mutation operator regardless to networksize. In fact, this oracle does not care about the joint/distributedperformances of other nodes and chooses the best operator from apure sequential/selfish point of view. Hence, it always choses the5 bit operators since it has a better performance (in average)com-pared to bit-flip for low fitness. However, this selfish strategy is notoptimal from the distributed point of view. In fact, let us considern nodes with initial solution0l. When they all use the same5 bitsmutation, the maximal fitness gain over the network after oneroundis always5. In that case, the fitness always increases by exactly5.When all nodes use the same1/l bit-flip mutation, the probabilitythat the fitness gain is strictly over5 is 1 − αλ·n whereα is theprobability that at most5 bits are flipped1 in one iteration. Thisprobability increases fast with the number of nodes. Hence,whenconsidering the whole distributed system, the most efficient muta-tion operator is the bit-flip which can not be predicted by a selfishoracle. This difference between an optimal localindependentdeci-sion, and an optimal localdistributeddecision, is naturally capturedby SBM-DAMS which clearly grants more bit-flip as the number
1More precisely,α =∑
5
i=0
(
l
i
)
(1/l)i(1− 1/l)l−i
of nodes increases. Note that this property is captured by SBM-DAMS without any global centralized control. In fact, whilebeingvery local and independent of the topology or any global knowl-edge of the network, the simple local communication policy usedSBM-DAMS seems to be very efficient.
Notice also that the idea of optimal distributed decision, whichis clearly demonstrated at the beginning of a run, could alsoappearlater in the run. However, it is difficult to compare our SBM-DAMSwith a global distributed oracle strategy as the possible ofdifferentof the system (for example here at mostln), and the number ofpossible metaheuristics distribution (here4n) is huge. This issue isleft as an open question and addressed in the conclusion.
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000 3500 4000 4500
Fre
quen
cyRounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
Figure 4: Evolution of the frequency of each mutation oper-ator for the complete network for different sizes (from top tobottom: n = 4, 16, and 36) and pmut = 10−3. Left figuresare particular runs which give the median performances, andright figures are the average performances, as a function of thenumber of rounds.
5.4 Parallelism propertiesIn this section, we study the performance of SBM-DAMS com-
pared to a pure sequential adaptive strategy. Our goal is to evaluatethe impact of parallelism and distributed coordination introducedby SBM-DAMS compared to a sequential setting. Let us first re-mark that at each round of SBM-DAMS, each node is allowed torun an iteration of a(1 + λ)-EA (among four operators). Thus,over all nodes, SBM-DAMS performsλ · n evaluations in parallel.Therefore, to be fair, SBM-DAMS should be compared to a puresequential strategy which is allowed to run a(1 + λ · n)-EA at
each iteration. This, exactly what we report in the following exper-iments, where the performance of SBM-DAMS is studied for fixedλ · n = 50.
In Fig 5, we report the performance of SBM-DAMS accordingto the metaheuristic mutation ratepmut using a complete graphwith n = 50 nodes andλ = 1, i.e., each node is allowed torun an iteration of a(1 + 1)-EA at each round. First, we observeas previously that SBM performs better for a mutation probabilityaround10−3. More importantly, for mutation ratepmut = 10−3,the average number of rounds before SBM finds the best solutionis 5441 (standard deviation is593). As depicted in Fig 5, SBM-DAMS is thus slightly better (in average) than the adaptive multi-armed bandit strategy (DMAB) from [3], but worse than the oraclestrategy and the best existing multi-armed bandit strategyfrom [5].However, we remark that DMAB algorithms are different in naturefrom SBM. In fact, DMAB is allowed to update the informationabout mutation operators each time a new individual (amongλ) isgenerated during the(1 + λ)-EA iteration, whereas SBM is notsince the(1 + 1)-EA in SBM-DAMS is executed in parallel byall nodes. Overall, we can yet reasonably state that SBM is com-petitive compared to other sequential strategies in terms of numberof rounds needed to reach to optimal solution. More importantly,since SBM is distributed/parallel in nature, an effective implemen-tation of SBM on a physical network can lead to better runningtime compared to DMAB which is inherently sequential2. In otherwords, the effective running time of SBM-DAMS could be dividedby λ = 50 (compared to DMAB) which is the best one can hope toobtain when using(1 + λ)-EAs.
4500
5000
5500
6000
6500
7000
7500
0.0001 0.001 0.01
Rou
nds
Mutation Rate
SBM-DAMSDMAB (5467)
ex-DMAB (5088)Oracle (4970)
Figure 5: Average of number of rounds of SBM (completetopology,n = 50 and λ = 1) according to metaheuristic muta-tion rate Vs DMAB, ex-DMAB and the sequential Oracle.
In Fig 6, we report the adaptation properties of SBM-DAMScompared to the sequential oracle strategy. Top left figure showsthat nodes act globally in average closely to the sequentialoracleillustrated by bottom figure. In particular, one can see thatnodesswitch (in average) to the3-bit and the rate-bit mutation operatorsaround the same round than the sequential oracle. This is an inter-esting property especially for these two operators which should beused during a small window of rounds to speed up the search. Topright figure displays one particular run which obtains the medianperformance over all runs. Again, one can clearly see that all nodescan switch from a mutation operator to a better one very quickly atruntime.
Finally, in Fig 7, we report the performance of SBM-DAMS fordifferent topologies using different sizesn such thatλ · n = 50,namely(λ, n) ∈ {(10, 5), (5, 10), (2, 25), (1, 50)}. Left figures
2Each time an individual (overλ) is generated by DMAB, the re-ward information about mutation operators needs to be updated.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 1000 2000 3000 4000 5000 6000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 1000 2000 3000 4000 5000 6000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 1000 2000 3000 4000 5000 6000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 1000 2000 3000 4000 5000 6000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 1000 2000 3000 4000 5000 6000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
Figure 6: Top left (resp. right): Evolution of the average num-ber of nodes (resp. number of nodes for the median run) ofeach mutation operator using a complete topology withn = 50,λ = 1, and pmut = 10−3. Bottom: Evolution of the mutationoperator in average according the the sequential oracle withλ = 50.
reports the robustness of SBM-DAMS according to mutation prob-ability pmut. Right figures displays the evolution of the numberof rounds for a mutation parameterpmut = 10−3 as a functionof n. One can see that for the complete topology, SBM-DAMS isrobust and its performances is almost constant for alll · n = 50.However, it is without surprise that for the grid and the cycle, SBM-DAMS performances degrade as the size of the network goes higher(equivalently asλ goes smaller). This can be intuitively explainedas following: (i) for smallerλ the probability that a given mutationoperator produces a better solution is smaller and (ii) the probabil-ity that a node, which has mutated from one operator to a betterone, informs all other nodes gets smaller as the network diametergets higher. Hence, from a pure parallel point of view, the con-sidered topology could have a non negligible impact on the overallspeedup of SBM-DAMS as previously suggested by Fig. 3.
6. CONCLUSION AND OPEN QUESTIONSIn this paper, we have proposed a new distributed adaptive method,
the select best and mutate strategy (SBM), in the generic frameworkof distributed adaptive metaheuristic selection (DAMS) based ona three-layer architecture. SBM strategy selects locally the meta-heuristic with the best instant reward and mutates it according to amutation rate parameter. The conducted experimental studyshowsSBM robustness and efficiency compared to naive distributedandsequential strategies. Many futures studies and open question aresuggested by DAMS. Firstly, it would be interesting to studyDAMSfor other problems and using other EAs. For instance, when con-sidering population-based metaheuristics many issues could be ad-dressed for population migration and population update. Inpar-ticular, one can use some distributed adaptive policies to guidepopulation evolution. Adaptive migration taking into account theperformance of crossover operators should also be introduced andstudied. DAMS could also be particularly accurate to derivenew
6000
8000
10000
12000
14000
0.0001 0.001 0.01 0.1 1
Rou
nds
Mutation Rate
l=10, n=5l=5, n=10l=2, n=25l=1,n=50
6000
8000
10000
12000
14000
5 10 15 20 25 30 35 40 45 50
Rou
nd
Size
6000
8000
10000
12000
14000
0.0001 0.001 0.01 0.1 1
Rou
nds
Mutation Rate
l=10, n=5l=5, n=10l=2, n=25l=1,n=50
6000
8000
10000
12000
14000
5 10 15 20 25 30 35 40 45 50
Rou
nd
Size
6000
8000
10000
12000
14000
0.0001 0.001 0.01 0.1 1
Rou
nds
Mutation Rate
l=10, n=5l=5, n=10l=2, n=25l=1,n=50
6000
8000
10000
12000
14000
5 10 15 20 25 30 35 40 45 50
Rou
nd
Size
Figure 7: From top to down: complete, grid, cycle topologies.Left: Average number of rounds according to metaheuristicmutation rate. Right: Average number of rounds accordingto sizen verifying λ · n = 50.
algorithms to tackle multi-objective optimization problems, since,for instance, one can use the parallelism properties of DAMStoexplore different regions of the search space by adapting the searchprocess according to different objectives.
For SBM-DAMS, many questions remain open. For instance,one may ask at what extent one can design a distributed strategybased on multi-armed bandits techniques to select metaheuristics,compute metaheuristic rewards, and adapt the search accordingly.Further work needs also to be conducted to study the impact ofthedistributed environment. For instance, asynchrony and lowleveldistributed issues on concrete high performance and large scalecomputing platforms, such as grid and parallel machines, shouldbe conducted. Another challenging issue is to adapt local commu-nications between nodes, for instance, to use the minimum numberof messages and obtain the maximum performances. In particular,it is not clear how to tune the number of nodes and the topologyin a dynamic way in order to balance the communication cost andthe computation time, i.e., parallel efficiency. From the theoreticalside, we also plan to study the parallelism of SBM analytically andits relation to a fully distributed oracle that still needs to be defined.
7. REFERENCES[1] X. Bonnaire and M.-C. Riff. Using self-adaptable probesfor
dynamic parameter control of parallel evolutionaryalgorithms. InFoundations of Intelligent Systems, volume3488 ofLNCS, pages 237–261. 2005.
[2] E. Burke, G. Kendall, J. Newall, E. Hart, P. Ross, andS. Schulenburg. Hyper-Heuristics: An Emerging Direction inModern Search Technology. InHandbook of Metaheuristics,chapter 16, pages 457–474. 2003.
[3] L. Da Costa, A. Fialho, M. Schoenauer, and M. Sebag.Adaptive operator selection with dynamic multi-armedbandits. In10th ACM conf. on Genetic and EvolutionaryComputation (GECCO’08), pages 913–920, 2008.
[4] A. E. Eiben, Z. Michalewicz, M. Schoenauer, and J. E.Smith. Parameter control in evolutionary algorithms. InParameter Setting in Evolutionary Algorithms, pages 19–46.2007.
[5] A. Fialho, M. Schoenauer, and M. Sebag. Towardcomparison-based adaptive operator selection. In12th ACMconf. on Genetic and Evolutionary Computation(GECCO’10), pages 767–774, 2010.
[6] N. Hansen and A. Ostermeier. Completely derandomizedself-adaptation in evolution strategies.EvolutionaryComputation, 9(2):159–195, 2001.
[7] J. L. Laredo, A. E. Eiben, M. Steen, and J. J. Merelo. Evag:ascalable peer-to-peer evolutionary algorithm.GeneticProgramming and Evolvable Machines, 11:227–246, 2010.
[8] J. L. J. Laredo, J. J. Merelo, C. Fernandes, A. Mora, M. I. G.Arenas, P. Castillo, and P. G. Sanchez. Analysing theperformance of different population structures for anagent-based evolutionary algorithm. InLearning andIntelligent Optimization (LION’05), pages 50–54, 2011.
[9] J. Lässig and D. Sudholt. The benefit of migration in parallelevolutionary algorithms. In12th ACM conf. on Genetic andevolutionary computation (GECCO’10), pages 1105–1112,2010.
[10] J. Lässig and D. Sudholt. General scheme for analyzingrunning times of parallel evolutionary algorithms. InParallelProblem Solving from Nature PPSN - XI, volume 6238 ofLNCS, pages 234–243. 2011.
[11] K. G. Srinivasa, K. R. Venugopal, and L. M. Patnaik. Aself-adaptive migration model genetic algorithm for datamining applications.Inf. Sci., 177:4295–4313, 2007.
[12] D. Thierens. An adaptive pursuit strategy for allocatingoperator probabilities. In7th conf. on Genetic andevolutionary computation (GECCO’05), pages 1539–1546,2005.
[13] M. Tomassini.Spatially Structured Evolutionary Algorithms:Artificial Evolution in Space and Time (Natural ComputingSeries). 2005.
[14] S. Tongchim and P. Chongstitvatana. Parallel geneticalgorithm with parameter adaptation.Inf. Process. Lett.,82:47–54, 2002.
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bitrate
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000 3500 4000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bitrate
220000
240000
260000
280000
300000
320000
340000
360000
380000
400000
0.0001 0.001 0.01 0.1 1
Eva
l
Mutation Rate
SBM-DAMSex-DMAB
SEA
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000 3500
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000 3500
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000 3500
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000 3500
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bitrate
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bitrate
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bitrate
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip
0
0.2
0.4
0.6
0.8
1
0 500 1000 1500 2000 2500 3000
Fre
quen
cy
Rounds
fitness5 bits3 bits1 bit
bit-flip