Hybrid Metaheuristics

Prof. E-G. TALBI OPAC (Parallel Cooperative Optimization)

INRIA DOLPHIN ProjectUniversity of Lille, France

http://www.lifl.fr/~talbi

Motivations

Large scale multi-objective optimization problems in different industrial domains (Telecommunications, Genomics, Transportation and Logistics, Engineering design, ...).Problem size more and more important (combinatorialexplosion) et/ou Delays more and more reduced.

Objective : Efficient Modelling and Solving of Large (Multi-objective) Combinatorial Optimization Problems

(PMO))(),...,(),()(min

21xxxxf fff

n=

s.c.

2≥n

Sx∈( )

Sx ∈(POC) )(min xf

Optimization Algorithms

• Exact methods are useless for large problems• Metaheuristics are efficient (lower bound, best known results, …).• Lack of theoretical results (applicable in a real context).

(neighborhood)Tendance in exploitation Tendance in exploration

Exact algorithms Heuristics

Branchand X

Dynamicprogramming CP Specific

(ε-approximation) Metaheuristics

Uniquesolution Population

Hill-climbing Simulatedannealing

Tabusearch

Evolutionaryalgorithms

Ants,SS, PSO

Problems of small size

Solution-based metaheuristics

“Improvement” of a solutionExploitation oriented

Based on the descentExploration of the neighborhood (intensification)

Local searchEscape from local optima: Simulated Annealing, TabuSearch, ILS, VNS, GLS, …

Neighborhood

ReplacementLocal

optima

Population-based metaheuristics

“Improvement” of a population Exploration oriented

Solutions distributed in the search spaceRecombination is explorative

Replacement

Recombination Parents

Offsprings

Replacement

Direct recombination

Evolutionary Algorithms: GA, GP, ES, EAScatter Search

Crossover

MutationPopulation

Offspring

ParentsSelection

Replacement

Indirect recombination: Ant colonies

Artificial ants: Dorigo (1992)Imitate the cooperative behavior of ant colonies to solve optimization problemsUse very simple communication mechanism : pheromone

Olfactive and volatile substanceEvolution : evaporation, reinforcement

M. Dorigo, & G. Di Caro. “The Ant Colony Optimization Meta-heuristic”.New Ideas in Optimization, 1999

A nature-inspired processNest

Food

Nest

Food

Obstacle

Food

Nest

Food

Nest

Equalprobability(left, right)

Higher levelof pheromoneon the left

During the trip, a pheromone is left on the ground.The quantity left depends on the amount of food found. The path is chosen accordingly to the quantity of pheromones.The pheromone has a decreasing action over time.

Nest

Food

Nest

Food

Obstacle

Food

Nest

Food

Nest

Ant : Solution construction, Pheromone update

Particle Swarm

Population based stochasticmetaheuristic

Dr. Eberhart and Dr. Kennedy (1995)

Inspired by social behavior of birdflocking or fish schooling

Kennedy, J. and Eberhart, R. C., "The particle swarm: social adaptation in information processing systems.," in Corne, D., Dorigo, M., and Glover, F.

(eds.) New Ideas in Optimization London, UK: McGraw-Hill, 1999

A nature-inspired process

Particles fly through the problem spaceFlight = add a velocity to the current positionSocial adaptation of knowledgeParticles follow the current optimum particles («follow the bird which is nearest to the food »)

Particle: Evaluate the velocities, Flight, Update the bests

Estimation of Distribution Algorithm

Based on the use of (unsupervised) density estimators/generative statistical modelsIdea is to convert the optimization problem into a search over probability distributionsThe probabilistic model is in some sense an explicit model of (currently) promising regions of the search space

• Initialize a probability model Q(x)

• Do • Create a population of points by sampling from Q(x)

• Evaluate the objective function for each point

• Update Q(x) using selected population and f() values

• While termination criterion not reached

EDA simplest probability model

Population-based incremental Learning (PBIL)Initial distribution D=(0.5, …, 0.5)Iteration :

Generation of the populationXi = 1 if r<Di (r uniform in [0,1]Xi = 0 else

Evaluate and sort the populationUpdate the distribution

S. S. BalujaBaluja, R. , R. CaruanaCaruana. Removing the Genetics from the Standard Genetic Algorithm. IC. Removing the Genetics from the Standard Genetic Algorithm. ICMLML’’9595

( ) bestXDD ⋅+−= αα1

Other probability models

Mutual Information Maximization for Input Clustering (MIMIC) regards pairwise dependances

J. De J. De BonetBonet, C. Isbell and P. Viola. MIMIC: Finding optima by estimating, C. Isbell and P. Viola. MIMIC: Finding optima by estimatingprobability densities. probability densities.

Advances in Neural Information Processing Systems, vol.9, 1997Advances in Neural Information Processing Systems, vol.9, 1997

Bayesian Optimization Algorithm (BOA) for multivariate dependances

M. M. PelikanPelikan, D. Goldberg and E. Cantu, D. Goldberg and E. Cantu--Paz. BOA: The Bayesian optimizationPaz. BOA: The Bayesian optimizationalgorithm. In Proc. GECCOalgorithm. In Proc. GECCO’’9999

Objective function

Variation Operators, Neighborhood

Parallellism and Distribution

Representation (Encoding)

Intensification, Exploitation

Diversification, Exploration

Hybridization

Main search components

Hybrid Metaheuristics

• E-G. Talbi, « A taxonomy of hybrid metaheuristics », Journal of heuristics, 8(5), 2002.

Why Hybridation ?• Equilibrate exploration / exploitation : Gain in performance, Robustness• Taxonomy : Grammar – Large variety of classified methods

Approximate PartialExact General SpecialistGlobal

Hierarchical

Flat

Resolution Search spaceApplication

Problemsuniformity

Hybrid Metaheuristics

Low-level High-level

Relay TeamWork Relay TeamWork

Design issues (Hierarchical)

Low-level / High-levelLow-level : Functional composition of a single method.High-level : Different methods are self-contained.

Relay / TeamworkRelay : Pipeline fashion.Teamwork : Parallel cooperating agents.

1. LRH 2. LTH 3. HRH 4. HTH4 Classes :

Hybrid Metaheuristics

Low-level High-level

Relay TeamWork Relay TeamWork

Low-Level Relay Hybrid

Cost

Configuration

« Kick »Intermediate

Local search

TrialStart

LRH : Hybrids in which a given metaheuristic isembedded into a single-solution metaheuristic

Example : Local search embedded in Simulated annealing(Markov chain explores only local optima) [Martin et Otto 92]

Simulated annealing

Low-Level Teamwork Hybrid

Greedyheuristic

Indiviuals

Crossover

Individual

Mutation

GATabou

LTH : A optimization method is embedded into a population-based method.

Examples :Greedy crossover [Grefenstette 85], local search for mutation [Fleurant 94, Chu 97] (lamarckian, baldwin, …) in GAs.Local search in Ant colonies [Taillard 97, Talbi et al. 99], Scatter search[Van Dat 97], and Genetic programming [O’Reilly 95].

[Talbi 94][Davis 85]

Good exploration Good exploitation

High-Level Relay Hybrid

GA

Tabu GA

Tabu

Greedy heuristic

Initialpopulation

Greedy heuristic

GA

Initial population

Population to exploit

HRH : Self-contained optimization methods are executedin a sequence.

Example :GA + Simulated annealing [Mahfoud 95]. GA + Tabu search [Talbi 94].

ES + Local search [Nissen 94]. Simulated annealing + GA [Lin 91]

Simulated AnnealingGenetic ProgrammingEvolution StrategyTabu searchAnt ColoniesScatter search

High-Level Teamwork Hybrid

GA

GA

GAGA

GA

GA GA

GA

HTH : Parallel cooperating self-contained algorithms.

Example :Island model for GAs [Tanese 87], Genetic programmig [Koza 95],Evolution strategies [Voigt 90].

Tabu search [Rego 96], Simulated annealing [De Falco 95], Antcolonies [Mariano 98], Scatter search [Van Dat et al. 99].

Topology, Frequency of migration, which individuals to migrate ?, Which individuals to replace ?, ...

High-Level Teamwork Hybrid

Example (meta exact): Cooperation between multi-objective genetic algorithm and a single objective Branch & Cut algorithm (sub-problem for Vehicle Routing Problem) [Jozefowiez, Talbi 04].

Multi-objectiveGA

Single-objective Branch & cut

(sub-problems)

Global versus Partial

Partial : Problem is decomposed in sub-problems. Eachalgorithm is dedicated to solve one sub-problem.

Examples :HTH : Tabu search (Vehicle routing) [Taillard 93], Simulatedannealing (placement of macro-cells) [Casoto et al. 86].

HTH : GA (Job-shop scheduling) [Husbands et al. 90].

Problem Specific

Problem

Sub-problem Sub-problem Sub-problem

Geneticalgorithm

Geneticalgorithm

Geneticalgorithm

Tabu searchSimulated annealing

Synchronisation : Build a global viable solution

Specialist versus General

Parallel tabu search to solve the QAPtabu tabu tabu

Communicationmedium

Frequencymemory

Initialsolutions

Genetic algorithm to solve a different optimization problem(diversification task)

Specialist : Algorithms which solve different problems.

Examples : HTH : GA + Tabu (QAP) [Talbi et al. 98]

HRH : GA + (SA | NM | GA | AC) [Krueger 93][Shahookar et al. 90][Abbattista et al. 95]

HTH

Geneticalgorithm

Simulatedannealing

HRH

Noisy methodGenetic algorithmAnt colonies

Optimizing the parametersof another heuristic

Robot path planning

(European project PAPAGENA)

[E-G. Talbi, P. Bessière, E. Mazer, J-M. Ahuactzin, 1994]

⎪⎩

⎪⎨⎧

−=

−=∈ mMinFMin

iaiSM MS S τ

τ1,

0),(

If a direct movement exist from mi to τ (i<a)

Else

Search Agent SEARCH (local optima)

Robot path planningDiversification agent EXPLORE

Fil d’Ariane (Real robots with Aleph Technologies)

λλ kaMeM mMinMaxFMinkee SMS −= Λ∈∈∈ )( { }λλλ n,...,, 21=Λ

Set of landmarks

EXPLORE(Parallel GA)

SEARCH(Parallel GA)

Hybrid Metaheuristics

E-G. Talbi et al. COSEARCH: A parallel cooperativ metaheuristic. Journal of Mathematical Modeling and algorithms JMMA, Vol.5(2), 2006.

COSEARCHCOSEARCH : 3 complementary agents cooperatevia an adaptive memory

Diversifying Agent Intensifying Agent

Search Agent

Explored regions Promising regionsAdaptative Memory

Good solutions

exploredspace

Promisingsolutions

Initial solutions

Ref

erst

o

Ref

erst

oSolutionsin

unexplored regions

COSEARCH for QAP

Path relinking Intensifier

Multiple Tabu Search

Bestsolution

found

Localfrequency

matrix

Solutions beingin promising

regions

Solutions beingin unexploredregions

Adaptive memory

Initial Solutions Elite SolutionsGlobal Frequencies

. . .. . .

Tabu search Tabu search Tabu search. . .

The fitness functionrefers to the

adaptive memory

Pickelitesolutions

Initialsolution

Diversifying Genetic Algo

Cooperation Meta-Exact

Branch andBound, Cut

CooperativeMetaheuristic

Upper bound,Partial solutions

Lower bound, Optimal Solution sous-pb,

Best neighbor

• M. Basseur, J. Lemesre, C. Dhaenens, E-G. Talbi, «Cooperation between branch and bound and evolutionary algorithms to solve a bi-objective flow-shop problem», WEA’2004, LNCS, Springer, 2004.

Heuristic approach :VLNS : Very Large Neighborhood Search (use of an exact method) Generation of different Sub-Problems solved by an exact method

Exact approach :Good solutions found by metaheuristics to reduce the visited search space

for exact methods

• M. Basseur, L. Jourdan E-G. Talbi, «Cooperation between exact methods and metaheuristics : A survey»,EJOR Journal.2007

Constraintprogramming

Grammar for extended schemes

< hybrid method > < design-issues > < implementation-issue > < design-issues > < hierarchical > < flat >< hierarchical > < LRH >|< LCH >|< HRH >|< HCH >< LRH > LRH (< method >( < method >))< LCH > LCH (< method >( < method >))< HRH > (< method >+ < method >)< HCH > HCH (< method>)< HCH > HCH (< method >,< method >)< flat > (< resolution >,< optimization >,< function >)< resolution > exact | approached

global | partial< optimization >

< function > general | specialist< implementation-issue > < scheduling >

sequential | parallel < scheduling >static | dynamic | adaptive

< method > < exact > | < heuristic >< heuristic > LS | TS | SA | GA | ES | GP | GH | AC | SS | NM | ... < hybrid method< exact > B&B | B&C | B&P | PL | PD | MS | ... < hybrid method >

Example of extended hybrid

HTH (HRH (GH + LTH(GA(LS))))

[Levine 94]

Set Partitioning ProblemAirline crew scheduling

Parallel staticimplementation

GA

GA

GAGA

GA

GA GA

GAGreedy

heuristic

Geneticalgorithm

Local search

[Braun 90]

Traveling salesmanproblem

Sequential implementation