metaheuristics techniques (2)
DESCRIPTION
Simulated annealingTRANSCRIPT
Company
LOGO
Scientific Research Group in Egypt (SRGE)
Meta-heuristics techniques (II)Simulated annealing
Dr. Ahmed Fouad AliSuez Canal University,
Dept. of Computer Science, Faculty of Computers and informatics
Member of the Scientific Research Group in Egypt
Company
LOGO Meta-heuristics techniques
Company
LOGO Outline
3. Simulated annealing (SA)(Background)3. Simulated annealing (SA)(Background)
4. SA (main concepts)4. SA (main concepts)
6. SA control parameters6. SA control parameters
5. SA algorithm5. SA algorithm
7. SA example7. SA example
2. Motivation2. Motivation
1. What is annealing?1. What is annealing?
8. SA applications8. SA applications
Company
LOGO What is annealing?
• Annealing is a physical process used
to harden metals, glass, etc.
• Starts at a high temperature and
cools slowly.
• Lower temperature of the glass slowly so that at each temperature the atoms can move just enough to begin adopting the most stable orientation.
Company
LOGO What is annealing?
• The temperature determines how much mobility the atoms have.
• How slowly you cool glass is critical.
• If the glass is cooled slowly enough, the atoms are able to "relax" into the most stable orientation.
• For example, the glass surrounding a hockey rink.
Company
LOGO Motivation
startingpoint
descenddirection
local minima
global minima
barrier to local search
?
Company
LOGOSimulated annealing (SA)(Background)
• The method was independently described by Scott Kirkpatrick, C. Daniel Gelatt and Mario P. Vecchi in 1983, and by Vlado Černý in 1985.
• SA is probably the most widely used meta-heuristic in combinatorial optimization problem.
• It was motivated by the analogy between the physical annealing of metals and the process of searching for the optimal solution in a combinatorial optimization problem.
• The main objective of SA method is to escape from local optima and so to delay the convergence.
Company
LOGO Simulated annealing (main concepts)
• SA proceeds in several iterations from an initial solution x0.
• At each iteration, a random neighbor solution x' is generated.
• The neighbor solution that improves the cost function is always accepted.
• Otherwise, the neighbor solution is selected with a given probability that depends on the current temperature T and the amount of degradation E of the objective function.
• E represents the difference in the objective value between the current solution x and the generated neighboring solution x'.
Company
LOGO Simulated annealing (main concepts)
• The probability follows, in general, the Boltzmann distribution as shown in the following equation
• Many trial solutions are generated as an exploration process at a particular level of temperature.
• The temperature is updated until stopping criteria satisfied.
Company
LOGOSimulated annealing algorithm
Inner loop
Outerloop
Company
LOGO SA control parameters
• Choosing too high temperature will cost computation time expensively.
• Too low temperature will exclude the possibility of ascent steps, thus losing the global optimization feature of the method.
• We have to balance between these two extreme procedures.
Initial temperature
Company
LOGO SA control parameters
• If the temperature is decreased slowly, better solutions are obtained but with a more significant computation time.
• A fast decrement rule causes increasing the probability of being trapped in a local minimum.
Choice of the temperature reduction strategy
Company
LOGO SA control parameters
In order to reach an equilibrium state at each temperature, a number of sufficient transitions (moves) must be applied.
The number of iterations must be set according to the size of the problem instance and particularly proportional to the neighborhood size.
Equilibrium State.
Company
LOGO SA control parameters
• Concerning the stopping condition, theory suggests a final temperature equal to 0.
• In practice, one can stop the search when the probability of accepting a move is negligible, or reaching a final temperature TF.
Stopping criterion
Company
LOGO SA examples1||SwjTj
Jobs 1 2 3 4
wj 4 5 3 5
pj 12 8 15 9
dj 16 26 25 27
Company
LOGO SA examples
Step 1. S0=S1=(1,3,2,4). G(S1)=136. Let T=10 and a=0.9 => b1=9
Step 2. Select randomly which jobs to swap, suppose a Uniform(0,1) random number is V1= 0.24 => swap first two jobsSc=(3,1,2,4), G(Sc)=174, P(Sk,Sc)=1.5%U1=0.91 => Reject Sc
Step 3. Let k=2
Iteration 1
Company
LOGO SA examples
Step 2. Select randomly which jobs to swap, suppose a Uniform(0,1) random number is V2= 0.46 => swap 2nd and 3rd jobsSc=(1,2,3,4), G(Sc)=115ÞS3=S0=Sc
Step 3. Let k=3
Iteration 2
Company
LOGO SA examples
Step 2. V3= 0.88 => swap jobs in 3rd and 4th position
Sc=(1,2,4,3), G(Sc)=67ÞS4=S0=Sc
Step 3. Let k=4
Iteration 3
Company
LOGO SA examples
Step 2. V4= 0.49 => swap jobs in 2nd and 3rd positionSc=(1,4,2,3), G(Sc)=72, b4=10(0.9)4=6.6P(Sk,Sc)=47%, U4=.90 => S5=S4
Step 3. Let k=5
Iteration 4
Company
LOGO SA examples
Step 2. V5= 0.11 => swap 1st and 2nd jobsSc=(2,1,4,3), G(Sc)=83, b5=10(0.9)5=5.9P(Sk,Sc)=7%, U5=0.61 => S6=S5
Step 3. Let k=6
Iteration 5
Company
LOGO SA Applications
Scheduling Quadratic assignme
nt
Frequency
assignment
Car pooling Capacitated p-
median,
Resource constrained
project scheduling
(RCPSP)
Vehicle routing
problems
Graph coloring
Retrieval Layout
Problem
Maximum Clique
Problem,
Traveling Salesman Problems
Database systems
Nurse Rostering Problem
Neural Nets Grammatical
inference,
Knapsack problems
SAT Constrain
t Satisfacti
on Problems
Network design
Telecomunication
Network
Global Optimizati
on
Company
LOGO References
Metaheuristics From design to implementation, El-Ghazali Talbi, University of Lille – CNRS – INRIA.
S. Kirkpatrick, C. Gelatt, M. Vecchi, Optimization by simulated annealing, Science 220 (1983), 671-680.
