13 od metaheuristics a-2008

1METAHEURISTICS

Introduction

Some problems are so complicated that are not

possible to solve for an optimal solution.

In these problems, it is still important to find a good

feasible solution close to the optimal.

A heuristic method is a procedure likely to find a very

good feasible solution of a considered problem.

Procedure should be efficient to deal with very large

problems, and be an iterative algorithm.

Heuristic methods usually fit a specific problem

rather than a variety of applications.

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho 433

Introduction

For a new problem, OR team would need to start

from scratch to develop a heuristic method.

This changed with the development of

metaheuristics.

A metaheuristic is a general solution method that

provides a general structure and strategy guidelines

for developing a heuristic method for a particular

type of problem.


Nature of metaheuristics

Example: Maximize

Function has three local optima (where?).

The example is a nonconvex programming problem.

f(x) is sufficiently complicated to solve analytically.

Simple heuristic method: conduct a local

improvement procedure.

5 4 3 2( ) 12 975 28,000 345,000 1,800,000f x x x x x x= + +

subject to0 31x


Example: objective function


Local improvement procedure

Starts with initial trial solution and uses a hill-climbing

procedure.

Example: gradient search procedure, bisection method,

etc.

Converges to a local optimum. Stops without

reaching global optimum (depends on initialization).

Typical sequence: see figure.

Drawback: procedure converges to local optimum. This

is only a global optimum if search begins in the

neighborhood of this global optimum.


2Local improvement procedure


Nature of metaheuristics

How to overcome this drawback?

What happens in large problems with many

variables?

Metaheuristic: general solution method that

orchestrates the interaction between local

improvement procedures and higher level strategies

to create a process to escape from local optima and

perform a robust search of a feasible region.

A trial solution after a local optimum can be inferior to

this local optimum.


Solutions by metaheuristics


Metaheuristics

Advantage: deals well with large complicated

problems.

Disadvantage: no guarantee to find optimal solution

or even a nearly optimal solution.

When possible, an algorithm that can guarantee

optimality should be used instead.

Can be applied to nonlinear or integer programming.

Most commonly it is applied to combinatorial

optimization problems.


Traveling Salesman Problem


Traveling Salesman Problem

Can be symmetric or asymmetric.

Objective: route that minimizes the distance (cost,

time).

Applications: truck delivering goods, drilling holes

when manufacturing printed circuit boards, etc.

Problem with n cities and a link between every pair of

cities has (n 1)!/2 feasible routes.

10 cities: 181 440 feasible solutions

20 cities: 61016 feasible solutions

50 cities: 31062 feasible solutions


3Traveling Salesman Problem

Some TSP problems can be solved using branch-and-

cut algorithms.

Heuristic methods are more general. A new solution

is obtained by making small adjustments to the

current solution. An example:

Sub-tour reversal adjusts sequence of visited cities

by reversing order in which subsequence is visited.

Solution of example: 1-2-3-4-5-6-7-1 Distance = 69

Reversing 3-4: 1-2-4-3-5-6-7-1 Distance = 65


Solving example


Sub-tour reversal algorithm

Initialization: start with any feasible solution.

Iteration: for current solution consider all possible

ways of performing a sub-tour reversal (except

reversal of entire tour). Select the one that provides

the largest decrease in traveled distance.

Stopping rule: stop when no sub-tour reversal

improves current trial solution. Accept solution as

final one.

Local improvement algorithm: does not assure

optimal solution!


Example

Iteration 1: starting with 1-2-3-4-5-6-7-1 (Distance = 69), 4 possible sub-tour reversals that improve solution are:

Reverse 2-3: 1-3-2-4-5-6-7-1 Distance = 68





Example

Iteration 2: continuing with 1-2-4-3-5-6-7-1 only 1 sub-tour reversal leads to improvement

Reverse 3-5-6: 1-2-4-6-5-3-7-1 Distance = 64.

Algorithm stops. Last solution is final solution (is it the optimal

one?).


Tabu Search

Includes a local search procedure, allowing non-

improvement moves to the best solution.

Referred to as steepest ascent/mildest descent

approach.

To avoid cycling a local optimum, a tabu list is added.

Tabu list records forbidden moves, knows as tabu

moves.

Thus, it uses memory to guide the search.

Can include intensification or diversification.


4Basic tabu search algorithm

Initialization: start with a feasible initial solution.

Iteration:

1. Use local search to define feasible moves in neighborhood.

2. Disregard moves in tabu list, unless they result in a better solution.

3. Determine which move provides best solution.

4. Adopt this solution as next trial solution.

5. Update tabu list.

Stopping rule: stop using fixed number of iterations,

fixed number of iterations without improvement,

etc. Accept best trial solution found as final solution.


Questions in tabu search

1. Which local search procedure should be used?

2. How to define the neighborhood structure?

3. How to represent tabu moves in the tabu list?

4. Which tabu move should be added to the tabu list in

each iteration?

5. How long should a tabu move remain in the tabu list?

6. Which stopping rule should be used?


Ex: minimum spanning tree problem

Problem without constraints: solved using greedy algorithm.


Adding constraints

Constraint 1: link AD can be included only if link DE is

also included.

Constraint 2: at most one of the three links AD, CD

and AB can be included.

Previous solution violates both constraints.

Applying tabu search:

Charge a penalty of 100 if Constraint 1 is violated.

Charge a penalty of 100 if two of the three links in Constraint 2 are included. Increase penalty to 200 if all three links are included.


Answers in tabu search - implementation

1. Local search procedure: choose best immediate neighbor not ruled out by its tabu status.

2. Neighborhood structure: immediate neighbor is one reached by adding a single link and then deleting one of the other links in the cycle.

3. Form of tabu moves: list links not to be deleted.

4. Addition of a tabu move: add chosen link to tabu list.

5. Maximum size of tabu list: two (half of total links).

6. Stopping rule: three iterations without improvement.


Solving example

Initial solution: solution of unconstrained version

Cost = 20 + 10 + 5 + 15 + 200 (why?) = 250

Iteration 1. Options to add a link are BE, CD and DE.

Add Delete Cost

BE CE 75 + 200 = 275

BE AC 70 + 200 = 270

BE AB 60 + 100 = 160

CD AD 60 + 100 = 160

CD AC 65 + 300 = 365

DE CE 85 + 100 = 185

DE AC 80 + 100 = 180

DE AD 75 + 0 = 75 Minimum


5Application of tabu search


Add DE to network.

Delete AD from network.

Add DE to tabu list

Iteration 2

Options to add a link are AD, BE and CD.

Add Delete Cost

AD DE* (Tabu move)

AD CE 85 + 100 = 185

AD AC 80 + 100 = 180

BE CE 100 + 0 = 100

BE AC 95 + 0 = 95

BE AB 85 + 0 = 85 Minimum

CD DE* 60 + 100 = 160

CD CE 95 + 100 = 195

*A tabu move; only considered if it results in better solution than

best trial solution found previously.


Add BE to tabu list

Iteration 3

Options to add a link are AB, AD and CD.

Add Delete Cost

AB BE* (Tabu move)

AB CE 100 + 0 = 100

AB AC 95 + 0 = 95

AD DE* 60 + 100 = 160

AD CE 95 + 0 = 95

AD AC 90 + 0 = 90

CD DE* 70 + 0 = 70 Minimum

CD CE 105 + 0 = 105

*A tabu move; only considered if result in better solution than best

trial solution found previously.


Add CD to tabu list

Delete DE from tabu list

Optimal solution


Traveling salesman problem example

1. Local search procedure: choose best immediate neighbor not ruled out by tabu status.

2. Neighborhood structure: immediate neighbor is the one reached by making a sub-tour reversal (add and delete two links of current solution).

3. Form of tabu moves: list links such that a sub-tour reversal would be tabu if both links are in the list.

4. Addition of a tabu move: add two new chosen links to tabulist.

5. Maximum size of tabu list: four links.

6. Stopping rule: three iterations without improvement.


Solving problem

Initial trial solution: 1-2-3-4-5-6-7-1 Distance = 69

Iteration 1: choose to reverse 3-4.

Deleted links: 2-3 and 4-5

Added links (tabu list): 2-4 and 3-5

New trial solution: 1-2-4-3-5-6-7-1 Distance = 65

Iteration 2: choose to reverse 3-5-6.

Deleted links: 4-3 and 6-7 (OK since not in tabu list)

Added links: 4-6 and 3-7

Tabu list: 2-4, 3-5, 4-6 and 3-7



6Solving problem

Only two immediate neighbors:

Reverse 6-5-3: 1-2-4-3-5-6-7-1 Distance = 65. (This would delete links 4-6 and 3-7 that are in the tabu list.)





Tabu list: 4-6, 3-7, 5-7 and 3-1



Sub-tour reversal of 3-7 (Iteration 3)


Iteration 4

Four immediate neighbors:

Reverse 2-4-6-5-7: 1-7-5-6-4-2-3-1 Distance = 65







Tabu list: 5-7, 3-1, 6-7 and 5-3

New trial (and final) solution: 1-2-4-6-7-5-3-1 Distance = 63


Sub-tour reversal of 5-7 (Iteration 4)


13 od metaheuristics a-2008

Documents