1

OPTIMISATION

Why do you need Meta Heuristics

Dr.S.P.Anbuudayasankar, M.E.., M.B.A., PhD

Associate Professor

Department of Mechanical Engineering

Amrita School of Engineering

Amrita Vishwa Vidyapeetham

Coimbatore - 641112

spanbu@yahoo.com

Optimization vs Decision Problems

Optimization Problem

A computational problem in which the object is to find the

best of all possible solutions

(i.e. find a solution in the feasible region which

has the minimum or maximum value of the

objective function.)

Decision Problem

A problem with a “yes” or “no” answer.2

Objective Function & Search Space

Objective Function

Max (Min) some function of decision variables

Subject to (s.t.)

equality (=) constraints

inequality ( ) constraints

Search Space

Range or values of decisions variables that will be searched

during optimization3

,,,

A solution to an optimization problem

specifies the values of the decision variables, and therefore also the

value of the objective function

A feasible solution

satisfies all constraints

An optimal solution

is feasible and provides the best objective function value

A near-optimal solution

is feasible and provides a superior objective function value, but not

necessarily the best.

Types of Solutions

Optimization problems can be

continuous (an infinite number of feasible solutions)

combinatorial (a finite number of feasible solutions).

Continuous vs Combinatorial

Constraints can be

Hard (must be satisfied) or

Soft (is desirable to satisfy)

Example:

In our course schedule

a hard constraint is that no classes overlap

a soft constraint is that no class be before 10 AM

Constraints

Objectives

Single or

Multiple

Size

number of decision variables,

range/count of possible values of decision variables

search space size

Degree of constraints

number of constraints

difficulty of satisfying constraints

proportion of feasible solutions to search space

Aspects of an Optimization Problem

Aspects of an Optimization Problem (Contd.)

Deterministic (all variables are deterministic) or

Stochastic (the objective function and/or some decision

variables and/or some constraints are random

variables)

Decomposition decomposite a problem into series problems,

and then solve them independently

Relaxation solve problem beyond relaxation, and then can

solve it back easier

8

9

Models

“The essence of Operations Research lies in the

construction and use of models.”

A model is a simplified representation of something real.

10

Optimization need Model

Guess…..What model is this ?

min i j cijxij

s.t. j xij < si, i,

i xij > dj, j,

xij > 0, i, j.

Transportation Model !

min i j cijxij

s.t. j xij < si, i,

i xij > dj, j,

xij > 0, i, j.

Costs PLANT Bearings

Supplier Chennai Madurai Cochin Coimbatore Available

Kumar 21 50 40 35 275,000

Ganesh 35 30 22 42 400,000

Sudhir 55 20 25 70 300,000

Gopal 43 25 37 58 500,000

Capacity 200,000 600,000 225,000 350,000

12

13

Modeling become

P

NP

NP- hard

&

NP – complete

What is P

P = Set of problems than can be solved in

polynomial time

What is Polynomial Time ?

Reasonable Time

What is NP

NP = Set of problems for which a solution

can be verified in polynomial time

What is NP Complete

Minesweeper

TSP

What is NP-Hard

Allocation Problem

Patient 1

Patient 2

Hospital 1

Hospital 2

5

4

6

3

18

? Commonly Believed

Relationship

P

NP NP-

Hard

NP Complete

19

When modeling become

NP-complete

&

NP-hard

We need….

Heuristics

and

Metaheuristics

20

21

Heuristics

Heuristics

are rules

to search to find optimal or near-optimal solutions

Problem Specific Algorithms

Examples

FIFO

LIFO

earliest due date first

largest processing time first

shortest distance first, etc.

Vogel‟s Approximation Method to solve transportation problem22

Heuristics

Examples of problems that are tackled with heuristic algorithms :

Cutting/Packing

Graph problems

Scheduling/Timetabling

23

Packing Problem

How it works (Packing stage)

2 3 41

Gap

How it works (Packing stage)

2 3 4

1

Gap

How it works (Packing stage)

2 3

41 X

How it works (Packing stage)

2 3 4

1 X

How it works (Packing stage)

2 3 4

1 X

Gap

How it works (Packing stage)

2

3 4

1 X

Gap

How it works (Packing stage)

2 3

4

1 X

Gap

How it works (Packing stage)

2 3

4

1 X

X

Gap

How it works (Packing stage)

2 3

4

1 X

X3 X

How it works (postprocessing)

Due to the nature of the algorithm, „towers‟ may form

„Towers‟ are boxes with large height dimensions and lower width

The tallest tower is removed from the packing, rotated

so that it is effectively „knocked down‟ and placed back into the

packing at the lowest point available

This is repeated until the solution is not improved any further

11 4 356 7

10

129 8 3 2

1

Gap

How it works

11 4 356 7

10

129 8 3 2

1

How it works

11 4 356 7

10

129 8 3 2

1

How it works

11 4 356 7

10

129 8 3 2

1

How it works

11 4 356 7

10

129 8 3 21

old height

new heightImprovement

How it works

Have you been paying attention?

Q) How many different orders can these boxes be placed

in?

Answer

4 x 3 x 2 x 1 = 24 different orders

12

34

Cracking a combination lock

How do we pack all of these?

?

(50 pieces)

That‟s not so easy…

50 pieces means 50 x 49 x…..2 x 1 different orderings

=

304140932017133780436126081660647688443776415689605

12000000000000

That’s quite a lot really. If a computer could evaluate

1000 orderings per second it’d still take approximately:

964424568801159882154128873860501295166718720476931

506 years! (and that’s without being allowed to rotate

the pieces)

Travelling Salesman Problem

One of the most researched Problems

43

44

Traveling Salesman Problem (TSP)

Depot

2

13

45

Combinatorial Explosion

46

Combinatorial Explosion

Cities Routes

1 1

2 1

3 3

4 12

5 60

6 360

7 2520

8 20160

9 181440

10 1814400

11 19958400

47

Combinatorial Explosion

A 10 city TSP has 181,000 possible solutions

A 20 city TSP has 10,000,000,000,000,000

possible solutions

A 50 City TSP has

100,000,000,000,000,000,000,000,000,000,00

0,000,000,000,000,000,000,000,000,000,000

possible solutions

There are 1,000,000,000,000,000,000,000

litres of water on the planet

Mchalewicz, Z, Evolutionary Algorithms for Constrained Optimization Problems, CEC 2000 (Tutorial)

Traveling Salesman Problem

If there are n cities,

How many possible solutions exist

The number of possible routes is (n!)/2

49

50

51

91.9 CPU years on a

Intel Xeon 2.8 GHz processor

Vehicle Routing Problems (VRPs)

Proven NP – hard problem

Depot

1

9

8

4

3

2

6

57

Heuristic for VRP

Depot

OVERLOAD

OK !

Depot

OVERLOAD

Heuristic for VRP

Depot

OK !

Heuristic for VRP

56

Meta-Heuristics– what are they?

Wikipedia says

“A metaheuristic is a heuristic method for solving a

very general class of computational problems by

combining user given black-box procedures — usually

heuristics themselves — in a hopefully efficient way.”

57

Meta-heuristics

Genetic algorithms

Simulated Annealing

Ant Colony Optimisation

Particle Swarm Optimisation

Tabu Search

Hill Climbing / Greedy search

… many more

58

Understanding the Concept

X

f (X)

Local

Minima

Global

Minima

Local

Maxima

Global

Maxima

Rastrigin‟s Function

Global minimum at ( 0, 0

)

61