ant colony optimization: an introductionchivdan/presentations/aco-03-04-2013.pdf · ant colony...

Ant Colony Optimization: an introduction

Daniel Chivilikhin03.04.2013

2

Outline

1. Biological inspiration of ACO2. Solving NP-hard combinatorial problems3. The ACO metaheuristic4. ACO for the Traveling Salesman Problem

3

Outline

1. Biological inspiration of ACO2. Solving NP-hard combinatorial problems3. The ACO metaheuristic4. ACO for the Traveling Salesman Problem

4

Biological inspiration: from real to artificial ants

5

Ant colonies

• Distributed systems of social insects• Consist of simple individuals• Colony intelligence >> Individual intelligence

6

Ant Cooperation

• Stigmergy – indirect communication between individuals (ants)

• Driven by environment modifications

7

Denebourg ’s double bridge experiments

• Studied Argentine ants I. humilis• Double bridge from ants to food source

8

Double bridge experiments: equal lengths (1)

9

Double bridge experiments: equal lengths (2)

• Run for a number of trials• Ants choose each branch ~ same number

of trials

10

Double bridge experiments: different lengths (2)

11

Double bridge experiments: different lengths (2)

• The majority of ants follow the short path

12

Outline

1. Biological inspiration of ACO2. Solving NP -hard combinatorial problems3. The ACO metaheuristic4. ACO for the Traveling Salesman Problem

13

Solving NP -hard combinatorialproblems

14

Combinatorial optimization

• Find values of discrete variables• Optimizing a given objective function

15

Combinatorial optimization

Π = (S, f, Ω) – problem instance• S – set of candidate solutions• f – objective function• Ω – set of constraints• – set of feasible solutions (with

respect to Ω)• Find globally optimal feasible solution s*

SS ⊆~

16

NP-hard combinatorial problems

• Cannot be exactly solved in polynomial time

• Approximate methods – generate near-optimal solutions in reasonable time

• No formal theoretical guarantees• Approximate methods = heuristics

17

Approximate methods

• Constructive algorithms• Local search

18

Constructive algorithms

• Add components to solution incrementally• Example – greedy heuristics:

add solution component with best heuristic estimate

19

Local search

• Explore neighborhoods of complete solutions

• Improve current solution by local changes– first improvement– best improvement

20

What is a metaheuristic ?

• A set of algorithmic concepts• Can be used to define heuristic methods• Applicable to a wide set of problems

21

Examples of metaheuristics

• Simulated annealing• Tabu search• Iterated local search• Evolutionary computation• Ant colony optimization• Particle swarm optimization• etc.

22

Outline

1. Biological inspiration of ACO2. Solving NP-hard combinatorial problems3. The ACO metaheuristic4. ACO for the Traveling Salesman Problem

23

The ACO metaheuristic

24

ACO metaheuristic

• A colony of artificial ants cooperate in finding good solutions

• Each ant – simple agent• Ants communicate indirectly using

stigmergy

25

Combinatorial optimization problem mapping (1)

• Combinatorial problem (S, f, Ω(t))• Ω(t) – time-dependent constraints

Example – dynamic problems

• Goal – find globally optimal feasible solution s*

• Minimization problem• Mapped on another problem…

26

Combinatorial optimization problem mapping (2)

• C = c1, c2, …, cNc – finite set of components

• States of the problem:

X = x = <ci, cj, …, ch, …>, |x| < n < +∞

• Set of candidate solutions:

XS ⊆

27

Combinatorial optimization problem mapping (3)

• Set of feasible states:

• We can complete into a solution satisfying Ω(t)

• Non-empty set of optimal solutions:

XX ⊆~

Xx~∈

SXS I~* ⊆

28

Combinatorial optimization problem mapping (4)

S* X~S

X• X – states

• S – candidate solutions

• – feasible states

• S* – optimal solutions

X~

29

Combinatorial optimization problem mapping (5)

• Cost g(s, t) for each

• In most cases –g(s, t) ≡ f(s, t)

• GC = (C, L) – completely connected graph

• C – set of components

• L – edges fully connecting the components (connections)

• GC – construction graph

Ss ∈Ss~∈∀

30

Combinatorial optimization problem mapping (last )

• Artificial ants build solutions by performing randomized walks on GC(C, L)

31

Construction graph

• Each component ci or connection lij have associated: heuristic information

pheromone trail

32

Heuristic information

• A priori information about the problem• Does not depend on the ants• On components ci – ηi

• On connections lij – ηij

• Meaning:cost of adding a component to the current solution

33

Pheromone trail

• Long-term memory about the entire search process

• On components ci – τi

• On connections lij – τij

• Updated by the ants

34

Artificial ant (1)

• Stochastic constructive procedure• Builds solutions by moving on GC

• Has finite memory for:– Implementing constraints Ω(t)

– Evaluating solutions– Memorizing its path

35

Artificial ant (2)

• Has a start state x• Has termination conditions ek

• From state xr moves to a node from the neighborhood – Nk(xr)

• Stops if some ek are satisfied

36

Artificial ant (3)

• Selects a move with a probabilistic rule depending on: Pheromone trails and heuristic information of

neighbor components and connections

Memory Constraints Ω

37

Artificial ant (4)

• Can update pheromone on visited components (nodes)

• and connections (edges)• Ants act:

Concurrently Independently

38

The ACO metaheuristic

While not doStop():ConstructAntSolutions()UpdatePheromones()DaemonActions()

39

ConstructAntSolutions

• A colony of ants build a set of solutions• Solutions are evaluated using the

objective function

40

UpdatePheromones

• Two opposite mechanisms: Pheromone deposit

Pheromone evaporation

41

UpdatePheromones : pheromone deposit

• Ants increase pheromone values on visited components and/or connections

• Increases probability to select visited components later

42

UpdatePheromones : pheromone evaporation

• Decrease pheromone trails on all components/connections by a same value

• Forgetting – avoid rapid convergence to suboptimal solutions

43

DaemonActions

• Optional centralized actions, e.g.: Local optimization

Ant elitism (details later)

44

ACO applications

• Traveling salesman• Quadratic assignment• Graph coloring• Multiple knapsack• Set covering• Maximum clique• Bin packing• …

45

Outline

1. Biological inspiration of ACO2. Solving NP-hard combinatorial problems3. The ACO metaheuristic4. ACO for the Traveling Salesman

Problem

46

ACO for the Traveling Salesman Problem

47

Traveling salesman problem

• N – set of nodes (cities), |N| = n• A – set of arcs, fully connecting N• Weighted graph G = (N, A)• Each arc has a weight dij – distance• Problem:

Find minimum length Hamiltonian circuit

48

TSP: construction graph

• Identical to the problem graph• C = N• L = A• states = set of all possible partial tours

49

TSP: constraints

• All cities have to be visited• Each city – only once• Enforcing – allow ants only to go to new

nodes

50

TSP: pheromone trails

• Desirability of visiting city j directly after i

51

TSP: heuristic information

• ηij = 1 / dij

• Used in most ACO for TSP

52

TSP: solution construction

• Select random start city• Add unvisited cities iteratively• Until a complete tour is built

53

ACO algorithms for TSP

• Ant System• Elitist Ant System• Rank-based Ant System• Ant Colony System• MAX-MIN Ant System

54

Ant System: Pheromone initialization

• Pheromone initializationΤij = m / Cnn,

where: m – number of ants

Cnn – path length of nearest-neighbor algorithm

55

Ant System: Tour construction

• Ant k is located in city i

• is the neighborhood of city i

• Probability to go to city :

∑∈

⋅⋅

=kiNl

ilil

ijijkijp βα

βα

ητητ

kiN

kiNj ∈

56

Tour construction: comprehension

• α = 0 – greedy algorithm• β = 0 – only pheromone is at work

quickly leads to stagnation

∑∈

⋅⋅

=kiNl

ilil

ijijkijp βα

βα

ητητ

57

Ant System: update pheromone trails – evaporation

Evaporation for all connections∀(i, j) ∈ L:

τij ← (1 – ρ) τij,

ρ ∈[0, 1] – evaporation rate

Prevents convergence to suboptimal solutions

58

Ant System: update pheromone trails – deposit

• Tk – path of ant k

• Ck – length of path Tk

• Ants deposit pheromone on visited arcs:

( ) Ljim

k

kijijij ∈∀∆+← ∑

=

,,1

τττ

( ) ∈

=∆otherwise

TjiC kkkij

,0

,,/1τ

59

Elitist Ant System

• Best-so-far ant deposits pheromone on each iteration:

bestij

m

k

kijijij e ττττ ∆+∆+← ∑

=1

( ) ∈

=∆otherwise

TjiC bestbestbestij

,0

,,/1τ

60

Rank-based Ant System

• Rank all ants• Each ant deposits amounts of pheromone

proportional to its rank

61

MAX-MIN Ant System

1. Only iteration-best or best-so-far ant deposits pheromone

2. Pheromone trails are limited to the interval [τmin, τmax]

62

Ant Colony System

• Differs from Ant System in three points: More aggressive tour construction rule

Only best ant evaporates and deposits pheromone

Local pheromone update

63

Ant Colony System

≤

= ∈

otherwiseSystemAntlike

qqifj

ililNl ki

,

,maxarg 0βητ

1.Tour Construction

2. Local pheromone update:

τij ← (1 – ξ)τij + ξτ0,

64

Comparing Ant System variants

65

State of the art in TSP

• CONCORDE http://www.tsp.gatech.edu/concorde.html

• Solved an instance of 85900 cities• Computation took 286-2719 CPU days!

66

Current ACO research activity

• New applications• Theoretical proofs

67

Further reading

• M. Dorigo, T. Stützle. Ant Colony Optimization. MIT Press, 2004.

• http://iridia.ulb.ac.be/~mdorigo/ACO/

68

Next time …

• Some proofs of ACO convergence

69

Thank you!

Any questions?

This presentation is available at:http://rain.ifmo.ru/~chivdan/presentations/

Daniel Chivilikhin [mailto: chivdan@gmail.com]

70

Used resources• http://teamaltman.com/wp-content/uploads/2011/06/Uncertainty-Ant-Apple-

1024x1024.jpg

• http://myrealestatecoach.files.wordpress.com/2012/04/ant.jpg• http://ars.els-cdn.com/content/image/1-s2.0-S1568494613000264-gr3.jpg

• http://www.theorie.physik.uni-goettingen.de/forschung/ha/talks/stuetzle.pdf

• http://www.buyingandsellingwebsites.com/wp-content/uploads/2011/12/Ant-150.jpg

• http://moodle2.gilbertschools.net/moodle/file.php/1040/Event-100_Days/Ant_Hormiga.gif

• http://4.bp.blogspot.com/_SPAe2p8Y-kg/TK3hQ8BUMtI/AAAAAAAAAHA/p75K-GcT_oo/s1600/ant+vision.GIF

• http://1.bp.blogspot.com/-tXKJQ4nSqOY/UQZ3vabiy_I/AAAAAAAAAjA/IK8jtlhElqk/s1600/16590492-illustration-of-an-ant-on-a-white-background.jpg