antsynnet: applying ant colony optimization to the network...

Carleton University Final Year Project in Computer Science

AntSynNet: Applying Ant Colony Optimization to the Network Synthesis Problem

Simon Wilkinson Student: 100229698

Supervisor: Dr. Tony White, SCS

December 19th, 2003

1. Introduction

The Ant Colony Optimization (ACO) meta-heuristic has been applied to many

discrete optimization problems such as the Traveling Salesman Problem (TSP) and the

quadratic assignment problem (QAP) with varying degrees of success. Many practical

problems have constraints that make them difficult for the ACO algorithms to process.

Therefore, new methods and extensions to the ACO algorithm have great importance to

the further development of this meta-heuristic technique.

In this paper, the application of ACO to a network synthesis problem is examined.

This type of problem has direct applications to the design of telecommunication systems.

The problem is highly constrained and, at times, finding feasible solutions can be

extremely difficult. This paper addresses an approach to this problem that is appropriate

for processing by ACO. "Extending ant like algorithms to situations like

telecommunications networks which are not found in nature will also increase our

understanding of the abstract and general abilities of such algorithms over and above

those applications found in nature." [Schoonderwoerd et al., 1996]

This paper is organized as follows. Section 2 gives background information on

the ACO meta-heuristic, while Section 3 discusses network design and outlines the

network synthesis problem. Section 4 defines the modeling approach and strategy used

to solve the problem. From this, the results will be presented in Section 5, and finally, in

Section 6, conclusion and future work are discussed.

2. Ant Colony Optimization

2.1 History

Ant Colony Optimization (ACO) studies artificial systems that take inspiration

from the behaviour of real ant colonies and which are used to solve discrete optimization

problems.

Ant algorithms were first proposed by Dorigo and colleagues [Dorigo et al., 1991]

as a multi-agent approach to difficult combinatorial optimization problems like the

traveling salesman problem (TSP) and the quadratic assignment problem (QAP), and

later introduced the ACO meta-heuristic [Dorigo and Di Caro, 1999]. There is currently a

lot of ongoing activity in the scientific community to extend/apply ant-based algorithms

to many different discrete optimization problems. Recent applications cover problems

like vehicle routing [Bullnheimer et al., 1999], graph coloring [Costa and Hertz, 1997],

and routing in communications networks [Di Caro and Dorigo, 1998][Varela and

Sinclair, 1999].

Ant algorithms were inspired by the observation of real ant colonies. Ants are

social insects, that is, insects that live in colonies and whose behavior is directed more to

the survival of the colony as a whole than to that of a single individual component of the

colony. Social insects have captured the attention of many scientists because of the high

level of structure their colonies can achieve, especially when compared to the relative

simplicity of the colony’s individuals. An important and interesting behavior of ant

colonies is their foraging behavior, and, in particular, how ants can find shortest paths

between food sources and their nest. While walking from food sources to the nest and

vice versa, ants deposit on the ground a substance called pheromone, forming in this way

a pheromone trail. Ants can smell pheromone and, when choosing their way, they tend to

choose, in probability, paths marked by strong pheromone concentrations. The

pheromone trail allows the ants to find their way back to the food source (or to the nest).

It can also be used by other ants to find the location of the food sources found by their

nest-mates. It has been shown experimentally that this pheromone trail following

behavior can give rise, once employed by a colony of ants, to the emergence of shortest

paths. That is, when more paths are available from the nest to a food source, a colony of

ants may be able to exploit the pheromone trails left by the individual ants to discover the

shortest path from the nest to the food source and back.

The Ant Colony Optimization meta-heuristic employs a co-operating colony of

ants in an attempt to find good solutions to difficult discrete optimization problems.

Cooperation is a key design component of ACO algorithms: The choice is to allocate the

computational resources to a set of relatively simple agents (artificial ants) that

communicate indirectly by stigmergy. Good solutions are an emergent property of the

agents’ cooperative interaction.

Artificial ants have a double nature. First, they are an abstraction of real ants

found in nature and of their behaviour. On the other hand, artificial ants have also been

enriched with some capabilities which are not found among their natural counterparts.

Since the artificial ants are employed in software to solve difficult discrete optimization

problems, it is reasonable to give the ants some capabilities, which while not found in

real ant behaviour, make them more efficient and effective.

2.2 Similarities and Differences With Real Ants

This section will explore some of the more important similarities and differences

between real ants and the artificial ants employed by the ACO meta-heuristic.

Most of the ideas of ACO stem from real ants and their behaviour. In particular,

the use of: (i) a colony of cooperating individuals, (ii) an (artificial) pheromone trail for

local stigmergic communication, (iii) a sequence of local moves to find shortest paths,

and (iv) a stochastic decision policy using local information and no lookahead. [Dorigo et

al., 1999]

Colony of cooperating individuals: As with real ant colonies, ant algorithms

comprise of a population, or colony, of concurrent and asynchronous entities co-

operating at a global level in order to find an optimal solution to the problem at hand.

Although each individual ant is given the task of building a feasible solution, high quality

solutions are the result of the cooperation of the individuals among the entire population.

Ants within the population communicate among themselves by means of the information

they read/write on the problems states that they visit. This is explained in more detail in

the next item.

Pheromone trail and stigmergy: As real ants move about, they deposit a chemical

substance, called pheromone, on the areas they visit. Artificial ants also modify some

aspects of their environments, by mimicking this pheromone depositing behaviour. This

artificial pheromone trail changes some numeric information locally stored in the

problem’s state they visit. This information takes into account the ant’s current

history/performance and can be read/written by any ant accessing the state. In ACO

algorithms local pheromone trails are the only communication channels among the ants.

This stigmergic form of communication plays a major role in the utilization of collective

knowledge. Its main effect is to change the way the environment (the problem landscape)

is locally perceived by the ants as a function of all the past history of the whole ant

colony. Just as real pheromone would evaporate over time, so do the artificial

pheromone trails. Pheromone evaporation helps prevent the ant colony from prematurely

converging upon a sub optimal solution, by allowing the ant colony to slowly forget it’s

past history so that it can search in new directions without being over-constrained by its

past decisions.

Shortest path searching and local moves: Both real and artificial ants share a

common goal; to find a shortest (minimum cost) path from an origin to a destination (nest

to food source). Both ant types move simply through their environment, going from one

problem state to another, adjacent problem state. These steps and states are, of course,

problem specific.

As was mentioned above, in order to help increase the efficiency and

effectiveness of the solution, artificial ants have some characteristics that are not apparent

in their real ant counterparts. [Dorigo et al., 1999]

- Artificial ants live in a discrete world, and their moves are from one discrete state

to the next.

- Artificial ants keep an internal state, which contains the memory of the ant’s past

actions.

- Artificial ants deposit an amount of pheromone that is a function of the quality of

solution found.

- Artificial ants timing in laying pheromone is problem dependent. In some cases,

pheromone trails are only updated after a solution is found. In other cases, high

quality solutions are given pheromone reinforcement.

- To help improve overall efficiency, ACO algorithms can also be enriched by

ideas such as lookahead, local optimization, anti-pheromone, and accumulated

experience.

2.3 ACO Meta-Heuristic

Within each step of the ACO process, each ant adds some element to its solution

until a termination condition has been met. How an ant chooses the next element is

problem specific, but is built upon some common ground work. Each ant maintains a

tabu list, J, of elements already contained in its solution. This guarantees that the ant will

not add the same element twice. In order to select the next element, a probabilistic model

is used. The transition probability for the ant to the next element is defined in Figure 1

below. The visibility (ηij ) of element i from j is defined as the inverse of the Euclidian

distance between the two elements, and τij(t) is the amount of pheromone on the edge

between elements i and j.

p ijk ( t ) =

[τ ij ( t )] α ⋅ [ η ij ] β

[τ il ( t )] α ⋅ [η il ] β

l∈ J ik

∑

if j belongs to Jik, and 0 otherwise

Figure 1: Transistion probability formula

The probability can differ for two ants on element i, as it depends on the partial solution

made by ant k (i.e., the tabu list of the ant).

After completing a tour, each ant k deposits ∆τijk(t) on each edge (i, j) it traversed

in compiling its solution. Τhis amount depends on how well the ant k performed,

according to the formula in Figure 2.

∆τijk(t) = Q / Lk(t)

if (i, j) belongs toTκ(t), and 0 otherwise.

Figure 2: Amount of pheromone to deposit

Here Tk(t) is the tour of ant k at iteration t, and parameter Q is set to a problem specific

value. To avoid stagnation, pheromone decays with coefficient ρ, 0 ≤ ρ < 1 (ρ must be <

1 to prevent unlimited pheromone accumulation). The pheromone update rule, which is

applied to all edges, is seen in Figure 3 below. At time t = 0, there is a small constant

amount of pheromone on all edges.

τij(t) = (1 - ρ) ∗ τij(t) + ∆ τij(t),

where ∆ τij(t) = ∑κ=1m ∆ τijk(t), and there are m ants.

Figure 3: Pheromone update rule These pieces are comnined together to create the overall ACO meta-heuristic, which is

summarized in Figure 4.

procedure ACO_meta-heuristic() while(termination condition not satisfied) schedule_activities ant_generation_and_activity() pheromone_evaporation() deposit_pheromone() end schedule activities end while end procedure procedure ant_generation_and_activity() for 1 -> numAnts create new ant ant_activity()

end for end procedure procedure ant_activity() while (current state != target state) compute_transition_probabilities() apply_decision_policy() move_to_next_state() end while evaluate_solution() clear_tabu_list() end procedure

Figure 4: ACO meta-heuristic pseudocode

(Adapted from [Brander and Sinclair, 1995])

3. The Network Synthesis Problem

The design of an efficient telecommunications network is a complicated task.

Typical problem domains include topology, connectivity, and routing requirements. The

measurement of efficiency is typically based on one or more of these facets. There has

been much work done on the design of networks, and comparing several meta-heuristic

design techniques. [Cantarella et al., 2002][Myslek and Karas, 2002] These types of

heuristics generally look at ideas such as Concave Link Elimination (CLE) and other

types of network reduction methods. It is these types of ideas that have been extended to

the use of artificial intelligence to network design. [White et al., 1999] [Berry et al.,

1999]

From this type of work stemmed the inspiration for the use of artificial

intelligence to routing in networks. This started with Genetic Algorithms (GAs) and has

continued on into the use of ant colonies. [Singh et al., 2003][Varela and Sinclair, 1999]

The problem of network synthesis is a common one in the design of

telecommunications networks. The problem aims to satisfy pre-defined traffic

requirements between sets of nodes at a minimum cost. The problem space for network

synthesis is large, as in a network of n nodes, there are 2n(n-1)/2 possible topologies. Thus,

for a network as small as ten nodes, there are trillions of possibilities. In order to assess

the problem, the following information is required: the traffic demand between each

origin-destination (O-D) pair, and a cost function for carrying traffic on each possible

link between nodes i and j.

There has been recent interest in the problem both in the application of GAs

[Berry et al., 1997] in which a GA is used to synthesize the network while a linear

programme (LP) is used to allocate the traffic amongst the routes of the network. There

has also been some work done in applying the ACO meta-heuristic to the problem.

[Randell and Tonks, 2001]

4. Application of ACO to the Network Synthesis Problem

In this implementation, the notion of pre-computed traffic routes is employed.

This idea was first proposed in a paper on a simulated annealing approach to network

design. [Randell et al., 1999] A route is a sequence of nodes between an origin and a

destination node, and can be limited by a hop limit. A hop is considered an edge between

any nodes i and j. Figure 5 shows a simple five-node network that has 4 routes. The pre-

computed route strategy has several advantages over a hop-by-hop approach. For

instance, hop limits are automatically satisfied and routes are always feasible.

Figure 5: An Example Network The set of possible routes in a fully connected network is quite large, and

therefore techniques have been developed for the creation of smaller subsets. The

technique used in this approach was to select the subset of k-shortest paths (KSP). There

have been many algorithms proposed for the implementation of this problem [Yen,

1971][Lawler, 1972]. In Yens algorithm the shortest path (containing p ≤ n nodes) is

found using a standard single shortest-path algorithm (i.e., Dijkstra's algorithm) and

placed into a results list (Yen's list A). Then every node in the shortest path, except the

terminating node, is taken and another shortest path (spur) is calculated from each

selected node to the terminating node. For each such node, the path from the start node to

The routes are: (1, 3, 5) (2, 1) (2, 3, 4) (4, 1)

1

2

35

4

the current node is the root path. Two restrictions are placed on the spur path: 1) It must

not pass through any node on the root path (i.e. it must be loopless), and 2) It must not

branch from the current node on any edge used by a previously found k-shortest path

with the same root. Node and edge marking is used to prevent the spur paths from

looping or simply following the route of a previous k-shortest path. If a new spur path is

found it is appended to the root path for that node, to form a complete path from origin to

destination node, which is then a candidate for the next KSP. All such paths are stored

(Yen's list B) and the shortest remaining unselected path is selected as the next KSP, and

transferred to the results list (Yen's list A). The same process is repeated, calculating a

spur path from each node in each new KSP, until the required number of KSP have been

found [Brander and Sinclair, 1995].

The algorithm developed for this implementation differs from above in that it

maintains edge-disjointness amoung all routes. That is, no two routes can share a

common edge. In the five and ten node networks that were used in testing this

implementation, this also imposed a hop limit of two. Loopless path are also gauranteed

using this algorithm, which is outlined in Figure 6 below.

procedure k-shortest paths() initialize graph for 1 -> k do

result <- dijkstra() //to find a single shortest path remove edges used in result from graph

end for end procedure procedure dijkstra() Q = Vertices of graph while Q is not empty do u <- extractMin(Q)

S <- S U {u} for each neighbour v of u do if distance to u through v is shorter than known distance updateDistance()

end for end while end procedure

Figure 6: K-shortest paths algorithm

The computational complexity of the above algorithm is O(kn3), where k is the required

number of shortest paths between every origin-destination pair, and O(n2) is due to the

shortest-path calculation. By removing edges at each iteration, the algorithmic speed and

complexity are improved upon as the algorithm progresses, due to the smaller number of

neighbours that are caluculated during Dijkstra's algorithm. The known computational

complexity for Yen's algorithm is also O(kn3).

The network synthesis problem now becomes one of progessively building up a

set of routes chosen from the smaller subset, until all the traffic requirements have been

met. In terms of ant behaviour, this means that at each step of an iteration, an ant will

select another route from the pre-computed route set and add it to its network. The

selection of the next route is done using the transistion probability function in Figure 1.

In this implementation, the visibility is the inverse of the sum of the hop distances. An

ant computes the transition probability for each route that satisfies the remaining origin-

destination pair requirements not in the ants tabu list. It then generates a random number

and chooses the next route, adding it to its tabu list. It terminates when a route to satisfy

every origin-destination pair has been choosen. Pheromone updates are done using an

inverse ranking scheme. The ants with the top e solutions lay down pheromone on the

routes used in their solutions, weighted by a factor of (e - rank + 1). So, if the value of e

were set to five, the top network would be given a weighting factor of five, the second

best solution a factor of four, the third a factor of 3, and so on.

The fitness of the solution is based upon the length of the route and the bandwidth

required on each hop. Bandwidth is allocated in discrete intervals of five, in the effort to

make sharing hops more advantageous, in terms of generating a smaller overall fitness.

In this sense, if one route (r1) required three units of bandwidth, 5 must be allocated.

Therefore, it would be advantageous for the network if another route that uses one or two

units shared hops with r1, as it must already allocate five. The algorithm to determine the

fitness of an ant's network is outlined in Figure 7.

procedure evaluate_fitness()

Vector v for i = 1 -> numRoutes do Route r <- ants.routeAt[i] for j = 1 -> numHopsInRoute do Hop h = r.hopAt[j] if v contains r add r.bandwidthRequired to v's entry else v.add(r) end for end for d = 0 f = 0 while v has more elements do Hop t = v.removeNext() round t.bandwidthRequired up to nearest 5 d += t.distance f += t.bandwidthRequired end while networkLength = d networkFitness = f

end procedure

Figure 7: Network fitness algorithm

The running time of the above algorithm is O(NR * H + NH), where NR is the number of

required origin-destination pair routes, H is the hop limit, and NH is the total number of

hops used in the network. The above algorithm keeps track of all the hops used in the

network, and computes values based on the hops, not the individual routes, in order to

avoid counting the same hop multiple times.

The entire application of ACO to the network synthesis problem is described in

Figure 8. The createDistanceTable() function simply computes the distances between

every pair of nodes and stores it in a lower triangular matrix. Triangular matrices were

used wherever possible to decrease the amount of memory needed to store information.

This was not a major concern for the networks used in testing, but would become more

prevalent as network size grew.

procedure AntSynNet() createDistanceTable() makeRoutes() initializePheromone() for 1 -> MAX_ITERATIONS do runIteration() end for end procedure procedure makeRoutes() for 1 -> numRoutesRequired do k-shortest_paths() end for end procedure procedure runIteration() for a = 1 -> numAnts do makeNetwork(a) evaluate_fitness() end for

evaporatePheromone() for a = 1 -> e do updatePheromone(a) end for

clearTabuLists() end procedure

Figure 8: The AntSynNet algorithm The function initialzePheromone() deposits a small amount of pheromone on every route.

In makeNetwork(a), the ant a chooses a route based on the transistion probability until

one for every required origin-destination pair has been added to its network.

5. Testing

5. 1 Control Parameters

In order to come up with appropriate control parameters for the testing, a

combination of known ideas and test results using small five-node networks. These

networks and test results are given in Appendix 1. While almost all parameters lead to

the same best fitness values, the parameters that lead to the best results on the average

were selected.

The number of ants was set to the same number as the number of nodes in the

network. Too many ants can lead to premature convergence upon a sub-optimal solution,

while too few ants creates a colony with little or no cooperation because of the

pheromone decay. The ants are initially placed on a randomly selected route. That is,

one route is chosen randomly from the pool of routes belonging to a randomly selected

origin-destination pair. This allows the transition probabilities of two ants to differ as

their partial solutions thus far (their tabu lists) will differ. The tabu lists keep an ant from

selecting another route out of the pool of k-routes for an origin-destination pair they have

already selected from.

The initial amount of pheromone deposited upon all routes was 10-6. Using a

small value here allows for good solutions to grow more slowly, allowing for greater

exploration of the search space. The variable Q (found in the formula in Figure 2) is set

to such a value that the best solution is given a 1% - 10% increase from the amount of

initial pheromone. For the five-node networks tests, this value was also 10-6. Keeping

this update value relatively small is important as it helps to prevent premature

convergence, and allows pheromone values to build slowly over successive iterations.

For similar reasons of preventing premature converge upon sub-optimal solutions p, the

pheromone evaporation rate (found in the formula in Figure 3) is set to a low value of

0.2.

The values for α and β (found in the equation in Figure 1) are also important, as

they play a large factor in the transition probability function. If α is set to a low value,

the equation becomes stochastically greedy, and shorter routes are likely to be selected.

If β is set to a low value the greater the attraction to pheromone levels, and the more

likelihood of rapid convergence upon sub-optimal solutions. A balance between these

two extremes must be found. During testing of the five-node networks, it was found that

a ratio of two to one for α to β yielded encouraging results, and so α was set to four and β

was set to two.

In order to test these chosen parameters for validity, a five-node network and

corresponding traffic matrix where constructed with specific link goals in mind. The

traffic matrix was designed so that routing over shared hops would be advantageous. The

traffic matrix and resulting matrix can be found in Appendix 2. The results of this

validity test were as desired, converging upon the optimal solution over 95% of the time.

5.2 Testing Results

Testing was done by running three different traffic matrices upon four different

ten-node network configurations. These traffic matrices are shown in Appendix 3, along

with examples of the networks and testing results. Each network had 10 trials run with

each traffic matrix. Unfortunately, there are no similar tests against which the results

could be compared. However, the significance of the results will be discussed below.

Test results showed a fairly wide variety of result networks. As expected,

differing traffic demands give rise to different "efficient" networks. The traffic matrices

used in these tests were all fairly high demand, having a range of 90% - 100% traffic

requirements between all nodes.

Results were varying within the same test groups, suggesting that the pheromone

feedback may have been causing premature convergence upon sub-optimal solutions.

This may also have been a result of the relatively small network size. A possible solution

to this, apart from simply increasing the test network size, would be to add a facility

whereby an ant can choose a route randomly from time to time (approximately 5% - 10%

of the time) in order to allow exploration to continue even in situations where a network

may have already emerged.

Also clearly seen from the results is that the shortest length network is not always

the network with the most desirable fitness. This is not necessarily a poor result, but in

actual telecommunication design, there is generally a cost associated with simply lying

down a physical cable (represented as an edge on the graphs). Therefore, one solution

would be to associate an edge penalty into the fitness function, whereby networks with

fewer edges (and therefore, smaller overall length) are rewarded over longer networks.

Some early testing with this idea shows promising results, as some improved network

designs were found. This also further encourages the notion of sharing edges. Another

possible solution would be to allocate bandwidth in intervals that are larger than units of

five.

Overall, test results were encouraging. They have shown that the ACO can be

applied to the Network Synthesis Problem effectively. This application has also lead to

some ideas of how to further extend the idea.

5.3 Future Work

Along with some of the suggestions made in the previous section, there are some

other improvements that could be made to AntSynNet. Firstly, the implementation of

another KSP algorithm may prove beneficial. This change could lead to improvements in

the computational complexity, as well as provide a more robust pool of paths, by not

maintaining a strict edge-disjointness property. In addition, other route selection

techniques could be used instead of, or in conjunction with KSP, in an effort to find a

more useful subset of routes.

There could also be the addition of more control parameters, such as hop count,

node degree limits, edge capacity restraints, and node capacity restraints. All of these

additions would correspond to issues that would need to be addressed in the design of an

actual physical network. Unfortunately, they were outside the scope and timeframe of

this project.

These types of additions would then also make it possible to compare results with

other work done on the network synthesis problem, using various other solution

techniques. [Minoux, M., 1989][Randell et al., 2002]


10 1969 21030 | | Average 1866.00 21047.50 | Minimum 1585 20970 Maximum 2301 21275 Example Network:



(Similar results held for other runs of test5-3, so those results will not be shown here.)

Traffic Matrices Used: 1) 0 3 9 5 8 10 5 1 6 7 3 0 5 4 3 6 0 4 6 0 9 5 0 8 10 6 1 1 2 0 5 4 8 0 10 0 9 0 4 7 8 3 10 10 0 8 1 0 9 2 10 6 6 0 8 0 1 4 10 9 5 0 1 9 1 1 0 0 6 9 1 4 1 0 0 4 0 0 10 9 6 6 2 4 9 10 6 10 0 9 7 0 0 7 2 9 9 9 9 0 2) 0 3 9 5 8 10 5 1 6 7 3 0 5 4 3 6 0 4 6 0 9 5 0 8 10 6 1 1 2 0 5 4 8 0 10 0 9 0 4 7 8 3 10 10 0 8 1 0 9 2 10 6 6 0 8 0 1 4 10 9 5 0 1 9 1 1 0 0 6 9 1 4 1 0 0 4 0 0 10 9 6 6 2 4 9 10 6 10 0 9 7 0 0 7 2 9 9 9 9 0 3) 0 6 10 4 10 2 7 3 4 4 6 0 1 8 8 6 8 10 10 7 1 5 0 10 7 7 9 4 3 9 10 6 1 0 7 10 3 7 5 1 9 8 5 7 0 10 1 7 8 7 2 7 10 4 3 0 7 4 5 10 10 8 8 2 7 5 0 10 5 9 10 1 6 2 10 2 7 0 1 7 6 1 4 4 6 3 7 6 0 9 9 4 3 8 6 8 6 9 9 0

antsynnet: applying ant colony optimization to the network...

Documents