massively parallel approximation algorithms for the ...ark/students/vrg5913/report.pdf3.2...

Massively Parallel Approximation Algorithms for

the Traveling Salesman Problem

Vaibhav Gandhi

May 14, 2015

Abstract

This paper introduces the reader to massively parallel approximationalgorithms which compute an optimal tour close to the best tour for atraveling salesman going through a set of cities. The algorithms mentionedin this paper take advantage of parallel computing. The paper comparesand analyzes the results of an exhaustive search against the massivelyparallel approximation algorithms.

1 Introduction

The traveling salesman problem (TSP) describes a salesman who needs to travelthrough a list of given cities exactly once by taking the shortest possible tour.The problem is NP-Complete since there are no known polynomial time algo-rithms to solve it.

The main motivation to solve this problem is that the applications of Travel-ing Salesman are immense. It can be used by banks to find an optimal route forupdating cash in ATMs, by delivery companies to chalk out the best possibleroute between the delivery centers and save up on fuel and man hours. It canbe used outside of delivery based applications too. For example it can be usedfor drilling holes in circuits.

The easiest way to solve the traveling salesman problem would be by usingbrute force search which is also called as exhaustive search. The algorithminvolves going through each possible ordered permutation of cities and findingout the cheapest route. The running time for this approach is O(n!) which isthe factorial of the number of cities. One can easily say that this kind of solutionis not feasible even for computing the route between 20 cities. This is a seriouslimitation.

Recent advances by vendors like Amazon, Google and Microsoft have madecloud services easily accessible to the public. One can easily harness the powerof the cloud to solve huge problems at a fraction of the cost in parallel by makinguse of multiple cores on multiple nodes. Traveling salesman is one such problemthat can be modified for such infrastructure.

1

This paper proposes two Massively Parallel Approximation algorithms whichcan be used to obtain approximate solutions close to the best solutions for thetraveling salesman problem.

The paper is broken down as follows: Section 2 provides a brief overview ofthe literature that was reviewed and that inspired the design of these algorithms.Section 3 provides description of the algorithms. Section 4 discusses the resultsof the experiments. Section 5 contains an analysis of the results and Section 6concludes the paper with final comments and future plans in this area.

2 Related Work

The following papers were reviewed and used a basis for designing the algorithmsproposed in this paper.

1. A parallel Tabu search algorithm for large traveling salesman problems [2]:Fiechter, the author, proposes the use of Tabu Search for global optimization.The algorithm starts of by generating a tour and then making modifications toit iteratively till we get the possible tour. The methods mentioned in the paperare adapted for parallel computing and used to find tours for up to 100,000cities.

2. Parallel Heuristics for TSP on MapReduce [3]: The authors of this paperpropose an adapted version of Neighbourhood Search and Tabu Search thatuses MapReduce to compute tours in parallel. The results of these 2 algorithmsare compared against a sequential algorithm and the authors conclude that thealgorithms are not well suited for MapReduce.

3. Ant Colony System: A Cooperative Learning Approach to the TravelingSalesman Problem[1]: The authors of this paper are intrigued by the way antsmove around and they adapt this knowledge to solve the traveling salesmanproblem.

4. New Parallel Randomized Algorithms for the Traveling Salesman Problem[5]: The authors demonstrate a new method based on randomized optimizationwhich partitions feasible regions into sub regions and manipulates the regionsbased on random sampling.

The techniques described further on in this paper take inspiration from theliterature reviewed but differs a lot from it. Concepts like Tabu search andRandomized optimizations were modified for these techniques. The central ideaof both of the techniques is to start with a tour and work on it to reach localminima whereas some of the papers reviewed usually worked by computing atour.

3 Algorithm Description

The algorithms mentioned below makes use of a seed to generate a graph ofcities in a 1000 x 1000 area. The seed is used to initialize the pseudo random

2

number generator. The random number generator randomly assigns each cityan x and y coordinate in this space.

The distance between any two cities is the Euclidean Distance which is thelength of the line segment connecting the two cities.

The cost of the tour or length of the route is computed by calculating theEuclidean Distance between subsequent cities.

3.1 Exhaustive Branch and Bound Search

Exhaustive search goes through each possible tour and finds the shortest possibleroute.

For the purpose of this research paper, a modified version of exhaustivesearch was implemented to work on a multi-core computer. Each core wouldcompute a different tour and the results from all the cores were combined to findthe shortest tour. Over all for a quad core computer, each core would compute(n!)/4 routes where n is the number of cities.

Despite executing the algorithm in parallel, the running time of the pro-gram was in exponential time. To overcome this, the exhaustive algorithm wasmodified to make use of branch and bound pruning. A starting city is known,which is the root node. The nodes of the tree are different cities. A branch is aroute from root to the leaf node. Cities do not repeat in a branch. Every timea branch is computed, a cost which is the length of the tour, is obtained. Thiscost is maintained globally. Every new branch computed is checked against thelowest cost of previous branches. This is called bounding. Any branch thatexceeds the bounds is pruned, i.e. the route is going to lead to a lengthier tourand thus is not worth computing. This results in fewer computations withoutaffecting the quality of the solution.

3.2 Approximation Algorithms

The algorithms mentioned below start by generating a tour and processing itto find the most optimal approximate tour.

3.2.1 Massively Parallel Stochastic Search

Stochastic Search is a modified Local Search. The problem of getting stuck inlocal minima in local search leads to not so optimal results. Stochastic Searchrandomizes the initialization step which leads to random solutions.

The stochastic algorithm randomly generates an initial tour using a combi-nation of user submitted seed and the iteration number. The cost of this tour iscomputed. The cities in this tour are randomly swapped and resulting in newroutes and after each swapping the cost is recomputed. This step is repeatedtill the cost of the current route is better than the cost of the previous route.The point where there is no improvement in the cost is the local minima. Thesteps listed above are repeated for a number of iterations and the lowest localminima from these iterations is the lowest cost route using this approach.

3

The program was modified to work on a multi-core computer. For a quadcore computer, each core would compute n/4 routes where n is the number ofiterations.

3.2.2 Massively Parallel Random Search

The Random Search algorithm randomly generates an initial tour using a com-bination of user submitted seed and the iteration number. The cost of this touris computed. This is repeated for a number of iterations and the lowest costfrom these iterations is the lowest cost route using this approach.

4 Results

A 16 core parallel computer was used for all the experiments mentioned in thispaper. Different systems were not used so as to compare the time complexityof both algorithms as well as the superiority of the results.

Parallel Java 2, a library developed by Prof. Alan Kaminsky [4] was used forhandling all the parallel communication, segmentation of iterations and man-agement of task between cores and aggregation of the results from the differentcores.

The results obtained from the Exhaustive Search are used as a benchmarkfor the Approximation Algorithms. Shown below in Table 1 is a copy of theresults for the exhaustive search for seed 456. Table 2 and Table 3 contain asnapshot of the data from Stochastic Search and Random Search for the sameseed.

Cities Time (ms) Cost

10 213 3068.73511 670 3136.6112 729 3158.14413 2721 3168.10214 17214 3289.77815 83966 3290.18416 400761 3315.76817 1687695 3329.47318 7445263 3329.512

Table 1: Exhaustive Cost values for Seed : 456

All the remaining graphical plots of the results are presented in AppendixA. The data for other seeds can be found in an accompanying cd drive.

4

Table 2: Stochastic and Random data for Seed : 456

5

Table 3: Continuation of Stochastic and Random data for Seed : 456

6

Figure 1: Plot of Cost Comparison for 10 iterations for Seed : 456Stochastic Search performs way better than Random Search as compared to

Exhaustive Search in terms of cost as expected for 10 iterations

Figure 2: Plot of Time Comparison for 10 iterations for Seed : 456Time required for Stochastic Search and Random Search is negligible (approx

10ms) for 10 iterations as compared to Exhaustive Search.

7

Figure 3: Plot of Cost Comparison for 10000 iterations for Seed : 456Stochastic Search performs close to Exhaustive Search in terms of cost but

Random Search is not good enough for 10000 iterations.

Figure 4: Plot of Time Comparison for 10000 iterations for Seed : 456Time required for Stochastic Search and Random Search is negligible (approx

100ms) for 10000 iterations as compared to Exhaustive Search.

8

Figure 5: Plot of Percentage Cost Difference for 10 cities for Seed : 456The percentage cost difference of Stochastic or Random Search vs Exhaustive

Search constantly decreases till it eventually reaches 0.

Figure 6: Plot of Percentage Cost Difference for 18 cities for Seed : 456The percentage cost difference of Stochastic or Random Search vs ExhaustiveSearch constantly decreases till it reaches a threshold after which there is no

improvement.

9

Figure 7: Plot of Percentage Cost Difference for Stochastic Search for Seed :456

Stochastic Search approaches cost obtained from Exhaustive Search.

Figure 8: Plot of Percentage Cost Difference for Random Search for Seed : 456For more number of cities, Random search performs poorly.

10

5 Analysis

Massively Parallel Stochastic Search works much better than Massively ParallelRandom Search as compared to the results from Branch and Bound ExhaustiveSearch. Stochastic Search comes to within 5% of the cost of the ExhaustiveSearch but in 0.05% time. Stochastic Search was expected to perform betterthan the Random Search and it did as shown in the results. This is becauseStochastic Search improves on the randomly generated tour in each iteration toreach a better local minima as compared to the Random Search.

As cities increase, Stochastic Search as well as Random Search require morenumber of iterations to reach the solution closest to the tour computed fromExhaustive Search. This is because the number of possibilities increase expo-nentially for each new city (O(n!) number of possibility where n is the numberof cities).

Both Random Search and Stochastic Search require only a fraction of thetime (0.05%) of Exhaustive Search despite the number of iterations becausethe algorithms do not compute the possibilities but rather process a randomlygenerated tour at each iteration which is very fast.

6 Future Work

As observed from the results, the Stochastic Search is able to evaluate thesame route as Exhaustive Search or in most cases computes a route with costapproaching less than 5% cost difference. This is achieved at a fraction of thetime of Exhaustive search. Random Search on the other hand does not get suchgood results. This was expected from the start.

In future, I want to try different approaches that would lead to better resultsin terms of cost and time. I also want to try the current algorithms on real datasourced from one of the maps to visualize the results.

References

[1] M. Dorigo and L. M. Gambardella. Ant colony system: a cooperative learn-ing approach to the traveling salesman problem. Evolutionary Computation,IEEE Transactions on, 1(1):53–66, 1997.

[2] C.-N. Fiechter. A parallel tabu search algorithm for large traveling salesmanproblems. Discrete Applied Mathematics, 51(3):243–267, 1994.

[3] S. Jain and M. Mallozzi. Parallel heuristics for tsp on mapreduce. BrownUniversity, 2010.

[4] A. Kaminsky. Parallel java 2 library, 2014.

[5] L. Shi, S. Olafsson, and N. Sun. New parallel randomized algorithms for thetraveling salesman problem. Computers & Operations Research, 26(4):371–394, 1999.

11

Appendix A : Graphical Plots

Figure 9: Plot of Cost Comparison for 10 iterations for Seed : 12345

12

Figure 10: Plot of Time Comparison for 10 iterations for Seed : 12345


13


Figure 13: Plot of Percentage Cost Difference for 10 cities for Seed : 12345

14


Figure 15: Plot of Percentage Cost Difference for Stochastic Search vs Exhaus-tive Search for Seed : 12345

15

Figure 16: Plot of Percentage Cost Difference for Random Search vs ExhaustiveSearch for Seed : 12345

16


17



18



19



20


21


22



23



24



25


26


27



28



29



30


31


32



33



34



35


36


37



38



39



40


41


42



43



44



45


46


47



48



49



50


51


52



53



54



55


56

massively parallel approximation algorithms for the ...ark/students/vrg5913/report.pdf3.2...

Documents