evolutionary computation optimization problems and random …szh/teaching/ec/lecture3... ·...

Evolutionary Computation

Evolutionary ComputationOptimization Problems and Random Search

Algorithms

Shan He

School for Computational ScienceUniversity of Birmingham

Modules 02411 and 22313: EC


What we learned and what we will learned

What we learned and what we will learned

I What we learned:I Randomised algorithmsI The important of randomness in algorithms

I What will we learn this week:I What is optimisationI How to solve optimisation problems using randomised

algorithms, local search and stochastic local search algorithms.I Evolutionary Computation ∈ Metaheuristics ∈ Heuristic

algorithms ∈ Search and enumeration algorithms


Outline

Outline of Topics

Optimisation

Optimisation problem examplesDiet problemTravelling salesman problemSolving TSP problems using heuristic algorithms

Randomised algorithms

Conclusion


Optimisation

What is optimisation

I Optimization: to find the best or optimal solution to aproblem

I Examples are everywhere:I Portfolio optimisation: you have an investment portfolio, e.g.,

cash and shares, you know the potential return and financialrisk of each asset. How to build the perfect investmentportfolio to maximise the return while keep risk in control?

I Other examples can be found in engineering, biology, medicine,etc.

I Things to consider when you have an optimisation problem:I Q: How to define ‘best’ or ‘optimal’ solution to the problem?I A: Formulate the problem as a mathematical model that a)

abstracts all the essential elements; and b) can evaluate thequality of the candidate solutions

I Q: Whether the solution is feasible?I A: We need to have some constraints on the solutions


Optimisation

Definition of optimisation

I Definition:I Given: a function f (x) : A → R from some set A to the real

numbersI Sought: an element x∗ in A such that f (x∗) ≤ f (x)

(“minimisation”) or f (x∗) ≥ f (x) (“maximisation”) for all x inA.

I function f (x) is called objective function, or cost function(minimisation), fitness function (maximisation and inEvolutionary Computation)

I A is called feasible set, which is some subset of the Euclideanspace specified by a set of constraints

I The domain A of f (x) is called the search space, while theelements of A, e.g., x ∈ A are called candidate solutions orfeasible solutions.


Optimisation

Categories of optimisation problems

I Depends on the nature of objective function:I Linear vs non-linearI If it is non-linear:

I Convex vs non-convex

I Multi-objective vs single objectiveI Constrained vs non-constrained

I Depends on the nature of solutions:I Continuous vs Discrete (also known as a combinatorial

optimization)


Optimisation

Algorithms for optimisation problems

I Mathematical programming algorithms, e.g., linearprogramming

I Search and enumeration algorithmsI Brute force algorithms, enumerating all possible candidate

solutions and checkI Improved brute force algorithms, e.g., branch and bound

algorithmsI Local search, e.g., greedy searchI Heuristic algorithms

I Randomised algorithms


Optimisation problem examples

Diet problem

Diet problem

I Diet problem: given a set of available foods, how to select aset foods to minimise the cost but also satisfy daily nutritionalrequirement?

I Studied in the 1930s and 1940s.I Motivated by USA Army’s desire to minimize the cost of

feeding soldiers while still providing a healthy diet

I Can be formulated as an linear programming problem.

I MS Excel provides a linear programming add-in, Excel solver.

I Here is an example how to solve Diet problem using ExcelSolver

http://office.microsoft.com/en-gb/excel-help/load-the-solver-add-in-HP010021570.aspx

http://switzernet.com/1/people/aram-gabrielyan/public/080221-excel-solver-diet-problem/

http://switzernet.com/1/people/aram-gabrielyan/public/080221-excel-solver-diet-problem/



Diet problem

Diet problem: example

Food Cost per serving Vitamin A Calories

Corn 0.35 107 72Milk 0.42 500 121Wheat Bread 0.25 0 65

I Objective Function: Minimise the total cost of the food

I Constraint Set 1: For each nutrient, e.g., Vitamin A, at leastmeet the minimum required level.

I Constraint Set 1: For each nutrient do not exceed themaximum allowable level.



Travelling salesman problem

TSP

I TSP: given a list of cities and the distances between each pairof them, how to find the shortest route that visits each cityexactly once and returns to the origin city?

I Mentioned in a travelling salesman handbook in 1832 [1]

I Formally defined by the Irish mathematician W. R. Hamilton

I One of the most prominent and widely studied combinatorialoptimisation problems in computer science and operationsresearch

I Conceptually simple but computationally difficult: bestbenchmark for development and evaluation of combinatorialoptimisation algorithms

[1] “Der Handlungsreisende wie er sein soll und was er zu tun hat, um Auftrge zu erhalten und eines glcklichenErfolgs in seinen Geschften gewi zu sein von einem alten Commis-Voyageur”




TSP example: 4 cities TSP

A

c

B

D

10

5

5

55

5

Optimal solution?




TSP exampleOptimal solution for “usa13509” from TSPLIB. Cities withpopulation at least 500 in the continental US (in 1995). In total13509 cities.

http://comopt.ifi.uni-heidelberg.de/software/TSPLIB95/



Solving TSP problems using heuristic algorithms

Why heuristic algorithms for TSP?

I Because it is difficult for other algorithms: shown to beNP-hard

I Brute force algorithms, e.g., try all permutations to find theshortest, the running time is O(n!), the factorial of the numberof cities.

I Improved brute force algorithms, e.g., branch and cutalgorithms, the running time could be O(1.9999n) [1]

I But still time consuming, e.g., for usa13509, the running timerequired is 213509

I In fact, usa13509 was solved in 1998 by Rice University usingtwo clusters of 44 CPUs, took approximately three monthsfrom start to finish (see News)

I The largest instance of TSP problem is an instance in TSPLIBof 85,900 cities, taking over 136 CPU-years [2]!

[1] Woeginger, G.J. (2003), ”Exact Algorithms for NP-Hard Problems: A Survey”, Combinatorial OptimizationEureka, You Shrink! Lecture notes in computer science, vol. 2570, Springer, pp. 185207. [2] Applegate, D. L.;Bixby, R. M.; Chvtal, V.; Cook, W. J. (2006), The Traveling Salesman Problem, ISBN 0-691-12993-2.

http://www.crpc.rice.edu/CRPC/newsArchive/techweb_6_29_98.html




Random search methods

I Essentially a kind of randomised algorithm, sometimes calledrandom sampling algorithm.

Generic random search algorithm

x0 := generate initial solutionterminationflag := falsex := x0while (terminationflag != true)

Randomly generate a candidate solution v ∈ AIf (f(v) < f(x)): x := vIf a termination criterion is met: terminationflag := true

Output x




Randomised search algoirthms

I Question: Monte Carlo or Les Vegas?

I How does it perform? Let’s take a look at a MATLABimplementation.

I What we learned?I Randomised search is NOT useful when the number of

acceptable solutions is just a small portion of the feasiblesolutions

I Randomised search might do well when there is no coherencein the search space, e.g., there is no sense about the searchspace.


Conclusion

Conclusion

I Optimisation problems can be very difficult

I Some of them are NP-hard: solution of NP-problems requires(in the worst case) an exhaustive search.

I TSP is known to be NP-hard so an optimal solution for evenmoderate size problem is intractable.

I Randomised search algorithm is not useful for most of theoptimisation problems

I Optimisation is all about exploration and exploitation

I Randomness can facilitate exploration, but not exploitation(local search)

I I will introduce local search and stochastic local searchalgorithm, e.g., simulated annealing on Thursday (23rd Jan.)

in LR5, Arts


Conclusion

Assignment: Diet optimisation

I Find out the daily recommended nutrient intake, and usefoods in your shopping list and their nutrient information toreduce the cost. You can also try reducing calories.

evolutionary computation optimization problems and random …szh/teaching/ec/lecture3... ·...

Documents