lecture 2: unconstrained optimization definitions and conditionsychen/543t/notes/lecture 2.pdf ·...

CSE543T: Algorithms for Nonlinear Optimization

Lecture 2: Unconstrained Optimization

– Definitions and Conditions

Unconstrained Optimization

Minimize f(x)where x is defined in X

What do we know? (Assumptions)

• Minimize f(x) but f(x) is:– Unknown

• Deterministic• Stochastic

– Known• No closed form• Closed form• Closed form gradient (1st order, 2nd order, …)

What do we know?

• Minimize f(x) but x is:– Continuous– Discrete– Mixed – Categorical

A blackbox problem

• Find x that minimizes E(x)

– E(x) is a blackbox

– x is defined over a finite discrete set X

– there are n elements in X

• Are there more efficient ways than simple

enumeration?

– Deterministic vs. expected running time

Noise

Highly nonlinear function

Nonlinear function

Simulated Annealing

Simulated Annealing

Simulated Annealing

Expected complexity

of SA depends on the

smoothness of the

objective function

Stochastic search algorithms

• Simulated annealing, genetic algorithms, ant colony algorithms, controlled random walk, etc.

• Much better performance than random sampling– Why?

• “slow” but valuable – Achieves global optimality– Very relaxed assumptions

Introduction to Genetic Algorithms 1

Introduction to Genetic Algorithms

Assaf Zaritsky

Ben-Gurion University, Israel

www.cs.bgu.ac.il/~assafza

Introduction to Genetic Algorithms 2

Genetic Algorithms (GA) OVERVIEW

A class of probabilistic optimization algorithms Inspired by the biological evolution process Uses concepts of “Natural Selection” and “Genetic Inheritance” (Darwin 1859) Originally developed by John Holland (1975)

Introduction to Genetic Algorithms 3

GA overview (cont)

Particularly well suited for hard problems where little is known about the underlying search space Widely-used in business, science and engineering

Introduction to Genetic Algorithms 4

Classes of Search Techniques Search Techniqes

Calculus Base Techniqes

Guided random search techniqes

Enumerative Techniqes

BFS DFS Dynamic Programming

Tabu Search Hill Climbing

Simulated Anealing

Evolutionary Algorithms

Genetic Programming

Genetic Algorithms

Fibonacci Sort

Introduction to Genetic Algorithms 5

A genetic algorithm maintains a population of candidate solutions for the problem at hand, and makes it evolve by iteratively applying a set of stochastic operators

Introduction to Genetic Algorithms 6

Stochastic operators

Selection replicates the most successful solutions found in a population at a rate proportional to their relative quality Recombination decomposes two distinct solutions and then randomly mixes their parts to form novel solutions Mutation randomly perturbs a candidate solution

Introduction to Genetic Algorithms 7

The Metaphor

Nature Genetic Algorithm

Environment Optimization problem

Individuals living in that environment

Feasible solutions

Individual’s degree of adaptation to its surrounding environment

Solutions quality (fitness function)

Introduction to Genetic Algorithms 8

The Metaphor (cont)

Nature Genetic Algorithm

A population of organisms (species)

A set of feasible solutions

Selection, recombination and mutation in nature’s evolutionary process

Stochastic operators

Evolution of populations to suit their environment

Iteratively applying a set of stochastic operators on a set of feasible solutions

Introduction to Genetic Algorithms 9

The Metaphor (cont)

The computer model introduces simplifications (relative to the real biological mechanisms),

BUT

surprisingly complex and interesting structures have emerged out of evolutionary algorithms

Introduction to Genetic Algorithms 10

Simple Genetic Algorithm produce an initial population of individuals

evaluate the fitness of all individuals

while termination condition not met do

select fitter individuals for reproduction

recombine between individuals

mutate individuals

evaluate the fitness of the modified individuals

generate a new population

End while

Introduction to Genetic Algorithms 11

The Evolutionary Cycle

selection

population evaluation

modification

discard

deleted members

parents

modified offspring

evaluated offspring initiate & evaluate

Introduction to Genetic Algorithms 12

Introduction to Genetic Algorithms 13

Example: the MAXONE problem

Suppose we want to maximize the number of ones in a string of l binary digits

Is it a trivial problem?

It may seem so because we know the answer in advance

However, we can think of it as maximizing the number of correct answers, each encoded by 1, to l yes/no difficult questions`

Introduction to Genetic Algorithms 14

Example (cont)

An individual is encoded (naturally) as a string of l binary digits The fitness f of a candidate solution to the MAXONE problem is the number of ones in its genetic code We start with a population of n random strings. Suppose that l = 10 and n = 6

Introduction to Genetic Algorithms 15

Example (initialization)

We toss a fair coin 60 times and get the following initial population: s1 = 1111010101 f (s1) = 7

s2 = 0111000101 f (s2) = 5

s3 = 1110110101 f (s3) = 7

s4 = 0100010011 f (s4) = 4

s5 = 1110111101 f (s5) = 8

s6 = 0100110000 f (s6) = 3

Introduction to Genetic Algorithms 16

Example (selection1)

Next we apply fitness proportionate selection with the roulette wheel method:

2 1 n

3

Area is Proportional to fitness value

Individual i will have a

probability to be chosen ∑

iif

if)(

)(

4

We repeat the extraction as many times as the number of individuals we need to have the same parent population size (6 in our case)

Introduction to Genetic Algorithms 17

Example (selection2)

Suppose that, after performing selection, we get the following population: s1` = 1111010101 (s1)

s2` = 1110110101 (s3)

s3` = 1110111101 (s5)

s4` = 0111000101 (s2)

s5` = 0100010011 (s4)

s6` = 1110111101 (s5)

Introduction to Genetic Algorithms 18

Example (crossover1)

Next we mate strings for crossover. For each couple we decide according to crossover probability (for instance 0.6) whether to actually perform crossover or not

Suppose that we decide to actually perform crossover only for couples (s1`, s2`) and (s5`, s6`). For each couple, we randomly extract a crossover point, for instance 2 for the first and 5 for the second

Introduction to Genetic Algorithms 19

Example (crossover2)

s1` = 1111010101 s2` = 1110110101

s5` = 0100010011 s6` = 1110111101

Before crossover:

After crossover: s1`` = 1110110101 s2`` = 1111010101

s5`` = 0100011101 s6`` = 1110110011

Introduction to Genetic Algorithms 20

Example (mutation1) The final step is to apply random mutation: for each bit that we are to copy to the new population we allow a small probability of error (for instance 0.1)

Before applying mutation:

s1`` = 1110110101

s2`` = 1111010101

s3`` = 1110111101

s4`` = 0111000101

s5`` = 0100011101

s6`` = 1110110011

Introduction to Genetic Algorithms 21

Example (mutation2)

After applying mutation:

s1``` = 1110100101 f (s1``` ) = 6

s2``` = 1111110100 f (s2``` ) = 7

s3``` = 1110101111 f (s3``` ) = 8

s4``` = 0111000101 f (s4``` ) = 5

s5``` = 0100011101 f (s5``` ) = 5

s6``` = 1110110001 f (s6``` ) = 6

Introduction to Genetic Algorithms 22

Example (end)

In one generation, the total population fitness changed from 34 to 37, thus improved by ~9%

At this point, we go through the same process all over again, until a stopping criterion is met

Introduction to Genetic Algorithms 23

Introduction to Genetic Algorithms 24

Components of a GA

A problem definition as input, and

Encoding principles (gene, chromosome) Initialization procedure (creation) Selection of parents (reproduction) Genetic operators (mutation, recombination) Evaluation function (environment) Termination condition

Introduction to Genetic Algorithms 25

Representation (encoding)

Possible individual’s encoding Bit strings (0101 ... 1100) Real numbers (43.2 -33.1 ... 0.0 89.2) Permutations of element (E11 E3 E7 ... E1 E15) Lists of rules (R1 R2 R3 ... R22 R23) Program elements (genetic programming) ... any data structure ...

Introduction to Genetic Algorithms 26

Representation (cont)

When choosing an encoding method rely on the following key ideas

Use a data structure as close as possible to the natural representation Write appropriate genetic operators as needed If possible, ensure that all genotypes correspond to feasible solutions If possible, ensure that genetic operators preserve feasibility

Introduction to Genetic Algorithms 27

Initialization

Start with a population of randomly generated individuals, or use - A previously saved population - A set of solutions provided by a human expert - A set of solutions provided by another heuristic algorithm

Introduction to Genetic Algorithms 28

Selection

Purpose: to focus the search in promising regions of the space Inspiration: Darwin’s “survival of the fittest” Trade-off between exploration and exploitation of the search space

Next we shall discuss possible selection methods

Introduction to Genetic Algorithms 29

Fitness Proportionate Selection

Derived by Holland as the optimal trade-off between exploration and exploitation

Drawbacks Different selection for f1(x) and f2(x) = f1(x) + c Superindividuals cause convergence (that may be premature)

Introduction to Genetic Algorithms 30

Linear Ranking Selection

Based on sorting of individuals by decreasing fitness

The probability to be extracted for the ith individual in the ranking is defined as

21,11)1(21)( ≤≤

−−

−−= βββni

nip

where β can be interpreted as the expected sampling rate of the best individual

Introduction to Genetic Algorithms 31

Local Tournament Selection

Extracts k individuals from the population with uniform probability (without re-insertion) and makes them play a “tournament”, where the probability for an individual to win is generally proportional to its fitness

Selection pressure is directly proportional to the number k of participants

Introduction to Genetic Algorithms 32

Recombination (Crossover)

* Enables the evolutionary process to move toward promising regions of the search space

* Matches good parents’ sub-solutions to construct better offspring

Introduction to Genetic Algorithms 33

Mutation

Purpose: to simulate the effect of errors that happen with low probability during duplication

Results: - Movement in the search space - Restoration of lost information to the population

Introduction to Genetic Algorithms 34

Evaluation (fitness function)

Solution is only as good as the evaluation function; choosing a good one is often the hardest part Similar-encoded solutions should have a similar fitness

Introduction to Genetic Algorithms 35

Termination condition

Examples:

A pre-determined number of generations or time has elapsed A satisfactory solution has been achieved No improvement in solution quality has taken place for a pre-determined number of generations

Introduction to Genetic Algorithms 36

What is your solution?

• Consider – Min f(x), x discrete, f(x) is a blackbox– Min f(x), x continuous, f(x) is a blackbox– Min f(x), x discrete, order of f(x) is known– Min f(x), x continuous, order of f(x) is known– Min f(x), x discrete, closed form f(x) is known– Min f(x), x continuous, closed form f(x) is known

[when does discrete/continuous make difference?]

LOCAL AND GLOBAL MINIMA

f(x)

x

Strict LocalMinimum

Local Minima Strict GlobalMinimum

Unconstrained local and global minima in one dimension.

Why do we look for local optima

• Sometimes, – local optimum == global optimum

• If you can enumerate all the local optima, you can find the global optima

NECESSARY CONDITIONS FOR A LOCAL MIN

• 1st order condition: Zero slope at a localminimum x∗

∇f(x∗) = 0

• 2nd order condition: Nonnegative curvatureat a local minimum x∗

∇2f(x∗) : Positive Semidefinite

• There may exist points that satisfy the 1st and2nd order conditions but are not local minima

x xx

f(x) = |x|3 (convex) f(x) = x3 f(x) = - |x|3

x* = 0 x* = 0x* = 0

First and second order necessary optimality conditions for

functions of one variable.

Why do we need optimality conditions

• Necessary vs. sufficient• Ways that do not work

– Enumerate and check– Enumerate and compare– Solving ∂f = 0 is nontrivial

• Provide guidance for design iterative search algorithms– Convergence– Conditions for the validity of points

PROOFS OF NECESSARY CONDITIONS

• 1st order condition ∇f(x∗) = 0. Fix d ∈ �n.Then (since x∗ is a local min), from 1st order Taylor

d′∇f(x∗) = limα↓0

f(x∗ + αd) − f(x∗)α

≥ 0,

Replace d with −d, to obtain

d′∇f(x∗) = 0, ∀ d ∈ �n

• 2nd order condition ∇2f(x∗) ≥ 0. From 2ndorder Taylor

f(x∗+αd)−f(x∗) = α∇f(x∗)′d+α2

2d′∇2f(x∗)d+o(α2)

Since ∇f(x∗) = 0 and x∗ is local min, there issufficiently small ε > 0 such that for all α ∈ (0, ε),

0 ≤ f(x∗ + αd) − f(x∗)α2

= 12d

′∇2f(x∗)d +o(α2)α2

Take the limit as α → 0.

CONVEXITY

Convex Sets Nonconvex Sets

x

y

αx + (1 - α)y, 0 < α < 1

x

x

y

y

xy

Convex and nonconvex sets.

αf(x) + (1 - α)f(y)

x y

C

z

f(z)

A convex function. Linear interpolation underestimates

the function.

MINIMA AND CONVEXITY

• Local minima are also global under convexity

αf(x*) + (1 - α)f(x)

x

f(αx* + (1- α)x)

x x*

f(x)

Illustration of why local minima of convex functions are

also global. Suppose that f is convex and that x∗ is a

local minimum of f . Let x be such that f(x) < f(x∗). By

convexity, for all α ∈ (0, 1),

f(αx∗ + (1 − α)x

)≤ αf(x∗) + (1 − α)f(x) < f(x∗).

Thus, f takes values strictly lower than f(x∗) on the line

segment connecting x∗ with x, and x∗ cannot be a local

minimum which is not global.

OTHER PROPERTIES OF CONVEX FUNCTIONS

• f is convex if and only if the linear approximationat a point x based on the gradient, underestimatesf :

f(z) ≥ f(x) + ∇f(x)′(z − x), ∀ z ∈ �n

f(z)f(z) + (z - x)'∇f(x)

x z

− Implication:

∇f(x∗) = 0 ⇒ x∗ is a global minimum

• f is convex if and only if ∇2f(x) is positivesemidefinite for all x

Convex Set and Convex Function