Computational ComplexityJang, HaYoung(hyjang@bi.snu.ac.kr)BioIntelligence Lab.

Algorithm AnalysisWhy Analysis?to predict the resources that the algorithm requires, such as computational time, memory, communication bandwidth, or logic gates.The running time of an algorithm is the number of primitives operations or step(machine independent) executed.

ComplextitySpace / MemoryTimeCount a particular operationCount number of stepsAsymptotic complexity

Time ComplexityWorst-casean upper bound on the running time for any inputAverage-caseWe shall assume that all inputs of a given size are equally likely.Best-caseto get the lower bound

Time ComplexitySequential search in a list of size nworst-case : n timesbest-case : 1 timesaverage-case :

Asymptotic NotationAsymptotic upper bound, we use -notation.For a given function g(n), we denote by (g(n)) the set of functions; (g(n)) = {f(n): there exist positive constants c and n0 such that f(n) = n0}.

Asymptotic Notation-notation provides an asymptotic lower bound.For a given function g(n), we denote by (g(n)) the set of functions (g(n)) = {f(n): there exist positive constants c and n0 such that f(n) >= cg(n) for all n >= n0}.

Asymptotic Notation(g(n)) = {f(n) : there exist positive constants c1, c2, and n0 such that c1g(n)

Asymptotic Notation

The sets (n2), (n2), and (n2)

Practical Complexities109 instructions/second computer

Impractical Complexities109 instructions/second computer

n

n4

n10

2n

1000

17min

3.2 x 1013

years

3.2 x 10283

years

10000

116

days

???

???

10^6

3 x 10^7

years

??????

??????

C:\sahni\cop3530\slides\table.rtf

Faster Computer Vs Better Algorithm

Algorithmic improvement more usefulthan hardware improvement.

E.g. 2n to n3

IntractabilityA polynomial-time algorithm is one whose worst- case time complexity is bounded above by a polynomial function of its input size.

example - worst-case time complexity - Polynomial-time : 2n, 3n3 + 4n, 5n + n10, n log n - Non polynomial-time : Intractable problem - No polynomial-time algorithm can solve it.W(n) (p(n))

Three Categories of Problems (1)1. Problems for which polynomial-time algorithms have been found2. Problems that have been proven to be intractable - The first type is problems that require a non-polynomial amount of output. e.g. Determining all Hamiltonian Circuits. - The second type of intractability occurs when our requests are reasonable and we can prove that the problem cannot be solved in polynomial time. e.g. Halting Problem, Presburger Arithmetic Problem

Three Categories of Problems (2) Presburger Arithmetic is the theory of integers with addition (Z,+,=,

The Sets P and NP (1)Definition P is the set of all decision problems that can be solved in polynomial-time.

A NP algorithm have two stages: 1. Guessing (in nondeterministic polynomial time) Stage 2. Verification (in deterministic polynomial time) StageDefinition NP is the set of all decision problems that can be solved in nondeterministic polynomial-time.

The Sets P and NP (2)

Genetic Algorithms

An Abstract View of GA generate initial population G(0); evaluate G(0); t := 0 ; repeat t := t + 1;generate G(t) using G(t-1);evaluate G(t); until termination condition has reached;

Search TechniquesSEARCH TECHNIQUESCalculus-basedtechniques.Guided RandomSearch TechniquesEnumerativeTechniquesDirectedMethodsIndirectedMethodsFibonacci NewtonSimulated EvolutionaryAnnealing AlgorithmsEvolutionary Genetic Algorithms strategiesDynamicPGMingParallel SequentialGAs GAsClasses of Search techniques

Simple Genetic Algo's components1. A mechanism to encode the solutions as binary strings2. A population of binary strings3. A fitness function4. Genetic operators5. Selection mechanism6. Control parameters

The GA CyclePopulation (chromosomes)Selection(mating pool )Evaluation(fitness)Genetic OperatorsMatesManipulationNew GenerationOffspring*Decoded stringsParents*Reproduction = Evaluation + Selection Crossover & Mutation

Fitness function (object function)The mechanism for evaluating each stringTo maintain uniformity over various problem domains, normalize the obj.function to 0 to 1.The normalized value of the obj. function = the fitness of the string

Selection Models nature's "survival-of-the-fittest " mechanismA fitter string receives higher number of offspring.

Proportionate selection scheme The roulette wheel selection scheme

After Selection, pairs of string are picked at randomIf string length = n,randomly choose a number from 1 to n - 1,then use it as a crossover point.GA invokes crossover ONLY IF a randomly generated no > pc .(pc = the crossover rate) Crossover

After crossover,string are subjected to mutation

Flipping bits : 0 1, 1 0

Mutation rate : Pm= probability that a bit will be flippedThe bits in a string are independently mutated.= Role : restoring lost genetic materialMutation

Function Definition Find x from the range [-1, 2] which maximizes the f

Analysis of function f

Representation (1)Representation of stringsix places after decimal pointThe range [-1,2] should be divided into at least 3000000 ranges.Binary representation : 22 bits

Mapping from a binary string into a real number xConvert the binary string from the base 2 to base 10:

Find a corresponding real number x:

Representation (2)String Example : string (1000101110110101000111)String00000000000000000000001111111111111111111111x-1.02.0

Initial PopulationCreate a population of strings, where each chromosome is a binary vector of 22 bits.All 22 bits for each string are initialized randomly.

Evaluation Functioneval(v) = f(x)For example,Stringsv1=(1000101110110101000111)v2=(0000001110000000010000)v3=(1110000000111111000101)xf(x)x1= 0.637197x2= -0.958973x3= 1.627888f(x1)= 1.586345f(x2)= 0.078878f(x3)= 2.250650The string v3 is the best of the three strings,since its evaluation returns the highest value.

Genetic Operators : MutationMutationMutation alters one or more genes with a probability equal to mutation rate.Mutation Example :

Genetic Operators : CrossoverCrossover Example :The crossover on v2 and v3Assume that the crossover point was randomly selected after 5th gene:

ParametersPopulation size = 50Probability of crossover = 0.25Probability of mutation = 0.01

Experimental resultsThe best chromosome after 150 generationsVmax = (1111001101000100000101)Xmax = 1.850773As expected, , and f(xmax) is slightly larger than 2.85.