Genetic Programming

Part 2

Jay Shaffstall

Genetic Programming

Review of Part 1

Program graphing

Program breeding

Program mutation

Fitness functions

Development life cycle

Real world applications

Conclusion

Review of Part 1

Genetic programming takes advantage of the power of evolution to grow a program

The fitness function determines which programs from any generation will survive to breed

The best programs out of every generation create the next generation

Review of Part 1

After millions of generations, we end up with a program that can solve the problem

But how does it all work?

Program Graphing

In order to breed programs, we have to represent them in some fashion. Genetic programming represents a program as a graph.

Programs are represented using a lisp like syntax, with the operation coming before the arguments

Program Graphing

Program Breeding

So we have two programs represented as program graphs. How do these two programs combine to form children?We randomly choose a node in each program. We delete the subtree from the first program and replace it with the subtree from the second program

Program Breeding -- Parents

Program Breeding -- Children

Program Mutation

Breeding is not the only way to change a program. We can also mutate a program. The program is changed in a random way, and the change will be passed on to any children.

Program Mutation

Types of mutationsReplace a random subtree with another randomly generated program

Duplicate a subroutine, and randomly half the calls to use the new subroutine

Duplicate arguments to a subroutine, and randomly change half the references to use the new argument

Program Mutation

Types of mutations (continued)Create a new subroutine

Delete an existing subroutine

Delete an existing argument

Fitness Functions

Eventually, between breeding and mutation, we can end up with some complex programs.

So how do we pick the ones that live and the ones that die?

Fitness Functions

The basic purpose of a fitness function is to determine the distance between one program’s solution and the ideal solution.

This allows us to pick the “closest” solutions for breeding.

Fitness Functions

An easy example is a program that must calculate the square of a number.

The fitness function can be something like this:

square (x) – program (x)

Fitness Functions

Those programs that score the lowest in the fitness function are those that are closest to the ideal solution.

Those are the programs that we choose to live and breed.

Fitness Functions

So how do we develop a fitness function for a complex problem, without running into the automated oracle problem?

For example, the problem of writing an order entry system?

Fitness Functions

The simple answer is that we cannot reliably create a fitness function for complex problems.

Does this mean that genetic programming is useless for more complex problems?

Not necessarily.

Fitness Functions

One possibility is to break the problem down into simpler modules that can be represented by a less complex fitness function.

We then evolve all the modules, which will work together to create the final product.

Fitness Functions

So this approach would be a combination of designing the system by a human, and evolving the pieces of the system by a computer.

Development Life Cycle

At first glance, genetic programming seems to require a waterfall method of development.

We’ve all seen that waterfall development only works for simple projects.

What life cycle do we use for complex projects using genetic programming?

Development Life Cycle

The best fit seems to be an iterative cycle, where a human designs the architecture down to simple modules that are then evolved to meet requirements.

As requirements change, only those modules that are affected need to be re-evolved.

Development Life Cycle

Since the modules are relatively simple, re-evolving them will not take as much time as re-evolving the entire program.

We also reduce the chance of getting unexpected features in the program, since the user-visible features will be specified in the human created design of the modules.

Development Life Cycle

In this type of life cycle, we use genetic programming as an entry level programmer, leaving humans free to do the higher level design work.

Real World Applications

So, how closely can genetic programming approach the results of a program written by a human?

John Koza, regarded as the inventor of genetic programming, took a number of problems in circuit design and evolved programs to see how competitive genetic programming was.

Real World Applications

The hardware he used to do this was a Beowulf style cluster of 1,000 Pentium II processors.

Clearly, for anything more than trivial problems, significant computing power is needed.

Real World Applications

What Koza found is that genetic programming does indeed produce results competitive with human designed circuits.

In fact, in several cases genetic programming produced results that were better than human created results

Real World Applications

Unfortunately, several of the genetically programmed designs also infringed on exiting U.S. patents.

This has led Koza to the idea that genetic programming can be used as an invention creator. Give it a problem you would like to solve, but have no idea how, and let it run.

Conclusion

We’ve seen how genetic programming works, and how it can produce competitive results for well defined problems.

Most of our problems in the business arena are not well defined, but I’ve suggested a way in which genetic programming could be a useful tool for implementing well defined pieces of a program.

Conclusion

While I don’t expect business to start using genetic programming, after doing the research for this presentation, I think it could very well have a place in a business environment.

ReferencesKoza, John, et al. (1999) Genetic Programming III Morgan Kaufmann PublishersJohn Koza's Home Page, includes basic information on genetic programming, available on the web at http://www.genetic-programming.com/johnkoza.html Koza, John, et al Human Competitive Machine Intelligence available on the web at http://www.genetic-programming.com/humancompetitive.html 

ReferencesQureshi, Adil GPsys 2b: Java Genetic Programmin System available on the web at http://www.cs.ucl.ac.uk/staff/A.Qureshi/gpsys.html Genetic Programming Conference homepage, available on the web at http://www.genetic-programming.org/Genetic Programming Tutorial, available on the web at http://www.geneticprogramming.com/Tutorial/

References

Genetic Programming, available on the web at http://www2.informatik.uni-erlangen.de/~jacob/Evolvica/GP/Java/html/genprog/gp.html

Fundamentals in Genetic Programming, available on the web at http://www.dd.chalmers.se/~f96edfa/