6/4/03genetic algorithm the genetic algorithm the research of robert axelrod the algorithm of john...
Post on 21-Dec-2015
214 Views
Preview:
TRANSCRIPT
6/4/03Genetic Algorithm
The Genetic Algorithm
The Research of Robert Axelrod
The Algorithm of John Holland
Reviewed by Eyal Allweil and Ami Blonder
6/4/03Genetic Algorithm
The Prisoner’s Dilemma
Player 1
Player 2
DefectCooperate
Defect:1
P(unishment)
1
0S(ucker)
5
Cooperate:5
T(emptation)
0
3R(eward)
3
6/4/03Genetic Algorithm
The Iterated Prisoner’s Dilemma (IPD)
• Two tournaments held by Robert Axelrod
• The highest average score was Anatol Rapoport’s Tit-For-Tat (TFT)
• Eight representatives explain 98% of the variance in the tournament results.
• We will also hear criticism of these claims. ( Ken Binmore)
6/4/03Genetic Algorithm
Introduction: Axelrod’s Motivation
• Axelrod wanted to prove a point: The success of Tit-For-Tat in his Iterated Prisoner’s Dilemma was not based on the preconceptions of those who submitted entries.
• The hammer and the nail.
6/4/03Genetic Algorithm
More practical motivations
• We have a group aim, and we want agents that can fulfill it. More accurately, we want our agents to be as efficient as possible in achieving this goal.
• Example: The Mars element-gathering, subsumption architecture experiment
6/4/03Genetic Algorithm
The Genetic Algorithm
1. Construct an initial population
2. Test and score population
3. Calculate number of offspring
4. Reproduction: Choose parental pairs
5. Reproduction: Crossover / Mutation
6/4/03Genetic Algorithm
Encoding a Strategy (1)
0123456 ……….. 69
0101010 . 1 . 0 . 1 ……… 0 . 1
•Axelrod encoded strategies as sequences of 70 bits. How does this work?
•Note: 0 = C, 1 = D
6/4/03Genetic Algorithm
Encoding a Strategy (2)
Move
-3 / him
Move
-3 / me
Move
-2 / him
Move
-2 / me
Move
-1 / him
Move
-1 / me
What do I do now?
0000000 = C
0010010 = C
…………………
1111111 = D
6/4/03Genetic Algorithm
1. Constructing an Initial Population
• Two possibilities for creating an initial population:
1. Initial strategies can be assembled (as in the first two tournaments) or -
2. They can be randomly created
6/4/03Genetic Algorithm
Constructing an Initial Population (cont’d)
• Axelrod used normalized population of 20
• A rule of thumb is that the product of the number in the population and the number of generations should exceed 100,000
• In addition, the number of individuals in the population must considerably exceed the number of genes in each individual's chromosome.
6/4/03Genetic Algorithm
2. Testing the Population
• In the Iterated Prisoner’s Dilemma, the outcome of each interaction is added to produce a player’s score for that generation.
• In general, an ordering function is needed.
• This function must be efficient!
6/4/03Genetic Algorithm
3. The Number of Offspring (in Axelrod’s simulation)
• Strategies that were one standard deviation above the average score produced two matings.
• Strategies that were one standard deviation below the average were barren- no offspring.
• Other strategies produced a single mating
• Each generation is disjoint from its predecessor
6/4/03Genetic Algorithm
The Number of Offspring (cont’d)
• There are other possibilities available when calculating offspring:
• Normalization / growth of population size
• Preservation of arbitrary amount of previous generation’s strategies (not done in Axelrod model)
6/4/03Genetic Algorithm
4. Choosing Parents
• In Axelrod’s simulations, pairs were chosen randomly to mate and produce, each, two offspring.
• Other possibilities exist:
• Mating by excellence (short term exploitation)
• Mating by geographic proximity
6/4/03Genetic Algorithm
5. Reproduction: Crossover
• “Crossover selects one or more places to break the parents’ chromosomes in order to construct two offspring each of whom has some genetic material from both parents.”
• Other forms of crossover are possible.
• Syntactic integrity must be preserved!
• What are the advantages of high/low crossover? (more on this later)
6/4/03Genetic Algorithm
5. Reproduction: Mutations
• In every offspring born, there is a small chance of bit reversal- a change in strategy.
• Don’t forget syntactic integrity!
• What are the advantages of high/low mutation rates?
6/4/03Genetic Algorithm
Technical Details
• Population size: 20
• Round length: 151 meetings
• Each population member met one of 8 “representatives” (not each other)
• Therefore 24000 meetings per generation
• Number of generations: 50
• 40 2-parent (sexual) experiments, 40 asexual ones
6/4/03Genetic Algorithm
Conclusions (1) : Effectiveness• “the problem for evolution can be conceptualized as a
search for relatively high points in a multidimensional field of gene combinations, where height corresponds to fitness.”
• Axelrod: TFT-like strategies are the big winner!
• The genetic algorithm produces results better than the second Axelrod tournament: 450 weighted score vs. 428 for TFT
• But these results were in 11 (out of 40) experiments which resulted in a strategy which tried to exploit “sucker” strategies! (more later)
6/4/03Genetic Algorithm
Successful Alleles: TFT-like behavior
1. Don’t rock the boat ( C after RRR )
2. Be provocable ( D after RRS )
3. Accept apologies: ( C after TSR )
4. Forget: ( C after SRR )
5. Accept a rut: ( D after PPP )
6/4/03Genetic Algorithm
Conclusions (2) : Sexual Reproduction
• In biology, sexual reproduction carries a stiff price – useless males
• Computationally, sexual reproduction is cheap.
• Asexual runs of IPD resulted in lower average scores (5 out of 11 had higher median scores than TFT)
• Parasite theory. (only if we have time)
6/4/03Genetic Algorithm
Conclusions (3) : Arbitrariness
• Hitch-hiking genes- typically present, not typically employed. (TFT example – PPR)
• Premises and their results ( TFT example – the original six bit premise )
• Two basin theory ( Ken Binmore ) : By choosing the correct premises, we can produce different (stable) evolutionary end-products
6/4/03Genetic Algorithm
Conclusions (4) : Tradeoffs
• The trade off that exists is between flexibility and specialization.
• This can be translated into short and long term gains (exploitation vs. exploration)
• Varying mutation based on how dynamic the environment is.
• Invasion
6/4/03Genetic Algorithm
Conclusions (5) : Irreversibility
• The possibility exists of getting stuck in a local maximum.
• This results from adaptation to a set of premises, which strengthens them in the future.
• There are those who claim that TFT is such a local maximum.
6/4/03Genetic Algorithm
Conclusions and Criticism
• Kristain Lindgren (1991) – found cyclical history of stability / instability
• Lombard (1996) : more extensive simulations, copy-and-innovate instead of standard genetic algorithm
• Lombard: Noise is a crucial factor
6/4/03Genetic Algorithm
More Criticism and Conclusions
• Binmore - TAT-FOR-TIT is a better strategy?
• Were more experiments called for? Probst (1996) found exploitive machines thriving after running longer experiments!
• His criticism is against the simulation results, not against the use of the algorithm in general.
• But this is a lesson that applies only in pairwise interactions. In multi-person interactions, it need not be the injured party who punishes a cheater.
top related