Reactive Agents to Improve a Parallel Genetic Algorithm

Solution

Ana Laureano1, José de la Cruz1, Javier Ramírez1 and Julio Solano2

1 Universidad Autónoma Metropolitana - Azcapotzalco

2 DISCA - IIMAS / UNAM

MultiAgentArchitectureReactive Agents80s BrooksExhausitive representation of the real worldThe behaviour is emergent and it is dependent on the perception of the environment at a given instant

Agents based on behaviour

A constant interaction with the environment Control mechanism allowing them to work with limited resources and incomplete informationThe advange of the use, lies in the speed of adaptation to unforeseen situations

Proposal ...

Analysis and design of reactive agents based Bottom-up design mecahnism

MaesLaureano, de Arriaga and García-Alegre

To improve a genetic algorithm

Parallel genetic algorithm

Efficiency of the classification of a universal set in two independent subsets

The balance between the capacity to converge to an optimum and the ability to explore new regions (pm, pc)

pm, pc are varied adaptively according to the fitness values of the solution

To improve a genetic algorithm

This architecture is implmented by establishing a similarity between the desires espressed by MAESEatingExploringFleenig Interchange individuals Improve the quality of the solution

Design of the Reactive Genetic Algorithm

Maes: Artificial creatures Beahviours (eating, drinking, exploring, going-

towards-food, going-towards-water, sleeping, fighting, fleeing-from-creature, going-towards-creature, and dodging-obstacle)

Motivations (thirst, hunger, sleepiness, fear, tiredness, safety and agressiveness)

Virtulal environment (water, food, another creature, an obstacle)

Each behaviour is associated with:an activation level represented by a real

numbera set of conditionsa threshold, represented by a real number

Behavior activation mechanisms are represented by:

the creature’s desire to perform a given behaviour

the present situation: the environment influences the creature’s desires

activation through the precedent

Parallel Genetic Algorithm with Reactive Characteristics

The purpose of incorporating a reactive behavior in the PGA, is improve the overall performance by up-dating the sub-populations according to the general behaviour of the algorithm avoiding getting stuck in local minima

The Reactive Parallel Genetic Algorithm (RPGA) will be able to send and receive individuals from and to any other sub-population. The former, at the time required, by observing the stimuli presented by the environment at a given time.

Characteristics and elements that will substitute the traditional migration

mechanism: A sub-population might send individuals to another sub-population in any generation. There is no need for both sub-populations to be synchronized. The sender does not know in what generation, nor to whom will it send individuals, until the very moment they are sent.

Due to the reactive migration plan, several cases can arise in the individuals interchange stages. These go from the minimal case in which in a given generation no population sends any individuals, to the maximum one in which, given a generation, all the populations send individuals to the n - 1 remaining ones. All the subsets of ordered pairs of the set of sub-populations must be considered.

In each generation, each sub-population (GA) must verify if individuals from other sub-populations have arrived. The former due to the lack of synchronization and migration topology. For that reason, all the sub-populations (GA), must check a special buffer, which will be called the station. If the station is empty, the GA continues on its task; if it is not, the individuals are extracted and added to the sub-population.

The sending of individuals from one sub-population to another is determined by certain non-deterministic impulses assigned to each GA. Those impulses would be controlled by a mechanism similar to the one used by MAES to simulate the behavior of simple creatures in a dynamic environment.

The new concepts are defined below: Migration Limit : is a value representing the limit

that the migration impulse must surpass Migration Impulse : is a value representing the

GA’s desire of sending individuals to a specific sub-population

Stimulus Range : is an interval comprising the allowed values of the stimulus in order to increment the migration impulse. The generated value will be called stimulus and it will be added to the present value of the migration impulse

An important aspect of the GA is the quality of the solution. For that reason, a mechanism is included in which each population requests to the n-1 remaining sub-populations the sending of individuals. Such request will be made based on the time elapsed from the last time the former best solution was improved. The time interval (number of generations) elapsed without improvement will be called improvement limit.

When in a GA a number of generations have passed and the improvement limit did not change, it sends a message to the other n-1 GAs. This message is interpreted as a request for individuals from the sender. The former will allow the rest of the GAs (sub-populations) to send their own individuals.

Generate Initial Population

Do While Population Evolution Criterion not Fulfilled

Find Aptitude

Select Individuals

Cross Individuals

Mutate Individuals

Increment Migration Impulse

If (Requests Arrived), Then

Increment Migration Impulse

End If

If (Migration_Impulse > Migration_Limit), Then

Send Individuals

End If

If (individuals arrived), Then

Add Individuals to Population

End If

If (solution not Improved in given time) Then

Send Individuals Request

End If

End Do

Station

Send

Quality

Environment - n

Station

Send

Quality

Environment - n

Station

Send

Quality

Environment - n

Station

Send

Quality

Environment - n

Experiments:

We had available a set of 201 samples:

• 49 from group 1

• 152 from group 2

•Comparison between genetic algorithms according to the effectiveness of each one, in order to classify accurately each individual in its proper group.

An objective function was designed for this purpose, determining the number of successes for each individual in the population.

m

jj

n

ii baCxf

11

where:

n is the number of samples from group 1.m is the number of samples from group 2.C a constant, used to assign a greater weight to successes in samples from group 1.

otherwise

groupfromisamplecorrectlyclassifiesifai 0

11

m

jj

n

ii baCxf

11

otherwise

groupfromisamplecorrectlyclassifiesifai 0

11

otherwise

groupfromjsamplecorrectlyclassifiesifb j 0

21

Two types of mutation were used: static and dynamic

max

max1 )(

A

AAAkp indavg

m

for Aind > Aavg

pm = k2 for Aind Aavg

where:

•pm = mutation probability.•Amax = aptitude of the best individual of the population in that generation.•Aavg = average aptitude of all individuals of the population in the generation.•Aind = aptitude of the individual for which pm is being estimated.•k1 = 0.5•k2 = 0.25

Parameters and Configuration

• The RPGA uses a maximum of 2000 generations.

• The universal stochastic selection method of reproduction is applied.

• The crossing mechanism is a discrete one with a probability of 1.0.

• The mutation mechanism is based on the one proposed by Mühlenbein and Schilerkamp-Voosen.

• The mutation probability is dynamic, based on the work of Patniak and Mandavilli.

Parallel Genetic Algorithm Reactive Genetic Algorithm

Run Time Aptitude Effect Time Aptitude Effect

1 370 0.9813 93.58 409 0.9798 93.08

2 372 0.9829 94.06 407 0.9829 94.06

3 368 0.9844 94.56 410 0.9829 94.06

4 369 0.9829 94.06 412 0.9813 93.58

5 375 0.9782 92.58 413 0.9798 93.08

6 375 0.9813 93.58 411 0.9844 94.56

7 389 0.9798 93.08 408 0.9813 93.58

8 371 0.9782 92.58 407 0.9844 94.56

9 372 0.9798 93.08 410 0.9844 94.56

10 371 0.9844 94.56 415 0.9829 94.06

Avera 373.2 0.9813 93.57 410.2 0.9824 93.92

Table 1. Static mutation

Parallel Genetic Algorithm Reactive Genetic Algorithm

Run Time Aptitude Effect Time Aptitude Effect

1 670 0.9829 94.06 707 0.9860 95.06

2 679 0.9875 95.55 710 0.9891 96.04

3 673 0.9860 95.06 711 0.9829 94.06

4 672 0.9813 93.58 709 0.9844 94.56

5 670 0.9860 95.06 710 0.9891 96.04

6 669 0.9860 95.06 708 0.9813 93.58

7 671 0.9829 94.06 700 0.9891 96.04

8 675 0.9813 93.58 710 0.9860 95.06

9 679 0.9829 94.06 712 0.9875 95.55

10 673 0.9829 95.55 715 0.9891 96.04

Avera 673.1 0.9840 94.56 709.2 0.9864 95.20

Table 2. Dynamic mutation

The RPGA has available a population of 800 individuals.

The best results were obtained when it was divided into 8 sub-populations of 100 each.

Members of the population migrate using the reactive mechanism with a migration percentage of 10%.

In order to implement a reactive migration scheme, several runs were conducted.

The best results were obtained for migration rates of: 10, 15 and 20.

To save communications we choose 20 as migration limit.

Therefore, migration impulses are set to zero and progressively increased in each generation randomly

Conclusions

A migration mechanism for genetic algorithms controlled by reactive agents is presented.

The results obtained show that both algorithms present their best results when dynamic mutation is used, the RPGA being the best for classifying, with a mean effectiveness percentage of 95.20%, while the PGA reaches 94.55%.

Conclusions

In that way the RPGA can be considered as an efficient method for the solution of complex problems.

Even if the computing time of the RPGA is larger, it has however a practical advantage, a higher classification average.