gbml

Introduction to Genetics-based

Machine Learning

Mahalingam.P.R

Semester III, M.Tech CSESIS, RSET

Contents

Background of Machine Learning

About Machine Learning

GA in Machine Learning

Evolution of GBML

Classifier System

Rule and Message System

Apportionment of Credit System

Genetic Algorithm

A simple classifier system in Pascal

Conclusion

2 Introduction to GBML

Introduction

Introduction to GBML3

Machine Learning

A machine learns whenever it changes its structure,

program, or data (based on its inputs or in response

to external information) in such a manner that its

expected future performance improves.

Machine learning usually refers to the changes in

systems that perform tasks associated with artificial

intelligence (AI).

Such tasks involve recognition, diagnosis, planning,

robot control, prediction, etc. The “changes" might be

either enhancements to already performing systems

or ab-initio synthesis of new systems.


A typical AI system

Here, the model progressively learns from the experience it earns from the environment


Need for Machine Learning

Some tasks cannot be defined well except byexample. We might be able to specify input/output pairs but not a

concise relationship between inputs and desired outputs.

We would like machines to be able to adjust their internalstructure to produce correct outputs for a large number ofsample inputs

suitably constrain their input/output function to approximate therelationship implicit in the examples.

It is possible that hidden among large piles of dataare important relationships and correlations. Machine learning methods can often be used to extract

these relationships (data mining).


Machine learning methods can be used for on-the-jobimprovement of existing machine designs.

The amount of knowledge available about certain tasksmight be too large for explicit encoding by humans. Machines that learn this knowledge gradually might be able to

capture more of it than humans would want to write down.

Environments change over time. Machines that can adapt to a changing environment would

reduce the need for constant redesign.

New knowledge about tasks is constantly beingdiscovered by humans (Vocabulary changes). There is a constant stream of new events in the world.

Continuing redesign of AI systems to conform to newknowledge is impractical.

Machine learning methods might be able to track much of it.


More on Machine Learning

Contributions to Machine Learning

Varieties of Machine Learning


Statistics

Brain Models

Adaptive Control Theory

Psychological Models

Artificial Intelligence

Evolutionary Models

Functions

Logic Programs and

Rule sets

Finite-state machines

Grammars

Problem solving

systems

GA in Machine Learning


Conventional GA systems work with the followingproperties: Probabilistic

Random

Enumerative

The structures adapted cater to the human-likemechanism adopted in the methodology.

This adaptation itself is the biggest hindrance whenincorporating GA into more complex, less completelydefined environments. Too much “guesses” when it comes to searching

Methodology safe under the “sandbox” of searching

Coming Up

Origins of GBML

Systems

Classifier System

Operation of Classifier

Systems

Implementation of a

Simple Classifier System

in Pascal

Testing classifier in a

problem domain –

Learning a Boolean

function


Study Machine Learning systems

that use genetic search as their

primary discovery heuristic.

Adapting GA structures to work in

complex environments like Machine

Learning

Origin of GBML systems



In nature, not only do individual animals learn to

perform better, but species evolve to be better fit in

their individual niches.

Since the distinction between evolving and learning

can be blurred in computer systems, techniques that

model certain aspects of biological evolution have

been proposed as learning methods to improve the

performance of computer programs.

Genetic algorithms [Holland, 1975] and genetic

programming [Koza, 1992, Koza, 1994] are the most

prominent computational techniques for evolution.

Theoretical Foundation for GBML


Laid by Holland (1962)

Outline for Adaptive Systems Theory

role of program replication as a method of emphasizing

past programs.

Fundamental role of recombination

Holland (1965)

Schemata Processors

Holland (1968-1971)

Application of a classifier system

Holland and Reitman (1978)


Modern classifier systems resemble schemata

processors in both outline and detail

Holland suggested four prototypes in the initial

proposal.

No experiments or implementation have been reported

yet!

Proposal coincided with the development of the

theory of schemata.

Prototype 1 Prototype 2


Stimulus-response (SR)

processor that would

link environmental

schemata (conditions)

with particular action

effectors.

Extend type 1 by adding

internal effectors

(internal states).

Prototype 3 Prototype 4


Build upon types 1 and

2 by including explicit

environmental state

prediction (a model of

the real world), and an

internal evaluation

mechanism.

Extend the other types

by incorporating the

capability to modify its

own effectors and

detectors, permitting

greater (or lesser) range

of data detection and a

larger behavioural

response.

Broadcast language


Derived from early proposals.

Broad

Unimplemented

Creating broadcast units (production rules) from a 10-letter alphabet.

The alphabet added wild card (single and multiple match)characters to an underlying binary alphabet.

If included, the following would have given sufficientpower for computational completeness andrepresentational convenience.

A fundamental punctuation mark, a persistence symbol forcontinued broadcast of a message.

A quotation character for taking the next character literally.


The proposal for broadcast language was

instrumental in unifying the earlier suggestions for

schemata operators by:

theoretically permitting a consistent representation of all

operators, data, and rules or instructions.

The generality gained in theory has not been

realized in practice.

First practical implementation


Three years following the broadcast languageproposal (Holland and Reitman, 1978).

Cognitive System Level One (CS-1) Trained to learn two maze-running tasks.

Performance system with message list and simple stringrules (classifiers).

GA composed of reproduction, crossover, mutation andcrowding.

Epochal learning mechanism where reward isapportioned to all classifiers active between successivepayoff events.

Learning mechanism has largely been supplanted byanother mechanism – a bucket brigade.

GBML Applications


Classifier System



A classifier system is a machine learning system that

learns syntactically simple string rules (called

classifiers) to guide its performance in an arbitrary

environment.

A classifier system consists of three main

components:

Rule and message system

Apportionment of credit system

Genetic algorithm


The rule and message system of a classifier system

is a special kind of production system.

A production system is a computational scheme that

uses rules as its only algorithmic device. The general

syntax of such systems are as follows:

if <condition> then <action>

The production means that the action may be taken

(rule is “fired”) when the condition is satisfied.


It has been shown that production systems are

computationally complete, as well as convenient.

A simple rule or set of rules can represent a complex

set of thoughts compactly.

But when it comes to situations in need of learning,

this is not advisable due to the complex rule syntax.

Many production systems permit involved

grammatical constructions for the condition and

action part of the rule.


Classifier systems depart from the mainstream by

restricting a rule to a fixed length representation.

This restriction has two benefits.

All strings under the permissible alphabet are syntactically

meaningful

A fixed string representation permits string operators of

the genetic kind. This leaves the door open for a Genetic

Algorithm search of the space of permissible rules.


Classifier systems use parallel rule activation,

whereas traditional systems use serial rule

activation.

They permit multiple activities to be coordinated

simultaneously. When choices must be made

between mutually exclusive environmental actions,

or size of matched rule set must be pruned to

accommodate the fixed length message list, the

choices are postponed to the last possible moment,

and the arbitration is then performed competitively.


In traditional expert systems, the value or rating of a rule relative to other is fixed by the programmer in conjunction with the expert or group of experts to be emulated. But this is not possible in rule learning system.

In such cases, the relative importance has to be “learned”. So, classifiers are forced to coexist in an information-based service economy. A competition is held among classifiers where the right to answer relevant messages goes to the highest bidder. Subsequent bids serve as the income for previously successful message senders.

This competitive nature ensures that good rules (profitable ones) survive and the bad rules (unprofitable) die off.

Internal Currency


The exchange and accumulation of an internal

currency provides a natural figure of merit for

applying GA.

Using the “bank balance” as a fitness function,

classifiers may be reproduced, crossed, and

mutated.

So, it can rank extant rules, and discover new,

possibly better rules by innovative combinations of

old ones.

Here, the stress is on “who gets replaced”, not

“replace entire populations”.

Next…

Each component is

discussed in detail and

the interconnections

studied.

•Rule and message

system

•Apportionment of credit

•Genetic Algorithm

Implementation in Pascal

Testing on real world

problem – learning a

Boolean function


Thus, apportionment of credit via

competition and rule discovery

using GA form a reasonable basis

for constructing a machine learning

system atop the computationally

convenient and complete

framework of classifiers.

This is the schematic of a

complete classifier

system.



The rule and message system forms the backbone of the silicon beast.

Information flows from the environment through the detectors, where it is decoded to one or more finite length messages.

The environmental messages are fed into a finite length message list where the messages activate string rules called classifiers.

When activated, the classifier posts a message on to the message list, which may in-turn invoke other classifiers, or cause an action to be taken through the system’s action triggers called effectors.


Classifiers combine environmental cues and internal

thoughts to determine what the system should do

and think next.

Coordinates the flow of information from where it is

sensed (detectors) to where it is processed

(message list and classifier store) to where it is

called to action (effectors).

Informational units

Messages

Classifiers


A message within a classifier system is simply a

finite-length string over some finite alphabet. If we

limit to the binary alphabet, we get

<message> ::= {0, 1}L

This means taking the concatenation of 0s or 1s for

“L” times.

Messages are the basic token of information

exchange in a classifier system.

The messages on the message list may match one

or more classifiers or string rules.


A classifier is a production rule with the syntax

<classifier> ::= <condition>:<message>

The condition is a simple pattern recognition device

where a wild card (#) is added to the underlying alphabet.

<condition> ::= {0, 1, #}L

So, the condition matches a message if at every position,

a 0 in the message matches a 0 in the condition, 1

matches a 1, or a # matches either a 0 or a 1.

For example, #01# matches 0010, 0011, 1010 and 1011. But it

doesn’t match 0000.

Similar to schema in GA


Once the classifier’s condition is matched, it

becomes a candidate to post its message to the

message list in the next time step.

Whether the candidate classifier posts its message

is determined by the outcome of an activation

auction, which in turn depends on evaluation of the

classifier’s value or weighting.

Sample simulation by hand


Initial message list is as follows.

Apportionment of Credit

Algorithm

THE BUCKET BRIGADE



Many classifiers attempt to rank or rate the individual

classifiers according to a classifier’s role in achieving

reward from the environment.

Most prevalent method

Bucket Brigade Algorithm

John Holland

About the algorithm


An information economy where the right to trade

information is bought and sold by classifiers.

Classifiers form a chain of middlemen from

information manufacturer (the environment) to

information consumer (the effectors).

Components of service economy

Auction

ClearingHouse


When classifiers are matched, they don’t directly

post their messages.

A matching message entitles a classifier to

participate in an activation auction.

Each classifier maintains a record of its net worth.

Called Strength (S).

Each matched classifier makes a bid B.

Bid proportional to its strength.

In this way, rules that are highly fit (have

accumulated a net worth) are given preference over

other rules.


Once a classifier is selected for activation, it must clear its payment through the clearinghouse, paying its bid to other classifiers for matching messages rendered.

A matched and activated classifier sends its bid B to those classifiers responsible for sending the messages that matched the bidder classifier’s condition.

Bid amount divided among the matching classifiers.

Division of payoff among contributing classifiers helps ensure the formation of an appropriately sized subpopulation of rules.

Different types of rules can cover different types of behavioral requirements without undue interspecies competition.


In a rule-learning system of any experience, we

cannot search for one master rule.

We must instead search for a co-adapted set of rules

that together cover a range of behavior that provides

ample payoff to the learning system.

Consider the

classifiers as before.


Now, lets follow the payments, with initial strength of

200.


For steady receipts, the bid value approaches the

receipt. For time-varying receipt values, we see that

the bid is a geometrically weighted average of the

input.

As such, it acts as a filter of the possibly intermittent

and noisy receipt values.

Genetic Algorithm



Bucket brigade

Clean procedure

Evaluation of rules

Decide among competing alternatives

But we have to devise a way of injecting new rules

into the system.

Similar to SGA, we can inject new rules using the

tripartite rules

Reproduction

Crossover

Mutation


The rules are placed in the population and processed by the auction, payment and reinforcement mechanism to properly evaluate their role in the system.

Pay attention to “who replaces whom” Not replacing the entire population

GA in classifier systems strongly resemble those used in search and optimization.

Main difference in Machine Learning

Non-overlapping population model not acceptable here.

In non-overlapping generations, complete generations are selected and replaced by a new population at every run.

Machine Learning Search and optimization


High level of on-line

performance.

Learn to perform more

proficiently.

Convergence

Offline performance

De Jong’s experiments Machine Learning


Conventional system

GA Parameter

Generation Gap (G)

Implement and test

overlapping population

Genetic Algorithms.

Whole population

should not be replaced.

Quantity

Selection Proportion

(proportion)

Replace that proportion

of the population at a

given algorithm run.

Coupled with a number

of other parameters.

Other Parameters


GA Period

Represented as Tga

Specifies number of time steps (rule and message cycles)

between GA calls.

Period can be treated deterministically

GA is called every Tga cycles

Or stochastically

GA is called probabilistically with average period Tga

Invocation of GA learning may be conditioned on

particular events such as:

Lack of match

Poor performance

Selection


Roulette Wheel Selection

Classifier’s strength S used as the fitness.

No longer generating entire populations

Careful when choosing population members for

replacement.

De Jong’s crowding procedure

Encourage replacement of similar population memebers

Mutation


Modified procedure

Here, ternary alphabet is used

SGA used a binary alphabet

Mutation probability pm defined as before

When a mutation is called for, we change the

mutated character to one of the other two with equal

probability.

0 { 1, # }

1 { 0, # }

# { 0, 1 }

Next-

A simple Classifier

System in Pascal

•Simple Classifier System

Data Structure

•The Performance System

•Apportionment of credit

algorithm

•Genetic search within the

Simple Classifier System

•Real-world testing

•Results

•Comparison with and

without GA


With all these changes to the

normal SGA routine, GA may be

dropped into the classifier system

and used in a manner not too

different from normal search and

optimization applications.

A Simple Classifier System in

Pascal



Construct a system designed to learn a boolean

function

A multiplexer

Collapse the finite-length message list to a single

message (the environmental message)

Immediate feedback

Simple payoff

Components


Simple Classifier System Data Structure Adapt to learning strategies

Performance System Heart of SCS

Matching procedures are the heart of the performance system

Apportionment of Credit Procedures

Auction

Clearinghouse

Taxcollector

Genetic Search within Simple Classifier System Similar to SGA

Learning the multiplexing system

Main procedure Reinforcement

Six – bit multiplexing system


Results using the Simple Classifier System

Without GA With GA


Conclusion


The following were discussed

Machine Learning

Role of GA in Machine Learning

The evolution of GA concepts in Machine Learning

Some applications of GBML

Classifier System

Components of Classifier System


Apportionment of Credit (The Bucket Brigade)

Genetic Algorithm

A practical implementation in Pascal

Increase in output efficiency when using GA

References

“Introduction to Machine Learning”, Nils J. Nilsson,

Robotics Laboratory, Department of Computer

Science, Stanford University

“Genetic Algorithms in Search, Optimization and

Machine Learning”, David E. Goldberg, pp 217-260


THANK YOU


gbml

Education

simple classifier

machine learning

classifier

machine learning

learning system

production

bucket brigade

genetic algorithms