Second GSFC/IEEE Workshop on Radical Agent

Concepts

Second GSFC/IEEE Workshop on Radical Agent

Concepts

Genetically Modified Software: Realizing Viable Autonomic Agency

A.G. Laws, A. Taleb-Bendiab & S.J. WadeLiverpool John Moores University, UK

Introduction & Overview Introduction & Overview

• Our search for a unifying conceptual framework and design model for the development of autonomic computing systems

• Management Models for Self-Managing Systems– Managerial Cybernetics and the Viable System Model

• A Cybernetic Reference Model for Autonomic Systems– and some drawbacks

• Some classical cybernetics and their implications• More speculatively

– Some small scale experiments in adaptation with genetic algorithms

– Lessons, limitations and possibilities– Dynamic Model Development for Open Environments

using LCS• Conclusions

The Viable System Model The Viable System Model • Underpinned by “classical” cybernetics

though structured using the human CNS as a model

• Beer’s VSM implements a control & communication structure via hierarchies of (homeostatic) feedback loops

• 6 major systems ensure ‘viability’ of the system

– Implementation S1– Monitoring S2– Audit S3*– Control S3– Intelligence S4– Policy S5

• Offers an extensible, recursive, model-based architecture, devolving autonomy to sub-systems

AutonomicSystems

AnticipatorySelf-awarenessDeliberative

System 5Policy

System 4Intelligence

System 3Control

System 3*Audit

System One (S1)

5

4

3

2

2

2

2

3*

1

Org

an

iza

tion

al E

nvi

ron

me

nt

System 2Coordination

System 2Coordination

System One (S1)

5

3

43*

1

System 2Coordination

System One (S1)

5

3

43*

1

System 2Coordination

Where this begins to get interesting

T h ree *S p o ra d ic A u d it

a lw a y s sta n d s fo r

T w o

T w o

T w oa n ti

o sc illa to ry

T h reein sid e & n o w se lf-o rg a n iza tio n a u to n o m ic reg u la tio n

F o u ro u tsid e & fu tu re se lf-re fe ren ce sim u la tio n p la n n in g

F iv e p o licy

a lg ed o n ic

lo ca l reg u la to ry

lo ca l reg u la to ry

O n e

O n e

5

5

3

4

4

3

2

2

2

2

3 *

3 *

1

1

• The recursive nature of the underlying model suggests that each System One should develop as a Viable System Model in its own right.

• If the recursion is pursued below the level of the human software team then we arrive at the level of the software system.

• This strongly implies that the software system should assume the same overall management structure as the human software process

• Consequently, we appear to have a theoretically supported management architecture both for the human process and for “self-managing autonomic software”

• This also fits closely with IBM’s “dynamic management by business rules/policies” aim for autonomic systems

BasicLevel 1

ManagedLevel 2

PredictiveLevel 3

AdaptiveLevel 4

AutonomicLevel 5

Multiple sources ofsystem generateddata

Requires extensive,highly skilled I T staff

Consolidation of datathrough managementtools

IT staff analyzes andtakes action

System monitors,correlates andrecommends actions

IT staff approvesand initiates actions

System monitors,correlates and takesaction

IT staff managesperformance againstSLAs

Integratedcomponentsdynamically managedby business rules/policiesIT staff focuses onenabling businessneeds

Geater systemsawareness

Improvedproductivity

Reduceddependency on deepskills

Faster, betterdecision-making

IT agility andresiliency withminimal humaninteraction

Business policy drivesIT management

Business agility andresiliency

ManualAutonomic

From IBM Global Services and Autonomic Computing, IBM White Paper, October 2002.

Bratman et al’s IRMA Architecture

• Bratman et al’s Intelligent Resource-Bounded Machine Architecture (IRMA), a classic BDI agent model

• It is noticeable that many of the elements contained in the model correlate directly to elements of the VSM

• So– Planning & Means-Ends

Reasoning S3– Opportunity Analyzer & Filtering S4– Beliefs about the world S5– Desires and Deliberative Process

S5– Intentions - S5 Policy passed to S3

for enactment• It is also apparent that some elements

are missing for Viability

Desires

Wor

ldM

odel Deliberation

Process

Intentions

OpportunityAnalyser

EnvironmentalScan

FilteringProcess

Plans

S

urvi

vin

g P

lans

Intentions

Planning Process

PlanLibrary

Reasoner

Internal Model

Internal Model

Status

ResourceBargaining

Process

Intentions

Overall Plans

Resou rcesA

cco

un

tab

ilit

y

Inten

tions

System Three

System Four

System Five

OperationalStatus

Scheduling

Overall Schedule

Local Plans

Local Schedule

Local Monitoring data

Local Monitoring data

Local Monitoring data

Ove

rall

Sch

edul

e M

onit

orin

g to

nex

t re

curs

ion

Monitoring data

A Viable Intelligent Agent Architecture?

• Using Bratman et al’s IRMA approach and using the VSM as a reference model, we designed a Viable Intelligent Agent Architecture realizable using a multi-agent approach.

• The design specifies, in some detail, the higher, deliberative processes of adaptation.

• The embedding of this architecture in a hierarchy of VSM’s keeps the “Man in the Loop”

• However, the VSM uses human agents to realize its systems. These provide implicit (unmapped) elements of creativity/learning to the organization.

Desires

Wor

ld

Mod

el DeliberationProcess

Intentions

OpportunityAnalyser

EnvironmentalScan

FilteringProcess

Plans

S

urvi

ving

Pla

ns

Intentions

Planning Process

PlanLibrary

Reasoner

Internal Model

InternalModel

Status

ResourceBargaining

Process

Intentions

Overall Plans

Resources

Acc

ount

abili

ty Intentions

System Three

System Four

System Five

Operational

Status

Scheduling

Ov er a ll Sch e du l e

Local Plans

Local Schedule

Local Monitoring data

Local Monitoring data

Local Monitoring data

Ove

rall

Sche

dule

Mon

itori

ng to

nex

t rec

ursi

on

Monitoring data

Ashbean CyberneticsAshbean Cybernetics• The Homeostat - ultra-stable system

capable of returning to stability after it has been disturbed in a way not envisaged by the designer.

• Self-vetoing homeostasis• The Law of Requisite Variety effective control

can only be attained if the repertoire of responses that the controller has at its disposal at least matches the disturbances the situation to be controlled can display.

• More recently, this has been supplemented by the Law of Requisite Knowledge, which states that the system must know i.e. learn, the most appropriate response to use in a given circumstance – i.e. optimized response

• Taken together these two laws underpin the Conant-Ashby theorem, namely “Every good regulator of a system must be (contain) a model of that system”.

K eyV = V a rie ty

E n v iro n m en t

V V

S y stem

M a n a g e m e n tU n it

V

= A tten u a tion

= A m p lifica tion

A (simple) Scenario to address the Law of Requisite Variety

A (simple) Scenario to address the Law of Requisite Variety

• Ashby’s Law states that a system must “derive” a repertoire of actions that can be deployed according to environmental circumstances

• Developed a “toy” system where the environment simply delivers arrays of integers for sorting

• The system is provide with a range of responses from which it tries to select the “optimal” solution using timings/statistical analysis

• Advantages– Fully controllable environment

• Finite (n = 8 = 40320 permutations)• Smooth/turbulent as you like• Measure the degree “sortedness” of

test cases easily

• However, more interesting is how the system acquires new responses to address new environmental circumstances.

SortingSystem

En

viro

nm

ent Quick

Shell

Merge

Bubble

Insert

Hillis’s Sorting NetworksHillis’s Sorting Networks• Hillis had used a “Connection Machine”

coupled with an approach using Holland’s Genetic Algorithms to “evolve” sorting networks for n = 16 arrays

• Although no Connection Machine was available, we decided to try with n = 8 arrays

• Here, for n = 8, a sorting network is represented by 5 pairs of 12 bit chromosomes

• Each 3 bits represents an array index (0 – 7)

• Each 12 bit chromosome holds 4 array indices and hence 2 comparisons

• E.g. Compare Index 1 with Index 2 and swap if necessary

• Chromosomes are paired so that duplicate entries do not appear in the phenotype.

• This gives a minimum of 10 comparisons and a maximum of 20 for the sorting network. Hence more/less “optimal” solutions?

0 0 1 1 0 1 1 1 1 1 1

1

0

3 7

1

5

(1 , 3) (7 , 5) (6 , 2)

0 1 1 1 0 0 101101

0 1 0 1 1 0 101000

1 1 0 1 0 1 101111

0 1 0 1 0 1 001001

0 1 1 1 0 0 101101

0 1 1 1 0 0 101101

0 0 0 1 0 0 101001

0 1 0 1 0 0 101000

0 0 1 1 0 0 101111

0 1 0 1 0 0 100001

1 0 1 1 1 1 1 0 10 0 1Chromosome A

1 0 1 1 1 0 0 1 00 0 1Chromosome B

Breeding GA’sBreeding GA’s• Used a population of 200

networks organized on a two-dimensional array and randomly filled with bits.

• Testing each against our engineered environmental test arrays and measuring reduction in “unsortedness” allowed the identification of “fitter” individuals and hence the new breeding population.

• Select two fit parents, using random crossover, each parent contributes 5 gametes to make a new 10 chromosome member in the new population.

• The new population is then subjected to random mutation (bit-flipping) using the standard probability rate of 0.001

Chromosome A

10 0 1Chromosome B

0 1 1 1 1 1 0 1

0 1 1 1 0 0 1 0

10 0 1

Parent 1

Chromosome A

10 0 1Chromosome B

0 1 1 1 1 1 0 1

0 1 1 1 0 0 1 0

10 0 1

10 0 1 0 1 1 1 0 0 1 0Gamete

Parent 2

10 0 1 0 1 1 1 0 0 1 0Gamete

10 0 1 0 1 1 1 0 0 1 0Chromosome A

10 0 1 0 1 1 1 0 0 1 0Chromosome B

Offspring

Emergence of Full Fitness IndividualsEmergence of Full Fitness Individuals

Full Fitness Individuals in the Population (200)

0

20

40

60

80

100

120

140

160

1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97

Generations

No

. of

Ind

ivid

ua

ls

• Somewhat surprisingly, fully-fit individuals appeared in the population very quickly.

• Here, a sorting network capable of correctly sorting the 100 test cases emerged after 11 generations (i.e. in minutes)

• Admittedly, this was using a “gentle” environment

– Inv. 0 “12345678”– Inv. 9 “12378654”

• Interestingly, more than one “type” of fully-fit individuals emerged – efficiency ?

• Unfortunately, less success in highly disordered environments

– Inv. 19 “87621345”– Inv. 28 “87654321” = 97%

• Of course, we’re not suggesting more generalized use, but perhaps “genetic programming?”

Learning Classifier Systems & The Law of Requisite Knowledge

Learning Classifier Systems & The Law of Requisite Knowledge

ClassifierSystem

cond 1 cond 2 message

IF THEN

MESSAGELIST

RULELIST

match

post

Output interfaceeffectors

Input interfacedetectors

Bucket brigade(adjusts rule strengths)

Genetic algorithm(generates new rules)

Payoff

Environment

Inpu

t mes

sage

s

Output m

essages

• So, assuming our system can develop a repertoire of responses tailored to specific environmental circumstances, we still need a means to achieve Requisite Knowledge

• One attractive candidate is Holland’s Learning Classifier Systems approach

• Here the system constructs “creates” candidate production rules in the form: if <condition> then <action> i.e. situation/response

• Rules compete (bid) to participate in a response to an environmental condition. The “solution” provided by the rules is tested in the environment and a “payoff” determined. This is distributed amongst contributing rules and either strengthens or weakens rules depending on success/failure

• A GA is used to generate new rules using provably fit rules.

• What emerges is an adaptive, learning “model” of the environment.

SummarySummary• So, we have looked at the VSM and what it has to contribute

– It clearly defines the types of systems that must exist for viability– Lead to a recursive architecture that appears to be able to incorporate both human systems

and autonomic software systems– Lends itself to the future development of a design pattern/grammar for autonomic system

development • We have noted some of the difficulties encountered by extending the model from

human to artificial systems• Model completeness in open environments• Although we didn’t refer to it today we need to define communications protocols• Etc.

• We briefly examined some of the cybernetic underpinnings for clues to a way forward• Presented some very small scale examples of the use of genetic algorithms to

generate a repertoire of responses and speculated on the use of other perhaps more generalizable genetic approaches e.g. artifical immune systems

• Speculated (even more briefly) on the potential use of Learning Classifier Systems as an approach to address the environmental modelling problems apparent in open systems

Thanks for your attention&

Any Comments or Questions would be welcome?