artificial immune systems...3/1/2006 immune systems - an evolutionary metaphor page 4 of 25 so for...

Artificial Immune Systems

Steve CayzerSemantic and Adaptive SystemsHewlett-Packard Laboratories,

Bristol

February 2006

of 25 3/1/2006 Immune Systems - an evolutionary metaphor

The Immune System is…

Immunity: state or quality of being resistant (immune), either by virtue of previous exposure (adaptive immunity) or as an inherited trait (innate immunity)

Immune system: a system that protectsthe body from foreign substances and pathogenic organisms by producing the immune response


So for the purposes of this seminar…

• The human body is constantly under attack from pathogens which produce antigens (foreign proteins)

• The immune system (lymphocytes) recognise the antigens and cause the pathogens to be destroyed

• … without destroying the host (self proteins)

• Each lymphocyte matches a range of proteins: as a population, the immune system (learns to) cover non-self space.

• Adaptive, self organising system: good paradigm for ‘new’computing?


Antibodies map non-self space

Self

X

Antibody (with recognition radius)

Non-Self



Self

Non-Self

X



Self

Autoreactive Antibody Destroyed

Non-Self

X



Self

Non-Self

XX

Antigen(matched by antibody)



Self

Non-Self

XX

Clonal MaturationWith hypermutation

XX


• basic model of an immune component (eg lymphocyte, often conflated with antibody)

• design informed from immunology• aimed at problem solving

Definition of AIS

de Castro and Timmis: “Artificial Immune Systems (AIS) are adaptive systems, inspired by theoretical immunology and observed immune functions, principles and models, which are applied to problem solving”

http://www.cs.kent.ac.uk/people/staff/jt6/aisbook/


Models, Design Features, Applications

Models

l Negative Selection

l Positive Selection

l Danger model

l Self Assertion

Features

l Learning

l Associative Memory

l Avoids ‘self’

l Autonomous

Applications

l Security

l Classification/Clustering

l Optimisation

l Modelling


Framework for AIS design

Application Domain

Representation

Affinity (cf fitness in GA)

Algorithms


GA – basic algorithm

Initialise population

WHILE (not finished)

Calculate fitnessSelectReproduceCrossover and mutationReplace

END WHILE


AIS – basic algorithm

Initialise population of lymphocytes


Calculate fitnessSelectReproduceCrossover and mutationReplace

END WHILE





Present antigenCalculate fitnessSelectReproduceCrossover and mutationReplace

END WHILE





Present antigenCalculate fitness (=match to antigen)SelectReproduceCrossover and mutationReplace

END WHILE





Present antigenCalculate fitness (=match to antigen)SelectReproduce (clonal expansion)Crossover and mutationReplace

END WHILE





Present antigenCalculate fitness (=match to antigen)SelectReproduce (clonal expansion)Mutation (no crossover)Replace

END WHILE





Present antigenCalculate fitness (=match to antigen)SelectReproduce (clonal expansion)Mutation (no crossover)Replace (variable population)

END WHILE





Present antigenCalculate fitness (=match to antigen)SelectReproduce (clonal expansion)Mutation (no crossover)Replace (variable population)Memory cells

END WHILE


Other notes

• Application areas– Classification– Clustering– Computer security

• Philosophical divide– GA or neural net- Or something else?

• New Approaches– Danger Theory – Gene Libraries

(end)


Supervised Tasks: AIRS

AIS for classification and clustering (1)

Application Domain

Representation


Algorithms K- Nearest Neighbour

Hamming (usu)

Varied (UCI datasets)

Data Mining


AIRS focus now shifting to parallel properties


82256

SonarDiabetesIonosphereIris

Watkins, A., Timmis, J., Boggess, L. 2004 “Artificial Immune Recognition System (AIRS): An Immune-Inspired Supervised Learning Algorithm”

Genetic Programming and Evolvable Machines 5 (3): 291-317

Supervised Tasks: AIRS

AIRS does competitively benchmarked against 35 classifiers on some standard datasets

(number represents ranking where 1 = best, ~35 = worst)


Unsupervised Tasks: aiNET


Application Domain

Representation


Algorithms Immune network model

Euclidean

Expression levels

Gene Expression Clustering


Unsupervised Tasks: aiNet

•an iterative clustering algorithm that performs data compression using a pattern recognition process inspired by the human immune system.

•applied to a benchmark data set of gene expression levels

•Capable accurately detecting the presence of clusters without a priori knowledge about the number of clusters


Bezerra, G. B., de Castro, L. N. (2003), “A Hybrid Approach for Gene Expression Data Clustering”, International Conference on Bioinformatics and Computational Biology, 2003

http://www.vision.ime.usp.br/~cesar/programa/pdf/112.pdf


Negative Selection: LISYS

AIS for Security (1)

Application Domain

Representation


Algorithms Negative Selection

r contiguous bits

Network connections

Intrusion Detection


•LISYS does quite well on test data

•Problems with false positives?

•Does it scale?

AIS for Security (1)

S. A. Hofmeyr and S. Forrest (1999) “Immunity by Design: An Artificial Immune System”Proceedings of the Genetic and Evolutionary Computation Conference (GECCO) pp. 1289-1296

Kim, J. and Bentley, P. J. (2001) "Evaluating Negative Selection in an Artificial Immune System for Network Intrusion Detection" , Genetic and Evolutionary Computation Conference 2001 (GECCO-2001) pp.1330 - 1337

Balthrop, J. Forrest, S. Glickman, M.R. (2002) “Revisiting LISYS: parameters and normal behavior” Proceedings of the 2002 Congress on Evolutionary Computation, CEC '02: 1045 - 50


Problems with the self-nonself worldview

• How do we produce lymphocytes that react against antigens and yet avoid self?

• One way is “Generate and Test”: negatively screen lymphocytes which react to self at production time

• But this is expensive!

• It’s difficult to screen against ALL self.

• Self also changes over time

• And it is not necessary to screen against all non-self – only dangerous non-self

Aickelin & Cayzer 2002 The Danger Theory and Its Application to Artificial Immune Systems Proc. International Conference on AIS (ICARIS 2002)


The Danger Theory

• In the danger model, the idea is to recognise ‘danger’rather than non self.

• The screening is accomplished post production through an external ‘danger’ signal.

• Thus the production of autoreactive lymphocytes (which react to self) is allowed.

• If an (eg autoreactive) lymphocyte matches a stimulus in the absence of danger, it is removed.

• Thus harmless antigens are tolerated, and changing self accommodated.

Matzinger (2002) The Danger Model: A renewed sense of self Science 296: 301-304


EPSRC Adventure Fund 2004-2007: HP, UCL, UWE, Nottingham

Aickelin U, Bentley P, Cayzer S, Kim J and McLeod J (2003): 'Danger Theory: The Link between AIS and IDS?', Proceedings ICARIS-2003, 2nd International Conference on Artificial Immune Systems, LNCS 2787, pp 147-155

“The danger theory suggests that the immune system reacts to threats based on the correlation of various (danger) signals, providing a method of ‘grounding’ the immune response, i.e. linking it directly to the attacker.”

www.dangertheory.com


Other ways of using danger

Danger = Crime, Antigen = Suspect

or...

Danger = Context ?

It could also be useful for data mining, where the ‘danger’signal is a proxy measure of interest

‘Danger Zone’ can be spatial or temporal

Andrew Secker, Alex Freitas, and Jon Timmis (2005) “Towards a danger theory inspired artificial immune system for web mining” in A Scime, editor, Web Mining: applications and techniques, pages 145-168 (Idea Group)


Gene Libraries

2 broken assumptionsrandom creation

uniform antigens

Gene Libraries bias lymphocyte creation

Steve Cayzer, Jim Smith, James Marshall and Tim Kovocs (2005) “What have Gene Libraries done for AIS?” Proc. 4th International Conference on AIS (ICARIS 2005)


So what can they do for AIS?

Cayzer & Smith 2006 Gene Libraries: Coverage, efficiency and diversity. AISB'06: Adaptation in Artificial and Biological Systems April 3rd-6th 2006 University of Bristol, Bristol, England

•Increase coverage?

•yes (but can be expensive)

•Decrease cost?

•Yes (but can reduce diversity)

•Map antigens?

•Probably! (expts ongoing)

•Cope with dynamic (co evolving) pathogens?

•Tbd…


Genetic algorithm • Creation (gene libraries)

• Emphasis on mutation

• Matching ~ fitness (?)

• Variable population size

Considerations

• Role of antigen

• Preservation of diversity

AIS can be thought of as a special case of:

Philosophical Divide






Considerations

• Role of antigen



AIS as optimiser



Neural network• Pattern classification

• Unsupervised learning

• Topographic mapping

• Variable network topology

Considerations

• Interpreting response

• Training regime





Considerations

• Role of antigen



AIS as optimiser



Neural network• Pattern classification

• Unsupervised learning

• Topographic mapping

• Variable network topology

Considerations

• Interpreting response

• Training regime





Considerations

• Role of antigen



AIS as optimiser AIS as classifier



… Alternatively …

Application Areas of AIS: Past, Present and Future E. Hart and J.Timmis. International Conference on Artificial Immune Systems (ICARIS), Banf, Canada. LNCS 3627. pp. 483-497. C.Jacob Et. al. (Eds) 2005.


New Approaches

• Innate response (works for most animals!)– Towards a Conceptual Framework for Innate Immunity Jamie Twycross and Uwe Aickelin Proc. of the

4th International Conference on Artificial Immune Systems, Banff, Canada, 2005 – R. Germain. An innately interesting decade of research in immunology. Nature Medicine,

10(12):1307–1320, 2004.

• Neuro-endocrine-immune interactions– J. Timmis and M. Neal. Once more unto the breach: Towards artificial homeostasis. Recent

Developments in Biologically Inspired Computing, pages 340–365, 2005.

• Danger Theory– Aickelin U and Cayzer S (2002): The Danger Theory and Its Application to Artificial Immune Systems',

Proceedings of the 1st International Conference on Artificial Immune Systems (ICARIS-2002), pp 141-148, Canterbury, UK

• Self Assertion (homeostasis)– Inspiration for the Next Generation of Artificial Immune Systems . P. Andrews and J.Timmis.

International Conference on Artificial Immune Systems (ICARIS), Banf, Canada. LNCS 3627. pp.126-138 . C.Jacob Et. al. (Eds) 2005


Or, maybe, the Immune System is…

• Francisco Varela: Homeostasis, self-develop an efficient communication pathway in order to create (assert) and maintain a coherent self

• John Stewart: rejection and memory are side effects of the homeostatic maintain.

Hugues Bersini: “While host defense is a critical function, it is hardly the only one of interest. Indeed the immune system might be regarded as primarily fulfilling an altogether different role…”

KNOW THYSELF? The Self Assertion View

Immune system only knows itself, no

recognition is at play

ftp://iridia.ulb.ac.be/pub/bersini/ImmunoSelf.pdf


AIS – Refined algorithm

Basic Matching Algorithml Population of B Cells (antibodies)

l Clonal expansion and hypermutation

Extensionsl Lifecycle events, screening (positive/negative selection)

l Other IS elements (T Cells, cytokines)

l Network interactions (idiotypic effects)

l Other – localization, self adaptation, population control

Choices l Genotype/Phenotype (Representation & Shape Space)

l Matching (Hamming, Euclidean, r-contiguous, other)


The Idiotypic Effect: Antibody-antibody interactions

Jerne’s Big Idea (1974)

Idiotype: specificity of antibody (epitopes to which it will bind)

Idiotope: An idiotypic epitope

Evidence: Antibodies produced againstantibodies of same species (cf individual)Antigen

P1

Idiotypic Set

I1

Anti-Idiotypic Set

-

P2 I2

Internal Image of Antigen

+

P3 I3


The Idiotypic Effect – Why do we care?

• Biological importance - ???

• Immunological models – Varela, Castellani

• Pattern recognition – Timmis & Hunt

• Non-stationary environments (idiotypic memory) –Gaspar & Collard

• Multimodal Optimisation – de Castro

• Recommendation communities – Cayzer & Aickelin


i

n

jjiji

N

j

N

jjiijjiji

i

xkyxmxxmkxxmc

ratedeath

recognisedantigens

recognisedamI

recognisedantibodies

cdtdx

211 1

1 −

+−=

−

+

−

=

∑∑ ∑== −

Modelling the Idiotypic Effect

• For N antibodies, n antigens.• xi is the concentration of antibody i• yi is the concentration of antigen I• c, k1 and k2 are scaling constants

• mij is a matching function


l Antibodies as:entire solutions

‘building blocks’

l Antigens as:objective functionsconstraintsfit/feasible solutionssolutions to subproblemsweight combinations spanning Pareto optimal front

l AIS usually hybridised with GA:Antibody selectionGene library creation

l Emergent fitness sharing (generalist/specialist)

AIS for Optimisation


Evaluationl Applied to TSP (of course), job shop scheduling, time series prediction, truss design, capacitor placement, time dependent optimization…

l Some good results on test problems

BUT…

l Often little ‘added value’ to GA

l AIS metaphor somewhat strained

l Difficult to find fair comparisons

l Best viewed as a collection of hybridising techniques

AIS for Optimisation


Example: Hajela & Yoo 2001

Crossover

Mutationbest

Antigen

Designs

Generalist AIS

Antibody

GA(unconstrained objective function)

Feasible Infeasible

allSmall s

artificial immune systems...3/1/2006 immune systems - an evolutionary metaphor page 4 of 25 so for...

Documents