Biologically Inspired Computing: Introduction

This is lecture one of

`Biologically Inspired Computing’

Contents:

Course structure, Motivation for BIC , BIC vs Classical computing, overview of BIC techniques

General and up-to-date information about this module

Go to my home page:

www.macs.hw.ac.uk/~dwcorne/

Find my Teaching Materials page, and go on from there.

Weeks Title What it’s about

1 General Intro to BIC

The differences between BIC and `ordinary’ computing, the kinds of problems we need BIC for (including basics of classification, optimisation, and problem complexity), motivation for BIC, and a broad overview of many BIC techniques and the kinds of problems they can solve.

2—5 Evolutionary Algorithms

Algorithms based on natural evolution, for solving real-world problems; various different algorithms based on this idea, several example applications

6—7 Swarm Intelligence

Algorithms inspired by natural swarming behaviour, with various applications (ant systems, particle swarm optimisation)

8 Alife / Cellular Automata

Simple but powerful ways to simulate biological systems, with many useful applications. Includes: L systems, Game of Life, self-organisation, and more complex CAs

9 Neural Computing Classification and prediction methods based on linked simple processing units, modelled on real neural tissues.

10 Other BIC methods A selection of other prominent BIC methods, e.g. Artificial Immune Systems, Cellular Computing, Foraging algorithms.

Course Contents

Course Delivery

Week beginning Monday 3:15 EM306

Wednesday 11:15 EM303

Thursday 4:15 EM307

EVENTS

12th Jan DC  overview of module

DC Optimisation and classification

NT Neural Computation

 

19th Jan DC Evolutionary Computation

DC Evolutionary Computation

NT Neural Computation

DC hands out coursework 1 worth 35% of module

26th Jan DC Swarm Intelligence

DC Swarm Intelligence

NT Neural Computation

 

2nd Feb DC Kohonen Networks

DC Cellular Automata

NT Neural Computation

 

9th Feb PF PF PF  

16th Feb PF PF PF  

23rd Feb PF PF PF  

2nd Mar   PF    

9th Mar NT PF NT  

16th Mar NT PF NT Friday hand-in for coursework 1

23rd Mar NT NT NT Revision Lecture

 

30th Mar DC Revision Lecture

PF Revision Lecture

   

DC

NT

PF

Examinable materials (DC)

• All the slides (all online)• A few additional papers and notes provided online

Exam + Coursework (whole module)

Exam: 50% DC c/w 35% PF c/w 15%

Programming/Expts assignment 25%Question Sheet 10%

• DC = David Corne, will generally lecture about bio-inspired methods for optimisation, with a focus on evolutionary computation (aka genetic algorithms) – broadly this is about how certain aspects of nature (evolution, swarm behaviour) lead to very effective optimisation and design methods.

• PF = Pier Frisco, will generally lecture about molecular computing – how computation is done within biological cells – and how that can be exploited, and how it inspires new ideas in computer science.

• NT = Nick Taylor, will generally lecture about neural computation – this is perhaps the most widely exploited bio-inspired technique, which underpins how we can build machines that learn from examples.

Course Delivery

Marks (BSc)The quiz sheets add up to 25 questions, each worth 0.4 marks. The Programming assignment is worth 10 marks.

Difference between BSc and MSc

The MSc students will have some additional reading materials. The web site will clearly indicate this.

The exams will be different

The MSc coursework will be based on the BSc coursework, but a bit harder with more to do.

This Lecture

Lecture 1: Classical computation vs biological computation Motivation for biologically inspired computation

Overview of several biologically inspired algorithms

What to take from this lecture:What `classical computing’ is, and what kinds of tasks it

is naturally suited for.What classical computing is not good at.An appreciation of how computation and problem solving

are manifest in biological systems.Appreciation of the fact that many examples of

computations done by biological systems are not yet matched by what we can do with computers.

An understanding of the motivation (consequent on the above) for studying how computation is done in nature.

A first basic knowledge of the main currently and successfully used BIC methods

Classical Computation vs Bio-Inspired Computation

• the fridge story• How do you tell the difference between dog and cat?• How do you tell the difference between male and female face?• How do you design a perfect flying machine? • How would we design the software for a robot that could make

a cup of tea in your kitchen?• What happens if you:

– Cut off a salamander’s tail?– Cut off a section of a CPU?

Classical Computation vs Bio-Inspired Computation

Classical computing is good at:

Number-crunchingThought-support (glorified pen-and-paper)Rule-based reasoningConstant repetition of well-defined actions.

Classical computing is bad at:

Pattern recognitionRobustness to damageDealing with vague and incomplete information;Adapting and improving based on experience

Why don’t we have software that can do the following things well?• Automatically locate a small outburst of

violent behaviour in a football crowd

• Classify a plant species from a photograph of a leaf.

• Design robust railway timetables

• Make a cup of tea?

Pattern Recognition andOptimisation

These two things tend to come up a lot when we think of what we would like to be able to do with software, but usually can’t do.

But these are things that seems to be done very well indeed in Biology.

So it seems like a good idea to study how these things are done in biology – i.e. (usually) how computation is done by biological machines

Basic notes on pattern recognitionand optimisation

Pattern recognition is often called classification

Formally, a classification problem is like this:

We have a set of things: S (e.g. images, videos, smells, vectors, …) We have n possible classes, c1, c2, …, cn, and we know that everything in S should be labelled with precisely one of these classes. In computational terms, the problem is: Can we design a computational process that takes a thing s (from S) as its input, and always outputs the correct class label for s?

Classification: examplesWhat S might be What the classes might beImages of peoples’ faces male, female

Smells (e.g. molecular spectra:

illicit_drugs, ok orfresh meat, good meat, rotten meat

Utterances of “hello” (e.g. in wav files)

child, man, woman or

authorised-person, unauthorised

Renditions of my signature genuine, fake

Images of artworks beautiful, good, reasonable, awful

Patient data – results of various blood tests

malignant, benign

Applications for loans good risk, medium risk, bad risk

Aircraft engine diagnostics safe, needs-maintenance, groundGround-penetrating radar image land-mine-here, safe

1

2

3

4

5

6

7

89

Some notes about those examplesThe idea of these examples is to:1. Remind you that pattern recognition is something you do

easily, and all the time, and you (probably) do it much better than we can do with classical computation. (e.g. 1, 2, 3, 5)

2. Remind (or inform) you that such complex pattern recognition problems are not yet done well by software (e.g. 1, 2, 3, 5)

3. Indicate that there are some very important problems that we would like to solve with software (9, 8, 6, 2, 7 are obvious, but of course we would like to do all nine and much more ), which are classification problems, and note that these are just as hard as examples 1, 2, 3, 5.

4. So, hopefully we can learn how brains do 1, 2, 5 etc …, so that we can build machines that find land mines, tell fake from genuine signatures, diagnose disease, and so on …

How brains seem to do pattern recognition – more in lectures 17—18

The business end of this is made of lots of these joined in networks like this

Our own computations are performed in/by this network

The key idea in brain-inspired computing

The brain is a complextangle of neurons, connectedby synapses

The key idea in brain-inspired computing

When neurons are active, theysend signals to others.

The key idea in brain-inspired computing

A neuron with lots of `strong’active inputs will become active.

The key idea in brain-inspired computing

And, when connected neuronsare active at the same time, the link between them gets stronger

The key idea in brain-inspired computing

So, suppose these neurons happento be active when you see a fluffy animal with big eyes, small ears and apointed face

The key idea in brain-inspired computing

So, suppose these neurons happento be active when you see a fluffy animal with big eyes, small ears and apointed face

… and suppose your mother then says “Cat”, which excites this additional neuron.

The key idea in brain-inspired computing

Links will then strengthen betweenthe active neurons

So when you see a similar animalagain, this neuron will probably Automatically be activated, helpingyou classify it.

A slightly different group of neuronswill respond to dogs, and sometimesboth the “cat” and “dog” groupwill be active, but one will be moreactive than the other …

Notice this in particular

What happens if we damage a singleneuron (remember, in reality there will be thousands involved in simpleclassification-style computations)?

Compare this with damaging a line ofcode.

In classical computing we provide rules; but biology seems to learn gradually from examples.

Basic Notes on OptimisationWe have 3 items as follows: (item 1: 20kg; item2: 75kg; item 3: 60kg)Suppose we want to find the subset of items with total weight closest to 100kg.

Well done, you just searched the space of possible subsets. Youalso found the optimal one. If the above set of subsets is called S, and the subsets themselves are s1, s2, s3, etc …, you just optimised the function “closest_to_100kg(s)”; i.e. you found the s which minimises thefunction |(weight—100)| .

101000

100

010

Which is the best?110

111

011001

Search and Optimisation In general, optimisation means that you are trying to

find the best solution you can (usually in a short time) to a given problem.

S

s1 s2 s3 …

We always have a set S of all possible solutions

S may be small (as just seen)

S may be very, very, very, very large(e.g all possible timetables for a 500-exam/3-week diet)… in fact something like 1030 is typical for real problems.S may be infinitely large – e.g. all real numbers.

How Biology Optimises

We wish to design something – we want the best possible (or, at least a very good) design.

The set S is the set of all possible designs. It is always much toolarge to search through this set one by one, however we want tofind good examples in S. In nature, this problem seems to be solved wonderfully well,

again and again and again, by evolutionNature has designed millions of extremely complex machines, eachalmost ideal for their tasks (assuming an environment that doesn’tgo haywire), using evolution as the only mechanism.

Clearly, this is worth trying for solving problems in science and industry.

Quick overview of BIC techniques we will learn about

Evolutionary algorithms:Use nature’s evolution mechanism to evolve solutions to

all kinds of problems. E.g. to find a very aerodynamic wing design, we essentially simulate evolution of a population of wing designs. Good designs stay in the population and breed to, poor designs die out. EAs are highly successful and come in many variants. There is also a lot to learn to understand how to apply them well to new problems. We will do quite a lot on EAs. EAs are all about optimisation, however classification is also an optimisation problem, so EAs work there too …

A genetically optimized three-dimensional truss with improved frequency response.

An EA-optimized concert-halldesign, which improves on humandesigns in terms of sound qualityaveraged over all listening points.

Swarm Intelligence How do swarms of birds, fish, etc … manage to

move so well as a unit? How do ants manage to find the best sources of food in their environment. Answers to these questions have led to some very powerful new optimisation methods, that are different to EAs. These include ant colony optimisation, and particle swarm optimisation.

Also, only by studying how real swarms work are we able to simulate realistic swarming behaviour

(e.g. as done in Jurassic Park, Finding Nemo, etc …)

Artifical Life and Cellular AutomataThis is a research area that tries to learn what the fundamental

computational structures and processes are that are necessary for the things that seem to go hand-in-hand with life. For example Growth, and Reproduction. One of the fruits of Alife are simple rule-based systems called L-systems that can be used to simulate very lifelike images of plants, that are used in computer graphics. Meanwhile, Cellular Automata (CA) are very simple computational systems that produce very complex behaviour, including `lifelike’ reproduction. CAs, as we will see, are also very useful for explaining/simulating biological pattern generation and other behaviours

Neural Computing

Pattern recognition using neural networks is the most widely used form of BIC in industry and science. We will learn about the most common and successful types of neural network.

This is Stanley, winnerof the DARPA grandChallenge – a greatexample of bio-inspiredcomputing winning overall other entries, whichwere largely `classical’

Other BIC techniquesThere are many other BIC areas under research, but not yet

found as successful in practice as those we have concentrated on in the course. But we will look at the most prominent `other’ techniques. At the moment these are:

Artifical Immune System methods – which lead to algorithms for optimisation and classification based on the workings of the human immune system.

Foraging Algorithms – which lead to optimisation methods based on how herds of animals decide where to graze. These are different from the current main algorithms that have arisen from swarm intelligence.

Week 1 Self-Study & Quiz

Before we get into looking at Evolutionary Algorithms (as well asother methods that do optimisation), we need to understand certain things about optimisation, such as

• When we need clever methods to do it, and when we don’t• What alternatives there are to EAs – no point designing an EA for an optimisation problem if it can be solved far more simply.

So the additional material and associated quiz this week is about optimisation problems in general, and some key pure computer-science things you need to know.

The next lecture will then introduce evolutionary algorithms.