neuronal evolution and the origins of language: towards a simulation platform eörs szathmáry...

Neuronal evolution and the origins of language: Towards a simulation platformEörs Szathmáry

Eötvös University Collegium Budapest

The group• Zoltán SzatmáryZoltán Szatmáry programming, neuroprogramming, neuro• Péter IttzésPéter Ittzés programming, bioprogramming, bio• Máté VargaMáté Varga programming, elect. eng.programming, elect. eng.• Ferenc HuszárFerenc Huszár informaticsinformatics• Anna FedorAnna Fedor bio, etholbio, ethol• István ZacharIstván Zachar bio, evolbio, evol• Gergő OrbánGergő Orbán biophys, Bayesian learnbiophys, Bayesian learn• Máté LengyelMáté Lengyel neuroneuro• Szabolcs SzámadóSzabolcs Számadóbio, evolbio, evol

SUPPORTED BY ECAGENTSSUPPORTED BY ECAGENTS

It all started with JMS…

• „You know Eörs, we have to consider language seriously in the book”

• The origin of language remains the primary motivation behind this work

The major transitions (JMS & ES, 1995)

***

*

* These transitions are regarded to be ‘difficult’

Some general lessons drawn

• Emergence of novel inheritance system

• Holistic digital BEWARE!

• Limited heredity unlimited heredity

• Solution of the cooperation problem is needed

• Unlimited heredity allows CUMULATIVE selection

Unique transitions are difficult

– Genetic code– Eukaryotic cell– Eukaryotic sex– Language

• Objective and subjective difficulty

• Limitation by selection

• Limitation by genetic variation

Recruitment (predaptation) is fine, except it is unlikely to give optimal

solutions

Initial engulfment of bacteria, BUT…

Hundreds of mutations must have gone to fixation!!!

The ‘momentum’ of evolution

• IF a trait is useful (functional)

• AND IF there is genetic variation for it

• AND IF it is not perfect to start with,

• THEN we can expect (some) improvement through evolution by natural selection!

Three interwoven processes

• Note the different time-scales involved• Cultural transmission: language transmits itself as

well as other things• A novel inheritance system

Trends Ecol. Evol. (2006)

A critical examination of ideas

Theories/Questions 1 2 3 4 5 6

Language as a mental tool (Jerison, 1991; Burling, 1993) + + - + - -Grooming hypothesis (Dunbar, 1998) - + - - - -Gossip (Power, 1998) + - - + - -Tool making (Greenfield, 1991) + + + + + -Mating contract (Deacon, 1997) - - - - - -Sexual selection (Miller, 2000) + - - - - -Status for information (Dessalles, 2000) + - - + - -Song hypothesis (Vaneechoutte & Skoyles, 1998) - - - - - +Group bonding/ ritual (Knight, 1998) - + - - - -Gestural theory (Hewes, 1973) + - + + - -Hunting theories (Washburn & Lanchester, 1968) + + + + - -

(1) selective advantage (2) honesty (3) grounded in reality (4) power of generalisations (5) cognitive abilities (6) uniqueness

An educated guess

• The origin of language had to do possibly with a combination of – Language as a mental tool – Gesturing– Tool making– Hunting

The coevolutionary ladder

cooperation language

The evolutionary approachgenes

development

behaviour

selection

learning

environmentImpact of evolution on the developmental genetics of the brain!

The genetics of complex behaviour is not easy…

• Pleiotropy: one gene affecting different traits• Epistasis: effects from different genes do not combine

independently• Intermediate phenotypes must be identified!

One method of finding out (within ECAgents)

• Simulated dynamics of interacting agents• Agents have a “nervous system”• It is under partial genetic control• Selection is based on learning performance

for symbolic and syntactical tasks• If successful, look and reverse engineer the

emerging architectures• HOW GENES RIG THE NETWORKS??

The most important precedent

„the purpose of this paper is to explore how genes could specify the actual neuronal network functional architectures found in the mammalian brain, such as those found in the cerebral cortex. Indeed, this paper takes examples of some of the actual architectures and prototypical networks found in the cerebral cortex, and explores how these architectures could be specified by genes which allow the networks when built to implement some of the prototypical computational problems that must be solved by neuronal networks in the brain”

Highly indirect genetic encoding

• There are special results with direct genetic encoding (one gene per neuron or per synapse)

• THIS IS NOT WHAT WE WANT• There are around 35 thousand genes• Only a fraction of them can deal with the

brain• Billions of neurons, many more synapses

Summary of our efforts

In: Nehaniv, C., Cangelosi, A & Lyon, C. (2006) Origin of Communication, in press. Springer-Verlag

ECAgents: project founded by the Future and Emerging Technologies program (IST-FET) of the European Community under EU R&D contract IST1940.

Software architecture

519 classes

99267 lines of C++ code

ECAgents: project founded by the Future and Emerging Technologies program (IST-FET) of the European Community under EU R&D contract IST1940.

Population dynamics and agent lifecycle

ECAgents: project founded by the Future and Emerging Technologies program (IST-FET) of the European Community under EU R&D contract IST1940.

Ontogenesis of a neuronal network

ECAgents: project founded by the Future and Emerging Technologies program (IST-FET) of the European Community under EU R&D contract IST1940.

A note on the importance of topographicity

• For each tropographical net, one can construct an equivalent topological net

• The nature of variation is very different for the two options

• Genes obviously affect topographical networks

ECAgents: project founded by the Future and Emerging Technologies program (IST-FET) of the European Community under EU R&D contract IST1940.

Parameter values

Population dynamics and games• Population size: 100. • Time steps: 500 (200 for the cloning test). • Number of games played per time step per agent:

100. • Death process: least fit (5). • Mating process: roulette wheel. • Number of offspring: Poisson with Lambda=5.

ECAgents: project founded by the Future and Emerging Technologies program (IST-FET) of the European Community under EU R&D contract IST1940.

Parameter values 2

Neurobiological parameters• Number of layers: randomly chosen from the range [1,3]

(mutation rate: 0.008). • Number of neuron classes: randomly chosen from the range

[1,3] (mutation rate: 0.2). • Number of neurons: randomly chosen from the range [10,30]

(mutation rate: 0.2). • Number of projections: randomly chosen from the range [1,3]

(mutation rate: 0.02). • Rate coding with linear transfer function [-1 , 1]. • Hebbian learning rules. • Reward matrix is same as the pay-off matrix of the given game

(below). • Brain update: 10 (same for listener and speaker).

ECAgents: project founded by the Future and Emerging Technologies program (IST-FET) of the European Community under EU R&D contract IST1940.

There are:• two kinds of environments, E={-1,1},• three types of cost-free signals S=[-1, 1, else],• three types of possible decisions D=[-1, 1],

where values other than –1 or 1 mean no signal and no response respectively.

Task: A two-person game

ECAgents: project founded by the Future and Emerging Technologies program (IST-FET) of the European Community under EU R&D contract IST1940.

-1/1

Population

Environment

A Coordination Game

Speaker Listener

Decision

Signal

Decision

Different types of gameDifferent types of game

Coodination game (Coop)Coodination game (Coop) Division of Labour (Div)Division of Labour (Div) Prisoners’ dilemma (PD)Prisoners’ dilemma (PD) Hawk- Dove game (SD)Hawk- Dove game (SD)

Environment -1 Environment 1

Coop (-1) Coop (1)

Div Div

PD (-1) PD (1)

SD (-1) SD (1)

PD (-1) Coop (-1)

PD (-1) CoopRev (1)

SD (-1) Coop (-1)

SD (-1) CoopRev (1)

D(1) D(-1)

D(1) 1 5

D(-1) 0 3

D(1) D(-1)

D(1) -1 5

D(-1) 0 3

D(1) D(-1)

D(1) 0 5

D(-1) 5 0

D(1) D(-1)

D(1) 5 1

D(-1) 0 0

Coodination gameCoodination game Division of LabourDivision of Labour Prisoners’ dilemmaPrisoners’ dilemma Hawk - Dove gameHawk - Dove game

Div/Div

0

500

1000

1500

2000

2500

3000

1 101 201 301 401 501 601 701 801 901

time

de

cis

ion

s

S:[-1] E:[-1] DL:[-1];DS:[1]

S:[-1] E:[-1] DL:[1];DS:[-1]

S:[-1] E:[1] DL:[-1];DS:[1]

S:[-1] E:[1] DL:[1];DS:[-1]

S:[1] E:[-1] DL:[-1];DS:[1]

S:[1] E:[-1] DL:[1];DS:[-1]

S:[1] E:[1] DL:[-1];DS:[1]

S:[1] E:[1] DL:[1];DS:[-1]

Coop/Coop

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

1 101 201 301 401 501 601 701 801 901

time

dec

isio

ns S:[-1] E:[-1] DL:[-1];DS:[-1]

S:[0] E:[-1] DL:[-1];DS:[-1]

S:[0] E:[-1] DL:[1];DS:[-1]

S:[0] E:[1] DL:[-1];DS:[1]

S:[0] E:[1] DL:[1];DS:[1]

S:[1] E:[1] DL:[1];DS:[1]

PD/PD

0

200

400

600

800

1000

1200

1400

1600

1800

2000

1 101 201 301 401 501 601 701 801 901

time

dec

isio

ns

S:[-1] E:[-1] DL:[-1];DS:[1]

S:[-1] E:[-1] DL:[1];DS:[1]

S:[-1] E:[1] DL:[-1];DS:[-1]

S:[-1] E:[1] DL:[1];DS:[-1]

S:[0] E:[-1] DL:[-1];DS:[1]

S:[0] E:[-1] DL:[1];DS:[1]

S:[0] E:[1] DL:[-1];DS:[-1]

S:[0] E:[1] DL:[1];DS:[-1]

S:[1] E:[-1] DL:[-1];DS:[1]

S:[1] E:[-1] DL:[1];DS:[1]

S:[1] E:[1] DL:[-1];DS:[-1]

S:[1] E:[1] DL:[1];DS:[-1]

other-reporting signals self-reporting signals

dishonest signals uninformative signals

no signal

SD/SD

0

500

1000

1500

2000

2500

3000

3500

4000

4500

1 101 201 301 401 501 601 701 801 901

time

dec

isio

ns S:[-1] E:[1] DL:[-1];DS:[-1]

S:[-1] E:[1] DL:[1];DS:[-1]

S:[-1] E:[1] DL:[1];DS:[1]

S:[1] E:[-1] DL:[-1];DS:[-1]

S:[1] E:[-1] DL:[-1];DS:[1]

S:[1] E:[-1] DL:[1];DS:[1]

Why is there communication in SD/SD?

• There is conflict of interest in the game, BUT:• There is mixed ESS: it pays to be the reverse of

the opponent!• Speaker sees the environment, chooses the selfish

strategy and informs the listener about it in the „hope” that the other behaves complementarily. The other has no real choice but to „believe” in it.

• Mixed ESS AND changing environments AND informational asymmetry RESULT IN communication

other-reporting signals self-reporting signals

dishonest signals uninformative signals

no signal

PD/Coop

0

500

1000

1500

2000

2500

1 101 201 301 401 501 601 701 801 901

time

dec

isio

ns

S:[-1] E:[-1] DL:[-1];DS:[-1]

S:[-1] E:[-1] DL:[-1];DS:[1]

S:[-1] E:[1] DL:[-1];DS:[-1]

S:[0] E:[-1] DL:[-1];DS:[-1]

S:[0] E:[-1] DL:[-1];DS:[1]

S:[0] E:[1] DL:[-1];DS:[-1]

S:[1] E:[-1] DL:[-1];DS:[1]

S:[1] E:[1] DL:[-1];DS:[-1]

PD/CoopRev

0

500

1000

1500

2000

2500

3000

1 101 201 301 401 501 601 701 801 901

time

dec

isio

ns

S:[-1] E:[-1] DL:[1];DS:[1]

S:[-1] E:[1] DL:[1];DS:[1]

S:[0] E:[-1] DL:[1];DS:[1]

S:[0] E:[1] DL:[1];DS:[1]

S:[1] E:[-1] DL:[1];DS:[1]

S:[1] E:[1] DL:[1];DS:[1]

SD/Coop

0

500

1000

1500

2000

2500

3000

3500

1 101 201 301 401 501 601 701 801 901

time

deci

sion

s

S:[-1] E:[-1] DL:[-1];DS:[-1]

S:[-1] E:[-1] DL:[-1];DS:[1]

S:[-1] E:[1] DL:[-1];DS:[-1]

S:[0] E:[-1] DL:[-1];DS:[-1]

S:[0] E:[-1] DL:[-1];DS:[1]

S:[0] E:[1] DL:[-1];DS:[-1]

S:[1] E:[-1] DL:[-1];DS:[-1]

S:[1] E:[-1] DL:[-1];DS:[1]

S:[1] E:[1] DL:[-1];DS:[-1]

SD/CoopRev

0

500

1000

1500

2000

2500

1 101 201 301 401 501 601 701 801 901

time

dec

isio

ns

S:[-1] E:[-1] DL:[1];DS:[-1]

S:[-1] E:[1] DL:[1];DS:[1]

S:[0] E:[-1] DL:[1];DS:[-1]

S:[0] E:[-1] DL:[1];DS:[0]

S:[0] E:[1] DL:[1];DS:[1]

S:[1] E:[-1] DL:[1];DS:[-1]

S:[1] E:[-1] DL:[1];DS:[1]

S:[1] E:[1] DL:[1];DS:[1]

ECAgents: project founded by the Future and Emerging Technologies program (IST-FET) of the European Community under EU R&D contract IST1940.

Early brains (t:10) Scenario: E1: complementary, E-1:same

Visual input

Audio input

Const input or unconnected

Mixed colours indicate input mixing.

ECAgents: project founded by the Future and Emerging Technologies program (IST-FET) of the European Community under EU R&D contract IST1940.

Advanced brain (t:750)

Scenario: E1: complementary, E-1:same

Visual input

Audio input

Constants input or unconnected

Mixed colours indicate input mixing

ECAgents: project founded by the Future and Emerging Technologies program (IST-FET) of the European Community under EU R&D contract IST1940.

Is there inheritance, despite highly indirect genetic encoding?

• Scatter plots for AudioIn, AudioOut, Const, Vision and Decision neurons

• Experiments on clones

ECAgents: project founded by the Future and Emerging Technologies program (IST-FET) of the European Community under EU R&D contract IST1940.

The central issue with indirect encoding is whether one can find heritability of the simulated, evolved neuronal networks. If our biomimetic, indirect encoding is successful; this should be the case.

Measuring the Heritability of Neural Connections

in ENGA-Generated Communicating Agents

Input/output neuron

h2

AudioIn 0.8689

AudioOut 0.8708

Const 0.8696

Decision 0.8123

Vision 0.8428

Estimated heritability values (h2) of the number of connections of the given input/output neurons (right).

This is a proof that ENGA works as we hoped: despite indirect encoding, there is hereditary variation between indivudal phenotypes on which simulated natural selection can act.

ECAgents: project founded by the Future and Emerging Technologies program (IST-FET) of the European Community under EU R&D contract IST1940.

Is there inheritance, or only council of the elders?

• The increase with age of time• The code of individuals in time• Green lines: individual living still the end of the simulation• Red: birth events

ECAgents: project founded by the Future and Emerging Technologies program (IST-FET) of the European Community under EU R&D contract IST1940.

Details of learning/heritability experiment

• Individuals are taken from an equilibrated Coop game• All are newborn, no close relatives• Smart and stupid individuals are included• Individuals were educated in a testbed• You see the average of the reward received in 1010

turns• Convention carved into pieces: two environments x

two types of input (audio and visual), measure the signal or the decision

ECAgents: project founded by the Future and Emerging Technologies program (IST-FET) of the European Community under EU R&D contract IST1940.

A minimalist version of the naming game

• 2 objects• Agents have two individual „concepts” (bit strings of

length 2) • One agent signals the other if shown an object • Success of communication is measured in terms of

fitness• Learning is indispensable

ECAgents: project founded by the Future and Emerging Technologies program (IST-FET) of the European Community under EU R&D contract IST1940.

Flow chart of the naming game

Mother nature

Concept

Speaker visual

Signal ouput

Listener ouput

Decision

Concept?

ECAgents: project founded by the Future and Emerging Technologies program (IST-FET) of the European Community under EU R&D contract IST1940.

What is ENGA good for?

• To test (some) ideas about language evolutionary scenarios

• Are certain suggested preadaptation ideas better than others?

• Can you select for recursion? How?• Put the networks into robots!• A USER-FRIENDLY PLATFORM IS TO BE

RELEASED