ics architecture (25 slides)

Cognitive Science 101 Cognition is computation But what type? Fundamental question of research on the human cognitive architecture

May 7, 2003

1

University of Amsterdam

Cognitive Architecture Rules operating on symbols grammar logic

Spreading activation in simple processors massively interconnected in a large network

Symbolic vs. Connectionist architecture? Integrated Connectionist/Symbolic (ICS) Architecture Grammar: Phonology

Architecture: defined by four components

May 7, 2003

2


dog+ s GA

dgz

The ICS Architecture

Representatio Grammar Function Algorithm A n G Constraints: optimal: k k NO{ODA C z t y t x optimal: x Activation Connection Harmony Spreading Processing Pattern Weights Optimization Activation (Learning)W (X B)5 4

dogs

W

3

2

1

0

0

1

2

3

4

5

6

7

8

9

Depth- 1 Unit s

May 7, 2003

3


Processing I: Activation Computational neuroscience Key sources Hopfield 1982, 1984 Cohen and Grossberg 1983 Hinton and Sejnowski 1983, 1986 Smolensky 1983, 1986 Geman and Geman 1984 Golden 1986, 1988May 7, 2003 4 University of Amsterdam

Processing I: ActivationCompetitive Net0.2

a10

-0.2 -0.4 Harmony -0.6 -0.8 -1 -1.2 -0.5 -0.2 0.1 a2

a 2 (i 2 = 0.5) a 2 (i 2 = 0.5)

(0.9)0 -0.5 1 1 0.5

a2

0.3 0.2 0.1 0 -0.1 -0.2 -0.3 0

da1 = a1 + i1 a2 dt0.2 0.4 0.6 0.8 1

Competitive Net

a1

0.4

0.7

a 1 (i 1 = 0.6)

i1 i2 (0.6 (0.5 Processing spreading ) ) H (aactivation2 a1a2 ) = a1i1 + a i2 is optimization:2Harmony (a1 + a22 ) maximization

Competitive Net1 0.8 0.6 0.4 0.2 0 -0.2 1 -0.4 -0.4 Time

a1

H

H = a W a

a2

May 7, 2003

5


catGA

kt


Representatio Grammar Function Algorithm A n G Constraints: optimal: k k NO{ODA C z t y t x optimal H:W (X B)5 4

W

3

2

1

0 0 1 2 3 4 5 6 7 8 9

Depth- 1 Unit s

Activation PatternMay 7, 2003

Connection Harmony Weights Optimization6

Spreading ActivationUniversity of Amsterdam

Processing II: Optimizationa1 (0.9) Cognitive psychology Key sources: a2 Hinton & Anderson 1981 Rumelhart, McClelland, & the PDP Group 1986

May 7, 2003

i1 i2 (0.6 (0.5 ) ) Processing spreading activation is optimization: Harmony

Competitive Net0.2 0 -0.2 -0.4 Harmony -0.6 -0.8 -1 -1.2 -0.5 -0.2 0.1 0.4 0.7 1

0 -0.5

1 0.5

a1

a2

7


a1 and a2 must not be simultaneously active (strength: a1 a2 ) (0.9)

Processing II: OptimizationHarmony maximization is satisfaction of parallel, violable well-formedness constraints a2 must be active (strength: 0.5)Competitive Net0.2 0 -0.2 -0.4 Harmony -0.6 -0.8 -1 -1.2 -0.5 -0.2 0.1 0.4 0.7 1

a1 must be active CONFLIC T (strength: 0.6) i1 i2 (0.6 (0.5 ) ) Processing spreading Optimal 0.21 activation is compromis 0.7 9 optimization: Harmony e:May 7, 2003

0 -0.5

1 0.5

a1

a2

8


Processing II: Optimization The search for an optimal state can employ randomness Equations for units activation values have random terms pr(a) eH(a)/T T (temperature) ~ randomness 0 during search Boltzmann Machine (Hinton and Sejnowski 1983, 1986); Harmony Theory (Smolensky 1983, 1986)May 7, 2003 9 University of Amsterdam

catGA

kt


Representatio Grammar Function Algorithm A n G Constraints: optimal: k k NO{ODA C z t y t x W (X B)5 4 3 2 1

opt.constr. sat.:5 6 7 8 9

W

0 0 1 2 3 4 Depth- 1 Unit s


Connection Weights

10

Harmony Spreading Opt./ Activation Constraint Satisfaction


Two Fundamental Questions Harmony maximization is2. What are the constraints?Knowledge representation

satisfaction of parallel, violable constraints

Prior question:

1. What are the activation patterns data structures mental representations evaluated by these constraints?May 7, 2003 11 University of Amsterdam

Representation Symbolic theory Complex symbol structures Generative linguistics (Chomsky & Halle 68) Particular linguistic representations

Markedness Theory (Jakobson, Trubetzkoy,

30s ) Good (well-formed) linguistic representations

Connectionism (PDP) Distributed activation patterns

ICS realization of (higher-level) complex symbolic structures in distributed patterns of activation over (lower-level) 12 University of Amsterdam units

May 7, 2003

Representation k t

{fi / ri } iActivation patterns: cat and its constituents

i fi ri

/r k/r0 /r01 t/r11 [ k [ t]]May 7, 2003

-1

4

9

14

Unit (Area = activation level)

13


catGA

kt


Representatio Grammar Function Algorithm A n G Constraints: optimal: k k NO{ODA C z t y t x opt.constr. sat.:W (X B)5 4

W

3

2

1

0 0 1 2 3 4 5 6 7 8 9 Depth- 1 Unit s


Connection Weights14

Harmony Opt./ Constraint Sat.


Constraints k cat t NOCODA: A syllable has no coda [Maori] * H(a[ k[ t] ) = sNOCODA < 0 W

*

violation

a[ k[ t]]

*15 University of Amsterdam

May 7, 2003

catGA

kt


Representatio Grammar Function Algorithm A n G Constraints: optimal: k k NO{ODA C y t x t opt.constr. sat.:W (X B)5 4

W

3

2

1

0 0 1 2 3 4 5 6 7 8 9 Depth- 1 Unit s





catGA

kt

Constraint Interaction ??Representatio Grammar Function Algorithm A n G Constraint optimal: k k s: NOCODA y t x tW (X B)5 4 3 2 1 0 0 1 2 3 4 5 6 7 8 9 Depth- 1 Unit s


opt.constr. sat.:

W



Harmony Opt./ Constraint


Constraint Interaction I Harmonic Grammar Legendre, Miyata, Smolensky 1990 et seq.

May 7, 2003

18


Constraint Interaction I The grammar generates the representation that maximizes H: this best-satisfies the constraints, given their differential strengths

H

k t

=H

H(k , > 0 H(, )ONSETMay 7, 2003

t

)< 0NOCODA

Any formal language can be so generated.

ij H (ci ,cj )19

= a W aUniversity of Amsterdam

=

catGA

kt

Constraint Interaction I: HGRepresentatio Grammar Function Algorithm A n G Constraint optimal H: k k s: NOCODA x t tW (X B)5 4 3 2 1


opt.constr. sat.:0 1 2 3 4 5 6 7 8 9 Depth- 1 Unit s

W

0





catGA

kt


Representatio Grammar Function Algorithm A n G Constraints: optimal: k k NOCODA t t

?

W (X B)5

4

3

2


W

1

0 0 1 2 3 4 Depth- 1 Unit s





catGA

kt

The ICS Architecture HG Powerful: French syntax

(Legendre, et al. 1990

P & Smolensky (1991 et seq.) Too powerful? rinceet seq.) Representatio Grammar Function Algorithm A n G Constraints: optimal: k k NOCODA t t

W (X B)5

4

3

2


W

1

0 0 1 2 3 4 Depth- 1 Unit s





catGA

kt

Constraint Interaction IIRepresentatio Grammar Function Algorithm A n G Constraint optimal: k k s: NOCODA k t t t


W (X B)5

4

3

2

1

0 0 1 2 3 4 5 6 7 8 9 Depth- 1 Unit s

opt.constr. sat.:

W





catGA

kt

Constraint Interaction II: OTRepresentatio Grammar Function Algorithm A n G Constraint optimal: k k s: NOCODA t tW (X B)5 4 3 2 1 0 0 1 2 3 4 5 6 7 8 9 Depth- 1 Unit s


CandidatesW

opt.constr. sat.:




a. H Spreading ActivationUniversity of Amsterdam

Constraint Interaction II: OT Strict domination Grammars cant count

Stress is on the initial heavy syllable iff the number of light sS syllables n obeys number H n< = any No wayTRESS EAVY

Ca id nd aUniversity of Amsterdam

sM AIN STRESSRIGHT

May 7, 2003

25

Intro to OT

May 7, 2003

26


catGA

kt


Representatio Grammar Function Algorithm A n G Constraint optimal H: k k s: NOCODA x t tW (X B)5 4 3 2 1

opt.constr. sat.:0 1 2 3 4 5 6 7 8 9 Depth- 1 Unit s

W

0





ics architecture (25 slides)

Documents