ACT-R Workshop

John R. AndersonDaniel Bothell

Christian LebiereNiels A. Taatgen

Schedule of Events: 9:00-10:30: ACT-R from CMU’s Perspective 11:00-12:30: Architecture 1:30-3:30: Extensions 4:00-5:30: Future of ACT-R from a non-CMU Perspective

And lots of Interaction!

ACT-R Workshop Schedule

Opening: ACT-R from CMU’s Perspective9:00 - 9:45 Overview of ACT-R -- John R. Anderson9:45 – 10:30 Details of ACT-R 6.0 -- Dan Bothell

Break: 10:30 – 11:00Presentations 1: Architecture

11:00 – 11:30 Functional constraints on architectural mechanisms -- Christian Lebiere11:30 – 12:00 Retrieval by Accumulating Evidence in ACT-R -- Leendert van Maanen12:00 – 12:30 A mechanism for decisions in the absence of prior reward -- Vladislav D. Veksler

Lunch: 12:30 – 1:30Presentations 2: Extensions

1:30 – 2:00 ACT-R forays into the semantic web -- Lael J. Schooler2:00 – 2:30 Making Models Tired: A Module for Fatigue -- Glenn F. Gunzelmann2:30 – 3:00 Acting outside the box: Truly embodied ACT-R -- Anthony Harrison3:00 - 3:30 Interfacing ACT-R with different types of environments and with different techniques: Issues and Suggestions.-- Michael J. Schoelles

Break: 3:30 – 4:00Panel: 4:00 – 5:30: Future of ACT-R from a non-CMU Perspective Danilo Fum, Kevin A. Gluck, Wayne D. Gray, Niels A. Taatgen, J. Gregory Trafton, Richard M. Young

Overview of ACT-R

John R. AndersonCarnegie Mellon University

Outline:

9:10: Big picture of what ACT-R is about

9:20: Evolution of the Procedural Module

9:30: Evolution of the Declarative Module

9:35: How ACT-R spreads

ACT-R is Not Monolithic

1. It is a community brought together by common theoretical assumptions and a commitment to the “No Magic” Principle -- cognitive theory has to run and it has to predict data. While ACT-R may be sustained from CMU it no longer resides at CMU. The community motto is “Let a thousand flowers grow”

2. It is a set of software for purposes of simulation. This software consists of a core LISP implementation, but there are many theoretically-motivated extensions and alternative practicality-motivated alternative implementations. In some cases the software provides the best definitions of what the theoretical claims are.

3. It is a theory that attempts to formalize and operationalize certain aspects of our understanding of the human mind. This includes assumptions that are more core and those that are more peripheral. It changes as our knowledge grows and has different interpretations in different hands.

ACT-R: The Oldest Core Principles

1. The Procedural-Declarative Distinctiona. The declarative component originated in Anderson & Bower (1973)

HAM network representation of memory.b. The procedural component originated in Newell’s (1973) production

system theory of cognitive control.c. Both the procedural and declarative components have evolved far

from these origins.2. The Symbolic-Subsymbolic Distinction

a. In addition to the symbolic level that represented knowledge there is a subsymbolic level that controls access to that knowledge.

b. The subsymbolic level was initially designed to reflect the 1970s & 1980s ideas about neural processing.

c. Guided by rational analysis the subsymbolic level was updated in 1993 to reflected the likelihood that the information was useful. This was the birth of ACT-R.

Evolution from ACT-R 2.0 (1993) to ACT-R 6.0 (2007)

1. There were 3 driving forces:a. The emergence of a user community around the publicly available

ACT-R 2.0.b. The realization that the “No Magic” principle required that we be

able to model the processing all the way from input to output. c. The insistence on not making assumptions that could not be

cashed out into neurally plausible computations.2. This converged in the modular architecture of ACT-R 6.0:

a. The allowed community members to try variations on existing ideas and extensions but keep what they wanted.

b. We borrowed the modular organization of EPIC for the perceptual-motor modules.

c. There was growing evidence that, while the brain was a complex parallel machine, different regions had their specializations.

Modules are high capacity, parallel, and asynchronous

Modules in ACT-R 6.0

Production system that contains rules that recognize patterns and react

Buffers provide narrow paths of communication -- only hold a chunk in ACT-R terms.

Manual Vocal

Visual Aural

Imaginal Declarative

Goal

Procedural

ACT-R Module-Region Mappings

Outline:

9:10: Big picture of what ACT-R is about

9:20: Evolution of the Procedural Module

9:30: Evolution of the Declarative Module

9:35: How ACT-R spreads

The Procedural Component in ACT-R has Evolved from Computer Science Notation to Description of

the Brain’s Action Selection

600517- 23523 4

The First Real ACT-R Production RuleIf the goal is to process a column and the top digit is not smaller than the bottom digit,Then write the difference between the digits as the answer

Responds to a Particular Pattern that Appears in the Buffers of a Set of Modules

Which consists of requests to other Modules

Imaginal> Top: 7 Relation: >= Bottom: 3

Goal> Task: Process-Column

Declarative> Type: subtraction Minuend: 7 Subtrahend: 3

Goal> Task: Subtracting

RequestDifference

Selects anAction

The Second Real ACT-R Production RuleIf the goal is to process a column and the top digit is not smaller than the bottom digit,Then write the difference between the digits as the answer

Goal> Task: Subtracting

Declarative>Type: subtractionDifference: 4

Manual> Action: write Digit: 4

Goal> Task: Next-column

Harvest Difference

Responds to a Particular Pattern that Appears in the the Buffers of a Set of Modules

Which consists of requests to other Modules

Selects anAction

Attributes of Production Rules

Production rules are stimulus-response bonds that have “gone over to the cognitive side” because among the stimuli they respond to are past memories, mental images, and control states.

Respond to conjunctions of elements in the various buffers. These buffers can represent relational structures -- e.g. A

above B. Note how innocuous the use of variables is -- it basically

copying information from one brain region to another.

Stewart, T.C. and Eliasmith, C. (2008). Building production systems with realistic spiking neurons. 30th Annual Meeting of the Cognitive Science Society.

Stocco, A., Lebiere, C., & Anderson, J. R. (in revision). Conditional routing of information to the cortex: A model of the role of basal ganglia in high-level cognition. Psychological Review

New Problem Situations

Following instructions (e.g. Multicolumn Subtraction)

Declarative Representations

Requires Deliberation

Analogy to Prior Experiences (e.g. Past Tense Model)

Interpreted

New Production Rules

EventuallyProduces

Production Compilation

Traces Feed Into

Learning of New Production Rules

DeductionFrom 1st Principles

Imaginal> Relation:

Goal> Task: Process-Column State: Imaginal

RetrieveOperator

Retrieval> Type: operator Pre:

Imaginal> Top: Bottom:

PerformSubtraction

Retrieval> relation: subtract arg1: top arg2: bottom post:

Retrieval> type: subtraction minuend: subtrahend:

Goal> Task:

StateFeature

StateFeature

ActionIdentity

ObjectReferent

ObjectReferent

ActionType

Operator

Pre

Post

arg2

Arg1

Action

Type

Op11-1

Top >= Bottom

Subtract

Retrieve

Bottom

Top

Subtracting

Imaginal> Top: Relation: >= Bottom:

Goal> Task: Process-Column

RequestDifference

Retrieval> Type: subtraction Minuend: Subtrahend:

Goal> Task: Subtracting

Origin of One of the Subtraction Rules

Production compilationcompresses general-purpose processing of knowledge into special case rules -- replacing deliberation by action.

Reinforcement of Competing Productions

Retrieve-Instruction (Reinforcement 10)If the goal is to process a columnThen retrieve an operator for that kind of column

Request-Difference-Subtract (Reinforcement 14)If the goal is to process a column and the top digit is not smaller than the bottom digit,Then subtract the bottom from the top

Request-Difference-Wrong (Reinforcement 14 or 0)If the goal is to process a columnThen subtract the smaller from the larger

Request-Difference-Borrow (Reinforcement 14)If the goal is to process a column and the top digit smaller than the bottom digit,Then add 10 to the top digit and set as a subgoal to borrow from the column to the left.

Utility Learning for Competing Productions

00 .10 .20 .30 .40 .50 .60 .70 .80 .9102 55 07 51 0 0E x p e rie n c e sIn s tru c tio nS u b tra c tB o rro wW ro n g

€

U i(n) =U i(n −1) +α [Ri(n) −U i(n −1)]

Considerable simplification of ACT-R utility learning based of reinforcement-like learning results from the basal ganglia

0

2

4

6

8

10

12

14

0 25 50 75 100Experiences

InstructionSubtractBorrowWrong

€

Pi =eUi / s

eUj / s

j

∑Standard ACT-R soft-max rule for choosing among productions according to their noisy utilities0

0.10.20.30.40.50.60.70.80.9

1

0 25 50 75 100

Experiences

InstructionSubtractBorrowWrong

Every time a rule created it is rewarded with the utility of its parent

Outline:

9:10: Big picture of what ACT-R is about

9:20: Evolution of the Procedural Module

9:30: Evolution of the Declarative Module

9:35: How ACT-R spreads

What has Happened to the Declarative Component in ACT-R?

It has bifurcated into two completely separate things:

1. An increasingly watered-down set of principles for the representation of knowledge, which comes to be the contents of module buffers. This is clearly a place where important new thinking is required.

2. An increasingly empirically well-founded set of principles (with a foundation in rational analysis) for how the brain performs controlled retrieval of information from declarative memory.

Buffers and Declarative Memory Buffers associated

with modules provide narrow paths of communication.

The contents of the buffers are called chunks.

Records of these chunks are placed in declarative memory.

These can be later retrieved and placed in the declarative buffer.

Manual Vocal

Visual Aural

Imaginal Declarative

Goal

Procedural

Chunk Activation Reflects Probability of Use

€

Log(Posterior(i |C)) = Log(Pr ior(i)) + Log(Likelihood( j | i)j∈C

∑ )Environmental Equation:

€

Ai = Bi + W jS ji

j∈C

∑Activation Equation:

Base-level Activation of memory i Association Strength from j to i

Posterior odds that memory i will be needed in context C

Likelihood ratio of element j in context given i is needed

Prior odds that i is needed: recency and frequency

Momentary Activation of memory i Weighting of Source j

Fan Experiment: Pirolli & Anderson (1985)

€

S ji = S − ln(Fan)

Growth of Activation

Ai = Bi + ΣWj S ji

0

0.5

1

1.5

2

2.5

1 2 3 4 5 6 7 8 9 10 Days of Practice

Activation Level1-1 Fan3-1 Fan3-3 Fan

Act

ivati

on L

evel

Recognition Latencies

0

200

400

600

800

1000

1200

1400

1 2 3 4 5 6 7 8 9 10

Days of Practice

Recognition Time (ms.)

3-3 Fan3-1 Fan1-1 FanSeries4Series5Series6

Reco

gnit

ion T

ime

(ms.

)

r = .986 is a parameter-free measure of the match between theory and data.

€

Time = I + Fe−A i Re trieval Time Equationintercept latency scale

Outline:

9:10: Big picture of what ACT-R is about

9:20: Evolution of the Procedural Module

9:30: Evolution of the Declarative Module

9:35: How ACT-R spreads

Temporal Module: An Example of How One Can Extend ACT-R

PacemakerPacemaker GateGate

StartSignalStart

Signal

AccumulatorAccumulator

MemoryMemory

ComparisonComparison

MatchingMatching

SelectionSelection

ExecutionExecutionPro

du

ctio

ns

Pro

du

ctio

ns

Declarative ModuleDeclarative Module

Visual ModuleVisual Module Manual ModuleManual Module

External WorldExternal World

Retrieval BufferRetrieval Buffer

Manual BufferManual BufferVisual BufferVisual Buffer

Goal BufferGoal Buffer

Problem BufferProblem BufferPacemakerPacemaker GateGate

StartSignalStart

Signal

AccumulatorAccumulator

Other Module Extensions for ACT-R

Salvucci’s Emma Module for Eye Movements.

My new Metacognitive Module.

Spatial Modules (Gunzelmann, Harrison & Trafton).

Fatigue Module (Gunzelmann) ????

Reasoning Module LarKC (Schooler)????

Module ModificationsSNIF-ACT (Fu & Pirolli): Procedural and Declarative.

Threaded Cognition (Salvucci & Taatgen): Goal

Spacing Effect (Pavlik): Declarative.

Blending (Lebiere): Declarative.

Race/A (van Maanen & Van Rijn): Declarative

Visual Saliency (Byrne): Visual.

Gray, Veksler, & and others of the RPI Co: Procedural.

Bothell & Leabra: Visual.

You Don’t Need to Change ACT-R to Have an Interesting Model

Fum & Stocco: Sugar Factory

Lebiere, Wallach, & Taatgen: Sugar Factory

Altmann & Trafton: Tower of Hanoi

Lewis & Vasishith: Parsing

Taatgen: Acquistion Past Tense Model

Anderson (2007) & Everybody (recently): Everything in fMRI

And indeed most of the published ACT-R models.

Getting ACT-R out of the Narrow Confines of Laboratory Experiments

Best & Lebiere: MOUT

St. Amant & Ritter: Segman

Bothell, Douglass, Lee: Unreal Tournament

Harrison & Trafton: Robotics

Destefano: Space Fortress

Schoelles: Lots of Interfaces

ACT-R is Not Monolithic

1. It is a community brought together by common theoretical assumptions and a commitment to the “No Magic” Principle -- cognitive theory has to run and it has to predict data. While ACT-R may be sustained from CMU it no longer resides at CMU. The community motto is “Let a thousand flowers grow”

2. It is a set of software for purposes of simulation. This software consists of a core LISP implementation, but there are many theoretically-motivated extensions and alternative practicality-motivated alternative implementations. In some cases the software provides the best definitions of what the theoretical claims are.

3. It is a theory that attempts to formalize and operationalize certain aspects of our understanding of the human mind. This includes assumptions that are more core and those that are more peripheral. It changes as our knowledge grows and has different interpretations in different hands.

Be Fruitful and Multiply!

(p. 12 Architecture of Cognition, 1983) 2007