. P-er

13"It

Ilienformation Processing

and Memory

Edward A. Feigenbaum 1

M. !/]/()/■(/ ( 4)/\C/s//\

Introduction ■'■'> 'Information Processing and Mcmonj -/-'J-j

A/i Information Processing Tlicorij oj Mcmonj .... ■/">/Fiirllu-r Considerations about Inuiirtliiitc Mcmonj "/"'""<further Considerations Concerning Acijiiisitiou

■/.-.rMemorij ''""''77ic /'.oiig-'l'erni Permanent Store I(i(>C.cncral Considerations 'WiProcesses '4'Structure I(}(

liclriecal from the Long-Term Store hi/ Problem

Sol,

ing ....'.('/inclusion ''■"

licfercuces ">'s

Introduction

FPAM (Flemcnlarv Perceiver and Memorize.*) is one of a class ofconiputcr-sinudation models of cognitive processes that have been de-veloped in the last decade. These- are models o\' human informationprocessing in certain learning and problem-solving tasks. This chapteris not intended as a

survey

ol this literature. The reader svho svishes lo

become acquainted svith a svicle variety of research projects in this area

is advised to seek out the book Computers and Thought (Feigenbaumand Feldman, 1963).

In ibis chapter, I shall lirsl sketch brielly the history of the FPAM

project, wit hoi it svhich Ihe remainder ol the discussion is not

very

mean-ingful.

\'e\l,

I will atliMi.pt to reinterpret the FPAM theory in terms ol

■ Ih.s ,

l,,

M ,|r, is

,r

(

HU.I,d Iron,

(lie Proa-clings ol the I ifl!, beiLelei/ Si/mposium on

\tntlii-muti,

al

Slalisti, s

ami Piolmbil ill/. Volume IV. biologi/ ami 1 1c„l ill . I'lt><>. ,Un UI.-\ .1,,,\ <IM U ol Cidilumi.i I'i.-ss. I'IC(4. | I In- Mi^rslnm ..1 lln

■ ;mlli.

n ,n.< I \n ill.

«"<

.ns.-nl ..|

nl-. ivl lln- ' !

ni\

iiMlx "I < '.ililiioii.i4r. I

\

ii

Iffi!/"Ii1!i'J_j

m

f

,i

m

mami'insWW,

452 EDWARD A. FEIGENBAUM 13: I

an emerging three-level theory of human memory. In the remainder ofthe chapter, I would like to explore some questions relating to a theoryol human long-term associative

memory.

Work on the various FPAM models' began mans scars ago. The- re-search has always been a joint effort by myself an'd'Professor HerbertA. Simon ofCarnegie Institute of Technology . We have been concernedwith modeling the information processes and structures that underliebehavior in a wide variety of verbal learning tasks. These include thestandard serial and paired-associate learning tasks, and other not-so-standard verbal learning tasks.

Through the refer-tile various FPAM niwould be well ads isjargon and notation h

languages, especially

In

Though the quest fchology itself, it hasand success in the la*tication in the art ofcneural processes andit is due also to the d-theory of the excitingination encoding andis the result of the itspread acceptance of,scribing cognitive pr<theory. Such terms a>

processing strategy arcthose psychologists ss Icessing theorists.

FPAM I was a very simple model—so simple, in fact, that a mathe-matical lonnulation, as well as a computer .simulation, was constructed.In FPAM I, we postulated a serial mechanism in which the learning ofverbal materials took a nontrivial amount of time. We explored diestrategies used by subjects to organize their total learning task. The'>"><lcl generated an accurate quantitative prediction of the well-knownbowed serial error curve, which plots the percentage of total errorsmade by subjects at each serial position of a relatively long seriallv-prcsented list of words.

FPAM II was ;. much more comprehensive model. || specified struc-tures in which memorized items are stored and retrieved. It specifiedprocesses for learning discriminations between items; for learning as-sociations among items; and for the familiarization of new items. ||specified mans other processes lor (among

many

things) responding," l "<'"!ion locus, ,,g, and the analysis of environ.i.e.. lai' feedback. Themodel generated qualitative and quantitative predictions for more than:i (,(,/( '" ( ,h< ' 4u Tad and well-known phenomena of human verballearning. FPAM I I ;4 described in Feigenbaum (1963) and also bsX'evvell and Simon ( 1963).

Tl"' FPAM 111 model was a reconceptualizalion and generalization«>f the processes and structures of FPAM 11. It attacked the problem ofbuilding up very general associative structures in

memory

(other thanword learning); the association of a familiarized stimulus , an arbitrarynumber of associative contexts: the construction and storage of internalrepresentations of new stimulus experiences by recognizing and bring-ing together already learned internal representations (lhaMs, ah'ea.h'"'miliar experiences). \\ ill, (his model, we made certain additional pre,1i,,1,n,s i ' 1,,, " l lll( ' «-lle«-ls of similarity, lam, hank and mea ,, mgl v Inessi" verbal learning tasks. Those p.ediotinns. as uell as a brief description'*- I'- I ' AM HI. are contained in Simon and Feigenb, ( |')6|).

This attempt to brirmemory has been tw<]ol memory processes aage structures that nu'iscuts the elements ofcorporating an integr;processes and

inform;

consists of inlorniatiorjormance in paired-as>of the FPAM perforiua;bom the

memory

shinmajor learning proces.-iariz.ation. The formelearned and those aire;to incorporate tests oncan take place with a in

i

13: INFORMATION PROCESSING A MEMORY .">:

Through the references cited, the reader can pursue the structure ol

the various EPAM models to the depth motivated by his interests, lie

would he well advised to he prepared to accept and understand the

jargon and notation of the nonnumeric symbol-manipulating computer_an<ma<'es, especially those that deal with listprocessing.

Information Processing and Memory

Though the quest for an adequate theory of memory is as old as psy-chology itself, it has been pursued with significantly increased vigorand success in the last decade. This is due in part to increased sophis-tication in the art of conducting experiments on memory, at the level of

neural processes and at the level of psychological processes, In part,it is due also to the desire to explore possible implications for memory

theory of the exciting developments in the theory of biological infor-

mation encoding and storage mechanisms. And, in part, this new vigor

is the result of the introduction in the I9s()'s. and subsequent wide-spread acceptance of. a new vocabulary of terms and concepts lor de-

scribing cognitive processes: the language of information processingtheory.' Such terms as buffer storage, coding, retrieval processes, and

processing strategy are familiar and commonly used labels, even among

those psychologists who do not think of themselves as information pro-cessing theorists.

This attempt to bring a reasonable amount of order to the study ol

memory has been two headed: the search for an adequate descriptionof memory processes and the search for models of the information stor-

age structures that might be involved. This section of the chapter pre-sents 'the elements of an information processing theory of memory in-

corporating an integrated set of hypotheses about both informationprocesses and information structures of memory. The KIV-Wl modelconsists of information processes and structures lor learning and pcr-[nrnnincc in paired-associate and serial verbal learning tasks. The jobof the KI'AM performance processes is to retrieve appropriate responses| mm the memory structures when the task so dictates. Kl*A\l has two

major learning processes: discrimination learning and stimulus familiari/ation. The former discovers dilfereiices hetsseen items being

learned and those already learned, and builds up the mentors sti.iclui.

|o mco. potato tests on these dillercnoes. so that storage and lehieva!,,,, lake place w ith a minimum oi si imu his genera I i/al nm ai id « oi.hisi,

>.

.

J VVA A t AU\'

The latter builds internal representations, called images, of verbal itemsbeing learned. It is an integrative process, in which previously famil-iarized parts of a stimulus item are first recognized and then ""assembled"(according to a strategy) to form the internal representation. As pre-viously mentioned, the EPAM model also contains a number of othermechanisms for attention focusing, organization of the learning task,associative recall, and so forth, which will not be -discussed here.

EPAM. as it stands, is a psychological theory of certain elementarycognitive processes, framed at the so-called information processinglevel. The primitives at this level are primitives concerning elementarysymbol manipulation processes. These primitives are not. at this stageof our knowledge, directly translatable into "neural language," that fs,statements about how the processes are realized in the underlying neuralmachinery. Some fruitful conjectures about this have been made (Prib-ram. Ahumada, Hartog, and Koos, 1964). however, and more can beexpected as increasing confidence in the adequacy of the informationprocessing theory is attained.

\n Information Processing Theon/ of Memonj

We proceed now to summarize elements of a theory of memory thatthe work on EPAM has suggested to us. That part of the presentationdealing with permanent storage is. to some extent, conjectural, sincethese mechanisms have not been precisely defined or "rigorously ex-plored, though they have been suggested by shortcomings in EPAM.I'l'lVll |IV i: COS II I. VI KS \IiOII I'KOCI.SS

{;l) At (,lr inlormation processing level, the central processing mecha-nism is essentially serial, that is, capable ol performing one. or at mosta v erv lew. elemental*, processes at any time.

(,)) ■'■" H ''« elenH'iitarv information process takes lime to perform. To(i,nA OM( '' s<'"'es of processes requires ||,\ (a>| an amount of tune loiighlvequal to the sun, of process ing t i mes of t he const iliient processes. Esei.I()1 Simple psychological processes, this processing time

may

besignilicantlv long when eoiupaied with, for example, item piesenlal ion11,,,s encountered m verbal learning e\pei i menls.

1 ■" M

'

|UM fundamental ideas .<" at the mol o| all 141' \\l pi ediel ions.MS ,S

' ,N

*

mr «'■ ,<' earliest model ■ || M f p,ed|e|ed ,, M |\ the se.ul

455

position effect (Eeigenbaum and Simon, 1962) as it is of the later, more 1

comprehensive, models (Simon and Eeigenbaum, 1964). Neither we norothers has been able to construct an alternate basic formulation thatachieves the same results. Postulate (b) we interpret as identical withthe "consolidation hypothesis" suggested by McGaugh (.1965) and otherson the basis of laboratory experiments with animals using electrocon-vulsive shock and various drug treatments.

The consolidation hypothesis is an empirical generalization. TheEPAM theory generates complex and accurate predictions of verballearning behavior on the basis of an identical postulate1 inferred froman entirely different empirical base 1 . Taken together, they provide strongconfirmation for the basic correctness of the position.

lIVI'OTIIKSKS AMOI'T STIUCITHK

We hypothesize three 1 type's of information storage structures(a) An immediate 1 memory: a buffer storage1 mechanism of extremely

small size 1, holding a few symbols. Inputs from the peripheral sensingand encoding mechanisms are 1 held here in a state 1 of availability forfurther central processing. The immediate memory provides the 1 only

channel of communication between the 1 central processes and the1

sensing processes at the 1 periphery. Central processes may use the im-mediate memory for temporary storage of internally generated symbols:

these then compete for storage with arriving input symbols. The 1 netresult ol such an immediate memory mechanism is that the total pro-cessing system has a very narrow "locus of attention, that is, the centralprocesses can attend to only a minuscule portion of the external stimulusenvironment at any time.

(b) Ae< juisition memory: the 1 term is chosen to contrast with long-termpermanent store [see (c), below |. It refers to a large 1 working memory

for discrimination and familiarization processes. In it are built theinternal representations of stimulus objects being learned, ln itsstructure is stored the information necessary to discriminate amongthe learned objects. This memory has a tree structure, called the dis-crimination net. At each nonterminal nodal level is stored a testing pro-cess, a discriminator, which tests some feature ol a stimulus object andsoils the object along the appropriate branch to the next nodal level.I he termini ol the tree are storage locations at which images, assem

41

,^»

blages ol sv in bo Is const it lit ing the i nteinal re pi c sen tat ioi i ol an externalobject , ale stored.

1

as

in

I

Itmi4'jA

4T111rH

i1

; ife-

Ii'Mi

-:<ix

%.-.■"lfmm?§:MV.I

:.JS-

■zir.:

%1

':/.«

13: INFORMATION PROCESSING AND MEMORY

"su

'M

1 EDWARD A. FEIGENBAUM456

!

!

I !

I1

I

i

It is this memory structure upon which most of the1 EPAM work hasbeen done, and whose structure is best understood.

(c) The permanent store: this memory structure considered for prac-tical purposes as being of essentially unlimited size, is the long-termpermanent repository of the images. The images are linked together ina highly interconnected web of cross associations. Thus, the1 structureof this memory is not treelike. However, it is plausible that this structureis "indexed" by a discrimination net like the one described above, forefficient cross referencing and searching.

HYPOTHKSKS ABOI'T PROCESS

The1 theory contains a number of hypotheses about memory-processingactivity, only a tew of which will be1 summarized at this point. Otherswill be 1 touched on in the1 discussions below.

(a) Working at the level of the acquisition memory, a matching pro-cess scans stimulus encodings and images serially for differences onthe 1 basis of which discriminators are1 constructed. "The 1 scan is controlledby a noticing order, an adaptive attention-focusing strategy.

(b) Image1 building in the 1 acquisition memory consists of assemblingat a terminal node 1 in an orderly wav (that is, controlled by a strategy)cue-tokens, which reference other images in the net.

(c) The discrimination net of the acquisition memory over time iselaborated (that is, necessary discriminators and branches are grown)as the task demands finer discriminations for siiccesslul perlormanee.The discrimination 11(4 is grown in a wholly pragmatic manner, itsgrowth at any stage reflecting what is just adequate lor correct per-formance. There is no attempt at this level to structure or restructurethe net lor efficiency or logical order.

(d) At the level ol the permanent storage, it is hypothesized that aprocess transfers images, discrimination information, and perhaps evensubnets ol the acquisition memory to the permanent store, dismantlingthe structure of the working memory as it processes it. The transferredinformation is reorganized and tied into the web structure ol the perma-nent store according to an organizational scheme which is more logicaland belter suited to the long-term retrieval needs ol the organism thanthe pragma! icallv built structure ol the acquisition nieinoiv.

Having thus siiniiuai i/e< I our basic hypotheses about stiiicluie andprocess m a three level iiieiuorv. we proceed to desciibe and discusseach o! I hese lev els il l n ioi c dela 11 .

13: INFORMATION PROCESSING AND MEMORY 457

:;>i

_'Further Considerations about Immediate Memory

Our theory holds that the immediate memory is a fast-access, low-capacity storage system whose function is primarily to buffer encodedsensory inputs. We conceive of the 1 immediate memory as being ultra-dvnamie, the average .ength of time of residence oi' a symbol thereinbeing of the order of seconds, although the stay can be extended undercontrol of a central process by recycling. At any given moment of time,the set of symbols in the immediate memory is the operational stimulusenvironment of the organism. This position is consistent with and con-tributory to our fundamental postulate that the central processes arebasically serial.

The need for such a buffer storage mechanism is twofold. First, sincethe 1 performance and acquisition processes consume a significantamountof time, it is necessary to hold on to the inputs lest they vanish beforeany processing can be done on them. Second, buffer storage provides a

necessary decoupling of the central processes from the peripheral pro-cesses sensing the external environment. This decoupling, relievingthe1 central processes o\' the impossible burden of instant-by-instantattention to the environment, is absolutely essential because many ofthe time-consuming acquisition processes are searching or, what isworse, manipulating memory structure and cannot be 1 interrupted atarbitrary times.

II is interesting to note 1 in passing that in large 1 nonbiological infor-mation-processing machines (any of the modern familiar digital com-

puter systems), buffered data channels are used for reasons identicalwith the decoupling argument given above. We cannot here explainthe difficulties encountered in operating the early digital computersystems without buffered information transfer. No modern computersystem of which we are aware is built without some input/outputbuffering.

further Considerations Concerning .\c(jaisit ion Memory

The acquisition memory is conceptualized as an intermediate 1 levelol storage between the ultradvnamic immediate memory and the telativelv slow Iv changing permanent store It is the "working niemor.

in which the discrimination and laini I iai i/af ion processes classify, andhnild internal representations 01. the current environmental context

CS

IS.■A*«i»

1

I111iv.

ii.

m#mm"'$

\i■Si

%I7*/8IM

I■:■&■in

m;mi"ii!an

p;h

'Qsi

Wlot.

1mM;WmI'.in

r ffr-t

a_

i

and the objects thereof. Performance of "current task" is done1 by

referencing the 1 net; images are 1 not stored in this memory indefinitely,

but rather, for times of the1 order of many minute's to a few hours.It follows that since the information in- the acquisition memory is

not vet permanently fixed, and since this memory level stores the re-cent, v learned context, it might be possible to disturb this memory.

destroying its contents, though this would not be so simple a matteras in the immediate memory, where merely a shift of attention suffices.We conjecture that the retrograde amnesia, affecting the memory oirecent learning, observed in animals given electroconvulsive shockshortly after a learning trial is a manifestation of just such a destructivedisturbance of acquisition memory before the permanent storage pro-cesses have had time to operate.

The discrimination net memory of EPAM is our model of the structure1

of the acquisition memory. When an object is presented to the acqui-sition processes for learning, the net is grown to provide a uniqueterminal location in which to build up the image of the object (that is,to familiarize it), if no such location is already available. Sorting anencoded object through the discrimination net will retrieve the storedimage 1 of the 1 object for further processing or tor response generation.44k1 discriminators used in growing the1 net to make1 finer and finer dis-criminations among the 1 objects entering the learned set are1 constructedfrom differences found by matching processes that compart 1 objectswith previously stored images. Recognition of an object is the 1 resultof sorting the object to a terminal and finding no differences betweenthe object and the image stored there.

familiarization of an object is done roughly as follows (though wecan not go into all the details of the process here). All images (exceptthe so-called elementary images, which are merely stored propcrt.strings) are built by listing a set of relerence pointers to the net locationsol the' image's of recognized (that is, already familial) subobjects of theobject being familiarized. 44ie.se pointers are cue-tokens ol the familiarparts comprising the object. The'ie is only one image lot each lamiliarobject in the net, but there may be any number ol cue-tokens ol thisimage stored in the context ol any number ol other images. When animage is processed for some reason, e.g., lor generating a response, thetokens of subpart images are used to retrieve the subpart images themselve as ne cessary. Thus, in summary, an object is lami I iai i/e< I in thisineinorv only by the process ol listing tokens ol alieadv lamiliar subobjects. II a subobjeel can not be recognized in the net, it must lust

13: INFORMATION PROCESSING AND MEMORY 459

i

«

itself be made familiar before it can be used in the construction of theimage of the higher-level object.

The image-building processes of EPAM are essentially recoding or"chunking" processes. No matter how complex a stimulus object maybe, after the image of that object has been built a single symbol, itscue-token, will be sufficient to signify its presence as a constituent ofany other complex stimulus context being memorized. Thus, all storedimages turn out to have roughly the same informational complexity(that is, number of symbols needed to represent them in the storage),though, of course, the processing that may have to be done to retrievedetails of a particular image may be a complex search. We see hereoperating the trade-off. often pointed out by computer scientists, be-tween the complexity of the storage representation and the complexityof the retrieval processes. The EPAM model, inclines toward simplicityand homogeneity in the storage representation and complexity of re-trieval processing. Thus, for example, the response generation processof EPAM is not a simple find-and-output affair but rather a multilaveredrecursive constructive process (Pribram has called this kind of process"remembering, as opposed to e/Tsmemibering").

Two additional observations about the discrimination net will be 1

useful. First, the structure of the 1 discrimination net is the 1 embodimentof all the 1 discrimination learning that has taken place during the 1 ac-quisition o\' the items stored in the 1 net. That is. "there is no' separatestorage 1 lor the information learned during discrimination learning.Second, the net is built by processes that are 1 under the 1 control of alearning strategy. Among other things, this strategy is responsible forthe 1 analysis of i\)(^ information concerning correct and incorrect per-formance that is \\'(\ back to the subject by the experimental procedure.It dee-ides what is causing incorrect responding and what io do aboutit. It does (his by the- application of the following "satisficing" heuristic:;m ;"l<h'timi or change to the net structure or image information thatjust works (gets rid of the immediate performance problem) is "good( '"

,,>

"-, L As a i-csull, the net is grown in a pragmatic fashion, no at-

tention being paid to the inherent "logic" of (he classification task that(he net is performing for the system. This heuristic strategy is useful

1,1

ir 5,1,,,( '■"" "I 'hat it allows EPAM to learn experimental tasks"111 reasonable amounts of processing effort. In general, it *s\ 1 1 1 not |,c,

*

,r 1m ' sI u ' ,n ln organize information lor purposes ol long term storageUr s,,;l *' "'("II. to this aiglliiienl in (he next section. 4'hese hypotheses■'boiil slmctiiie ,md process in (he acquisition meuioiv has,' many in

«460 EDWARD A. FEIGENBAUM

i

I

teresting consequences in terms of the learning and performance be-havior of EPAM. I he'se consequences constitute1 the validating e\*idencefor the EPAM theory. We 1 shall mention just one result, a rather startlingone, since it is a direct consequence of no single1 process or structure 1,

but is rather a complex consequence of the interaction of many pro-cesses and the discrimination net.

The result is this: EPAM exhibits forgetting behavior even thoughthere is no destruction or decay of images or tokens stored in the mem-ory. Using traditional labels, this behavior is described as oscillation (inthe learning of a single list of items) and retroactive inhibition (in the1

learning of more than one list).

44.ese two types of forgetting behavior have a single EPAM expla-nation. The discrimination net must grow to store new items beinglearned. 44k1 cue-token information used by the performance processto retrieve the' image of some' stimulus item's response associate isgenerally just sufficient to correctly retrieve the response 1 from thediscrimination net at the 1 time 1 the association is learned. However, asrepetitive trials proceed, and the net grows over time 1 to include newitems, this cuing information may become inadequate for retrievingthe 1 correct response. In this event, what may be 1 retrieved is a subnetof items (all similar to the 1 correct response) which includes the correctresponse A random process then selects a response item from this sub-net as a guess. (Note that because of the 1 wav in which the net is built,this response, if in error, will be an error of response generalization.)W hen such an error is made, the processes that anal, ze the informativ efeedback can correct it by storing additional cuing information. Withinthe learning ol a single list, when S-K pairs learned on later trials inter-fere in this wav with pairs learned on earlier trials, oscillation results.In multilist experiments, when pairs learned in later lists interfere inthis wav with pairs learned in earlier lists, retroactive inhibition isobserved in the test sci jueiices.

The Long-Term Permanent Store

We have been led to the idea of a permanent associative stoic ol seiv

large size loi a i in nibei ol i easons hay i ng lo do w i I h coi icepl ua I pi oblenis

( '.cm r<d ( 'ousidcra / i<>n.\

13: INFORMATION PROCESSING AND MEMORY 461

of the EPAM theory. On these grounds alone, however, the hypothesisof an additional level of memory is not completely convincing, thoughsome of the problems would be resolved neatly under the hypothesis.The existence of empirical evidence that suggests different storagemechanisms for working-\ersus-permanent storage is therefore en-

couraging (Deutsch. 1962; McGaugh, 1965).

The notion of permanence of the storage of symbols in the long-termmemory is an assumption for which there is not much empirical evi-dence. To be more precise about this hypothesis, it is assumed that thosesymbols upon which a significant amount of processing has been donewill never disappear from the long-term storage structure. This hy-pothesis is not the same as a naive "tape recorder" hypothesis, underwhich all information sensed is thereby recorded permanently; becauseof the demands oi' tasks and the effect of attention focusing processes,some inputs will never receive the processing necessary to quality themas candidate's for long-term storage. The hypothesis regarding per-manence is, we think, reasonable at the current state of knowledge.There is no directly controverting evidence, and there 1 is some measureof support from the earlier EPAM modeling efforts, namely, that be-havioral evidence oi' forgetting can be accounted for satisfactorily asloss of access to stored symbols caused by dynamic changes of memorystructure.

We 1 view the processes of retrieval of symbols from the long-termpermanent storage as a problem-solv ing process. By "problem sob ing,we mean a process that finds an "answer" path through a large 1 maze ofpossible alternatives using search-limiting heuristics, as has been widelydiscussed in the literal lire on computer simulation ol cognitive processesand artificial intelligence models (Eeigenbaum and Feldman, 196.4).

Ketrieval times alone would indicate 1 quite a different retrieval processacting in the long-term memory from that involved in the recognitionof a familial object (processes of testing and sorting that are used inEPAM 111). Introspection on retrieval episodes, where it is possible tobe fairly sell-conscious about the underlying processes suggests prob-lem solving— that is, trying out various strategies to guide search,testing hypotheses, exploring particular avenues in depth, pice ing to-gether clues in various patterns (as in puzzle solving), a great deal oltrial and error searching, and sometimes (as with problem-solving purgrains) the successlul termination ol search with great suddenness.

\ not her phenol neno 1 1 ol long form memory Iha I needs |o I >«" explained

iiHi-

29trf

1'Ii;iIus

life

jin'si

M\*fiw1:i¥

■<■£..■#■■«&

"jr.

'31

i■■}[

mHIp1m

1mm■:»S:'US4

;

:T

Til.

m«f:

is the fluctuating availability of the symbols stored therein. Sometimesa particular sought-after symbol can be retrieved quickly and easily; at

other times, it may appear irretrievable, no matter how much effort isspent trying to recall it. until some circumstances (or merely the passageof time) appear to "bring it to mind." We shall have some comments

about this later in connection with the problem-solving nature of re-

trieval processing.An additional and more subtle question that early influenced our

thinking about a level of long-term storage involves an adequate ex-

planation of proactive inhibition, to which we now turn our attention.The memory of an organism at the beginningof a learningexperience,

it can be plausibly assumed, contains a large number of symbols storedduring past learning. How can this total memory context affect as-

sociative recall in current learning? In the present EPAM model, itcannot, that is, EPAM exhibits no proactive inhibition (though there isa "proactive" effect on rate of learning). By the end of the criterion trialthe current symbol context is adequately discriminated from previousones, and no confusion by generalization is possible. In general terms,

the problem can be- resolved simply by the notion that the recentlylearned symbols are1, over time, assimilated into the total memory con-

text by a transfer process.Experimental evidence suggests that subjects acquire and use 1

seemingly extraneous features of a stimulus environment in the learningof the 1 task oriented part of the 1 total environment. This information isrelevant locally in place 1 and time to the 1 objects of the task. (liven a

simulated environment enriched with such contextual information, andan augmented list of features for the noticing process to work with.EPAM could learn an experimental task using such local contextualinformation. 4'he locally relevant information would be used in buildingdiscriminators, consistent with the EPAM heuristic that whatever in-formation "works" is satisfactory (for example, the discrimination: "thesyllable beginning with the 1 letter \\ and learned 'early' in the ex-

perience" versus "the syllable beginning with the letter I. and learnedlate'").

Though such information might be uselul in speeding up the learningin the experi mental session, its utility quickl \ fades as time passes andstimulus environments change Focal contextual information does not

"work" well in discriminating objects and guiding retrieval over theloi ig It i in.

13: INFORMATION PROCESSING AND MEMORY 463

Processes

Considerations of this kind lead us to suggest a transfer process con-

trolling long-term storage, with these properties.(a) It "reprocesses" the working memory, copying recently learned

images to the permanent store (with the appropriate associative linksas -determined from the discriminator and cue-token information)- Inso doing, it makes decisions about temporary versus permanent rele-vance of the information. It ignores the temporarily relevant information,

which thereafter plays no further role. The storage is reused by theacquisition processes in subsequent processing.

(b) It is a strategy, in that its decisions concerning long-term relevancemay change over time based upon experience with environments or

upon instruction.(c) It is a low-priority process. The high-priority processes are those

that attend to the demands of the environment and the acquisition ofthe task. Since it must share with these the processing time of the serialmechanism we have just postulated, and since it must grow a memory

structure that is good (useful and relevant) for long-term processing,not merely "just adequate," we conclude that the so-called "consoli-dation" of the long-term storage will extend over a considerable elapsedtime. This time may be of the order of hours, or even days (as suggestedby some drug studies), depending upon the activity of the organism andthe other information-processing demands it is satisfying, and depend-ing also upon the complexity of the' learned task.

It may be 1 that the 1 permanent storage 1 process is slow for anotherre ison, namely that the 1 underlying biological permanent storage pro-cess is intrinsically a very slow process. One 1 sees this, for example, insome of the nonbiological memory models that have been constructed.To cite two extremes, the 1 chemical thread-growing memory built byPask (1961, pp. 105-108) store's information thousands of times more 1

slowly than the fastest magnetic memories of present day computers.Indeed, within a computer system itself, the data rate's of the main"working" memory are many times taster than the 1 data rates ol thehuge "bulk" memories used for secondary storage 1 .

in this connection ( Ihorover's result I 1 9(i 4) showing very fast con

solidation is disturbing but not totally at variance with our position, at

least for very simple learning tasks. MeCaugh, in a personal eomnmni

cation, indicates that his experiments suggest big differences in con

StI2B

isi?I»Itil

■ttt-

§43?

n1I!lt**ai\&:mi

;'"*■.;:/!

:f:9

I::_?«,_lH

w'■';Isk'.

w

mmmm.W

m.m."si;m_w*f

§M

.41■'-rrt

mm

464 EDWARD A. FEIGENBAUM

i

solielation speed between simple 1 tasks and complex tasks. On the' otherhand, Chorover's result is at variance with many previous results in theexperimental work on consolidation.

Such a process suggests the solution to the questions poseel earlierabout the mechanism needed to account for proactive inhibition. Itmerges the recently learned context with the total symbol context ofthe permanent memory, and in so doing, throws away some of the dis-crimination information that was responsible for perfect performanceduring the criterion phase of the recent learning. The consequence ofthis is generalization with the symbols of the total memory context,typically for some, but not all, of the recently learned items.

Structure

The structure of long-term memory is viewed here as an extensionof the EPAM 111 (acquisition memory) structures, not as an entirelyseparate level of organization. The primary memory structure of EPAM111 is the' discrimination net, described earlier. The images stored atthe 1 bottom of the 1 discrimination net are1 richly interconnected so thatusually they form a large 1 and complicated graph. An image 1 is built atthe bottom of the net as a collection of tokens for already learned sub-images; the subimages are themselves built up in this general wav

;

andso on. In this graph of interconnected images, those 1 that are connectedto the 1 discrimination net are said to be in acquisition memory. Othersconnected into the 1 graph but not directly accessible through the dis-crimination net are said to be in the 1 long-term store

Thus, the nodes of the memory graphs are the 1 familiar images ol ob-jects. Connections between nodes are 1 either attribute-value links de-fining relations between images at nodes, or specifically the whole-partrelations that are "built in" by the way EPAM constructs images.

4'here is a simple 1 wav of looking at this structure 1 . If the capacity ofthe long-term store is very great, and il symbols stored permanentlytherein are to be retrieved without very great inlorniat ion-processingpenalties and long search times, there must be multiple entry points tothe store These might be thought of as index points to a large file. Thediscrimination net ol the acquisition memory is, in essence, the indexto the long-term memory, liidei our present conception ol storage processes acting in the long term memory, the discrimination net growsand contracts it grows dining discrimination learning, thereby in

" i^iwnifinI .. .

creasing the 1 number of points at which the net accesses the long-termmemory graph, and contracts as the net is reprocessed by the transferprocess described above.

The search for images in the long-term store need not be restrictedto merely a movement from node to node using whole-part or attributevalue relational structure. The discrimination net is always availableas an indexing device for selecting a new entry point. Search strategiescan be constructed that makes use of this fact, thereby adding an ad-ditional "dimension" to the search.

Retrieval from the Pong-Term Store by Problem Solving

Stimulus events set up retrieval problems. In the laboratory situation,

they are part of the task that the subject is called upon to master. In per-formance mode, he accesses symbols stored during present and pastlearning activity. In learning mode, he accesses previously storedsymbols to build up higher level images. In our present conception,retrieval of information is either direct or by problem solving.

Direct retrieval is accomplished in the discrimination net: a path olthe net links directly to the image being sought. In the sense describedearlier, this directly retrieved image is in the acquisition memory.

If the 1 image 1 being sought cannot be- retrieved directly in this fashion,

then a problem-solving search for the item is conducted in the neighbor-hood of the "entry" point" give 1!) by sorting through the discriminationnet the stimulus situation that gave rise to the 1 search. Usually entry to

the graph will be obtained at a region containing information similarto the information being sought, since this is how the net and long-termmemory graph is built up in the first place 1 .

For this purpose, an adequate problem-solving model is the CencralProblem Solver (GPS) [Newell and Simon (1964)]. Stated very briefly,GPS solves problems that can be 1 put in the 1 following general lorm.Given descriptions ol an initial problem state and a target state andgiven a set of operators for transforming states by reducing difference'sbetween problem states, find a sequence of operators that will transformthe initial slate into the target stale

h, the memory retrieval problems being discussed, the initial stateis the entry node given by the discrimination net. Some ol the extrinsicinformation that gave rise to the' retrieval problem is used up in ac-

cessing the appropriate "local" portion ol the memory graph. I he re

1

Ik

I4fr

Iw. :

in;

till?.&:<lk-

";'*,

,

|l$II

I1

'■;#

1nif

wI_f

mwI..-"9*

11.■_e.

;;_*

«

...ft

'Bit.

■-ft

466 EDWARD A. FEIGENBAUM4

mainder is available as a description of the target state (the image beingsought). This description is the basis for recognizing the target nodewhen it is encountered. Upon encounter, a cluster of symbols is accessed(those associated with the target image), one or more of which may bethe sought after symbol(s), for example the name associated with thetarget image. The operators, the means by which states are transformedrespond to the various ways of moving from one node to another inthe graph .

We wish to conclude this section by looking again at the problem ofexplaining fluctuating availability of stored symbols in the light of theproposed problem solving nature of the memory retrieval process. GPS.or the Logic Theory Program that was its forefather (New ell, Shaw, andSimon, 1963) or any one of a number of programs (such as Slagle, 1963.his SAINT) that are cousins to the Logic Theorist, are fairly powerfulheuristic problem solvers in the sense that, over the domains of theirapplicability, they solve problems about as complex as people can solve.\vt all of these problem-solving efforts (programs or people) appear tohave a common characteristic: the average number of steps in the solu-tion derivations, and in the means-ends reasoning chains, is not large.The' longest proof generated by LT was eight steps deep; the average 1

perhaps half that. 44ie average number of steps in GPS means-endschains for the tasks that have' been explored is probably about six. Inthe well-known Checker-Playing Program (Samuel, 1963) and in somechess-playing programs, analysis proceeds four half moves deep in the"look ahead" search.

Suppose that in memory retrieval problems, under a CPS-Jike regi-men, comparable limits on "solution complexity" were to be encoun-tered (a reasonable assumption). Then on some particular retrieval at-tempts, searches may be unsuccessful (subjectively, frustratingly so)because the item being searched for is not within the "span" coveredby the problem solver from the entry node given it by the index, that is,the discrimination net. In other words, the selected entry node was not"close enough" to the target node for the "path length to solution" tobe within the bounds of average depth of search. The sought for itemis thus inaccessible unless a better entry node is selected.

One wav to achieve a better solution is to postpone the retrievalproblem lor some time awaiting the circumstance (testing periodically)'hat the contact nodes ol the discrimination net with the memory graphv ill be more lavoiahle This is a possible solution because in the noimaleourso of e\ei,|s, the discrimination net is expanding and contracting

v/I

_.

4%

under the 1 impact of the1 changing environment, as described earlier.Thus, we have here1 a possible explanation for fluctuating response avail-ability, even in the absence of a conscious retrieval strategy.

However, deliberate strategies are also wav s of inducing shifts ofentry nodes. Some strategies, for example, might employ early or inter-mediate products of search in various arrangements as inputs to thediscrimination net for new entry node selections. Another strategywhich appears to be commonly used is the systematic generation of"stimuli" (produced and used internally), which we would interpret asa search for an appropriately "local" portion of the memory graph inwhich to search for a particular item. A common example of this is thetrick of "going down through the alphabet" when trying to rememberthe1 name1 of some object. This strategy is very much a trial-and-errorprocess. like most other heuristics, but it often works. Here we haveanother piece e)t the explanation for fluctuating availability e.f storedsymbols, this time strategv-elirecteel.

4"here are1 other general inferences one could make from a model ofthe type that views memory retrieval as a problem-solving process, buteliscussion of these' is best postponed until after a computer simulationof the model is written and tested.

Conclusion

In this chapter, I have proposed a three-level theory ol memory:

(a) an immediate memory of very small size, in which information isstored for very brief intervals, which acts as a bufler storage1 to decouplethe input (peripheral encoding) processes from the central processesand as a temporary storage for central processing;

(b) an acquisition memory, a working memory with the structure ofthe EPAM discrimination net, in which discrimination learning takesplace and in which the internal representations of stimulus objects arebuilt:

(c) a permanent storage in which the internal representations areorganized and stored lor long-term retrieval.

the l'4'AM model is a precise formulation ol the immediate and ac-quisition memories, and we have been able to demonstrate and validatetin' consequences ol these parts ol the theory. I'he theory ol the perma-nent stoiage is a logical extension ol I*4 VAN I suggested to the theorist

a icsoliitioii ol certain <1 1 llicu 1 1 les with the present model. Since it

:-.

130

!iffiI;

4?,;

Im'\m

IiSI*.-'m&m■J.

I:wM\

i.

ins--.iv'."f_r.mlm'.:P*'imi':??'mm1:

%$i

M:

V3i-1mM■'.ma

5j

M

♦

>■»

has not been precisely described or tested by means of computer simu-lation, it is offered in a tentative spirit.

The discrimination net of the acquisition memory is viewed as anindex to the permanent storage. Retrieval of information from thepermanent storage is viewed as a problem-solving process, along thelines of the General Problem Solver model.

ReferencesCuohovek, S. L. Neural information processing and memory. In J. L. McGaugh (Chin.).

Information processing am* memory. Symposium presented at the American Psy-chological Association Annual Meeting. Los Angeles. September 1964.

Deutscu, J. A. Higher nervous function: The physiological bases of

memory.

AnnualRecieic ofPhysiology. 1962. 24, 259-286.

Fkh;k.\h,u*M.

F. A. Simulation of verbal learning behavior. In K. A. Feigenbaum andJ. Feldman (Fds.). Computers and thought. Xesv York: McGraw-Hill 1963 Pp 297--"309. '

'

Fkicenbacm. F. A., and Fia.n.viA.v J. (Eds.) Computers and thought. Ness York McCraw-II ill, 1963.

Fkick.XßAl m. F. A., and Simon. H. A. A theory of the serial position effect. British journalof Psychology. 1962, 53, 307-320.

MeC.ucn, J. L. Facilitation and impairment of

memory

storage. In 1). P. Kimble (Fd.).The anatomy of memory. Proceedings of the first conference on remembering.learning, and forfeiting. Palo Alto, California: Science and Bellas ior Books 1965Pp. 240-291.

\i:v\ K1.1., A.. SIIXW. J. (4. and

SIMON,

11. A. F.npirical explorations svith the- Logic TheoryMachine: A case studs in heuristics. In 14. A. Feigenbaum and |. Feldman 44ds.').Computers and thought. \es\ York: McGraw-Hill. 196:5. Pp. 109-133.

\i-:wku„ A., AM) Simon. 11. A. Computer simulation of eognitise processes. In li. I).Lmr*

]l H - Hlls1 '- i,1,(! I- < Planter (Fds.). Handbook of mathematical psychology.\ol. I. New York: Wiles. 1963. Pp. 361- I2S.MvsEU.,

\.,

and Simon. 11.. A. CPS. a program that s,n, .dales human thought. In 14 A.'"<"'genhai.ni ;m( ' .1- Feldman (Fds.). Computers and thought \es\ York- M.-Ci mHill, 1963. Pp. 279-293.P.VSK. C. An approach to cybernetics. \ew York: Harper 1961I'mmrvvi, X., Am vrvnv. A.. Maktoc;. J., and Boos, L. A progress repor! on (!,<■ neuro-logical processes disturbed by frontal lesions in primates. In |. VI. Warren and K. Akerl(Fds.), The frontal granular cortex and behavior. \ew "lork \lc( irasv - 1 1 ill |')(, |

Pp. 2S-55.SVVII

-.;■..

A. L. Some studies , niachiiic learning using die gaum of checkers. In 14 \.|,V, .- , ' ,11,i" ,,n ',l,(l J- Feldman (fids.). Computers and thought \e\\ York- VloG.-.uHill, 1963. Pp. 71-106,

Simon. 11. A.. VNl> Fkioi nu s.

vi,

17. A. An mlormal.on p.oeess.ng theors ol some elleels

"' s"'"'-" | 4 la. ..ilia. its. ami n .can. ngl ,. I mss ,n \r.lul 1,..,,,,,,,,. fauna/ oj \cbalI .earning and \'crlnd lich,n ior. 196 1,3. 3.55 3!)6.Sl X< ''

*

' l{ X l"'H.ehc pioo.am thai solses

5\,,,b01,e

inlcmal.on ,„o|,len,s ,„ | ,|,i„,,i,

'" 11,,1,,1v '" X V l-'-enbanm and | !■ e1d....... < F.ds >. (', anpnl,

,„„/ //„,„../,/

\,.uo.k vie< ;.,iw i hii !!»>.'. i'|, mi g{),\