Gorn 1-

7*f*»tj IMrertv Libraries

cm JtjJo >3^ L -*****Sox _ J^__ g^jSfj^-r,Fo

i

!

'N'ORMATtON

AN,,

,„v,« (^

On the Mechanical Simulation ofHabit-Forming and Learning*

Sail Gorn

''-"'MCA"-iM;::;;r.M:-r"

mechanism,

i,

di,,,,

' ,'., r T\?\ g'^ lypcs

of

simulated,fJu^TenUoTu ' I°""* "'"' l"aia* '° '*guage

of

comouter J" """ m" f" r'"'""t. It uses the lan-■he l„«, ,"',,, ' "^""'"'"K '"

JeserilH,

the flow

of

control, andvariousr,,,,'

1,,

': ', 'd:' 1 P"ba

hili,

y to analyze the

effectof

latingZTr "r T "" ""' "W"* >«h.vior efsimu-fe ::,; ;:;:h ';,;r::;, ,i:"r "BT;n 'i"s— h-he "factored,„„-.. ' ' simulating process

may

ments Ji, "",:'"' -''st„,g„,shed by whether the reinforce-S^ver-l'' '"'"-^'"■l.v or up„„ "comparison with , goal."

sketch i.s given of th..

HM,--:

ui> .. . a "'' Ull(l ,jUre - n

rer" rr

of

-;'- r.SNational Science Foun 1, , .T*V? " :ir'' ma'lfl ■**""' We hv " «r»n«

fron

' ""Portion,

of

Lt< mi " ' '""t° thel '''"■r.-ityofl>en„svlv„ni:l('„m l

„„erCen,«.

summer coursesr"r\ \ , "'" presented :" several University

of

Michigani" August 1057 „,, i ..Y ,'I', "':l'.""

,s

"f Lotsic to Digital Computer Programming"■i^■"i«'J«»:X)^^;lTrfD^w^^" toArtifiH,,lD,,^TheirCapal,ili,L"i ;, 'o the semmar on "Automatic Computers nodKngineeri,,, and th. £!!." * i """"mi '" 1957-S8 '>y theMoore School

of

Electricaleer.ng and the Society

fo,

Industrial and Applied Mathematics226

■■*-■_,

BECHANICAI

SIMI I.ATION

OF HABIT-FORMING AND LEARNING 227

INTRODUCTION

\Vhv should "life" scientists be interested in mechanical models forexperimentation? . „

,■

.

"

. r,U is because the living subject,.being "irritable" and

intricate,

often

yields more informationabout the disturbance caused by the experimentthan it does about the factor being investigated.

A machine, on the other hand, can be constructed to abstract onlywhat isunder investigation.

This paper will concern itself with a class of mechanical models ol thetype obtained by programming a general purpose digital computer tosimulate many special mechanical models. Each such program

IS

alinguistic description in "command" form of the special mechanicalmodel desired. (In just such a way a complete abstract theory is a"linguistic mechanism.") These programs are relatively easy to form,and one could try hundreds of such mechanismsand experimentson thesrae general purpose machine. In fact, one could have a "library ofsuch programs and "compile" intricate larger mechanisms by suitablyprogramming the order in which the sub-mechanismsare to appear.

This paper, because it discusses programmed models of the "reinforce-ment" type, will, perforce, use two types of language in their descrip-

tion. The first is a language suitable for presenting the flow of controlsin the models, i.e. the "command language" of programming as pre-sented in flow charts. It will provide the simplest means to show howcontrol selections are made, delayed, and randomized; for, whatever elsethey may be, habit-forming and learning are delayed random

select,,,,,

processes. The second is a language suitable for presenting the asymp-totic analysis of the feedback of communications in these models, hat«, the "descriptive language" of mathematics m it is used to

analyze

the random "memory" processes being mechanized.We can expect such mixed command anddescriptive languages to ap-

pear whenever mechanismsare specified and analyzed.Such emphasis on language, is,

however,

forced upon v. by th fact.hat our models are programmed. We should here pause to remark onother mechanical approaches. i„„

r

„;„„ n,.,v l,eti

■

, " i ,;„, f hab t-fornimg and learning may lieThe mechameal simulation \[^« mechanisms appear inwalog or digital. Example of am, w. Grey Wa,ter'he path-breaking work by VV. ttosa

-->■■

3 \ '. . ,

,„,,,„,,

(1MB). In these studies, selection by the mechanism . Iron, a c0n,,,,,,-./

>'

■-

N.

v*- ■--R

5-

.

22800R.N

Otisdata range rather than from discrete

i„f

emphasis ,s on the study of behavior ' "

f,

:r",:"" ," *»■*."

>l,

of selection procedures. I 'l'"hty mther

th;

'» »" thevan,"If the meclianical simulation is digital ii «i

,"

,apeeial purpose machine or by pn,. r „ ? beactue^ eitherbJachine. The special purpose JS^r^-^^^*than programming. Such machines lev hardware, rather"-***»

1-'"l^'"-.^

for all prog, ,„,s to

g

',;"'MB

C

°mpletelyzzz:y:::B~":aa:z:z^

ma iTh«

,h

,e nr chart °f ii programindicate ight be

"and-nodes"o,

not ant^'r "Tc P,TM ma('l,ines are «wtWly serial, and-nodestheo,LZ /"

S

'tal "0W ClmrtS M

,h

"-V d" '" anal°S «" Whenno ZZ -r V °PtT g geMral P "rp°^e di«

,t:ll

««»P«tei. appear, it willthem m 7 l^TT* t0 SimU,atc simultaneous actions by stringthem out anally The equivalence of programming and hardware

,ull

Sumnedutey apparent; flow chart, for routines and logical de-Irrotv JhT*' 7 maChineS wi" match «** f-'r ho* and arrow forarrow if the degree ofdetailof the language describing each is the same.'ehant.T',n,M Ut ,"? "* ***"&& «WUM if we describe the nie-forSr " habit-fl,rnii"S learning in terms of programor g ne al purpose machines. The distinction is in manner of summationInaloLl , !V ,Ut " SimU,ated' «"«e is essentially true in theanalog-digital dichotomy.the^star? dich,JtoI»>' "' "loarning" j, concerned with whetherDhei Th , '" eXPerience d""-,g the process is explicit or im-plicit. The explicit method stores suitably coded and classified li* d

*,t^:z^z'zt:^to 10,,,,i in tha "—a" ihf

.':-

gff" "i-V 1

■lII.MCM. SIMULATION

OF

HABIT-FOUMING AND

LEARNING

229

„t experiences to be used in the delayed selection of the propernse to astimulus. The implicit method affects the

memory

merelythe more appropriate responses whenever an experienceLarrants without any more detailed storage of information.Control

at the moment of selection is automatic ami immediate,without elab-

orate digestion oi' past information.Evidently the explicit method is more likely to simulate the intelligent

development of gestalls in habit-forming and learning. But ,t must heused with discretion, for, with no other intermediary, the piling up ofexperience rapidly develops an informationretrievalproblem. The moreexperience piles up. the longer is th.- selection of information therefrom.

The action of the control choosing the proper response to the stimulusbecomes more delayed. It is as though advice were asked of an old, oldman-the type who responds, in ancient mariner fashion, with a lengthybiographical sketch of largely irrelevant experiencesbefore arriving, il

ever, at the required suggestion for action. .,_,The implicit method, on the other hand, though it yields faster re-

sponses to stimuli, simply simulates reinforcement or avoidance reac-tions; it is thereby less ■■intelligent" and more like brute habit-formmgor like learning of the conditioning or reflex type.

There is no doubt, then, that when experimental modes are con-structed, they should include a number of judicious combinationsof heimplicit and explicit simulations. This paper will be restricted to a dis-cussion of the implicit type. The reader will find interesting informationon the explicit type in the studies by Newell and Simon (1957).

In this paper,'.!,,- question of asymptotic

bohavio,

-

w,l

emerge as the

result of studying stochastic difference equationsrather thano examm-

ing singular pointsof differentialequations,as wouldbe donew„h analog

"Study, then, functional equations connect,,! with dccisiomniakingPresses such as the Bales-Householder model which is analyzed by,

"

' „

. "

nQVVI and used extensively by Bush andHarris, Bellman, and Shapiro (V.M) anu u^MosteUer (1955). election)processes discussed,

Although in

the,

ig ta decision (that

is,

km i if

,„g„,

i B established is an important one,the question of whethera habit will be est ,M|,»m«»lv«,' . . .„ »,i,„i„as be theseection procedures themselves.the main issue will nevertheless ne »„. . , !" „„„.;nff selection methods, the or-noaes re-ft c therefore begin by discussing s«i<lerred to above.

■\

■ ■■

i

= T3 -;-

230

I

I

■

COUS-

IN OF HABIT-FOBMINO AND

LEARNINCI

231MECHANICAL SIMULATION

SELECTION METHODSAy^::yy:.y:::::!:^- « ,

whether the ° """" ', 'S °

l

■

■

;

j

ti

!

I

Ii

HORN

233MECHANICAL SIMULATION OF

HABIT-FORMING

ANT) LEARNING

r^A;:YYz:.!YY::^r^^to ex., 3 - -would simulatenot a procedure but ! ?"* % a"d M '"sen.se. This is especially true if ,I, ProCess "' ,hc statisticalhack to the same ,111^1 "B*"h'hta—*random phenomena We couTHf " ' ""* ** made

,0

simuL*to «he i„put (,f th o, , „ ",d;i(: , .';;;;-" f«h *>" random p

rw&

.Sightofore",.rT ?uhod ls «"' entirelysatisfactory,

record these inputs and T "T ""' PMea We could ofc^In thatc^we couldhn !"

wi,h

t}- -P-d inputs:". the first PM T e?T a tUUe "f rand°m nUmbera' «««Wa simple rou« ne , ",mi '7"^:"ld fc< «* ■*paradox that rami ' product,,,,, of a random table. The£ exphi, ed bv f/l "" mUy , '° BimUkted ""

S

***"**& machinedS The h """ "' , ', ""nV' M°°re- Sha"°"" "rfShp,„,^, IIW,r7 Btato. r"»«hly, that if a probabilistic maleZib

1,

in I' ;"

f,

rat "m Which a computable distribution,

i,

bmi ZS^* "*** '* *" ""te te-

We will represent any such subroutine which generates "pseudo-

random" numbers by the symbol

—> i generate p I —

►

in a flow chart. .The stage is now set to program random selection routines. As in Mg.

1 «c canachieve the effect of the random or-node2a immediately, as inRg 2b We are now ready to achieve the effect of the delayed randomselection2c by combining the method of 2b with thatof lc. We will thenbe able to look upon a and what precedes it as a -stimulus" with theresponses a\ , a-. , and a.( .

THE GENERAL REINFORCEMENT SUBROUTINEAa the introduction states, we are restricting our discussion to the

programmed simulation of habit forming and learning as delayed selec-

fo£ TW^ It"r,

m,i

"0S 1"'"

Sff^t.^.^r-^- * -1 *£selectlon-P.wllbetheorobabilitvthat„ hi

" '

+ P" = 1;aMcan be achieved by „,ak,„g h vecLl .^"^^^^««* feedback to the selecting sub outLe i M di%"'earning type from the habi,

for,,

, , 1 "7 distin*«*occurs in learning when there i '

USe

,he

""ionof,

'«'»'»*. the traitfor t ;:;;:!,anW" "ith -n-1-l.Tl^id»'» -10ci,,,, subroutie In , ' must °"routside the*r,>»«« have to occur o ,'. ''' " '"""""r: SUch transformation fasubroutine, then „ ', ' Mlbroutmf- For " habit-forminjr* of the v«S p the 1ZTI "' ' "** " ' "rt "f transfoni^scribing the d«tnbut.on of responses; here, is an index*.deternLsic o FT? '"" ind«-»«««* nu,faof the sti,nuu, " tk ' :: ;v ':;,h

'

?vhii''i d"scrife the"^kresponse i ( ,,.„,/ ,° f trausforD>ed at each stimulus withand aVg, s f." M^ *>. W* history of respo,,,,:,the stin ih

hI

the

X,

VeCt°r r '""' "-'>" fOT the next appliel,,than' nep nr;'y'

COm

P°oeut * >W»" m ght ,■ 1 HP' th6re " U habit-f°"ng tendency to response*;

whole h.,; a PoB' tiVe" °r "rei«forced" response. However, .heof 'co ""''''^ Patter" might tend *" a"'d responses

heca,,

In the ",llU''lr'S; ?" fth «"»»' t,tc -b the resulting process (Markoviaii ornot.;, and similar factors.mast If « iPTr WiU reBtrict our discussion to a number of modelswhich are special ca.ses of the general reinforcement type habit-

Fio. i. Ti : reinforcement

functions;

p - (p, . P*. ""> P-); ''-"''"'—i »;£*.» p, - 1.forming subroutines indicated by Kig. 3. In this figure f. are the variousreinforcement

functions,

p = (p, ,

/>■;

. ""

■

. P«)> P' =I and

I—1i L i ..nnl.l rlefer the reinforcement, T,p —- p,In such a sU .routine we coula out, >■"-

until after the response a. has been taken. By doingso we can ,-c .

resulting subroutine either for habit forming or for lear,

g.

fact become a pure random selector. On any machine withloop control

».,,

wiuiui p,,0 achieved more compactly as(Corn, 1957) tins randomRector^y ta a ingtructions suchshown m the "schematic «- ' if fictions. By such a methodas ,+ , , must

mod,

ya mm.tar*we have "factored out trie pmc n»"*

■

the reinforcement functions. . ru.rnfor«T., , „j„na variety of choices

for

the operators i , ,Because of the tremendous varies

*'.

■

M

■

\y

■'■

i = 1,2, ""■ ,n,

UJ

/

I

236

CORN

237MECHANICAI. SIMULATION

OF

HABIT-FORMING AND LEARNING

°. 4. General random selector fora vectorfor response,t

£^I!^^^^^-«i.-^r*" r*- ■■■ = 7Mp= ( P,, P3,---, P„, Pl)

U,e. ;; >, ?M mr

,'':,,' ,< 'SS , at a"- SUPP°SC' agttin' that " = 2.«i«ta,m f ;"

238 i

■

d

"

I

-

I

i

HORN ATION OF HABIT-FOUMING

AND

LEARNING

239

I,

1 ) in the second. This means that

HIMILATIONmatrix upon (llreDhei,,,

=:^—;~ ofsavage s,i,,^;;;; T Th0"">- ("WW.rf and -«*>■»« the rr IZZ TIZVS °*****-**TP ~ "l'+ 0- M, where Xis a fixJvc , " TMd °f

,he

f'«N°te^-*envecto^^^T, =

7',

= .. . = r = r X = (pi, , ih- ,■ ■ ■ B )." the notationof Fig 5 );.,.„.,. ....totic distribution ofrespond r -Km{or

1

240

CORN-

NO AND LEARNING 241-ECHAKICAI SIMULATION

OF

HABIT-FORMINO

l

,r»

«nee the subroutine is of the "perfect learning" type,it^KtoS iS oIT by placing i„st before the exit a' the flowCtcS' of how model //„ could be extended to allow for morellull two responses is given in Fig.

10.

Self

sealing exit

for

perfect learning models

.. '"" " H

2,

, qSymmetric, additive Bhgorhi„- , ,ye, atworbing boundary reinforcement (model H,Fio 8.

n-2.I

*

■

B^iX ', ?' "» -;"- -responding to model #/„ hAny routine usbg , I',7/ " shott'» »' »"» ""hies, , and the initial ill ""Z) Thu °T^ "*" T '"^tine we will find the in ,t„ , ' some"'here in the mam rou-«. - a Tha the °"" Sett '"g '- and substituting ?„ - « andequations "'

U

242

i

i

:

i

*

■

CORNM[-.-rnM,At. SIMULATION OF

HABIT-FORMING

AND

LEARNING

2i'i

(Ti,),

Fn:. 12.

Reinforcement

(model B„),

n-2,

200

i Fig. 9), and it is possible to generalize the model to more than tworesponses, as in .Fig. 10.

For each of these models, the following fundamental ,|U -stums arise:(al Is there an asymptotic response pattern established, or is there

,i

probability greater than zero that a floundering between responseswill continue indefinitely?

(b) In the ea.se where it can be shown that an asymptotic responsepattern is established (with probability 1 ), can we define a measure ofthe "degree" with which it is established, and, for any such degree, canwe find the number of stimuli expected to produce such a degree ofestablishment?

'In = {(3)

l+(,+ ,i)-l,o]

ifrM,„-,

Again, this

conditio,,

;."v""' 's symmetric because

For the most trivial of the models here shown, model //,, , thesequestions will be answered completely.

ANALYSIS

OF

ASYMPTOTIC MODELS

For asymptotic models we expect that (T,p), = 1 for p - (

1,

0),

and that(T.ph = 0 for p = (0, I) as in Fig. 6. If we let x = p, thenlet us define the functions as in Fig. 0, expressed as functions ofx a c.as follows:

"■"W' + IBIJ-iSl'']As with model

//

. "

"

"» , it is possible to attach a self-sealing

program

Thus the linear asymptotic model of Eqs. (1) has:

and we note that, for0 £ x i 1. «"> have «,(*) 2

"■

\ '

"

'■

244

OORN

Theorem!. Both „(*) and ., , '■"'■^ Ihe functional equation;A'-r) = */«,(*)) + (] -*}/(«,(,))with boundary conditions- (6 >

*~*K.*r the Uiiear asymptotic mode, of (5) weJ(9)then

*(*)

=/,

,4_.(1 -*)Theorem 4. For each set of boundary conditions:

(10)

/(0) =/„ and /(l) m jXiwhere

otonic f th-it is rl

*',

\ "^ ',Yr 'ls "olution is absolutely mon-

TH.ORKM 5. lJl+%Vl-] btegerS Z1 a"d anal^

C

'

a funcHoW

,,al

eqult,io,rymP,o,io Pr°babilit* distribution which .satisfies

"

■>

MECHANICAL SIMULATION OF

HABIT-FORMING

AND

LEARNING

243

b The uniqueness of the solution shows directly that the probability0[ "floundering" is zero.

c. The distribution is analytic.Thus question v at the end of the last section is completely settled,

and it is fairly obvious that questionb can be handled, though a con-structive formulationof it will be left an open question. The next sectionwill show how the analysis of absorbing boundary models has a com-pletely different flavor.

Definition. Let It be the sequence of events

where each event B„ is the appearance ofeither the response a, , or theresponse a; . A sequence li is said to "conclude

a,"

if there is a numbern such that B„ =a, if '» g»io similar definition applies to the expres-sion "B concludes a".MUM

LJ urm rum

.1

«: "

Definition. Let n(x) be the probability of concluding a, when, at theinitial trial B, , we have p, = I.llllllilllllillI'l , "i

"" » '

fI

-"

Proof of Theorem I.

I,

concludes a, if: either B, =a, and (8. , ■"

■

,

8. , -" " ) concludes a, with p, = M), or B, =

(J

240

I

I!

i

I

t

OORN

U"- EqUatiOn(,o)i-h- ' —ary,,^,,applicable to anyfunction Ad) forwhiehn <

i,

A then, a functional operator with the f,7, S ' Wh °"° 5' S >'a- It preserves, he

b,„

I d"" th( '''Howingproperties:*(0. and ,M il =*1'; |I :""" I '"'V -'"'- "- *if*. = AMhenA.,o, .that"Am.,^;, t, :r uv:,i>o, ';iveo^nutnh.;, „ " '" ,tSCIf- "

S

""ll '"l>- -" *** ail na^iI rr"! Ulu d

A/

' ''- also continuous in [( , _ x)]*. by c, A"[x( i _ ,)] is poBitive Hence thp K{x) form a monotonic

MECHANICAL SIMULATION OF HABIT-FORMIXG ANOI.F.AItMN,; 247

-ouence hounded by 0 and 1, nondecreasing if o, + a

S

1, and non-. ;,„, if a4-a£ 1. In either case, convergence follows, and, if+„_| *

0,

lim A"[-rl 1-J" I M 0. Taking the hunt ofA.+i =

AA,

lows that the limit satisfies (01. From the expressionfor Mi obtainedfrom (5) one shows that /

I }Vy i Ll Ul AJA f '.!■-. rj-

i

CO <

■

3

I

. f 2

2

i

absorption at the nth trials by „, or „, . Even their generating, ■are nonanalytic. On the other hand we will obtau, £*answering question

I,

of the section before last 'a'os i- m

ll ___ J, r-1- r250

!

1

00R\

'"fdd:ld,;::::d;d;:;:;M!;;MM:;;;-- ,,,. ' '* in os,gl: : ' "

:lrl,

the blowing piw.IMv^

/Vv

t ,f.r-

( .\_

h,Au) -C,(x,«). J fo'Theorem 8 Tl "" Uldefiuite floundering,.*** rational iiullTTTxTilT" * """ '° : 'l^"rPf "m * ' P^*«-K-'^^l-tffri+l),; ,= .-.,.V+ 1

1 )',/,i) -,°4r -(y-i )wy- v'u'i fotl-tAf-j+D^^.,. MI.---.V

If,

for example, A' =oas in Pic «,, c" m '" l '&■ o, then from (11)Dq = xAs 1

MLCHANICAL

SIMULATION

OF

HABIT-FORMING

AND

LEARNING

251

Therefore, for the successive intervals, we get from (15), (16), and (17)

for0 § x < 1 - 21,

far 1 - 2< S -

S f,

A(x) = /,,(,. MM) l"M(-r -00 -x) + x(l -x-t)\and similarly for the last two siibintervals.

Thus,

for X = \ we have

«»>-STTFor example, for

/

= A, I-(h^ ~ *f-

Proof oj Theorem 6. If t < x < ' - ',

andD,ix,t,u) = i _,(, _ x _ l)u,

_. , (x + 2t)(x+ 1)IHfi,(j-,„) = -—rD,ix,t,u)_, . (1 - x)„\\ -(x + 0(1 -x - 2/).,! ]GAX.II) = f-~.—;—j

/,-,

. Fi,iix,l,l) = 2(x +0'+(1 - 0'' />d(.r,/,l) D>(xA»)(j + l)xu'(,i(.r,u) = ; -- rr—.1 - x{ I — .r — ()"

„, . (1 - x)u(r-A.r,lt) = .; —.. ■„1 — x(l — i - 0"2p/ , _ « I±M—

/M

" ~ D,(xAl) 1 -x(l -»-*)'for IgiJl-l,

r , s (x± DxV(" lr-" ) - i _|(x-«)(l-x) + x(l - x - t)]u'-ri . - o~x + t)(l r£dM—G,(.r,») - j._ lu _ t)(1 _ I j + ,{ j _.r - ,)]„.

/\(x, n+l) = *Pii* + ',") +d- x)Plix - t,„)

!»,(,,»+ 1) = *ft(*i

"

■* -Ml - *>'"'(' - '. ")

I

|[

I

)'!'

I

i

252

OORN

The boundary conditions are(O for j- < l ,P'^.o)= a„dA„,O, M JS °M 'or J 6 1 )ni oMx,„,= OforxSO, orforxS 1 and „> n

P,(x,«)= Oforxg |,

orfo,-,

S oa„d„>oThus, for I-/S/S1 v,. have:

The boundary conditions on the generating function are, therefore:Gtr

\

- Illfx «E ' Oifxg 1[Oifx£o [lifxgOwhile the* generating functions fulfill the following conditions:

for I -I jj|l

for 0 g x S

Gtix,

v)for / < x < 1 - <

(1 - x)«G,(x - (, «)forl-t g x g 1

MECHANICAL SIMULATION

(18)

I.N

OF

HABIT-FORMING AND

LEARNING

253

Now ifOJiSI-M

then

and

If therefore, in the functional equations (18), we successively sub-stitute x + t ■■■ X + (2V - l)t for xin the middle equations, andt + Nt for 'x in the third of each set, we will obtain two systems ofV + 1 equations in the unknowns G,-(x, «), C.-(x +«, «), "■

'

- '[ x J(.V - 1)/, .), and G.(x + 2W, a) lin both cases the matnx of coefficienthas the determinant D„«(x, I, «), and the system is

va id whenever

0 < x Z 1 - .V(. The constant terms will all be 0 except for the lastin The G, system, namely (x + AM, and the first in the ft systemnamely (1 - x)«. Similarly, whenever I - Nt S * f\«"°£«substitutions yield two systems of N equations each in the quantitieso,(x, v), -. , G.(x + (.V - 1)1, v). Applying Cramer s rule to the

P,(x, »+!)-(, - x)/>,(x -1, „)

,f

„>o,Pl(x,l)= xandP,(x,n + 1) = (1 - x)Pl(x _ tll)

while, for 0 g x

S

t we have:P,ix,n+l) = xP,(x +t> n) l{ n> 0( /Mx> l} m1_ x

andP,(x,»+ 1) = xl\(x +

l,

n)

x«G,(x + /, tt) forO S x SIxuG,(x + l, u ) +(] - x)„G,(x -t, v)

c'

E

:

■

i'

*il!

ri2,>1

MKHAVICAL SIMULATION

OF

HABIT-FORMING AND LEARNING 255first set of intervals and .,- -(V - I w r. a > 1 ■obtain Theorem ti. ~ A lor x for the sTO

,„d

„, hf||,(, „,e numerator for G(x +tf - 1 >«, ")"' C niml'r 9 mle

,S

Proof of Themm7. I.et D x̂> „)+ («- 1 )F.

v+

,,(x, I, v)

f/lx,

„) = 0l(jj (,

S!. absorption Js rf remam_

)+ I equa .ons in G(x +(y _ , )f> „, ,*_ f. . y+f whose *c first stimulus. SimUarly,for each ofthe m^aodetennmant sagain 0„+l(l , „,J^J- ing two intervals C^/-**,^ £*££thelec^d. Con-Now the sl d";,' ("r" lal VCCtOr K» - *)«. 0, """, 0 (x + .W4 "on ma* "

CTl,r

at the "

rSt

intervals of length 3"

may

histthe sun, of all the columns of />,+ ,(x,

I,

v) is the vec or *"*"« thi*

P"cess,

we sec' J^- a, „(h „„ ■„ expanded( 1 .. . ,w, ,_L „ throuKh re -■ 1 stimuli but must t t,ltio|ls (occUrring in the1 ' ■»)(!- «)+ {(I - x)«, 0, ■■-,0, (x + .V0«]; in radix 3, it can

have,

at most, two rep

-

\

:'

:i

■

■;

250

CORN

able. It is, therefore , t r J''' ,? ''fl°"d^"* at /*tion is one. ' """"

,wl

> e"Jenf that the probabilityof abs«J'he preceding a 1 iT'T 1 > '^^** f"'fe ioothC X "

n<

' M -"' f'LuttJ?-

Inw!thhSthI

»

254

:

'.

j1 IE

i

.

I

"

CORN

tnrcr chieved -- «Pie, theBash-Gala„ter-I , f muit,P« loop.F„r d"*«- University of I^Sltn 1"'" '« "'ay that therandom selector subro „ ''L h*man ■ *of Tig. I-l.

' 1,1,r

""""e has essentially thescha^

tion of the simulation r, ' Kepamtln« the r:""'"n>he simulated lllZlr T'T-- ''"", "^The Ma*function" methods or I- ' ""'

,h

"o :l skettble "hhran".wh eh Sd ,' ""'' r V " U mUSter P«««» «"ed a '■compiler'-aE£ „ ' Tl r" '' "**"**" :'" aPPr»P"te selection from sucbpsvcMo^ , °f ft M"*"***- Such a procedure would free

,!,,

KS d 'n °"C',"f the mum problems »"tura' «*»'«*' beyondwhhouT r ,m: "" US °°f machine models is "' ".» thee.xpcrime.,lappwp etc m Mf d'StUrbanCe- Penologists can then chLth.dfre T " T amo"g ",; '"-v b? comparing their outcomes withdirect observations of the systems being simulatedReceived: August 26, 1958;revised March Hi, 1950.

References

£"IRWRR- (T-"De"ign ' Brain," W , x„, y„rkD*W«m^UDl?ir MOB!ELL.

I!B,ANDG

(19M). Aformal

stnif

ideUbcw X Mo

,.":.""u""°"s-

/n

"Decision Processes.- Wiley,New York.Wlitv'hv dd, ,"","!''

Shannon,

C.

E.,

and Shapiro, N. (1050). Computa-YmIIZ , VZ

DU,ChineB

- '" "Automata Studies" (C E. Shannonand jMcCarthy, eds.). PnacetOD VuW p^p,^^ New Jerafy

MECIHNICAL SIMULATION" OF HABIT-FORMING AND

LEARNING

259

p - UK (1950). Toward a statistical theory

of

learning.Psychol. Rev. 67.Flli's W. (1957)."An Introduction to Probability Theory and Its Applications,"

Vol 1. Wiley, New York.Gobs S. (195"). Standardized programming methods and universal

coding. J.

Issoc Computing Machinery

4,

July issue.HABMS.BBi.tMAN, and Shapiro, (1953). Studies in functional equations occur-

ring in derision processes. Rand Corp. Hep,. I'

-3*2.

Hu-SOOBFP, F. (1957).

"Sit

Theory."

Chelsea,

New York.Jcsoba, M. I- (1953). Random number generation on the BRL high-speed com-

puting machines. Ballistic Research Lab. Kept. *855.New,;,.,.,

A.,

and H. A.

Simon

(1957). Programming the logic theory

machine;

Proc. Western Joint Computer Conf.Minsky M 1. (1956). Heuristic aspects

of

the artificial intelligenceproblem.Was's. ln.it r.vrWl. tincoln tab. Croup Repl. 34 55.

December,

1956.

Tacssb»-Todd,

I), and T J. (1955)-. Generation and testing

of

pseudo-random

nun,

hers.

Salt.

Bur. Standards if'.S.) Repl. 3370.Rawer, W. li. H953). -The Living Brain."

Norton,

Sew York