the psychological review · by b. f. skinner harvard university certain basic assumptions,...

VOL. 57, No. 4 JULY, 1950

THE PSYCHOLOGICAL REVIEWARE THEORIES OF LEARNING NECESSARY?

BY B. F. SKINNER

Harvard University

Certain basic assumptions, essentialto any scientific activity, are sometimescalled theories. That nature is orderlyrather than capricious is an example.Certain statements are also theoriessimply to the extent that they are notyet facts. A scientist may guess at theresult of an experiment before the ex-periment is carried out. The predictionand the later statement of result maybe composed of the same terms in thesame syntactic arrangement, the differ-ence being in the degree of confidence.No empirical statement is wholly non-'theoretical in this sense, because evi-dence is never complete, nor is any pre-diction probably ever made wholly with-out evidence. The term "theory" willnot refer here to statements of thesesorts but rather to any explanation ofan observed fact which appeals to eventstaking place somewhere else, at someother level of observation, described indifferent terms, and measured, if at all,in different dimensions.

Three types of theory in the field oflearning satisfy this definition. Themost characteristic is to be found in thefield of physiological psychology. Weare all familiar with the changes thatare supposed to take place in the nerv-ous system when an organism learns.

1 Address of the president, Midwestern Psy-chological Association, Chicago, Illinois, May,1949.

Synaptic connections are made orbroken, electrical fields are disruptedor reorganized, concentrations of ionsare built up or allowed to diffuse away,and so on. In the science of neuro-physiology statements of this sort arenot necessarily theories in the presentsense. ' But in a science of behavior,where we are concerned with whether ornot an organism secretes saliva when abell rings, or jumps toward a gray tri-angle, or says bik when a cards readstuz, or loves someone who resembles hismother, all statements about the nerv-ous system are theories in the sense thatthey are not expressed in the same termsand could not be confirmed with thesame methods of observation as thefacts for which they are said to account.

A second type of learning theory is inpractice not far from the physiological,although there is less agreement aboutthe method of direct observation, Theo-ries of this type have always dominatedthe field of human behavior. They con-sist of references to "mental" events, asin saying that an organism, learns to be-have in a certain way because it "findssomething pleasant" or because it "ex-pects something to happen." To thementalistic psychologist these explana-tory events are no more theoretical thansynaptic connections to the neurophysi-ologist, but in a science of behaviorthey are theories because the methods

193

194 B. F. SKINNER

and terms appropriate to the events tobe explained differ from the methodsand terms appropriate to the explainingevents.

In a third type of learning theory theexplanatory events are not directly ob-served. The writer's suggestion that theletters CNS be regarded as representing,not the Central Nervous System, butthe Conceptual Nervous System (2, p.421), seems to have been taken seri-ously. Many theorists point out thatthey are not talking about the nerv-ous system as an actual structure un-dergoing physiological or bio-chemicalchanges but only as a system with acertain dynamic output. Theories ofthis sort are multiplying fast, and so areparallel operational versions of mentalevents. A purely behavioral definitionof expectancy has the advantage thatthe problem of mental observation isavoided and with it the problem of howa mental event can cause a physical one.But such theories do not go so far as toassert that the explanatory events areidentical with the behavioral facts whichthey purport to explain. A statementabout behavior may support such atheory but will never resemble it interms or syntax. Postulates are goodexamples. True postulates cannot hercome facts. Theorems may be deducedfrom them which, as tentative state-ments about behavior, may or may notbe confirmed, but theorems are nottheories in the present sense. Postulatesremain theories until the end.

It is not the purpose of this paper toshow that any of these theories cannotbe put in good scientific order, or thatthe, events to which they refer may notactually occur or be studied by appro-priate sciences. It would be foolhardyto deny the achievements of theories ofthis sort in the history of science. Thequestion of whether they are necessary,however, has other implications and isworth asking. If the answer is no, then

it may be possible to argue effectivelyagainst theory in the field of learning.A science of behavior must eventuallydeal with behavior in its relation to cer-tain manipulable variables. Theories—whether neural, mental, or conceptual—talk about intervening steps in these re-lationships. But instead of promptingus to search for and explore relevantvariables, they frequently have quitethe opposite effect. When we attributebehavior to a neural or mental event,real or conceptual, we are likely to for-get that we still have the task of ac-counting for the neural or mental event.When we assert that an animal acts in agiven way because it expects to receivefood, then what began as the task ofaccounting for learned behavior becomesthe task of accounting for expectancy.The problem is at least equally complexand probably more difficult. We arelikely to close our eyes to.it and to usethe theory to give us answers in place ofthe answers we might find through fur-ther study. It might be argued that theprincipal function of learning theory todate has been, not to suggest appropri-ate research, but to create a false senseof security, an unwarranted satisfactionwith the status quo.

Research designed with respect totheory is also likely to be wasteful.That a theory generates research doesnot prove its value unless the researchis valuable. Much useless experimenta-tion results from theories, and muchenergy and skill are absorbed by them.Most theories are eventually overthrown,and the greater part of the associatedresearch is discarded. This could bejustified if it were true that productiveresearch requires a theory, as is, ofcourse, often claimed. It is argued thatresearch would be aimless and disor-ganized without a theory to guide it.The view is supported by psychologicaltexts that take their cue from the logi-cians rather than empirical science and

ARE THEORIES OF LEARNING NECESSARY? 195

describe thinking as necessarily involv-ing stages of hypothesis, deduction, ex-perimental test, and confirmation. Butthis is not the way most scientists actu-ally work. It is possible to design sig-nificant experiments for other reasonsand the possibility to be examined isthat such research will lead more di-rectly to the kind of information thata science usually accumulates.

The alternatives are at least worthconsidering. How much can be donewithout theory? What other sorts ofscientific activity are possible? Andwhat light do alternative practices throwupon our present preoccupation withtheory?

It would be inconsistent to try to an-swer these questions at a theoreticallevel. Let us therefore turn to someexperimental material in three areas inwhich theories of learning now flourishand raise the question of the function oftheory in a more concrete fashion.2

The Basic Datum in Learning

What actually happens when an or-ganism learns is not an easy question.Those who are interested in a scienceof behavior will insist that learning is achange in behavior, but they tend toavoid explicit references to responses oracts as such. "Learning is adjustment,or adaptation to a situation." But ofwhat stuff are adjustments and adapta-tions made? Are they data, or infer-ences from data? "Learning is improve-ment." But improvement in what? Andfrom whose point of view? "Learning isrestoration of equilibrium." But what

2 Some of the material that follows was ob-tained in 1941-42 in a cooperative study onthe behavior of the pigeon in which KellerBreland, Norman Guttman, and W. K. Estescollaborated. Some of it is selected from sub-sequent, as yet unpublished, work on the pi-geon conducted by the author at Indiana Uni-versity and Harvard University. Limitationsof space make it impossible to report full de-tails here.

is in equilibrium and how is it put there?"Learning is problem solving." But whatare the physical dimensions of a problem—or of a solution? Definitions of thissort show an unwillingness to take whatappears before the eyes in a learningexperiment as a basic datum. Particu-lar observations seem too trivial. Anerror score falls; but we are not readyto say that this is learning rather thanmerely the result of learning. An or-ganism meets a criterion of ten success-ful trials; but an arbitrary criterion isat variance with our conception of thegenerality of the learning process.

This is where theory steps in. If itis not the time required to get out of apuzzle box that changes in learning, butrather the strength of a bond, or theconductivity of a neural pathway, orthe excitatory potential of a habit, thenproblems seem to vanish. Getting outof a box faster and faster is not learn-ing; it is merely performance. Thelearning goes on somewhere else, in adifferent dimensional system. And al-though the time required depends uponarbitrary conditions, often varies dis-continuously, and is subject to reversalsof magnitude, we feel sure that thelearning process itself is continuous, or-derly, and beyond the accidents ofmeasurement. Nothing could betterillustrate the use of theory as a refugefrom the data.

But we must eventually get back toan observable datum. If learning is theprocess we suppose it to be, theri it mustappear so in the situations in which westudy it. Even if the basic process be-longs to some other dimensional system,our measures must have relevant andcomparable properties. But productiveexperimental situations are hard to find,particularly if we accept certain plau-sible restrictions. To show an orderlychange in the behavior of the averagerat or ape or child is not enough, sincelearning is a'process in the behavior of

196 B. F. SKINNER

the individual. To record the beginningand end of learning or a few discretesteps will not suffice, since a series ofcross-sections will not give completecoverage of a continuous process. Thedimensions of the change must springfrom the behavior itself; they must notbe imposed by an external judgment ofsuccess or failure or an external criterionof completeness. But when we reviewthe literature with these requirementsin mind, we find little justification forthe theoretical process in which we takeso much comfort.

The energy level or work-output ofbehavior, for example, does not changein appropriate ways. In the sort of be-havior adapted to the Pavlovian experi-ment (respondent behavior) there maybe a progressive increase in the magni-tude of response during learning. Butwe do not shout our responses louderand louder as we learn verbal material,nor does a rat press a lever harder andharder as conditioning proceeds. Inoperant behavior the energy or magni-tude of response changes significantlyonly when some arbitrary value is dif-ferentially reinforced—when such achange is what is learned.

The emergence of a right response incompetition with wrong responses is an-other datum frequently used in thestudy of learning. The maze and thediscrimination box yield results whichmay be reduced to these terms. But abehavior-ratio of right vs. wrong can-not yield a continuously changing meas-ure in a single experiment on a singleorganism. The point at which one re-sponse takes precedence over anothercannot give us the whole history of thechange in either response. Averagingcurves for groups of trials or organismswill not solve this problem.

Increasing attention has recently beengiven to latency, the relevance of which,like that of energy level, is suggested bythe properties of conditioned and uncon-

ditioned reflexes. But in operant be-havior the relation to a stimulus is dif-ferent. A measure of latency involvesother considerations, as inspection ofany case will show. Most operant re-sponses may be emitted in the absenceof what is regarded as a relevant stimu-lus. In such a case the response islikely to appear before the stimulus ispresented. It is no solution to escapethis embarrassment by locking a leverso that an organism cannot press ituntil the stimulus is presented, since wecan scarcely be content with temporalrelations that have been forced intocompliance with our expectations. Run-way latencies are subject to this objec-tion. In a typical experiment the doorof a starting box is opened and the timethat elapses before a rat leaves the boxis measured. Opening the door is notonly a stimulus, it is a change in thesituation that makes the response pos-sible for the first time. The time meas-ured is by no means as simple as a la-tency and requires another formulation.A great deal depends upon what therat is doing at the moment the stimu-lus is presented. Some experimenterswait until the rat is facing the door,but to do so is to tamper with the meas-urement being taken. If, on the otherhand, the door is opened without refer-ence to what the rat is doing, the firstmajor effect is the conditioning of fa-vorable waiting behavior. The rat even-tually stays near and facing the door.The resulting shorter starting-time isnot due to a reduction in the latency ofa response, but to the conditioning offavorable preliminary behavior.

Latencies in a single organism do notfollow a simple learning process. Rele-vant data on this point were obtained aspart of an extensive study of reactiontime. A pigeon, enclosed in a box, isconditioned to peck at a recessed discin one wall. Food is presented as rein-forcement by exposing a hopper through


a hole below the disc. If responses arereinforced only after a stimulus hasbeen presented, responses at other timesdisappear. Very short reaction timesare obtained by differentially reinforc-ing responses which occur very soonafter the stimulus (4). But responsesalso come to be made very quickly with-out differential reinforcement. Inspec-tion shows that this is due to the de-velopment of effective waiting. Thebird comes to stand before the disc withits head in good striking position. Un-der optimal conditions, without differ-ential reinforcement, the mean time be-tween stimulus and response will be ofthe order of % sec. This is not a truereflex latency, since the stimulus is dis-criminative rather than eliciting, butit is a fair example of the latency usedin the study of learning. The point isthat this measure does not vary con-tinuously or in an orderly fashion. Bygiving the bird more food, for example,we induce a condition in which it doesnot always respond. But the responsesthat occur show approximately the sametemporal relation to the stimulus (Fig.1, middle curve). In extinction, of spe-cial interest here, there is a scatteringof latencies because lack of reinforce-ment generates an emotional condition,

STANDARD HUNGER{All responses re/fibrced)

VERY LOW HUNGER(An responses reinforced)

STANDARD HUNGERIfXTINCTlCH)

4 ~T 4 5 6 7 8 9 10 H 12 13 14 I

RESPO'NSE TIME IN TENTHS OF A SECOND

Fro. 1

Some responses occur sooner and othersare delayed, but the commonest valueremains unchanged (bottom curve inFig. 1). The longer latencies are easilyexplained by inspection. Emotional be-havior, of which examples will be men-tioned later, is likely to be in progresswhen the ready-signal is presented. Itis often not discontinued before the"go" signal is presented, and the resultis a long starting-time. Cases also be-gin to appear in which the bird simplydoes not respond at all during a speci-fied time. If we average a large num-ber of readings, either from one bird ormany, we may create what looks like aprogressive lengthening of latency. Butthe data for an individual organism donot show a continuous process.

Another datum to be examined is therate at which a response is emitted.Fortunately the story here is different.We study this rate by designing a situa-tion in which a response may be freelyrepeated, choosing a response (for ex-ample, touching or pressing a small leveror key) that may be easily observedand counted. The responses may be re-corded on a polygraph, but a more con-venient form is a cumulative curve fromwhich rate of responding is immediatelyread as slope. The rate at which a re-sponse is emitted in such a situationcomes close to our preconception ofthe learning process. As the organismlearns, the rate rises. As it unlearns(for example, in extinction) the ratefalls. Various sorts of discriminativestimuli may be brought into controlof the response with correspondingmodifications of the rate. Motivationalchanges alter the rate in a sensitiveway. So do those events which wespeak of as generating emotion. Therange through which the rate variessignificantly may be as great as of theorder of 1000:1. Changes in rate aresatisfactorily smooth in the individualcase, so that it is not necessary to aver-

198 B. F. SKINNER

age cases. A given value is often quitestable: in the pigeon a rate of four orfive thousand responses per hour maybe maintained without interruption foras long as fifteen hours.

Rate of responding appears to be theonly datum that varies significantly andin the expected direction under condi-tions which are relevant to the "learn-ing process." We may, therefore, betempted to accept it as our long-sought-for measure of strength of bond, excita-tory potential, etc. Once in possessionof an effective datum, however, we mayfeel little need for any theoretical con-struct of this sort. Progress in a scien-tific field usually waits upon the dis-covery of a satisfactory dependent vari-able. Until such a variable has beendiscovered, we resort to theory. Theentities which have figured so promi-nently in learning theory have servedmainly as substitutes for a directly ob-servable and productive datum. Theyhave little reason to survive when sucha datum has been found.

It is no accident that rate of respond-ing is successful as a datum, because itis particularly appropriate to the funda-mental task of a science of behavior.If we are to predict behavior (and pos-sibly to control it), we must deal withprobability of response. The businessof a science of behavior is to evaluatethis probability and explore the condi-tions that determine it. Strength ofbond, expectancy, excitatory potential,and so on, carry the notion of prob-ability in an easily imagined form, butthe additional properties suggested bythese terms have hindered the search forsuitable measures. Rate of respondingis not a "measure" of probability but itis the only appropriate datum in aformulation in these terms.

As other scientific disciplines can at-test, probabilities are not easy to han-dle. We wish to make statements aboutthe likelihood of occurrence of a single

future response, but our data are in theform of frequencies of responses thathave already occurred. These responseswere presumably similar to each otherand to the response to be predicted.But this raises the troublesome problemof response-instance vs. response-class.Precisely what responses are we to takeinto account in predicting a future in-stance? Certainly not the responsesmade by a population of different or-ganisms, for such a statistical datumraises more problems than it solves. Toconsider the frequency of repeated re-sponses in an individual demands some-thing like the experimental situationjust described.

This solution of the problem of a ba-sic datum is based upon the view thatoperant behavior is essentially an emis-sive phenomenon. Latency and magni-tude of response fail as measures be-cause they do not take this into ac-count. They are concepts appropriateto the field of the reflex, where the allbut invariable control exercised by theeliciting stimulus makes the notion ofprobability of response trivial. Con-sider, for example, the case of latency.Because of our acquaintance with sim-ple reflexes we infer that a response thatis more likely to be emitted will beemitted more quickly. But is this true?What can the word "quickly" mean?Probability of response, as well as pre-diction of response, is concerned withthe moment of emission. This is a pointin time, but it does not have the tem-poral dimension of a latency. The exe-cution may take time after the responsehas been initiated, but the moment ofoccurrence has no duration.3 In recog-

3 It cannot, in fact, be shortened or length-ened. Where a latency appears to be forcedtoward a minimal value by differential re-inforcement, another interpretation is calledfor. Although we may differentially reinforcemore energetic behavior or the faster execu-tion of behavior after it begins, it is meaning-less to speak of differentially reinforcing re-


nizing the emissive character of operant'behavior and the central position ofprobability of response as a datum, la-tency is seen to be irrelevant to ourpresent task.

Various objections have been made tothe use of rate of responding as a basicdatum. For example, such a programmay seem to bar us from dealing withmany events which are unique occur-rences in the life of the individual. Aman does not decide upon a career, getmarried, make a million dollars, or getkilled in an accident often enough tomake a rate of response meaningful.But these activities are not responses.They are not simple unitary events lend-ing themselves to prediction as such. Ifwe are to predict marriage, success, acci-dents, and so on, in anything more thanstatistical terms, we must deal with thesmaller units of behavior which lead toand compose these unitary episodes. Ifthe units appear in repeatable form, thepresent analysis may be applied. Inthe field of learning a similar objectiontakes the form of asking how the pres-ent analysis may be extended to experi-mental situations in which it is impos-sible to observe frequencies. It does

spouses with short or long latencies. Whatwe actually reinforce differentially are (a)favorable waiting behavior and (b) mote vig-orous responses. When we ask a subject torespond "as soon as possible" in the humanreaction-time experiment, we essentially askhim (a) to carry out as much of the responseas possible without actually reaching the cri-terion of emission, (b) to do as little else aspossible, and (c) to respond energetically afterthe stimulus has been given. This may yielda minimal measurable time between stimulusand response, but this time is not necessarilya basic datum nor have our instructions al-tered it as such, A parallel interpretation ofthe differential reinforcement of long "laten-cies" is required. This is easily established byinspection. In the experiments with pigeonspreviously cited, preliminary behavior is con-ditioned that postpones the response to thekey until the proper time. Behavior that"marks time" is usually conspicuous.

not follow that learning is not takingplace in such situations. The notionof probability is usually extrapolated tocases in which a frequency analysis can-not be carried out. In the field of be-havior we arrange a situation in whichfrequencies are available as data, butwe use the notion of probability inanalyzing and formulating instances oreven types of behavior which are notsusceptible to this analysis.

Another common objection is that arate of response is just a set of latenciesand hence not a new datum at all. Thisis easily shown to be wrong. When wemeasure the time elapsing between tworesponses, we are in no doubt as to whatthe organism was doing when we startedour clock. We know that it was justexecuting a response. This is a naturalzero—quite unlike the arbitrary pointfrom which latencies are measured. Thefree repetition of a response yields arhythmic or periodic datum very differ-ent from latency. Many periodic physi-cal processes suggest parallels.

We do not choose rate of respondingas a basic datum merely from an analy-sis of the fundamental task of a scienceof behavior. The ultimate appeal is toits success in an experimental science.The material which follows is offered asa sample of what can be done. It isnot intended as a complete demonstra-tion, but it should confirm the factthat when we are in possession of adatum which varies in a significant fash-ion, we are less likely to resort to theo-retical entities carrying the notion ofprobability of response.

Why Learning Occurs

We may define learning as a changein probability of response but we mustalso specify the conditions under whichit comes about. To do this we mustsurvey some of the independent vari-ables of which probability of response is

200 B. F. SKINNER

a function. Here we meet another kindof learning theory,

An effective class-room demonstrationof the Law of Effect may be arrangedin the following way. A pigeon, re-duced to 80 per cent of its ad lib weight,is habituated to a small, semi-circularamphitheatre and is fed there for sev-eral days from a food hopper, which theexperimenter presents by closing a handswitch. The demonstration consists ofestablishing a selected response by suit-able reinforcement with food. For ex-ample, by sighting across the amphi-theatre at a scale on the opposite wall,it is possible to present the hopperwhenever the top of the pigeon's headrises above a given mark. Higher andhigher marks are chosen until, within afew minutes, the pigeon is walking aboutthe cage with its head held as high aspossible. In another demonstration thebird is conditioned to strike a marbleplaced on the floor of the amphitheatre.This may be done in a few minutes byreinforcing successive steps. Food ispresented first when the bird is merelymoving near the marble, later when itlooks down in the direction of themarble, later still when it moves it? headtoward the marble, and finally when itpecks it. Anyone who has seen such ademonstration knows that the Law ofEffect is no theory. It simply specifiesa procedure for altering the probabilityof a chosen response.

But when we try to say why rein-forcement has this effect, theories arise.Learning is said to take place becausethe reinforcement is pleasant, satisfying,tension reducing, and so on. The con-verse process of extinction is explainedwith comparable theories. If the rateof responding is first raised to a highpoint by reinforcement and reinforce-ment then withheld, the response is ob-served to occur less and less frequentlythereafter. One common theory ex-plains this by asserting that a state is

built up which suppresses the behavior.This "experimental inhibition" or "re-action inhibition" must be assigned to adifferent dimensional system, since noth-ing at the level of behavior correspondsto opposed processes of excitation andinhibition. Rate of responding is simplyincreased by one operation and de-creased by another. Certain effectscommonly interpreted as showing re-lease from a suppressing force may beinterpreted in other ways. Disinhibi-tion, for example, is not necessarily theuncovering of suppressed strength; itmay be a sign of supplementary strengthfrom an extraneous variable. The proc-ess of spontaneous recovery, often citedto support the notion of suppression,has an alternative explanation, to benoted in a moment.

Let us evaluate the question of whylearning takes place by turning again tosome data. Since conditioning is usu-ally too rapid to be easily followed, theprocess of extinction will provide uswith a more useful case. A number ofdifferent types of curves have been con-

, sistently obtained from rats and pigeonsusing various schedules of prior rein-forcement. By considering some of therelevant conditions we may see whatroom is left for theoretical processes.

The mere passage of time betweenconditioning and extinction is a vari-able that has surprisingly little effect.The rat is too short-lived to make anextended experiment feasible, but thepigeon, which may live ten or fifteenyears, is an ideal subject. More thanfive years ago, twenty pigeons were con-ditioned to strike a large translucentkey upon which a complex visual pat-tern was projected. Reinforcement wascontingent upon the maintenance of ahigh and steady rate of responding andupon striking a particular feature of thevisual pattern. These birds were setaside in order to study retention. Theywere transferred to the usual living


quarters, where they served as breeders.Small groups were tested for extinctionat the end of six months, one year, twoyears, and four.years. Before the testeach bird was transferred to a separateliving cage. A controlled feeding sched-ule was used to reduce the weight to ap-proximately 80 per cent of the ad libweight. The bird was then fed in thedimly lighted experimental apparatus inthe absence of the key for several days,during which emotional responses to theapparatus disappeared. On the day ofthe test the bird was placed in the dark-ened box. The translucent key waspresent but not lighted. No responseswere made. When the pattern wasprojected upon the key, all four birdsresponded quickly and extensively. Fig.2 shows the largest curve obtained.This bird struck the key within twoseconds after presentation of a visualpattern that it had not seen for fouryears, and at the precise spot uponwhich differential reinforcement hadpreviously been based. It continued torespond for the next hour, emittingabout 700 responses. This is of the or-der of one-half to one-quarter of theresponses it would have emitted if ex-tinction had not been delayed fouryears, but otherwise, the curve is fairlytypical.

Level of motivation is another vari-able to be taken into account. An ex-ample of the effect of hunger has been

-MO

-600

reported elsewhere (3). The responseof pressing a lever was established ineight rats with a schedule of periodicreinforcement. They were fed the mainpart of their ration on alternate days sothat the rates of responding on succes-sive days were alternately high and low.Two subgroups of four rats each werematched on the basis of the rate main-tained under periodic reinforcement un-der these conditions. The response wasthen extinguished—in one group on al-ternate days when the hunger was high,in the other group on alternate dayswhen the hunger was low. (The sameamount of food was eaten on the non-experimental days as before.) The re-sult is shown in Fig. 3. The uppergraph gives the raw data. The levels ofhunger are indicated by the points at Pon the abscissa, the rates prevailing un-der periodic reinforcement. The subse-quent points show the decline in extinc-tion. If we multiply the lower curvethrough by a factor chosen to super-impose the points at P, the curves arereasonably closely superimposed, asshown in the lower graph. Several otherexperiments on both rats and pigeonshave confirmed this general principle.If a given ratio of responding prevailsunder periodic reinforcement, the slopesof later extinction curves show the sameratio. Level of hunger determines theslope of the extinction curve but not itscurvature.

EXTINCTION 4 YEARSAFTER CONDITIONING

MINUTES

FIG. 2

202 B. F. SKINNER

soo

Low Degree of Hunger

Hiqh Degree

P I 2 3 4 5 6 7

Doily Periods of One Hour Each

10 Low Oegrts of Hunqei

,, High Degree

Daily Periods of One Hour Each

FIG. 3

Another variable, difficulty of re-sponse, is especially relevant because ithas been used to test the theory of re-action inhibition (1), on the assump-tion that a response requiring consider-able energy will build up more reactioninhibition than an easy response andlead, therefore, to faster extinction.The theory requires that the curvatureof the extinction curve be altered, notmerely its slope. Yet there is evidencethat difficulty of response acts like levelof hunger simply to alter the slope.Some data have been reported but notpublished (5). A pigeon is suspendedin a jacket which confines its wings and

legs but leaves its head and neck free torespond to a key and a food magazine.Its behavior in this situation is quanti-tatively much like that of a bird movingfreely in an experimental box. But theuse of the jacket has the advantage thatthe response to the key may be madeeasy or difficult by changing the dis-tance the bird must reach. In one ex-periment these distances were expressedin seven equal but arbitrary units. Atdistance 7 the bird could barely reachthe key, at 3 it could strike without ap-preciably extending its neck. Periodicreinforcement gave a straight base-lineupon which it was possible to observethe effect of difficulty by quickly chang-ing position during the experimental pe-riod. Each of the five records in Fig. 4covers a fifteen minute experimental pe-riod under periodic reinforcement. Dis-tances of the bird from the key are indi-cated by numerals above the records.It will be observed that the rate of re-sponding at distance 7 is generally quitelow while that at distance 3 is high.Intermediate distances produce inter-mediate slopes. It should also be notedthat the change from one position toanother is felt immediately. If repeatedresponding in a difficult position wereto build a considerable amount of re-action inhibition, we should expect therate to be low for some little time afterreturning to an easy response. Con-trariwise, if an easy response were tobuild little reaction inhibition, we shouldexpect a fairly high rate of respondingfor some time after a difficult positionis assumed. Nothing like this occurs.The "more rapid extinction" of a diffi-cult response is an ambiguous expres-sion. The slope constant is affected andwith it the number of responses in ex-tinction to a criterion, but there maybe no effect upon curvature.

One way of considering the questionof why extinction curves are curved isto regard extinction as a process of ex-

ABE THEORIES op LEARNING NECESSARY? 203

haustion comparable to the loss of heatfrom source to sink or the fall in thelevel of a reservoir when an outlet isopened. Conditioning builds up a pre-disposition to respond—a "reserve"—which extinction exhausts. This is per-haps a defensible description at the levelof behavior. The reserve is not neces-sarily a theory in the present sense,since it is not assigned to a different di-mensional system. It could be opera-tionally denned as a predicted extinc-tion curve, even though, linguistically, itmakes a statement about the momentarycondition of a response. But it is not a

particularly useful concept, nor does theview that extinction is a process of ex-haustion add much to the observed factthat extinction curves are curved in acertain way.

There are, however, two variablesthat affect the rate, both of which oper-ate during extinction to alter the curva-ture. One of these falls within the fieldof emotion. When we fail to reinforcea response that has previously been re-inforced, we not only initiate a processof extinction, we set up an emotionalresponse—perhaps what is often meantby frustration. The pigeon coos in an

TIME JN MINUTES

Fio. 4

204 B. F. SKINNER

identifiable pattern, moves rapidly aboutthe cage, defecates, or flaps its wingsrapidly in a squatting position that sug-gests treading (mating) behavior. Thiscompetes with the response of strikinga key and is perhaps enough to accountfor the decline in rate in early extinc-tion. It is also possible that the prob-ability of a response based upon fooddeprivation is directly reduced as partof such an emotional reaction. What-ever its nature, the effect of this vari-able is eliminated through adaptation.Repeated extinction curves becomesmoother, and in some of the schedulesto be described shortly there is little orno evidence of an emotional modifica-tion of rate.

A second variable has a much more•serious effect. Maximal responding dur-ing extinction is obtained only when theconditions under which the response wasreinforced are precisely reproduced. Arat conditioned in the presence of alight will not extinguish fully in the ab-sence of the light. It will begin to re-spond more rapidly when the light isagain introduced. This is true for otherkinds of stimuli, as the following class-room experiment illustrates. Nine pi-geons were conditioned to strike a yel-low triangle under intermittent reinforce-ment. In the session represented byFig. 5 the birds were first reinforced onthis schedule for 30 minutes. The com-bined cumulative curve is essentially astraight line, showing more than 1100responses per bird during this period.A red triangle was then substituted forthe yellow and no responses were rein-forced thereafter. The effect was asharp drop in responding, with only aslight recovery during the next fifteenminutes. When the yellow triangle wasreplaced, rapid responding began im-mediately and the usual extinction curvefollowed. Similar experiments haveshown that the pitch of an incidentaltone, the shape of a pattern being

1800

1600

1400

1200

inB 1000

I13 800EC

600

400

200

-Aperiodic• reinforcement

EXTINCTION-

20 30 40 SO 60 70TIME IN MINUTES

FIG. S

struck, or the size of a pattern, if pres-ent during conditioning, will to someextent control the rate of respondingduring extinction. Some properties aremore effective than others, and a quan-titative evaluation is possible. Bychanging to several values of a stimu-lus in random order repeatedly duringthe extinction process, the gradient forstimulus generalization may be readdirectly in the rates of responding un-der each value.

Something very much like this mustgo on during extinction. Let us supposethat all responses to a key have beenreinforced and that each has been fol-lowed by a short period of eating.When we extinguish the behavior, wecreate a situation in which responsesare not reinforced, in which no eatingtakes place, and in which there areprobably new emotional responses. Thesituation could easily be as novel as ared triangle after a yellow. If so, itcould explain the decline in rate duringextinction. We might have obtained a


smooth curve, shaped like an extinctioncurve, between the vertical lines in Fig.5 by gradually changing the color of thetriangle from yellow to red. This mighthave happened even though no othersort of extinction were taking place.The very conditions of extinction seemto presuppose a growing novelty in theexperimental situation. Is this why theextinction curve is curved?

Some evidence comes from the dataof "spontaneous recovery." Even afterprolonged extinction an organism willoften respond at a higher fate for atleast a few moments at the beginning ofanother session. One theory contendsthat this shows spontaneous recoveryfrom some sort of inhibition, but an-other explanation is possible. No mat-ter how carefully an animal is handled,the stimulation coincident with the be-ginning of an experiment must be ex-tensive and unlike anything occurringin the later part of an experimental pe-riod. Responses have been reinforcedin the presence of, or shortly following,the organism is again placed in the ex-perimental situation, the stimulation isthis stimulation. In extinction it ispresent for only a few moments. When

restored; further responses are emittedas in the case of the yellow triangle.The only way to achieve full extinctionin the presence of the stimulation ofstarting an experiment is to start the ex-periment repeatedly.

Other evidence of the effect of nov-elty comes from the study of periodicreinforcement. The fact that intermit-tent reinforcement produces bigger ex-tinction curves than continuous rein-forcement is a troublesome difficultyfor those who expect a simple relationbetween number of reinforcements andnumber of responses in extinction. Butthis relation is actually quite complex.One result of periodic reinforcement isthat emotional changes adapt out. Thismay be responsible for the smoothnessof subsequent extinction curves butprobably not for their greater extent.The latter may be attributed to the lackof novelty in the extinction situation.Under periodic reinforcement many re-sponses are made without reinforcementand when no eating has recently takenplace. The situation in extinction istherefore not wholly novel.

Periodic reinforcement is not, how-ever, a simple solution. If we reinforce

tnUJU)

0Q.<nUJa:

TIME

FIG. 6

206 B. F. SKINNER

100

TIME IN MINUTES

Fio. 1

on a regular schedule—say, every min-ute—the organism soon forms a dis-crimination. Little or no respondingoccurs just after reinforcement, sincestimulation from eating is correlatedwith absence of subsequent reinforce-ment. How rapidly the discriminationmay develop is shown in Fig. 6, whichreproduces the first five curves obtainedfrom a pigeon under periodic reinforce-ment in experimental periods of fifteenminutes each. In the fifth period (orafter about one hour of periodic rein-forcement) the discrimination yields apause after each reinforcement, result-ing in a markedly stepwise curve. Asa result of this discrimination the birdis almost always responding rapidlywhen reinforced. This is the basis foranother discrimination. Rapid respond-ing becomes a favorable stimulatingcondition. A good example of the ef-fect upon the subsequent extinctioncurve is shown in Fig. 7. This pigeonhad been reinforced once every minuteduring daily experimental periods offifteen minutes each for several weeks.In the extinction curve shown, the birdbegins to respond at the rate prevailingunder the preceding schedule. A quickpositive acceleration at the start is lostin the reduction of the record. The

pigeon quickly reaches and sustains arate that is higher than the overall-rateduring periodic reinforcement. Duringthis period the pigeon creates a stimu-lating condition previously optimallycorrelated with reinforcement. Even-tually, as some sort of exhaustion inter-venes, the rate falls off rapidly to amuch lower but fairly stable value andthen to practically zero. A conditionthen prevails under which a response isnot normally reinforced. The bird istherefore not likely to begin to respondagain. When it does respond, however,the situation is slightly improved and,if it continues to respond, the condi-tions rapidly become similar to thoseunder which reinforcement has been re-ceived. Under this "autocatalysis" ahigh rate is quickly reached, and morethan SOO responses are emitted in asecond burst. The rate then declinesquickly and fairly smoothly, again tonearly zero. This curve is not by anymeans disorderly. Most of the curva-ture is smooth. But the burst of re-sponding at forty-five minutes showsa considerable residual strength which,if extinction were merely exhaustion,should have appeared earlier in thecurve. The curve may be reasonablyaccounted for by assuming that the


bird is largely controlled by the preced-ing spurious correlation between rein-forcement and rapid responding.

This assumption may be checked byconstructing a schedule of reinforce-ment in which a differential contingencybetween rate of responding and rein-forcement is impossible. In one suchschedule of what may be called "aperi-odic reinforcement" one interval be-tween successive reinforced responses isso short that no unreinforced responsesintervene while the longest interval isabout two minutes. Other intervals aredistributed arithmetically between thesevalues, the average remaining one min-ute, The intervals are roughly random-ized to compose a program of reinforce-ment. Under this program the prob-ability of reinforcement does not changewith respect to previous reinforcements,and the curves never acquire the step-wise character of curve E in Fig. 6.(Fig. 9 shows curves from a similarprogram.) As a result no correlationbetween different rates of respondingand different probabilities of reinforce-ment can develop.

An extinction curve following a briefexposure to aperiodic reinforcement isshown in Fig. 8. It begins character-istically at the rate prevailing underaperiodic reinforcement and, unlike thecurve following regular periodic rein-

forcement, does not accelerate to ahigher overall rate. There is no evi-dence of the "autocatalytic" productionof an optimal stimulating condition.Also characteristically, there are nosignificant discontinuities or suddenchanges in rate in either direction. Thecurve extends over a period of eighthours, as against not quite two hoursin Fig. 7, and seems to represent asingle orderly process. The total num-ber of responses is higher, perhaps be-cause of the greater time allowed foremission. All of this can be explainedby the single fact that we have made, itimpossible for the pigeon to form a pairof discriminations based, first, uponstimulation from eating and, second,upon stimulation from rapid responding.

Since the longest interval between re-inforcement was only two minutes, acertain novelty must still have been in-troduced as time passed. Whether thisexplains the curvature in Fig. 8 may betested to some extent with other pro-grams of reinforcement containing muchlonger intervals. A geometric progres-sion was constructed by beginning with10 seconds as the shortest interval andrepeatedly multiplying through by aratio of 1.S4. This yielded a set ofintervals averaging S minutes, the long-est of which was more than 21 minutes.Such a set was randomized in a program

4QOOr ONE PIGEON, ONECONTINUOUS PERIOD

to 3000en

o 2000Q.CrtLJ

K 1000

EXTINCTION(After aperiodic reinforcement,

arithmetic series,mean: one reinforcement per minute)

_L4

TIME IN

FIG. 8

J_5

HOURS

_L8

208 B. F. SKINNER

FIG. 9

of reinforcement repeated every hour.In changing to this program from thearithmetic series, the rates first declinedduring the longer intervals, but the pi-geons were soon able to sustain a con-stant rate of responding under it. Tworecords in the form in which they wererecorded are shown in Fig. 9. (Thepen resets to zero after every thousandresponses. In order to obtain a singlecumulative curve it would be necessaryto cut the record and to piece the sec-tions together to yield a continuous line.The raw form may be reproduced withless reduction.) Each reinforcement isrepresented by a horizontal dash. Thetime covered is about 3 hours. Recordsare shown for two pigeons that main-tained different overall rates under thisprogram of reinforcement.

Under such a schedule a constant rateof responding is sustained for at least21 minutes without reinforcement, afterwhich a reinforcement is received. Lessnovelty should therefore develop duringsucceeding extinction. In Curve 1 ofFig. 10 the pigeon had been exposed toseveral sessions of several hours eachwith this geometric set of intervals.The number of responses emitted in ex-tinction is about twice that of the curvein Fig. 8 after the arithmetic set of in-

tervals averaging one minute, but thecurves are otherwise much alike. Fur-ther exposure to the geometric sched-ule builds up longer runs during whichthe rate does not change significantly.Curve 2 followed Curve 1 after two andone-half hours of further aperiodic re-inforcement. On the day shown inCurve 2 a few aperiodic reinforcementswere first given, as marked at the be-ginning of the curve. When reinforce-ment was discontinued, a fairly con-stant rate of responding prevailed forseveral thousand responses. After an-other experimental session of two andone-half hours with the geometric se-ries, Curve 3 was recorded. This ses-sion also began with a short series ofaperiodic reinforcements, followed by asustained run of more than 6000 unrein-forced responses with little change inrate (A). There seems to be no reasonwhy other series averaging perhaps morethan five minutes per interval and con-taining much longer exceptional inter-vals would not carry such a straightline much further.

In this attack upon the problem of ex-tinction we create a schedule of rein-forcement which is so much like theconditions that will prevail during ex-tinction that no decline in rate takes


place for a long time. In other wordswe generate extinction with no curva-ture. Eventually some kind of exhaus-tion sets in, but it is not approachedgradually. The last part of Curve 3(unfortunately much reduced in the fig-ure) may possibly suggest exhaustion inthe slight overall curvature, but it is asmall part of the whole process. Therecord is composed mainly of runs of afew hundred responses each, most ofthem at approximately the same rate asthat maintained under periodic rein-forcement. The pigeon stops abruptly;when it starts to respond again, itquickly reaches the rate of respondingunder which it was reinforced. Thisrecalls the spurious correlation betweenrapid responding and reinforcement un-

der regular reinforcement. We havenot, of course, entirely eliminated thiscorrelation. Even though there is nolonger a differential reinforcement ofhigh against low rates, practically allreinforcements have occurred under aconstant rate of responding.

Further study of reinforcing sched-ules may or may not answer the ques-tion of whether the novelty appearing inthe extinction situation is entirely re-sponsible for the curvature. It wouldappear to be necessary to make theconditions prevailing during extinctionidentical with the conditions prevailingduring conditioning. This may be im-possible, but in that case the question isacademic. The hypothesis, meanwhile,is not a theory in the present sense,

EXTINCTION(After continued aperiodic reinforcement,

geometric series)

T I M E

FIG. 10

210 B. F. SKINNER

since it makes no statements about aparallel process in any other universe ofdiscourse.*

The study of extinction after differ-ent schedules of aperiodic reinforcementis not addressed wholly to this hypothe-sis. The object is an economical de-scription of the conditions prevailingduring reinforcement and extinction andof the relations between them. In us-ing rate of responding as a basic datumwe may appeal to conditions that areobservable and manipulable and we mayexpress the relations between them inobjective terms. To the extent that ourdatum makes this possible, it reducesthe need for theory. When we observea pigeon emitting 7000 responses at aconstant rate without reinforcement, weare not likely to explain an extinctioncurve containing perhaps a few hundredresponses by appeal to the piling up ofreaction inhibition or any other fatigueproduct. Research which is conductedwithout commitment to theory is morelikely to carry the study of extinctioninto new areas and new orders ofmagnitude. By hastening the accumu-lation of data, we speed the departureof theories. If the theories have playedno part in the design of our experi-ments, we need not be sorry to seethem go.

Complex Learning

A third type of learning theory isillustrated by terms like preferring,choosing, discriminating, and matching.An effort may be made to define thesesolely in terms of behavior, but in tradi-tional practice they refer to processesin another dimensional system. A re-

4 It is true that it appeals to stimulationgenerated in part by the pigeon's own be-havior. This may be difficult to specify ormanipulate, but it is not theoretical in thepresent sense. So long as we are willing toassume a one-to-one correspondence betweenaction and stimulation, a physical specificationis possible.

sponse to one of two available stimulimay be called choice, but it is com-moner to say that it is the result ofchoice, meaning by the latter a theo-retical pre-behavioral activity. Thehigher mental processes are the bestexamples of theories of this sort; neuro-logical parallels have not been wellworked out. The appeal to theory isencouraged by the fact that choosing(like discriminating, matching, and soon) is not a particular piece of behavior.It is not a response or an act with speci-fied topography. The term character-izes a larger segment of behavior inrelation to other variables or events.Can we formulate and study the behav-ior to which these terms would usuallybe applied without recourse to the theo-ries which generally accompany them?

Discrimination is a relatively simplecase. Suppose we find that the prob-ability of emission of a given responseis not significantly affected by chang-ing from one of two stimuli to the other.We then^make reinforcement of the re-sponse contingent upon the presence ofone of them. The well-established re-sult is that the probability of responseremains high under this stimulus andreaches a very low point under theother. We say that the organism nowdiscriminates between the stimuli. Butdiscrimination is not itself an action, ornecessarily even a unique process. Prob-lems in the field of discrimination maybe stated in other terms. How muchinduction obtains between stimuli ofdifferent magnitudes or classes? Whatare the smallest differences in stimulithat yield a difference in control? Andso on. Questions of this sort do notpresuppose theoretical activities in otherdimensional systems.

A somewhat larger segment must bespecified in dealing with the behavior ofchoosing one of two concurrent stimuli.This has been studied in the pigeon byexamining responses to two keys differ-


ing in position (right or left) or in someproperty like color randomized with re-spect to position. By occasionally rein-forcing a response on one key or theother without favoring either key, weobtain equal rates of responding on thetwo keys. The behavior approaches asimple alternation from one key to theother. This follows the rule that tend-encies to respond eventually correspondto the probabilities of reinforcement.Given a system in which one key or theother is occasionally connected with themagazine by an external clock, then ifthe right key has just been struck, theprobability of reinforcement via the leftkey is higher than that via the rightsince a greater interval of time haselapsed during which the clock mayhave closed the circuit to the left key.But the bird's behavior does not cor-respond to this probability merely outof respect for mathematics. The spe-cific result of such a contingency ofreinforcement is that changing-to-the-other-key-and-striking is more often re-inforced than striking-the-same-key-a-second-time. We are no longer dealingwith just two responses. In order toanalyze "choice" we must consider asingle final response, striking, withoutrespect to the position or color of thekey, and in addition the responses ofchanging from one key or color to theother.

Quantitative results are compatiblewith this analysis. If we periodicallyreinforce responses to the right keyonly, the rate of responding on theright will rise while thajt on the left willfall. The response of changing-from-right-to-left is never reinforced whilethe response of changing-from-left-to-right is occasionally so. When the birdis striking on the right, there is no greattendency to change keys; when it isstriking on the left, there is a strongtendency to change. Many more re-sponses come to be made to the right

key. The need for considering the be-havior of changing over is clearly shownif we now reverse these conditions andreinforce responses to the left key only.The ultimate result is a high rate of re-sponding on the left key and a low rateon the right. By reversing the condi-tions again the high rate can be shiftedback to the right key. In Fig. 11 agroup of eight curves have been aver-aged to follow this change during sixexperimental periods of 45 minuteseach. Beginning on the second day inthe graph responses to the right key(RR) decline in extinction while re-sponses to the left key (RL) increasethrough periodic reinforcement. Themean rate shows no significant varia-

o

toUJCO

1•ffl-

1 2 3 4 5 6

DAYSFIG. 11

212 B. F. SKINNER

TIME IN MINUTESFIG. 12

tion, since periodic reinforcement is con-tinued on the same schedule. The meanrate shows the condition of strength ofthe response of striking a key regard-less of position. The distribution of re-sponses between right and left dependsupon the relative strength of the re-sponses of changing over. If this weresimply a case of the extinction of oneresponse and the concurrent recondi-tioning of another, the mean curvewould not remain approximately hori-zontal since reconditioning occurs muchmore rapidly than extinction.6

The rate with which the bird changesfrom one key to the other depends uponthe distance between the keys. This dis-tance is a rough measure of the stimu-lus-difference between the two keys. It

B Two topographically independent responses,capable of emission at the same time and hencenot requiring change-over, show separate proc-esses of reconditioning and extinction, and thecombined rate of responding varies.

also determines the scope of the re-sponse of changing-over, with an im-plied difference in sensory feed-back.It also modifies the-spread of reinforce-ment to responses supposedly not rein-forced, since if the keys are close to-gether, a response reinforced on oneside may occur sooner after a precedingresponse on the other side. In Fig. 11the two keys were about one inch apart.They were therefore fairly similar withrespect to position in the experimentalbox. Changing from one to the otherinvolved a minimum of sensory feed-back, and reinforcement of a responseto one key could follow very shortlyupon a response to the other. Whenthe keys are separated by as much asfour inches, the change in strength ismuch more rapid. Fig. 12 shows twocurves recorded simultaneously from asingle pigeon during one experimentalperiod of about 40 minutes. A high rate


to the right key and a low rate to theleft had previously been established. Inthe figure no responses to the right werereinforced, but those to the left were re-inforced every minute as indicated bythe vertical dashes above curve L. Theslope of R declines in a fairly smoothfashion while that of L increases, alsofairly smoothly, to a value comparableto the initial value of R. The bird hasconformed to the changed contingencywithin a single experimental period.The mean rate of responding is shownby a dotted line, which again shows nosignificant curvature.

What is called "preference" entersinto this formulation. At any stage ofthe process shown in Fig. 12 preferencemight be expressed in terms of the rela-tive rates of responding to the two keys.This preference, however, is not in strik-ing a key but in changing from one keyto the other. The probability that thebird will strike a key regardless of itsidentifying properties behaves independ-ently of the preferential response ofchanging from one key to the other.Several experiments have revealed anadditional fact. A preference remainsfixed if reinforcement is withheld. Fig.13 is an example. It shows simultane-ous extinction curves from two keysduring seven daily experimental periodsof one hour each. Prior to extinctionthe relative strength of the responsesof changing-to-R and changing-to-Lyielded a "preference" of about 3 to 1

DAYS

FIG. 13

for R. The constancy of the ratethroughout the process of extinction hasbeen shown in the figure by multiply-ing L through by a suitable constantand entering the points as small circleson R. If extinction altered the prefer-ence, the two curves could not be super-imposed in this way.

These formulations of discriminationand choosing enable us to deal withwhat is generally regarded as a muchmore complex process—matching tosample. Suppose we arrange threetranslucent keys, each of which may beilluminated with red or green light. Themiddle key functions as the sample andwe color it either red or green in ran-dom order. We color the two side keysone red and one green, also in randomorder. The "problem" is to strike theside key which corresponds in color tothe middle key. There are only fourthree-key patterns in such a case, andit is possible that a pigeon could learnto make an appropriate response to eachpattern. This does not happen, at leastwithin the temporal span of the experi-ments to date. If we simply present aseries of settings of the three colors andreinforce successful responses, the pi-geon will strike the side keys withoutrespect to color or pattern and be rein-forced SO per cent of the time. This is,in effect, a schedule of "fixed ratio" re-inforcement which is adequate to main-tain a high rate of responding.

Nevertheless it is possible to get apigeon to match to sample by rein-forcing the discriminative responsesof striking-red-after-being-stimulated-by-red and striking-green-after-being-stimulated-by-green while extinguishingthe other two possibilities. The diffi-culty is in arranging the proper stimu-lation at the time of the response. Thesample might be made conspicuous—for example, by having the sample colorin the general illumination of the ex-perimental box. In such a case the pi-

214 B. F. SKINNER

geon would learn to strike red keys ina red light and green keys in a greenlight (assuming a neutral illuminationof the background of the keys). Buta procedure which holds more closelyto the notion of matching is to inducethe pigeon to "look at the sample" bymeans of a separate reinforcement. Wemay do this by presenting the color onthe middle key first, leaving the sidekeys uncolored. A response to the mid-dle key is then reinforced (secondarily)by illuminating the side keys. The pi-geon learns to make two responses inquick succession—to the middle key andthen to one side key. The response tothe side key follows quickly upon thevisual stimulation from the middle key,which is the requisite condition for adiscrimination. Successful matching wasreadily established in all ten pigeonstested with this technique. Choosingthe opposite is also easily set up. Thediscriminative response of striking-red-after-being-stimulated-by-red is appar-ently no easier to establish than strik-ing-red-after-being-stimulated-by-green.When the response is to a key of thesame color, however, generalization maymake it possible for the bird to match anew color. This is an extension of thenotion of matching that has not yetbeen studied with this method.

Even when matching behavior hasbeen well established, the bird will notrespond correctly if all three keys arenow presented at the same time. Thebird does not possess strong behaviorof looking at the sample. The experi-menter must maintain a separate rein-forcement to keep this behavior instrength. In monkeys, apes, and hu-man subjects the ultimate success inchoosing is apparently sufficient to re-inforce and maintain the behavior oflooking at the sample. It is possiblethat this species difference is simply adifference in the temporal relations re-quired for reinforcement.

The. behavior of matching survivesunchanged when all reinforcement iswithheld. An intermediate case hasbeen established in which the correctmatching response is only periodicallyreinforced. In one experiment one colorappeared on the middle^ key for oneminute; it was then changed or notchanged, at random, to the other color.A response to this key illuminated theside- keys, one red and one green, inrandom order. A response to a sidekey cut off the illumination to both sidekeys, until the middle key had againbeen struck. The apparatus recordedall matching responses on one graphand all non-matching on .another. Pi-geons which have acquired matchingbehavior under continuous reinforce-ment have maintained this behaviorwhen reinforced no oftener than onceper minute on the average. They maymake thousands of matching responsesper hour while being reinforced for nomore than sixty of them. This sched-ule .will not necessarily develop match-ing behavior in a naive bird, for theproblem can be solved in three ways.The bird will receive practically asmany reinforcements if it responds to(1) only one key or (2) only one color,since the programming of the experi-ment makes any persistent responseeventually the correct one.

A sample of the data obtained in acomplex experiment of this sort is givenin Fig. 14. Although this pigeon hadlearned to match color under continuousreinforcement, it changed to the spuri-ous solution of a color preference underperiodic reinforcement. Whenever thesample was red, it struck both the sam-ple and the red side key and received allreinforcements. When the sample wasgreen, it did not respond and the sidekeys were not illuminated. The resultshown at the beginning of the graph inFig. 14 is a high rate of responding onthe upper graph, which records match-


ing responses. (The record is actuallystep-wise, following the presence or ab-sence of the red sample, but this is lostin the reduction in the figure.) A colorpreference, however, is not a solution tothe problem of opposites. By chang-ing to this problem, it was possible tochange the bird's behavior as shown be-tween the two vertical lines in the fig-ure. The upper curve between theselines shows the decline in matching re-sponses which had resulted from thecolor preference. The lower curve be-tween the same lines shows the develop-ment of responding to and matching theopposite color. At the second verticalline the reinforcement was again madecontingent upon matching. The uppercurve shows the reestablishment ofmatching behavior while the lower curveshows a decline in striking the oppositecolor. The result was a true solution:the pigeon struck the sample, no mat-ter what its color, and then the corre-sponding side key. The lighter lineconnects the means of a series of points

30 60 90 120TIME IN MINUTES

ITxelvt experimental periods joined)

FIG. 14

on the two curves. It seems to followthe same rule as in the case of choos-ing: changes in the distribution of re-sponses between two keys do not in-volve the over-all rate of responding toa key. This mean rate will not remainconstant under the spurious solutionachieved with a color preference, as atthe beginning of this figure.

These experiments on a few higherprocesses have necessarily been verybriefly described. They are not of-fered as proving that theories of learn-ing are not necessary, but they maysuggest an alternative program in thisdifficult area. The data in the field ofthe higher mental processes transcendsingle responses or single stimulus-re-sponse relationships. But they appearto be susceptible to formulation in termsof the differentiation of concurrent re-sponses, the discrimination of stimuli,the establishment of various sequencesof responses, and so on. There seemsto be no a priori reason why a completeaccount is not possible without appealto theoretical processes in other dimen-sional systems.

Conclusion

Perhaps to do without theories alto-gether is a tour de force that is toomuch to expect as a general practice.Theories are fun. But it is possiblethat the most rapid progress toward anunderstanding of learning may be madeby research that is not designed to testtheories. An adequate impetus is sup-plied by the inclination to obtain datashowing orderly changes characteristicof the learning process. An acceptablescientific program is to collect data ofthis sort and to relate them to ma-nipulable variables, selected for studythrough a common sense exploration ofthe field.

This does not exclude the possibilityof theory in another sense. Beyond thecollection of uniform relationships lies

216 B. F. SKINNER

the need for a formal representation ofthe data reduced to a minimal numberof terms. A theoretical constructionmay yield greater generality than anyassemblage of facts. But such a con-struction will not refer to another di-mensional system and will not, there-fore, fall within our present definition.It will not stand in the way of oursearch for functional relations becauseit will arise only after relevant variableshave been found and studied. Thoughit may be difficult to understand, it willnot be easily misunderstood, and it willhave none of the objectionable effects ofthe theories here considered.

We do not seem to be ready fortheory in this sense. At the momentwe make little effective use of empirical,let alone rational, equations. A few ofthe present curves could have beenfairly closely fitted. But the most ele-mentary preliminary research shows that

there are many relevant variables, anduntil their importance has been experi-mentally determined, an equation thatallows for them will have so many arbi-trary constants that a good fit will be amatter of course and a cause for verylittle satisfaction.

REFERENCES

1. MOWRER, 0. H., & JONES, H. M, Extinc-tion and behavior variability as func-tions of effortfulness of task. /. exp.Psychol, 1943, 33, 369-386.

2. SKINNER, B. F. The behavior of organisms.New York: D. Appleton-Century Co.,1938.

3. . The nature of the operant reserve.Psychol. Butt., 1940, 37, 423 (abstract).

4. , Differential reinforcement with re-spect to time. Amer. Psychol., 1946, 1,274-275 (abstract).

S. . The effect of the difficulty of a re-sponse upon its rate of emission. Amer.Psychol, 1946, 1, 462 (abstract).

[MS. received December 5, 1949]

the psychological review · by b. f. skinner harvard university certain basic assumptions,...

Documents