469 number (november) variable-ratio mcdowell and

JOURNAL OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR

VARIABLE-RATIO SCHEDULES AS VARIABLE-INTERVALSCHEDULES WITH LINEAR FEEDBACK LOOPS

J. J MCDOWELL AND JOHN T. WIXTED

EMORY UNIVERSITY

The mathematical theory of linear systems has been used successfully to describe responding onvariable-interval (VI) schedules. In the simplest extension of the theory to the variable-ratio (VR)case, VR schedules are treated as if they were VI schedules with linear feedback loops. The assump-tion entailed by this approach, namely, that VR and VI-plus-linear-feedback schedules are equivalent,was tested by comparing responding on the two types of schedule. Four human subjects' lever pressingproduced monetary reinforcers on five VR schedules, and on five VI schedules with linear feedbackloops that reproduced the feedback properties of the VR schedules. Pressing was initiated by instruc-tions in 2 subjects, and was shaped by successive approximation in the other 2. The different methodsof response initiation did not have differential effects on behavior. For each of the 4 subjects, the VRand the comparable VI-plus-linear-feedback schedules generated similar average response rates andsimilar response patterns. The subjects' behavior on both types of schedule was similar to that of avianand rodent species on VR schedules. These results indicate that the assumption entailed by the VI-plus-linear-feedback approach to the VR case is valid and, consequently, that the approach is worthpursuing. The results also confute interresponse-time theories of schedule performance, which requireinterval and ratio contingencies to produce different response rates.

Key words: variable-ratio schedules, feedback, linear system theory, interresponse-time theory, shap-ing, ratio strain, lever press, humans

McDowell and Kessel (1979) used themathematical theory of linear systems (Asel-tine, 1958) to describe responding on variable-interval (VI) schedules of reinforcement. Thelinear system theory is a set of mathematicaltechniques that can be used to calculate theresponse of a system to a known input, pro-vided the system can be described at least inprinciple by a linear differential equation. Theresult of the analysis for the VI case was amean-value rate equation (cf. McDowell, Bass,& Kessel, 1983), which can be written(McDowell, 1980):

ROUT = {ln[1 + PB (e 1 1)]} (1)

This equation expresses response rate, ROUT,on a VI schedule as a joint function of rein-

This research was supported by a grant to J. JMcDowell from the Emory University Research Com-mittee. Preparation of the manuscript was supported inpart by National Institute of Mental Health GrantMH38046 to Emory University, J. J McDowell, prin-cipal investigator. We thank Margaret M. Quinlin forpreparing the figures, and Mark Hoyert for commentingon an earlier version of the paper. Requests for reprintsshould be sent to J. J McDowell, Department of Psy-chology, Emory University, Atlanta, Georgia 30322.

forcement rate, RIN, reinforcer value, PR, andresponse aversiveness, PB. Gamma is a scalarconstant that is characteristic of the organism.The quantities, PR and PB, as well as methodsof measuring them, have been discussed in de-tail elsewhere (McDowell, 1980, 1986).

Equation 1 provides an excellent descriptionof VI responding (McDowell, 1980, 1986;McDowell & Kessel, 1979) that is superior tothe descriptions provided by seven other math-ematical accounts of the VI case (Catania, 1973;Killeen, 1981, 1982; Rachlin, 1978; Staddon,1977, 1979), including Herrnstein's (1970)matching-based account (McDowell et al.,1983; McDowell & Kessel, 1979; McDowell& Wood, 1984, 1985). Although several prop-erties of Equation 1 remain to be tested(McDowell, 1986), the success of the linear-system description of the VI case indicates thatapplications of the theory to other cases mightalso be successful. In the present article weconsider how the theory might be applied toresponding on variable-ratio (VR) schedules.McDowell (1979, 1980) pointed out that

the simplest mathematical approach to the VRcase was to treat it as VI respondirg, but withthe addition of a feedback loop. Of course feed-back also occurs on VI schedules, but it isrestricted to a small range of low response rates

315

1986, 469 315-329 NUMBER 3 (NOVEMBER)

J. J McDOWELL and JOHN T. WIXTED

and is minimal in comparison to VR feedback,which operates at all response rates (seeMcDowell, 1980). Feedback approaches tounderstanding behavior are not new; they havefigured prominently in the work of Baum(1973, 1981), Rachlin (1978), and Staddonand Motheral (1978), among others.

Feedback on a VR schedule is determinedby the average ratio requirement, fn, that theschedule arranges. If the responses specifiedby this average requirement are emitted slowly,reinforcers will be delivered slowly, and if theyare emitted rapidly, reinforcers will be deliv-ered rapidly. The direct relationship betweenreinforcement rate and response rate on VRschedules is described by the feedback function,

RIN = (1/fi)ROUT, (2)

which is a line with intercept equal to zeroand slope equal to the reciprocal of the aver-age ratio requirement. For VR schedules,Equation 2 is true by definition because itdescribes a defining, or necessary, property ofthese schedules. However, the linear feedbackdescribed by Equation 2 can be arranged in-dependently of a ratio contingency because 1/fnis simply a number that can assume any finitevalue greater than zero.

Because Equation 1 was written to describeresponding on simple VI schedules, it is evi-dent that the linear system theory requires thecomposite of Equation 2 on Equation 1 todescribe responding on a VI schedule to whicha linear feedback loop has been added. Thistype of schedule is not difficult to arrange.Reinforcement on a VI-plus-linear-feedbackschedule is delivered according to a time-based,or interval, contingency, just like on a simpleVI schedule. However, reinforcement rate onthis type of schedule increases linearly withresponse rate according to Equation 2. In otherwords, throughout the session the effective av-erage interreinforcement interval (or VI value,i.e., 1/RIN) varies directly with the averageinterresponse time (1/RouT). Higher responserates produce smaller mean VI values (i.e.,higher reinforcement rates) and lower re-sponse rates produce larger mean VI values(i.e., lower reinforcement rates). According tothe linear system theory, responding on thistype of schedule must occur in the followingmanner. An initial response rate produces aninitial reinforcement rate according to Equa-

tion 2. This reinforcement rate produces a newresponse rate according to Equation 1, whichin turn produces a new reinforcement rate ac-cording to Equation 2, and so on, until re-sponding reaches equilibrium, where Equa-tions 1 and 2 are satisfied simultaneously.

In McDowell's (1979, 1980) suggested ap-proach to the VR case, VR schedules aretreated mathematically as if they were VIschedules with linear feedback loops. In otherwords, the two types of schedule are assumedto be equivalent. Implicit in this assumptionis the view that the only important differencebetween VI and VR schedules is the linearfeedback that the latter arrange. If this viewis correct, then response characteristics thatare unique to VR schedules must be due tothe linear feedback, rather than to any specialproperty of the response-based ratio contin-gency per se. It follows that VI-plus-linear-feedback schedules, which arrange linearfeedback in the absence of a ratio contingency,should produce response outputs that are in-distinguishable from those produced by VRschedules. For example, the high responserates that VR schedules are known to gener-ate (Baum, 1981; Zeiler, 1977) should also beproduced by VI schedules with linear feed-back loops. As another example, strained, two-valued (i.e., zero or very rapid) responding issometimes observed on VR schedules whenthe mean ratio requirement is large (e.g.,Ferster & Skinner, 1957). If strained re-sponding is observed on a VR schedule, thenit should also be observed on the comparableVI-plus-linear-feedback schedule.The first step in pursuing the VI-plus-lin-

ear-feedback account of the VR case is to testthe assumption that the two types of scheduleare equivalent. If the assumption holds, thenthe VI-plus-linear-feedback approach can bepursued further. If it does not hold, then anew approach to the VR case will be required.The purpose of the present experiment wasto test the assumption by comparing humansubjects' responding on VR schedules to theirresponding on VI schedules with linear feed-back loops. This experiment has five addi-tional benefits. First, it adds to the sparse para-metric literature on VR responding (Baum,1981). Second, it represents the first para-metric study of human responding on VRschedules. Third, it introduces a novel methodof arranging response-rate feedback. Fourth,

VR SCHEDULES AS VI SCHEDULES WITH LINEAR FEEDBACK

it compares two methods of initiating re-sponding in human subjects, namely, instruc-tions and shaping by successive approxima-tion. And fifth, it bears on traditionalinterresponse-time (IRT) theories (Zeiler,1977) of schedule performance. According toIRT theories, relatively long IRTs are likelyto be reinforced on schedules that arrange in-terval contingencies, and relatively short IRTsare likely to be reinforced on schedules thatarrange ratio contingencies. As a consequence,the former schedules should generate rela-tively low response rates and the latter shouldgenerate relatively high response rates (Fers-ter & Skinner, 1957; Morse, 1966; Peele,Casey, & Silberberg, 1984; Skinner, 1938;Zeiler, 1979). Clearly, interresponse-timetheory requires VI-plus-linear-feedbackschedules to generate lower response rates thancomparable VR schedules because of the dif-ferent contingencies that the two types ofschedule arrange.

METHOD

SubjectsFour humans aged 31 to 41 years (three

male, one female), who were recruited by ad-vertisement, served in the experiment. Allsubjects were either unemployed or employedpart-time while participating, and none werecollege students. One subject (H31) had pre-vious experience on VI schedules. The othersubjects were experimentally naive.

ApparatusSubjects worked in a small room facing a

54.6-cm (width) by 64.8-cm (height) consolethat tilted away from the subject at an angleof 23.20 from the vertical. A straight 24.5-cmlever extended horizontally from the center ofthe panel. A downward force on the lever ofapproximately 120 N produced an audible clickand counted as a response. A 6-digit LEDdisplay, an amber (reinforcement) light, a smallspeaker, a green (session) light, and a row offive blue (schedule) lights were attached to a17.6-cm (width) by 8.7-cm (height) metal boxthat was mounted on top of the console. Duringsessions the room was dimly illuminated by a7.5-W houselight and continuous white noisemasked extraneous sounds. A small viewer

mounted in the door permitted one-way ob-servation of the experimental room. The con-sole was controlled and data were recorded bya computer located in an adjoining room.

ProcedureAll subjects' lever pressing produced mon-

etary reinforcers in the two phases of theexperiment. In one phase the subjects wereexposed to VR 15, 30, 60, 120, and 240 sched-ules. Individual ratio requirements were cal-culated by Fleshler and Hoffman's (1962)method. In the other phase, the subjects wereexposed to five VI-plus-linear-feedbackschedules each of which had feedback prop-erties comparable to one of the five VR sched-ules. Feedback was arranged according toEquation 2 which, when inverted, is written

IRI = fn(IRT). (3)

The quantity, IRI, is the mean interreinforce-ment interval, or VI value, and IRT is themean interresponse time. Feedback compa-rable to that of a given VR schedule was ar-ranged by setting ni equal to the average ratiorequirement of the comparable VR. Equation3 and Fleshler and Hoffman's equation werethen used to schedule the reinforcers one at atime on-line. Fleshler and Hoffman's equa-tion, which is often used to calculate individ-ual interreinforcement intervals (IRIs) forsimple VI schedules, is written

IRIZ = IRI[1 + ln N + (N - i)ln(N -i)-(N-i +1)ln(N-i + 1)], (4)

where i = {1, 2, 3, . . ., N}. The index, i,enumerates the individual IRIs and N is thenumber of distinct IRIs in the schedule. Inthe present experiment N = 20. After each re-inforcement, i in Equation 4 was set equal toa randomly selected integer between 1 and 20.After each response (consider, e.g., the jth re-sponse), a computer calculated the IRT basedon all IRTs since the last reinforcement andmultiplied it by the average ratio requirement(ni) of the comparable VR, as Equation 3 re-quires. This yielded a new "fed back" IRI, ormean VI value, which was then used to cal-culate an individual IRI from Equation 4.This individual IRI was then scheduled. Ifthe next (i.e., the [j + 1]th) response occurredafter the lapse of the scheduled IRI, then it

317


was reinforced, a new integer value between1 and 20 was assigned at random to i, and thescheduling process began again. If, however,the next (again, the [j + 1]th) response oc-curred before the lapse of the scheduled IRI,then that IRI was canceled and a new IRTwas calculated, which yielded a new IRI fromEquation 3. The new IRI was used to calcu-late a new individual IRI from Equation 4,which was then scheduled. Timing continuedduring the very brief reinforcer presentations(<1 s, as described below) such that the firstIRT in a given IRI was the time between theresponse that produced reinforcement and thefirst response thereafter. Variable-intervalschedules with linear feedback loops arrangedin this way have the following properties: (1)the time between reinforcements depends onresponse rate since the last reinforcement only,just as on VR schedules, (2) faster responserates produce faster reinforcement rates andslower response rates produce slower rein-forcement rates (according to Equation 2), justas on simple VR schedules, (3) only intervalcontingencies are arranged, (4) the scheduledIRI changes with each response, and (5) thefirst response on a given schedule and the firstresponse after reinforcement can never bereinforced because initial IRTs are not cal-culated until these responses have occurred. Itis important to recognize the difference be-tween VR and VI-plus-linear-feedbackschedules. Although they have the same feed-back properties, the two types of schedule ar-range different contingencies, that is, they usedifferent criteria to set up reinforcement.Variable-interval-plus-linear-feedback sched-ules arrange interval contingencies, whichmeans that reinforcement is set up on the basisof elapsed time. Every response on this typeof schedule can be reinforced (except as notedabove), provided the subject waits until thescheduled interval lapses before responding.This is also possible on simple VI schedules.Variable-ratio schedules, on the other hand,arrange ratio contingencies, which means thatreinforcement is set up on the basis of re-sponse count. On this type of schedule it is notpossible for every response to be reinforced.

Except during the initial training sessionsfor Subjects H32 and H36, the following pro-cedure was used in both phases of the exper-iment. All five schedules were presented ineach session. The subject worked on one

schedule for 10 min, rested for 5 min, workedon the next schedule for 10 min, rested for 5min, and so on until all schedules were pre-sented. The sequence of schedules was quasi-random within sessions, with the restrictionthat each schedule appear exactly once persession. One of the blue schedule lights wascorrelated with each schedule. During workperiods the houselight, session light, and theappropriate schedule light were illuminated.During rest periods the subject was requiredto remain in the experimental room with onlythe session light illuminated. Subjects H3 1 andH36 were exposed to the five VR schedulesfirst; the other 2 subjects were exposed to thefive VI-plus-linear-feedback schedules first.

For Subjects H31 and H32, reinforcementconsisted of the addition of 3.75¢ to the digitaldisplay, a brief (<< 1 s) flash of the amber re-inforcement light, a brief (<1 s) offset of thehouselight, and a brief (<1 s) sounding of a1,000-Hz tone. For H36 and H37, reinforce-ment consisted of the addition of 4.25¢ to thedigital display and presentation of the threebrief stimuli just mentioned.

All subjects signed a contract before the startof the experiment in which they agreed to par-ticipate for 60 sessions or until they were re-leased, whichever occurred first. The contractalso stated that their earnings would dependon their performance, that they would be paidat the end of the experiment (although smalladvances were arranged for some subjects), andthat they would be subject to a penalty formissing more than two sessions (forfeiture ofone session's average pay per additional ses-sion missed) or for early withdrawal from theexperiment (forfeiture of one session's averagepay per session remaining in the contract).The penalties were approved by the EmoryUniversity Human Subjects Committee andmeet APA guidelines regarding informed con-sent. Subjects also were advised that they mightbe observed during sessions through the viewerin the door. To ensure that subjects did nothave timepieces in the experimental room, theywere told that metal jewelry might interferewith the operation of the equipment, and theywere asked to leave such items with the ex-perimenter.

For H31 and H37, lever pressing was ini-tiated by instructions. At the start of the firstsession these 2 subjects were read the follow-ing instructions:


This is a situation in which you can earnmoney. The green light will be on for the en-tire session, about 70 minutes. You earn moneysimply by pressing this lever. You can tellwhether or not you have pressed hard enoughby listening for the click from inside the ma-chine. Look at these blue lights. When thehouselight and a blue light are on, it meansthat you are able to earn money. At the begin-ning of the session, one of the lights will comeon and it will remain on for about 10 minutes.Throughout this time you will be able to earnmoney. Sometimes when you press the leverthis yellow light will flash and a tone willsound. Each time this happens, 3.75 cents [4.25cents for H37] will be added to the total. Thetotal amount you have earned will be shownon this counter. After 10 minutes, all lightswill go off for about five minutes. During thistime, you are to stay in this room and rest. Donot read during this period or leave the room.After the rest period, another blue light willcome on and you can earn more money. Thenthere will be another rest period and so onuntil all five lights have been presented. At theend of the session, I will take a reading fromthe counter and give you a receipt for the moneyyou have earned. You will be paid in a lumpsum at the end of the experiment.

Questions at the first and all subsequent ses-sions were answered by rereading relevantportions of the instructions.

For H32 and H36, lever pressing wasshaped by successive approximation. SubjectH32 required two training sessions; H36 re-quired four. At the start of the first trainingsession both subjects were read the followinginstructions:

This is a situation in which you can earnmoney. The green light will be on for the en-tire session, about 70 minutes. The total amountof money you have earned will be shown onthis counter. At the end of the session I willtake a reading from the counter and give youa receipt for the money you have earned. Youwill be paid in a lump sum at the end of theexperiment. Please do not ask for additionalinformation about what you are to do.

The subjects' behavior was then observedthrough the viewer in the door and leverpressing was shaped by hand. Both subjectspressed the lever within 10 min of the start ofthe session. For each subject, each of the first25 lever presses was reinforced and then fiveVR schedules were arranged, each of whichremained in effect for 10 reinforcers. The VR

schedules had mean ratio requirements (ni) of5, 10, 15, 25, and 50 responses, and they werepresented in order of increasing ni. The sched-ules were not accompanied by discriminativestimuli, and transitions between them wereunsignaled. Subject H32's second training ses-sion consisted of the procedure and schedulesof Phase 1, except that the schedules werepresented in order of increasing ni. After thesetwo training sessions, Phase 1 began for H32.In H36's second and third training sessions,various VR schedules were presented in orderof increasing ni. The schedules gradually in-creased to larger fis and finally culminated ina VR 150. The VR schedules were not accom-panied by discriminative stimuli, and transi-tions between them were unsignaled. Thefourth training session for H36 consisted ofthe procedure and schedules of Phase 1, ex-cept that the schedules were presented in orderof increasing ni. After these four trainingsessions, Phase 1 began for H36.

All subjects participated in two sessions perday. Sessions in each phase continued until asubject's response rates across sessions showedno trends at any schedule value. There wasno break between phases and there were nostimulus changes or additional instructions atthe start of Phase 2. Some of the proceduraldetails of the experiment, including the totalnumber of sessions in each phase, are sum-marized in Table 1.

RESULTSAll subjects satisfied the terms of their con-

tracts such that no penalties were incurred.Each subject's reinforcement and response

rates were averaged over the last five sessionson the VR and the VI-plus-linear-feedbackschedules. The average rates are listed in Ta-ble 2, along with the standard errors of theresponse-rate means, and the obtained num-ber of responses per reinforcement for the lastfive sessions on each schedule. On most of theVR and VI-plus-linear-feedback schedules,the obtained number of responses per rein-forcement agreed fairly closely with the sched-uled ni. The disagreements for H36 at fi = 240on both the VR and the VI-plus-linear-feed-back schedules, and for H37 at ni = 240on the VI-plus-linear-feedback schedule, weredue to sampling error. Only a few response-

319


Table 1Reinforcer magnitude, method of response initiation, typeof schedule (VI+ = VI-plus-linear-feedback schedule),and total number of sessions for each subject in the twophases of the experiment. For each subject, reinforcermagnitude was the same in both phases.

Num-ber of

Rein- ses-forcer sionsmagni- (Phasetude Schedule 1,

(¢/rein- Responding type (Phase PhaseSubject forcer) initiated by 1, Phase 2) 2)

H31 3.75 instructions VR, VI+ 12, 12H32 3.75 shaping VI+, VR 14, 10H36 4.25 shaping VR, VI+ 20, 9H37 4.25 instructions VI+, VR 18, 8

per-reinforcement units accumulated duringthe last five sessions on these schedules (cf.H37's closer agreement at ni = 240 on the VRschedule).The average response rates listed in Table

2 are plotted against ni in Figure 1. Filledcircles represent data from the VR schedules,and unfilled circles represent data from theVI-plus-linear-feedback schedules. Error bars(± 1 standard error) also appear, unless theywere equal to or smaller than the diameter ofthe data point.The data in Table 2 and Figure 1 show

that response rate decreased with ni for allsubjects on both the VR and the VI-plus-lin-ear-feedback schedules. The decline in re-sponse rate was much greater for H36 andH37 than for the other 2 subjects. The ses-sion-to-session variability in response rate at

Table 2

Obtained number of responses per reinforcement (rsp/rft), mean obtained reinforcement rate(rft/hr = reinforcements per hour), mean response rate (rsp/min = responses per minute),and the standard error (SE) of the mean response rate for each of the 4 subjects in the VRand the VI-plus-linear-feedback phases of the experiment. The quantity, n, is the reciprocalof the slope of the linear feedback function (Equation 2). For the VR schedules, in also rep-resents the average ratio requirement. All quantities in the table were calculated from un-rounded data.

VR VI-plus-linear-feedbackrsp/min rsp/min

rsp/rft rft/hr ±SE rsp/rft rft/hr ±SE

Subject H3115 14.8 493.2 121.9 ± 1.1 15.8 468.0 123.5 ± 0.630 28.2 255.6 120.1 ± 1.4 30.2 244.8 123.1 ± 0.860 52.5 135.6 118.6 ± 1.8 57.8 126.0 121.3 ± 1.0120 122.1 56.4 114.8 ± 1.9 126.6 56.4 119.0 ± 1.3240 196.4 34.8 113.9 ± 2.3 236.6 30.0 118.3 ± 1.4

Subject H3215 15.0 495.6 124.0 ± 0.9 15.6 480.0 124.5 ± 2.130 30.8 230.4 118.2 ± 0.9 31.0 238.8 123.3 ± 0.860 62.8 111.6 116.8 ± 1.7 64.0 111.6 119.0 ± 0.7120 135.5 50.4 113.8 ± 1.6 123.2 57.6 118.3 ± 0.7240 277.0 24.0 110.8 ± 1.6 233.0 30.0 116.5 ± 0.8

Subject H3615 15.1 496.8 124.7 ± 4.5 15.7 530.4 138.4 ± 1.130 29.9 223.2 111.2 ± 8.4 30.0 267.6 134.0 ± 5.060 58.6 116.4 113.6 ± 5.5 58.0 122.4 118.3 ± 12.3120 120.6 54.0 108.6 ± 9.5 108.8 56.4 102.3 ± 13.8240 679.0 2.4 27.2 ± 24.7 170.7 14.4 41.0 ± 17.9

Subject H3715 15.0 584.4 146.0 ± 2.4 15.5 554.4 143.1 ± 4.330 30.4 289.2 146.4 ± 5.9 30.5 278.4 141.6 ± 5.060 58.0 145.2 140.4 ± 4.9 57.8 124.8 120.1 ± 15.5120 113.3 52.8 99.7 ± 18.1 121.6 60.0 121.6 ± 7.0240 211.5 7.2 25.4 ± 25.4 415.3 7.2 49.8 ± 26.3

320


150rH31

125

H32

1001

75

501

25

100 200 300

IV} VR1sEo VI +

100 200 300

H37

100 200 300 100 200 300

i -~~~~~~~~~~~~iI ~~~~~~~~n IFig. 1. Response rate on VR (filled circles) and VI-plus-linear-feedback (VI+; unfilled circles) schedules as a

function of the reciprocal of the slope of the linear feedback function (nT). For the VR schedules, ni is also the mean

ratio requirement. Error bars, which were omitted when equal to or less than the diameter of the data point, represent± 1 standard error.

most nis (indicated by the error bars) was alsogreater for these 2 subjects.As shown in Figure 1, the VR schedules

and the VI-plus-linear-feedback schedules fora given subject produced similar rates of re-sponding at a given value of ni. For H31 andH32, response rates on the VR and the VI-plus-linear-feedback schedules at a given niwere nearly identical. For H36 and H37, dif-ferences between rates were usually accom-

panied by large, overlapping error bars. Thus,as is clear from the figure, the two types ofschedule produced similar response rate versusni functions for each subject.

Similarities in behavior on the VR and theVI-plus-linear-feedback schedules also ex-

tended to the details of responding. Represen-tative cumulative records for each subject are

reproduced in Figures 2 through 5. Each ofthe four records in a figure shows a subject'sfinal 50 min of responding on a schedule.

Figures 2 and 3 show H31's and H32'sresponding on the two types of schedule at ni =15 and fi = 240. Their performances weresimilar at the other fis. Both subjects re-

sponded on all schedules at high, steady ratesthat decreased gradually as ni increased. Thepattern of responding on all schedules wasnearly invariant from session to session. Sub-ject H31 did not pause during the final 50min on any schedule. Subject H32's final 50min on both the VR and the VI-plus-linear-feedback schedules showed infrequent, briefpauses that ended in abrupt transitions to theresponse rate that prevailed before the pause.

These pauses can be seen in all four recordsin Figure 3. At each ni, H31's and H32's cu-

mulative records on the VR and the VI-plus-linear-feedback schedules were indistinguish-able, as the examples in Figures 2 and 3 show.

Reproduced in Figures 4 and 5 are H36'sand H37's cumulative records of responding

150

125

100

75'-

50',

25

Ta)E

0)cc

as

321


H31VR

VI+

Fig. 2. Cumulative records of H31's final 50 min of responding on VR and VI-plus-linear-feedback schedules(VI+) at ni = 15 (top two records) and n = 240 (bottom two records). The pen reset every 10 min, and within 10-min periods it reset every 400 responses. Downward deflections of the pen indicate reinforcer deliveries.

at i = 15 and fi = 240 on the VR and the VI-plus-linear-feedback schedules. At ni = 15,H36 and H37 responded on both types ofschedule at high steady rates. The pattern ofresponding was fairly constant from session tosession. Subject H36 occasionally showedpauses and rate fluctuations. Very infre-quently, H37 showed brief pauses that endedin abrupt returns to the prevailing responserate. The performances of these 2 subjects atni = 30, 60, and 120 were similar, except thatthe frequency and duration of the pausing in-creased somewhat at the larger fns. At fi =

240, H36 and H37 showed strained respond-ing on both types of schedule. Response rateswere either zero or high and steady, with oc-casional periods of fluctuating, intermediaterates. At each fi, H36's and H37's cumulativerecords on the VR and the VI-plus-linear-feedback schedules were very similar, as theexamples in Figures 4 and 5 show.

DISCUSSIONFor each subject, the VR and the VI-plus-

linear-feedback schedules produced similar

322

11 VI+1=

i4


H32VR

t-4

VI+

Fig. 3. Cumulative records of H32's final 50 min of responding on VR and VI-plus-linear-feedback schedules(VI+) at a = 15 (top two records) and ni = 240 (bottom two records). The pen reset every 10 min, and within 10-min periods it reset every 400 responses. Downward deflections of the pen indicate reinforcer deliveries.

average response rates at each value of ni, sim-ilar response rate versus fi functions, and sim-ilar patterns of responding. The behavior ofH36 and H37 differed from that of the other2 subjects in three ways: They showed agreater decline in response rate with fi, moresession-to-session variability in response rate,and strained responding at fi = 240 on boththe VR and the VI-plus-linear-feedbackschedules. These between-subject differencescannot be accounted for by the order of pre-sentation of the two types of schedule, or bythe method of initiating responding. The only

procedural detail that could have been re-sponsible is the magnitude of the reinforcer,which was slightly larger for H36 and H37than for the other 2 subjects. It is also possiblethat these differences simply reflect between-subject variability. Ferster and Skinner (1957)and Brandauer (1958) both reported consid-erable between-subject variability in the be-havior of pigeons on VR schedules. For ex-ample, in both experiments, sustainedresponding was observed in some birds butnot in others on VR schedules with high meanratio requirements (ni = 360, 400, and 600).

323

11 VI+

111=


H36VR

I IVR

VR [s

VI+

Fig. 4. Cumulative records of H36's final 50 min of responding on VR and VI-plus-linear-feedback schedules(VI+) at a = 15 (top two records) and fn = 240 (bottom two records). The pen reset every 10 min, and within 10-min periods it reset every 400 responses. Downward deflections of the pen indicate reinforcer deliveries.

An additional finding of the present experi-ment is that the two methods used to initiateresponding, namely, instructions and shapingby successive approximation, did not have dif-ferential effects on behavior (cf. Matthews,Shimoff, Catania, & Sagvolden, 1977; Shi-moff, Catania, & Matthews, 1981).

Certain burst-pause patterns of respondingcould have been differentially reinforced onthe VI-plus-linear-feedback schedules in thisexperiment. For example, a pattern consistingof a few rapid responses after reinforcement,

followed by a pause, would have been differ-entially reinforced. No burst-pause patternsof responding were apparent in any subject'scumulative records, although a fine-grainedanalysis could not be carried out because in-dividual response times were not recorded.The response rates on the VR and the VI-

plus-linear-feedback schedules in this exper-iment can be compared with response rates ofhuman subjects on ordinary VI schedules.Four series of VI schedules from McDowelland Wood's (1984) study of monetarily rein-


H37

1i/VR

VW-

aI5 uwissi

Fig. 5. Cumulative records of H37's final 50 min of responding on VR and VI-plus-linear-feedback schedules(VI+) at n = 15 (top two records) and n = 240 (bottom two records). The pen reset every 10 min, and within 10-min periods it reset every 400 responses. Downward deflections of the pen indicate reinforcer deliveries.

forced lever pressing in humans entailed forcerequirements and reinforcer magnitudes com-parable to those used in the present experi-ment. These series were from H18, H19, H20,and H23, where reinforcer magnitude was 4¢.The median asymptotic response rate (i.e., themedian estimate across subjects of Herrn-stein's [1970] k) on these series was 69.4 re-

sponses/min. The median of the eight highestresponse rates from the present experiment(one per type of schedule per subject) was124.6 responses/min. The latter median,

which exceeds 2 responses/s, was about 80%greater than the median asymptotic responserate on the VI schedules. This result is con-sistent with earlier findings of higher responserates on VR than on VI schedules in humansubjects (Matthews et al., 1977).The behavior of the human subjects in the

present experiment was similar to that of avianand rodent species on VR schedules. For ex-

ample, response rates of pigeons (Brandauer,1958 [excluding fixed-ratio 1 schedules];Green, Kagel, & Battalio, 1982) and rats

AR

I"

11Or

325

VI+

v

111=

t

i r


(Mazur, 1983) on VR schedules have beenfound to decrease from the low to the highend of the ni range. Similarly, high, steadyrates of responding on VR schedules have beenreported in both pigeons (Brandauer, 1958;Ferster & Skinner, 1957) and rats (Mazur,1983). Higher response rates on VR than onVI schedules have also been reported in pi-geons (Catania, Matthews, Silverman, & Yo-halem, 1977; Ferster & Skinner, 1957; Zuriff,1970) and rats (Mazur, 1983). In addition,many of the response details observed in thebehavior of the human subjects in the presentexperiment have also been reported in the be-havior of other species. For example, Fersterand Skinner observed the following details inpigeons' key pecking on VR schedules: (1)brief pauses that ended abruptly in returns tothe prevailing rate of responding, (2) occa-sional periods of fluctuating, intermediate re-sponse rates in otherwise steady, high-raterecords, (3) some between-session variabilityin overall response rate, and (4) behavior athigh mean ratio values (ni = 360) that con-sisted of zero or very rapid response rates withoccasional periods of fluctuating, intermediaterates (i.e., strained responding). Brandauer(1958) reported similar details of respondingin his pigeons' key pecking, including an in-creased frequency of pausing at larger valuesof ni. Mazur (1983) reported brief, abruptlyending pauses in his rats' lever pressing, andan increased frequency of pausing at largervalues of ni. In addition, as noted above, bothFerster and Skinner, and Brandauer reportedconsiderable between-subject variability in thebehavior of their pigeons. These similaritiesin behavior across species are especially note-worthy in view of the different experimentalprocedures that were used with the differentsubject species. For example, consummatoryreinforcers were used in the experiments withpigeons and rats, and it is unlikely that thesesubjects had significant uncontrolled extraex-perimental histories of reinforcement andpunishment. By contrast, nonconsummatoryreinforcers were used in the present experi-ment, and the subjects' uncontrolled extraex-perimental histories were extensive.

Interresponse-time theories of scheduleperformance (Zeiler, 1977) are not supportedby the results of the present experiment. Ac-cording to one version of IRT theory (Ferster& Skinner, 1957; Morse, 1966; Skinner,1938), behavior interacts with interval and

ratio contingencies to produce the characteristicVI and VR performances. More specifically,because responses tend to be emitted in clus-ters, reinforcement on ratio schedules is likelyto occur after short IRTs whereas reinforce-ment on interval schedules is likely to occurafter long IRTs. This differential reinforce-ment is said to produce the relatively highresponse rates on VR schedules and the rel-atively low response rates on VI schedules. Asecond version of IRT theory (e.g., Peele etal., 1984) ignores properties of responding andsimply asserts that VI schedules differentiallyreinforce long IRTs because the probability ofreinforcement on a VI schedule increases withIRT duration (up to the limiting value of uni-ty). By a similar argument, variable-ratioschedules are said not to differentially rein-force any class of IRTs because the probabil-ity of reinforcement on a VR schedule is un-related to IRT duration. Clearly, this versionof IRT theory also requires response rates onVI schedules to be lower than those on VRschedules. A third version of IRT theory (e.g.,Zeiler, 1979) entails two opposing processes.One is the tendency of reinforcement to"strengthen" behavior, that is, to increase itsrate of occurrence. This tendency is presentfor both VI and VR responding. On VI sched-ules, however, the strengthening effect of re-inforcement is partially counteracted by thedifferential reinforcement of long IRTs. OnVR schedules, the strengthening effect oper-ates unopposed. Again, the result is lower re-sponse rates on VI than on VR schedules. Allthree versions of IRT theory require a rela-tively low rate of responding to develop when-ever a time-based, interval contingency is ineffect, and a relatively high rate of respondingto develop whenever a response-based, ratiocontingency is in effect. The critical factor ingenerating the response-rate difference is thetype of contingency (i.e., interval vs. ratio).According to IRT theories, a relatively lowrate of responding should develop on a VI-plus-linear-feedback schedule (as compared toa VR schedule) because it arranges intervalcontingencies, which ensure higher probabil-ities of reinforcement for longer IRTs. Thepresent results are contrary to this predictionand, consequently, confute all three versionsof IRT theory. Zeiler (1979) has noted thatdata from differential-reinforcement-of-low-rate schedules also fail to support the theory.

Interresponse-time theory has also been

326


used to account for the empirical observationthat behavior can be maintained better by alean VI schedule than by a VR schedule witha large mean ratio requirement, all else beingequal (Zeiler, 1977). Interval, but not ratio,contingencies are said to be capable of "re-generating" responding (Zeiler, 1977, 1979)because they arrange higher probabilities ofreinforcement for longer IRTs. As respondingweakens on a lean VI schedule (i.e., as theIRT increases), reinforcement becomes moreprobable. The next response is likely to bereinforced, and so responding may be main-tained, at least at a low rate. But as respondingweakens (i.e., as the IRT increases) on a VRschedule with a large mean ratio requirement,reinforcement does not become more probable.Hence, responding may become strained or itmay cease altogether. The results of the pres-ent experiment are not in agreement with thisaccount. Subjects H36 and H37 showedstrained responding at ni = 240 on both theVRand VI-plus-linear-feedback schedules (Fig-ures 4 and 5), even though the interval con-tingencies arranged by the latter schedule en-sured higher probabilities of reinforcement forlonger IRTs. According to IRT theory, theseinterval contingencies should have "regener-ated" responding, that is, produced a roughlysteady, if low, response rate.The present results also appear to be in-

consistent with findings recently reported byPeele et al. (1984, Experiment 3), who stud-ied the responding of pigeons on a two-com-ponent multiple schedule. One component ofthe schedule was a tandem VI VR and theother was a tandem VR VI. The ratio sched-ule in each component was a VR 100. TheIRI of the VI schedule in each component wasyoked to the duration of the VR in the othercomponent such that the durations of the twomultiple-schedule components were equal.According to Peele et al., the feedback func-tions in the two components were identical.Of course the components differed in that oneended in a VR and the other ended in a VI.Given identical feedback loops, a feedback ac-count predicts equal rates of responding in thetwo components, whereas IRT theory predictsa higher response rate in the tandem VI VRcomponent. Peele et al. found a higher re-sponse rate in the tandem VI VR component,and they concluded that IRT theory was sup-ported. But this conclusion is questionable be-cause of a confound introduced by the yoking

procedure. The between-component yokingensured that reinforcement in one componentdepended not only on responding in that com-ponent, but also on responding in the othercomponent. For example, rapid responding onthe VR in one component reduced the dura-tion of the VI in the other. Hence, reinforce-ment in each component was not a functionof responding in that component only, but wasa joint function of responding in both com-ponents. Because the within-component feed-back was confounded with the between-com-ponent feedback, Peele et al.'s results aredifficult to interpret. Even if the between-component feedback is ignored, however, thereis a second problem, namely, that the feedbackfunctions in the two tandem components ac-tually may have been different. The data intheir Table 5 show that similar reinforcementrates were produced by different responserates. If the two feedback functions were iden-tical, this could happen only if responding fellin a region where the slope of the feedbackfunction was nearly zero. But if the two feed-back functions were different, similar rein-forcement rates would be produced, as a rule,by different response rates. Because the exactforms and parameter values of the feedbackfunctions cannot be determined from Peele etal.'s data, the possibility that the two feedbackfunctions were different cannot be ruled out.Contrary to IRT theories, the results of the

present experiment suggest that the principaldifference between VI and VR schedules isthe linear feedback that the latter arrange. Inother words, the assumption entailed by theVI-plus-linear-feedback account of the VRcase is supported. This indicates that the VI-plus-linear-feedback approach is worth fur-ther study. The next step, which is currentlyin progress, is to work out the formal detailsof the theory. This entails finding the simul-taneous solution sets of Equations 1 and 2, anddetermining their properties. The theory mustbe able to describe the form of the responserate versus in function, and to account for var-ious known properties ofVR responding, suchas ratio strain and high response rates.As a final point, we note that the method

of arranging feedback used here has addi-tional applications. For example, just as lin-ear feedback can be added to a VI schedule,it can be removed from a VR schedule. Thisis accomplished by solving Equation 2 for fiand setting 1/RIN equal to the mean IRI of a

327

328 J. J McDOWELL and JOHN T. WIXTED

comparison VI. The resulting equation is thenused to calculate ni and the current ratio re-quirement after each response. More gener-ally, it is worth noting that the present methodof arranging feedback is not restricted to lin-ear relationships. Indeed, feedback functionsof any form whatsoever can be arranged sim-ply by writing the desired equation in placeof Equation 2. In addition, the operation ofthe feedback loop itself can be studied by vary-ing the width of the feedback window, whichis the amount of time or number of responsesused to calculate the response rate upon whichthe loop operates. The present method canalso be used in the absence of molecular re-sponse-reinforcement contingencies. For ex-ample, a feedback loop of any desired formcan be added to a variable-time schedule toproduce an environment that entails only themolar rate contingency arranged by the feed-back function itself (cf. Baum, 1973).

REFERENCESAseltine, J. A. (1958). Transform method in linear system

analysis. New York: McGraw-Hill.Baum, W. M. (1973). The correlation-based law of

effect. Journal of the Experimental Analysis of Behavior,20, 137-153.

Baum, W. M. (1981). Optimization and the matchinglaw as accounts of instrumental behavior. Journal ofthe Experimental Analysis of Behavior, 36, 387-403.

Brandauer, C. M. (1958). The effects of uniform prob-abilities of reinforcement on the response rate of thepigeon. Dissertation Abstracts, 19, 3377. (UniversityMicrofilms No. 59-1478)

Catania, A. C. (1973). Self-inhibiting effects of rein-forcement. Journal of the Experimental Analysis of Be-havior, 19, 517-526.

Catania, A. C., Matthews, T. J., Silverman, P. J., &Yohalem, R. (1977). Yoked variable-ratio and vari-able-interval responding in pigeons. Journal of the Ex-perimental Analysis of Behavior, 28, 155-161.

Ferster, C. B., & Skinner, B. F. (1957). Schedules ofreinforcement. New York: Appleton-Century-Crofts.

Fleshler, M., & Hoffman, H. S. (1962). A progressionfor generating variable-interval schedules. Journal ofthe Experimental Analysis of Behavior, 5, 529-530.

Green, L., Kagel, J. H., & Battalio, R. C. (1982). Ratioschedules of reinforcement and their relation to eco-nomic theories of labor supply. In M. L. Commons,R. J. Herrnstein, & H. Rachlin (Eds.), Quantitativeanalyses of behavior: Vol. 2. Matching and maximizingaccounts (pp. 395-429). Cambridge, MA: Ballinger.

Herrnstein, R. J. (1970). On the law of effect. Journalof the Experimental Analysis of Behavior, 13, 243-266.

Killeen, P. R. (1981). Averaging theory. In C. M.Bradshaw, E. Szabadi, & C. F. Lowe (Eds.), Quan-tification of steady-state operant behaviour (pp. 21-34).

Amsterdam: Elsevier /North-Holland BiomedicalPress.

Killeen, P. R. (1982). Incentive theory. In D. J. Bern-stein (Ed.), Nebraska symposium on motivation 1981:Vol. 29. Response structure and organization (pp. 169-216). Lincoln: University of Nebraska Press.

Matthews, B. A., Shimoff, E., Catania, A. C., & Sagvol-den, T. (1977). Uninstructed human responding:Sensitivity to ratio and interval contingencies. Journalof the Experimental Analysis of Behavior, 27, 453-467.

Mazur, J. E. (1983). Steady-state performance on fixed-,mixed-, and random-ratio schedules. Journal of the Ex-perimental Analysis of Behavior, 39, 293-307.

McDowell, J. J. (1979). Some contributions to themathematical description of behavior. Dissertation Ab-stracts International, 40, 1931B-1932B. (UniversityMicrofilms No. 79-22,058)

McDowell, J. J. (1980). An analytic comparison ofHerrnstein's equations and a multivariate rate equa-tion. Journal of the Experimental Analysis of Behavior,33, 397-408.

McDowell, J. J. (1986). A mathematical theory of rein-forcer value and its application to reinforcement delayin simple schedules. In M. L. Commons, J. E. Mazur,J. A. Nevin, & H. Rachlin (Eds.), Quantitative anal-yses of behavior: Vol. 5. Effects ofdelay and of interveningevents on reinforcement value (pp. 77-106). Hillsdale,NJ: Erlbaum.

McDowell, J. J, Bass, R., & Kessel, R. (1983). Vari-able-interval rate equations and reinforcement and re-sponse distributions. Psychological Review, 90, 364-375.

McDowell, J. J, & Kessel, R. (1979). A multivariaterate equation for variable-interval performance. Jour-nal of the Experimental Analysis of Behavior, 31, 267-283.

McDowell, J. J, & Wood, H. M. (1984). Confirmationof linear system theory prediction: Changes in Herrn-stein's k as a function of changes in reinforcer mag-nitude. Journal of the Experimental Analysis ofBehavior,41, 183-192.

McDowell, J. J, & Wood, H. M. (1985). Confirmationof linear system theory prediction: Rate of change ofHerrnstein's k as a function of response-force require-ment. Journal of the Experimental Analysis of Behavior,43, 61-73.

Morse, W. H. (1966). Intermittent reinforcement. InW. K. Honig (Ed.), Operant behavior: Areas of researchand application (pp. 52-108). New York: Appleton-Century-Crofts.

Peele, D. B., Casey, J., & Silberberg, A. (1984). Pri-macy of interresponse-time reinforcement in account-ing for rate differences under variable-ratio and vari-able-interval schedules. Journal of ExperimentalPsychology: Animal Behavior Processes, 10, 149-167.

Rachlin, H. C. (1978). A molar theory of reinforcementschedules. Journal of the Experimental Analysis of Be-havior, 30, 345-360.

Shimoff, E., Catania, A. C., & Matthews, B. A. (1981).Uninstructed human responding: Sensitivity of low-rate performance to schedule contingencies. Journal ofthe Experimental Analysis of Behavior, 36, 207-220.

Skinner, B. F. (1938). The behavior of organisms. NewYork: Appleton-Century.

Staddon, J. E. R. (1977). On Herrnstein's equation andrelated forms. Journal of the Experimental Analysis ofBehavior, 28, 163-170.

VR SCHEDULES AS VI SCHEDULES WITH LINEAR FEEDBACK 329

Staddon, J. E. R. (1979). Operant behavior as adap-tation to constraint. Journal of Experimental Psychol-ogy: General, 108, 48-67.

Staddon, J. E. R., & Motheral, S. (1978). On matchingand maximizing in operant choice experiments. Psy-chological Review, 85, 436-444.

Zeiler, M. (1977). Schedules of reinforcement: The con-trolling variables. In W. K. Honig & J. E. R. Staddon(Eds.), Handbook of operant behavior (pp. 201-232).Englewood Cliffs, NJ: Prentice-Hall.

Zeiler, M. D. (1979). Output dynamics. In M. D. Zeiler

& P. Harzem (Eds.), Advances in analysis of behaviour:Vol. 1. Reinforcement and the organization of behaviour(pp. 79-115). Chichester, England: Wiley.

Zuriff, G. E. (1970). A comparison of variable-ratioand variable-interval schedules of reinforcement. Jour-nal of the Experimental Analysis of Behavior, 13, 369-374.

Received September 26, 1985Final acceptance August 17, 1986

469 number (november) variable-ratio mcdowell and

Documents