intertrial responses in defensive instrumental learning

Intertrial responses in defensive instrumental learning

-

Kazimierz Zielinski

Department of Neurophysiology, Nencki Institute of Experimental Biology, 3 ~asteur St., 02-093 Warsaw, Poland

Abstract. Data by the author, as well as from other laboratories, show residual fear as the main source of intertrial responses (ITR) during defensive instrumental training in rats, cats and dogs. Between-subject differences in ITR rate arise in the course of training. The differences depend on training conditions and on subject's emotionality. Under the same conditions higher ITR rates correspond to more rapid avoidance learning. The increase of ITR rate is regularly observed in the following circumstances: at early stages of training, after transfer to a less salient discriminative stimulus, and after a change of the task requirements. The decrease of ITR rate corresponds to the plateau level of avoidance or escape performance. Both a lower ITR rate and reduced flexibility of behavior are found after lesions of the amygdala nuclei.

Key words: intertrial response, behavioral activation, fear, frustration, avoidance, escape, amygdala

216 K. Zielidski

INTRODUCTION

Most of our knowledge about what is learned and how it is learned has been obtained in experiments employing alternation of conditioning trials with longer periods during which a subject is only under the influence of constant elements of an experimental context. The situation, in which each trial is in- itiated by presentation of a discriminative stimulus, seems to be especially suitable for investigating the rules by which associations between a conditioned (CS) and an unconditioned stimulus (US) or between a response and a reinforcement are estab- lished. However, responses indicating the formation of associations and other related changes of behavior occur not only during sporadic CS pres- entations, but also in the absence of CSi. In Pavlo- vian conditioning experiments, dogs salivate not only to a metronome, a light or other CSi, but also between trials, although at a lower rate. One index of stimulus control of behavior is the higher probability of a given response during the action of a specific stimulus than in other situations. For this reason, intertrial responses were considered in Pav- lovian laboratories as a result of insufficient learning (Pavlov 1927). When the CS is not fully discriminated from the background, any unex- pected stimulus can elicit the learned response.

Intertrial responding is one of several behavioral phenomena about which the origin and role in learning processes are not clear. In some respects intertrial responding is similar to the problem of animal superstition, which seems to be pointlessly wasteful activity. However, intimate relations between superstitious behavior and learning processes have been discussed in theoretical (Herrnstein 1966) and experimental (Devenport and Halloway 1980) papers, whereas the problem of intertrial responding has been largely neglected in contemporary theories of learning.

In this review only data on instrumental defensive conditioning will be considered. The term "intertrial response" will be used for responses similar in form to the conditioned response (CR) performed

in the experimental situation in absence of the sporadic CS. As a rule, the changes in the frequency and also in the form of ITRs during instrumental training have been found to be more substantial than those observed in studies of classical conditioning.

FEAR AND ITRs RATE

The distribution of responses emitted during an intertrial interval and the pattern of changes of the ITR frequency in the course of training provide ar- guments against the notion that poor discrimination of a cue from an experimental context or accidental stimuli produce a learned response in the absence of a CS. Our data on cats trained to avoid a scrambled shock applied through a grid floor (Zieliriski and Plewako 1980) are an example. These experiments were conducted in a cage enabling locomotion of the subject. A bar was easily reached during an en- tire session. A trial started with the onset of a 70 dB wide-band noise CS and had to be terminated by a bar press response. A 5 s CS-US interval was used for all cats. Ten trials were given daily with intertrial intervals of 40-, 60- and 80-s in a semi-random order. Training continued until a cat reached a criterion of 90 avoidance responses in 100 consecutive trials. The course of training was analyzed using the "vincentizing" method (Vincent 19 12, Hilgard 1938), whereby the total number of training and criterion trials for a given subject was divided into five blocks of equal length. Then for each block of trials, group means were calculated for several measures of behavior. In the present paper only the ITR rate (frequencies of bar presses per s of the intertrial intervals) will be considered.

Two groups of cats differed in the consequences of bar press responses emitted during the 5 s CS-US interval. For one group, a bar press executed within 5 s after CS onset immediately terminated the CS and prevented the shock. For the other group, the bar press executed within 5 s of the CS action was equally effective in preventing the shock US, however, the CS was terminated only after the fixed elapse of 5 s from onset. If an avoidance response did not occur within 5 s of the CS, the grid floor was

ITR and avoidance 217

s a f te r t r i a l

Fig. 1. Intertrial response rate during consecutive 5 s periods of the intertrial intervals (black histograms) and extra response rate (hatched bars) for cats permitted to terminate CS with an avoidance response (upper) and cats trained with the CS of 5 s minimal duration (bottom) in consecutive (I-V) Vincentized fifths of training.

activated and both stimuli lasted until the cat performed the bar press. Responses executed during the intertrial intervals had no consequence.

Cats trained under conditions in which avoidance responses did not terminate the CS, performed additional bar presses during the prolonged CS action on about 2.5 % of the trials. The frequency of these extra responses was compared with a similar measure for intertrial responses and are presented for each group and consecutive vincentized fifth of training in Fig. 1.

In each of the five blocks of training, most of the ITRs were executed soon after termination of the trial, then the frequency of responses decreased, and by 15-20 s after the CS termination the probability of ITRs was very low. Such a distribution of ITRs may be a consequence of residual fear which did not dissipate immediately after performance of the in-

strumental response and trial termination. The data on extra responses seem to support this interpretation. With advanced training, the frequencies of extra responses began to fit the exponentially decreasing curves of the ITR frequency in the early portion of intertrial intervals. The logarithmic values of the frequencies of the extra responses and of the JTRs in few successive 5 s periods after the CS termination showed a linear relationship with time.

According to such an understanding of ITR origin, the rise of the fear drive ought to result in the increase of ITR frequency. Comparison of the ITR frequencies after non-shock and shock trials for each group and Vincentized fifth of training supported this inference. More ITRs were performed after escape than after avoidance responses and this difference increased in the course of training (Zielifiski and Plewako 1980).

The monotonicaly decreasing ITR rate during intertrial intervals was found also when cats were trained to escape the shock from a grid-floor (Zielinski et al. 1983). A comparison of data from different experiments for cats not subjected to any brain lesions and trained in the same apparatus, showed that the ITR rate was higher during escape (Werka 1980, Zielinski et al. 1983) than during avoidance (Zielinski and Plewako 1980) training.

Observations of other investigators also indicate that fluctuations of classically conditioned fear are responsible for the occurrence of intertrial responses in the avoidance situation (Kamin 1954, Brush 1962, Bolles et al. 1966). The changes of the ITR distribution within intertrial intervals of fixed duration after prolonged training may serve as another supportive demonstration. At the beginning of training in a shuttle box with a 10 s CS-US interval and a 120 s intertrial interval, dogs performed jump- ing ITRs soon after the trial termination, whereas in later training, the distribution of ITRs shifted toward the end of the intertrial interval (Black and Carlton 1959). This change indicated a shift of fear as the time approached for the next presentation of the danger signal.

DYNAMICS OF ITRs IN TRAINING

The changes of ITR frequency found during avoidance training of cats were also in agreement with the two-factor theory of avoidance. As seen in Fig. 1, the ITR rate increased at early stages of avoidance training and then slowly decreased. A comparison with other measures analyzed in this experiment indicated that maximum ITR rate corre- sponded to the 50 % avoidance performance (Zielinski and Plewako 1980). The low level of ITRs at the first Vincentized block cannot be related to insufficient motor learning, since the bar press response was performed on each trial. The frequency of extra responses also increased during early training, but did not decrease after. The gradual rise of the extra response frequency may be a consequence of a strengthening of the fear drive associated with the CS. The initial increase of ITR frequency in the

course of avoidance training was enhanced in cats trained in the more difficult paradigm in which avoidance response did not terminate the CS immediately.

A marked increase of ITR frequency at early stages of training was obtained in another study conducted on cats that were required to terminate unsignaled shock by a bar press response (Werka 1980). The shortening of escape response latencies observed after a few training sessions was accompanied in control cats and in cats with amygdala lesions by an increase of bar presses during intertrial intervals.

The enhancement of intertrial responding can be a symptom of behavioral activation (Zieliriski 1990). The behavior of cats subjected before training to bilateral electrolytic lesions restricted to a fragment of the dorso-lateral part of the central nucleus of the amygdala, seems to support the hypothesis that the rise of ITR rate promotes defensive instrumental learning. The symmetrical bilateral amygdala lesions increased threshold to shock and, moreover, cats with such a loss responded to low shock intensities with longer latencies that short- ened progressively with the increase of shock intensity (Werka 1980, Zielinski et al. 1983). Reduced capacity of cats with amygdala lesions to excite stimulus-motor act associations when the eliciting stimulus was weak could be partially compensated by the behavioral activation mechanism. Accordingly, the enhancement of ITR rate was greater and lasted longer in lesioned than in control cats (Fig. 2). In all groups of cats of this study the ITR rate decreased at the stage of training when escape response latencies reached plateau.

The potential use of the behavioral activation as a compensation mechanism seems to depend on the time span between the lesion and the beginning of training as well as on task requirements. The cats that started avoidance training ten days after nucleus centralis of the amygdala lesions were unable to increase ITR rate and showed very slow learning in contrast to cats trained 35 days after the lesion (Werka and Zielinski 1978). The retarded shuttle- box avoidance learning and very low ITR level in


trials

Fig. 2. The mean ITR rate (histograms) and the mean latency of escape responses in 50 trial blocks for normal cats (circles), for cats starting training 10 days (squares) or 35 days (hexagons) after bilateral lesions in the nucleus centralis of amygdala.

rats trained ten days after lesions of the nucleus centralis or the baso-lateral nucleus of amygdala was shown in a recent study (Werka and Zieliriski 1992).

VARIABILITY OF THE ITR FORM

The most interesting data concerning changes of the ITR form were obtained on dogs trained in spe- cial modification of the avoidance paradigm. A CS of fixed duration, but divided into two parts of different significance, was used. The first part signalled forthcoming painful shock applied through the electrodes located on one paw of the dog. The shock may be avoided or escaped only if this very paw is flexed and maintained above a definite height (Petropavlovskii 1934). In most experiments it is required that the paw is kept within a narrow "safety zone" confined within two altitudes (Stul 1974). The optimal strategy consists in flexion of the paw at the end of the CS-US interval, maintenance of fixed position of the leg at the required height during the shock availability period and to stand the paw on the floor just after the CS termi-

nation. Due to complex interactions between the unconditioned flexion response to painful stimulation, instrumental control of the flexion amplitude and classically conditioned fear response, the training is difficult and final success depends on the method of training, shock intensity and a dog's characteristic (Frolov 1983).

The form of the instrumental response learned in this procedure changed in the course of training (Shirkova 1956,1958). At the beginning, dogs performed phasic flexion followed by lowering of the paw, thus resulting in shock application eliciting, in turn, the next flexion, and so on, until the termination of the trial. Then, the typical response was a tonic flexion of a paw starting before and lasting longer than the shock availability period. Finally, leg flexion became restricted to the danger period only (Fig. 3). Since the shape of the CR and the ITR change in parallel, and the execution of intertrial response were preceded by a similar respiratory pattern as observed after the CS onset, the common instrumental origin of responses performed during the CS and during the intertrial intervals was postulated (Shirkova 1956).

220 K. Zielinski

Fig. 3. Variability of avoidance and intertrial responses in dogs trained by the Petropavlovskii Method (after Shirkova 1956). From top to bottom of each rec- ord: movement of the leg, CS, shock US, time in s. Interrupted line denote beginning of the danger period.

New data concerning this problem have been presented by Dolbakyan (1988,1991). A fixed duration of the CS employed in this procedure of avoidance training enabled detailed analysis of heart rate changes. It was shown on dogs, which attained high and stable avoidance performance levels, that the CS onset elicited heart rate increases reaching maximum soon before the leg flexion. This rise was followed by a reduction of heart rate up to the pre-CS period when the leg was kept in the safety zone during the danger period. Immediately after the CS termination and lowering of the paw, some short heart rate increase was observed, which was considered as a rebound response.

In spite of the efficient avoidance of shock on nearly all trials, ITRs were observed even after prolonged training. They were of two different forms: (I) phasic flexion of the paw or series of such responses and (2) prolonged tonic flexion similar in form to the well trained CR. Before execution of the ITR, the heart rate increased, more before phasic than before tonic responses. The heart rate increased further during the phasic flexion emitted in intertrial intervals, whereas some gradual decrease

of heart rate in the course of tonic ITRs was observed. In five of the six dogs, heart rate was higher before the ITRs than before the CRs. In the paper a hypothesis was formulated that phasic ITRs were reminiscent of classically conditioned motor responses to shock, while tonic responses emitted in absence of the CS may be considered true instrumental ITRs (Dolbakyan 1991).

The possibility of a different origin of responses performed by a subject in the same experimental context has been analyzed earlier by authors inter- ested in the problem of how reflexes are instrumen- talized (Konorski 1967, Shettleworth 1979). Extending their considerations further, it may be ar- gued that representation of the shock stimulus (US) at early stages of training is associated with the CS, the experimental context or with both. Excitation of US representation results in performance of phasic flexion. However, such paw movements do not guarantee a successful avoidance and/or escape of painful stimulation. Thus, the inborn motor flexion reaction is gradually transformed in a complex instrumental act, and the representation of the acquired motor pattern becomes associated with the


CS and the experimental situation. Excitation of the representation of the acquired motor pattern (and not of the US representation) leads to the execution of tonic flexion, either CR or ITR.

Phasic leg flexion ITRs, single or performed in series, remind of unconditioned response to painful electric shock from electrodes located on the paw. Such ITRs were preceded by a sharp rise of heart rate which gradually decrease after termination of the response. Tonic leg flection ITRs are similar to the learned instrumental response, and during execution of such ITR a gradual decrease of heart rate, similar to that accompanying performance of the CR, has been observed. The two forms of ITRs are consequences of different associative mechanisms. The mechanism based on excitation of the representation of complex motor pattern and involved in the "normal" instrumental response is acquired later and is absent at the early stages of training. The decrease of heart rate accompanying execution of the tonic leg flection CR during the danger period before the CS termination and, similarly, in the case of ITR not accompanied by any change of external stimulation, seems to indicate reduction of fear. Thus, it may be proposed that tonic ITRs not only possess adaptive function, but their performance is reinforced by a similar mechanism as involved in the strengthening of the instrumental defensive CR.

It must be recalled that higher heart rate increases were observed before the ITRs than for the CRs. This finding may indicate a facilitatory effect of the CS on evocation of the instrumental CR, which has to be compensated by a higher increase of the fear drive in the absence of the CS to elicit ITRs similar in form to CRs. Moreover, fear drive reduction caused by execution of the tonic ITR, as evidenced by heart rate changes, seems to be smaller than that following the CR. The difference in magnitude of drive reduction occurring due to the CR or the ITR execution may provide a mechanism for the gradual decrease of the ITR frequency in the course of training. However, a smaller reinforcement is possibly needed for maintenance than for acquisition of an instrumental response (Herrnstein 1966).

THE DIFFICULTY OF A TASK AND ITR RATE

We have recently showed the effects of stimulus modality on active avoidance learning and performance in rats trained in shuttle box. One experimental design was especially suitable for the analysis of avoidance and intertrial responses relations (Zieliriski et al. 1991, Exp. 11). Each group of rats was trained with a visual CS and an auditory CS in succession. To ease the transfer of the avoidance CR, a single session was inserted between the two stages of training, in which both stimuli, that used already and the new one, were given in a compound. The daily sessions consisted of 50 trials, each terminated with either an avoidance or an escape crossing response. During all training a 5 s CS-US interval and 15-, 20-, 25-s intertrial intervals, randomly distributed, were used. Crossing responses during intertrial intervals were permitted. The visual CS was darkness provided by termination of the ceiling light in the compartment occupied by the rat, and the auditory CS was onset of the 70 dB wide-band noise, also presented in the danger compartment.

Avoidance performance clearly differentiate two stages of training and the transfer session. Ac- quisition of the avoidance CR was more rapid to the auditory than to the visual CS. During the transfer session the same level of avoidance was observed in both groups. After the change of stimuli, the avoidance performance was determined by the modality of the CS actually used: higher for the auditory and lower for the visual CS. Rats trained originally with the visual and then switched to the auditory CS immediately reached the avoidance performance level that would be expected for rats trained with the auditory CS all the time. For those rats, performance during the first stage of training underestimate the attained level of learning.

The ITR frequency was also directly related to the modality of the CS used at different stages of the experiment. At the outset of training the ITR frequency was similar in both groups and a difference appeared in the course of learning. Already by the

222 K. Zielinski

1 2 3 4 5 6 7 8 9 sessions

Fig. 4. Numbers of avoidance and intertrial responses in five 10-trial blocks of consecutive nine training sessions for Group N-ND-D (above) and for Group D-ND-N (bottom). Histograms for intertrial responses and lines for avoidance responses. Filled triangles denote the darkness (D), circles denote the 70 dB noise (N), filled triangles in circles denote the compound CS con- sisting of 70 dB noise and darkness (ND).

second session, rats trained with the visual CS emitted several times more ITRs than found in rats trained with the auditory CS. Introduction of the noise, presented during the transfer session in compound with darkness, soon decreased the ITR frequency in rats trained previously with the visual CS. In the other group, the change from the noise to the darkness CS resulted in an immediate enhancement of the ITR frequency. The drastic increase of the ITR frequency when an auditory CS was changed to a less salient visual CS may be considered as a reaction to an increase of the task difficulty*.

In these experiments a monotonic increase of the avoidance performance was observed within each

training session. The avoidance performance at the beginning of each successive session was lower than that attained toward the end of the previous session. Insufficient retention of the avoidance response was observed on early sessions in all groups, but later was evident only in rats trained with the visual CS. The number of ITRs displayed a similar increasing within-session trend but only when rats were trained with the visual CS (Fig. 4).

Divergent experimental data indicate that, in contrast to visual stimuli, the onset and the termination of auditory stimuli are easily discriminated. We consequently postulated that the high frequency of ITRs found in rats trained with a visual CS was due

"Nearly identical results were obtained in another experiment in which rats were trained in two-way avoidance with wide-band noise of 60 dB intensity and ceiling light onset as the CSi (Zieliriski et al. 1991, Exp. 111). The similarity of the effects observed with the darkness CS and the light during the intertrial intervals and with the opposite arrangement of visual stimulation indicates that the behavior of rats depends rather on modality of the stimuli than on their specific intensity characteristics.


to insufficient discrimination of the opposite changes in illumination level signalling respective- ly danger and safe periods in the shuttle-box (Zielinski et al. 1991). Monotonic increases of both CR and ITR performance of rats trained with a visual CS were found in the sessions that started with series of shock trials terminated by the escape response. Dur- ing those trials the fear cumulated and its response invigorating effect resulted in an increase of the tendency for a running response. In the shuttle-box situation the rats' innate tendency for running when in danger is opposed by the resistance to enter a compartment in which rat has been shocked previously (Anisman and Wahlstein 1974). Crossing responses during intertrial interval may be accom- plished more readily than during the sporadic CS which action increases a fear level. Thus, performance of the ITRs facilitates locomotion toward the central partition and helps to shape the instrumental response. When the avoidance performance reachedplateau (which was lower for visual than for auditory CS) the ITR rate decreased and bimodal or irregular within-session distributions of ITR frequency were observed.

Such an understanding of the origin and the role of ITRs in two-way avoidance training provides the explanation of the parallel within-session increase of avoidance performance and ITR frequency. The opportunity to execute behaviors similar in form to the CR in the absence of the sporadic CS increases the flexibility of behavior.

ITRs IN EXTINCTION AND REVERSAL LEARNING

Enhancement of the ITR rate discussed in the previous section was elicited not by a transfer to the stimulus of a different modality per se, but by the change to the less salient CS and resulting increase of the number of shock trials. We found also marked changes in an experiment in which the requirements of the task were changed after acquisition of the instrumental response (Zielinski et al. 1983). Cats were at first trained to escape pain from an activated

grid-floor by a bar press response (350 trials), then this instrumental response was subjected to extinction (100 trials), retrained (150 trials) and finally transformed to an avoidance bar press response (1 50 trials). The mean duration of the grid-floor activation was used as a measure of the efficacy of the instrumental performance, independently of the task requirements. In Fig. 5 this measure and the mean ITR rate are presented for three groups of cats: intact, after bilateral lesions of the nucleus lateralis or the nucleus centralis of amygdala.

At all stages of the experiment, the ITR rate was markedly lower in lesioned than in control cats. In- troduction of a constant 5 s shock, not terminated by the bar press response, resulted in an enhancement of ITR rate in normal cats and, although on a very low level, in cats after nucleus centralis lesions. Extra responses appeared on this stage of training in each group of cats. The escape response extinction procedure resulted also in a subsequent lengthening of escape latencies in all cats but, as seen in Fig. 5, during the 2nd and the 3rd sessions of retraining the shocks remained long only in cats with nucleus centralis lesions. Only the normal cats, i.e., those with the highest ITR rate, were able to acquire efficient avoidance behavior as evidenced by very short mean shock duration on the last stage of training. Thus, group of cats with the lowest ITR rate suffer more than the others from the procedure aimed to weaken the instrumental response. The same cats were also not able to shorten the shock duration when the avoidance procedure was introduced.

We found a drastic enhancement of the ITR and extra response rate during the reversal learning, when stimuli signalling danger and safety were ex- changed. In an experiment conducted on cats the 1,000-Hz tone and 51s click stimuli, both of 65 dB intensity, were used (Zieliriski and Czarkowska 1974). At first an avoidance bar press response to one CS was trained with a 5 s CS-US interval. Ten trials daily were presented with 40-, 60- and 80-s intertrial intervals randomly distributed. After reaching a criterion of 90 avoidance responses in 100 trials, the other stimulus was introduced and presented also 10 times daily for 5 s, but never

224 K. Zielinski

Fig. 5. The mean ITR rate (histograms) and the mean shock duration for normal cats (circles), cats with bilateral lesions in the nucleus lateralis (triangles) or in the nucleus centralis of the amygdala (squares). The data were averaged over the last two sessions of each stage of training: escape training (T), extinction (E), retraining of the escape (R), and avoidance training (A).

paired with shock. The order of the stimuli presentation was according to Gellerman series. The 90 % criterion of correct responses for each kind of trial in 10 sessions had to be reached twice with 10 days pause between the two runs of sessions. After such overtraining the signalling values of the stimuli were reversed and training continued for the next 30 sessions.

Two groups of normal cats analyzed here differed in the arrangement of stimuli signalling danger and safety. Data for the end of the original differentiation and for the reversal learning, averaged for five session blocks, are presented in Fig.6. As seen, at the beginning of reversal training the ITR and extra response rate increased ten times in each group and, in spite of a marked reduction in consecutive blocks, did not return to the previous level.

The change of the signalling value of the stimuli resulted in a drop of avoidance performance and an increase of shock trials terminated by escape re-

sponses. The rapidity of avoidance transfer and the extinction efficacy of the previously excitatory CS differed between two groups. Transfer of the avoidance response from the tone to the more salient click CS was easier than from the click to the less salient tone CS. Similarly, extinction of responding to the click was more protracted than to the tone. Com- parison of the two groups suggests that the effort related to correct evaluation of the meaning of each stimulus was the main reason for the ITR rate enhancement, independent of the exact number of increased shock trials or the number of errors committed to the stimulus that now signalled safety.

A similar study on dogs clearly showed, however, that the main disturbing factor in normal animals was the inability to terminate the previously excitatory stimulus by the bar press response (Kowalska et al. 1975). Transfer of the avoidance response to a new positive stimulus was very rapid, independently on stimuli arrangements. On the other side, bar presses to the previously excitatory


-

diff. reversal di i f . reversal

Fig. 6. The mean ITR rate (histograms), the mean percentage of avoidance responses (filled symbols) and disinhibited trials (open symbols) averaged in five session blocks of the end of the original differentiation and the reversal learning in two groups of cats. Circles for the tone and triangles for the click stimulus.

tone gradually decreased, whereas those to the click stimulus were performed on each reversal trial. The dogs that were unable to extinguish bar presses to the click stimulus, emitted during the reversal training five times more ITRs and extra responses than those extinguishing bar presses to the tone CS.

The data collected at the beginning of the original differentiation training were not conclusive. Marked individual variability of the ITR rate was observed, both during criterion sessions of the avoidance response acquisition and when a stimulus signalling safety was introduced. The two opposite processes, a gradual decrease of ITRs at the stage when avoidance performance reached plateau and an increase of ITR rate related to introduction of the new CS, gave very variable outcome in individual subjects.

Our data from reversal training suggest that the change of the task requirements and not the increase of shock density was a primary reason for the ITR enhancement. This statement is in agreement with

classic papers on avoidance behavior in dogs. A change from acquisition to extinction in which no shock was used resulted in a marked increase of ITRs emitted by dogs trained in a two-way avoidance with either delayed (Church et al. 1956) or trace-conditioning (Kamin 1954) procedures. Frus- tration inevitably elicited by all of those changes summed with and enhanced the fear drive, resulting in very pronounced increase of the ITR frequency.

REACTIVITY, MOTOR ACTIVITY AND AVOIDANCE

In a number of studies, a correspondence between ITR frequency and avoidance performance was observed. In an experiment in which running wheel avoidance of shock was investigated in eight groups of rats differing in the consequences of the instrumental response on CS termination and avoid-

ance of shock, it was shown that the number of ITRs highly correlates across conditions with avoidance performance (Bolles et al. 1966). Successful acquisition of two-way avoidance in rats was characterized by a rapid initial increase of ITRs, whereas rats that failed to reach avoidance criterion performed a small number of ITRs (Abuladze and Chuchulashvili 1982). We found a significant positive correlation between ITR frequency and avoidance performance during early sessions of two-way avoidance training of rats (Zielinski et al. 1991, Exp. IV).

Experiments aimed at genetic selection for shuttle-box avoidance in rats also showed a gener- ally positive relations between ITR frequency and avoidance performance. Roman High Avoidance rats were characterized by higher general activity (Bignami 1965) and also by higher ITR rate (Broadhurst and Bignami 1965, Holland and Gupta 1966) than the Roman Low Avoidance line. In another study, selection for good or poor two-way avoidance was combined for both lines with an additional criterion of limited responding to the to-be- CS during pretest trials. The results indicated that "avoidance behavior is a heritable characteristic which can be selected independently of activity level" (Brush et al. 1979, p.315), since the frequency of crossings during pretest was not related to the difference in avoidance behavior. However, tht introduction of shock during training sup- pressed crossing responses performed during intertrial intervals more in Syracuse Low Avoidance (SLA) rats than in Syracuse High Avoidance (SHA) line (Brush et al. 1985). In effect, on many gener- ations SHA performed more ITRs than SLA rats. Moreover, the two lines did not differ in shock sensitivity but testing in the open-field suggest that rats of SLA line were more emotionally reactive than those of the SHA line (Brush et al. 1985). The SLA rats acquired conditioned suppression faster and showed greater suppression of baseline responding indicating their greater emotionality than the SHA animals (Brush et al. 1988).

All of the above considerations suggest that avoidance performance and intertrial responding

are controlled by independent factors. Individual and group differences in ITR frequency occurs during training and depends on training conditions, CS characteristic being an important factor, and on emotionality of the genetic line and each individual subject. The instrumental defensive learning may be achieved without any ITR, when responses similar in form to the CR are physically prevented, however, performance of intertrial responses is an important factor promoting rapid and successful training. Decrease of the ITR frequency by introduction of a partition between two compartments or punishment of intertrial responses has a deleterious effect on the two-way avoidance acquisition (Big- nami et al. 1985, Bignami 1989). Crossing a partition requires more precise coordination of the body movements than for a regular running response. Punishment of the ITR necessarily decreases probability of crossing response and increases a tendency for freezing - a response incompatible with active avoidance behavior.

GENERAL CONSIDERATIONS

Changes in intertrial responding necessarily influence the correlation between response and reinforcement. The Thorndikean Law of Effect postulated that the consequences of a response cause the strengthening or weakening of the S-R associations. Examples of positive correlations between the high level of ITRs - i.e., responses not reinforced, reinforced by smaller drive reduction or reinforced after longer delay than the CR - and the more efficient learning, are all embarrassing for strict S-R theories.

Attempts to compare the efficacy of learning in procedures differing only in the possibility to per- form the acquired response during intertrial intervals raise serious interpretation problems. It was shown recently (Hachiya and Ito 1991) that rats trained to press a lever for food learned the discrimination of tonal frequencies much easier when a response lever was available all the time than if it was retracted from a chamber during intertrial intervals.


The frequency of responding to a tone signalling no food differed markedly between the groups. This finding was interpreted as an overshadowing or masking effects exerted by the lever presented sim- ultaneously with the tonal stimuli on the trial onset. No data were presented, however, concerning influence of the two procedures on learning parame- ters of the lever press response to a single stimulus before differentiation learning.

Termination of the trial by an instrumental response reduces the excitation elicited by the onset of a CS but not completely. In the defensive situation, fear associated with the experimental context surnmate with residual fear of the previous trial and may summate also with fear of a next expected trial. The data reviewed in this paper indicate that fluctuations of fear are responsible for the occurrence of ITRs during training of instrumental defensive behavior.

A hypothesis may be proposed that ITRs, which are a consequence of emotional tension, i.e., fear, are a tool for the decrease of the tension. The execution of ITRs would have similar positive effects on learning and performance of a conditioned response as lengthening of intertrial intervals. It was shown that the stress accompanying avoidance training was greater with short than with long intertrial intervals, thus an increase of the intervals separating adjacent trials facilitates avoidance training (Brush et al. 1964, Kurtz and Shafer 1967, Bignarni et al. 1985).

It seems that behavioral activation, encompas- sing a state of "arousal of the motor system", considered by Konorski (1 967), plays an important role in learning. Analysis of the method employed originally by Konorski for reward training in dogs (Konorski and Miller 1936) showed that the change from continuous to the partial food reinforcement elicited an enhancement of motor activity. Soon after the required movement was performed on most of the trials the motor excitement disappeared (Zielinski 1980). It has been postulated that the increase of the motor activity and enhancement of ITRs are manifestations of innate mechanisms promoting discovery of the relations between important events in an environment and formation of new

response-reinforcer and stimulus-reinforcer associations (Zielinski 1989).

By now participation of the mechanisms responsible for occurrence of intertrial responses and for their further performance are largely neglected. It seems that more attention has to be paid to the effects of behaviors manifested during intertrial intervals on the formation and strengthening of the associations between discriminative stimulus, response and reinforcement. Presumably, only future theories of learning will be able to summarize the existing data concerning intertrial responding and point out its role in the acquisition and maintenance of CRs.

ACKNOWLEDGEMENTS

Preparation of this paper was supported by a statutable grant from the State Committee for Scien- tific Research to the Nencki Institute. The author is grateful to Dr. J.F. Brennan for interesting discus- sions and correction of the English style.

REFERENCES

Abuladze G.V., Chuchulashvili N.A. (1982) Analysis of emotional component of two-way avoidance conditioned reflex (in Russian). Zh. Vyssh. Nervn. Deyat. I. P. Pavlova 32: 269-275.

Anisman H., Wahlsten D. (1974) Response initiation and di- rectionability as factors influencing avoidance performance. J. Comp. Physiol. Psychol. 84: 11 19-1 128.

Bignami G. (1965) Selection for high rates and low rates of avoidance conditioning in the rat. Anim. Behav. 13: 221- 227.

Bignami G. (1989) To go or not to go in aversive paradigms: Preparedness and other questions. In: Aversion, avoidance, and anxiety. Perspectives on aversively motivated behavior (Eds. T.Archer and L.-G. Nilsson). Hillsdale Erl- baum, New York, p. 343-370.

Bignami G., Alleva E., Amorico L., De Acetis L., Giardini V. (1985) Bidirectional avoidance by mice as a function of CS, US, and apparatus variables. Anim. Learn. Behav. 13: 439-450.

Black A.H., Carlson N.J. (1954) Traumatic avoidance learning: A note on intertrial-interval responding. J. Comp. Physiol. Psychol. 52: 759-760.

Bolles R.C., Stokes L.W., Younger M.S. (1966) Does CS termination reinforce avoidance behavior? J. Comp. Physiol. Psychol. 62: 201-207.

228 K. Zielinski

Broadhurst P.L., Bignami G. (1965) Correlative effects of psychogenic selection: A study of the Roman High and Low Avoidance strains of rats. Behav. Res. Ther. 2: 273- 280.

Brush F.R. (1962) The effects of intertrial interval on avoidance learning in the rat. J. Comp. Physiol. Psychol. 55: 888-892.

Brush F.R., Froehlich J.C., Sakellaris P.C. (1979) Genetic selection for avoidance behavior in the rat. Behav. Genet. 9: 309-316.

BrushF.R., Baron S., Froehlich J.C., Ison J.R., Pellegrino L.J., Phillips D.S., Sakellaris P.C., Williams V.N. (1985) Genetic differences in avoidance learning by Rattus nor- vegicus: EscapeIAvoidance responding, sensitivity to electric shock, discrimination learning, and open-field behavior. J. Comp. Psychol. 99: 60-73.

Brush F.R., Myer J.S., Palmer M.E. (1964) Joint effects of intertrial and intersession interval upon avoidance learning. Psychol. Rep. 14: 31-37.

Brush F.R:, Del Paine S.N., Pellegrino L.J., Dess N.K., Rykaszewski I.M., Collins P.Y. (1988) CER suppression, passive-avoidance learning, and stress-induced suppression of drinking in the Syracuse High- and Low-Avoidance strains of rats (Rattusnowegicus). J. Comp. Physiol. Psy- chol. 102: 337-349.

Church R.M., Brush F.R., Solomon R.L. (1956) Traumatic avoidance learning: The effects of CS-US interval with a delayed-conditioning procedure in a free-responding situation. J. Comp. Physiol. Psychol. 49: 301-308.

Devenport L.D., Holloway F.A. (1980) The rat's resistance to superstition: role of the hippocampus. J. Comp. Physiol. Psychol. 94: 691-705.

Dolbakyan E.E. (1988) New data on correlation of heart and motor components of defensive instrumental conditioned reflex in dogs (in Russian). Zh. Vyssh. Nervn. Deyat. I. P. Pavlova 38: 65 1-659.

Dolbakyan E.E. (1991) Conditioned avoidance reflex and intertrial responses: Cardiac-motor relationships (in Rus- sian). Zh. Vyssh. Nervn. Deyat. I. P. Pavlova 41: 267-280.

Frolov A.G. (1983) The effect of instrumentalization of inborn reaction on its transformation into contrary directed escape response in dogs and the problem of reinforcement. Acta Neurobiol. Exp. 43: 1-14.

Hachiya S., Ito M. (1991) Effects of discrete-trial and free- operant procedures on the acquisition and maintenance of successive discrimination in rats. J. Exp. Anal. Behav. 55: 3-10.

Herrnstein R.J. (1966) Superstition: A corollary of the princi- ples of operant conditioning. In: Operant behavior. Areas of research and application (Ed. E.K.Honig). Appleton- Century-Crofts, New York, p. 33-51.

Hilgard E.R. (1938) A summary and evaluation of alternative procedures for the construction of Vincent curves. Psy- chol. Bull. 35: 282-297.

Holland N.C., Gupta B.D. (1966) Some correlated measures of activity and reactivity in two strains of rats selectively bred for differences in the acquisition of a conditioned avoidance response. Anim. Behav. 14: 574- 580.

Kamin L.J. (1954) Traumatic avoidance learning: The effects of CS-US interval with a trace-conditioning procedure. J. Comp. Physiol. Psychol. 47: 65-72.

Konorski J. (1967) Integrative activity of the brain: An inter- disciplinary approach. Univ. Chicago Press, Chicago.

Konorski J., Miller S. (1936) Conditioned reflexes of the motor analyzer (in Russian). Tr. Fiziol. Lab. Akad. 1.P.Pavlova 6: 119-288.

Kowalska D., Dabrowska J., Zieliriski K. (1975) Effects of partial prefrontal lesions in dogs on go-no go avoidance reflex differentiation and reversal learning. Acta Neurobiol. Exp. 35: 549-580.

Kurtz P.S., Shafer J.N. (1967) The interaction of UCS intensity and intertrial interval in avoidance learning. Psychon. Sci. 8: 465-466.

Pavlov, I.P. (1927) Conditionedreflexes (G. V. Anrep, trans.). Oxford University Press, London.

Petropavlovskii V.P. (1934) To the method of motor conditioned reflexes (in Russian). Fiziol. Zh. SSSR im. I.M. Sechenova 17: 217-226.

Shettleworth, S.J. (1979) Constrains on conditioning in the writings of Konorski. In: Mechanisms of learning and mo- tivation: A memorial volume to Jerzy Konorski (Eds. A. Dickinson and R.A. Boakes). Lawrence Erlbaum Associ- ates, Hillsdale, NJ, 399-416.

Shirkova G.I. (1956) On the neural mechanism of so called voluntary movements: study of intertrial responses (in Russian). Tr. Inst. Vyssh. Nervn. Deyat. Acad. Nauk SSSR Ser. Fiziol. 2: 75-89.

Shirkova G.I. (1958) On some peculiarities of motor defensive conditioning in dogs (in Russian). Zh. Vyssh. Nervn. Deyat. I. P. Pavlova 8: 393-402.

Stul N.I. (1974) Precise instrumental conditioned reflexes and the role of different sensory modalities in their execution in dogs. Acta Neurobiol. Exp. 34: 377-391.

Vincent S.B. (1912) The function of the vibrissae in the behavior of the white rat. Behav. Monogr. 1 : 1-8 1.

Werka T. (1980) Acquisition of the escape reflex in cats after the nucleus centralis of the amygdala lesions. Acta Neurobiol. Exp. 40: 433-449.

Werka T., Zielinski K. (1978) Effects of lesions in the amygdaloid nucleus centralis on acquisition and retention of avoidance reflexes in cats. Acta Neurobiol. Exp. 38: 247-270.

Werka T.T., Zielinski K. (1992) CS modality transfer of two- way avoidance in rats with central and baso-lateral amygdala lesions. First International Congress of the Pol- ish Neuroscience Society, Warsaw, 20-23 Sept. 1992. Acta Neurobiol. Exp. 52: 188.


Zielinski K. (1980) Involvement of the partial reinforcement procedure in reward training: opening address. In: The Warsaw Colloquium on instrumental conditioning and brain research (Eds. B .~ernicki and K.Zielinski). Martinus Nijhoff, the Hague, p. 5-16.

Zielinski K. (1989) The biology of conditioning. ActaPhysiol. Pol. 40: 145-155.

Zielinski K. (1990) Changes of behavioral strategy as a mechanism of relearning instrumental responses after cerebral lesions. Acta Neurobiol. Exp. 50: 151-162.

Zielinski K., Czarkowska J. (1974) Quality of stimuli and prefrontal lesions effects on reversal learning in go-no go avoidance reflex differentiation in cats. Acta Neurobiol. Exp. 34: 43-68.

Zielinski K., Plewako. M. (1980) The role of conditioned stimulus termination in short-latency avoidance responding in cats. Behav. Brain Res. 1: 379-395.

Zielinski K., Werka T., Nanieshvili T. (1983) Do amygdaloid nucleus centralis and nucleus lateralis serve similar func- tions in defensive responding in cats? Acta Neurobiol. Exp. 43: 141-163.

Zieliliski K., Werka T., Nikolaev E. (1991) Intertrial responses of rats in two-way avoidance learning to visual and auditory stimuli. Acta Neurobiol. Exp. 5 1 : 7 1-88.

Paper presented at the 1st International Congress of the Polish Neuroscience Society; Session: Learning and memory

intertrial responses in defensive instrumental learning

Documents