Sense modality switching and serialleaming

ADRIAN CHAN,1 UNIVERSITY OF WISCONSIN-MILWAUKEEAND ROBERT M. W. TRAVERS, WESTERN MICHIGAN UNIVERSITY

This investigation examined the effect ofsense modality switching ofinformation atvarious positions of a serial learning task.Different numbers of flashes of light anddifferent numbers of "blips" ofa pure tonewere used as signalsin a four-position seriallearning task. The design of the studyinvolved a switch in sense modality at eachone of the four possible positions in theseries. Thus, it was possible to determine ifthe transmission of information throughone sense modality makes it progressivelymore difficult or more time consuming toswitch to another source of informationtransmitted through another modality. Theresults generally supported the contentionthat, as information is received through aparticular modality, there is a build-up ofthe disruption involved in switching sensemodality.

A previous experiment by Chan andTravers (1965) studied the effects of sensemodality switching on the learning ofelements in a serial learning task. The studyinvolved a task with seven nonsensesyllables, each of which could be presentedin either the auditory or visual modality.Switching the sense modality oftransmission took place with some Ss onthe second position and with other Ss onthe fifth position. At the higher of the twospeeds of presentation used, there was nosignificant effect on learning produced byswitching modality in the second position.However, the switching of modality in thefifth position produced (1) a significantincrease in learning in the syllable on whichthe switch occurred and (2) a significantdecrement in learning of the followingsyllable (the sixth position), whichinvolved a switch back to the originalmodality. The increment was interpreted asa von Restorff effect, and the decrementwas considered to be a result of loss oftime involved in switching-a lossestablished by research that has beenreviewed and added to by Reid (1964).

The fact that in the Chan and Travers(1965) study the effect of switching wasmuch more marked in the fifth positionthan in the second suggests that, asinformation is received through a particularsense modality, switching to anothermodality requires an increasing amount oflost time. The present study investigatesfurther this tendency for switching effectsto become more marked as a particular

sense modality continues to be used forinformation transmission.

The lack of equivalence of nonsensesyllables suggested that it would beadvisable to substitute for them signals thatwould be more nearly equivalent to eachother. For this reason, the present studyused as signals either brief series of flashesof light or brief series of sound "blips."Each signal consisted of 3,4,5, or 6 flashesor blips. Four signals were givenconsecutively through either the auditoryor the visual modality. The task of the Swas to report at the end of each series offour signals the number of flashes or blipsin each of the signals.

While no recent related study usingflashes and blips could be found, Judd(1927) has used a similar task in whichadult Ss were to estimate the number offlashes presented at various rates fromthree to five per second and sound thatvaried from three to eight per second. Hisdata showed that (1) increasing rates ofpresentation resulted in increasing errors,(2) errors increased as the number offlashes or sounds increased in number, and(3) auditory signals were more readilyestimated numerically than were flashes.The latter finding was attributed by Juddto the fact that adults have had moreexperience in counting flashes.

METHODProcedure

The task of the S was to estimate thenumber of blips and/or flashes in each offour consecutive signals. Each blip or flashwithin each signal involved' a .09-secexposure of either light or sound followedby a rest interval of .03 sec. Thus threeblips and their rest intervals following themoccupied .36 sec, four blips occupied.48 sec, and so forth. The blips wereproduced by recording a 900-Hz sine wavetone on tape. The flashes were producedby a 3·W neon lamp placed 5 ft from the Sin a dimly lit room. A single signalconsisting of 3, 4, 5, or 6 blips or flasheswas transmitted and then was followed bya I-sec pause before the next signal wastransmitted. The l-sec pause betweensignals was established empirically byrunning Ss with intervals between signalsthat varied from 0.2 sec upwards. Intervalslower than 1 sec did not permit the 5 toprovide any accuracy in estimating thenumber of flashes or blips.

The experimental facilities consisted of aroom with a table and five chairs, aloudspeaker, and a tape recordersynchronized with a relay device connectedto a 3-W neon lamp. The auditory stimuliwere produced through the loudspeaker,which was placed behind the Ss, while thevisual stimuli were reproduced through theneon bulb mounted on a black piece ofcardboard located directly in front of theSs.

As the participants came in they weregiven written instructions that stated thatthe experiment they were about toparticipate in was a study to determinehow well they could remember seeing aseries of flash signals and how well theycould remember hearing a series of blipsignals. Each S was told that he would bepresented with 30 trials, and each trialconsisted of four consecutive series ofsignals via the auditory channel and/or thevisual channel. Each signal was to consistof 3, 4, 5, or 6 auditory blips or visualflashes. After each presentation trial, Sswere given 15 sec to write down thenumber of blips and/or flashes for each ofthe four serial positions.

Since the dependent variable was thenumber of errors on each condition, eachresponse on the S's answer sheet wasscored either right or wrong and themagnitude of each error was not taken intoaccount.

DesignTable 1 presents the 10 conditions used

in the experiment. The first eightconditions represent the experimentalconditions where switching of the auditoryand/or the visual signals in each of the four

Table 1Position of the Auditory (A) and Visual (V)

Signals in Each Condition

Position of Signal

Condition 2 3 4

1 Experimental V A A A2 Experimental A V V V

3 Experimental A V A A4 Experimental V A V V

5 Experimental A A V A6 Experimental V V A A

7 Experimental A A A V8 Experimental V V V A

9 Control A A A A10 Control V V V V

Perception & Psychophysics, 1970, Vol. 8 (1) Copyright 1970, Psychonomic Journals, Inc., Austin, Texas 37

positions is involved. The last twoconditions represent the control conditionswhere no switching of modality is involved.Each S was exposed three times to eachone of the 10 conditions.

The experiments required that there be30 sets of four signals each, that is, 30trials presented, with each trial containinga series of four signals. These wereprepared by assigning the numbers 3,4,5,and 6 at random to the cells of a 30 by 4matrix, with the restriction that no numberappear more than once in a single row offour cells. Each signal within a set of foursignals could be transmitted either throughthe auditory 01" the visual modalitydepending upon the particular conditioninvolved. In addition, since a particularcondition might provide an easier task:when it occurred in one position than inanother in the series of 30 trials, fourdifferent orders for the learning conditionswithin the 30 trials were prepared. Forexample, Condition 1 in Table 1 providedfor three different trials of four consecu-:tive signals-visual, auditory, auditory, andauditory, respectively. One trial of thisV A A A sequence may present 5 flashes, 4blips, 6 blips, and 3 blips consecutively. Asecond trial in the V A A A sequence maypresent 3 flashes, 4 blips, 5 blips, and 6blips consecutively. Finally, a third trial inthis V A A A sequence may present 6flashes, 3 blips, 5 blips, and 4 blipsconsecutively. Conditions 2, 3, 4, and soforth to Condition 10 would each consistof three different trials of a particularsequence of blips and flashes.

SubjectsThe Ss were derived from undergraduate

courses in educational psychology. FortySs were divided into four groups (10 Ss ineach group), each of which was assigned toone of the four randomized presentationorders of 30 trials.

each S participating in the experiment wasgiven 30 total trials.

The dependent variable was the numberof errors committed by each S on eachcondition and on combined similarconditions. The data were analyzedaccording to the number of errors made oneach of the four positions of learningsequence. From the four positions,assessment could be made as to thefacilitation or the disruption of learningwhen switching or switching-back of themodality presentation occurs.

Table 2 presents the mean number oferrors and the standard deviations of eachserial position of the experimentalconditions and the control conditions. Itshould be noted that these experimentalconditions are paired according to whichserial position the switch and/orswitch-back occurred. For example, if theswitch and switch-back presentationsoccurred in the second and third positions,then number of errors in the experimentalconditions involvingA V A A and V A V Vwould be combined. Likewise, the sameprocedure was used for the other serialpositions for the experimental groups. Asfor the control groups, the number oferrors in the A A A A condition and theV V VV condition were combined.

Recall also that each separate condition,e.g., A A A A, was composed of threedifferent trials, each trial using a differentsequence of blips and a different numberof blips. Thus, any combining of twosimilar conditions would really involve thesum of six different trials in terms of theirnumber of errors, three trials from onecondition and another three trials from theother condition.

The nature of the experimental designmade possible the use of t tests betweencorrelated means, mainly comparinglearning in the switch and switch-back

positions with learning in thecorresponding control positions. Correlatedt tests were also used in determining thesignificance of the differences in learningwhen sensory switching occurs inprogressively later positions as in thesecond, third, and fourth positions.

The :witch positions. Consider Table 2and note the mean number of errors forthe second-position switch involvingExperimental Conditions 1 and 2 and 3and 4. Each mean represented the meannumber of errors out of six trials. Forexample, consider the first entry in thetable (0.49 errors). This meant that each Swas exposed to Conditions 1 and 2 forthree trials each, so the table implies thatout of six trials the 40 Ss made an averageof 0.49 errors. That is to say, about oneresponse in 12 was wrong. Visualcomparison between these means and themean of the corresponding second positionin the control group (1.14 errors and 1.09errors as compared to 1.11 errors) revealsvery little difference. A formal comparisonof the data from these switch positionswith the control position indicated nostatistically significant differences.

However, the third position switch,involving Conditions 5 and 6, seems tohave considerably more error than thecorresponding position of the controlgroup. A significant difference was foundwhen comparing the two group means(t = 3.59, df = 78, P < .01). A significantdifference was also found when comparingExperimental Groups 7 and 8 on thefourth position switch with itscorresponding control position (t = 8.00,df=78, p<.OOI). In summary, moreerrors occurred in learning the blips andflashes when the switch occurred in thelater serial positions. Thus, in the secondposition, there was no marked decrementin learning, as measured by number of

RESULTSThe general experimental design called

for 40 Ss to be divided into four groups of10 Ss each, with each group representingone of four different randomized orders ofpresentations. Each randomizedpresentation consisted of 10 conditions(eight experimental and two controlconditions) as shown in Table 1. As isevident in Table 1, each conditionrepresented the different arrangements ofthe audio and visual transmission ofinformation in a four-position seriallearning sequence. More specifically, eachcondition consisted of three trials, witheach trial consisting of a different sequenceand number of blips and flashes. In short,

38

Table2Mean Number of Errors for Six Trials and Standard Deviations for Each Serial Position of Related

Pairs of Experimental Conditions and Control Conditions (N = 40 per cell;a total of six errors is possible for each cell).

Position

Experimental 2 3 4

Conditions M SO M SO M SO M SO

1 & 2 VAAA0.49 0.79 1.14* 1.53 1.10 1.43 0.84 0.84AVVV

3 & 4 AVAA0.94 1.31 1.09* 1.42 1.05** 1.31 0.86 1.18VAVV

5 & 6 AAVA0.67 1.00 0.94 .97 1.11* 1.05 1.01** 1.20VVAV

7 & 8 AAAV0.69 1.08 0.74 1.07 0.93 0.63 1.41* 1.72VVVA

Control Conditions9 & 10 AAAA

.86 1.36 1.11 1.43 .80 1.06 .74 .99VVVV

- switch position--

switch-back position

Perception & Psychophysics, 1970, Vol. 8 (1)

__________~M SO

Table 3Mean Number of Errors and StandardDeviations of All Audio or Predominantly

Audio Conditions and All Visual orPredominantly Visual Conditions

errors, whereas in the later positions, i.e.,the third and fourth positions, considerabledecrement in learning occurred.

The switch-back positions. There areonly two pairs of experimental conditionsthat yield information about theswitch-back position, and these areConditions 3 and 4 and Conditions 5 and6. In Conditions 3 and 4, the switch-backoccurred in the third position and inConditions 5 and 6 the switch-backoccurred in the fourth position. Incomparing the mean of 1.05 errors for thethird position switch-back of Conditions 3and 4 with the mean of 0.80 errors for thethird position of the control group, asignificant difference was found betweenthese two means (t = 5.84, df= 78,P < .01). A significant difference was alsofound when the means of the fourthposition switch-back of Conditions 5 and 6(1.01 errors) were compared with thecorresponding mean (0.74 errors) of thecontrol group (t = 2.38, df= 78, P < .05).

Comparison of the auditory conditionswith the visual conditions. The difficultyof the task in the all-auditory orpredominantly auditory conditions couldbe compared with the difficulty of the taskin the all-visual or predominantly visualconditions. From Table 3 it is evident thatthe visual treatments yielded more errors inlearning. A significant t indicated the highdegree of confidence that could be placedin the difference between the two groups(t = 3.90, df = 799, P < .01).

Conditions AAAA, VAAA,AVAA,AAVA,AAAV

Conditions VVVV, AVVV,VAVV, VVAV, VVVA

0:84

1.01

:88

.85

DISCUSSIONOf primary importance is the gradual

increment in difficulty that occurs as theposition where switching occurs is movedfrom the second to the third and to thefourth serial positions. This similarphenomenon occurred in the previousstudy by Chan and Travers (1965) wherethe later position switching resulted in afacilitation of learning on the switch andthen a disruption of learning on theswitch-back. The explanation given wasthat the earlier syllables in the presentationbuild up an expectation that theinformation is to be received through thesame modality and, as a result, thetransmission of information through adifferent modality becomes progressivelymore unexpected and, hence, increases thedifficulty of making the switch as well asthe time taken to make the switch. This, intum, leaves progressively less time forprocessing the information after theswitch. This explanation seems to fit verywell the results discussed here. Theevidence presented seems to suggest thatthe decrement due to switching isdependent on the serial position of theswitch, that is, the amount of disruptionincreases as the switch occurs later in theseries of signals. A similar experimentshould be repeated with signals receivedthrough one modality over much longerperiods in order to determine the extent towhich this build-up may progress.

Several features of this study seem to beinconsistent with some of the resultsobtained in the Chan and Travers (1965)study and in the study by Judd (1927). Inthe present study, analysis of the data bothin the switch position and in theswitch-back position clearly indicated adecrement in performance, whereas, in the '1965 study, analysis of the results showedan increment in learning in the switchposition and a decrement in theswitch-back positions (for the fifth andsixth positions).

The differences in findings could well beattributed to the differences in the tasks inthe two studies. The task in the presentstudy places emphasis on perceptualrecognition and short-term storage of theinformation, that is to say, storage for amatter of a few seconds. In contrast, thetask in the 1965 study places littleemphasis on perceptual processes, for eachof the syllables is easily read in the timeallotted, but it requires retention for amuch longer period of time. The difficultyof the task in the 1965 study comes fromthe difficulty of discriminating the signals.The difficulty of the task in the earlierstudy would appear to derive from therequirement that a considerable amount ofreadily recognizable information has to beretained for a period that may be as long asseveral minutes.

Finally, the data from the present studyare consistent with those of Judd, whoreported that Ss found it easier torecognize the number of auditory blipsthan the number of flashes of light. Judd'sexplanation that adults have moreexperience in counting intermittent soundsthan they have in counting flashes stillseems plausible.

REFERENCESCHAN, A., & TRAVERS, R. M. W. Effect of

sense modality switching on serial learning.Perceptual & Motor Skills, 1965, 20,1185-1191.

JUDD, C. H. Psychological analysis of thefundamentals of arithmetic. SupplementaryEducational Monographs, 1927, 32, 16-32.

REID, I. E. Sense modality switching in relationto learning. Unpublished doctoral dissertation,University of Utah, 1964.

NOTEI. Address: University of

Wisconsin-Milwaukee, Milwaukee, Wisconsin53201.

(Accepted {OI'publkJztion October 30, 1969.}

Perception & Psychophysics, 1970, Vol. 8 (1) 39