probabilistic motor sequence yields greater offline and

ORIGINAL RESEARCHpublished: 02 March 2016

doi: 10.3389/fnhum.2016.00087

Probabilistic Motor Sequence YieldsGreater Offline and Less OnlineLearning than Fixed SequenceYue Du 1,2*†, Shikha Prashad 1,3†, Ilana Schoenbrun 1 and Jane E. Clark 1,3

1 Department of Kinesiology, University of Maryland, College Park, MD, USA, 2 Applied Mathematics and Statistics, andScientific Computation Program, University of Maryland, College Park, MD, USA, 3 Neuroscience and Cognitive ScienceProgram, University of Maryland, College Park, MD, USA

Edited by:Ovidiu Lungu,

Université de Montréal, Canada

Reviewed by:Erin J. Wamsley,

Furman University, USADavid L. Wright,

Texas A&M University, USA

*Correspondence:Yue Du

[email protected]

†These authors have contributedequally to this work.

Received: 29 October 2015Accepted: 19 February 2016Published: 02 March 2016

Citation:Du Y, Prashad S, Schoenbrun I andClark JE (2016) Probabilistic Motor

Sequence Yields Greater Offline andLess Online Learning than

Fixed Sequence.Front. Hum. Neurosci. 10:87.

doi: 10.3389/fnhum.2016.00087

It is well acknowledged that motor sequences can be learned quickly through onlinelearning. Subsequently, the initial acquisition of a motor sequence is boosted orconsolidated by offline learning. However, little is known whether offline learning candrive the fast learning of motor sequences (i.e., initial sequence learning in the firsttraining session). To examine offline learning in the fast learning stage, we asked fourgroups of young adults to perform the serial reaction time (SRT) task with either afixed or probabilistic sequence and with or without preliminary knowledge (PK) of thepresence of a sequence. The sequence and PK were manipulated to emphasize eitherprocedural (probabilistic sequence; no preliminary knowledge (NPK)) or declarative(fixed sequence; with PK) memory that were found to either facilitate or inhibit offlinelearning. In the SRT task, there were six learning blocks with a 2 min break betweeneach consecutive block. Throughout the session, stimuli followed the same fixed orprobabilistic pattern except in Block 5, in which stimuli appeared in a random order.We found that PK facilitated the learning of a fixed sequence, but not a probabilisticsequence. In addition to overall learning measured by the mean reaction time (RT), weexamined the progressive changes in RT within and between blocks (i.e., online andoffline learning, respectively). It was found that the two groups who performed the fixedsequence, regardless of PK, showed greater online learning than the other two groupswho performed the probabilistic sequence. The groups who performed the probabilisticsequence, regardless of PK, did not display online learning, as indicated by a declinein performance within the learning blocks. However, they did demonstrate remarkablygreater offline improvement in RT, which suggests that they are learning the probabilisticsequence offline. These results suggest that in the SRT task, the fast acquisition of amotor sequence is driven by concurrent online and offline learning. In addition, as theacquisition of a probabilistic sequence requires greater procedural memory comparedto the acquisition of a fixed sequence, our results suggest that offline learning is morelikely to take place in a procedural sequence learning task.

Keywords: online learning, offline learning, probabilistic sequence, sequence knowledge, procedural memory,declarative memory, fast motor sequence learning

Frontiers in Human Neuroscience | www.frontiersin.org 1 March 2016 | Volume 10 | Article 87

http://www.frontiersin.org/Human_Neuroscience

http://www.frontiersin.org/Human_Neuroscience/editorialboard

http://www.frontiersin.org/Human_Neuroscience/editorialboard

http://dx.doi.org/10.3389/fnhum.2016.00087

http://crossmark.crossref.org/dialog/?doi=10.3389/fnhum.2016.00087&domain=pdf&date_stamp=2016-03-02

http://journal.frontiersin.org/article/10.3389/fnhum.2016.00087/abstract



http://loop.frontiersin.org/people/75352/overview




https://creativecommons.org/licenses/by/4.0/

mailto:[email protected]

http://dx.doi.org/10.3389/fnhum.2016.00087


http://www.frontiersin.org/

http://www.frontiersin.org/Human_Neuroscience/archive

Du et al. Sequence Type Modulates Learning Processes

INTRODUCTION

In the laboratory, studies employing the serial reaction time(SRT) task (Nissen and Bullemer, 1987) have demonstratedthat adults can learn a motor sequence quickly within a singletraining session (i.e., in 4 to 8 practice blocks; Nissen andBullemer, 1987; Willingham et al., 1989; Robertson, 2007).This initial stage of motor sequence learning is referred to asfast learning that leads to the initial acquisition of sequences(Honda et al., 1998; Karni et al., 1998; Walker et al., 2002;Dayan and Cohen, 2011; Censor et al., 2012). Fast learningdevelops over the course of a single training session, where anindividual practices a new motor sequence and demonstratesconsiderable performance improvement. It has been suggestedthat such improvement in the performance of motor sequencesare driven by online learning (Bornstein and Daw, 2012,2013; Verstynen et al., 2012), where performance progressivelyimproves as the task is practiced. After the fast learning stage,performance is strengthened without further practice (i.e., offlinelearning) by an early offline boost (Hotermans et al., 2006;Schmitz et al., 2009) or memory consolidation (Robertson et al.,2004b, 2005; Brown and Robertson, 2007a; Nettersheim et al.,2015). To date, it is unclear whether offline learning drivesthe acquisition of motor sequence in the fast learning stage.The purpose of this study, therefore, is to examine whetherfast learning of a motor sequence arises from offline learning.Furthermore, given that offline learning in the SRT task hasbeen found to be associated with procedural memory (Robertsonet al., 2004a; Brown and Robertson, 2007a,b), we furtherinvestigate whether a bias towards procedural or declarativememory in the SRT task modulates offline and online sequencelearning.

Learning motor sequences in the SRT tasks typically involvesboth procedural and declarative memory (Willingham et al.,1989; Curran and Keele, 1993; Reber and Squire, 1994;Willingham and Goedert-Eschmann, 1999; Destrebecqz andCleeremans, 2001; Brown and Robertson, 2007a; Robertson,2007). In this task, participants press keys on the keyboardto respond to sequential visual stimuli that are presentedin a pattern (e.g., a fixed order). Since participants are notinformed of the presence of the sequence, learning in the SRTtask requires procedural memory. However, participants mayrecognize the presence of the sequence after they perform thetask and thus form a declarative memory of the sequence(Perruchet et al., 1997; Willingham and Goedert-Eschmann,1999). This entanglement of procedural and declarative learningsuggests the infeasibility of eliminating or isolating either ofthem from the SRT task. Nonetheless, manipulating the sequencetype and the preliminary knowledge (PK) of the sequence canmodulate procedural or declarative learning. Particularly, it hasbeen shown that learning a probabilistic sequence favors moreprocedural memory compared to learning a fixed sequence(Jiménez et al., 1996; Song et al., 2007). In contrast, PK of thesequence facilitates declarative learning (Curran and Keele, 1993;Curran, 1997; Destrebecqz, 2004).

In this study, we bias the involvement ofprocedural/declarative memory by manipulating the sequence

type and PK of the sequence in the SRT task to examine whetheroffline or online learning mediate the acquisition of motorsequences in the fast learning stage. Before the experiment, weinformed half of the participants that the visual stimuli followeda specific pattern, but no further information was providedabout the sequence. No information about the presence of asequence was provided to the other participants. The participantswere further divided into two groups. In one group, the visualstimuli followed a fixed sequence (i.e., 10 repetitions of a 12-trialsequence) while in the other group; the visual stimuli followed aprobabilistic sequence that was generated by a first-orderMarkovprocess. We found that a motor sequence is learned quicklythrough concurrent online and offline learning. However, theinvolvement of procedural or declarative memory mediated theuse of online and offline learning. Particularly, learning of afixed sequence arose from greater online learning. In contrast,acquisition of a probabilistic sequence resulted from significantoffline learning, regardless of PK. These results suggest that theinvolvement of procedural and declarative memory modulateshow a motor sequence is learned in the fast learning stage.

MATERIALS AND METHODS

This study was carried out in accordance with therecommendations and approval of the Institutional ReviewBoard at the University of Maryland, College Park. Allparticipants signed consent forms prior to their participation.Each participant received $10 after the completion of theexperiment.

ParticipantsForty-eight right-handed adults (24 males, see Table 1) wererandomly assigned to one of four groups: fixed sequencewith PK of the sequence (PK_Fixed; mean age: 21.8 ± 1.91),fixed sequence without PK of the sequence (NPK_Fixed;mean age: 21.5 ± 1.41), probabilistic sequence with PKof the sequence (PK_Prob; mean age: 21.2 ± 0.893), andprobabilistic sequence without PK of the sequence (NPK_Prob;mean age; 21.3 ± 0.830). All participants completed a healthquestionnaire to exclude those with any neurological and motorimpairments, the Edinburgh Handedness Inventory (Oldfield,1971) to assess that participants were right-handed, and theGlobal Physical Activity Questionnaire (Armstrong and Bull,2006) to insure that groups did not differ in their level of physicalactivity.

TABLE 1 | Participant demographic information.

Group Age (years)### Sex

PK_Fixed 21.8 ± 1.91 6 females; 6 malesNPK_Fixed 21.5 ± 1.41 6 females; 6 malesPK_Prob 21.2 ± 0.893 6 females; 6 malesNPK_Prob 21.3 ± 0.830 6 females; 6 males

#There were no significant differences between the groups in age, F(3,47) = 0.564,

p = 0.642.






Serial Reaction Time TaskParticipants were seated in front of a computer monitor (19′′)and keyboard. Participants placed the middle finger of their lefthand on the keyboard’s ‘‘D’’ key, the index finger of their lefthand on the ‘‘F’’ key, the index finger of their right hand onthe ‘‘J’’ key, and the middle finger of their right hand on the‘‘K’’ key (see Figure 1A). At the beginning of each trial, a mouseappeared in one of four squares on the screen and the participantpressed the key that corresponded to the location of the stimulus.After the participant pressed a key, the next stimulus appearedafter an interval of 300 ms. No visual feedback was provided toparticipants and a wooden board blocked vision of their fingerposition. Participants were first randomly assigned to either thePK group or no preliminary knowledge (NPK) group and werefurther randomly assigned to either the fixed or probabilisticsequence. The probabilistic sequence was created based on aMarkov chain transitional matrix with probabilities associatedwith each stimulus (Figures 1C,D). The probabilistic sequencewas constrained such that the same stimuli were not repeated oneafter the other and that each stimulus appeared an equal numberof times in each block.

There were a total of six blocks for all groups (see Figure 1B),each consisting of 120 trials. Prior to the first block, participantspracticed a random sequence. These initial trials were includedto ensure that participants were able to accurately associate eachfinger with a corresponding key before the experimental practiceblocks commenced. That is, we observed that participants didnot produce reaction times (RTs: amount of time taken to pressthe corresponding button after the stimulus was presented) thatwere slower than 2000 m because of incorrect key pressing.After the practice block, participants in the PK groups wereinformed that a sequence would be present in the subsequentblocks and that they should look for the sequence. No otherinformation about the nature of the sequence was provided. Thefirst four blocks (Blocks 1–4) were the learning blocks consistingof the 120-trial probabilistic sequence or the fixed sequence inwhich the sequence was repeated 10 times in each block. Block 5consisted of 120 trials of stimuli occurring in a random order andblock 6 consisted of the assigned probabilistic or fixed sequence(Figures 1C,D). Participants were given a 2minmandatory breakbetween each block. The participants’ RT was recorded for eachtrial.

All participants completed a posttest after the completion ofthe six blocks to determine the amount of declarative knowledgeof the sequence. Participants were first asked to recall thesequence and attempted to write down the 12 items of thesequence and rated how confident they were that the sequencethey wrote was correct. Participants were then asked to completea recognition task. They were given eight chunks (i.e., four three-element and four four-element chunks where two of each werecorrect) and were asked to choose the chunks they thought wereincluded in the sequence.

Data AnalysisThe RTs were trimmed according to the individual participant’smean and standard deviation. Within each block for an

individual participant, any RT greater or less than 2.5 standarddeviations was excluded from the analysis (Ratcliff, 1993;Whelan, 2008). Mean RTs were calculated for each block andwere averaged across participants in each group. Learningwas measured through a decrease in RT from block 1 to 4(stimuli in assigned sequence) and an increase in RT fromblock 4 (stimuli in assigned sequence) to block 5 (stimuli inrandom order). Online learning was defined as the amountof learning within a block and was determined by performinga linear regression on the 120 RTs within a block. Offlinelearning was computed as the RT change after a short breakwithout performing the task. Given that the fixed sequenceconsisted of 10 repetitions of a 12-item long sequence, thedifference between mean RT of the last 12 taps in one blockand that of the first 12 taps in the succeeding block wasused to quantify offline learning. In addition, since participantstypically acquire the sequence transitions of higher probabilitiesin probabilistic sequence learning (Hunt and Aslin, 2001;Howard et al., 2004; Bornstein and Daw, 2012), we expect thatparticipants in the two probabilistic sequence groups would onlylearn sequential stimuli that were associated with transitionalprobabilities of 0.3 and 0.6 and fail to learn those associatedwith transitional probability of 0.1. Thus, we computed meanRT, offline- and online-learning of stimuli with transitionalprobabilities of 0.3 and 0.6 in the two probabilistic sequencegroups.

A controversy regarding offline improvement inRT is whether the improvement results from reactiveinhibition/fatigue (Rickard et al., 2008; Brawn et al., 2010)or it is driven by active learning mechanisms (i.e., offlinelearning; Eysenck and Frith, 1977; Robertson et al., 2004a).According to Eysenck and Frith (1977), in the case of reactiveinhibition/fatigue-induced offline improvement, post-restperformance should return to the starting performancelevel before the rest or so called pre-rest performance, butwithout improvement over that level. In contrast, post-rest performance is superior to the pre-rest performanceif offline improvement arises from offline learning. Giventhat RT increased (i.e., became slower) within blocks insome participants so that the mean RT of the last 12 tapsmay not reflect the pre-rest performance, we calculatedcorrected offline learning. Specifically, if RT increased(i.e., became slower) within the previous block, correctedoffline learning was calculated by subtracting the amountof RT deterioration (i.e., negative online learning) withinthe previous block from the amount of offline learning sothat the corrected offline learning reflects the differencebetween the pre-rest and post-rest performance. If RTimproved (i.e., became faster) within the previous block,indicating no RT deterioration, corrected offline learningwas the same as offline learning, computed as the differencebetween mean RT of the last 12 taps in the block and thatof the first 12 taps in the succeeding block. We expect thatall groups should exhibit the same amount of correctedoffline learning (none), if offline improvement in RTobserved in this study were caused by reactive inhibition orfatigue.






FIGURE 1 | (A) Experimental setup. At the beginning of each trial, a stimulus appeared in one of four squares on the screen and the participant pressed the key thatcorresponded to the location of the stimulus. Participants placed the middle finger of their left hand on the keyboard’s “D” key, the index finger of their left hand onthe “F” key, the index finger of their right hand on the “J” key, and the middle finger of their right hand on the “K” key. (B) Experimental paradigm. Participantsperformed the learning blocks (blocks 1–4) with either the fixed or probabilistic sequence, followed by randomly ordered stimuli in block 5, and ended with the samesequence in block 6. All blocks consisted of 120 trials. Participants were given a mandatory 2 min break between each block. (C) Sequence Types. Participants wererandomly assigned to the fixed sequence group or the probabilistic sequence group. The probabilistic sequence was created using the probabilities defined in thetransitional matrix, T. (D) Example of how the probabilistic sequence was created using matrix T. If the current stimulus is 2, there is a probability of 0.6 that the nextstimulus will be 1, a 0.3 probability that the next stimulus will be 3, and a 0.1 probability that the next stimulus will be 4.

To measure the amount of declarative knowledge of thesequence, we calculated the recognition score as the numberof correct chunks that participants chose in the recognitiontask. The recognition score was normalized by four as therewere four correct chunks. To compare the recall score amongparticipants, we calculated the number of three-element chunksthat participants could recall. Given there were 12 three-elementchunks in the fixed sequence, the number that a participantrecalled was normalized by 12 to compute a percentage. Tomake the amount of declarative knowledge between probabilisticand fixed sequences comparable, the number of three-element

chunks that participants could recall was also used in thetwo groups who performed the probabilistic sequence. Sinceparticipants only learned the stimulus transition with transitionalprobabilities of 0.3 and 0.6 (for details, see ‘‘Results’’ Section),there were 16 three-element chunks in the probabilistic sequence.Thus, the percent of recalled chunks was normalized by 16 inthe two probabilistic sequence groups. Importantly, the chancelevel for guessing differed between the fixed and probabilisticsequence. Specifically, the chance level for a three-element chunkin the fixed sequence was 18.75% (i.e., given the first element,75% chance for the second element and 25% chance for the






third element) while it was 25% for a three-element chunk in theprobabilistic sequence (i.e., given the first element, 50% chancefor the second and third elements), we corrected the percentageof recalled chunks by the chance level specific to each sequencegroup.

Statistical AnalysisA three-way (block × knowledge × sequence) repeatedmeasures analysis of variance (ANOVA) was used to comparedifferences in RT between the blocks and groups. Separatepairwise comparisons were conducted on the priori contrastsof interest (block 1 vs. block 4 and block 4 vs. block 5) todetermine any significant differences between the sequencedblocks and the random block. A three-way (block × knowledge× probability) ANOVA was used to compare differences in RTof stimuli with different probabilities in the two probabilisticgroups. All repeated measures ANOVAs were performed inSAS with the MIXED procedure. Thus, the co-variance matrixstructures were determined by the Akaike information criterion(AIC). A two-way (knowledge × sequence) ANOVA wasemployed to examine the effects of PK and sequence typeon online, offline learning, and corrected offline learning. Atwo-way (knowledge × sequence) ANOVA was employed toexamine the effects of PK and sequence type on the recallscore. Given the violation of the normality assumption, theeffects of PK and sequence type on the recognition scorewas examined by the Scheirer-Ray-Hare test. Tukey-Kramerpost hoc tests were used to decompose any significant effects.Student’s t-tests/Wilcoxon tests were used to examine whetherrecall/recognition scores were different from the correspondingchance level for each group. The statistical significance level wasset as α = 0.05.

RESULTS

Figure 2A shows the mean RT across the six blocks. The repeatedmeasures ANOVA reveals a significant interaction between PK,sequence type, and block (F(5,44) = 2.79, p < 0.05). Post hocanalyses with the Tukey-Kramer correction found that all fourgroups produced comparable RTs in all blocks (all p > 0.2).However, RT in two groups who performed the fixed sequences(i.e., PK_Fixed and NPK_Fixed) improved from blocks 1 to 4and 6 (all p < 0.0001). In contrast, RT remained the samefrom block 1 to 4 in the other two probability sequencegroups (i.e., PK_Prob and NPK_Prob, Figure 2B; all p > 0.1).Nevertheless, RT was faster in block 6 compared to block 1in the NPK_Prob group (p < 0.01) and this improvementapproached significance in the PK_Prob group (p = 0.09). Inaddition, when a random sequence was introduced in block5, RT in the PK_Fixed and NPK_Fixed groups deteriorated(both p < 0.0001) while it remained the same between blocks4 and 5 in the PK_Prob and NPK_Prob groups (both p = 1;Figure 2C).

The inferior learning in the probabilistic sequence (asexpressed in no change in RT from block 1 to 4 andbetween blocks 4 and 5) is consistent with the hypothesisthat probabilistic sequences are harder to learn compared to

fixed sequences (Schvaneveldt and Gomez, 1998). However,given our hypothesis that participants typically acquire thesequence transitions of higher probabilities (Hunt and Aslin,2001; Howard et al., 2004; Bornstein and Daw, 2012), themarginal learning effect on the probabilistic sequence likelyresulted from the difference in RT among stimuli with differenttransitional probabilities (Figure 1C). Thus, we comparedRTs between these stimuli (Figure 2D) in the probabilisticsequence. A three-way (block × knowledge × probability)repeated measures ANOVA found that PK does not significantlyaffect RT and there was a significant interaction betweenblock and probability (F(10,220) = 17.07, p < 0.0001). Posthoc analyses with the Tukey-Kramer correction revealed thatRTs of stimuli with a transitional probability of 0.1 werecomparable to that of stimuli with transitional probabilityof 0.3, while RTs of stimuli with transitional probability of0.3 were slower than that of probability of 0.6 (p < 0.01).However, as learning progressed, RTs of stimuli with atransitional probability of 0.1 remained the same. In contrast,RTs improved from blocks 1 to 4 in stimuli with highertransitional probabilities 0.3 (p < 0.01) and 0.6 (p < 0.0001),suggesting learning of these higher transitional probabilities(Figure 2E). Additionally, introduction of a random sequencein block 5 did not impair RT of stimuli with transitionalprobabilities of 0.1 and 0.3, but RTs of stimuli with atransitional probability of 0.6 deteriorated in block 5 (p< 0.0001;Figure 2F). These results confirm that the participants learnedstimulus transitions with higher probabilities, specifically 0.6 andperhaps 0.3.

Since participants only learned higher transitionalprobabilities when stimuli followed a probabilistic pattern,we re-compared the learning effects among groups by usingRT for stimuli with transitional probabilities 0.3 and 0.6 inPK_Prob and NPK_Prob groups. A repeated measures ANOVArevealed a significant interaction among the effects of block, PK,and sequence (F(5,44) = 3.1, p < 0.05). Tukey-Kramer-correctedpost hoc analyses suggest that all groups had comparable meanRTs across all blocks (Figure 3A). In addition, all groupsdemonstrated improved mean RT from block 1 to 4 (allp < 0.0001) and deteriorated mean RT from block 4 to 5(all p < 0.005). However, contrast analyses showed that thePK_Fixed group had the greatest change in RT from block 1 to 4compared to the NPK_Fixed (p < 0.05), PK_Prob (p < 0.0005),and NPK_Prob groups (p < 0.0005; Figure 3B), while thelatter three groups exhibited the same change in RT. Similarly,the RT change from block 4 to 5 was greater in the PK_Fixedgroup compared to the other three groups (all p < 0.01) whohad the same RT change (Figure 3C). These results suggestthat the PK_Fixed group learned better than the other threegroups.

Although participants learned either fixed or probabilisticsequences with or without PK of the sequence, learningacross trials exhibited different patterns (Figure 4A).Specifically, learning of a fixed sequence exhibits decreasedRT within blocks while learning of a probabilistic sequenceexhibits reduced RT after rest without practice. A two-way(knowledge × sequence) ANOVA found a significant effect






FIGURE 2 | (A) Mean RT and SE bars across the six blocks for all four groups. (B) Difference between the RT in block 1 and block 4 to assess whether sequencelearning occurred. (C) Difference between block 5 and block 4 to assess whether RT increases in block 5 when a random sequence is presented. (D) Mean RT andSE bars across the six blocks for only the probabilistic sequence in which the three transitional probabilities (Pro 0.1, Pro 0.3, and Pro 0.6) have been extracted andplotted separately. (E) Difference between RT in block 1 and block 4. (F) In block 5 and block 4 separated for the three transitional probabilities in the probabilisticsequence. PK, preliminary knowledge; NPK, no preliminary knowledge; RT, reaction time; SE, standard error.

of sequence on offline learning (F(1,44) = 8.84, p < 0.005).Particularly, the acquisition of the probabilistic sequence arisesfrom greater offline learning compared to the acquisitionof the fixed sequence (Figure 4B). Although sequencetype was also found to significantly affect online learning(F(1,44) = 18.72, p < 0.0001), it was shown that greater onlinelearning was produced when a fixed sequence was learned(Figure 4B). Interestingly, when learning a probabilisticsequence, participants did not exhibit online learning. Instead,RT became slower within blocks. We further comparedwhether online or offline learning contributed more to theacquisition of a motor sequence. A two-way (knowledge ×sequence) ANOVA on the RT difference between offline andonline learning revealed a significant effect of sequence type(F(1,44) = 15.27, p < 0.0005). Student’s t-tests found equalonline and offline learning when a fixed sequence is performed(p = 0.59), while greater offline compared to online learning

was found when a probabilistic sequence was performed(p< 0.0001).

We also analyzed the corrected offline learning. The sameresults were found compared to the original offline learningdata (Figure 4C). A two-way (knowledge × sequence) ANOVAfound a significant effect of sequence (F(1,44) = 4.99, p < 0.05).Specifically, there was greater corrected offline learning inPK_Prob and NPK_Prob groups compared to PK_Fixed andNPK_Fixed groups. These results suggest that offline learningrather than reactive inhibition/fatigue underlies the offlineimprovement in RT.

In the posttest, we found that the recognition score didnot differ from chance (i.e., 50%) in all four groups andthere were no effects of sequence type and PK on the scores.Figure 5A shows the percentage of recalled three-elementchunks. It is clear that participants in the fixed sequencegroups had higher than chance recall, while recall was at






FIGURE 3 | Mean RT and SE bars to assess learning. Only the RT of stimuli with transitional probabilities of 0.3 and 0.6 were extracted and are shown for theprobabilistic sequences. (A) Mean RT across the six blocks. (B) Difference between the RT in block 1 and block 4 to assess whether sequence learning occurred.(C) Difference between block 5 and block 4 to assess whether RT increases in block 5 when a random sequence is presented. PK, preliminary knowledge; NPK, nopreliminary knowledge; RT, reaction time; SE, standard error.

FIGURE 4 | (A) Mean RT of each 12 taps to reflect online and offline learning. (B) Comparison of online and offline learning between the groups. (C) Comparison ofonline and corrected offline learning between the groups. Error bars represent standard errors. PK, preliminary knowledge; NPK, no preliminary knowledge; RT,reaction time.

chance in the two probabilistic sequence groups. The correctedpercentage according to the chance level was shown in Figure 5B.A two-way ANOVA found a significant effect of sequencetype (F(1,44) = 6.75, p < 0.01). Specifically, recall of thefixed sequence was superior compared to that of probabilisticsequence. In addition, using Student’s t-tests with an adjusted

p level of α4 = 0.0125 to control the familywise error rate for

the four simultaneous t-tests, the recall in the PK_Fixed wassignificantly higher than chance level (p < 0.0001) The recallin the NPK_Fixed did not differ from chance (but approachedsignificance, p = 0.0146), In contrast, recall in the two groupsthat performed the probabilistic sequence (i.e., PK_Prob and






FIGURE 5 | (A) Percentage of three-element chunks recalled in the posttest. (B) Corrected percentage according to chance level. Error bars represent standarderrors. PK, preliminary knowledge; NPK, no preliminary knowledge.

NPK_Prob) was not significantly different from the chance level(both p> 0.2).

DISCUSSION

In this study, we demonstrated that both fixed and probabilisticmotor sequences can be learned quickly (i.e., in one trainingsession). Further, this initial acquisition of a fixed sequencein the fast learning stage arises from both online and offlinelearning, while acquisition of a probabilistic sequence isdriven predominantly by offline learning. Given that learninga probabilistic or fixed sequence requires greater proceduralor declarative memory, respectively, our results suggest thata bias toward procedural or declarative memory modulateshow a motor sequence is learned in the fast learningstage.

Offline learning, as a salient feature underlying motorsequence learning (Robertson et al., 2004a), can boost thememory of a newly acquired sequence 5–30 min after the initialacquisition (Albouy et al., 2006; Hotermans et al., 2006, 2008;Schmitz et al., 2009; Nettersheim et al., 2015) or consolidate thememory a few hours later without sleep (Robertson et al., 2004a;Brown and Robertson, 2007a,b) or after sleep (Walker et al.,2002; Robertson et al., 2004b; Censor et al., 2012; Nettersheimet al., 2015). Thus, offline learning has been widely considered tooccur only after the initial acquisition of sequences that developsover the course of a single training session, referred to as fastlearning (Honda et al., 1998; Karni et al., 1998;Walker et al., 2002;Dayan and Cohen, 2011; Censor et al., 2012). Unlike the widely-found offline learning that occurs following the fast learningstage, we observed offline learning that drives the fast acquisitionof sequences within a first single training session. This resultsuggests that in addition to online learning (Cleeremans andMcClelland, 1991; Bornstein and Daw, 2012), offline learningalso contributes to rapid improvements in performance thatallow sequences to be learned quickly in a single trainingsession.

The concurrent effect of online and offline learningcould be modulated by the involvement of declarative andprocedural memory. It is widely accepted that both memorysystems cooperate and compete during motor sequence learning(Meulemans et al., 1998; Brown and Robertson, 2007b).Remarkably, the presence of declarative memory inhibitsoffline learning of procedural memory and thus disruption ofdeclarative memory induces offline improvement in proceduralskills 4 h after the initial acquisition (Brown and Robertson,2007a). In our study, similar effects of declarative andprocedural memory were observed on offline learning inthe fast learning stage. The recognition and recall testswere used to measure the engagement of declarative andprocedural memory in the SRT task. Although the recognitiontest shows no differences in the amount of declarativeknowledge acquired by participants regardless of the sequencetype and PK (see details below), the recall scores revealthat, participants acquired less declarative knowledge of theprobabilistic sequence. Notably, participants exhibited greateroffline learning when performing probabilistic sequences,suggesting that the offline learning in the fast learningstage was strengthened when greater procedural memory andless declarative memory were required to learn the motorsequences. On the other hand, when greater declarativememory was involved in learning fixed sequences, as indicatedby higher recall scores, reduced offline and greater onlinelearning were observed. This inverse relationship betweenonline and offline learning confirms the inhibition effect ofdeclarative memory on offline learning. More importantly,our finding extends our understanding of the competitionbetween multiple memory systems. That is, unlike previousstudies that demonstrated this competition after skills areacquired (Poldrack et al., 2001; Foerde et al., 2006; Brownand Robertson, 2007b), we demonstrated that the competitionbegins as soon as learning starts and that declarative andprocedural memorymay be identified by their distinct behavioralexpressions.






The offline learning observed within a single training session(i.e., the fast learning stage) is associated with proceduralmemory as is offline learning that takes place hours after theinitial acquisition and is responsible for memory consolidation.However, it remains unclear whether this offline learning thatallows fast initial acquisition of a motor sequence is relatedto offline learning that consolidates the memory of a newlyacquired sequence. It is possible that offline learning that drivesthe fast acquisition is a precursor of the later occurring memoryconsolidation, or theymay be the same process. To elucidate theirrelationship, further systematic investigations are needed.

A debate within the offline learning literature is whetheroffline improvement in performance after rest, referred to asreminiscence (Eysenck and Frith, 1977), results from fatigueor reactive inhibition (Rickard et al., 2008; Brawn et al., 2010)or an active learning mechanism (Eysenck and Frith, 1977;Robertson et al., 2004a). It has been suggested that offlinelearning and reactive inhibition/fatigue are usually combinedto lead to reminiscence (Eysenck, 1965), thus making itdifficult to determine if reactive inhibition/fatigue is a potentialcause of reminiscence. However, observations from our datafavor offline learning to reactive inhibition/fatigue as theprimary mechanism underlying offline improvement in RT orreminiscence observed in the SRT task. Specifically, with thesame amount of practice, only participants who performedthe probabilistic sequence slowed down their RT, while such‘‘fatigue’’ was not observed when participants performed afixed sequence. In addition, if fatigue appeared as soon asparticipants in the probabilistic sequence groups started toperform the task, it would be unlikely that their learningwould arise quickly (i.e., over four learning blocks) and toa comparable level as the participants in the fixed sequencegroups who did not exhibit fatigue. Moreover, according toEysenck and Frith (1977), reminiscence is task-specific. Forexample, reminiscence that results from reactive inhibitionor fatigue usually occurs in a task that does not involvelearning, where performance on the task is already perfectwhen an individual starts to perform the task. In contrast,reminiscence that arises from offline learning usually takesplace in a learning task. Obviously, the SRT task involvessequence learning and our data demonstrated that participantslearned the sequence. Further evidence supporting offlinelearning rather than reactive inhibition or fatigue comesfrom the observation on corrected offline learning. In theprobabilistic sequence groups, performance after the shortbreak is superior to the best performance level before thebreak. Therefore, without fully excluding the effect of reactiveinhibition/fatigue, our results favor the statement that theoffline improvement in RT is driven by offline learningrather than reactive inhibition or fatigue. Meanwhile, wesuggest that it is necessary to systematically examine thereactive inhibition or fatigue effects in future sequence learningstudies.

Although it appears that offline learning rather than reactiveinhibition or fatigue is the primary mechanism underlying theoffline improvement in RT, the cause of increased RT whenlearning a probabilistic sequence is unclear. One likely reason

is the interference of stimuli transitions with a probabilityof 0.1. It has been found that adults learned a sequence byiteratively updating the internal model of the motor sequence(Cleeremans and McClelland, 1991; Bornstein and Daw, 2012,2013; Verstynen et al., 2012) and our data provide consistentevidence that participants acquired the stimulus transitionswith probabilities of 0.3 and 0.6. However, the introduction ofstimulus transition governed by a probability of 0.1 may misleadthe updating of the internal model (i.e., transitional probabilitymatrix) and thus impair RT when the probabilistic sequence wasperformed.

In addition to the primary findings on online and offlinelearning, our results provide insights into the learning ofprobabilistic sequences. Sequence structure plays a critical role inmotor sequence learning (Curran and Keele, 1993; Jiménez et al.,1996; Bennett et al., 2007; Song et al., 2007). To date, a varietyof probabilistic sequences have been used in the SRT task, butonly a few studies have employed probabilistic sequences thatrepresent the stochastically related events of daily life, such assequences produced by a finite state grammar (Jiménez et al.,1996) or a Markov chain. We found that participants acquiredstimulus transitions with higher probabilities of 0.3 and 0.6 andthe learning of these higher stimulus transitions was comparableto that of the fixed sequence. Moreover, the facilitating effect ofPK of a sequence depends on the sequence structure, which isconsistent with previous studies (Jiménez et al., 1996; Stefaniaket al., 2008). Specifically, PK only facilitates the learning ofa simple sequence, such a fixed sequence (Curran and Keele,1993; Frensch and Miner, 1994; Curran, 1997; Destrebecqz,2004; Stefaniak et al., 2008) and not a sequence with a complexstructure.

Finally, one caveat worthy of further study is themeasurementof the amount of declarative knowledge. Both recognition andrecall tests are most widely used to examine procedural learningin the SRT task (Shanks and Johnstone, 1999; Wilkinson andShanks, 2004; Destrebecqz and Peigneux, 2005). In particular,these tests examine whether participants can explicitly recollectthe acquired sequence knowledge. However, results from therecognition tests are equivocal in the literature (Perruchet andAmorim, 1992; Willingham et al., 1993; Reed and Johnson, 1994;Shanks and Johnstone, 1999). Similarly in our study, unlike therecall tests demonstrating the common finding that probabilisticsequence learning favors more procedural memory (Jiménezet al., 1996; Song et al., 2007), the recognition tests revealsno difference in the amount of acquired declarative knowledgedespite the sequence type and PK. In addition, the recognitionscores in all four groups were not greater than chance. Given thatin the recognition test, participants were presented with sequencesegments and were asked to determine whether these segmentsare from the sequence they learned or a new sequence they didnot see in the SRT task, it is hard to know whether the chance-level score was due to the participant’s inability to explicitlyrecollect sequence knowledge or that the participant did not learnsome segments of the sequence. These two possibilities that maysimultaneously account for the chance-level recognition must beaddressed by other tests in future studies. Moreover, in our study,only four correct sequence segments were given to participants,






while there were more than 10 segments within the learnedsequence, the chance-level recognition score was caused possiblybecause some participants may learn segments other than thefour displayed in the recognition test.

In summary, we found that concurrent online and offlinelearning allows motor sequences to be acquired quickly in thefast learning stage and can be identified by their manifestationsin the progressive changes in RT. Remarkably, online and offlinelearning can be mediated by the declarative and proceduralmemory that are required to learn motor sequences. Inaddition, the modulation of online and offline learning mayreflect the competition between both memory systems during

motor sequence learning that begins in the fast learning stage.How the offline learning that drives the initial acquisition ofsequences is related to the offline learning that is responsiblefor memory consolidation occurring hours after the initialacquisition remains to be investigated.

AUTHOR CONTRIBUTIONS

YD, SP, IS, and JEC designed the experiment. IS performed theexperiment. YD and SP analyzed the data and prepared all tablesand figures. YD, SP, and JEC wrote the manuscript. All authorsreviewed the manuscript.

REFERENCES

Albouy, G., Ruby, P., Phillips, C., Luxen, A., Peigneux, P., and Maquet, P.(2006). Implicit oculomotor sequence learning in humans: time course ofoffline processing. Brain Res. 1090, 163–171. doi: 10.1016/j.brainres.2006.03.076

Armstrong, T., and Bull, F. (2006). Development of the world health organizationglobal physical activity questionnaire (GPAQ). J. Public Health 14, 66–70.doi: 10.1007/s10389-006-0024-x

Bennett, I. J., Howard, J. H., and Howard, D. V. (2007). Age-related differencesin implicit learning of subtle third-order sequential structure. J. Gerontol. BPsychol. Sci. Soc. Sci. 62, P98–P103. doi: 10.1093/geronb/62.2.p98

Bornstein, A. M., and Daw, N. D. (2012). Dissociating hippocampal and striatalcontributions to sequential prediction learning. Eur. J. Neurosci. 35, 1011–1023.doi: 10.1111/j.1460-9568.2011.07920.x

Bornstein, A. M., and Daw, N. D. (2013). Cortical and hippocampal correlatesof deliberation during model-based decisions for rewards in humans. PLoSComput. Biol. 9:e1003387. doi: 10.1371/journal.pcbi.1003387

Brawn, T. P., Fenn, K. M., Nusbaum, H. C., and Margoliash, D. (2010).Consolidating the effects of waking and sleep on motor-sequence learning.J. Neurosci. 30, 13977–13982. doi: 10.1523/jneurosci.3295-10.2010

Brown, R. M., and Robertson, E. M. (2007a). Inducing motor skill improvementswith a declarative task. Nat. Neurosci. 10, 148–149. doi: 10.1038/nn1836

Brown, R. M., and Robertson, E. M. (2007b). Off-line processing: reciprocalinteractions between declarative and procedural memories. J. Neurosci. 27,10468–10475. doi: 10.1523/jneurosci.2799-07.2007

Censor, N., Sagi, D., and Cohen, L. G. (2012). Common mechanisms of humanperceptual and motor learning. Nat. Rev. Neurosci. 13, 658–664. doi: 10.1038/nrn3315

Cleeremans, A., and McClelland, J. L. (1991). Learning the structure of eventsequences. J. Exp. Psychol. Gen. 120, 235–253. doi: 10.1037/0096-3445.120.3.235

Curran, T. (1997). Effects of aging on implicit sequence learning: accounting forsequence structure and explicit knowledge. Psychol. Res. 60, 24–41. doi: 10.1007/bf00419678

Curran, T., and Keele, S. W. (1993). Attentional and nonattentional forms ofsequence learning. J. Exp. Psychol. Learn. Mem. Cogn. 19, 189–202. doi: 10.1037/0278-7393.19.1.189

Dayan, E., and Cohen, L. G. (2011). Neuroplasticity subserving motor skilllearning. Neuron 72, 443–454. doi: 10.1016/j.neuron.2011.10.008

Destrebecqz, A. (2004). The effect of explicit knowledge on sequence learning: agraded account. Psychol. Belg. 44, 217–247. doi: 10.5334/pb-44-4-217

Destrebecqz, A., and Cleeremans, A. (2001). Can sequence learning be implicit?New evidence with the process dissociation procedure. Psychon. Bull. Rev. 8,343–350. doi: 10.3758/bf03196171

Destrebecqz, A., and Peigneux, P. (2005). Methods for studying unconsciouslearning. Prog. Brain Res. 150, 69–80. doi: 10.1016/s0079-6123(05)50006-2

Eysenck, H. J. (1965). A three-factor theory of reminiscence. Br. J. Psychol. 56,163–182. doi: 10.1111/j.2044-8295.1965.tb00956.x

Eysenck, H. J., and Frith, C. D. (1977). Reminiscence, Motivation, and Personality.London: Plenum.

Foerde, K., Knowlton, B. J., and Poldrack, R. A. (2006). Modulation of competingmemory systems by distraction. Proc. Natl. Acad. Sci. U S A 103, 11778–11783.doi: 10.1073/pnas.0602659103

Frensch, P. A., and Miner, C. S. (1994). Effects of presentation rate and individualdifferences in short-term memory capacity on an indirect measure of seriallearning.Mem. Cognit. 22, 95–110. doi: 10.3758/BF03202765

Honda, M., Deiber, M. P., Ibáñez, V., Pascual-Leone, A., Zhuang, P., andHallett, M. (1998). Dynamic cortical involvement in implicit and explicit motorsequence learning. A PET study. Brain 121, 2159–2173. doi: 10.1093/brain/121.11.2159

Hotermans, C., Peigneux, P., de Noordhout, A. M., Moonen, G., and Maquet, P.(2008). Repetitive transcranial magnetic stimulation over the primary motorcortex disrupts early boost but not delayed gains in performance in motorsequence learning. Eur. J. Neurosci. 28, 1216–1221. doi: 10.1111/j.1460-9568.2008.06421.x

Hotermans, C., Peigneux, P., Maertens de Noordhout, A., Moonen, G., andMaquet, P. (2006). Early boost and slow consolidation in motor skill learning.Learn. Mem. 13, 580–583. doi: 10.1101/lm.239406

Howard, J. H., Howard, D. V., Dennis, N. A., Yankovich, H., and Vaidya, C. J.(2004). Implicit spatial contextual learning in healthy aging. Neuropsychology18, 124–134. doi: 10.1037/0894-4105.18.1.124

Hunt, R. H., and Aslin, R. N. (2001). Statistical learning in a serial reaction timetask: access to separable statistical cues by individual learners. J. Exp. Psychol.Gen. 130, 658–680. doi: 10.1037/0096-3445.130.4.658

Jiménez, L., Méndez, C., and Cleeremans, A. (1996). Comparing direct andindirect measures of sequence learning. J. Exp. Psychol. Learn. Mem. Cogn. 22,948–969. doi: 10.1037/0278-7393.22.4.948

Karni, A., Meyer, G., Rey-Hipolito, C., Jezzard, P., Adams, M. M., Turner, R., et al.(1998). The acquisition of skilled motor performance: fast and slow experience-driven changes in primary motor cortex. Proc. Natl. Acad. Sci. U S A 95,861–868. doi: 10.1073/pnas.95.3.861

Meulemans, T., Van der Linden, M., and Perruchet, P. (1998). Implicit sequencelearning in children. J. Exp. Child Psychol. 69, 199–221. doi: 10.1006/jecp.1998.2442

Nettersheim, A., Hallschmid, M., Born, J., and Diekelmann, S. (2015). The role ofsleep in motor sequence consolidation: stabilization rather than enhancement.J. Neurosci. 35, 6696–6702. doi: 10.1523/JNEUROSCI.1236-14.2015

Nissen, M. J., and Bullemer, P. (1987). Attentional requirements of learning:evidence from performance measures. Cogn. Psychol. 19, 1–32. doi: 10.1016/0010-0285(87)90002-8

Oldfield, R. C. (1971). The assessment and analysis of handedness: theEdinburgh inventory. Neuropsychologia 9, 97–113. doi: 10.1016/0028-3932(71)90067-4

Perruchet, P., and Amorim, M. A. (1992). Conscious knowledge and changesin performance in sequence learning–evidence against dissociation. J. Exp.Psychol. Learn. Mem. Cogn. 18, 785–800. doi: 10.1037/0278-7393.18.4.785

Perruchet, P., Bigand, E., and BenoitGouin, F. (1997). The emergence of explicitknowledge during the early phase of learning in sequential reaction time tasks.Psychol. Res. 60, 4–13. doi: 10.1007/bf00419676






Poldrack, R. A., Clark, J., Paré-Blagoev, E. J., Shohamy, D., Moyano, J. C.,Myers, C., et al. (2001). Interactivememory systems in the human brain.Nature414, 546–550. doi: 10.1038/35107080

Ratcliff, R. (1993). Methods for dealing with reaction time outliers. Psychol. Bull.114, 510–532. doi: 10.1037/0033-2909.114.3.510

Reber, P. J., and Squire, L. R. (1994). Parallel brain systems for learning with andwithout awareness. Learn. Mem. 1, 217–229.

Reed, J., and Johnson, P. (1994). Assessing implicit learning with indirect tests:determining what is learned about sequence structure. J. Exp. Psychol. Learn.Mem. Cogn. 20, 585–594. doi: 10.1037/0278-7393.20.3.585

Rickard, T. C., Cai, D. J., Rieth, C. A., Jones, J., and Ard, M. C. (2008). Sleep doesnot enhance motor sequence learning. J. Exp. Psychol. Learn. Mem. Cogn. 34,834–842. doi: 10.1037/0278-7393.34.4.834

Robertson, E. M. (2007). The serial reaction time task: implicit motorskill learning? J. Neurosci. 27, 10073–10075. doi: 10.1523/jneurosci.2747-07.2007

Robertson, E. M., Pascual-Leone, A., and Miall, R. C. (2004a). Current conceptsin procedural consolidation. Nat. Rev. Neurosci. 5, 576–582. doi: 10.1038/nrn1426

Robertson, E. M., Pascual-Leone, A., and Press, D. Z. (2004b). Awareness modifiesthe skill-learning benefits of sleep. Curr. Biol. 14, 208–212. doi: 10.1016/j.cub.2004.01.027

Robertson, E. M., Press, D. Z., and Pascual-Leone, A. (2005). Off-line learning andthe primary motor cortex. J. Neurosci. 25, 6372–6378. doi: 10.1523/jneurosci.1851-05.2005

Schmitz, R., Schabus,M., Perrin, F., Luxen, A.,Maquet, P., and Peigneux, P. (2009).Recurrent boosting effects of short inactivity delays on performance: an ERPsstudy. BMC Res. Notes 2:170. doi: 10.1186/1756-0500-2-170

Schvaneveldt, R. W., and Gomez, R. L. (1998). Attention and probabilisticsequence learning. Psychol. Res. 61, 175–190. doi: 10.1007/s004260050023

Shanks, D. R., and Johnstone, T. (1999). Evaluating the relationship betweenexplicit and implicit knowledge in a sequential reaction time task. J. Exp.Psychol. Learn. Mem. Cogn. 25, 1435–1451. doi: 10.1037/0278-7393.25.6.1435

Song, S. B., Howard, J. H., and Howard, D. V. (2007). Implicit probabilisticsequence learning is independent of explicit awareness. Learn. Mem. 14,167–176. doi: 10.1101/lm.437407

Stefaniak, N., Willems, S., Adam, S., and Meulemans, T. (2008). What is theimpact of the explicit knowledge of sequence regularities on both deterministicand probabilistic serial reaction time task performance? Mem. Cognit. 36,1283–1298. doi: 10.3758/MC.36.7.1283

Verstynen, T., Phillips, J., Braun, E., Workman, B., Schunn, C., and Schneider, W.(2012). Dynamic sensorimotor planning during long-term sequence learning:the role of variability, response chunking and planning errors. PLoS One7:e47336. doi: 10.1371/journal.pone.0047336

Walker, M. P., Brakefield, T., Morgan, A., Hobson, J. A., and Stickgold, R. (2002).Practice with sleepmakes perfect: sleep-dependentmotor skill learning.Neuron35, 205–211. doi: 10.1016/S0896-6273(02)00746-8

Whelan, R. (2008). Effective analysis of reaction time data. Psychol. Rec. 58,475–482.

Wilkinson, L., and Shanks, D. R. (2004). Intentional control and implicit sequencelearning. J. Exp. Psychol. Learn. Mem. Cogn. 30, 354–369. doi: 10.1037/0278-7393.30.2.354

Willingham, D. B., and Goedert-Eschmann, K. (1999). The relation betweenimplicit and explicit learning: evidence for parallel development. Psychol. Sci.10, 531–534. doi: 10.1111/1467-9280.00201

Willingham, D. B., Greeley, T., and Bardone, A. M. (1993). Dissociation in a serialresponse time task using a recognition measure: comment on Perruchet andAmorim (1992). J. Exp. Psychol. Learn. Mem. Cogn. 19, 1424–1430. doi: 10.1037/0278-7393.19.6.1424

Willingham, D. B., Nissen, M. J., and Bullemer, P. (1989). On the developmentof procedural knowledge. J. Exp. Psychol. Learn. Mem. Cogn. 15, 1047–1060.doi: 10.1037/0278-7393.15.6.1047

Conflict of Interest Statement: The authors declare that the research wasconducted in the absence of any commercial or financial relationships that couldbe construed as a potential conflict of interest.

Copyright © 2016 Du, Prashad, Schoenbrun and Clark. This is an open-access articledistributed under the terms of the Creative Commons Attribution License (CC BY).The use, distribution and reproduction in other forums is permitted, provided theoriginal author(s) or licensor are credited and that the original publication in thisjournal is cited, in accordance with accepted academic practice. No use, distributionor reproduction is permitted which does not comply with these terms.


http://creativecommons.org/licenses/by/4.0/




probabilistic motor sequence yields greater offline and

Documents