bayesiantransferinacomplexspatiallocalizationtaskcommunity.dur.ac.uk/marko.nardini/reprints/kiryakova_jov... ·...

Journal of Vision (2020) 20(6):17, 1–19 1

Bayesian transfer in a complex spatial localization task

Reneta K. KiryakovaDepartment of Psychology, Durham University,

Durham, UK

Stacey AstonDepartment of Psychology, Durham University,

Durham, UK

Ulrik R. BeierholmDepartment of Psychology, Durham University,

Durham, UK

Marko NardiniDepartment of Psychology, Durham University,

Durham, UK

Prior knowledge can help observers in varioussituations. Adults can simultaneously learn two locationpriors and integrate these with sensory information tolocate hidden objects. Importantly, observers weightprior and sensory (likelihood) information differentlydepending on their respective reliabilities, in line withprinciples of Bayesian inference. Yet, there is limitedevidence that observers actually perform Bayesianinference, rather than a heuristic, such as forming alook-up table. To distinguish these possibilities, we askwhether previously learned priors will be immediatelyintegrated with a new, untrained likelihood. If observersuse Bayesian principles, they should immediately putless weight on the new, less reliable, likelihood(“Bayesian transfer”). In an initial experiment, observersestimated the position of a hidden target, drawn fromone of two distinct distributions, using sensory and priorinformation. The sensory cue consisted of dots drawnfrom a Gaussian distribution centered on the truelocation with either low, medium, or high variance; thelatter introduced after block three of five to test forevidence of Bayesian transfer. Observers did not weightthe cue (relative to the prior) significantly less in the highcompared to medium variance condition, counter toBayesian predictions. However, when explicitly informedof the different prior variabilities, observers placed lessweight on the new high variance likelihood (“Bayesiantransfer”), yet, substantially diverged from ideal. Muchof this divergence can be captured by a model thatweights sensory information, according only to internalnoise in using the cue. These results emphasize thelimits of Bayesian models in complex tasks.

Imagine that you are trying to give your cat a bath,but as soon as it sees the bathtub, it gets scared andruns away to the garden (Kording et al., 2007; Vilares& Kording, 2011). So, you are walking around your

garden, trying to figure out where your cat has hidden,and you hear a “meow” (auditory cue). This perceptualcue is useful but not perfectly reliable and will not allowyou to pinpoint exactly the cat’s position. However,from previous experience, you may have learned thatyour cat often hides in the bushes, furthest from thepond (priors). The uncertainty in the two pieces ofinformation that you have (the auditory cue andthe prior information) allow them to be expressedas probability distributions over location and theoptimal strategy for estimating the cat’s location is tointegrate the sensory and prior information accordingto the rules of Bayesian Decision Theory (BDT).Recent studies show that people behave as if theydeal with uncertainty in this way, for example, whenestimating the position of a hidden target (Berniker,Voss, & Kording, 2010; Körding & Wolpert, 2004;Tassinari, Hudson, & Landy, 2006; Vilares, Howard,Fernandes, Gottfried, & Kording, 2012), direction ofmotion (Chalk, Seitz, & Series, 2010), speed (Stocker &Simoncelli, 2006; Weiss, Simoncelli, & Adelson, 2002),or the duration of a time interval (Acerbi, Wolpert, &Vijayakumar, 2012; Ahrens & Sahani, 2011; Jazayeri &Shadlen, 2010; Miyazaki, Nozaki, & Nakajima, 2012).In all of these studies, human observers integratedknowledge of the statistical structure of the experiment(acquired from feedback in previous trials) with sensoryinformation, taking a weighted average according totheir relative reliabilities in order to maximize his orher score on the task (Ma, 2012). However, otherstudies report suboptimal behavior, finding that eventhough observers take into account the uncertainty ofthe current and prior information, the weights do notmatch those of an ideal Bayesian observer (Bowers &Davis, 2012; Jones & Love, 2011; Rahnev & Denison,2018). The fact that human performance ranges from

Citation: Kiryakova, R. K., Aston, S., Beierholm, U. R., & Nardini, M. (2020). Bayesian transfer in a complex spatial localization task.Journal of Vision, 20(6):17, 1–19, https://doi.org/10.1167/jov.20.6.17.

https://doi.org/10.1167/jov.20.6.17 Received June 13, 2019; published June 24, 2020 ISSN 1534-7362 Copyright 2020 The Authors

This work is licensed under a Creative Commons Attribution 4.0 International License.Downloaded from jov.arvojournals.org on 07/07/2020

mailto:[email protected]




https://doi.org/10.1167/jov.20.6.17

http://creativecommons.org/licenses/by/4.0/

Journal of Vision (2020) 20(6):17, 1–19 Kiryakova, Aston, Beierholm, & Nardini 2

close-to-optimal to largely suboptimal suggests thatBayesian models may describe behavior well in somecases, but not in others. Understanding when BDTcan and cannot provide an accurate model of humanbehavior is an important step toward understanding thecomputations and approximations used by the brain tosupport adaptive behavior.

One factor that may influence how close performanceis to BDT (“optimal”) predictions is task complexity.For example, Bejjanki, Knill, and Aslin (2016) askedobservers to estimate the position of an unknowntarget (a bucket at a fairground), whose true locationwas randomly drawn from one of two Gaussiandistributions, with different means and variances(priors). On each trial, eight dots were drawn from aGaussian distribution centered on the true location witha low, medium, or high variance to form a dot cloud thatserved as a noisy cue to target location (likelihoods).To successfully estimate the position of the target,subjects could use both the likelihood, obtained fromthe displayed dots, and the prior, obtained from thedistribution of previous target positions that theycould learn from the trial-by-trial feedback. The studyfound that subjects adjusted their responses accordingto the reliability of sensory and prior information,giving more weight to the centroid of the dot cloud(likelihood) when the variance of the prior was highand the variance of the likelihood was low; a signatureof probabilistic inference (Ma, 2012). More generally,these results are also in agreement with previouslydescribed work, which used a single prior change (e.g.Berniker et al., 2010; Vilares et al., 2012). However,unlike in the studies using only a single prior change,the weight placed on the likelihood differed from thatof the ideal-Bayes observer whenever the likelihooduncertainty was medium or high. The magnitude ofthe difference varied with both likelihood and prioruncertainty. As the likelihood became more uncertain,the difference from optimal increased, participantsplacing more weight on the likelihood than optimal.In addition, the difference from optimal was greater inthe narrow prior condition. In a follow-up experiment,participants experienced one prior distribution only,with double the amount of trials used in the originalstudy, finding that subjects’ weights on the likelihoodsapproach optimal with increasing task exposure,suggesting more time is required to accurately learn thevariances of the prior distributions and that learningis disrupted when trying to learn two distributionssimultaneously (Bejjanki et al., 2016).

Even in cases when likelihood-weighting mightmatch the prediction of an ideal-Bayes observer,Maloney and Mamassian (2009) noted that such“optimal” or “near-optimal” performance alone is notenough to show that the brain is following Bayesianprinciples. Maloney andMamassian (2009) showed thata reinforcement-learning model that learns a mapping

from stimulus to response (learning a separate look-uptable for each prior and likelihood pairing in the typesof tasks discussed here) will also appear to optimallyweight prior and likelihood information withoutlearning the individual probability distributions.

Maloney and Mamassian (2009) suggested that thetwo models may be distinguished by asking whethersubjects are able to immediately transfer probabilisticinformation from one condition to another (hereafterknown as Bayesian transfer). These transfer criteriaare a strong test for use of Bayesian principles becausethey make very different predictions for how theobserver will behave when presented with a new levelof sensory noise halfway through the task. If peoplefollow Bayesian principles, we would expect them toimmediately adapt to the new sensory uncertainty,and integrate it with an already-learned prior, withoutany need for feedback-driven learning. On the otherhand, the reinforcement-learner would require acertain amount of exposure to the new likelihoodand prior pairing (with feedback) in order to forma look-up table that could then lead to optimalperformance.

To our knowledge, only one study has tested Bayesiantransfer in the context of sensorimotor learning. Satoand Kording (2014) tested the ability of participantsto generalize a newly learned prior to a previouslylearned likelihood. In their task, Sato and Kording(2014) first trained participants to complete the taskwhen only a single Gaussian prior was present (eithernarrow or wide) that could be paired with either alow or high uncertainty likelihood by giving feedbackon every trial. After 400 trials, the prior switchedto the other level of uncertainty (narrow to wideor wide to narrow) and for the following 200 trials,participants saw the new prior paired only with one ofthe likelihoods (either low or high) and continued toreceive feedback. In the second part of the experiment,subjects still saw the second prior variability, butnow with the first likelihood again, which they hadso far only seen paired with the first prior. They didnot provide any feedback in this part of the task toexamine how subjects transferred their knowledge ofthe prior to the new likelihood. The weight placed onthe likelihood in the newly reintroduced likelihoodcondition was immediately different to the weightplaced on the same likelihood before the change in theprior. In other words, participants’ behavior in thislikelihood condition changed dependent upon prioruncertainty without any explicit training with this priorand likelihood pairing. This is evidence of Bayesiantransfer and, hence, that participants solved the taskusing Bayesian principles — representing probabilities— rather than a simpler strategy, such as a look-uptable learned by reinforcement.

Whether the same will hold in more complexscenarios is unclear. Indeed, it has been repeatedly

Downloaded from jov.arvojournals.org on 07/07/2020


pointed out that exact Bayesian computations demandconsiderable computational resources (e.g. workingmemory and attention) such that the brain might notbe able to perform these computations in more complextasks and will instead resort to heuristics (Beck, Ma,Pitkow, Latham, & Pouget, 2012; Gardner, 2019). Wehave already seen that performance in more complextasks is far from optimal (e.g. Bejjanki et al., 2016),suggesting that there are limits to humans’ ability tolearn and optimally integrate prior distributions withsensory information when tasks become more complex.Establishing the limits of BDT as a model of humanbehavior will inform models of information processingin the human brain.

Here, we ask whether people will show Bayesiantransfer in a complex situation with multiple priorsand likelihood variances, similar to Bejjanki et al.(2016). We report three experiments in which a targetis sampled from one of two possible prior distributions(with different means and different variances) and thencued with one of three possible likelihood variances(with the variance itself also displayed). Likelihoodand prior variances were identical to those used inBejjanki et al. (2016) in terms of visual angle, in orderto match the true (objective) reliabilities of the cueand prior across the studies. The first two experimentstested for Bayesian transfer by only introducing thehigh likelihood variance in blocks four and five ofthe task. The only difference was that, in the secondexperiment, participants were explicitly told that theprior variances differed, to test whether this wouldpromote closer-to-optimal performance. The lastexperiment was used to check whether removing theadditional burden of transfer allows participantsto learn the complex environment correctly bypresenting all likelihood conditions from the start ofthe experiment — a replication of Bejjanki et al. (2016).

To summarize, in the first experiment, we foundthat observers did not show evidence for Bayesiantransfer. When a new high variance likelihood wasintroduced in blocks four and five, they did notweight it less than the familiar medium variancelikelihood. This is at odds with the idea that observersperform full Bayesian inference, combining prior andlikelihood based on relative variances, and thus doesnot provide evidence for Bayesian transfer. In thesecond experiment, observers did show evidence fortransfer, weighting the newly-introduced high variancelikelihood significantly less than the medium variancelikelihood. However, in both experiments, the weightsplaced on the medium and high variance likelihoodswere much higher than optimal. These weights remainedsuboptimal in the final experiment where all likelihoodvariances were present from the start of the task.These results extend our knowledge of how potentiallyBayesian perceptual processes function in complexenvironments.

Experiment 1: Testing transfer to anew level of likelihood variance

In the first experiment, we tested whether Bayesiantransfer would occur in a complex environment withtwo priors, similar to the one used by Bejjanki et al.(2016). We trained participants on a spatial localizationtask with two likelihood variances and two priordistributions (with different means and variances).In initial training, they were exposed to all fourcombinations (trials interleaved), with feedback. If, likeparticipants in Bejjanki et al. (2016), they weighted thelikelihood and the prior differently across conditionsin line with their differing reliabilities, this wouldshow that they had learned and were using the priors.However, such reliability weighting could either be donevia Bayesian inference — representing probabilities —or via a simpler strategy akin to learning a look-uptable (Maloney & Mamassian, 2009). To distinguishthese possibilities, after the training trials, we tested for“Bayesian transfer” by adding a new higher-variancelikelihood distribution to the task. If participants dealwith this newly introduced likelihood in a Bayesianmanner, they should immediately rely less on this newlikelihood information than they did on the likelihoodsin previously trained conditions. Alternatively, if theirinitial learning is more rote in nature (i.e. more likea look-up table), participants would begin to placea different weight on the new likelihood only afterextensive training with feedback.

Methods

OverviewSubjects performed a sensorimotor decision making

task on a computer monitor where they estimatedthe horizontal location of a hidden octopus. Thetrue location was sampled from one of two distinctGaussian distributions that differed in mean andvariance (narrow or wide priors). On each trial, therelevant prior distribution was indicated by the colorof the likelihood information — eight dots that weredescribed to the participant as the “tentacles” of theoctopus. The horizontal locations of the eight dotswere sampled from a Gaussian distribution centeredon the true location that had low, medium, or highvariability (the likelihood). To estimate the octopus’position, participants could use (although this wasnever explicitly mentioned) both the likelihood andprior information, with the subjects able to learn thelatter via trial-to-trial feedback. Participants completedfive blocks of trials. Crucially, in blocks one to three,only the low and medium likelihood variances werepaired with the narrow or wide priors. The high



likelihood condition was only introduced in blocks fourand five to test for evidence of Bayesian transfer.

ParticipantsParticipants were recruited from the Durham

Psychology department participant pool, DurhamUniversity newsletter, and by word-of-mouth. Twenty-six participants were recruited in total (13 women, meanage: 20.1 years, age range: 18-30 years). All participantshad normal or corrected-to-normal visual acuity, andno history of neurological or developmental disorders.Each participant received either course credits or a cashpayment of £10 for their time.

EthicsEthical approval was received from the Durham

University Psychology Department Ethics Board. Allparticipants gave written, informed consent prior totaking part in the study.

Stimuli and apparatusStimuli were displayed on a 22-inch iiyama monitor

(1680 × 1050 pixels), viewed at a distance of 60cm, using the Psychophysics Toolbox for MATLAB(Brainard, 1997; Kleiner et al., 2007; Pelli, 1997). Thestimuli were set against a blue background (to representthe sea).

The position of the octopus was sampled from oneof two Gaussian distributions (priors): the narrow(standard deviation (SD) of σ pl = 1% of screen width)or wide (SD of σ ph = 2.5% of screen width) priors.The side of the screen associated with each prior wascounterbalanced across participants. One was always35% of the way across the screen (from left to right),and the other 70%. When the narrow prior was centeredon 35%, for example, the wide prior had a mean in theopposite side of the screen (i.e. to the right, centered on70%). When the octopus appeared on the left-hand side(drawn from the prior centered on 35%) it was white,and when it appeared on the right (drawn from theprior centered on 70%) it was black.

At the beginning of each trial, a cloud of eight dots(0.5% of screen width in diameter) appeared on thescreen. The horizontal position of each dot was drawnfrom a Gaussian distribution centered on the trueoctopus location with either low (σ ll = 0.6% screenwidth), medium (σ lm = 3% screen width), or high (σ lh= 6% screen width) SD (in the following referring to aslow, medium, and high variance likelihood conditions).The horizontal positions of the dots were scaled sothat their SD was equal to the true SD (σ ll, σ lm, or σ lh)on each trial while preserving the mean of the dots.We performed this correction so that participantswould “see” the same variability across trials for each

likelihood condition. This ensures that an observer whocomputes the reliability for the likelihood informationtrial by trial would always calculate the same valuewithin likelihood trial types. The vertical positions ofthe dots were spaced at equal intervals from the verticalcenter of the screen, with half of the dots appearingabove, and the other half below the center. The verticaldistance between each dot was fixed and equal to 1%screen width. Given that the vertical positions of thedots were fixed, only the horizontal position of thetarget was relevant. Participants estimated location onlyalong the horizontal axis by moving a vertical greenrectangle (measuring 1% of screen width in width and3% of screen width in height) left or right, making this aone-dimensional estimation task. Participants receivedfeedback in the form of a red dot (0.5% of screen widthin diameter) that represented the true target position.

The combination of two priors and three likelihoodsled to six trial types (all possible prior × likelihoodpairings). The task was split into five blocks of trialswith 300 trials per block. In the first three blocks of thetask, only four trial types were used (75 trials of eachparing), with the high likelihood condition not shownin combination with either prior. The high likelihoodcondition was introduced in blocks four and five (50trials per paring), in order to test for Bayesian transfer.Within each block, all trial types were interleaved. Thetrials were broken into runs of 20 trials. Within eachrun, the trials for each prior type were arranged suchthat an ideal learner would have an exact estimate ofthe mean and variance of the prior distributions ifevidence was accumulated over those 20 trials.

We also included prior-only trials where subjects weretold that a black/white octopus was hiding somewhereon the screen and they were instructed to find it.No sensory information was provided (no likelihoodinformation). These trials were interleaved with therest of the trials (one every nine trials for each prior),and there were 83 trials in total, for each of the priors(narrow and wide).

Procedure

Participants were instructed to estimate the positionof a “hidden” octopus, indicating their estimate byadjusting the horizontal location of a “net” (greenrectangle). Each trial started with the presentationof eight dots that remained on screen until the endof the trial (the likelihood information, describedto the participants as the tentacles of the octopus)(Figure 1A). The eight dots could have one of threelevels of uncertainty: low, medium, or high variancelikelihood trials (Figure 1B). When the level ofuncertainty was higher, the dots were more dispersed onthe screen and, therefore, were a less reliable indicator ofthe true location of the octopus. Participants used the



Figure 1. (A) Illustration of the task. Participants were asked toestimate the position of a hidden target (the “octopus”,represented as the red dot) by horizontally moving a net (greenvertical bar). At the beginning of each trial, participants weregiven noisy information about the location of the hidden targetin the form of eight dots (the likelihood). Participants thenmoved the net to the estimated location and clicked to confirmtheir response, after which the actual target location wasdisplayed. If the target was inside, or overlapped with, the net,a score was added to the participant’s score. (B) The threelikelihood variances. (C) Illustration of a Bayes-optimalobserver. A Bayesian observer would combine informationabout the prior uncertainty (learnt from the distribution ofprevious target locations) with the likelihood information on agiven trial to optimally estimate the target location.

mouse to move the net to their guessed position, usinga right click to confirm their choice (no time limit).Following a response, the true position of the octopuswas shown as a red dot on the screen. Over the courseof the experiment, the feedback served as a second cueto location because the true locations of the black andwhite octopi were drawn from different distributions.In other words, participants could learn a prior overeach octopus’ location. We provided performancefeedback on a trial-to-trial basis so that the priorscould be learned. Specifically, subjects could potentiallylearn that the two sets of octopi (black/white) weredrawn from separate Gaussian distributions centeredat different locations on the screen and with differinglevels of uncertainty (narrow and wide variance priortrials).

To keep participants engaged, we incorporatedan animation when the participant picked the rightlocation of a cartoon octopus moving into a bucketcentered at the bottom of the screen. In addition,participants would get one point added to their scoreif they “caught” the octopus. An octopus was caughtif the true octopus position overlapped with the netplacement by at least 50% of the red feedback dot’s size.

The cumulative score was displayed at the end of eachtrial. Participants completed five blocks of 300 trialseach for a total of 1500 trials. The five experimentalblocks were performed in succession with a shortbreak between each one. The experiment duration wasapproximately an hour and a half.

Data analysis

For each individual participant, we regressedestimated octopus’ position against the centroid (mean)of the cloud of dots (likelihood) on each trial. Allregression analyses were done using a least squaresprocedure (the polyfit function in MATLAB). Theslope of the fitted regression line quantifies the extentto which participants rely on the current sensoryevidence (likelihood), as opposed to prior information.A slope of one suggests that participants only uselikelihood information and a slope of zero suggeststhat participants rely only on their prior knowledge,ignoring the likelihood. A slope between zero and onesuggests that both likelihood and prior information aretaken into account, and the steeper the slope, the moreparticipants rely on the likelihood and less on the priorinformation. Accordingly, we will refer to the fittedslope values as the weight placed on the likelihood.

We also computed the weight that would begiven to the likelihood in each condition by an idealBayesian observer with perfect knowledge of the priorand likelihood distributions (see Figure 1C for anillustration). The optimal weight on the likelihood wascomputed as:

woptimal =1

σ 2l /n

1σ 2l /n + 1

σ 2p

where σ 2l is the variance of the likelihood, n is the

number of dots that indicate the likelihood (in this case,there were 8 dots), and σ 2

p is the variance of the prior.To determine the proportion of the variance in

responses that is accounted for by change in the estimatefrom the sensory cue, the coefficient of determination(R2) was calculated by linearly regressing participants’responses against each estimate participants could havetaken from the cue (i.e. arithmetic mean, robust average,median, or mid-range). This was done for the combineddata of all subjects in each experiment, across all blocksand trial types (prior and likelihood pairings). Theestimate with the highest R2 value was taken to be theestimate participants had most likely used.

Statistical differences were analyzed using repeated-measures ANOVA with a Greenhouse-Geissercorrection (Greenhouse & Geisser, 1959) of the degreesof freedom in order to correct for violations of the



Figure 2. Mean weight placed on the likelihood information, separated by block and prior width in Experiment 1. Lower valuesrepresent a greater weight on the prior. Blue is low-variance likelihood (a tight array of dots), green is medium-variance likelihood(dots somewhat spread out), red is the later-introduced high-variance likelihood (highly spread out dots). Dashed lines showoptimal-predicted values. Error bars are +/− 1 SEM. The far right is the average over blocks.

sphericity assumption if ε ≤ 0.75 and a Huynh-Feldtcorrection otherwise.

We discarded a trial from analysis if the absoluteerror for that trial was in the top 1% of all absoluteerrors, computed separately for each prior andlikelihood pairing across all blocks and participants(this rule excluded at most 13 trials per pairing for anindividual subject).

Results and discussion for experiment 1

We first checked whether subjects took the mean asan estimate from the sensory cue, and not a heuristic,such as the robust average. In tasks similar to ours(Bejjanki et al., 2016; Chambers, Gaebler-spira, &Kording, 2018; Vilares et al., 2012), authors assume thatobservers use the mean of the dots as their best estimateof true location from the likelihood information.However, we did not explicitly tell our participantshow the eight dots that formed the likelihood weregenerated, or that the best estimate they could takefrom them was their mean position, leaving open thepossibility that observes may have taken a differentestimate from the cue than the mean (de Gardelle &Summerfield, 2011; Van Den Berg & Ma, 2012). Themean horizontal position was found to explain the

most amount of variance in participants’ responses (R2

= 0.996), relative to the robust average (R2 = 0.995),median (R2 = 0.995), or the mid-range of the dots (R2

= 0.992). This suggests that the mean of the dots is theestimate that participants take from the sensory cue.

We then examined whether the weight participantsplaced on the likelihood, relative to the prior, variedwith respect to trial type (prior/likelihood pairing) forall the trial types present from the beginning of theexperiment. Without this basic result — a replication ofthe pattern found by Bejjanki et al. (2016) — we couldnot expect them to transfer knowledge of the learnedprior distributions to the new high variance likelihoodin the later blocks. This was a qualified success: forthese trial types (blue and green bars in Figure 2),participants showed the predicted pattern, but placedmore weight on the likelihood than is optimal (comparebar heights to dashed lines in Figure 2), in line withprevious research (Bejjanki et al., 2016; Tassinari et al.,2006). We conducted a 2 (narrow versus wide varianceprior) × 2 (low versus medium variance likelihood) ×5 (block) repeated measures ANOVA with the weightgiven to the likelihood (the displayed dots) as thedependent variable. These results are shown in Table 1and summarized here. There was a main effect ofprior variance, with less weight on the likelihoodwhen the prior was narrower (p < 0.001). There was



F df, dferror p Effect size (η2p)

Likelihood 104.40 1, 25 < 0.001 0.81Prior 28.08 1, 25 < 0.001 0.53Block 7.72 4, 100 < 0.001 0.24Likelihood x prior 12.77 1, 25 0.001 0.34Likelihood x block 3.32 4, 100 0.01 0.12Prior x block 0.51 4, 100 0.21 0.06

Table 1. Results from a 2 (prior) × 2 (likelihood) × 5 (block)repeated measures ANOVA for the likelihood variances presentfrom the beginning of the task (low and medium) inExperiment 1.

also a main effect of likelihood variance (p < 0.001),where participants relied less on the medium variancelikelihood. However, there was also a significantinteraction effect of likelihood and prior (p = 0.001).When the prior was narrow, the decrease in reliance onthe likelihood was smaller as the likelihood varianceincreased (t(25) = 3.57, p = 0.001).

We found a main effect of block (p < 0.001) andan interaction between block and likelihood (p =0.014), with the medium variance likelihood weightedsignificantly differently across blocks (simple main effectof block, F (4, 100) = 5.84, p < 0.001, η2

p = 0.189,weights decrease with increasing exposure), but not thelow variance likelihood (no simple main effect of block,F (4, 100) = 1.64, p = 0.169, η2

p = 0.063). This suggeststhat participants adjusted, through practice, theirweights on the medium variance likelihood, gettingcloser to optimal.

Examination of the prior-only trials shows successfullearning of the priors. On average, subjects’ responseswere not significantly different from the prior mean inthe wide prior condition (t(25) = − 0.77, p = 0.450).They were significantly different in the narrow priorcondition (t(25) = − 2.78, p = 0.010), although the biaswas extremely small (95% confidence interval [CI]: 0.06,0.41 percent of the screen width to the left). The medianSD of responses for all subjects was 1.4% (narrow prior)and 2.5% (wide prior): almost identical to the true priorSDs of 1.3% and 2.5%, respectively.

Participants qualitatively followed the predictedoptimal pattern of reweighting: like the dashed lines(predictions) in Figure 2, actual likelihood weights(bars) were higher for the wide prior (right) than thenarrow prior (left), and higher for the low variancelikelihood (blue) than the medium variance likelihood(green). However, comparing bar heights withdashed lines (predictions) shows that quantitatively,their weights were far from optimal. Participantssystematically gave much more weight than is optimalto the likelihood when its variance was medium (seeFigure 2, green bars versus lines — p < 0.001 in allblocks for the medium likelihood when paired with


Likelihood 50.08 2, 50 < 0.001 0.67Prior 15.52 1, 25 0.001 0.38Block 1.21 1, 25 0.28 0.05Likelihood x prior 2.39 1.62, 40.41 0.10 0.09Likelihood x block 0.75 2, 50 0.48 0.03Prior x block 4.35 1, 25 0.05 0.15

Table 2. Results from a 2 (prior) × 3 (likelihood) × 2 (block)repeated measures ANOVA for all likelihood variances inExperiment 1.

either prior). This over-reliance on the likelihood is inline with previous studies (e.g. Bejjanki et al., 2016),although stronger in the present study. Participants,therefore, accounted for changes in the probabilitiesinvolved in the task (e.g. weighted the likelihood lesswhen it was more variable), but did not perform exactlyas predicted by the optimal strategy.

Having found that participants’ performance wasin line with the predicted patterns, we could thenask if they would generalize their knowledge tothe new high likelihood trials added in blocks fourand five (“Bayesian transfer”), as predicted for anobserver following Bayesian principles. This shouldlead immediately to a lower weight for the new highvariance likelihood than the familiar medium variancelikelihood. By contrast, lack of a significant differencein weights between the medium and high likelihoodtrial types would suggest that the observer is employingan alternative strategy, such as simply learning alook-up table. To test this, we performed a 2 (prior) ×3 (likelihood) × 2 (block) repeated measures ANOVA(summarize only blocks four and five — those with alllikelihoods present). These results are shown in Table 2and summarized here. As above, we found a main effectof likelihood, participants placing less weight on thelikelihood as it became more uncertain (p < 0.001).However, post hoc analyses showed that the weightplaced on the high likelihood was not significantly lowerthan the weight placed on the medium likelihood (p =0.103). Only the comparison of the weights placed onthe likelihood in low and high variance trial types wassignificant (p < 0.001). Moreover, there was no maineffect of block (p = 0.28), nor an interaction betweenblock and likelihood (p = 0.48), suggesting that theweight placed on the newly introduced likelihoodvariance did not decrease with increasing exposure.

Finally, we compared mean weights in block fiveagainst the optimal Bayesian values for each priorand likelihood pairing. In the low variance likelihoodtrials, we did not observe significant deviation from theBayesian prediction, irrespective of prior (low variancelikelihood, narrow prior: t(25) = .784, p = 0.440); lowvariance likelihood, wide prior: t(25) = − 1.12, p =



0.270). Subjects’ weights differed significantly fromoptimal in all other conditions (p < 0.001 in all cases).

Overall, our results do not exactly match thepredictions of a Bayesian observer because we findonly weak evidence of Bayesian transfer. Specifically,although we find a main effect of likelihood, the weighton the high variance likelihood is not significantlydifferent to that placed on the medium variancelikelihood (although the change is in the predicteddirection). That said, our results are not simply moreconsistent with a rote process, because the weightplaced on the high likelihood does not decrease withincreasing exposure (no interaction between likelihoodand block).

Our results point mostly away from a simple varianceweighted Bayesian model being a good model of humanbehavior in this particular task. The correct patternof weights was present, but evidence of transfer wasweak. Participants were also significantly suboptimal,overweighting the likelihood whenever its variance wasmedium or high. Previous studies have also found thatobservers give more weight to the sensory cue than isoptimal (e.g. Bejjanki et al., 2016); even so, the level ofsuboptimality that we observe here is still drasticallyhigher, compared to previous reports. However,Sato and Kording (2014) found better, near-optimalperformance in those participants who were told thatthe sensory information can have one of two levels ofvariance, and that the variance will sometimes change,compared to those who were not provided with thisinformation. We, therefore, reasoned that if observersare given additional information about the structureand statistics of the task (e.g. that the variances of theprior distributions are different), the weight they giveto the sensory cue may move closer to optimal. If wefind weights closer to optimal, we may be better ableto detect whether transfer had taken place becausethe effect size of a change in the likelihood would bebigger. In fact, we wonder whether the size of thiseffect could be an important factor behind the lackof significant differences observed in Experiment 1(i.e. that the effect size of the change from medium tohigh likelihood was too small for our statistical analysisto reliably detect). In view of this, we set out to testwhether additional instructions will lead to weightingof likelihood and prior information that is closer tooptimal.

Experiment 2: Additionalinstructions about prior variance

Experiment 2 was identical to Experiment 1 exceptfor a change in instructions. In this experiment,subjects were explicitly (albeit indirectly) informed of

the different variances of the prior. We hypothesizedthat giving participants additional information aboutthe model structure of the task will move weightscloser to optimal and make any transfer effects morepronounced.

Methods

Twelve participants (8 women, mean age: 20.3 years,age range: 19-22 years) participated in Experiment 2. Allparticipants had normal or corrected-to-normal visualacuity, no history of neurological or developmentaldisorders, and had not taken part in Experiment 1.Each participant received either course credits or cashcompensation for their time.

The experimental set had the same layout as the mainexperiment, with the following difference: in additionto the previously described instructions, subjects inthis version of the task were told that “it is importantto remember that one of the octopuses tends to stayin a particular area, whereas the other one movesquite a bit!” (i.e. they were indirectly informed that thevariances of the priors were different) (see Appendix Ain the Supplementary Material for full instructions).

Results and discussion for experiment 2

Similarly to what we saw in Experiment 1, the meanand robust average of the dots explained the sameamount of the variance in participants’ responses (R2

= 0.991 for both), followed by the median (R2 = 0.990)and the mid-range (R2 = 0.989). We, thus, proceed withthe mean as the estimate from the likelihood.

Figure 3 shows that the pattern of results wasqualitatively similar to those of Experiment 1(see Figure 2). A 2 (prior) × 2 (likelihood) × 5 (block)repeated measures ANOVA (analyzing only the lowand medium likelihood trials) revealed that subjectsplaced less weight on the likelihood as its uncertaintyincreased (main effect of likelihood, p < 0.001) and asthe prior uncertainty decreased (main effect of prior, p= 0.002). However, unlike in Experiment 1, there wasno significant interaction of these factors (p = 0.123)(see Table 3).

We found a main effect of block (p=0.02)and an interaction between block and likelihood(p=0.01), with participants weighting the likelihoodsignificantly less with increasing task exposure(regardless of prior) when its variance was medium(F (2.21, 25.34) = 3.81, p = 0.03, η2

p = 0.257, with aGreenhouse-Geisser correction), but not when it waslow (F (4, 44) = 0.70, p = 0.60, η2

p = 0.060).As before, we analyzed subjects’ responses in

the prior-only trials, finding a good quantitativeagreement with the true prior mean (narrow prior:



Figure 3. Mean weight placed on the likelihood information in each block of Experiment 2. Blue is the low-variance likelihood, green isthe medium-variance likelihood, red is the high-variance likelihood. Dashed lines show optimal values. Error bars are +/− 1 SEM. Thefar right is the average over blocks.


Likelihood 46.14 1, 11 < 0.001 0.81Prior 17.28 1, 11 0.002 0.61Block 3.15 4, 44 0.02 0.22Likelihood x prior 2.80 1, 11 0.12 0.20Likelihood x block 3.73 4, 44 0.01 0.25Prior x block 1.32 2.07, 22.77 0.29 0.11

Table 3. Results from a 2 (prior) × 2 (likelihood) × 5 (block)repeated measures ANOVA for the likelihood variances presentfrom the beginning (low and medium) in Experiment 2.

t(11) = − 0.002, p = 0.999; wide prior: t(11) = 0.35,p = 0.734). The median SD of responses was alsoremarkably similar to the true prior SDs (narrow prior:1.6% vs. 1.3% in screen units; wide prior: 2.5% forboth).

Again, subjects’ overall performance was suboptimal(as can be seen by comparing the height of the barsagainst the dashed lines — the optimal predictions —in Figure 3). Subjects’ placed more weight on both themedium and high variance likelihoods than is optimal(p < 0.001 in both cases, for both priors). However, itis worth noting that the weights placed on the mediumand high likelihoods are closer to optimal than they


Likelihood 37.98 2, 22 < 0.001 0.78Prior 18.36 1, 11 0.001 0.63Block .23 1, 11 0.64 0.02Likelihood x prior 1.80 2, 22 0.19 0.14Likelihood x block 2.09 2, 22 0.15 0.16Prior x block .17 1, 11 0.69 0.02


were in Experiment 1 (compare bar heights in Figures 2and 3).

Finally, we tested for transfer to the newly-introducedhigh likelihood in blocks four and five. We conducted a2 (prior) × 3 (likelihood) × 2 (block) repeated measuresANOVA (analyzing only blocks four and five with alllikelihoods present). These results are shown in Table 4and summarized here. There was a main effect oflikelihood, with less weight placed on the likelihoodas it became more uncertain (p < 0.001). Unlike inExperiment 1, post hoc analysis showed that the weightplaced on the high likelihood was significantly lowerthan the weight placed on the medium likelihood (p= 0.034). The weights placed on the likelihood in the



low variance trial type were significantly lower thanthose in the medium and high variance trial types (p <0.001 for both). Moreover, there was no main effect ofblock (p = 0.64), or an interaction effect of block andlikelihood (p = 0.15), meaning that the weight placedon the newly added likelihood information did not varywith increasing exposure.

Again, we find a significant difference betweensubjects’ weights (in block 5) and optimal predictionswhen the likelihood variance was medium or high(p < 0.001 in both cases), but not when it was low,irrespective of prior variance (low likelihood, narrowprior: t(11) = .120, p = 0.907); low likelihood, wideprior: t(11) = − 1.29, p = 0.163).

In line with our prediction of transfer, here, we showthat the observers put lower weight on the high variancelikelihood than the medium variance likelihood theyhave experienced before. This is strengthened by the factthat participants’ weights did not change significantlywith increasing exposure across blocks four and five.

To check more directly for differences due toexperimental instructions, we compared subjects’performance in the last two blocks across thetwo experiments. We ran a 2 (instructions) ×2 (prior) × 3 (likelihood) × 2 (block) mixedANOVA. We found a main effect of instructions(F (1, 36) = 6.78, p = 0.013, η2

p = 0.159) with subjectsweighting the likelihood significantly less withexplicit instructions (Experiment 2) (i.e. closerto the optimal weightings). We also found aninteraction between instructions and likelihood(F (1, 36) = 4.79, p = 0.011, η2

p = 0.118), indicatingthat the main effect of instructions is due to asignificant decrease in the weight placed on the high,relative to the medium likelihood in the explicitinstructions (Experiment 2) task, but not the originaltask (Experiment 1).

These results show that adding extra instructions tothe task that make the participant aware of a changein uncertainty between the two priors has an effect.The weights placed on the likelihood moved closerto optimal, and the transfer criterion was met, whichsuggests that, perhaps, observers are more likely toadopt a Bayes-optimal strategy when more explicitexpectations about the correct model structure ofthe task are set. However, even with the additionalinstructions, the weight given to the sensory cue wasstill systematically higher than the “optimal” weight.Arguably, expecting people to perform optimally israther unrealistic, as it presumes that the observerperfectly knows the environmental statistics. However,Bejjanki et al. (2016) found performance much closerto optimal than what we have seen in either of ourprevious experiments. The major difference betweentheir experiment and ours’ is the fact that Bejjanki et al.(2016) presented all likelihood variances from the startof the task. Therefore, unlike Experiments 1 and 2,

which were designed in order to provide some evidenceof transfer, Experiment 3 sought to test whethersubjects’ weights would move closer to optimal if wepresent all likelihood variances from the beginning, ina more direct replication of Bejjanki et al. (2016). Thelikelihood and prior variance parameters were identicalto those used in Bejjanki et al. (2016), and we used asimilar number of trials per prior and likelihood pairing(250 vs. 200 trials in Bejjanki et al. (2016)).

Experiment 3: All likelihoods fromthe beginning

Experiment 3 was identical to Experiment 1 (lackingthe extra instructions of Experiment 2) except that alllikelihood variances were included from the beginningof the task. The participants experienced all six trialtypes in every block.

Methods

Twelve participants (10 women, mean age: 22.6 years,age range: 19-30 years) took part in Experiment 3. Allparticipants had normal or corrected-to-normal visualacuity, no history of neurological or developmentaldisorders, and had not taken part in Experiment 1 and2. Each participant received either course credits orcash compensation for their time.

The stimuli and task were identical to those describedfor Experiment 1, except that all likelihood conditions(low, medium, and high) were now present from thebeginning (50 trials of each likelihood/ prior pairinginterleaved in the same block).

Results and discussion of experiment 3

Again, the mean position of the dots explained themost amount of variance in participants’ responses (R2

= 0.990). The amount of variance explained decreasedfor the robust average (R2 = 0.989), median (R2 =0.988), and the mid-range of the dots (R2 = 0.985). We,thus, proceed with the mean as the estimate from thelikelihood.

Figure 4 shows a similar pattern of results toExperiments 1 and 2. Again, a 2 (prior) × 3 (likelihood)× 5 (block) repeated measures ANOVA shows thatthe likelihood information was weighted less as itbecame more unreliable (main effect of likelihood,p < 0.001). Specifically, subjects placed significantlymore weight on the low likelihood than on themedium (p = 0.001) or high likelihood (p < 0.001),and more weight on the medium likelihood than the



Figure 4. Mean weight placed on the likelihood information in each block of Experiment 3. Blue is the low-variance likelihood, green isthe medium-variance likelihood, red is the high-variance likelihood. Dashed lines show optimal values. Error bars are +/− 1 SEM. Thefar right is the average over blocks.


Likelihood 29.90 2, 22 < 0.001 0.73Prior 2.74 1, 11 0.13 0.20Block .28 4, 44 0.89 0.03Likelihood x prior 2.67 2, 22 0.09 0.19Likelihood x block .52 8, 88 0.84 0.05Prior x block .87 4, 44 0.49 0.07


high likelihood (p = 0.005). No other main effects orinteractions were significant (see Table 5 for a summaryof results).

For the prior-only trials, subjects’ responses were, onaverage, statistically indistinguishable from the mean ofthe wide prior distribution (t(11) = − 1.14, p = 0.278),but were significantly different from the mean of thenarrow prior (t(11) = − 3.91, p = 0.002) (although wenote that the bias was small (95% CI: 0.24, 0.87 percentof the screen width to the left). The median SD ofresponses was 2.2% for the narrow prior condition and2.6% for the wide prior condition; the SD of responseswas, therefore, only close to the true variance of thewide prior (which was 2.5%). Together, these findings

suggest that subjects had not learned either the mean,or the variance of the narrow prior condition. Thismay explain the lack of difference in performancebetween the narrow and wide prior conditions in thistask.

A comparison of subjects’ weights on the likelihoodsin block five against Bayesian predictions showeda significant difference for all likelihood and priorpairings (p < 0.001), with the exception of the wideprior/ low likelihood condition (t(11) = − .362, p =0.724).

To sum up, although the correct pattern ofweights was present, subjects were still substantiallysub-optimal, even after experiencing all likelihoodvariances from the start of the task.

Accounting for suboptimality

Even when we replicate Bejjanki et al. (2016) veryclosely with all likelihoods from the beginning, ourparticipants are strikingly suboptimal. However, wenote that in our initial calculations of optimal behavior,we assumed that observers’ weight sensory and priorinformation according only to the variance of thedot distribution (i.e. external noise). However, manyof the studies in the cue combination field that havefound near-optimal performance used cues that only



Figure 5. Mean weight placed on the likelihood information in each block of Experiment 1. Blue is the low-variance likelihood, green isthe medium-variance likelihood, red is the high-variance likelihood. Dashed lines show optimal values. Dotted lines show predictedweights when only weighting by internal noise. Error bars are +/− 1 SEM.

have internal noise, not external (Alais & Burr, 2006;Körding & Wolpert, 2006). It is, therefore, possible thatour participants are suboptimal because they fail to takeaccount of external noise, only weighting the sensoryand prior information by the internal variability (i.e.error intrinsic to them) of the sensory cue. Keeping thisis mind, we considered the predicted weights of a modelthat only takes into account the internal variability σliin using the sensory cue.

We performed a separate control experiment to seehow good participants were at finding the centroidsof dot clouds in the absence of prior information (seeAppendix B in the Supplementary Material for moredetails). From this control data, we could calculateobservers’ internal noise as their responses were notsubject to bias from the prior. For each participant,their internal variability σli was calculated by taking theSD of their errors from the centroid of the dots (error= dot centroid - response). The predicted weight on thesensory cue was then calculated as w = σ 2

p/(σ 2li + σ 2

p ).This equation was the same as the full Bayesianmodel, the only difference being that the external SDof the likelihood (as defined by the experimenters)

was substituted for the internal SD of the likelihood(measured in the control experiment). The variance ofthe prior was still included in the model.

We compared subjects’ weights (block 5,Experiment 1) with those predicted when onlyweighting by internal noise and found that they weresignificantly different for all likelihood and priorpairings (p < 0.01). Indeed, Figure 5 shows that theinternal noise model still predicts less weight on thesensory cue than we see in our data (compare barsand dotted lines). This could reflect participantsdownweighing the prior because it is, in fact, subjectto additional internal noise, stemming from a need toremember and recall the correct prior from memory.Even so, empirical weights were closer to the internalnoise predictions, compared to those predicted by theoptimal strategy (with experimentally controlled cuevariance, dashed lines).

In addition, we examined the predictions for anobserver model that weights sensory information,according to overall variability in the sensory cue. Whencalculating the optimal predicted weights initially, weassumed that the optimal observer knew how reliable



Figure 6. Mean weight placed on the likelihood information in each block of Experiment 1. Blue is the low-variance likelihood, green isthe medium-variance likelihood, red is the high-variance likelihood. Dashed lines show optimal values. Dotted lines show predictedweights when overall variability in using the likelihood is taken into account. Error bars are +/− 1 SEM.

the dots (i.e. the likelihood) were, and could averagethem perfectly. Because participants will not be perfectat averaging dots, they will be more variable in using thesensory cue than the optimal observer. Therefore, thetruly optimal thing to do is for participants to weightthe sensory cue, according to their overall variability,by taking into account both the variance of the dotdistribution and the internal variability in estimatingthe average of the dots. Because the sensory cue is nowless reliable (due to the added internal variability), wewould expect participants to put less weight on it, andmore weight on the prior.

We calculated the overall variability in using thesensory cue as:

σ 2lo = σ 2

l /n + σ 2li

where σ 2l /n is the external noise in the sensory cue, and

σ 2li is the individual internal variability.As is expected, Figure 6 shows that the predicted

weights in this case were lower than the optimal weights(compare dotted and dashed lines) as participants areworse than the optimal observer in averaging the dots.They placed less weight on the sensory cue and more

weight on the prior than the optimal observer. We alsocompared these predicted weights to subjects’ weightsin the final block (5) in Experiment 1, and found thatthey were still significantly different from the empiricaldata when the variance of the likelihood was mediumor high (irrespective of prior variance) and when thelikelihood variance was low and the prior variance wasnarrow (all p < 0.001). No significant difference wasobserved when the likelihood variance was low, and theprior variance was wide (p = 0.79). This means thataccounting for the added internal variability fails toexplain our results as observers are placing more weighton the sensory cue than is optimal, not less.

We compared the mean squared error (MSE) for eachof the three models we tested: (1) the original optimalmodel (using the experimentally imposed likelihoodvariance); (2) the model with only the internal noise;and (3) the model with the overall variability (includingboth the experimentally imposed likelihood varianceand the internal noise). The internal noise modelhad the lowest MSE, which confirms that this modelprovides a better explanation for subjects’ behaviorthan other models (see Figure 7).

In summary, our data are best described by amodel based on subjects’ internally generated noise,



Figure 7. Average mean squared error (MSE) for the externalnoise, internal noise and overall noise models. MSE calculationswere based only on participants, for whom control data wasavailable (N = 12; 6 had participated in Experiment 2 and 6 hadparticipated in Experiment 3).

as opposed to either a model with the experimentallyimposed likelihood variance, or a model that accountsfor both the experimentally imposed likelihood varianceand the internal noise.

Discussion

We set out to test more strictly than before forBayes-like combination of prior knowledge withsensory information in the context of a sensorimotordecision making task (Beierholm, Quartz, & Shams,2009; Bejjanki et al., 2016; Berniker et al., 2010;Tassinari et al., 2006; Vilares et al., 2012) by addingtransfer criteria to the task (Maloney & Mamassian,2009).

In Experiment 1, we did this by investigating whetherobservers are able to learn the variances of two priordistributions, and instantly integrate this knowledgewith a new level of sensory uncertainty added mid-waythrough a task. We found that observers placed moreweight on the sensory cue (likelihood) when its variancewas low and the variance of the prior was high; behaviorthat is in broad agreement with the Bayesian prediction.However, we found only partial evidence of transfer.The weight placed on the high variance likelihood wasnot significantly lower than that placed on the mediumvariance likelihood, which is at odds with the predictionof transfer. Importantly, even though qualitatively,our participants behaved like Bayesian observers, theirperformance fell markedly short of optimal.

In two further experiments we asked: (1) howbehavior would be affected by additional instructions,

which can clarify whether this suboptimality stemsfrom using the incorrect model structure of the task;and (2) whether experiencing the high likelihoodvariance condition from the start of the experimentwould lead to closer-to-optimal weighting of the priorand likelihood information. In the first of these twofurther experiments, Experiment 2, we found thatsubjects’ performance moved closer to optimal whenthey were (indirectly) instructed that the prior varianceswere different — possibly why we were able to detectsignificant evidence of transfer in the task. However,they were still significantly suboptimal in multipleexperimental conditions. Participants remainedsignificantly suboptimal in the final experiment(Experiment 3), when the need for transfer was removed(all trials types were present from the start of the task)and the experiment became a more direct replication ofBejjanki et al., (2016).

Suboptimal weighting of prior and likelihoodinformation

We show that observers take uncertainty intoaccount, giving more weight to the sensory cue as itsvariance decreases, a result that is consistent across allthree experiments. Equally, for a Bayes-like observer,we expect to find that the weight on the sensory cue ishigher as the prior variance increases, but we founda main effect of prior in Experiments 1 and 2 only,and not in Experiment 3. Moreover, although ourmanipulation to the instructions in Experiment 2moved the weights placed on the likelihood closer tooptimal, they were still significantly different to theoptimal predictions.

To examine to what extent additional sensoryvariability in estimating the centers of dot-cloudscould have affected predictions and performance,we ran a separate, control experiment (see AppendixB in the Supplementary Material). This shows thatobservers are less efficient in their use of the likelihoodinformation than an ideal observer: the variability oftheir responses is significantly larger than the truevariability of the sensory cue in both the low andmedium variance likelihood conditions. However, thisfails to account for suboptimal performance: idealweights for the likelihood that are computed using themeasured likelihood variabilities in the control task arestill significantly lower than those in the empirical data.

Suboptimal weighting of the prior and likelihoodinformation may also be caused by incomplete orincorrect learning of the prior information. However,the prior-only trials suggest that the observers learnthe means of the priors and distinguish betweentheir variances at least in Experiments 1 and 2, ifnot Experiment 3 (under the assumption that SDs



of subjects’ responses are related to the learnedprior variances). Suboptimal weighting of the priorcould also be due to the use of an incorrect Bayesiangenerative model (causal structure) by subjects (e.g. ifthey believe that the prior will change over trials thenthey should apply a smaller weight to the prior; couldbe conceptualized as a meta-prior or hyperprior inBayesian terms, Gelman, Carlin, Stern, & Rubin, 2013).The fact that we found an effect of instructions impliesthat the causal structure assumed by subjects can indeedgreatly influence behavior (Shams & Beierholm, 2010).

Other research groups have performed similarexperiments but using only a single prior distribution(Acerbi, Vijayakumar, & Wolpert, 2014; Berniker etal., 2010; Chambers et al., 2018; Tassinari et al., 2006;Vilares et al., 2012). These studies also find deviationsfrom the optimal predictions; however, the deviationscan be accounted for by adding extra sources ofinefficiency to the model that are due to motor errors,centroid calculation errors, and aim point (in reachingtasks) calculation errors (Tassinari et al., 2006).Moreover, when trials are blocked by prior conditions, ithas been shown that learning after a switch in the priorvariance is slower when the prior variance decreasesthan when it increases, suggesting participants mayperform optimally after further exposure to the task(Berniker et al., 2010).

We considered elements of the experimental designthat could have resulted in suboptimal behavior in a tasksimilar to others in the literature where performancewas closer to optimal (e.g. Bejjanki et al., 2016). First,describing the dots as the tentacles of an octopusmay have caused participants to assume that anothermethod of gaining an estimate from the dot color, otherthan taking the mean horizontal position, was moreappropriate in this task (de Gardelle & Summerfield,2011; Van Den Berg & Ma, 2012). However, ouranalysis shows that participant responses are not betterpredicted by the median, mid-range, or robust average,than they are by the mean. Second, our correction ofthe dot positions so that their SD on each trial wasequal to the true likelihood SD may have influencedparticipant’s inferred reliability for the likelihood.However, an observer who computes the reliability forthe likelihood trial by trial, by taking the dot cloud SD,would infer that the likelihood was less reliable ( 1

σ 2l) as a

cue to true location than the centroid of the dots wouldbe for an observer who could perfectly calculate themean of the dots ( 1

σ 2l/n). This would lead to an observer

placing less weight on the likelihood than the idealobserver. Participants in our experiment place moreweight on the likelihood than the ideal observer, so thisis not the source of suboptimality in our experiment.

Finally, whereas the true likelihood and priorreliabilities used in our task were matched to those inBejjanki et al. (2016), observers may have perceived the

cue (dots) as more reliable than it actually was, which,in turn, would result in more weight placed on the cuethan in previous studies (Bejjanki et al., 2016; Vilares &Kording, 2011).

It is possible that subjects did not experience enoughtrials of each prior and likelihood uncertainty to reachoptimal performance, and, indeed, we find evidence ofdecreasing weights on the likelihood with increasingtask exposure in both Experiments 1 and 2 (main effectof block, although not the rise from block 4 to 5 inExperiment 2). Crucially, however, our participantsexperienced more trials per prior than in Bejjanki et al.(2016) (750 compared to 600) where weights were closerto optimal, ruling out the possibility that observersdid not experience enough trials to learn the complexfeatures of the distributions.

The result that our participants’ performance was sodifferent, in terms of level of suboptimality, comparedto Bejjanki et al. (2016) might be explained by adifference in instructions. Specifically, their instructionshave a social element that ours do not (i.e. in their task,participants were instructed to interpret the likelihooddots as “locations that other people have previouslyguessed the bucket is located at”; versus tentacles ofthe octopus in ours). This means that in Bejjanki et al.(2016), participants would have to take into accounthow accurate they think other people’s guesses are. Ifwe assume that people give lower weight to informationthat is allegedly based on other people’s guesses, thismight explain why observers in Bejjanki et al. (2016)generally weighted the likelihood less than in ourexperiments (Martino, Bobadilla-suarez, Nouguchi,Sharot, & Love, 2017). Another aspect about theinstructions that is worth mentioning here is that,perhaps, our participants are more likely to assume thatthe body of an octopus is in the center of its tentacles,compared to previous guesses of other participants(Bejjanki et al., 2016) or splashes from a coin (Tassinariet al., 2006; Vilares et al., 2012). However, had thisbeen the case, we would have expected participants’responses to be better predicted by another estimate,such as the robust average, than the mean of thedots, and we found no evidence of this in thedata.

Another explanation is that observers were being“optimally lazy”; that is, they deviated from optimalperformance in a way that had minimal effects on theirexpected reward (Acerbi, Vijayakumar, & Wolpert,2017). In this case, we would expect the obtainedreward to match well with the predictions of theoptimal Bayesian model; instead, the predicted rewardresulting from optimally combing sensory and priorinformation was higher than that obtained by ourobservers, particularly when the variance of the priorwas narrow (see Appendix C in the SupplementaryMaterial). Therefore, we have no reason to believe thatthe suboptimal behavior we observed in our task was



due to our participants being “optimally lazy” (Acerbiet al., 2017).

Nevertheless, we could show that the suboptimalbehavior in our task can be best explained by assumingthat participants were weighting sensory information(relative to prior information) only according to theinternal variability in using the cue, ignoring externalnoise. It is, thus, interesting to consider it as onepotential explanation, on the computational level, forthe deviations from optimal consistently reported insimilar studies on combination of sensory and priorinformation (Bejjanki et al., 2016; Berniker et al., 2010;Sato & Kording, 2014; Tassinari et al., 2006; Vilareset al., 2012). However, we note that attending to internalnoise may be easily mistaken for underweighting thetotal noise; future work could investigate the extent towhich suboptimal behavior is specifically linked to theuse of internal variability, and not simply a generalunderestimation of the total noise in the stimuli.

Note that the observer model, based on the internallygenerated noise, can still be considered “subjectively”optimal (fully Bayesian), in the sense that observers takeinto account and act according to their internal noisevariability (Acerbi et al., 2014). This strategy lookssensible but is arguably not Bayes-optimal as an idealobserver has to take into account external sources ofnoise in addition to his or her own sensory uncertainty(Kersten, Mamassian, & Yuille, 2004; Knill & Richards,1996).

Evidence for bayesian transfer

We found only partial evidence for transfer inExperiment 1, as there was no significant changein the weight placed on the likelihood between themedium and high likelihood conditions. In fact,subjects seemed to treat the high variance likelihoodthe same as the medium variance likelihood (that theyhad experience with), suggesting that observers didnot adopt a statistically optimal Bayesian strategy.Nonetheless, performance did not improve with moretrials, suggesting that subjects were not implementinga look-up table decision rule, either (Maloney &Mamassian, 2009). However, we note that in our data,observers placed much more weight on the medium andhigh likelihoods than is optimal. This means that theeffect size of a change in likelihood is much less thanwas expected; thus, the observed lack of significantdifferences might simply be due to lack of statisticalpower in our analysis to detect such small effect sizes.

Why do we see more convincing evidence of transferin the instructions task? Bayes-like computationsdemand considerable computational resources (e.g.working memory load and attentional focus); itis, therefore, reasonable to expect that if a task issufficiently complex, and there is a lot to learn,

subjects will start behaving suboptimally. The impactof additional instructions in Experiment 2 may be tofree up cognitive resources by providing subjects with(indirect) information about the variances of the twoprior distributions at the start of the task (Ma, 2012;Ma & Huang, 2009).

Our findings do not allow us to clearly distinguishbetween the reinforcement-learning and Bayesianinterpretations. We found that when we introduced anew level of (known) uncertainty to the likelihood,observers immediately changed how they used this newinformation in a way that is largely consistent withoptimal predictions; this effect was significant in thesecond experiment, but not in the first. Thus, we notethat this effect is not particularly robust as it dependson the experimental procedure used to measure it.Indeed, our findings demonstrate that whether thiseffect is observed in the first place is greatly affected bysmall changes in experimental layout (e.g. instructionsand number of trials). The fact that we observe nolearning during Experiment 3 (no main effect ofblock), coupled with the observation that the weighton the likelihood in the fourth block of Experiment 3were remarkably similar to those in the fourth blockof Experiments 1 and 2 makes a weak suggestion ofa Bayesian interpretation. However, a stronger testof transfer would be if participants had received nofeedback for the new level of uncertainty. We providedtrial-by-trial feedback (true target position) to ensurethat participants were able to learn and recall the correctprior distributions. Therefore, we cannot rule out thepossibility that our participants used the feedback todirectly learn a mapping between the high variancelikelihood and each prior, instead of the distribution oflocations.

Sato and Kording (2014) showed that subjects behavein a Bayes-optimal fashion in a sensorimotor estimationtask, where they transferred their knowledge from the“learning phase” to the prior in the testing phase inthe absence of trial-to-trial feedback, suggesting thatpeople did not learn a simple likelihood-prior mapping.This means that the features of our experiments set anapproximate upper bound on learning; in other words,we can generally expect subjects’ performance to be lessaccurate when performance feedback is not provided.

Nevertheless, in order to meaningfully test whetherobservers can transfer probabilistic information acrossdifferent conditions, an experiment where trial-by-trialfeedback is limited, or excluded altogether, is needed.Hudson, Maloney, and Landy (2008) argued that pro-viding only blocked performance feedback, for example,would prevent participants from using a “hill-climbing”strategy in the high variance likelihood condition (i.e.updating their estimates, based on the feedback fromtrial to trial). Alternatively, Acerbi, Vijayakumar, andWolpert (2014) found that partial feedback (whereparticipants are told whether they “hit” or “missed” the



target, but the actual target position is not displayed) issufficient to maintain participant engagement; however,no meaningful information can be extracted from thefeedback, preventing participants from using it to bettertheir performance. Future work could investigate howthe removal of full performance feedback would affectbehavior in more complex scenarios.

What are observers if not Bayesian?

Some studies suggest that BDT is generally a gooddescriptive model of people’s perceptual and motorperformance, but quantitative comparison showsdivergence from Bayes-optimal behavior (Bejjanki et al.,2016; Zhou, Acerbi, & Ma, 2018), not unlike what wereport in this study. These deviations from optimalitymay have arisen because rather than performing thecomplex computations that a typical Bayesian observerwould do, observers draw on simpler non-Bayesian,perhaps even non-probabilistic, heuristics (Gigerenzer& Gaissmaier, 2011; Zhou et al., 2018). Laquitaineand Gardner (2018) developed a model that switchedbetween the prior and sensory information, instead ofcombining the two, which was found to explain the databetter than standard Bayesian models. The authorsconcluded that people can approximate an optimalBayesian observer by using a switching heuristic thatforgoes multiplying prior and sensory likelihood. Inanother study, Norton, Acerbi, Ma, and Landy (2018)compared subjects’ behavior to the “optimal” strategy,as well as several other heuristic models. The modelfit showed that participants consistently computedthe probability of a stimulus as belonging to one oftwo categories as a weighted average of the previouscategory types, giving more weight to those seen morerecently; subjects’ responses also showed a bias towardseeing each prior category equally often (i.e. with equalprobability). We note that a Reinforcement-Learning(RL) model was also tested, where participants couldsimply update the decision criterion after makingan error with no assumptions about probability; noparticipant was best fit by the RL model. This suggeststhat observers are, in fact, probabilistic (i.e. take intoaccount probabilities), although not necessarily in theoptimal way; instead, they seem to resort to heuristicstrategies. However, future work should explore which,if any, of these models can capture the behavior on thistype of complex localization task.

Keywords: bayesian transfer, sensorimotor estimation,decision making, bayesian models

Acknowledgments

Supported by research grants from the LeverhulmeTrust (RPG-2017-097; Marko Nardini, PI) and North

East Doctoral Training Centre (ES/J500082/1; RenetaKiryakova). We would like to thank Dr. James Negenfor helpful comments on the manuscript.

Commercial relationships: none.Corresponding author: Reneta K. Kiryakova.Email: [email protected]: Department of Psychology, DurhamUniversity, Stockton Road, DH1 3LE, Durham, UK.

References

Acerbi, L., Vijayakumar, S., & Wolpert, D. M.(2014). On the origins of suboptimality in humanprobabilistic inference. PLoS ComputationalBiology, 10, e1003661.

Acerbi, L., Vijayakumar, S., & Wolpert, D. M. (2017).Target uncertainty mediates sensorimotor errorcorrection. PLoS One, 12, e0170466.

Acerbi, L., Wolpert, D. M., & Vijayakumar, S. (2012).Internal representations of temporal statistics andfeedback calibrate motor-sensory interval timing.PLoS Computational Biology, 8, e1002771.

Ahrens, M. B., & Sahani, M. (2011). Observers exploitstochastic models of sensory change to helpjudge the passage of time. Current Biology, 21,200–206.

Alais, D., & Burr, D. (2006). The ventriloquist effectresults from near-optimal bimodal integration.Current Biology, 14, 257–262.

Beck, J. M., Ma, W. J., Pitkow, X., Latham, P. E., &Pouget, A. (2012). Not noisy, just wrong: the roleof suboptimal inference in behavioral variability.Neuron, 74, 30–39.

Beierholm, U. R., Quartz, S. R., & Shams, L. (2009).Bayesian priors are encoded independently fromlikelihoods in human multisensory perception.Journal of Vision, 9, 23–23.

Bejjanki, V. R., Knill, D. C., & Aslin, R. N. (2016).Learning and inference using complex generativemodels in a spatial localization task. Journal ofVision, 16, 1–13.

Berniker, M., Voss, M., & Kording, K. (2010). Learningpriors for Bayesian computations in the nervoussystem. PLoS One, 5, 1–9.

Bowers, J. S., & Davis, C. J. (2012). Bayesianjust-so stories in psychology and neuroscience.Psychological Bulletin, 138, 389–414.

Brainard, D. H. (1997). The psychophysics toolbox.Spatial Vision, 10, 433–436.

Chalk, M., Seitz, A. R., & Series, P. (2010). Rapidlylearned stimulus expectations alter perception ofmotion. Journal of Vision, 10, 2.



Chambers, C., Gaebler-spira, D., & Kording, K. P.(2018). The development of Bayesian integrationin sensorimotor estimation. Journal of Vision, 18,1–16.

de Gardelle, V., & Summerfield, C. (2011). Robustaveraging during perceptual judgment. Proceedingsof the National Academy of Sciences, 108,13341–13346.

Gardner, J. L. (2019). Optimality and heuristics inperceptual neuroscience. Nature Neuroscience, 22,514–523.

Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D.B. (2013). Bayesian data analysis. Chapman andHall/CRC, 146, 165–169.

Gigerenzer, G., & Gaissmaier, W. (2011). Heuristicdecision making. Annual Review of Psychology, 62,451–482.

Greenhouse, S. W., & Geisser, S. (1959). On methods inthe analysis of profile data. Psychometrika, 24(2),95–112.

Hudson, T. E., Maloney, L. T., & Landy, M. S. (2008).Optimal compensation for temporal uncertainty inmovement planning. PLoS Computational Biology,4, e1000130.

Jazayeri, M., & Shadlen, M. N. (2010). Temporalcontext calibrates interval timing. NatureNeuroscience, 13, 1020–1026.

Jones, M., & Love, B. C. (2011). Bayesianfundamentalism or enlightenment ? On theexplanatory status and theoretical contributionsof Bayesian models of cognition. The BehaviouralBrain Sciences, 34, 169–188; discussion 188–231.

Kersten, D., Mamassian, P., & Yuille, A. L. (2004).Object perception as Bayesian inference. AnnualReview of Psychology, 55, 271–304.

Kleiner, M., Brainard, D., Pelli, D., Ingling, A.,Murray, R., & Broussard, C. (2007). What’s new inPsychtoolbox-3? Perception, 36, 70821.

Knill, D. C., & Richards, W. (1996). Perception asBayesian Inference. Cambridge University Press,Cambridge, UK.

Kording, K. P., Beierholm, U., Ma, W. J., Quartz, S.,Tenenbaum, J. B., & Shams, L. (2007). Causalinference in multisensory perception. PLoS One, 2,e943.

Körding, K.P., & Wolpert, D. M. (2004). Bayesianintegration in sensorimotor learning. Nature, 427,244–247.

Körding, Konrad P., &Wolpert, D. M. (2006). Bayesiandecision theory in sensorimotor control. Trends inCognitive Sciences, 10, 319–326.

Laquitaine, S., & Gardner, J. L. (2018). A switchingobserver for human perceptual estimation. Neuron,97, 462–474.

Ma, W. J. (2012). Organizing probabilistic modelsof perception. Trends in Cognitive Sciences, 16,511–518.

Ma, W. J., & Huang, W. (2009). No capacity limit inattentional tracking : evidence for probabilisticinference under a resource constraint. Journal ofVision, 9, 1–30.

Maloney, L. T., & Mamassian, P. (2009). Bayesiandecision theory as a model of human visualperception: testing Bayesian transfer. VisualNeuroscience, 26, 147.

Martino, B. De, Bobadilla-suarez, S., Nouguchi, T.,Sharot, T., & Love, X. C. (2017). Social informationis integrated into value and confidence judgmentsaccording to its reliability. Journal of Neuroscience,37, 6066–6074.

Miyazaki, M., Nozaki, D., & Nakajima, Y. (2012).Testing Bayesian models of human coincidencetiming. Journal of Neurophysiology, 94, 395–399.

Norton, E. H., Acerbi, L., Ma, W. J., & Landy, M. S.(2018). Human online adaptation to changes inprior probability. PLoS Computational Biology, 15,e1006681.

Pelli, D. G. (1997). The VideoToolbox software forvisual psychophysics: transforming numbers intomovies. Spatial Vision, 10, 437–442.

Rahnev, D., & Denison, R. N. (2018). Suboptimalityin perceptual decision making. https://doi.org/10.1017/S0140525X18000936. [Online ahead of print].

Sato, Y., & Kording, K. P. (2014). How much to trustthe senses: likelihood learning. Journal of Vision,14, 1–13.

Shams, L., & Beierholm, U. R. (2010). Causal inferencein perception. Trends in Cognitive Sciences, 14,425–432.

Stocker, A. A., & Simoncelli, E. P. (2006). Noisecharacteristics and prior expectations in humanvisual speed perception. Nature Neuroscience, 9,578–585.

Tassinari, H., Hudson, T. E., & Landy, M. S. (2006).Combining priors and noisy visual cues in arapid pointing task. Journal of Neuroscience, 26,10154–10163.

Van Den Berg, R., &Ma, W. J. (2012). Robust averagingduring perceptual judgment is not optimal.Proceedings of the National Academy of Sciences,109, E736–E736.


https://doi.org/10.1017/S0140525X18000936


Vilares, I., Howard, J. D., Fernandes, H. L., Gottfried,J. A., & Kording, K. P. (2012). Differentialrepresentations of prior and likelihood uncertaintyin the human brain.Current Biology, 22, 1641–1648.

Vilares, I., & Kording, K. (2011). Bayesian models:the structure of the world, uncertainty, behavior,and the brain. Annals of the New York Academy ofSciences, 1224, 22–39.

Weiss, Y., Simoncelli, E. P., & Adelson, E. H. (2002).Motion illusions as optimal percepts. NatureNeuroscience, 5, 598–604.

Zhou, Y., Acerbi, L., & Ma, W. J. (2018). The roleof sensory uncertainty in simple perceptualorganization. https://www.biorxiv.org/content/10.1101/350082v1.


https://www.biorxiv.org/content/10.1101/350082v1

bayesiantransferinacomplexspatiallocalizationtaskcommunity.dur.ac.uk/marko.nardini/reprints/kiryakova_jov... ·...

Documents