research article a functional model of sensemaking in a ...computationalintelligence and...

Hindawi Publishing CorporationComputational Intelligence and NeuroscienceVolume 2013 Article ID 921695 29 pageshttpdxdoiorg1011552013921695

Research ArticleA Functional Model of Sensemaking ina Neurocognitive Architecture

Christian Lebiere1 Peter Pirolli2 Robert Thomson1 Jaehyon Paik2

Matthew Rutledge-Taylor1 James Staszewski1 and John R Anderson1

1 Carnegie Mellon University Pittsburgh PA 15213 USA2 Palo Alto Research Center Palo Alto CA 94304 USA

Correspondence should be addressed to Christian Lebiere clcmuedu

Received 21 December 2012 Revised 26 April 2013 Accepted 8 July 2013

Academic Editor Giorgio Ascoli

Copyright copy 2013 Christian Lebiere et al This is an open access article distributed under the Creative Commons AttributionLicense which permits unrestricted use distribution and reproduction in any medium provided the original work is properlycited

Sensemaking is the active process of constructing a meaningful representation (ie making sense) of some complex aspect ofthe world In relation to intelligence analysis sensemaking is the act of finding and interpreting relevant facts amongst thesea of incoming reports images and intelligence We present a cognitive model of core information-foraging and hypothesis-updating sensemaking processes applied to complex spatial probability estimation and decision-making tasks While the modelwas developed in a hybrid symbolic-statistical cognitive architecture its correspondence to neural frameworks in terms of bothstructure and mechanisms provided a direct bridge between rational and neural levels of description Compared against data fromtwo participant groups the model correctly predicted both the presence and degree of four biases confirmation anchoring andadjustment representativeness and probability matching It also favorably predicted human performance in generating probabilitydistributions across categories assigning resources based on these distributions and selecting relevant features given a priorprobability distributionThis model provides a constrained theoretical framework describing cognitive biases as arising from threeinteracting factors the structure of the task environment the mechanisms and limitations of the cognitive architecture and the useof strategies to adapt to the dual constraints of cognition and the environment

1 Introduction

We present a computational cognitive model developed inthe ACT-R architecture [1 2] of several core information-foraging and hypothesis-updating processes involved in acomplex sensemaking task Sensemaking [3ndash6] is a conceptthat has been used to define a class of activities and tasksin which there is an active seeking and processing of infor-mation to achieve understanding about some state of affairsin the world Complex tasks in intelligence analysis andsituation awareness have frequently been cited as examples ofsensemaking [3ndash5] Sensemaking as in tomake sense impliesan active process to construct a meaningful and functionalrepresentation of some aspects of the world A variety oftheories and perspectives on sensemaking have been devel-oped in psychology [3 4] human-computer interaction [6]information and library science [7] and in organizational

science [8] In this paperwe present a cognitivemodel of basicsensemaking processes for an intelligence analysis task

A major concern in the intelligence community is theimpact of cognitive biases on the accuracy of analyses [9]Two prominent biases are confirmation bias in which an ana-lyst disproportionately considers information that supportsthe current hypothesis and anchoring bias inwhich an initialjudgment is insufficiently revised in the face of new evidenceIn the task used in this paper sensemaking is instantiated interms of estimation of probability distributions over hypoth-esis space Rational Bayesian optima are defined over thosedistributions with cognitive biases defined as deviations fromthose optima In this framework confirmation bias can thenbe defined as a distribution ldquopeakierrdquo than the Bayesianoptimum whereas anchoring bias is a flatter-than-rationaldistribution reflecting an insufficient adjustment from theoriginal uniform prior We present simulation results that

2 Computational Intelligence and Neuroscience

Data Framerecognize data defines what counts

from world as data organizes

Elaboration cycle Reframing cycle

Reframe- Compare frames- Seek a new frame

Elaborate a frame Question a frame- Detect inconsistencies- Track anomalies- Judge data quality

- Add and fill slots- Discover data- Infer new

- Discard datarelationships

Preserve

Figure 1 The Data-Frame model of sensemaking Image repro-duced by Klein et al [4]

exhibit several cognitive biases including confirmation biasanchoring and adjustment probability matching and base-rate neglect Those biases are not engineered in the modelbut rather result from the interaction of the structure andstatistics of the task the structure and mechanisms of ourcognitive architecture and the strategies that we select toperform the former using the latter

Figure 1 presents the DataFrame theory of sensemaking[3] The DataFrame theory assumes that meaningful mentalrepresentations called frames define what counts as data andhow those data are structured for mental processing [4]A similar conceptual model was employed in Pirolli andCard [5] to perform a cognitive task analysis of intelligenceanalysis [10ndash12] Frames can be expressed in a variety offorms including stories maps organizational diagrams orscripts Whereas frames define and shape data new data canevoke changes to frames In this framework sensemakingcan involve elaboration of a frame (eg filling in details)questioning a frame (eg due to the detection of anomalies)or reframing (eg rejecting a frame and replacing it withanother) The DataFrame theory proposes that backward-looking processes are involved in forming mental modelsthat explain past events and forward-looking mental sim-ulations are involved in predicting how future events willunfold We describe how frames can be represented in acognitive architecture and how the architectural mechanismscan implement general sensemaking processes We thendemonstrate how the dynamics of sensemaking processes ina cognitive architecture can give rise to cognitive biases in anemergent way

The structure of this paper is as follows Section 2 definesthe AHA (Abducting Hotspots of Activity) experiment con-sisting of a suite of six sensemaking tasks of increasing com-plexity Section 3 outlines our cognitive modeling approachto sensemaking it describes the ACT-R architecture how itis used to prototype neuralmodels which cognitive functionscompose the model and how four cognitive biases can beaccounted for by the model Section 4 presents the measuresused to assess the cognitive biases in the AHA frameworkand then compares human and model results Section 5presents a test of the modelrsquos generalization on a data set

that was unavailable at the time of initial model developmentFinally Section 6 summarizes our account of cognitive biasescentered around themechanisms and limitations of cognitivearchitectures the heuristic that these mechanisms use toadapt to the structure of the task and their interaction withthe task environment

2 The Task Environment

The AHA experiment consists of a series of six tasksdeveloped as part of the IARPA (Intelligence AdvancedResearch Projects Activity) ICArUS (Integrated Cognitive-neuroscience Architectures for the Understanding of Sense-making) program whose goal is to drive the developmentof integrated neurocognitive models of heuristic and biasesin decision-making in the context of intelligence analysisThe AHA tasks can be subdivided into two classes the firstfocusing on learning the statistical patterns of events locatedon a map-like layout and generating probability distribu-tions of category membership based on the spatial locationand frequency of these events (Tasks 1ndash3) and the secondrequiring the application of probabilistic decision rules aboutdifferent features displayed on similar map-like layouts inorder to generate and revise probability distributions ofcategory membership (Tasks 4ndash6)

The AHA tasks simulate the analysis of artificial geospa-tial data presented in a manner consistent with and informedby current intelligence doctrine (Geospatial IntelligenceBasic Doctrine httpwwwfasorgirpagencyngadoctrinepdf) The tasks involve the presentation of multiple featuresconsistent with intelligence data which are presented in aGIS(Geographic Information System) display not unlike Googlemaps (httpsmapsgooglecom) These features include

HUMINT information collected by human sourcessuch as detecting the location of eventsIMINT information collected from imagery of build-ings roads and terrain elementsMOVINT analysis of moving objects such as trafficdensitySIGINT analysis of signals and communicationsSOCINT analysis of social customs and attitudes ofpeople communities and culture

The display (see Figure 2) includes access to the missiontutorial and instructions (the top-right corner) a legendto understand the symbols on the map (the left pane) themap (the center pane) and participantsrsquo current and pastresponses (the right pane)

For Tasks 1ndash3 the flow of an average trial proceedsaccording to the following general outline First participantsperceive a series of events (SIGACTs SIGnals of ACTivity)labeled according to which category the event belongedCategorieswere both color- and shape-codedwith the appro-priate label AquaBromineCitrine or Diamond listed inthe legend After perceiving the series of events a probe eventis displayed (represented as a ldquordquo on the display) Participantswere asked to generate a center of activity (eg prototype)

Computational Intelligence and Neuroscience 3

Your responsesMission areaIntelligencereceived (top)

legend (bottom)

Figure 2 The image is a sample of the display in Task 4 To the left is a legend explaining all the symbols on the map (center) To the rightare the probability distributions for the four event categories (both for the current and prior layer of information) The panel across the topprovides step-by-step instructions for participants

for each categoryrsquos events reflect on how strongly theybelieved the probe belonged to each category and generatea probability estimate for each category (summed to 100across all groups) using the sliders or by incrementing thecounters presented on the right side of the Task interfaceAs an aid the interface automatically normalized the totalprobability such that the total probability summed acrosseach category equaled 100 Participants were not providedfeedback at this step Scoring was determined by comparingparticipants distributions to an optimal Bayesian solution(see Section 4 for a detailed description of how the probabilityestimate scores are calculated) Using these scores it waspossible to determine certain biases For instance partici-pantsrsquo probability estimates that exhibited lower entropy thanan optimal Bayes model would be considered to exhibit aconfirmation bias while probability estimates having higherentropy than an optimal Bayes model would be considered toexhibit an anchoring bias

After finalizing their probability estimates participantswere then asked to allocate resources (using the same right-side interface as probability estimates) to each category withthe goal of maximizing their resource allocation score whichwas the amount of resources allocated to the correct categoryParticipants would receive feedback only on their resource

allocation score For Tasks 1ndash3 the resource allocationresponse was a forced-choice decision to allocate 100 oftheir resources to a single category If that category producedthe probe event then the resource allocation score was 100out of 100 for choosing the correct category otherwise 0out of 100 for choosing an incorrect category Following thisfeedback the next trial commenced

For Tasks 4ndash6 the flow of an average trial was structurallydifferent as intelligence ldquofeaturesrdquo governed by probabilisticdecision rules (see Table 1) were presented sequentially asseparate layers of information on the display These Tasksrequired reasoning based on rules concerning the relationof observed evidence to the likelihood of an unknown eventbelonging to each of four different categories Participantsupdated their beliefs (ie likelihoods) after each layer ofinformation (ie feature) was presented based on the prob-abilistic decision rules described in Table 1

For instance in Task 4 after determining the centerof activity for each category (similar in mechanism toTasks 1ndash3) and reporting an initial probability estimate theSOCINT (SOCial INTelligence) layer would be presented bydisplaying color-coded regions on the display representingeach categoryrsquos boundary After reviewing the informationpresented by the SOCINT layer participants were required


Table 1 Rules for inferring category likelihoods based on knowl-edge of category centroid location and an observed feature

Features Rules

HUMINTIf an unknown event occurs then the likelihood ofthe event belonging to a given category decreases asthe distance from the category centroid increases

IMINT

If an unknown event occurs then the event is fourtimes more likely to occur on a Government versusMilitary building if it is from category A or B If anunknown event occurs then the event is four timesmore likely to occur on aMilitary versus Governmentbuilding if it is from category C or D

MOVINT

If an unknown event occurs the event is four timesmore likely to occur in dense versus sparse traffic if itis from category A or C If an unknown event occursthe event is four times more likely to occur in sparseversus dense traffic if it is from category B or D

SIGINT

If SIGINT on a category reports chatter then thelikelihood of an event by that category is seven timesas likely as an event by each other categoryIf SIGINT on a category reports silence then thelikelihood of an event by that category is one-third aslikely as an event by each other category

SOCINT

If an unknown event occurs then the likelihood ofthe event belonging to a given category is twice aslikely if it is within that categoryrsquos boundary(represented as a colored region on the display)

to update their likelihoods based on this information and thecorresponding probabilistic decision rule

When all the layers have been presented (two layers inTask 4 five layers in Task 5 and four layers in Task 6)participants were required to generate a resource allocationIn these Tasks the resource allocation response was pro-duced using the same interface as probability estimates Forinstance assuming that resources were allocated such thatA = 40B = 30C = 20D = 10 and if the probebelonged to category A (ie that A was the ldquoground truthrdquo)then the participant would receive a score of 40 out of 100whereas if the probe instead belonged to category B theywould score 30 pointsThe resource allocation score providedthe means of measuring the probability matching bias Theoptimal solution (assuming one could correctly predict theright category with over 25 accuracy) would be to alwaysallocate 100 of onersquos resources to the category with thehighest probability Allocating anything less than that couldbe considered an instance of probability matching

Finally participants were not allowed to use any assistivedevice (eg pen paper calculator or other external devices)as the intent of the Task was tomeasure howwell participantswere able to make rapid probability estimates without anyexternal aids

21 Task 1 In Task 1 participants predicted the likelihoodthat a probe event belonged to either of two categoriesAqua or Bromine Categories were defined by a dispersionvalue around a centroid location (eg central tendency)with individual events produced probabilistically by sampling

Figure 3 Sample output fromTask 1 Participants must generate thelikelihood that a probe event (denoted by the ldquordquo) was produced byeach category and then perform a forced-choice resource allocationto maximize their trial score Likelihoods are based on the distancefrom each categoryrsquos centroid and the frequency of events Forinstance Aqua has a higher likelihood because its centroid is closerto the probe and it has a higher frequency (ie more events) thanBromine

in a Gaussian window using a similar function as seen inprototype distortion methodologies from dot pattern catego-rization studies [13] The interface was presented spatially ona computer screen (see Figure 3) in a 100 times 100 grid pattern(representing 30 square miles grid not shown)

Participants were instructed to learn about each cate-goryrsquos tendencies according to three features the categoryrsquoscenter of activity (ie centroid) the dispersion associatedwith each categoryrsquos events and the frequency of eventsfor each category Using these three features participantsdetermined the likelihood that the probe event belonged toeach category

A trial consisted of 10 events with 9 events presentedsequentially at various locations about the interface with par-ticipants required to click ldquonextrdquo after perceiving each eventThe 10th event was the probe event which was presented asa ldquordquo on the interface Each participant completed 10 trialswith events accumulating across trials such that 100 eventswere present on the interface by the end of the task

After perceiving the probe event participants wereinstructed to generate likelihoods that the probe eventbelonged to each category based on all the events that theyhave seen not just the recent events from the current trialThese likelihoods were expressed on a scale from 1 to 99 foreach category and summing to 100 across both categoriesIf necessary the interface would automatically normalize alllikelihoods into probabilities summing to 100

Finally participants entered a forced-choice resourceallocation response analogous to a measure of certaintyResource allocation was a forced-choice decision to allocate100 of their resources to a single category If that categoryproduced the probe event then the participant would receivefeedback that was either 100 out of 100 for choosing the


Figure 4 Sample output from Task 2 Participants must generatethe likelihood that a probe event (denoted by the ldquordquo) was producedby each category and then do a forced-choice resource allocation tomaximize their trial score In addition participants had to draw a 2-to-1 boundary for each category whose boundary encapsulates 23of that categoryrsquos events and whose center represents the center ofactivity for that category Likelihoods are based on the distance fromeach categoryrsquos centroid and the frequency of events For instanceCitrine has the highest likelihood because it has a higher frequencythan the other categories while Diamond has a marginally higherlikelihood than Aqua and Bromine because it has the closestdistance

correct category or 0 out of 100 for choosing an incorrectcategory Following this feedback the next trial commenced

22 Task 2 In Task 2 participants predicted the likeli-hood that a probe event belonged to either of four cate-gories AquaBromineCitrine or Diamond The interfaceand procedure were similar to Task 1 with the followingdifferences A trial consisted of 20 events with 19 events pre-sented sequentially at various locations about the interfaceThe 20th event was the probe event which was presented as aldquordquo on the interface Each participant completed 5 trials withevents accumulating across trials such that 100 events werepresent on the interface by the end of the task Participantswere further required to estimate each categoryrsquos centroid anddispersion by drawing a circle for each category representinga 2-to-1 boundary with 23 of the categoryrsquos events insidethe circle and 13 outside (see Figure 4) Participants clickedwith the mouse to set the centroid and dragged out withthe mouse to capture the 2-to-1 boundary releasing themouse to set the position It was possible to adjust both theposition and dispersion for each category after their initialset Estimating category centroids and dispersion precededgenerating likelihoods

Finally participants entered a similar forced-choiceresource allocation response as in Task 1 Resource alloca-tion was a forced-choice decision to allocate 100 of theirresources to a single category If that category produced theprobe event then the participant would receive feedback thatwas either 100 out of 100 for choosing the correct category or

Figure 5 Sample output from Task 3 Participants must generatethe likelihood that a probe event (denoted by the ldquordquo) was producedby each category and then do a forced-choice resource allocationto maximize their trial score Likelihoods are based on the roaddistance from each categoryrsquos centroid and the frequency of eventsFor instance Citrine has the highest likelihood because it is theclosest category

0 of out 100 for choosing an incorrect category Following thisfeedback the next trial commenced

23 Task 3 In Task 3 participants predicted the likelihoodthat a probe event belonged to either of four categoriessimilar to Task 2 with the following differences Instead of theinterface instantiating a blank grid it displayed a network ofroads Events were only placed along roads and participantswere instructed to estimate distance along roads rather thanldquoas the crow fliesrdquo In addition participants no longer hadto draw the 2-to-1 boundaries but instead only identify thelocation of the category centroid

A trial consisted of 20 events with 19 events presentedsequentially at various locations about the interfaceThe 20thevent was the probe event which was presented as a ldquordquoon the interface Each participant completed 5 trials withevents accumulating across trials such that 100 events werepresent on the interface by the end of the task Participantswere further required to estimate each categoryrsquos centroid byplacing a circle for each category (see Figure 5) Participantsclicked with the mouse to set the centroid It was possible toadjust the position for each category after the initial set Basedon the requirement to judge road distance Task 3 (and Task4) involved additional visual problem solving strategies (egspatial path planning and curve tracing) [14]

Finally participants entered a similar forced-choiceresource allocation response as in Task 2 Resource alloca-tion was a forced-choice decision to allocate 100 of theirresources to a single category If that category produced theprobe event then the participant would receive feedback thatwas either 100 out of 100 for choosing the correct category or0 out of 100 for choosing an incorrect category Following thisfeedback the next trial commenced


Figure 6 Sample output from Task 4 Participants must generatethe likelihood that a probe event (denoted by the Diamond) wasproduced by each category (1ndash4) first by the HUMINT layer(distance from category centroids to probe event) and then by theSOCINT layer (likelihoods are doubled for the category in whoseregion the probe event falls) Finally participants allocate resourcestomaximize their trial score For instance category 4 has the highestlikelihood because it is the closest category and the probe falls withinits boundary

24 Task 4 Beginning with Task 4 instead of gatheringinformation from a sequence of events participants insteadgenerated and updated likelihoods after being presented witha number of features as separate layers of information Thesefeatures were governed by probabilistic decision rules [15]described previously in Table 1 In Task 4 two featureswere presented to participants in a fixed order The firstlayer was HUMINT (HUMan INTelligence) which revealedthe location of the category centroid for each category Thesecond layer was SOCINT (SOCial INTelligence) whichrevealed color-coded regions on the display representing eachcategoryrsquos boundary (see Figure 6) If a probe event occurredin a given categoryrsquos boundary then the probability that theprobe belonged to that category was twice as high as the eventbelonging to any of the other categories

Participants were instructed that the feature layers pro-vided ldquocluesrdquo revealing intelligence data (called INTs) and theprobabilistic decisions rules (called PROBs rules) providedmeans to interpret INTs Participants were instructed to referto the PROBs handbook (based on Table 1 see the appendixfor the complete handbook) which was accessible by clickingon the particular layer in the legend on the left side of thedisplay or by reviewing the mission tutorial in the top-rightcorner They were further instructed that each feature layerwas independent of other layers

The same simulated geospatial display from Task 3 wasused in Task 4 however instead of a trial consisting of a seriesof individual events a trial instead consisted of reasoningfrom category centroids to a probe event by updating likeli-hoods after each new feature was revealed A trial consisted oftwo features presented in sequence (HUMINT and SOCINTresp) The HUMINT layer revealed the centroids for each

category along with the probe event Participants reportedlikelihoods for each category 1 2 3 or 4 based on the roaddistance between the probe and each categoryrsquos centroidSimilar to previous tasks likelihoods were automatically nor-malized to a probability distribution (ie summing to 100)After this initial distribution was input the SOCINT featurewas presented by breaking the display down into four coloredregions representing probabilistic category boundaries Usingthese boundaries participants applied the SOCINT rule andupdated their probability distribution

Once their revised probability distribution was enteredparticipants were required to generate a resource allocationThe resource allocation response was produced using thesame interface as probability estimates For instance assum-ing that resources were allocated such that 1 = 40 2 =30 3 = 20 4 = 10 and if the probe belonged to category1 (ie that 1 was the ldquoground truthrdquo) then the participantwould receive a score of 40 out of 100 whereas if the probeinstead belonged to category 2 they would score 30 pointsAfter completing their resource allocation the display wasreset and a new trial started

Participants completed 10 trials Unlike Tasks 1ndash3 eachtrial was presented on a unique road network with all fourcategory locations presented in a unique location

25 Task 5 In Task 5 all five features were revealed to partici-pants in each trial with theHUMINT feature always revealedfirst (and the rest presented in a randomorder)Thus partici-pants began each trial with each categoryrsquos centroid presentedon the interface and the Bayesian optimal probability distri-bution already input on the right-side response panel (seeFigure 7)The goal of Task 5 was to examine how participantsfused multiple layers of information together Unlike Task4 the correct probability distribution for HUMINT wasprovided to participants This was done both to reduce thevariance in initial probabilities (due to the noisiness of spatialroad distance judgments) and also to reduce participantfatigue After perceiving HUMINT and being provided thecorrect probability distribution each of the four remainingfeatures (SOCINT IMINT MOVINT and SIGINT on asingle category) was revealed in a random order After eachfeature was revealed participants updated their probabilitydistribution based on applying the corresponding decisionrules Similar to Task 4 after the final feature was revealedparticipants allocated resources

The same methodology was used as for Task 4 only withfive layers of features presented instead of two Participantsreported likelihoods for each category Aqua BromineCitrine or Diamond based on the information revealedby the feature at each layer according to the rules of thePROBs handbook Likelihoods were automatically normal-ized to a probability distribution (ie summing to 100)After HUMINT was revealed four other features (SOCINTMOVINT IMINT and SIGINT) were revealed in randomorder The SOCINT feature was presented by breaking thedisplay down into four colored regions representing proba-bilistic category boundaries The IMINT (IMagery INTelli-gence) feature was presented by showing either a governmentor military building at the probe location The MOVINT


Figure 7 Sample output fromTask 5 Participantsmust generate thelikelihood that a probe event (denoted by the probe event ldquo1rdquo) wasproduced by each category The HUMINT layer is always displayedfirst and the initial probability distribution based on road distanceis provided to participants Participants must update this initialdistribution as new features are revealed In the current example thelikelihoods of categories A and C are increased due to theMOVINTlayer revealing sparse traffic at the probe event location (see PROBsrules in Table 1)

(MOVement INTelligence) feature was presented by showingeither sparse or dense traffic at the probe location Finallythe SIGINT (SIGnal INTelligence) feature was presentedby showing either the presence or absence of chatter for aspecific category at the probe location After each feature wasrevealed participants applied the relevant PROBs rule andupdated their probability distribution

After all feature layers were revealed and probabilitydistributions were revised participants were required to gen-erate a resource allocation The resource allocation responsewas produced using the same interface as in Task 4 Aftercompleting their resource allocation the display was resetand a new trial started Participants completed 10 trials Notethat participants needed to update likelihoods four times pertrial (thus 40 times in total) in addition to a single resourceallocation per trial (10 total) Similar to Task 4 each trial waspresented on a unique road network with all four categorylocations presented in a unique location

26 Task 6 In Task 6 participants were able to choose threeof four possible features to be revealed in addition to theorder in which they are revealed (see Figure 8) The goal ofTask 6 was to determine participantsrsquo choices and ordering inselecting features (which we refer to as layer selection) Thismethodology determined whether participants were biasedto pick features whose corresponding decision rule con-firmed their leading hypothesis or possiblymaximized poten-tial information gain Participants were instructed to chooselayers thatmaximized information at each step to increase thelikelihood of a single category being responsible for the event

As for Task 5 a trial began by perceiving the HUMINTlayer and being provided the correct probability distribution

Figure 8 Sample output fromTask 6 Participantsmust generate thelikelihood that a probe event (denoted by the probe event ldquo1rdquo) wasproduced by each category The HUMINT layer is always displayedfirst and the initial probability distribution based on road distanceis provided to participants Participants must update this initialdistribution as new features are revealed In the current example thelikelihoods of categories A and C are increased due to theMOVINTlayer revealing sparse traffic at the probe event location

Participants must then choose a feature to be revealed(SOCINT IMINT MOVINT or SIGINT on a single cat-egory) When participants chose the SIGINT layer theyneeded to further specify which category they were inspect-ing (listening for chatter) After the chosen feature wasrevealed participants updated their probability distributionbased on applying the corresponding decision rules Thisprocess was repeated twice more with different features fora total of three layers being chosen Participants must updatecategory likelihoods AquaBromineCitrine or Diamondafter each layer was revealed based on the information pro-vided by the corresponding feature at each layer according tothe rules of the PROBs handbook As in the other tasks like-lihoods were automatically normalized to sum to 100 acrosscategories Note that with only three layer selection choicesparticipants were not able to reveal one feature on each trial

After participants completed the process of choosinga feature and updating their likelihoods for each of threeiterations participants were required to generate a resourceallocation The resource allocation response was producedusing the same interface as in Tasks 4-5 After completingtheir resource allocation the display was reset and a newtrial commenced Participants completed 10 trials Note thatwith three layer selections participants actually updatedprobabilities 30 times (3 times per trial) in addition toallocating resources once for each trial Similar to Tasks 4-5 each trial was presented on a unique road network with allfour category locations presented in a unique location

3 An ACT-R Model of Sensemaking

31 Overview of ACT-R Our aim has been to develop afunctional model of several core information-foraging and


hypothesis-updating processes involved in sensemaking Wedo this by developing ACT-R models to specify how ele-mentary cognitive modules and processes are marshaled toproduce observed sensemaking behavior in a set of complexgeospatial intelligence Tasks These tasks involve an iterativeprocess of obtaining new evidence from available sourcesand using that evidence to update hypotheses about potentialoutcomes One purpose of the ACT-R functional model isto provide a roadmap of the interaction and sequencing ofneural modules in producing task performance (see nextsection) A second purpose is to identify and understand acore set mechanisms for producing cognitive biases observedin the selection and weighting of evidence in informationforaging (eg confirmation bias)

TheACT-R architecture (see Figure 9) is organized as a setof modules each devoted to processing a particular kind ofinformation which are integrated and coordinated througha centralized production system module Each module isassumed to access and deposit information into buffersassociated with the module and the central productionsystem can only respond to the contents of the buffers not theinternal encapsulated processing of the modules Each mod-ule including the production module has been correlatedwith activation in particular brain locations [1] For instancethe visual module (occipital cortex and others) and visualbuffers (parietal cortex) keep track of objects and locations inthe visual field The manual module (motor cortex cerebel-lum) and manual buffer (motor cortex) are associated withcontrol of the hands The declarative module (temporal lobehippocampus) and retrieval buffer (ventrolateral prefrontalcortex) are associated with the retrieval and awareness ofinformation from long-term declarative memory The goalbuffer (dorsolateral prefrontal cortex) keeps track of thegoals and internal state of the system in problem solvingFinally the production system (basal ganglia) is associatedwith matching the contents of module buffers and coordinat-ing their activity The production includes components forpattern matching (striatum) conflict resolution (pallidum)and execution (thalamus) A production rule can be thoughtof as a formal specification of the flow of information frombuffered information in the cortex to the basal ganglia andback again [16]

The declarative memory module and production systemmodule respectively store and retrieve information thatcorresponds to declarative knowledge and procedural knowl-edge [17] Declarative knowledge is the kind of knowledgethat a person can attend to reflect upon andusually articulatein some way (eg by declaring it verbally or by gesture)Procedural knowledge consists of the skills we display in ourbehavior generally without conscious awareness Declarativeknowledge in ACT-R is represented formally in terms ofchunks [18 19] The information in the declarative mem-ory module corresponds to personal episodic and semanticknowledge that promotes long-term coherence in behaviorIn this sense a chunk is like a data frame integratinginformation available in a common context at a particularpoint in time in a single representational structure The goalmodule stores and retrieves information that represents the

Environment

Prod

uctio

ns(b

asal

gan

glia

)

Retrieval buffer (VLPFC)

Matching (striatum)

Selection (pallidum)

Execution (thalamus)

Goal buffer

Visual buffer (parietal)

Manual buffer (motor)

Manual module (motorcerebellum)

Visual module (occipitaletc)

Intentional module(not identified)

Declarative module(temporalhippocampus)

ACT-R cognitive architecture

(DLPFC)

Figure 9ACT-R functions as a production system architecturewithmultiple modules corresponding to different kinds of perceptionaction and cognitive information stores Modules have been identi-fied with specific brain regions In addition to those shown abovethe Imaginal module has been associated with posterior parietalactivation

internal intention and problem solving state of the system andprovides local coherence to behavior

Chunks are retrieved from long-termdeclarativememoryby an activation process (see Table 2 for a list of retrievalmechanisms in ACT-R) Each chunk has a base-level acti-vation that reflects its recency and frequency of occurrenceActivation spreads from the current focus of attentionincluding goals through associations among chunks indeclarative memory These associations are built up fromexperience and they reflect how chunks cooccur in cognitiveprocessing The spread of activation from one cognitivestructure to another is determined by weighting values onthe associations among chunks These weights determine therate of activation flow among chunks Chunks are comparedto the desired retrieval pattern using a partial matchingmechanism that subtracts from the activation of a chunkits degree of mismatch to the desired pattern additively foreach component of the pattern and corresponding chunkvalue Finally noise is added to chunk activations to makeretrieval a probabilistic process governed by a Boltzmann(softmax) distributionWhile themost active chunk is usuallyretrieved a blending process [20] can also be applied whichreturns a derived output reflecting the similarity between thevalues of the content of all chunks weighted by their retrievalprobabilities reflecting their activations and partial-matchingscores This blending process will be used intensively inthe model since it provides both a tractable way to learnto perform decisions in continuous domains such as theprobability spaces of the AHA framework and a directabstraction to the storage and retrieval of information inneural models (see next section)


Table 2 The list of sub-symbolic mechanisms in the ACT-R architecture

Mechanism Equation Description

Activation 119860119894= 119861119894+ 119878119894+ 119875119894+ 120576119894

119861119894 base-level activation reflects the recency and frequency of use of

chunk i119878119894 spreading activation reflects the effect that buffer contents have on

the retrieval process119875119894 partial matching reflects the degree to which the chunk matches

the request120576119894 noise value includes both a transient and (optional) permanent

components (permanent component not used by the integratedmodel)

Base level 119861119894= ln(

119899

sum119895=1

119905minus119889

119895) + 120573

119894

119899 the number of presentations for chunk i119905119895 the time since the jth presentation119889 a decay rate (not used by the integrated model)120573119894 a constant offset (not used by the integrated model)

Spreading activation119878119894= sum119896

sum119895

119882119896119895119878119895119894

119896 weight of buffers summed over are all of the buffers in the model119895 weight of chunks which are in the slots of the chunk in buffer k119882119896119895 amount of activation from sources j in buffer k

119878119895119894 strength of association from sources j to chunk i

119878119895119894= 119878 minus ln (fan

119895119894)

119878 the maximum associative strength (set at 4 in the model)fan119895119894 a measure of how many chunks are associated with chunk j

Partial matching 119875119894= sum119896

119875119872119896119894

119875 match scale parameter (set at 2) which reflects the weight given tothe similarity119872119896119894 similarity between the value k in the retrieval specification and

the value in the corresponding slot of chunk iThe default range is from 0 to minus1 with 0 being the most similar and minus1being the largest difference

Declarative retrievals 119875119894=

119890119860119894119904

sum119895119890119860119895119904

119875119894 the probability that chunk 119894 will be recalled119860119894 activation strength of chunk 119894

sum119860119895 activation strength of all of eligible chunks 119895

119904 chunk activation noise

Blended retrievals 119881 = minsum119894

119875119894(1 minus Sim

119894119895)2 119875

119894 probability from declarative retrieval

Sim119894119895 similarity between compromise value 119895 and actual value 119894

Utility learning

119880119894(119899) = 119880

119894(119899 minus 1) + 120572 [119877

119894(119899) minus 119880

119894(119899 minus 1)]

119880119894(119899 minus 1) utility of production i after its 119899 minus 1st application

119877119894(119899) reward production received for its nth application

119880119894(119899) utility of production i after its nth application

119875119894=

119890119880119894119904

sum119895119890119880119895119904

119875119894 probability that production i will be selected119880119894 expected utility of the production determined by the utility

equation above119880119895 the expected utility of the competing productions j

Production rules are used to represent procedural knowl-edge inACT-RThat is they specify procedures that representand apply cognitive skill (know-how) in the current contextand how to retrieve and modify information in the buffersand transfer it to other modules In ACT-R each productionrule has conditions that specify structures that arematched inbuffers corresponding to information from the externalworldor other internal modules Each production rule has actionsthat specify changes to be made to the buffers

ACT-R uses a mix of parallel and serial processing Mod-ules may process information in parallel with one anotherSo for instance the visual modules and the motor modulesmay both operate at the same time However there are twoserial bottlenecks in process First only one production maybe executed during a cycle Second each module is limitedto placing a single chunk in a buffer In general multipleproduction rules can be applied at any point Production

utilities learned using a reinforcement learning scheme areused to select the single rule that fires As for declarativememory retrieval production selection is a probabilisticprocess

Cognitive model development in ACT-R [21] is in partderived from the rational analysis of the task and informationstructures in the external environment (eg the design ofthe tasks being simulated or the structure of a graphical userinterface) the constraints of the ACT-R architecture andguidelines frompreviousmodels of similar tasks A successfuldesign pattern in specifying cognitive process sequencing inACT-R [21] is to decompose a complex task to the level ofunit tasks [22] Card et al [22] suggested that unit taskscontrol immediate behaviorUnit tasks empirically take about10 seconds To an approximation unit tasks are where ldquotherubber of rationality meets the mechanistic roadrdquo To anapproximation the structure of behavior above the unit task


level largely reflects a rational structuring of the task withinthe constraints of the environment whereas the structurewithin and below the unit task level reflects cognitive andbiological mechanisms in accordance with Newellrsquos bandsof cognition [23] Accordingly in ACT-R unit tasks areimplemented by specific goal types that control productionsthat represent the cognitive skills for solving those tasks

ACT-R has been the basis for several decades of researchon learning complex cognitive tasks such as algebra andprogramming [24 25] In general the long-run outcome oflearning such tasks is a large set of highly situation-specificproductions whose application is sharply tuned by ACT-Rutility mechanisms (a form of reinforcement learning) How-ever it is also generally assumed that achieving such expertlevels of learning requires 1000s of hours of experience Weassume that the participants in the AHA tasks will not havethe opportunity to achieve such levels of expertise Insteadwe hypothesize that participants will rely on direct recogni-tion or recall of relevant experience from declarativememoryto guide their thinking or failing that will heuristically inter-pret and deliberate through the rules and evidence providedin the challenge tasks This compute-versus-retrieve processis another design pattern that typically structures ACT-Rmodels [21] The notion that learners have a general-purposemechanism whereby situation-action-outcome observationsare stored and retrieved as chunks in ACT-R declarativememory is derived from instance-based learning theory(IBLT) [26 27] Gonzalez et al [26] present argumentsthat IBLT is particularly pertinent to modeling naturalisticdecisionmaking in complex dynamic situations andmany ofthose arguments would transfer to making the case that IBLTis appropriate for sensemaking

Relevant to the Bayesian inspiration for the AHAtasks ACT-Rrsquos subsymbolic activation formula approximatesBayesian inference by framing activation as log-likelihoodsbase-level activation (119861

119894) as the prior the sum of spreading

activation and partial matching as the likelihood adjustmentfactor(s) and the final chunk activation (119860

119894) as the posterior

The retrieved chunk has an activation that satisfies themaximum likelihood equation ACT-R provides constraintto the Bayesian framework through the activation equationand production system The calculation of base levels (iepriors) occurs within both neurally and behaviorally con-sistent equations (see Table 2) providing for behaviorallyrelevantmemory effects like recency and frequencywhile alsoproviding a constrainedmechanism for obtaining priors (iedriven by experience)

In addition the limitations on matching in the produc-tion system provide constraints to the Bayesian hypothesisspace and as a result the kinds of inferences that can bemade For instance there are constraints on the kinds ofmatching that can be accomplished (eg no disjunctionmatching only to specific chunk types within buffers) andwhile user-specified productions can be task-constrained theproduction system can generate novel productions (throughproceduralization of declarative knowledge) using produc-tion compilation In addition the choice of which productionto fire (conflict resolution) also constrains which chunks

(ie hypotheses) will be recalled (limiting the hypothesisspace) and are also subject to learning via production utilities

It has been argued that ACT-Rrsquos numerous parametersdo not provide sufficient constraint on modeling endeav-ors However the use of community and research-justifieddefault values the practice of removing parameters bydeveloping more automatized mechanisms and the devel-opment of common modeling paradigmsmdashsuch as instance-based learning theorymdashmitigate these criticisms by limitingdegrees of freedom in the architecture and thus constrainingthe kinds of models that can be developed and encouragingtheir integration

32 ACT-R Prototyping for Neural Models ACT-R can beused in a prototyping role for neural models such asEmergent which uses the Leabra learning rule [28] In ACT-R models can be quickly developed and tested and theresults of thesemodels then help informmodeling efforts anddirect training strategies in Emergentmodels [29 30] ACT-Rmodels can be created quickly because ACT-Rmodels acceptpredominantly functional specifications yet they produceneurally relevant results The ACT-R architecture is also flex-ible enough that innovations made in neurocomputationalmodels can be implemented (to a degree of abstraction)within new ACT-R modules [31]

There are several points of contact between ACT-R andEmergent the most tangible of which is a commitment toneural localization of architectural constructs in both archi-tectures (see Figure 9) In both architectures a central controlmodule located in the basal ganglia collects inputs from avariety of cortical areas and outputs primarily to the frontalcortex which maintains task relevant information [16 32]Additionally both include a dedicated declarativeepisodicmemory system in the hippocampus and associated corticalstructures Lastly both account for sensory and motor pro-cessing in the posterior cortex

The architectures differ in that the brain regions areexplicitly modeled in Emergent whereas they are implicitin ACT-R In ACT-R the basal ganglia are associated withthe production system the frontal cortex with the goalmodule the parietal cortex with the imaginal module thehippocampus with the declarative memory module andfinally the posterior cortices with the manual vocal auraland vision modules This compatibility of ACT-R and Emer-gent has been realized elsewhere by the development ofSAL (Synthesis of ACT-R and LeabraEmergent) a hybridarchitecture that combinesACT-R andEmergent and exploitsthe relative strengths of each [33] Thus ACT-R connects tothe underlying neural theory of Emergent and can providemeaningful guidance to the development of neural models ofcomplex tasks such as sensemaking

In effect ACT-Rmodels provide a high-level specificationof the information flows that will take place in the neuralmodel between component regions implemented in Emer-gent Since ACT-R models have been targeted at preciselythis level of description they can provide for just the rightlevel of abstraction while ignoring many implementationaldetails (eg number of connections) at the neural level


Conceptually the ACT-R architecture provides a bridgebetween the rational Bayesian level and the detailed neurallevel In terms of Marr [34] levels of analysis the Bayesiancharacterization of the task solutions is situated at the com-putational level describing the computations that should beperformed without specifying how AnACT-R account of thetasks is at the algorithmicrepresentational level specifyingwhat representations are created whichmechanisms are usedto manipulate them and which structure constrains bothrepresentations and processes Finally a neural account is sit-uated at the physicalimplementational level fully specifyingall the details of how the computations are to be carried out inthe brain Thus just as in Marrrsquos analysis it would not makesense to try to bridge directly the highest and lowest levelsa functional cognitive architecture such as ACT-R providesa critical link between abstract computational specificationssuch as Bayesian rational norms and highly detailed neuralmechanisms and representations

Moreover ACT-R does not just provide any intermediatelevel of abstraction between computational and implemen-tational levels in a broad modular sense Rather just as theACT-R mechanisms have formal Bayesian underpinningsthey also have a direct correspondence to neural mecha-nisms and representations The fundamental characteristicsof modern neural modeling frameworks are distributedrepresentations local learning rules and training of networksfrom sets of input-output instances [35]

Distributed representations are captured in ACT-Rthrough similarities between chunks (and other sets of valuessuch as number magnitudes) that can be thought of ascorresponding to the dotproduct between distributed repre-sentations of the corresponding chunks The generalizationprocess operates over distributed representations in neuralnetworks effectively matching the learned weights from theinput units resulting in a unit containing the representation ofthe current inputThis is implemented in ACT-R using a par-tial matching mechanism that combines a chunkrsquos activationduring the memory retrieval process with its degree of matchto the requested pattern as determined by the similaritiesbetween chunk contents and pattern [36]

Local learning rules in neural networks are used toadjust weights between units based on information flowingthrough the network The base-level and associative learningmechanisms in ACT-R perform a similar function in thesame manner Both have Bayesian underpinnings [37] butalso direct correspondence to neural mechanisms Base-levellearning is used to adjust the activation of a chunk basedon its history of use especially its frequency and recencyof access This corresponds to learning the bias of a unitin a network determining its initial activation which isadded to inputs from other units Associative learning adjuststhe strengths of association between chunks to reflect theirdegree of coactivation While the original formulation wasBayesian in nature a new characterization makes the linkto Hebbian-like learning explicit in particular introducingthe same positive-negative learning phases as found in manyconnectionist learning algorithms including Leabra [31]

Neural models are created by a combination of modeler-designed structure and training that adjusts the networkrsquos

weights in response to external inputs The instance-basedlearning approach in ACT-R similarly combines a repre-sentational structure provided by the modeler with contentacquired from experience in the form of chunks that repre-sent individual problem instances The set of chunks storedin declarative memory as a result can be thought of as theequivalent to the set of training instances given to the neuralnetwork While the network compiles those instances intoweights during training ACT-R can instead dynamicallyblend those chunks together during memory retrieval toproduce an aggregate response that reflects the consensus ofall chunks weighted by their probability of retrieval reflectingthe activation processes described above [20]

Thus ACT-R models can be used to prototype neuralmodels because they share both a common structure of infor-mation flow as well as a direct correspondence from themoreabstract (hence tractable) representations andmechanisms atthe symbolicsubsymbolic level and those at the neural level

33 Cognitive Functions Engaged in the AHA Tasks Theintegrated ACT-R model of the AHA tasks has been used toprototype many cognitive effects in neural models includinggenerating category prototypes of centroids from SIGACTevents (centroid generation) [29] spatial path planningalong road networks [30] adjusting probabilities based onincoming information [29] choosing how many resourcesto allocate given a set of probabilities and prior experience[29] and selecting additional intelligence layers (see Table 3for an overview) The modelrsquos output compared favorablywith human behavioral data and provides a comprehensiveexplanation for the origins of cognitive biases in the AHAframework most prominently the anchoring and adjustmentbias All the functions described below were integrated in asingleACT-Rmodel that performed all 6AHA tasks using thesame parameters That model was learned across trials andtasks We will describe later in details how its performancein the later trials of a task can depend critically upon itsexperience in the earlier trials (even just the first trial) inparticular leading to a distinct conservatism bias Similarlyits performance in the later tasks depends upon its experiencein earlier tasks leading directly to probability matching biasin resource allocation

The model performs the task in the same manner ashuman subjects Instructions such as the probabilistic deci-sion rules are represented in declarative memory for laterretrieval when needed The model perceives the eventsrepresents them in the imaginal buffer and then storesthem in declarative memory where they influence futurejudgments In Tasks 1ndash3 the model uses those past eventsin memory to generate the category centroid when given aprobe In Tasks 3-4 the model programmatically parses themap to represent the road network declaratively and thenuses that declarative representation to generate paths andestimate road distances In all tasks probability adjustmentis performed using the same instance-basedmechanismwithexperience from earlier tasks accumulated inmemory for usein later tasks Resource allocation is also performed in alltasks using the same instance-based approach with results


Table 3 Overview of cognitive functions of ACT-R model

Cognitive function Overview of operation

Centroid generationTasks 1ndash3

Buffers implicated blending imaginal and goalBiases instantiated base-rate neglect anchoring and adjustmentThe model generates a category centroid by aggregating overall of the perceived events (SIGACTs) in memoryvia the blended memory retrieval mechanism Judgments are based on generating a centroid-of-centroids byperforming a blended retrieval over all previously generated centroids resulting to a tendency to anchor to earlyjudgments Because there is an equal number of centroids per category this mechanism explicitly neglects baserate

Path planningTasks 3-4

Buffers implicated retrieval imaginal and goalBiases instantiated anchoring and adjustmentThe model parses the roads into a set of intersections and road segments The model hill-climbs by starting atthe category centroid and appends contiguous road segments until the probe event is reached Road segmentlengths are perceived veridically however when recalled the lengths are influenced by bottom-up perceptualmechanisms (eg curve complexity and length) simulated by a power law with an exponent less than unity Thisleads to underestimation of longer and curvier segments resulting in a tendency to anchor when perceivinglong segments

Probability adjustmentTasks 1ndash6

Buffers implicated blending imaginal and goalBiases instantiated anchoring in weighing evidence confirmation biasThe model represents the prior probability and multiplicative factor rule and then attempts to estimate thecorrect posterior by performing a blended retrieval over similar chunks in memory in a form of instance-basedlearning The natural tendency towards regression to the mean in blended retrievals leads to anchoring bias inhigher probabilities and confirmation bias in lower probabilities The partial matching mechanism is used toallow for matches between the prior and similar values in DM

Resource allocationTasks 1ndash6

Buffers implicated blending imaginal and goalBiases instantiated probability matchingThe model takes the probability assigned to a category and then estimates an expected outcome by performing ablended retrieval using the probability as a cue The outcome value of the retrieved chunk is the expectedoutcome for the trial Next an additional blended retrieval is performed based on both the probability andexpected outcome whose output is the resources allocationAfter feedback the model stores the leading category probability the resources allocated and the actualoutcome of the trial Up to two counterfactuals are learned representing what would have happened if awinner-take-all or pure probability matching resources allocation had occurred Negative feedback on forcedwinner-take-all assignments in Tasks 1ndash3 leads to probability matching in Tasks 4ndash6

Layer selectionTask 4ndash6

Buffers implicated blending goalBiases instantiated confirmation biasIn Task 6 the model uses partial matching to find chunks representing past layer-selection experiences that aresimilar to the current situation (the distribution of probabilities over hypotheses) If that retrieval succeeds themodel attempts to estimate the utility of each potential layer choice by performing a blended retrieval over theutilities of past layer-choice outcomes in similar situations The layer choice that has the highest utility isselected If the model fails to retrieve past experiences similar to the current situations it performs aldquolook-aheadrdquo search by calculating the expected utility for some feature layers The number of moves mentallysearched will not often be exhaustiveThe blended retrieval mechanism will tend to average the utility of different feature layers based on priorexperiences from Tasks 4 and 5 (where feature layers were provided to participants) in addition to prior trialson Task 6

from forced-choice selections in Tasks 1ndash3 fundamentallyaffecting choices in later Tasks 4ndash6 Finally layer selection inTask 6 uses experiences in Tasks 4-5 to generate estimates ofinformation gain and select the most promising layer Thusthe integrated model brings to bear constraints from all tasksand functions

331 Centroid Generation The ACT-R integrated modelgenerates category centroids (ie the prototype or centraltendency of the events) in Tasks 1ndash3 by aggregating overall ofthe representations of events (eg spatial-context frames) inmemory via the blended memory retrieval mechanism The

goal buffer maintains task-relevant top-down informationwhile the blending buffer createsupdates centroids fromboth the set of perceived SIGACTs to date and prior createdcentroids Centroid generation approximates a stochasticleast-MSE derived from distance and based on the 2D Carte-sian coordinates of the individual SIGACTs Specifically themismatch penalty (119875

119894) used in the blended retrieval is a linear

difference

119875119894=

2 sdot10038161003816100381610038161198891 minus 1198892

1003816100381610038161003816

lowastmax rangelowast (1)

where 119889 is the perceived distance and lowastmax rangelowast is thesize of the display (100 units)The imaginal buffer (correlated


with parietal activity) is used to hold blended chunks beforebeing committed to declarative memory When centroidsare generated directly from SIGACTs the blending processreflects a disproportionate influence of the most recentevents given their higher base-level activation A strategyto combat this recency bias consisted of generating a finalresponse by performing a blended retrieval over the currentand past centroids thereby giving more weight to earlierSIGACTsThis is because the influence of earlier centroids hasbeen compounded over the subsequent blended retrievalsessentially factoring earlier SIGACTs into more centroidsThis second-order blended retrieval is done for each categoryacross their prior existing centroids which we refer to as thegeneration of a centroid-of-centroidsThis blending over cen-troids effectively implements an anchoring-and-adjustmentprocess where each new centroid estimate is a combinationof the previous ones together with the new evidence Afundamental difference with traditional implementation ofanchoring-and-adjustment heuristic is that this process isentirely constrained by the architectural mechanisms (espe-cially blending) and does not involve additional degrees offreedom Moreover because there are an equal number ofcentroid chunks (one per category created after each trial)there is no effect of category base rate on the modelrsquos laterprobability judgments even though the base rate for eachcategory is implicitly available in the model based on thenumber of recallable events

332 Path Planning The ACT-R model uses the declarativememory and visual modules to implement path planningwhich simulate many of the parietal functionalities that werelater implemented in a Leabra model [30] Two examplesinclude

(1) perceptually segmenting the road network so thatthe model only attends to task-relevant perceptualelements

(2) implementing visual curve tracing to model the psy-chophysics of how humans estimate curved roads

The model segments the road network in Tasks 3-4 intosmaller elements and then focuses perceptual processes suchas curve tracing distance estimation and path planning onthese smaller road segments [38] Specifically the modelidentifies the intersections of different roads as highly salientHUMINT features and then splits the road network into roadsegments consisting of two intersections (as the ends of thesegment) the general direction of the road and the length ofroad Intersections are generally represented as a particularlocation on the display in Cartesian119883-119884 coordinates

For each trial the probe location is also represented as alocal HUMINT feature and in Task 3 the individual eventsare represented as local HUMINT features for the purposesof calculating category centroids At the end of each trialthe probe is functionally removed from the path planningmodel although a memory trace of the previous location stillremains in declarative memory

We have implemented a multistrategy hill-climber toperform path planning The model starts with a category

centroid and appends contiguous road segments until theprobe location is reached (ie the model is generatingand updating a spatial-context frame) The path planningldquodecision-makingrdquo is a function of ACT-Rrsquos partial matchingIn partial matching a similarity is computed between asource object and all target objects that fit a set of matchingcriteria This similarity score is weighted by the mismatchpenalty scaling factor The hill-climber matches across mul-tiple values such as segment length and remaining distanceto probe location In general for all road segments adjoiningthe currently retrieved intersection or category centroidthe model will tend to pick the segment where the nextintersection is nearest to the probe This is repeated until thesegment with the probe location is retrievedThese strategiesare not necessarily explicit but are instead meant to simulatethe cognitive weights of different perceptual factors (egdistance direction and length) guiding attentional processesThe partial matching function generates probabilities fromdistances and calculates similarity between distances usingthe same mismatch penalty as in Tasks 1 and 2

Human performance data on mental curve tracing [14]show that participants take longer to mentally trace along asharper curve than a relatively narrower curveThis relation isroughly linear (with increased underestimation of total curvelength at farther curves) and holds along the range of visualsizes of roads that were seen in the AHA tasksThis modelingassumption is drawn from the large body of the literature onvisuospatial representation and mental scanning [39] Whenroad network is parsed a perceived length is assigned toeach road segment This length is more or less representedveridically in declarative memory The dissociation betweena veridical perceived magnitude of distance and a postpro-cessed cognitive distance estimate is consistent with priorliterature [40] We represent a cognitive distance estimateusing Stevensrsquo Power Law [41] Stevensrsquo Power Law is a well-studied relationship between themagnitude of a stimulus andits perceived intensity and serves as a simple yet powerfulabstraction of many low-level visual processes not currentlymodeled in ACT-R

The function uses the ratio of ldquoas the cow walksrdquo distanceto ldquoas the crow fliesrdquo distance to create an estimate of curvecomplexity [41]The higher the curve complexity the curvierthe road To represent the relative underestimation of dis-tance for curvier segments this ratio is raised to an exponentof 82 [41ndash43] The effect of this parameter is that for eachunit increase in veridical distance the perceived distance isincreased by a lesser extent The closer the exponent to 1the more veridical the perception and the closer to zerothe more the distance will be underestimated This value forcurve complexity is then multiplied by a factor representingstraight-line distance estimation performance (102) [43ndash45]

119863 = 102 (CowWalkCrowFlies

) + CrowFlies 82

(2)

where 119863 is the cognitive judgment of distance for the roadsegment CowWalk is the veridical perception of the curvatureof the road and CrowFlies is the veridical perception of theEuclidean distance between the source and target locations


0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability

Probability adjustment (linear similarities)

RationalModel

(a)

0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability

Probability adjustment (ratio similarities)

RationalModel

(b)

Figure 10 Results from an ACT-R model of probability adjustment with linear (a) and ratio (b) similarities

The factor of 102 represents a slight overestimation of smallerstraight-line distances Similar to the exponent any factorabove unity represents an overestimation of distance and anyfactor below unity represents an underestimation of distance

333 Probability Adjustment Lebiere [20] proposed amodelof cognitive arithmetic that used blended retrieval of arith-metic facts to generate estimates of answers without explicitcomputations The model was driven by number of sim-ilarities that correspond to distributed representations fornumber magnitudes in the neural model and more generallyto our sense of numbers [46] It used partial matching tomatch facts related to the problem and blended retrievalsto merge them together and derive an aggregate estimatedanswer The model reproduced a number of characteristicsof the distribution of errors in elementary school childrenincluding both table and nontable errors error gradientsaround the correct answer higher correct percentage for tieproblems and most relevant here a skew toward under-estimating answers a bias consistent with anchoring andadjustment processes

To leverage this approach for probability adjustment theACT-R modelrsquos memory was populated with a range of factsconsisting of triplets an initial probability an adjustmentfactor and the resulting probability These triplets form thebuilding blocks of the implementation of instance-basedlearning theory [47] and correspond roughly to the notion ofa decision frame [3 4] In the AHA framework the factor isset by the explicit rules of the task (eg an event in a categoryboundary is twice as likely to belong to that category) Themodel is then seeded with a set of chunks that correspondto a range of initial probabilities and an adjustment factortogether with the posterior probability that would result frommultiplying the initial probability by the adjustment factor

then normalizing it When the model is asked to estimate theresulting probability for a given prior and multiplying factorit simply performs a blended retrieval specifying prior andfactor and outputs the posterior probability that representsthe blended consensus of the seeded chunks Figure 10displays systematic results of this process averaged over athousand runs given the variations in answers resulting fromactivation noise in the retrieval process When providedwith linear similarities between probabilities (and factors)the primary effect is an underestimation of the adjustedprobability for much of the initial probability range with anoverestimate on the lower end of the range especially forinitial values close to 0The latter effect is largely a result of thelinear form of the number similarities function While linearsimilarities are simple they fail to scale both to values nearzero and to large values

A better estimate of similarities in neural representationsof numbers is a ratio function as reflected in single cellrecordings [1] This increases dissimilarities of the numbersnear zero and scales up to arbitrarily large numbers Whenusing a ratio similarity function the effects from the lin-ear similarity function are preserved but the substantialoverestimate for the lower end of the probability range isconsiderably reduced While the magnitude of the biasescan be modulated somewhat by architectural parameterssuch as the mismatch penalty (scaling the similarities) orthe activation noise (controlling the stochasticity of memoryretrieval) the effects themselves are a priori predictions ofthe architecture in particular its theoretical constraints onmemory retrieval

Particular to the integrated model of the AHA tasks themismatch penalty (119875

119894) was computed as a linear difference

119875119894= 2 lowast

10038161003816100381610038161003816119872119896minus119872119895

10038161003816100381610038161003816 (3)


where 119872119896is the possible target probability and 119872

119895is the

probability in the blended retrieval specification As will bedescribed below this linear differencematches extremely wellto human behavioral data

The results of the ACT-R mechanism for probabilityadjustment provided a benchmark against which neuralmodels were evaluated and were used to generate traininginstances for the neural model which had already embodiedthe proper bias In our opinion this constitutes a novel wayto use functional models to quickly prototype and interpretneural models

334 Resource Allocation The resource allocation mecha-nism in the model makes use of the same instance-basedlearning paradigm as the probability adjustment mechanismThis unified mechanism has no explicit strategies but insteadlearns to allocate resources according to the outcomes ofpast decisions The model generates a resource allocationdistribution by focusing on the leading category and deter-mining how many resources to allocate to that category Theremaining resources are divided amongst the remaining threecategories in proportion to their assigned probabilities Thisinstance-based learning not only occurs during Tasks 4ndash6 butalso inTasks 1ndash3 for forced-choice allocations Because of thisthe model has some prior knowledge to draw upon in Task 4when it first has the opportunity to select howmany resourcesto assign to the leading category

As mentioned above this instance-based model hasthe same structure as the model of probability adjustmentRepresentation of a trial instance consists of three parts adecision context (in this case the probability of the leadingcategory) the decision itself (ie the resource allocation tothe leading category) and the outcome of the decision (iethe payoff resulting from the match of that allocation to theground truth of the identity of the responsible category)Thisrepresentation is natural because all these pieces of infor-mation are available during a resource allocation instanceand can plausibly be bound together in episodic memoryHowever the problem is how to leverage it tomake decisions

Decision-making (choice)models based on this instance-based learning approach iterate through a small number ofpossible decisions generating outcome expectancies fromthe match of context and decision and then choose thedecision with the highest expected outcome [47 48] Controlmodels apply the reverse logic given the current context anda goal (outcome) state they match context and outcome togenerate the expected action (usually a control value from acontinuous domain) that will get the state closest to the goal[49 50] However our problem does not fit either paradigmunlike choice problems it does not involve a small numberof discrete actions but rather a range of possible allocationvalues and unlike control problems there is no known goalstate (expected outcome) to be reached

Our modelrsquos control logic takes a more complex hybridapproach involving two steps of access to experiences indeclarative memory rather than a single one The first stepconsists of generating an expected outcome weighted overthe available decisions given the current context The second

step will then generate the decision that most likely leadsto that outcome given to the context Note that this processis not guaranteed to generate optimal decisions and indeedpeople do not Rather it represents a parsimonious way toleverage our memory of past decisions in this paradigm thatstill provides functional behavior A significant theoreticalachievement of our approach is that it unifies control modelsand choice models in a single decision-making paradigm

When determining how many resources to apply to thelead category the model initially has only the probabilityassigned to that category The first step is to estimate anexpected outcome This is done by performing a blendedretrieval on chunks representing past resource allocationdecisions using the probability as a cue The outcome valueof the retrieved chunk is the expected outcome for thetrial Next based on the probability assigned to the leadingcategory and the expected outcome an additional blendedretrieval is performed The partial matching mechanism isleveraged to allow for nonperfect matches to contribute tothe estimation of expected outcome and resource quantityThe resource allocation value of this second blended allocatechunk is the quantity of resources that the model assigns tothe leading category After feedback is received the modellearns a resource allocation decision chunk that associatesthe leading category probability the quantity of resourcesassigned to the leading category and the actual outcome ofthe trial (ie the resource allocation score for that trial)Additionally up to two counterfactual chunks are committedto declarative memory The counterfactuals represent whatwould have happened if a winner-take-all resource assign-ment had been applied and what would have happened ifa pure probability-matched resource assignment (ie usingthe same values as the final probabilities) had been appliedThe actual nature of the counterfactual assignments is notimportant what is essential is to give the model a broadenough set of experience representing not only the choicesmade but also those that could have been made

The advantage of this approach is that the model is notforced to choose between a discrete set of strategies suchas winner-take-all or probability matching rather variousstrategies could emerge from instance-based learning Bypriming the model with the winner-take-all and probabilitymatching strategies (essentially the boundary conditions) itis possible for the model to learn any strategy in betweenthem such as a tendency to more heavily weigh the leadingcandidate (referred to as PM+) or even suboptimal strategiessuch as choosing 25 for each of the four categories (assuringa score of 25 on the trial) if the model is unlucky enoughto receive enough negative feedback so as to encourage riskaversion [47]

335 Layer Selection Layer selection in Task 6 dependson learning the utilities of layer choices in Tasks 4-5 andrelies on four processes instance-based learning (similar toprobability adjustment and resource allocationmechanisms)difference reduction heuristic reinforcement learning andcost-satisfaction During Tasks 4ndash6 participants were askedto update probability distributions based on the outcome of


each layer (ie feature and relevant decision rule) In Tasks 4-5 participants experienced 20 instances of the SOCINT ruleand 10 instances each of the IMINT MOVINT and SIGINTrules They also had a variable number of instances fromTask 6 based on their layer selections Some of the layersand outcomes might support their preferred hypothesis butsome of them might not Based on the results of each layerrsquosoutcome the gain towards the goal of identifying a singlecategory might vary and those experiences affect future layerselection behavior through reinforcement learning

A rational Bayesian approach to Tasks 4ndash6 might involvethe computation of expected information gains (EIGs) com-puted overall possible outcomes that might result from theselection of a feature layer Under such a rational strategySIGINT and SOCINT layers would require more calculationcost than IMINT andMOVINT In particular the calculationof EIG for SIGINT requires considering four categories withtwo outcomes each and the calculation of EIG for SOCINTrequires four outcomes however the calculation of EIGfor an IMINT or MOVINT layer requires consideration ofonly two outcomes We assume participants might considerthe cognitive costs of exploring layer selection outcomes inpreferring certain layer selection

We chose to use a difference reduction heuristic (ie hill-climbing) because we assume that an average person is notable to compute and maintain the expected information gainfor all layers A hill-climbing heuristic enables participantsto focus on achieving states that are closer to an ideal goalstate with the same requirement for explicit representationbecause all that needs to be represented is the differencebetween the current state and a preferred (ie goal) state

In Task 6 all prior instances were used to performevaluations of layer selection First the model attends to thecurrent problem state including the distribution of likelihoodof attacks as represented in the goal (Prefrontal CortexPFC) and imaginal (Parietal Cortex PC) buffers Then themodel attempts to retrieve a declarative memory chunk(HippocampusMedial Temporal Lobe HCMTL) whichencodes situation-action-outcome-utility experiences of pastlayer selections This mechanism relies on partial matchingto retrieve the chunks that best match the current goalsituation and then on blending to estimate the utility of layer-selectionmoves based on past utilities If the retrieval requestfails then themodel computes possible layer-selectionmoves(ie it performs a look-ahead search) using a difference-reduction problem-solving heuristic In difference reductionfor each mentally simulated layer-selection action the modelsimulates and evaluates the utility of the outcome (withsome likelihood of being inaccurate) Then the model storesa situation-action-outcome-utility chunk for each mentallysimulated move It is assumed that the number of movesmentally searched will not often be exhaustiveThis approachis similar to the use of counterfactuals in the resourceallocation model

TheACT-Rmodel of Task 6 relies on the use of declarativechunks that represent past Tasks 4 5 and 6 experiences Thisis intended to capture a learning process whereby participantshave attended to a current probability distribution chosen alayer revised their estimates of the hypotheses and finally

assessed the utility of the layer selection they just madeThe model assumes that chunks are formed from theseexperiences each representing the specific situation (proba-bility distribution over groups) selected intelligent layer andobserved intelligence outcome and information utility wherethe utilities are computed by the weighted distancemetric (119889)below

119889 = sum119894isinHypotheses

119901119894(1 minus 119901

119894) (4)

where each119901 is a posterior probability of a group attack basedon rational calculation and zero is the optimum The ACT-R model uses this weighted distance function and assumesthat the participantrsquos goal is to achieve certainty on one of thehypotheses (ie 119901

119894= 1)

At a future layer selection point a production rule willrequest a blendedpartial matching retrieval from declar-ative memory based on the current situation (probabilitydistribution over possible attacking groups) ACT-R will usea blended retrieval mechanism to partially match againstprevious experience chunks and then blend across the storedinformation utilities for each of the available intelligence layerchoices For each layer this blending over past experienceof the information utilities will produce a kind of expectedinformation utility for each type of intelligence for specificsituations Finally the model compares the expected utilityof different intelligence layers and selects the one with thehighest utility

The ACT-R model performs reinforcement learningthroughout Tasks 4 to 6 After updating probability distribu-tion based on a layer and its outcomes the model evaluateswhether it has gained or lost information by comparingthe entropy of the prior distribution with the posteriordistribution If it has gained information the production forthe current layer receives a reward if it has lost it receives apunishment This reinforcement learning enables the modelto acquire a preference order for the selection of intelligencelayers and this preference order list was used to determinewhich layer should be explored first in the beginning of thelayer selection process

34 Cognitive Biases Addressed Anchoring and confirmationbiases have been long studied in cognitive psychology andthe intelligence communities [9 51ndash55] As we have alreadymentioned these biases emerge in several ways in the ACT-Rmodel of AHA tasks (see Table 4 for an overview) In generalour approach accounts for three general sources of biases

The first source of bias is the architecture itself bothin terms of mechanisms and limitations In our model aprimary mechanistic source of bias is the ACT-R blendingmechanism that is used to make decisions by drawingon instances of similar past problems While it providesa powerful way of aggregating knowledge in an efficientmanner its consensus-driven logic tends to yield a regressionto themean that often (but not always) results in an anchoringbias Limitations of the architecture such as workingmemorycapacity and attentional bottlenecks can lead to ignoringsome information that can result in biases such as base-rate


Table 4 Source of cognitive biases in the ACT-R integrated model of the AHA tasks

Cognitive bias Mechanism Source of bias in functional model (ACT-R)

Confirmation bias

Attentional effect (seeking) Feature selection behavior such as selecting SIGINT too early Blendedretrieval during layer choice using stored utilities

Overscaling in rule application(weighing)

Bias in blended retrieval of mappings from likelihood factor to revisedprobability (low value) Weighted-distance utilities used for layer selectionsshows confirmation bias in weighing

Anchoring in learningUnderscaling in rule application Bias in blended retrieval of mappings from likelihood factor to revised

probability (high values)

Centroid computation Inertia from centroid estimates to consolidated values to DM Productionsencoding thresholds in distance for centroid updating

Representativeness Base-rate neglectBase rate not a cue for matching to a category Compute distance to categorycentroid rather than cloud of events Blended retrievals ignore number ofevents

Probability matching Resource allocationUse of instance-based learning leads to tendency of risk aversion againstwinner-take-all instances leading to the tendency for the blended retrieval ofinstances between pure probability matching and winner-take-all

neglect (ignoring background frequencies when making ajudgment of conditional probabilities)

The second source of bias is the content residing inthe architecture most prominent strategies in the form ofprocedural knowledge Strategies can often lead to biaseswhen they take the form of heuristic that attempt to conservea limited resource such as only updating a subset of theprobabilities in order to save time and effort or overcomea built-in architectural bias such as the centroid-of-centroidstrategy intended to combat the recency effect in chunkactivations that in turn leads to an anchoring bias

The third and final source of biases is the environmentitself more specifically its interaction with the decision-maker For instance the normalization functionality in theexperimental interface can lead to anchoring bias if it resultsin a double normalization Also the feedback provided bythe environment or lack thereof can lead to the emergenceor persistence of biases For instance the conservatism biasthat is often seen in the generation of probabilities couldpersist because subjects do not receive direct feedback as tothe accuracy of their estimates

In the next subsections we discuss in detail the sources ofthe various biases observed

341 Anchoring and Adjustment Anchoring is a cognitivebias that occurswhen individuals establish some beliefs basedon some initial evidence and then overly rely on this initialdecision in their weighting of new evidence [54] Humanbeings tend to anchor on some estimates or hypotheses andsubsequent estimates tend to be adjustments that are influ-enced by the initial anchor pointmdashthey tend to behave as ifthey have an anchoring + adjustment heuristic Adjustmentstend to be insufficient in the sense that they overweight theinitial estimates and underweight new evidence

Anchoring and adjustment in learning (AL) can occur inthe first three tasks due to the nature of the task specificallythe iterative generation of the centroids of each categoryacross each trial For each trial participants estimate acentroid for the events perceived to date by that category

then observe a new set of events and issue a revised estimateThis process of issuing an initial judgment and then revisingmight lead to anchoring and adjustment processes Thusin Tasks 1ndash3 anchoring can occur due to the centroid-of-centroid strategy to prevent being overly sensitive to themostrecent events

Tasks 4ndash6 can also elicit anchoring biases Anchoringbias in weighing evidence might be found when participantsrevise their belief probabilities after selecting and interpretinga particular feature The estimates of belief probabilities thatwere set prior to the new feature evidence could act as ananchor and the revised (posterior) belief probabilities couldbe insufficiently adjusted to reflect the new feature (ie whencompared to some normative standards) Insufficient adjust-ment may happen because the blended retrieval mechanismtends to have a bias towards the mean

The model also updates only the probabilities corre-sponding to the positive outcomes of the decision rulesFor example if it is discovered that the probe occurs on amajor road the model would update the probabilities forcategories A and B and neglect to adjust downward theprobabilities for categories C and D This neglect is assumedto result from a desire to save labor by relying on the interfacenormalization function and by the difficulty of carrying outthe normalization computations mentally In turn this is acause of an underestimation of probabilities (anchoring) thatresults from the normalization of the probabilities in theinterface

342 Confirmation Bias Confirmation bias is typicallydefined as the interpretation of evidence in ways that arepartial to existing beliefs expectations or a hypothesis inhand [53] the tendency for people to seek information andcues that confirm the tentatively held hypothesis or beliefand not seek (or discount) those that support an oppositeconclusion or belief [56] or the seeking of informationconsidered supportive of favored beliefs [53] Studies [57ndash59]have found evidence of confirmation bias in tasks involvingintelligence analysis and there is a common assumption that


many intelligence failures are the result of confirmation biasin particular [9 60]

Confirmation bias in weighing evidence might occur inthe probability adjustment process in Tasks 4ndash6 For examplewe have seen that the probability adjustment process appliedto small probability values sometimes resulted in over-adjustment Certain strategies for probability adjustmentmight also result in confirmation bias When applying aparticular feature such as IMINT (which supports twohypotheses) participants may only apply the adjustment tothe preferred hypothesis while neglecting the other categorythat is also supported by evidence or weight the evidence toostrongly in favor of the preferred category

Confirmation bias can also occur in the evidence seekingprocess in Task 6 as the participants might select intelligencelayers that maximize information gains about the currentpreferred category For instance when applying the IMINTand MOVINT rules one could only apply the adjustmentto the preferred hypothesis (assuming it is one of the tworeceiving favorable evidence from that layer) while neglectingthe other categories also supported by the evidence Thisstrategic decision could reflect the desire both to minimizeeffort and to maximize information gain

A significant difference in layer selection features is theSIGINT feature which requires the selection of one particularcategory to investigate If that feature is applied to the leadingcategory and chatter is detected then the category likelihoodgains considerable weight (by a factor of 7) However if nochatter is detected then the category likelihood is stronglydowngraded which throws considerable uncertainty over thedecision processThus the decision to select the SIGINT layertoo early (before a strong candidate has been isolated) orto apply it to strictly confirm the leading category ratherthan either confirm or invalidate a close second might beconstrued as a form of confirmation bias in layer selection

343 Base-Rate Neglect Base-rate neglect is an error thatoccurs when the conditional probability of a hypothesis isassessed without taking into account the prior backgroundfrequency of the hypothesisrsquo evidence Base-rate neglect cancome about from three sources

(1) Higher task difficulties and more complex environ-ments can lead to base-rate neglect due to the sheervolume of stimuli to remember To reduce memoryload some features may be abstracted

(2) Related to the above there can be base-rate neglectdue to architectural constraints For instance short-termmemory is generally seen to have a capacity of 7plusmn2 chunks of information available Oncemore chunksof information need to be recalled some informationmay either be abstracted or discarded

(3) Finally there can be explicit knowledge-level strategicchoices made from an analysis of (1) and (2) above

The strategic choice (3) of the ACT-R model leads tobase-rate neglect in calculating probabilities for Tasks 1ndash3 In particular the fact that the ACT-R model generates

probabilities based on category centroids leads to base-rate neglect This is because base-rate information is notdirectly encoded within the category centroid chunk Theinformation is still available within the individual SIGACTsstored in the model but it is not used directly in generatingprobabilities

344 Probability Matching We endeavored to develop amodel that leveraged subsymbolic mechanisms that oftengive rise naturally to probability matching phenomena [61]Subsymbolic mechanisms in ACT-R combine statistical mea-sures of quality (chunk activation for memory retrievalproduction utility for procedural selection) with a stochasticselection process resulting in behavior that tends to select agiven option proportionately to its quality rather than in awinner-take-all fashionThis approach is similar to stochasticneural models such as the Boltzmann Machine [35]

In our model resource allocation decisions are basednot on discrete strategies but rather on the accumulation ofindividual decision instances Strategies then are an emergentproperty of access to those knowledge bases Moreover tounify our explanation across biases we looked to leveragethe same model that was used to account for anchoring (andsometimes confirmation) bias in probability adjustment

Results also approximate those seen by human partic-ipants a wide variation between full probability matchingand winner-take-all several individual runs tending towardsuniform or random distributions and the mean fallingsomewhere between probability matching and winner-take-all (closer to matching)

Probability matching in resource allocation occurs dueto the trade-off inherent in maximizing reward versus mini-mizing risk A winner-take-all is the optimal strategy overallhowever there are individual trials with large penalties (a zeroscore) when a category other than the one with the highestprobability is the ground truth When such an outcomeoccurs prominently (eg in the first trial) it can have adeterminant effect on subsequent choices [47]

4 Data and Results

The results of the ACT-R model on the AHA tasks werecompared against 45 participants who were employees of theMITRE Corporation All participants completed informedconsent and debriefing questionnaires that satisfied IRBrequirements To compare the ACT-R model to both humanparticipants and a fully Bayesian rational model several met-rics were devised by MITRE [62ndash64] which are summarizedbelow

As an overall measure of uncertainty across a set ofhypotheses we employed a Negentropy (normalized negativeentropy) metric119873 computed as

119873 =(119864max minus 119864)

119864max (5)

where 119864 is the Shannon entropy computed as

119864 = minussumℎ

119875ℎlowast log2119875ℎ (6)


where the summation is over the probabilities 119875ℎ assigned

to hypotheses Negentropy can be expressed on a scale of0 to 100 where 119873 = 0 implies maximum uncertainty(ie maximum entropy or a uniform probability distributionover hypotheses) and 119873 = 100 implies complete certainty(ie zero entropy or a distribution in which one hypothesisis assigned a maximum probability of 1) The normalizationis provided by 119864max = 2 in the case of four hypotheses (Tasks2ndash6) and 119864max = 1 in the case of two hypotheses (Task 1)

Comparisons of human and normative (eg Bayesian)assessments of certainty as measured by Negentropy permitthe evaluation of some cognitive biases For instance onecan compare human Negentropy 119873

119867to Negentropy for a

rational norm 119873119876

following the revision of probabilitiesassigned to hypotheses after seeing new intelligence evidenceIf 119873119867gt 119873119876 then the human is exhibiting a confirmation

bias because of overweighing evidence that confirms themostlikely hypothesis On the other hand if 119873

119867lt 119873119876 then the

human is exhibiting conservatism which might arise from ananchoring bias

In addition to measuring biases we also compared theprobability estimation and resource allocation functions ofthe model against both human participants and a Bayesianrational model (ie an optimal model)The Kullback-LeiblerDivergence (119870) is a standard information-theoretic measurefor comparing two probability distributions like those of ahuman (119875) and model (119872) 119870

119875119872measures the amount of

information (in bits) by which the two distributions differwhich is computed as follows

119870119875119872= 119864119875119872minus 119864119875

= minussumℎ

119875ℎlowast log2119872ℎ+ sumℎ

119875ℎlowast log2119875ℎ

(7)

where similar to the Negentropy measure 119864119875119872

is the cross-entropy of human participants (119875) and theACT-Rmodel (119872)and 119864

119875is the entropy of human participants It is important

to note that 119870119875119872

= 0 when both distributions are the sameand119870

119875119872increases as the two distributions diverge119870 ranges

from zero to infinity but 119870 is typically less than 1 unless thetwo distributions have large peaks in different hypotheses

A normalizedmeasure of similarity (119878) on a 0ndash100 scalesimilar to that of Negentropy can be computed from119870

119878 = 100 lowast 2minus119870 (8)

As the divergence 119870 ranges from zero to infinity thesimilarity 119878 ranges from 100 to 0 Thus 119878

119876119875and 119878119876119872

canbe useful for comparing the success of humans or modelsin completing the task (compared by their success relativeagainst a fully rational Bayesian model) This measure will bereferred to as an S1 score

To address the overall fitness of the model output com-pared with human data the most direct measure would bea similarity comparing the human and model distributions(119878119875119872

) directly However this would not be a good measureas it would be typically higher than 50 (119870 is typicallyless than 1) thus we scaled our scores on a relative basisby comparing against a null model A null model (eg auniform distribution 119877 = 025 025 025 025) exhibits

maximum entropy which implies ldquorandomrdquo performance insensemaking Thus 119878

119875119877was used to scale as a lower bound in

computing a relative success rate (RSR) measure as follows

RSR =(119878119875119872minus 119878119875119877)

(100 minus 119878119875119877) (9)

The modelrsquos RSR was zero if 119878119875119872

is equal to or less than119878119875119877 because in that case a null model 119877 would provide the

same or better prediction of the human data as the modelThe RSR for a model119872 will increase as 119878

119875119872increases up to

a maximum RSR of 100 when 119878119875119872

= 100 For example ifa candidate model 119872 matches the data 119875 with a similarityscore of 119878

119875119872= 80 and the null model 119877 matches 119875 with

a similarity score of 119878119875119877

= 60 then the RSR for model 119872would be (80 minus 60)(100 minus 60) = (2040) = 50

In Task 6 because each participant potentially receivesdifferent stimuli at each stage of the trial as they choosedifferent INTs to receive RSR was not appropriate Insteadthe model was assessed against a relative match rate (RMR)which is defined below

After receiving the common HUMINT feature layer atthe start of each trial in Task 6 human participants have achoice amongst four features (IMINT MOVINT SIGINT orSOCINT)Thenext choice is among three remaining featuresand the last choice is among two remaining features Thusthere are 4 lowast 3 lowast 2 = 24 possible sequences of choices thatmight be made by a subject on a given trial of Task 6 Foreach trial the percentage of subjects that chooses each of the24 sequences was computedThemodal sequence (maximumpercentage) was used to define a benchmark (119905 119904max) foreach trial (119905) where 119865 is the percentage of a sequence and119904max refers to the sequence with maximum 119865 for trial 119905 Foreach trial the model predicted a sequence of choices 119904modand the percentage value of 119865(119905 119904mod) for this sequence wascomputed from the human data In other words 119865(119905 119904mod) isthe percentage of humans that chose the same sequence as themodel chose on a given trial 119905

RMR (119905) =119865 (119905 119904mod)

119865 (119905 119904max) (10)

For example assume a model predicts a sequence ofchoices 119904mod on a trial of Task 6 Assume also that 20 ofhuman subjects chose the same sequence but a differentsequence was themost commonly chosen by human subjectsfor example by 40 of subjects In that case 119865(119905 119904mod) = 20and 119865(119905 119904max) = 40 so RMR(119905) = 2040 = 50

Finally a measure of resource allocation was derived byassigning a value (S2) based on the resources allocated to thecategory that was the ground truth Thus if a participant (ormodel) was assigned a resource allocation of A = 40B =30C = 20D = 10 and the ground truth was categoryB then the S2 score for that trial would be 30 Thus tomaximize the score an optimal model or participant wouldneed to assign 100 of their resources to the ground truth(ie adopt a winner-take-all strategy to resource allocation)

41 Data The integrated ACT-R AHAmodel performs (andlearns incrementally across) all 6 tasks using the same


Table 5 S1 S2 RSR (RMR for Task 6) and linear regression (1199032) scores broken down by task and layer

Task S1 score S2 score RSRRMRModel Human 119877

2 Model Human 1199032

1 781 687 929lowast 550 691 219 6502 682 537 313lowast 787 791 990lowast 7993 826 745 001 452 453 253lowast 5954-1 922 756 730lowast 7614-2 927 762 461lowast 477 440 510lowast 9065-1 966 681 037 8565-2 915 774 078 7765-3 853 698 115 7805-4 823 663 262lowast 404 452 637lowast 6186 912 910 867lowast 348 312 902lowast 788lowastP lt 01

knowledge constructs (production rules and chunks otherthan those it learns as part of executing the task) and param-eters The results comparing human and model performancepresented below are broken down by task and expandedon trial-by-trial and layer-by-layer analyses For instancewhile a similar instance-based representation is used acrossall tasks for probability adjustment resource allocation andlayer selection the output from the path planningmechanismis only used in Tasks 3 and 4 Thus it is best to examine Task3 and Task 4-1 (the first layer of Task 4) alone in determiningthe efficacy of the path planning mechanism

The model was run the same number of times asparticipants in the dataset (45) with the average modelresponse were compared to the average human performanceThe natural variability in the model (stochastic elementsinfluencing instance-based learning) approximates some ofthe individual differences of the human participants Whilethe average distribution of the ACT-R model is slightlypeakier than the average human (the ACT-R model is closerto Bayesian rational than humans are) the overall fits (basedon RSRRMR) are quite high with an overall score over 7(a score of 1 indicates a perfect fit [63 64] see Table 5) Inaddition a linear regression comparing model and humanperformance at each block of each layer indicates that themodel strongly and significantly predicts human behavior onAHA tasks

Supporting these results the trial-by-trial performance ofthe model (see Figure 11) predicted many of the variationsseen in usersrsquo dataWhile the ACT-Rmodel tended to behavemore rationally than human participants (ie the modelexhibited a higher S1 score) the model tended to capturemuch of the individual variation of humanparticipants acrosstrials (the S1 scores on Task 2 and S2 scores on Task 3 beingthe exceptions)

In addition to the fit to human data based on probabilities(S1RSR) and resource allocation (S2) the model was alsocompared to human participants in terms of the anchoringand adjustment and confirmation biases (see Figure 12)Whenever both the human behavioral data and modelexhibit a lower Negentropy than the Bayesian rational model

they are both exhibiting anchoring bias (and converselythey exhibit confirmation bias when they have a higherNegentropy) As shownbelow theACT-Rmodel significantlypredicts not only the presence or absence of a bias but also thequantity of the bias metric reflected in an overall 1198772 = 645for Negentropy scores across all tasks

411 Tasks 1 and 2 In Task 1 the ACT-R model producesa probability distribution and forced-choice resource allo-cation for each trial The probability distribution is basedon the blended probability adjustments using instance-basedlearning as described above and results in an increasedprevalence of anchoring (ie less peaky distributions) overthe normative solution in a manner similar to (yet strongerthan) human data

Similar to Task 1 in Task 2 the model follows the generaltrends of human participants for both S1 and especially S2scores With 4 groups to maintain in Task 2 we assume thatthere is more base-rate neglect in humans (which results inACT-R from the centroid representation of groups that losesbase-rate information) which increases theRSR score to 799However the 1198772 for S1 drops from 929 in Task 1 to 313 inTask 2 because the ACT-R model does not capture the sametrial-by-trial variability despite being closer to mean humanperformance

In Task 1 the ACT-Rmodel exhibited ameanNegentropyscore (119873

119872= 076) well below that of the Bayesian solution

(119873119876

= 511) thus there was an overall strong trendtowards anchoring and adjustment in learning (AL) for themodel Humans exhibited a similar AL bias (119873

119867= 113)

Additionally on a trial-by-trial comparison of the model tothe Bayesian solution both humans and the ACT-R modelshowed AL for each individual trial

In Task 2 the results were similar (119873119876= 791119873

119872= 206

119873119867= 113) with both the model and humans exhibiting

anchoring and adjustment in learning in every trial

412 Task 3 In Task 3 the ACT-R model was first requiredto generate category centroids based on a series of eventsand then was required to use the path planning mechanism


0

20

40

60

80

100

Task 1

1 2 3 4 5 6 7 8 9 10

1 2 3 4 50

20

40

60

80

100

Task 2

1 2 3 4 50

20

40

60

80

100

Task 3

1 2 3 4 5 6 7 8 9 1060708090

100708090

100

Task 4

1 2 3 4 5 6 7 8 9 1040

60

80

100708090

100

Task 5

Human S1 Model S1

(a)

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 50

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

Human S2 Model S2

(b)

Figure 11 (a) is the trial-by-trial (horizontal axis) fit between the ACT-R model and human data for Tasks 1ndash5 using the S1 metric (verticalaxis) which compares humans andmodel to Bayesian rational (b) is the fit for the S2metric determining resource allocation score For Tasks4-5 the top tile is the fit for the first feature layer and the bottom tile is the fit for the final feature layer


0

02

04

06

08

1

1 2 3 4 5 6 7

Task 1

8 9 10

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

1 2 3 4 5 6 7 8 9 100

01

02

02040608

1

03

RationalModelHuman

(a)

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04Task 5-1

Task 5-2

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-3

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

minus02

minus01

01

02

03

04

RationalModelHuman

(b)

Figure 12 Trial-by-trial Negentropy scores for Tasks 1ndash5 (Δ Negentropy between layers for Tasks 4-2 and 5) for the fully rational Bayesoutcome the ACT-R model and human participants Values less than normative (ie Bayesian rational) are considered an anchoring biasand values greater than normative are considered confirmation bias

to estimate the distance between each category centroid anda probe location While the model captured average humanperformance on the task it was not able to capture individualhuman behavior This was in part due to wide variabilityand negative skew in the raw human data and a difficulty inthe ACT-R model correctly placing category centroids whenevents fell across multiple roads

However when examining bias metrics the ACT-Rmodel exhibited both AL and confirmation biases as didhuman participants Both ACT-R and human participantsexhibited an AL bias on Trials 1 3 and 5 and confirmationbias on Trials 2 and 4 Overall both the model and humansexhibit a similar AL (119873

119876= 412119873

119872= 372 and119873

119867= 311)

Thus while the model was not capturing the exact distance


estimates of human participants it was able to capture thevariability in the bias metrics

413 Task 4 In Task 4 the issue with centroid generationover multiple roads is avoided since centroids are providedby the task environment resulting in aHUMINT layer RSR =761 and 1198772 = 730 Thus the path-planning mechanism itselfis functioning correctly and providing excellent fits to humandata In addition the model provided excellent fits to thesecond layer (the SOCINT layer) in Task 4 with an RSR fitof 905

Beginning with Task 4 layer 2 the measure of anchoringand adjustment (Delta Negentropy) is based on whethercategory probabilities were revised sufficiently by followingthe probabilistic decision rules There was an overall trendtowards anchoring and adjustment in both learning andinference with a slight trend towards confirmation bias forthe humans The main difference is when using SOCINTthe ACT-R model tends to exhibit an anchoring bias whilehuman participants tended to exhibit a confirmation biaswhen applying the SOCINT layer We speculate that thereason why humans would exhibit a confirmation bias onSOCINT which is the easiest of the rules to apply might bethat it has a compelling visual interpretation that participantsare more likely to trust

Also beginning with Task 4 resource allocation judg-ments are a distribution instead of a forced-choiceThemodellearns the outcomes of probability matching (PM) versuswinner-take-all (WTA forced-choice) through experience onTasks 1ndash3 in the formof IBL chunks From this experience themodel adopts a strategy (not a procedural rule but emergentfrom blended retrieval of chunks) that is somewhere betweenPM and WTA with a bias towards PM Based on the S2fits for Tasks 4ndash6 (see Table 5) the resource allocationmechanism which also relies on the same instance-basedlearning approach as the probability adjustment mechanismprovides an excellent match to human data

414 Task 5 In Task 5 the RSR fits for Layers 1ndash3 are quitehigh (856 776 and 780 resp) with some drop-off in Layer4 (618) due to human participantsrsquo distributions being closerto uniform and an RSR singularity (a near-uniform Bayesianhuman and model distribution leading to all nonperfect fitsreceiving a near-zero score since the random model near-perfect predicts human behavior) It may also be the casethat humans after getting several pieces of confirmatoryand disconfirmatory evidence express their uncertainty byflattening out their distribution in the final layer rather thanapplying each layer mechanically

As seen in the Delta Negentropy graphs for each layer(see Figure 12) ACT-R correctly predicts the overall trend ofanchoring (119873

119867lt 119873119876and119873

119872lt 119873119876) for each layer

Layer 1119873119902= 080119873

ℎ= 016119873

119898= minus007

Layer 2119873119902= 110119873

ℎ= 033119873

119898= 025

Layer 3119873119902= 138119873

ℎ= 056119873

119898= 024

Layer 4119873119902= 000119873

ℎ= minus007119873

119898= minus011

0

1

02

04

06

08

Prob

abili

ty

ACT-R modelHuman

IM-M

O-S

IIM

-MO

-SO

IM-S

O-M

OIM

-SO

-SI

MO

-IM

-SI

MO

-IM

-SO

MO

-SI-

IM

IM-S

I-M

OIM

-SI-

SO

SO-S

I-M

OSO

-SI-

IMSO

-MO

-SI

SO-M

O-I

M

SI-S

O-M

OSO

-IM

-MO

SO-I

M-S

I

SI-M

O-S

OSI

-SO

-IM

MO

-SO

-IM

SI-M

O-I

M

MO

-SI-

SO

MO

-SO

-SI

SI-I

M-M

OSI

-IM

-SO

Figure 13 Layer selection sequences both the ACT-R model andhuman data (IM for IMINT MO for MOVINT SI for SIGINT andSO for SOCINT)

Across each layer the model correctly predicts anchoringon all 10 trials of Layer 2 correctly predicts anchoring on8 trials of Layer 3 and correctly predicts the confirmationon the other 2 trials correctly predicts anchoring on 8 trialsof Layer 4 and correctly predicts confirmation on the other2 and correctly predicts anchoring on 4 trials of Layer 5and correctly predicts confirmation on 5 other trials Over40 possible trials ACT-R predicts human confirmation andanchoring biases on 39 of the trials (trial 10 of Layer 5 beingthe only exception)

415 Task 6 In Task 6 both the model and participantsare able to choose 3 feature layers before specifying a finalprobability distribution Figure 13 shows the probability dis-tribution of layer selection sequences for our ACT-R modeland human data To measure the similarity of the probabilitydistribution of layer selection sequences between the ACT-R model and human data we performed Jensen-Shannondivergence analysis which is a method of measuring thesimilarity between two distributionsThe divergence betweenthe two distributions is 35 indicating that the ACT-R modelstrongly predicts the human data patterns

5 Generalization

To determine the generalizability of the integrated ACT-Rmodel of the AHA tasks the same model that produced theabove results was run on novel stimuli in the same AHAframework The results of the model were then compared tothe results of a novel sample gathered from 103 students atPenn State University This new data set was not availablebefore the model was run and no parameters or knowledgestructures were changed to fit this data set Unlike theoriginal 45-participant dataset the Penn State sample usedonly people who had taken course credit towards a graduateGeospatial Intelligence Certificate Overall the RSR and 1198772


Table 6 Set of S1 S2 RSR (RMR in Task 6) and linear regression (1199032) scores broken down by task for novel dataset and participants



1 817 807 011 591 634 141 6252 685 789 347lowast 542 546 765lowast 5343 721 797 121 347 738 701lowast 6924 944 876 006 475 467 992lowast 8935 845 845 000 420 455 943lowast 8646 853 883 447lowast 484 446 990lowast 854lowastP lt 01

fits on S2 scores improved while the 1198772 fits on S1 scoresdropped (see Table 6) The increase in RSR was mainly dueto the Penn State population behaving more rationally (iehigher S1 scores see Figure 14) than the population from theinitial datasetThis is consistent with the increased educationand experience of the Penn State sample That said the PennState sample most likely utilized some different strategies insolving the tasks as the trial-by-trial S1 fits were not as closeimplying some difference in reasoning that the ACT-Rmodelwas not capturing

Overall the improved model fits indicate that the ACT-R model of the AHA tasks is able to capture average humanperformance at the task level for S1 scores and at the trial-by-trial level for S2 scores Conversely this justifies the reliabilityof the AHA tasks as a measure of human performance in asensemaking task environment

Finally the ACT-R model fits for anchoring and con-firmation biases (see Figure 15) were also similar in thePenn State dataset The model correctly predicted both thepresence and degree of anchoring on every block in Tasks 1ndash3and followed similar trial-by-trial trends for both anchoringand confirmation in Tasks 4-5 1198772 of Negentropy scores wasa similar 591 to the original dataset

6 Conclusion

The decision-making literature has established a lengthy listof cognitive biases under which human decision makingempirically deviates from the theoretical optimum Thosebiases have been studied in a number of theoretical (egbinary choice paradigms) and applied (egmedical diagnosisand intelligence analysis) settings However as the list ofbiases and experimental results grows our understandingof the mechanisms producing these biases has not followedpace Biases have been formulated in an ad hoc task- anddomain-specific manner Explanations have been proposedranging from the use of heuristic to innate individual pref-erences What is lacking is an explicit unified mechanisticand theoretical framework for cognitive biases that provides acomputational understanding of the conditions under whichthey arise and of themethods bywhich they can be overcome

In this paper we present such a framework by developingunified models of cognitive biases in a computational cogni-tive architecture Our approach unifies results along a pairof orthogonal dimensions First the cognitive architectureprovides a functional computational bridge from qualitative

theories of sensemaking to detailed neural models of brainfunctions Second the framework enables the integrationof results across paradigms from basic decision making toapplied fields Our basic hypothesis is that biases arise fromthe interaction of three components the task environmentincluding the information and feedback available as wellas constraints on task execution the cognitive architectureincluding cognitive mechanisms and their limitations andthe use of strategies including heuristic as well as formalremediation techniques This approach unifies explanationsgrounded in neurocognitive mechanisms with those assum-ing a primary role for heuristicThe goal is to derive a unifiedunderstanding of the conditions underwhich cognitive biasesappear as well as those under which they can be overcomeor unlearned To achieve this unification our model uses asmall set of architectural mechanisms leverages them using acoherent task modeling approach (ie instance-based learn-ing) performs a series of tasks using the same knowledgestructures and parameters generalizes across different sets ofscenarios andhumanparticipants and quantitatively predictsa number of cognitive biases on a trial-to-trial basis

In particular we showbiases to be prevalent under system1 (automatic) processes [65] and that a unified theory ofcognition can provide a principled way to understand howthese biases arise from basic cognitive and neural substratesAs system 2 (deliberative) processes make use of the samecognitive architecture mechanisms in implementing accessto knowledge and use of strategies we expect biases to alsooccur in particular as they relate to the interaction betweenthe information flow and the structure of the architectureHowever at the same time we show that system 2 processescan provide explicit means to remediate most effects ofthe biases such as in the centroid-of-centroid generationstrategy where a strong recency bias is replaced with an(slight) anchoring bias

Moreover it is to be expected that a rational agent learnsand adapts its strategies and knowledge its metacognitivecontrol (eg more deliberate processing of information) andits use of the task environment (eg using tools to performcomputations or serve as memory aids) so as to at leastreduce the deteriorating effects of these biases Howeverbiases are always subjective in that they refer to an implicitassumption about the true nature of the world For instancethe emergence of probability matching in the later tasks can


1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 1

1 2 3 4 5

Task 2

0

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

Task 3

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 4

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 5

Human S1 Model S1

(a)

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Human S2 Model S2

(b)

Figure 14 (a) is the trial-by-trial fit between the ACT-R model and human data for Tasks 1ndash5 using the S1 metric which compares humansand model to Bayesian rational (b) is the fit for the S2 metric determining resource allocation score For Tasks 4-5 the graph represents thefinal layer fit These results are for the final Penn State dataset


0

02

04

06

08

1

1 2 3 4 5 6 7 8 9 10

Task 1

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

02040608

1

minus04minus03minus02minus01

001

RationalModelHuman

1 2 3 4 5 6 7 8 9 10

(a)

00102030405

1 2 3 4 5 6 7 8 9 10

minus01

Task 5-1

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-2

Task 5-3

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

01

02

03

04

05

minus01

RationalModelHuman

(b)

Figure 15 Trial-by-trial Negentropy scores for Tasks 1ndash5 (Δ Negentropy between layers for Tasks 4-2 and 5) for the fully rational Bayesoutcome the ACT-R model and human participants These results are for the Penn State dataset

be seen as a response to the uncertainty of the earlier tasks andthe seemingly arbitrary nature of the outcomes Thus peoplerespond by hedging their bets a strategy that can be seen asbiased in a world of exact calculations but one that has shownits robust adaptivity in a range of real-world domains such asinvesting for retirement As is often the case bias is in the eyeof the beholder

Appendix

PROBs Handbook

Figures 16 17 18 19 and 20 were the individual pages of thePROBs (Probabilistic Decision Rules) handbook explaininghow to update category likelihoods based on the informationrevealed by each feature


50 10 15 20 25 30 35 40

5

0

10

15

20

25

30

35

Distance from group center along road network (miles)

Groupcenter

If a group attacks then the relative likelihood of attackdecreases as the distance from the group center increases

Prob

abili

ty o

f atta

ck (

)

(i) The likelihood of attack is about 40 at the group center(ii) The likelihood of attack is about 35 at 5 miles(iii) The likelihood of attack is about 25 at 10 miles(iv) The likelihood of attack is about 5 at 20 miles(v) The likelihood of attack is nearly 0 at 40 miles

HUMINT human intelligence

Figure 16TheHUMINT feature representing distance along a roadnetwork between a category and a probe event

If the attack is near agovernment buildingthen attack by A or B

is 4 times as likely asattack by C or D

If the attack is near amilitary building

then attack by C or Dis 4 times as likely as

attack by A or B

A B C D A B C D

IMINT image intelligence

Figure 17The IMINT feature representing imagery of governmentor military buildings located at a probe event location

Acknowledgments

Thiswork is supported by the IntelligenceAdvancedResearchProjects Activity (IARPA) via Department of the Interior(DOI) Contract no D10PC20021 The US Government is

MOVINT movement intelligence

If the attack is indense traffic thenattack by A or C is4 times as likely asattack by B or D

If the attack is insparse traffic thenattack by B or D is

attack by A or C4 times as likely as

A B C D A B C D

Figure 18TheMOVINT feature representing vehicular movementinformation located at a probe event location

A B C D

A B C DA B C DA B C DA B C D

A B C DA B C DA B C D

SIGINT SIGNAL intelligence

If SIGINT on a group reports chatter then attack by

If SIGINT on a group reports silence then attack by

Chatter Chatter Chatter Chatter

Silent Silent SilentSilent

that group is 7 times as likely as attack by each other group

that group is 13 as likely as attack by each other group

Figure 19 The SIGINT feature representing chatter located at aprobe event location Note that SIGINTmust be picked for a specificcategory

SOCINT sociocultural intelligence

Region A Region B Region C Region D

If the attack is in a grouprsquos region then attack by that group is2 times as likely as attack by each other group


Figure 20 The SOCINT feature representing sociocultural infor-mation about the region boundary for each category


authorized to reproduce and distribute reprints for Govern-mental purposes notwithstanding any copyright annotationthereon The views and conclusions contained hereon arethose of the authors and should not be interpreted asnecessarily representing the official policies or endorsementseither expressed or implied of IARPA DOI or the USGovernment

References

[1] J R AndersonHowCan the HumanMind Occur in the PhysicalUniverse Oxford University Press Oxford UK 2007

[2] J R Anderson D Bothell M D Byrne S Douglass C Lebiereand Y Qin ldquoAn integrated theory of the mindrdquo PsychologicalReview vol 111 no 4 pp 1036ndash1060 2004

[3] G Klein B Moon and R R Hoffman ldquoMaking sense of sense-making 1 alternative perspectivesrdquo IEEE Intelligent Systems vol21 no 4 pp 70ndash73 2006

[4] G Klein B Moon and R R Hoffman ldquoMaking sense of sense-making 2 a macrocognitive modelrdquo IEEE Intelligent Systemsvol 21 no 5 pp 88ndash92 2006

[5] P Pirolli and S K Card ldquoThe sensemaking process and leveragepoints for analyst technologyrdquo inProceedings of the InternationalConference on Intelligence Analysis McLean Va USA May2005

[6] D M Russell M J Stefik P Pirolli and S K Card ldquoThe coststructure of sensemakingrdquo in Proceedings of the INTERACT andCHI Conference on Human Factors in Computing Systems pp269ndash276 April 1993

[7] B Dervin ldquoFrom the mindrsquos eye of the user the sense-making of qualitative-quantitative methodologyrdquo in Sense-Making Methodology Reader Selected Writings of BrendaDervin B Dervin L Foreman-Wenet and E Lauterbach Edspp 269ndash292 Hampton Press Cresskill NJ USA 2003

[8] K Weick Sensemaking in Oragnizations Sage Thousand OaksCalif USA 1995

[9] R J Heuer Psychology of Intelligence Analysis Center for theStudy of Intelligence Washington DC USA 1999

[10] S G Hutchins P Pirolli and S K Card ldquoWhat makes intelli-gence analysis difficult A cognitive task analysis of intelligenceanalystsrdquo in Expertise Out of Context R R Hoffman Edpp 281ndash316 Lawrence Erlbaum Associates Mahwah NJ USA2007

[11] L G Militello and R J B Hutton ldquoApplied cognitive taskanalysis (ACTA) a practitionerrsquos toolkit for understandingcognitive task demandsrdquo Ergonomics vol 41 no 11 pp 1618ndash1641 1998

[12] J M Schraagen S F Chipman and V L Shalin Eds CognitiveTask Analysis Lawrence Erlbaum Associates Mahwah NJUSA 2000

[13] M I Posner R Goldsmith and K E Welton Jr ldquoPerceiveddistance and the classification of distorted patternsrdquo Journal ofExperimental Psychology vol 73 no 1 pp 28ndash38 1967

[14] C Lefebvre R Dellrsquoacqua P R Roelfsema and P JolicaeligurldquoSurfing the attentional waves during visual curve tracingevidence from the sustained posterior contralateral negativityrdquoPsychophysiology vol 48 no 11 pp 1510ndash1516 2011

[15] F G Ashby andW T Maddox ldquoComplex decision rules in cat-egorization contrasting novice and experienced performancerdquoJournal of Experimental Psychology vol 18 no 1 pp 50ndash71 1992

[16] A Stocco C Lebiere R C OrsquoReilly and J R Anderson ldquoTherole of the anterior prefrontal-basal ganglia circuit as a biolog-ical instruction interpreterrdquo in Biologically Inspired CognitiveArchitectures 2010 A V Samsonovich K R Johannsdottir AChella and B Goertzel Eds vol 221 of Frontiers in ArtificialIntelligence and Applications pp 153ndash162 2010

[17] G RyleThe Concept of Mind Hutchinson London UK 1949[18] G A Miller ldquoThe magical number seven plus or minus

two some limits on our capacity for processing informationrdquoPsychological Review vol 63 no 2 pp 81ndash97 1956

[19] H A Simon ldquoHow big is a chunkrdquo Science vol 183 no 4124pp 482ndash488 1974

[20] C Lebiere ldquoThe dynamics of cognition an ACT-R model ofcognitive arithmeticrdquo Kognitionswissenschaft vol 8 no 1 pp5ndash19 1999

[21] N Taatgen C Lebiere and J R Anderson ldquoModelingparadigms inACT-Rrdquo inCognition andMulti-Agent InteractionFrom Cognitive Modeling to Social Simulation R Sun EdCambridge University Press New York NY USA 2006

[22] S K Card T P Moran and A Newell The Psychology ofHuman-Computer Interaction Lawrence Erlbaum AssociatesHillsdale NJ USA 1983

[23] A Newell Unified Theories of Cognition Harvard UniversityPress Cambridge Mass USA 1990

[24] J R Anderson ldquoAcquisition of cognitive skillrdquo PsychologicalReview vol 89 no 4 pp 369ndash406 1982

[25] J RAndersonRules of theMind LawrenceErlbaumAssociatesHillsdale NJ USA 1993

[26] C Gonzalez J F Lerch and C Lebiere ldquoInstance-basedlearning in dynamic decision makingrdquo Cognitive Science vol27 no 4 pp 591ndash635 2003

[27] C Lebiere C Gonzalez and W Warwick ldquoMetacognition andmultiple strategies in a cognitive model of online controlrdquoJournal of Artificial General Intelligence vol 2 no 2 pp 20ndash372010

[28] R C OrsquoReilly T E Hazy and S A Herd ldquoThe leabra cognitivearchitecture how to play 20 principles with nature and winrdquo inOxford Handbook of Cognitive Science S Chipman Ed OxfordUniversity Press Oxford UK

[29] M Ziegler M Howard A Zaldivar et al ldquoSimulation ofanchoring bias in a spatial estimation task due to cholinergic-neuromodulationrdquo submitted

[30] Y Sun and H Wang ldquoThe parietal cortex in sensemakingspatio-attentional aspectsrdquo in press

[31] R Thomson and C Lebiere ldquoConstraining Bayesian inferencewith cognitive architectures an updated associative learningmechanism in ACT-Rrdquo in Proceedings of the 35th AnnualMeeting of the Cognitive Science Society (CogSci rsquo13) BerlinGermany July-August 2013

[32] C Lebiere and J R Anderson ldquoA connectionist implementationof the ACT-R production systemrdquo in Proceedings of the 15thAnnual Meeting of the Cognitive Science Society (CogSci rsquo93) pp635ndash640 Lawrence Erlbaum Associates June 1993

[33] D J Jilk C Lebiere R C OrsquoReilly and J R AndersonldquoSAL an explicitly pluralistic cognitive architecturerdquo Journal ofExperimental and Theoretical Artificial Intelligence vol 20 no3 pp 197ndash218 2008

[34] D Marr ldquoSimple memory a theory for archicortexrdquo Philosoph-ical Transactions of the Royal Society of London B vol 262 no841 pp 23ndash81 1971


[35] D E Rumelhart J L McClelland and PDP Research GroupParallel Distributed Processing Explorations in the Microstruc-ture of Cognition Vol 1 Foundations MIT Press CambridgeMass USA 1986

[36] C Lebiere J R Anderson and L M Reder ldquoError modelingin the ACT-R production systemrdquo in Proceedings of the 16thAnnual Meeting of the Cognitive Science Society pp 555ndash559Lawrence Erlbaum Associates 1994

[37] J R Anderson The Adaptive Character of Thought LawrenceErlbaum Associates 1990

[38] A Klippel H Tappe and C Habel ldquoPictorial representationsof routes chunking route segments during comprehensionrdquo inSpatial Cognition III C Freksa W Brauer C Habel and K FWender Eds vol 2685 of Lecture Notes in Computer Sciencepp 11ndash33 2003

[39] S M Kosslyn Image and Brain The Resolution of the ImageryDebate MIT Press Cambridge Mass USA 1994

[40] W C Gogel and J A da Silva ldquoA two-process theory of theresponse to size and distancerdquo Perception amp Psychophysics vol41 no 3 pp 220ndash238 1987

[41] W M Wiest and B Bell ldquoStevensrsquos exponent for psychophys-ical scaling of perceived remembered and inferred distancerdquoPsychological Bulletin vol 98 no 3 pp 457ndash470 1985

[42] J A da Silva ldquoScales for perceived egocentric distance in a largeopen field comparison of three psychophysical methodsrdquo TheAmerican Journal of Psychology vol 98 no 1 pp 119ndash144 1985

[43] J A Aznar-Casanova E H Matsushima N P Ribeiro-Filhoand J A da Silva ldquoOne-dimensional and multi-dimensionalstudies of the exocentric distance estimates in frontoparallelplane virtual space and outdoor open fieldrdquo The SpanishJournal of Psychology vol 9 no 2 pp 273ndash284 2006

[44] C A Levin and R N Haber ldquoVisual angle as a determinant ofperceived interobject distancerdquo Perception amp Psychophysics vol54 no 2 pp 250ndash259 1993

[45] R Thomson The role of object size on judgments of lateralseparation [PhD dissertation]

[46] S Dehaene and L Cohen ldquoLanguage and elementary arith-metic dissociations between operationsrdquo Brain and Languagevol 69 no 3 pp 492ndash494 1999

[47] C Lebiere C Gonzalez andMMartin ldquoInstance-based deci-sion making model of repeated binary choicerdquo in Proceedings ofthe 8th International Conference on Cognitive Modeling (ICCMrsquo07) Ann Arbor Mich USA July 2007

[48] I Erev E Ert A E Roth et al ldquoA choice prediction competitionchoices from experience and from descriptionrdquo Journal ofBehavioral Decision Making vol 23 no 1 pp 15ndash47 2010

[49] DWallach and C Lebiere ldquoConscious and unconscious knowl-edge mapping to the symbolic and subsymbolic levels of ahybrid architecturerdquo in Attention and Implicit Learning LJimenez Ed John Benjamins Publishing Amsterdam Nether-lands 2003

[50] C Lebiere C Gonzalez and W Warwick ldquoA comparativeapproach to understanding general intelligence predictingcognitive performance in an open-ended dynamic taskrdquo in Pro-ceedings of the 2nd Conference on Artificial General Intelligence(AGI rsquo09) pp 103ndash107 Arlington Va USA March 2009

[51] J Klayman ldquoVarieties of confirmation biasrdquo inDecisionMakingfrom a Cognitive Perspective J Busemeyer R Hastie and D LMedin Eds vol 32 of Psychology of Learning and Motivationpp 365ndash418 Academic Press New York NY USA 1995

[52] J Klayman and Y-W Ha ldquoConfirmation disconfirmation andinformation in hypothesis testingrdquoPsychological Review vol 94no 2 pp 211ndash228 1987

[53] R S Nickerson ldquoConfirmation bias a ubiquitous phenomenonin many guisesrdquo Review of General Psychology vol 2 no 2 pp175ndash220 1998

[54] A Tversky and D Kahneman ldquoJudgment under uncertaintyheuristics and biasesrdquo Science vol 185 no 4157 pp 1124ndash11311974

[55] P Wason ldquoOn the failure to eliminate hypotheses in a concep-tual taskrdquoTheQuarterly Journal of Experimental Psychology vol12 no 3 pp 129ndash140 1960

[56] C D Wickens and J G Hollands Engineering Psychology andHuman Performance Prentice Hall Upper Saddle River NJUSA 3rd edition 2000

[57] B A Cheikes M J Brown P E Lehner and L AdelmanldquoConfirmation bias in complex analysesrdquo Tech Rep MITRECenter for Integrated Intelligence Systems BedfordMass USA2004

[58] G Convertino D Billman P Pirolli J P Massar and JShrager ldquoCollaborative intelligence analysis with CACHE biasreduction and information coveragerdquo Tech Rep Palo AltoResearch Center Palo Alto Calif USA 2006

[59] M A Tolcott F F Marvin and P E Lehner ldquoExpert decision-making in evolving situationsrdquo IEEE Transactions on SystemsMan and Cybernetics vol 19 no 3 pp 606ndash615 1989

[60] C Grabo and J Goldman Anticipating Surprise Rowman ampLittlefield 2004

[61] J R Anderson and C Lebiere The Atomic Components ofThought Lawrence Erlbaum Associates Mahwah NJ USA1998

[62] K Burns ldquoMental models and normal errorsrdquo in How Pro-fessionals Make Decision H Montgomery R Lipshitz andB Brehmer Eds pp 15ndash28 Lawrence Erlbaum AssociatesMahwah NJ USA 2004

[63] MITRE Technical Report ldquoIARPArsquos ICArUS program phase 1challenge problem design and test specificationrdquo in progress

[64] MITRE Technical Report ldquoA computational basis for ICArUSchallenge problem designrdquo in progress

[65] D Kahneman and S Frederick ldquoA model of heuristic judg-mentrdquo inThe Cambridge Handbook of Thinking and ReasoningK J Holyoak and R GMorrison Eds pp 267ndash293 CambridgeUniversity Press New York NY USA 2005

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Distributed Sensor Networks


Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014


ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014


Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications


Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia


Biomedical Imaging


ArtificialNeural Systems

Advances in


RoboticsJournal of



Computational Intelligence and Neuroscience

Industrial EngineeringJournal of


Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in



Data Framerecognize data defines what counts

from world as data organizes

Elaboration cycle Reframing cycle

Reframe- Compare frames- Seek a new frame

Elaborate a frame Question a frame- Detect inconsistencies- Track anomalies- Judge data quality

- Add and fill slots- Discover data- Infer new

- Discard datarelationships

Preserve

Figure 1 The Data-Frame model of sensemaking Image repro-duced by Klein et al [4]

exhibit several cognitive biases including confirmation biasanchoring and adjustment probability matching and base-rate neglect Those biases are not engineered in the modelbut rather result from the interaction of the structure andstatistics of the task the structure and mechanisms of ourcognitive architecture and the strategies that we select toperform the former using the latter

Figure 1 presents the DataFrame theory of sensemaking[3] The DataFrame theory assumes that meaningful mentalrepresentations called frames define what counts as data andhow those data are structured for mental processing [4]A similar conceptual model was employed in Pirolli andCard [5] to perform a cognitive task analysis of intelligenceanalysis [10ndash12] Frames can be expressed in a variety offorms including stories maps organizational diagrams orscripts Whereas frames define and shape data new data canevoke changes to frames In this framework sensemakingcan involve elaboration of a frame (eg filling in details)questioning a frame (eg due to the detection of anomalies)or reframing (eg rejecting a frame and replacing it withanother) The DataFrame theory proposes that backward-looking processes are involved in forming mental modelsthat explain past events and forward-looking mental sim-ulations are involved in predicting how future events willunfold We describe how frames can be represented in acognitive architecture and how the architectural mechanismscan implement general sensemaking processes We thendemonstrate how the dynamics of sensemaking processes ina cognitive architecture can give rise to cognitive biases in anemergent way

The structure of this paper is as follows Section 2 definesthe AHA (Abducting Hotspots of Activity) experiment con-sisting of a suite of six sensemaking tasks of increasing com-plexity Section 3 outlines our cognitive modeling approachto sensemaking it describes the ACT-R architecture how itis used to prototype neuralmodels which cognitive functionscompose the model and how four cognitive biases can beaccounted for by the model Section 4 presents the measuresused to assess the cognitive biases in the AHA frameworkand then compares human and model results Section 5presents a test of the modelrsquos generalization on a data set

that was unavailable at the time of initial model developmentFinally Section 6 summarizes our account of cognitive biasescentered around themechanisms and limitations of cognitivearchitectures the heuristic that these mechanisms use toadapt to the structure of the task and their interaction withthe task environment

2 The Task Environment

The AHA experiment consists of a series of six tasksdeveloped as part of the IARPA (Intelligence AdvancedResearch Projects Activity) ICArUS (Integrated Cognitive-neuroscience Architectures for the Understanding of Sense-making) program whose goal is to drive the developmentof integrated neurocognitive models of heuristic and biasesin decision-making in the context of intelligence analysisThe AHA tasks can be subdivided into two classes the firstfocusing on learning the statistical patterns of events locatedon a map-like layout and generating probability distribu-tions of category membership based on the spatial locationand frequency of these events (Tasks 1ndash3) and the secondrequiring the application of probabilistic decision rules aboutdifferent features displayed on similar map-like layouts inorder to generate and revise probability distributions ofcategory membership (Tasks 4ndash6)

The AHA tasks simulate the analysis of artificial geospa-tial data presented in a manner consistent with and informedby current intelligence doctrine (Geospatial IntelligenceBasic Doctrine httpwwwfasorgirpagencyngadoctrinepdf) The tasks involve the presentation of multiple featuresconsistent with intelligence data which are presented in aGIS(Geographic Information System) display not unlike Googlemaps (httpsmapsgooglecom) These features include

HUMINT information collected by human sourcessuch as detecting the location of eventsIMINT information collected from imagery of build-ings roads and terrain elementsMOVINT analysis of moving objects such as trafficdensitySIGINT analysis of signals and communicationsSOCINT analysis of social customs and attitudes ofpeople communities and culture

The display (see Figure 2) includes access to the missiontutorial and instructions (the top-right corner) a legendto understand the symbols on the map (the left pane) themap (the center pane) and participantsrsquo current and pastresponses (the right pane)

For Tasks 1ndash3 the flow of an average trial proceedsaccording to the following general outline First participantsperceive a series of events (SIGACTs SIGnals of ACTivity)labeled according to which category the event belongedCategorieswere both color- and shape-codedwith the appro-priate label AquaBromineCitrine or Diamond listed inthe legend After perceiving the series of events a probe eventis displayed (represented as a ldquordquo on the display) Participantswere asked to generate a center of activity (eg prototype)



legend (bottom)









Features Rules


IMINT


MOVINT


SIGINT


SOCINT















































Environment

Prod

uctio

ns(b

asal

gan

glia

)


Matching (striatum)



Goal buffer








(DLPFC)







Activation 119860119894= 119861119894+ 119878119894+ 119875119894+ 120576119894






Base level 119861119894= ln(

119899

sum119895=1

119905minus119889

119895) + 120573

119894



sum119895

119882119896119895119878119895119894



119878119895119894= 119878 minus ln (fan

119895119894)



119875119872119896119894




119890119860119894119904

sum119895119890119860119895119904





119875119894(1 minus Sim

119894119895)2 119875



Utility learning

119880119894(119899) = 119880

119894(119899 minus 1) + 120572 [119877

119894(119899) minus 119880

119894(119899 minus 1)]




119875119894=

119890119880119894119904

sum119895119890119880119895119904

















































difference

119875119894=

2 sdot10038161003816100381610038161198891 minus 1198892

1003816100381610038161003816















) + CrowFlies 82

(2)



0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability


RationalModel

(a)

0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability


RationalModel

(b)









119875119894= 2 lowast

10038161003816100381610038161003816119872119896minus119872119895

10038161003816100381610038161003816 (3)



119895is the



















119901119894(1 minus 119901

119894) (4)


119894= 1)








Confirmation bias


































4 Data and Results



119873 =(119864max minus 119864)

119864max (5)



119875ℎlowast log2119875ℎ (6)









119867lt 119873119876 then the





119870119875119872= 119864119875119872minus 119864119875

= minussumℎ



(7)




to note that 119870119875119872





119878 = 100 lowast 2minus119870 (8)


119876119875and 119878119876119872







RSR =(119878119875119872minus 119878119875119877)

(100 minus 119878119875119877) (9)












RMR (119905) =119865 (119905 119904mod)

119865 (119905 119904max) (10)


















(119873119876


119867= 113)



119872= 206





0

20

40

60

80

100

Task 1

1 2 3 4 5 6 7 8 9 10

1 2 3 4 50

20

40

60

80

100

Task 2

1 2 3 4 50

20

40

60

80

100

Task 3

1 2 3 4 5 6 7 8 9 1060708090

100708090

100

Task 4

1 2 3 4 5 6 7 8 9 1040

60

80

100708090

100

Task 5

Human S1 Model S1

(a)

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 50

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7

Task 1

8 9 10

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

1 2 3 4 5 6 7 8 9 100

01

02

02040608

1

03

RationalModelHuman

(a)

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04Task 5-1

Task 5-2

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-3

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

minus02

minus01

01

02

03

04

RationalModelHuman

(b)




119876= 412119873

119872= 372 and119873

119867= 311)









119867lt 119873119876and119873

119872lt 119873119876) for each layer

Layer 1119873119902= 080119873

ℎ= 016119873

119898= minus007

Layer 2119873119902= 110119873

ℎ= 033119873

119898= 025

Layer 3119873119902= 138119873

ℎ= 056119873

119898= 024

Layer 4119873119902= 000119873

ℎ= minus007119873

119898= minus011

0

1

02

04

06

08

Prob

abili

ty

ACT-R modelHuman

IM-M

O-S

IIM

-MO

-SO

IM-S

O-M

OIM

-SO

-SI

MO

-IM

-SI

MO

-IM

-SO

MO

-SI-

IM

IM-S

I-M

OIM

-SI-

SO

SO-S

I-M

OSO

-SI-

IMSO

-MO

-SI

SO-M

O-I

M

SI-S

O-M

OSO

-IM

-MO

SO-I

M-S

I

SI-M

O-S

OSI

-SO

-IM

MO

-SO

-IM

SI-M

O-I

M

MO

-SI-

SO

MO

-SO

-SI

SI-I

M-M

OSI

-IM

-SO




5 Generalization










6 Conclusion







1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 1

1 2 3 4 5

Task 2

0

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

Task 3

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 4

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 5

Human S1 Model S1

(a)

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7 8 9 10

Task 1

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

02040608

1


001

RationalModelHuman

1 2 3 4 5 6 7 8 9 10

(a)

00102030405

1 2 3 4 5 6 7 8 9 10

minus01

Task 5-1

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-2

Task 5-3

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

01

02

03

04

05

minus01

RationalModelHuman

(b)



Appendix

PROBs Handbook



50 10 15 20 25 30 35 40

5

0

10

15

20

25

30

35


Groupcenter


Prob

abili

ty o

f atta

ck (

)








attack by A or B

A B C D A B C D



Acknowledgments






A B C D A B C D


A B C D


















References










































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in





legend (bottom)









Features Rules


IMINT


MOVINT


SIGINT


SOCINT















































Environment

Prod

uctio

ns(b

asal

gan

glia

)


Matching (striatum)



Goal buffer








(DLPFC)







Activation 119860119894= 119861119894+ 119878119894+ 119875119894+ 120576119894






Base level 119861119894= ln(

119899

sum119895=1

119905minus119889

119895) + 120573

119894



sum119895

119882119896119895119878119895119894



119878119895119894= 119878 minus ln (fan

119895119894)



119875119872119896119894




119890119860119894119904

sum119895119890119860119895119904





119875119894(1 minus Sim

119894119895)2 119875



Utility learning

119880119894(119899) = 119880

119894(119899 minus 1) + 120572 [119877

119894(119899) minus 119880

119894(119899 minus 1)]




119875119894=

119890119880119894119904

sum119895119890119880119895119904

















































difference

119875119894=

2 sdot10038161003816100381610038161198891 minus 1198892

1003816100381610038161003816















) + CrowFlies 82

(2)



0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability


RationalModel

(a)

0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability


RationalModel

(b)









119875119894= 2 lowast

10038161003816100381610038161003816119872119896minus119872119895

10038161003816100381610038161003816 (3)



119895is the



















119901119894(1 minus 119901

119894) (4)


119894= 1)








Confirmation bias


































4 Data and Results



119873 =(119864max minus 119864)

119864max (5)



119875ℎlowast log2119875ℎ (6)









119867lt 119873119876 then the





119870119875119872= 119864119875119872minus 119864119875

= minussumℎ



(7)




to note that 119870119875119872





119878 = 100 lowast 2minus119870 (8)


119876119875and 119878119876119872







RSR =(119878119875119872minus 119878119875119877)

(100 minus 119878119875119877) (9)












RMR (119905) =119865 (119905 119904mod)

119865 (119905 119904max) (10)


















(119873119876


119867= 113)



119872= 206





0

20

40

60

80

100

Task 1

1 2 3 4 5 6 7 8 9 10

1 2 3 4 50

20

40

60

80

100

Task 2

1 2 3 4 50

20

40

60

80

100

Task 3

1 2 3 4 5 6 7 8 9 1060708090

100708090

100

Task 4

1 2 3 4 5 6 7 8 9 1040

60

80

100708090

100

Task 5

Human S1 Model S1

(a)

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 50

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7

Task 1

8 9 10

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

1 2 3 4 5 6 7 8 9 100

01

02

02040608

1

03

RationalModelHuman

(a)

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04Task 5-1

Task 5-2

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-3

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

minus02

minus01

01

02

03

04

RationalModelHuman

(b)




119876= 412119873

119872= 372 and119873

119867= 311)









119867lt 119873119876and119873

119872lt 119873119876) for each layer

Layer 1119873119902= 080119873

ℎ= 016119873

119898= minus007

Layer 2119873119902= 110119873

ℎ= 033119873

119898= 025

Layer 3119873119902= 138119873

ℎ= 056119873

119898= 024

Layer 4119873119902= 000119873

ℎ= minus007119873

119898= minus011

0

1

02

04

06

08

Prob

abili

ty

ACT-R modelHuman

IM-M

O-S

IIM

-MO

-SO

IM-S

O-M

OIM

-SO

-SI

MO

-IM

-SI

MO

-IM

-SO

MO

-SI-

IM

IM-S

I-M

OIM

-SI-

SO

SO-S

I-M

OSO

-SI-

IMSO

-MO

-SI

SO-M

O-I

M

SI-S

O-M

OSO

-IM

-MO

SO-I

M-S

I

SI-M

O-S

OSI

-SO

-IM

MO

-SO

-IM

SI-M

O-I

M

MO

-SI-

SO

MO

-SO

-SI

SI-I

M-M

OSI

-IM

-SO




5 Generalization










6 Conclusion







1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 1

1 2 3 4 5

Task 2

0

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

Task 3

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 4

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 5

Human S1 Model S1

(a)

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7 8 9 10

Task 1

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

02040608

1


001

RationalModelHuman

1 2 3 4 5 6 7 8 9 10

(a)

00102030405

1 2 3 4 5 6 7 8 9 10

minus01

Task 5-1

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-2

Task 5-3

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

01

02

03

04

05

minus01

RationalModelHuman

(b)



Appendix

PROBs Handbook



50 10 15 20 25 30 35 40

5

0

10

15

20

25

30

35


Groupcenter


Prob

abili

ty o

f atta

ck (

)








attack by A or B

A B C D A B C D



Acknowledgments






A B C D A B C D


A B C D


















References










































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in





Features Rules


IMINT


MOVINT


SIGINT


SOCINT















































Environment

Prod

uctio

ns(b

asal

gan

glia

)


Matching (striatum)



Goal buffer








(DLPFC)







Activation 119860119894= 119861119894+ 119878119894+ 119875119894+ 120576119894






Base level 119861119894= ln(

119899

sum119895=1

119905minus119889

119895) + 120573

119894



sum119895

119882119896119895119878119895119894



119878119895119894= 119878 minus ln (fan

119895119894)



119875119872119896119894




119890119860119894119904

sum119895119890119860119895119904





119875119894(1 minus Sim

119894119895)2 119875



Utility learning

119880119894(119899) = 119880

119894(119899 minus 1) + 120572 [119877

119894(119899) minus 119880

119894(119899 minus 1)]




119875119894=

119890119880119894119904

sum119895119890119880119895119904

















































difference

119875119894=

2 sdot10038161003816100381610038161198891 minus 1198892

1003816100381610038161003816















) + CrowFlies 82

(2)



0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability


RationalModel

(a)

0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability


RationalModel

(b)









119875119894= 2 lowast

10038161003816100381610038161003816119872119896minus119872119895

10038161003816100381610038161003816 (3)



119895is the



















119901119894(1 minus 119901

119894) (4)


119894= 1)








Confirmation bias


































4 Data and Results



119873 =(119864max minus 119864)

119864max (5)



119875ℎlowast log2119875ℎ (6)









119867lt 119873119876 then the





119870119875119872= 119864119875119872minus 119864119875

= minussumℎ



(7)




to note that 119870119875119872





119878 = 100 lowast 2minus119870 (8)


119876119875and 119878119876119872







RSR =(119878119875119872minus 119878119875119877)

(100 minus 119878119875119877) (9)












RMR (119905) =119865 (119905 119904mod)

119865 (119905 119904max) (10)


















(119873119876


119867= 113)



119872= 206





0

20

40

60

80

100

Task 1

1 2 3 4 5 6 7 8 9 10

1 2 3 4 50

20

40

60

80

100

Task 2

1 2 3 4 50

20

40

60

80

100

Task 3

1 2 3 4 5 6 7 8 9 1060708090

100708090

100

Task 4

1 2 3 4 5 6 7 8 9 1040

60

80

100708090

100

Task 5

Human S1 Model S1

(a)

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 50

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7

Task 1

8 9 10

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

1 2 3 4 5 6 7 8 9 100

01

02

02040608

1

03

RationalModelHuman

(a)

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04Task 5-1

Task 5-2

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-3

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

minus02

minus01

01

02

03

04

RationalModelHuman

(b)




119876= 412119873

119872= 372 and119873

119867= 311)









119867lt 119873119876and119873

119872lt 119873119876) for each layer

Layer 1119873119902= 080119873

ℎ= 016119873

119898= minus007

Layer 2119873119902= 110119873

ℎ= 033119873

119898= 025

Layer 3119873119902= 138119873

ℎ= 056119873

119898= 024

Layer 4119873119902= 000119873

ℎ= minus007119873

119898= minus011

0

1

02

04

06

08

Prob

abili

ty

ACT-R modelHuman

IM-M

O-S

IIM

-MO

-SO

IM-S

O-M

OIM

-SO

-SI

MO

-IM

-SI

MO

-IM

-SO

MO

-SI-

IM

IM-S

I-M

OIM

-SI-

SO

SO-S

I-M

OSO

-SI-

IMSO

-MO

-SI

SO-M

O-I

M

SI-S

O-M

OSO

-IM

-MO

SO-I

M-S

I

SI-M

O-S

OSI

-SO

-IM

MO

-SO

-IM

SI-M

O-I

M

MO

-SI-

SO

MO

-SO

-SI

SI-I

M-M

OSI

-IM

-SO




5 Generalization










6 Conclusion







1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 1

1 2 3 4 5

Task 2

0

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

Task 3

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 4

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 5

Human S1 Model S1

(a)

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7 8 9 10

Task 1

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

02040608

1


001

RationalModelHuman

1 2 3 4 5 6 7 8 9 10

(a)

00102030405

1 2 3 4 5 6 7 8 9 10

minus01

Task 5-1

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-2

Task 5-3

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

01

02

03

04

05

minus01

RationalModelHuman

(b)



Appendix

PROBs Handbook



50 10 15 20 25 30 35 40

5

0

10

15

20

25

30

35


Groupcenter


Prob

abili

ty o

f atta

ck (

)








attack by A or B

A B C D A B C D



Acknowledgments






A B C D A B C D


A B C D


















References










































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in






































Environment

Prod

uctio

ns(b

asal

gan

glia

)


Matching (striatum)



Goal buffer








(DLPFC)







Activation 119860119894= 119861119894+ 119878119894+ 119875119894+ 120576119894






Base level 119861119894= ln(

119899

sum119895=1

119905minus119889

119895) + 120573

119894



sum119895

119882119896119895119878119895119894



119878119895119894= 119878 minus ln (fan

119895119894)



119875119872119896119894




119890119860119894119904

sum119895119890119860119895119904





119875119894(1 minus Sim

119894119895)2 119875



Utility learning

119880119894(119899) = 119880

119894(119899 minus 1) + 120572 [119877

119894(119899) minus 119880

119894(119899 minus 1)]




119875119894=

119890119880119894119904

sum119895119890119880119895119904

















































difference

119875119894=

2 sdot10038161003816100381610038161198891 minus 1198892

1003816100381610038161003816















) + CrowFlies 82

(2)



0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability


RationalModel

(a)

0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability


RationalModel

(b)









119875119894= 2 lowast

10038161003816100381610038161003816119872119896minus119872119895

10038161003816100381610038161003816 (3)



119895is the



















119901119894(1 minus 119901

119894) (4)


119894= 1)








Confirmation bias


































4 Data and Results



119873 =(119864max minus 119864)

119864max (5)



119875ℎlowast log2119875ℎ (6)









119867lt 119873119876 then the





119870119875119872= 119864119875119872minus 119864119875

= minussumℎ



(7)




to note that 119870119875119872





119878 = 100 lowast 2minus119870 (8)


119876119875and 119878119876119872







RSR =(119878119875119872minus 119878119875119877)

(100 minus 119878119875119877) (9)












RMR (119905) =119865 (119905 119904mod)

119865 (119905 119904max) (10)


















(119873119876


119867= 113)



119872= 206





0

20

40

60

80

100

Task 1

1 2 3 4 5 6 7 8 9 10

1 2 3 4 50

20

40

60

80

100

Task 2

1 2 3 4 50

20

40

60

80

100

Task 3

1 2 3 4 5 6 7 8 9 1060708090

100708090

100

Task 4

1 2 3 4 5 6 7 8 9 1040

60

80

100708090

100

Task 5

Human S1 Model S1

(a)

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 50

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7

Task 1

8 9 10

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

1 2 3 4 5 6 7 8 9 100

01

02

02040608

1

03

RationalModelHuman

(a)

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04Task 5-1

Task 5-2

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-3

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

minus02

minus01

01

02

03

04

RationalModelHuman

(b)




119876= 412119873

119872= 372 and119873

119867= 311)









119867lt 119873119876and119873

119872lt 119873119876) for each layer

Layer 1119873119902= 080119873

ℎ= 016119873

119898= minus007

Layer 2119873119902= 110119873

ℎ= 033119873

119898= 025

Layer 3119873119902= 138119873

ℎ= 056119873

119898= 024

Layer 4119873119902= 000119873

ℎ= minus007119873

119898= minus011

0

1

02

04

06

08

Prob

abili

ty

ACT-R modelHuman

IM-M

O-S

IIM

-MO

-SO

IM-S

O-M

OIM

-SO

-SI

MO

-IM

-SI

MO

-IM

-SO

MO

-SI-

IM

IM-S

I-M

OIM

-SI-

SO

SO-S

I-M

OSO

-SI-

IMSO

-MO

-SI

SO-M

O-I

M

SI-S

O-M

OSO

-IM

-MO

SO-I

M-S

I

SI-M

O-S

OSI

-SO

-IM

MO

-SO

-IM

SI-M

O-I

M

MO

-SI-

SO

MO

-SO

-SI

SI-I

M-M

OSI

-IM

-SO




5 Generalization










6 Conclusion







1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 1

1 2 3 4 5

Task 2

0

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

Task 3

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 4

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 5

Human S1 Model S1

(a)

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7 8 9 10

Task 1

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

02040608

1


001

RationalModelHuman

1 2 3 4 5 6 7 8 9 10

(a)

00102030405

1 2 3 4 5 6 7 8 9 10

minus01

Task 5-1

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-2

Task 5-3

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

01

02

03

04

05

minus01

RationalModelHuman

(b)



Appendix

PROBs Handbook



50 10 15 20 25 30 35 40

5

0

10

15

20

25

30

35


Groupcenter


Prob

abili

ty o

f atta

ck (

)








attack by A or B

A B C D A B C D



Acknowledgments






A B C D A B C D


A B C D


















References










































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in






Activation 119860119894= 119861119894+ 119878119894+ 119875119894+ 120576119894






Base level 119861119894= ln(

119899

sum119895=1

119905minus119889

119895) + 120573

119894



sum119895

119882119896119895119878119895119894



119878119895119894= 119878 minus ln (fan

119895119894)



119875119872119896119894




119890119860119894119904

sum119895119890119860119895119904





119875119894(1 minus Sim

119894119895)2 119875



Utility learning

119880119894(119899) = 119880

119894(119899 minus 1) + 120572 [119877

119894(119899) minus 119880

119894(119899 minus 1)]




119875119894=

119890119880119894119904

sum119895119890119880119895119904

















































difference

119875119894=

2 sdot10038161003816100381610038161198891 minus 1198892

1003816100381610038161003816















) + CrowFlies 82

(2)



0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability


RationalModel

(a)

0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability


RationalModel

(b)









119875119894= 2 lowast

10038161003816100381610038161003816119872119896minus119872119895

10038161003816100381610038161003816 (3)



119895is the



















119901119894(1 minus 119901

119894) (4)


119894= 1)








Confirmation bias


































4 Data and Results



119873 =(119864max minus 119864)

119864max (5)



119875ℎlowast log2119875ℎ (6)









119867lt 119873119876 then the





119870119875119872= 119864119875119872minus 119864119875

= minussumℎ



(7)




to note that 119870119875119872





119878 = 100 lowast 2minus119870 (8)


119876119875and 119878119876119872







RSR =(119878119875119872minus 119878119875119877)

(100 minus 119878119875119877) (9)












RMR (119905) =119865 (119905 119904mod)

119865 (119905 119904max) (10)


















(119873119876


119867= 113)



119872= 206





0

20

40

60

80

100

Task 1

1 2 3 4 5 6 7 8 9 10

1 2 3 4 50

20

40

60

80

100

Task 2

1 2 3 4 50

20

40

60

80

100

Task 3

1 2 3 4 5 6 7 8 9 1060708090

100708090

100

Task 4

1 2 3 4 5 6 7 8 9 1040

60

80

100708090

100

Task 5

Human S1 Model S1

(a)

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 50

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7

Task 1

8 9 10

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

1 2 3 4 5 6 7 8 9 100

01

02

02040608

1

03

RationalModelHuman

(a)

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04Task 5-1

Task 5-2

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-3

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

minus02

minus01

01

02

03

04

RationalModelHuman

(b)




119876= 412119873

119872= 372 and119873

119867= 311)









119867lt 119873119876and119873

119872lt 119873119876) for each layer

Layer 1119873119902= 080119873

ℎ= 016119873

119898= minus007

Layer 2119873119902= 110119873

ℎ= 033119873

119898= 025

Layer 3119873119902= 138119873

ℎ= 056119873

119898= 024

Layer 4119873119902= 000119873

ℎ= minus007119873

119898= minus011

0

1

02

04

06

08

Prob

abili

ty

ACT-R modelHuman

IM-M

O-S

IIM

-MO

-SO

IM-S

O-M

OIM

-SO

-SI

MO

-IM

-SI

MO

-IM

-SO

MO

-SI-

IM

IM-S

I-M

OIM

-SI-

SO

SO-S

I-M

OSO

-SI-

IMSO

-MO

-SI

SO-M

O-I

M

SI-S

O-M

OSO

-IM

-MO

SO-I

M-S

I

SI-M

O-S

OSI

-SO

-IM

MO

-SO

-IM

SI-M

O-I

M

MO

-SI-

SO

MO

-SO

-SI

SI-I

M-M

OSI

-IM

-SO




5 Generalization










6 Conclusion







1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 1

1 2 3 4 5

Task 2

0

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

Task 3

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 4

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 5

Human S1 Model S1

(a)

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7 8 9 10

Task 1

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

02040608

1


001

RationalModelHuman

1 2 3 4 5 6 7 8 9 10

(a)

00102030405

1 2 3 4 5 6 7 8 9 10

minus01

Task 5-1

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-2

Task 5-3

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

01

02

03

04

05

minus01

RationalModelHuman

(b)



Appendix

PROBs Handbook



50 10 15 20 25 30 35 40

5

0

10

15

20

25

30

35


Groupcenter


Prob

abili

ty o

f atta

ck (

)








attack by A or B

A B C D A B C D



Acknowledgments






A B C D A B C D


A B C D


















References










































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in













































difference

119875119894=

2 sdot10038161003816100381610038161198891 minus 1198892

1003816100381610038161003816















) + CrowFlies 82

(2)



0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability


RationalModel

(a)

0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability


RationalModel

(b)









119875119894= 2 lowast

10038161003816100381610038161003816119872119896minus119872119895

10038161003816100381610038161003816 (3)



119895is the



















119901119894(1 minus 119901

119894) (4)


119894= 1)








Confirmation bias


































4 Data and Results



119873 =(119864max minus 119864)

119864max (5)



119875ℎlowast log2119875ℎ (6)









119867lt 119873119876 then the





119870119875119872= 119864119875119872minus 119864119875

= minussumℎ



(7)




to note that 119870119875119872





119878 = 100 lowast 2minus119870 (8)


119876119875and 119878119876119872







RSR =(119878119875119872minus 119878119875119877)

(100 minus 119878119875119877) (9)












RMR (119905) =119865 (119905 119904mod)

119865 (119905 119904max) (10)


















(119873119876


119867= 113)



119872= 206





0

20

40

60

80

100

Task 1

1 2 3 4 5 6 7 8 9 10

1 2 3 4 50

20

40

60

80

100

Task 2

1 2 3 4 50

20

40

60

80

100

Task 3

1 2 3 4 5 6 7 8 9 1060708090

100708090

100

Task 4

1 2 3 4 5 6 7 8 9 1040

60

80

100708090

100

Task 5

Human S1 Model S1

(a)

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 50

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7

Task 1

8 9 10

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

1 2 3 4 5 6 7 8 9 100

01

02

02040608

1

03

RationalModelHuman

(a)

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04Task 5-1

Task 5-2

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-3

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

minus02

minus01

01

02

03

04

RationalModelHuman

(b)




119876= 412119873

119872= 372 and119873

119867= 311)









119867lt 119873119876and119873

119872lt 119873119876) for each layer

Layer 1119873119902= 080119873

ℎ= 016119873

119898= minus007

Layer 2119873119902= 110119873

ℎ= 033119873

119898= 025

Layer 3119873119902= 138119873

ℎ= 056119873

119898= 024

Layer 4119873119902= 000119873

ℎ= minus007119873

119898= minus011

0

1

02

04

06

08

Prob

abili

ty

ACT-R modelHuman

IM-M

O-S

IIM

-MO

-SO

IM-S

O-M

OIM

-SO

-SI

MO

-IM

-SI

MO

-IM

-SO

MO

-SI-

IM

IM-S

I-M

OIM

-SI-

SO

SO-S

I-M

OSO

-SI-

IMSO

-MO

-SI

SO-M

O-I

M

SI-S

O-M

OSO

-IM

-MO

SO-I

M-S

I

SI-M

O-S

OSI

-SO

-IM

MO

-SO

-IM

SI-M

O-I

M

MO

-SI-

SO

MO

-SO

-SI

SI-I

M-M

OSI

-IM

-SO




5 Generalization










6 Conclusion







1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 1

1 2 3 4 5

Task 2

0

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

Task 3

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 4

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 5

Human S1 Model S1

(a)

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7 8 9 10

Task 1

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

02040608

1


001

RationalModelHuman

1 2 3 4 5 6 7 8 9 10

(a)

00102030405

1 2 3 4 5 6 7 8 9 10

minus01

Task 5-1

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-2

Task 5-3

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

01

02

03

04

05

minus01

RationalModelHuman

(b)



Appendix

PROBs Handbook



50 10 15 20 25 30 35 40

5

0

10

15

20

25

30

35


Groupcenter


Prob

abili

ty o

f atta

ck (

)








attack by A or B

A B C D A B C D



Acknowledgments






A B C D A B C D


A B C D


















References










































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in















) + CrowFlies 82

(2)



0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability


RationalModel

(a)

0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability


RationalModel

(b)









119875119894= 2 lowast

10038161003816100381610038161003816119872119896minus119872119895

10038161003816100381610038161003816 (3)



119895is the



















119901119894(1 minus 119901

119894) (4)


119894= 1)








Confirmation bias


































4 Data and Results



119873 =(119864max minus 119864)

119864max (5)



119875ℎlowast log2119875ℎ (6)









119867lt 119873119876 then the





119870119875119872= 119864119875119872minus 119864119875

= minussumℎ



(7)




to note that 119870119875119872





119878 = 100 lowast 2minus119870 (8)


119876119875and 119878119876119872







RSR =(119878119875119872minus 119878119875119877)

(100 minus 119878119875119877) (9)












RMR (119905) =119865 (119905 119904mod)

119865 (119905 119904max) (10)


















(119873119876


119867= 113)



119872= 206





0

20

40

60

80

100

Task 1

1 2 3 4 5 6 7 8 9 10

1 2 3 4 50

20

40

60

80

100

Task 2

1 2 3 4 50

20

40

60

80

100

Task 3

1 2 3 4 5 6 7 8 9 1060708090

100708090

100

Task 4

1 2 3 4 5 6 7 8 9 1040

60

80

100708090

100

Task 5

Human S1 Model S1

(a)

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 50

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7

Task 1

8 9 10

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

1 2 3 4 5 6 7 8 9 100

01

02

02040608

1

03

RationalModelHuman

(a)

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04Task 5-1

Task 5-2

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-3

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

minus02

minus01

01

02

03

04

RationalModelHuman

(b)




119876= 412119873

119872= 372 and119873

119867= 311)









119867lt 119873119876and119873

119872lt 119873119876) for each layer

Layer 1119873119902= 080119873

ℎ= 016119873

119898= minus007

Layer 2119873119902= 110119873

ℎ= 033119873

119898= 025

Layer 3119873119902= 138119873

ℎ= 056119873

119898= 024

Layer 4119873119902= 000119873

ℎ= minus007119873

119898= minus011

0

1

02

04

06

08

Prob

abili

ty

ACT-R modelHuman

IM-M

O-S

IIM

-MO

-SO

IM-S

O-M

OIM

-SO

-SI

MO

-IM

-SI

MO

-IM

-SO

MO

-SI-

IM

IM-S

I-M

OIM

-SI-

SO

SO-S

I-M

OSO

-SI-

IMSO

-MO

-SI

SO-M

O-I

M

SI-S

O-M

OSO

-IM

-MO

SO-I

M-S

I

SI-M

O-S

OSI

-SO

-IM

MO

-SO

-IM

SI-M

O-I

M

MO

-SI-

SO

MO

-SO

-SI

SI-I

M-M

OSI

-IM

-SO




5 Generalization










6 Conclusion







1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 1

1 2 3 4 5

Task 2

0

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

Task 3

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 4

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 5

Human S1 Model S1

(a)

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7 8 9 10

Task 1

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

02040608

1


001

RationalModelHuman

1 2 3 4 5 6 7 8 9 10

(a)

00102030405

1 2 3 4 5 6 7 8 9 10

minus01

Task 5-1

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-2

Task 5-3

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

01

02

03

04

05

minus01

RationalModelHuman

(b)



Appendix

PROBs Handbook



50 10 15 20 25 30 35 40

5

0

10

15

20

25

30

35


Groupcenter


Prob

abili

ty o

f atta

ck (

)








attack by A or B

A B C D A B C D



Acknowledgments






A B C D A B C D


A B C D


















References










































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in




0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability


RationalModel

(a)

0

01

02

03

04

05

06

07

08

09

1

0 01 02 03 04 05 06 07 08 09 1

Adju

sted

prob

abili

ty

Initial probability


RationalModel

(b)









119875119894= 2 lowast

10038161003816100381610038161003816119872119896minus119872119895

10038161003816100381610038161003816 (3)



119895is the



















119901119894(1 minus 119901

119894) (4)


119894= 1)








Confirmation bias


































4 Data and Results



119873 =(119864max minus 119864)

119864max (5)



119875ℎlowast log2119875ℎ (6)









119867lt 119873119876 then the





119870119875119872= 119864119875119872minus 119864119875

= minussumℎ



(7)




to note that 119870119875119872





119878 = 100 lowast 2minus119870 (8)


119876119875and 119878119876119872







RSR =(119878119875119872minus 119878119875119877)

(100 minus 119878119875119877) (9)












RMR (119905) =119865 (119905 119904mod)

119865 (119905 119904max) (10)


















(119873119876


119867= 113)



119872= 206





0

20

40

60

80

100

Task 1

1 2 3 4 5 6 7 8 9 10

1 2 3 4 50

20

40

60

80

100

Task 2

1 2 3 4 50

20

40

60

80

100

Task 3

1 2 3 4 5 6 7 8 9 1060708090

100708090

100

Task 4

1 2 3 4 5 6 7 8 9 1040

60

80

100708090

100

Task 5

Human S1 Model S1

(a)

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 50

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7

Task 1

8 9 10

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

1 2 3 4 5 6 7 8 9 100

01

02

02040608

1

03

RationalModelHuman

(a)

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04Task 5-1

Task 5-2

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-3

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

minus02

minus01

01

02

03

04

RationalModelHuman

(b)




119876= 412119873

119872= 372 and119873

119867= 311)









119867lt 119873119876and119873

119872lt 119873119876) for each layer

Layer 1119873119902= 080119873

ℎ= 016119873

119898= minus007

Layer 2119873119902= 110119873

ℎ= 033119873

119898= 025

Layer 3119873119902= 138119873

ℎ= 056119873

119898= 024

Layer 4119873119902= 000119873

ℎ= minus007119873

119898= minus011

0

1

02

04

06

08

Prob

abili

ty

ACT-R modelHuman

IM-M

O-S

IIM

-MO

-SO

IM-S

O-M

OIM

-SO

-SI

MO

-IM

-SI

MO

-IM

-SO

MO

-SI-

IM

IM-S

I-M

OIM

-SI-

SO

SO-S

I-M

OSO

-SI-

IMSO

-MO

-SI

SO-M

O-I

M

SI-S

O-M

OSO

-IM

-MO

SO-I

M-S

I

SI-M

O-S

OSI

-SO

-IM

MO

-SO

-IM

SI-M

O-I

M

MO

-SI-

SO

MO

-SO

-SI

SI-I

M-M

OSI

-IM

-SO




5 Generalization










6 Conclusion







1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 1

1 2 3 4 5

Task 2

0

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

Task 3

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 4

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 5

Human S1 Model S1

(a)

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7 8 9 10

Task 1

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

02040608

1


001

RationalModelHuman

1 2 3 4 5 6 7 8 9 10

(a)

00102030405

1 2 3 4 5 6 7 8 9 10

minus01

Task 5-1

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-2

Task 5-3

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

01

02

03

04

05

minus01

RationalModelHuman

(b)



Appendix

PROBs Handbook



50 10 15 20 25 30 35 40

5

0

10

15

20

25

30

35


Groupcenter


Prob

abili

ty o

f atta

ck (

)








attack by A or B

A B C D A B C D



Acknowledgments






A B C D A B C D


A B C D


















References










































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in





119895is the



















119901119894(1 minus 119901

119894) (4)


119894= 1)








Confirmation bias


































4 Data and Results



119873 =(119864max minus 119864)

119864max (5)



119875ℎlowast log2119875ℎ (6)









119867lt 119873119876 then the





119870119875119872= 119864119875119872minus 119864119875

= minussumℎ



(7)




to note that 119870119875119872





119878 = 100 lowast 2minus119870 (8)


119876119875and 119878119876119872







RSR =(119878119875119872minus 119878119875119877)

(100 minus 119878119875119877) (9)












RMR (119905) =119865 (119905 119904mod)

119865 (119905 119904max) (10)


















(119873119876


119867= 113)



119872= 206





0

20

40

60

80

100

Task 1

1 2 3 4 5 6 7 8 9 10

1 2 3 4 50

20

40

60

80

100

Task 2

1 2 3 4 50

20

40

60

80

100

Task 3

1 2 3 4 5 6 7 8 9 1060708090

100708090

100

Task 4

1 2 3 4 5 6 7 8 9 1040

60

80

100708090

100

Task 5

Human S1 Model S1

(a)

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 50

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7

Task 1

8 9 10

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

1 2 3 4 5 6 7 8 9 100

01

02

02040608

1

03

RationalModelHuman

(a)

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04Task 5-1

Task 5-2

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-3

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

minus02

minus01

01

02

03

04

RationalModelHuman

(b)




119876= 412119873

119872= 372 and119873

119867= 311)









119867lt 119873119876and119873

119872lt 119873119876) for each layer

Layer 1119873119902= 080119873

ℎ= 016119873

119898= minus007

Layer 2119873119902= 110119873

ℎ= 033119873

119898= 025

Layer 3119873119902= 138119873

ℎ= 056119873

119898= 024

Layer 4119873119902= 000119873

ℎ= minus007119873

119898= minus011

0

1

02

04

06

08

Prob

abili

ty

ACT-R modelHuman

IM-M

O-S

IIM

-MO

-SO

IM-S

O-M

OIM

-SO

-SI

MO

-IM

-SI

MO

-IM

-SO

MO

-SI-

IM

IM-S

I-M

OIM

-SI-

SO

SO-S

I-M

OSO

-SI-

IMSO

-MO

-SI

SO-M

O-I

M

SI-S

O-M

OSO

-IM

-MO

SO-I

M-S

I

SI-M

O-S

OSI

-SO

-IM

MO

-SO

-IM

SI-M

O-I

M

MO

-SI-

SO

MO

-SO

-SI

SI-I

M-M

OSI

-IM

-SO




5 Generalization










6 Conclusion







1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 1

1 2 3 4 5

Task 2

0

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

Task 3

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 4

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 5

Human S1 Model S1

(a)

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7 8 9 10

Task 1

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

02040608

1


001

RationalModelHuman

1 2 3 4 5 6 7 8 9 10

(a)

00102030405

1 2 3 4 5 6 7 8 9 10

minus01

Task 5-1

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-2

Task 5-3

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

01

02

03

04

05

minus01

RationalModelHuman

(b)



Appendix

PROBs Handbook



50 10 15 20 25 30 35 40

5

0

10

15

20

25

30

35


Groupcenter


Prob

abili

ty o

f atta

ck (

)








attack by A or B

A B C D A B C D



Acknowledgments






A B C D A B C D


A B C D


















References










































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in











119901119894(1 minus 119901

119894) (4)


119894= 1)








Confirmation bias


































4 Data and Results



119873 =(119864max minus 119864)

119864max (5)



119875ℎlowast log2119875ℎ (6)









119867lt 119873119876 then the





119870119875119872= 119864119875119872minus 119864119875

= minussumℎ



(7)




to note that 119870119875119872





119878 = 100 lowast 2minus119870 (8)


119876119875and 119878119876119872







RSR =(119878119875119872minus 119878119875119877)

(100 minus 119878119875119877) (9)












RMR (119905) =119865 (119905 119904mod)

119865 (119905 119904max) (10)


















(119873119876


119867= 113)



119872= 206





0

20

40

60

80

100

Task 1

1 2 3 4 5 6 7 8 9 10

1 2 3 4 50

20

40

60

80

100

Task 2

1 2 3 4 50

20

40

60

80

100

Task 3

1 2 3 4 5 6 7 8 9 1060708090

100708090

100

Task 4

1 2 3 4 5 6 7 8 9 1040

60

80

100708090

100

Task 5

Human S1 Model S1

(a)

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 50

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7

Task 1

8 9 10

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

1 2 3 4 5 6 7 8 9 100

01

02

02040608

1

03

RationalModelHuman

(a)

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04Task 5-1

Task 5-2

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-3

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

minus02

minus01

01

02

03

04

RationalModelHuman

(b)




119876= 412119873

119872= 372 and119873

119867= 311)









119867lt 119873119876and119873

119872lt 119873119876) for each layer

Layer 1119873119902= 080119873

ℎ= 016119873

119898= minus007

Layer 2119873119902= 110119873

ℎ= 033119873

119898= 025

Layer 3119873119902= 138119873

ℎ= 056119873

119898= 024

Layer 4119873119902= 000119873

ℎ= minus007119873

119898= minus011

0

1

02

04

06

08

Prob

abili

ty

ACT-R modelHuman

IM-M

O-S

IIM

-MO

-SO

IM-S

O-M

OIM

-SO

-SI

MO

-IM

-SI

MO

-IM

-SO

MO

-SI-

IM

IM-S

I-M

OIM

-SI-

SO

SO-S

I-M

OSO

-SI-

IMSO

-MO

-SI

SO-M

O-I

M

SI-S

O-M

OSO

-IM

-MO

SO-I

M-S

I

SI-M

O-S

OSI

-SO

-IM

MO

-SO

-IM

SI-M

O-I

M

MO

-SI-

SO

MO

-SO

-SI

SI-I

M-M

OSI

-IM

-SO




5 Generalization










6 Conclusion







1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 1

1 2 3 4 5

Task 2

0

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

Task 3

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 4

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 5

Human S1 Model S1

(a)

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7 8 9 10

Task 1

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

02040608

1


001

RationalModelHuman

1 2 3 4 5 6 7 8 9 10

(a)

00102030405

1 2 3 4 5 6 7 8 9 10

minus01

Task 5-1

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-2

Task 5-3

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

01

02

03

04

05

minus01

RationalModelHuman

(b)



Appendix

PROBs Handbook



50 10 15 20 25 30 35 40

5

0

10

15

20

25

30

35


Groupcenter


Prob

abili

ty o

f atta

ck (

)








attack by A or B

A B C D A B C D



Acknowledgments






A B C D A B C D


A B C D


















References










































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in






Confirmation bias


































4 Data and Results



119873 =(119864max minus 119864)

119864max (5)



119875ℎlowast log2119875ℎ (6)









119867lt 119873119876 then the





119870119875119872= 119864119875119872minus 119864119875

= minussumℎ



(7)




to note that 119870119875119872





119878 = 100 lowast 2minus119870 (8)


119876119875and 119878119876119872







RSR =(119878119875119872minus 119878119875119877)

(100 minus 119878119875119877) (9)












RMR (119905) =119865 (119905 119904mod)

119865 (119905 119904max) (10)


















(119873119876


119867= 113)



119872= 206





0

20

40

60

80

100

Task 1

1 2 3 4 5 6 7 8 9 10

1 2 3 4 50

20

40

60

80

100

Task 2

1 2 3 4 50

20

40

60

80

100

Task 3

1 2 3 4 5 6 7 8 9 1060708090

100708090

100

Task 4

1 2 3 4 5 6 7 8 9 1040

60

80

100708090

100

Task 5

Human S1 Model S1

(a)

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 50

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7

Task 1

8 9 10

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

1 2 3 4 5 6 7 8 9 100

01

02

02040608

1

03

RationalModelHuman

(a)

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04Task 5-1

Task 5-2

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-3

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

minus02

minus01

01

02

03

04

RationalModelHuman

(b)




119876= 412119873

119872= 372 and119873

119867= 311)









119867lt 119873119876and119873

119872lt 119873119876) for each layer

Layer 1119873119902= 080119873

ℎ= 016119873

119898= minus007

Layer 2119873119902= 110119873

ℎ= 033119873

119898= 025

Layer 3119873119902= 138119873

ℎ= 056119873

119898= 024

Layer 4119873119902= 000119873

ℎ= minus007119873

119898= minus011

0

1

02

04

06

08

Prob

abili

ty

ACT-R modelHuman

IM-M

O-S

IIM

-MO

-SO

IM-S

O-M

OIM

-SO

-SI

MO

-IM

-SI

MO

-IM

-SO

MO

-SI-

IM

IM-S

I-M

OIM

-SI-

SO

SO-S

I-M

OSO

-SI-

IMSO

-MO

-SI

SO-M

O-I

M

SI-S

O-M

OSO

-IM

-MO

SO-I

M-S

I

SI-M

O-S

OSI

-SO

-IM

MO

-SO

-IM

SI-M

O-I

M

MO

-SI-

SO

MO

-SO

-SI

SI-I

M-M

OSI

-IM

-SO




5 Generalization










6 Conclusion







1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 1

1 2 3 4 5

Task 2

0

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

Task 3

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 4

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 5

Human S1 Model S1

(a)

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7 8 9 10

Task 1

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

02040608

1


001

RationalModelHuman

1 2 3 4 5 6 7 8 9 10

(a)

00102030405

1 2 3 4 5 6 7 8 9 10

minus01

Task 5-1

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-2

Task 5-3

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

01

02

03

04

05

minus01

RationalModelHuman

(b)



Appendix

PROBs Handbook



50 10 15 20 25 30 35 40

5

0

10

15

20

25

30

35


Groupcenter


Prob

abili

ty o

f atta

ck (

)








attack by A or B

A B C D A B C D



Acknowledgments






A B C D A B C D


A B C D


















References










































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in


















4 Data and Results



119873 =(119864max minus 119864)

119864max (5)



119875ℎlowast log2119875ℎ (6)









119867lt 119873119876 then the





119870119875119872= 119864119875119872minus 119864119875

= minussumℎ



(7)




to note that 119870119875119872





119878 = 100 lowast 2minus119870 (8)


119876119875and 119878119876119872







RSR =(119878119875119872minus 119878119875119877)

(100 minus 119878119875119877) (9)












RMR (119905) =119865 (119905 119904mod)

119865 (119905 119904max) (10)


















(119873119876


119867= 113)



119872= 206





0

20

40

60

80

100

Task 1

1 2 3 4 5 6 7 8 9 10

1 2 3 4 50

20

40

60

80

100

Task 2

1 2 3 4 50

20

40

60

80

100

Task 3

1 2 3 4 5 6 7 8 9 1060708090

100708090

100

Task 4

1 2 3 4 5 6 7 8 9 1040

60

80

100708090

100

Task 5

Human S1 Model S1

(a)

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 50

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7

Task 1

8 9 10

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

1 2 3 4 5 6 7 8 9 100

01

02

02040608

1

03

RationalModelHuman

(a)

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04Task 5-1

Task 5-2

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-3

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

minus02

minus01

01

02

03

04

RationalModelHuman

(b)




119876= 412119873

119872= 372 and119873

119867= 311)









119867lt 119873119876and119873

119872lt 119873119876) for each layer

Layer 1119873119902= 080119873

ℎ= 016119873

119898= minus007

Layer 2119873119902= 110119873

ℎ= 033119873

119898= 025

Layer 3119873119902= 138119873

ℎ= 056119873

119898= 024

Layer 4119873119902= 000119873

ℎ= minus007119873

119898= minus011

0

1

02

04

06

08

Prob

abili

ty

ACT-R modelHuman

IM-M

O-S

IIM

-MO

-SO

IM-S

O-M

OIM

-SO

-SI

MO

-IM

-SI

MO

-IM

-SO

MO

-SI-

IM

IM-S

I-M

OIM

-SI-

SO

SO-S

I-M

OSO

-SI-

IMSO

-MO

-SI

SO-M

O-I

M

SI-S

O-M

OSO

-IM

-MO

SO-I

M-S

I

SI-M

O-S

OSI

-SO

-IM

MO

-SO

-IM

SI-M

O-I

M

MO

-SI-

SO

MO

-SO

-SI

SI-I

M-M

OSI

-IM

-SO




5 Generalization










6 Conclusion







1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 1

1 2 3 4 5

Task 2

0

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

Task 3

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 4

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 5

Human S1 Model S1

(a)

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7 8 9 10

Task 1

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

02040608

1


001

RationalModelHuman

1 2 3 4 5 6 7 8 9 10

(a)

00102030405

1 2 3 4 5 6 7 8 9 10

minus01

Task 5-1

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-2

Task 5-3

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

01

02

03

04

05

minus01

RationalModelHuman

(b)



Appendix

PROBs Handbook



50 10 15 20 25 30 35 40

5

0

10

15

20

25

30

35


Groupcenter


Prob

abili

ty o

f atta

ck (

)








attack by A or B

A B C D A B C D



Acknowledgments






A B C D A B C D


A B C D


















References










































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in











119867lt 119873119876 then the





119870119875119872= 119864119875119872minus 119864119875

= minussumℎ



(7)




to note that 119870119875119872





119878 = 100 lowast 2minus119870 (8)


119876119875and 119878119876119872







RSR =(119878119875119872minus 119878119875119877)

(100 minus 119878119875119877) (9)












RMR (119905) =119865 (119905 119904mod)

119865 (119905 119904max) (10)


















(119873119876


119867= 113)



119872= 206





0

20

40

60

80

100

Task 1

1 2 3 4 5 6 7 8 9 10

1 2 3 4 50

20

40

60

80

100

Task 2

1 2 3 4 50

20

40

60

80

100

Task 3

1 2 3 4 5 6 7 8 9 1060708090

100708090

100

Task 4

1 2 3 4 5 6 7 8 9 1040

60

80

100708090

100

Task 5

Human S1 Model S1

(a)

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 50

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7

Task 1

8 9 10

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

1 2 3 4 5 6 7 8 9 100

01

02

02040608

1

03

RationalModelHuman

(a)

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04Task 5-1

Task 5-2

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-3

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

minus02

minus01

01

02

03

04

RationalModelHuman

(b)




119876= 412119873

119872= 372 and119873

119867= 311)









119867lt 119873119876and119873

119872lt 119873119876) for each layer

Layer 1119873119902= 080119873

ℎ= 016119873

119898= minus007

Layer 2119873119902= 110119873

ℎ= 033119873

119898= 025

Layer 3119873119902= 138119873

ℎ= 056119873

119898= 024

Layer 4119873119902= 000119873

ℎ= minus007119873

119898= minus011

0

1

02

04

06

08

Prob

abili

ty

ACT-R modelHuman

IM-M

O-S

IIM

-MO

-SO

IM-S

O-M

OIM

-SO

-SI

MO

-IM

-SI

MO

-IM

-SO

MO

-SI-

IM

IM-S

I-M

OIM

-SI-

SO

SO-S

I-M

OSO

-SI-

IMSO

-MO

-SI

SO-M

O-I

M

SI-S

O-M

OSO

-IM

-MO

SO-I

M-S

I

SI-M

O-S

OSI

-SO

-IM

MO

-SO

-IM

SI-M

O-I

M

MO

-SI-

SO

MO

-SO

-SI

SI-I

M-M

OSI

-IM

-SO




5 Generalization










6 Conclusion







1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 1

1 2 3 4 5

Task 2

0

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

Task 3

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 4

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 5

Human S1 Model S1

(a)

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7 8 9 10

Task 1

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

02040608

1


001

RationalModelHuman

1 2 3 4 5 6 7 8 9 10

(a)

00102030405

1 2 3 4 5 6 7 8 9 10

minus01

Task 5-1

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-2

Task 5-3

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

01

02

03

04

05

minus01

RationalModelHuman

(b)



Appendix

PROBs Handbook



50 10 15 20 25 30 35 40

5

0

10

15

20

25

30

35


Groupcenter


Prob

abili

ty o

f atta

ck (

)








attack by A or B

A B C D A B C D



Acknowledgments






A B C D A B C D


A B C D


















References










































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in

















(119873119876


119867= 113)



119872= 206





0

20

40

60

80

100

Task 1

1 2 3 4 5 6 7 8 9 10

1 2 3 4 50

20

40

60

80

100

Task 2

1 2 3 4 50

20

40

60

80

100

Task 3

1 2 3 4 5 6 7 8 9 1060708090

100708090

100

Task 4

1 2 3 4 5 6 7 8 9 1040

60

80

100708090

100

Task 5

Human S1 Model S1

(a)

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 50

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7

Task 1

8 9 10

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

1 2 3 4 5 6 7 8 9 100

01

02

02040608

1

03

RationalModelHuman

(a)

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04Task 5-1

Task 5-2

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-3

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

minus02

minus01

01

02

03

04

RationalModelHuman

(b)




119876= 412119873

119872= 372 and119873

119867= 311)









119867lt 119873119876and119873

119872lt 119873119876) for each layer

Layer 1119873119902= 080119873

ℎ= 016119873

119898= minus007

Layer 2119873119902= 110119873

ℎ= 033119873

119898= 025

Layer 3119873119902= 138119873

ℎ= 056119873

119898= 024

Layer 4119873119902= 000119873

ℎ= minus007119873

119898= minus011

0

1

02

04

06

08

Prob

abili

ty

ACT-R modelHuman

IM-M

O-S

IIM

-MO

-SO

IM-S

O-M

OIM

-SO

-SI

MO

-IM

-SI

MO

-IM

-SO

MO

-SI-

IM

IM-S

I-M

OIM

-SI-

SO

SO-S

I-M

OSO

-SI-

IMSO

-MO

-SI

SO-M

O-I

M

SI-S

O-M

OSO

-IM

-MO

SO-I

M-S

I

SI-M

O-S

OSI

-SO

-IM

MO

-SO

-IM

SI-M

O-I

M

MO

-SI-

SO

MO

-SO

-SI

SI-I

M-M

OSI

-IM

-SO




5 Generalization










6 Conclusion







1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 1

1 2 3 4 5

Task 2

0

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

Task 3

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 4

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 5

Human S1 Model S1

(a)

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7 8 9 10

Task 1

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

02040608

1


001

RationalModelHuman

1 2 3 4 5 6 7 8 9 10

(a)

00102030405

1 2 3 4 5 6 7 8 9 10

minus01

Task 5-1

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-2

Task 5-3

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

01

02

03

04

05

minus01

RationalModelHuman

(b)



Appendix

PROBs Handbook



50 10 15 20 25 30 35 40

5

0

10

15

20

25

30

35


Groupcenter


Prob

abili

ty o

f atta

ck (

)








attack by A or B

A B C D A B C D



Acknowledgments






A B C D A B C D


A B C D


















References










































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in




0

20

40

60

80

100

Task 1

1 2 3 4 5 6 7 8 9 10

1 2 3 4 50

20

40

60

80

100

Task 2

1 2 3 4 50

20

40

60

80

100

Task 3

1 2 3 4 5 6 7 8 9 1060708090

100708090

100

Task 4

1 2 3 4 5 6 7 8 9 1040

60

80

100708090

100

Task 5

Human S1 Model S1

(a)

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 50

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7

Task 1

8 9 10

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

1 2 3 4 5 6 7 8 9 100

01

02

02040608

1

03

RationalModelHuman

(a)

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04Task 5-1

Task 5-2

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-3

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

minus02

minus01

01

02

03

04

RationalModelHuman

(b)




119876= 412119873

119872= 372 and119873

119867= 311)









119867lt 119873119876and119873

119872lt 119873119876) for each layer

Layer 1119873119902= 080119873

ℎ= 016119873

119898= minus007

Layer 2119873119902= 110119873

ℎ= 033119873

119898= 025

Layer 3119873119902= 138119873

ℎ= 056119873

119898= 024

Layer 4119873119902= 000119873

ℎ= minus007119873

119898= minus011

0

1

02

04

06

08

Prob

abili

ty

ACT-R modelHuman

IM-M

O-S

IIM

-MO

-SO

IM-S

O-M

OIM

-SO

-SI

MO

-IM

-SI

MO

-IM

-SO

MO

-SI-

IM

IM-S

I-M

OIM

-SI-

SO

SO-S

I-M

OSO

-SI-

IMSO

-MO

-SI

SO-M

O-I

M

SI-S

O-M

OSO

-IM

-MO

SO-I

M-S

I

SI-M

O-S

OSI

-SO

-IM

MO

-SO

-IM

SI-M

O-I

M

MO

-SI-

SO

MO

-SO

-SI

SI-I

M-M

OSI

-IM

-SO




5 Generalization










6 Conclusion







1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 1

1 2 3 4 5

Task 2

0

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

Task 3

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 4

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 5

Human S1 Model S1

(a)

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7 8 9 10

Task 1

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

02040608

1


001

RationalModelHuman

1 2 3 4 5 6 7 8 9 10

(a)

00102030405

1 2 3 4 5 6 7 8 9 10

minus01

Task 5-1

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-2

Task 5-3

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

01

02

03

04

05

minus01

RationalModelHuman

(b)



Appendix

PROBs Handbook



50 10 15 20 25 30 35 40

5

0

10

15

20

25

30

35


Groupcenter


Prob

abili

ty o

f atta

ck (

)








attack by A or B

A B C D A B C D



Acknowledgments






A B C D A B C D


A B C D


















References










































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in




0

02

04

06

08

1

1 2 3 4 5 6 7

Task 1

8 9 10

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

1 2 3 4 5 6 7 8 9 100

01

02

02040608

1

03

RationalModelHuman

(a)

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04Task 5-1

Task 5-2

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-3

1 2 3 4 5 6 7 8 9 10

minus01

01

0

02

03

04

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

minus02

minus01

01

02

03

04

RationalModelHuman

(b)




119876= 412119873

119872= 372 and119873

119867= 311)









119867lt 119873119876and119873

119872lt 119873119876) for each layer

Layer 1119873119902= 080119873

ℎ= 016119873

119898= minus007

Layer 2119873119902= 110119873

ℎ= 033119873

119898= 025

Layer 3119873119902= 138119873

ℎ= 056119873

119898= 024

Layer 4119873119902= 000119873

ℎ= minus007119873

119898= minus011

0

1

02

04

06

08

Prob

abili

ty

ACT-R modelHuman

IM-M

O-S

IIM

-MO

-SO

IM-S

O-M

OIM

-SO

-SI

MO

-IM

-SI

MO

-IM

-SO

MO

-SI-

IM

IM-S

I-M

OIM

-SI-

SO

SO-S

I-M

OSO

-SI-

IMSO

-MO

-SI

SO-M

O-I

M

SI-S

O-M

OSO

-IM

-MO

SO-I

M-S

I

SI-M

O-S

OSI

-SO

-IM

MO

-SO

-IM

SI-M

O-I

M

MO

-SI-

SO

MO

-SO

-SI

SI-I

M-M

OSI

-IM

-SO




5 Generalization










6 Conclusion







1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 1

1 2 3 4 5

Task 2

0

20

40

60

80

100

0

20

40

60

80

100

1 2 3 4 5

Task 3

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 4

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Task 5

Human S1 Model S1

(a)

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

1 2 3 4 5

0

20

40

60

80

100

1 2 3 4 5

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

Human S2 Model S2

(b)



0

02

04

06

08

1

1 2 3 4 5 6 7 8 9 10

Task 1

Task 2

1 2 3 4 50

02

04

06

08

1

Task 3

1 2 3 4 50

02

04

06

08

1

Task 4-1

Task 4-2

02040608

1


001

RationalModelHuman

1 2 3 4 5 6 7 8 9 10

(a)

00102030405

1 2 3 4 5 6 7 8 9 10

minus01

Task 5-1

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-2

Task 5-3

00102030405

minus01

1 2 3 4 5 6 7 8 9 10

Task 5-4

1 2 3 4 5 6 7 8 9 10

0

01

02

03

04

05

minus01

RationalModelHuman

(b)



Appendix

PROBs Handbook



50 10 15 20 25 30 35 40

5

0

10

15

20

25

30

35


Groupcenter


Prob

abili

ty o

f atta

ck (

)








attack by A or B

A B C D A B C D



Acknowledgments






A B C D A B C D


A B C D


















References










































































Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in



research article a functional model of sensemaking in a ...computationalintelligence and...

Documents