motivation and cognition: from regulatory fit to reinforcement learning
DESCRIPTION
Motivation and Cognition: From Regulatory Fit to Reinforcement Learning. Darrell A. Worthy University of Texas, Austin. Motivation and Cognition. Why study motivation? Need to understand how goals and rewards influence cognition and behavior. More complicated than anecdotal notions - PowerPoint PPT PresentationTRANSCRIPT
Motivation and Cognition: Motivation and Cognition: From Regulatory Fit to From Regulatory Fit to
Reinforcement LearningReinforcement Learning
Darrell A. WorthyDarrell A. Worthy
University of Texas, AustinUniversity of Texas, Austin
Motivation and CognitionMotivation and Cognition Why study motivation?Why study motivation?
Need to understand Need to understand how goals and rewards how goals and rewards influence cognition and influence cognition and behavior.behavior.
More complicated than More complicated than anecdotal notionsanecdotal notions
Approach vs. avoidance Approach vs. avoidance distinctiondistinction
Global incentive vs. local Global incentive vs. local goal pursuit mechanism goal pursuit mechanism
Leads to regulatory fit or Leads to regulatory fit or mismatchmismatch
Regulatory fit affects Regulatory fit affects cognition and behaviorcognition and behavior
Overview of TalkOverview of Talk Regulatory Fit FrameworkRegulatory Fit Framework Regulatory fit affects cognitionRegulatory fit affects cognition Tests of the Regulatory Fit HypothesisTests of the Regulatory Fit Hypothesis
Extend framework to examine effects of Extend framework to examine effects of social pressure.social pressure.
Regulatory Fit and Decision-makingRegulatory Fit and Decision-making
Future DirectionsFuture Directions
Regulatory Fit Regulatory Fit FrameworkFramework
FitFit MismatchMismatch
MismatchMismatch FitFit
Promotion Focus
Prevention Focus
Gain
sL
oss
es
Local
Goal
Pu
rsu
it
Mech
an
ism
Global Incentive
Global incentive Global incentive focus interacts focus interacts with local with local reward structure reward structure
Produces a Fit or Produces a Fit or a Mismatch (e.g. a Mismatch (e.g. Higgins, 2000).Higgins, 2000).
Almost all Almost all cognitive cognitive research involves research involves promotion focus promotion focus with gains with gains reward reward structure.structure.
Manipulation of Regulatory Manipulation of Regulatory FocusFocus
(Global Task Goal)(Global Task Goal)Promotion Promotion Focus Focus (Approach)(Approach)
Achieve Global Task Achieve Global Task Performance Criterion Performance Criterion Raffle ticket for $50Raffle ticket for $50
Prevention Prevention Focus Focus (Avoidance)(Avoidance)
Achieve Global Task Achieve Global Task Performance Criterion Performance Criterion Keep $50 raffle ticket given Keep $50 raffle ticket given initiallyinitially
Manipulation of Goal-Pursuit Manipulation of Goal-Pursuit MechanismMechanism
(Local Trial-by-trial Task Goal)(Local Trial-by-trial Task Goal)
GainsGainsCorrect Response = 3 pointsCorrect Response = 3 points
Incorrect Response = 1 pointIncorrect Response = 1 point
LossesLossesCorrect Response = -1 pointCorrect Response = -1 point
Incorrect Response = -3 pointIncorrect Response = -3 point
Effects of Regulatory Fit Effects of Regulatory Fit Previous researchPrevious research
Regulatory Fit leads to:Regulatory Fit leads to: Increased sense of ‘feeling right’ (Higgins, Increased sense of ‘feeling right’ (Higgins,
2000)2000) Increased motivational strength (Spiegel et Increased motivational strength (Spiegel et
al., 2004)al., 2004) Increased “cognitive flexibility” (Shah et al., Increased “cognitive flexibility” (Shah et al.,
1998)1998)
Flexibility can be defined within tasksFlexibility can be defined within tasks Category-learning -willingness to test Category-learning -willingness to test
various strategiesvarious strategies Decision-making -willingness to explore Decision-making -willingness to explore
the environmentthe environment
Perceptual Perceptual ClassificationClassification
Excellent for testing the effects of Excellent for testing the effects of regulatory fitregulatory fit
Stimuli with small number of dimensionsStimuli with small number of dimensions Lines that vary in length, orientation and Lines that vary in length, orientation and
positionposition ‘‘Gabor’ patches that vary in frequency and Gabor’ patches that vary in frequency and
orientationorientation Experimenter control of category structureExperimenter control of category structure Extensive set of tools for modeling Extensive set of tools for modeling
performance of individual participantsperformance of individual participants Can assess the strategies participants use in Can assess the strategies participants use in
the taskthe task
Explicit, Explicit, Hypothesis-testingHypothesis-testing system system mediates learning of mediates learning of “rule-based” (RB) category structures.“rule-based” (RB) category structures.
-Frontally mediated-Frontally mediated
-Verbalizable rules -Verbalizable rules
Implicit, Implicit, Procedural Procedural learning systemlearning system mediates learning mediates learning of “information-integration” (II) category structures.of “information-integration” (II) category structures.
-Striatally mediated-Striatally mediated
- Verbalizable rules - Verbalizable rules hurt hurt performanceperformance(Maddox and Ashby, 2004; Ashby et al., 1998)(Maddox and Ashby, 2004; Ashby et al., 1998)
Multiple systems mediate different classification tasks
Categorization TasksCategorization Tasks
Rule-Based Information-Integration
Bar Width
Ori
enta
tion
A B
Bar Width
Ori
enta
tion
A
B
Learned Explicitly Learned Implicitly
Increased cognitive flexibility will Increased cognitive flexibility will increase rule useincrease rule use Enhance performance on rule-based tasksEnhance performance on rule-based tasks Will harm performance on information-integration taskWill harm performance on information-integration task
Rule-use disrupts the procedural systemRule-use disrupts the procedural system
Recent tests of this hypothesis (Markman et al., 2005; Maddox Recent tests of this hypothesis (Markman et al., 2005; Maddox et al., 2006; Grimm et al., 2008)et al., 2006; Grimm et al., 2008) Manipulated regulatory focus and reward structure between subjectsManipulated regulatory focus and reward structure between subjects Used rule-based and information-integration tasksUsed rule-based and information-integration tasks
Influence of Regulatory Fit
Regulatory Fit and Regulatory Fit and ClassificationClassification
Rule-based performance was better in a fit Information-integration performance was better in a mismatch
Fit increases rule-useHelps on rule-based, hurts on information-integration
Choking & Choking & Excelling Under Excelling Under
PressurePressure
Worthy, Markman, & Maddox, 2009a, 2009b; Worthy, Markman, & Maddox, 2009a, 2009b; Worthy, Markman, & Maddox, 2008; Markman, Worthy, Markman, & Maddox, 2008; Markman,
Maddox & Worthy 2006Maddox & Worthy 2006
Choking Under PressureChoking Under Pressure
Anecdotal phenomenon (e.g. sports, Anecdotal phenomenon (e.g. sports, test-taking, etc.)test-taking, etc.)
People perform worse than normal People perform worse than normal when under pressurewhen under pressure
Some also seem to excel under Some also seem to excel under pressurepressure
Might pressure be similar to a Might pressure be similar to a prevention focus?prevention focus?
Motivation and Pressure Motivation and Pressure Working Memory Distraction Hypothesis of chokingWorking Memory Distraction Hypothesis of choking
Pressure reduces WM capacityPressure reduces WM capacity Should see main effectsShould see main effects Pressure decreases rule-usePressure decreases rule-use
Alternative: Pressure affects cognition through its Alternative: Pressure affects cognition through its effects on the motivational stateeffects on the motivational state
Working Hypothesis:Working Hypothesis: Pressure induces an “avoidance” or “prevention” Pressure induces an “avoidance” or “prevention”
motivational statemotivational state Interacts with goal pursuit mechanism to influence Interacts with goal pursuit mechanism to influence
regulatory fitregulatory fit
Pressure and Category-Pressure and Category-LearningLearning
Low pressure – “do Low pressure – “do your best”your best”
High pressure:High pressure:-Paired with a ‘partner’-Paired with a ‘partner’-If both of you reach -If both of you reach criterion, both get $6criterion, both get $6-If one of you fails -If one of you fails neither get $6 bonusneither get $6 bonus-Partner has already -Partner has already reached criterionreached criterion-Trying to -Trying to preventprevent the the negative end-state of negative end-state of letting their partner letting their partner downdown
Run gains and lossesRun gains and losses
FitFit MismatchMismatch
MismatchMismatch FitFit
Promotion Focus Prevention Focus
Gai
ns
Los
ses
Loc
al G
oal P
urs
uit
Mec
han
ism
Global Incentive
Low Pressure High Pressure
WM Distraction vs. WM Distraction vs. Regulatory FitRegulatory Fit
Pressure decreases Pressure decreases WMWM Poor rule-based Poor rule-based
performanceperformance Better information-Better information-
integrationintegrationPredictions Based on Regulatory Fit
Hypothesis
Gain Loss Gain Loss
Rule-Based Information-Integration
Pro
po
rtio
n C
orr
ec
tLowPressure
High Pressure
Predictions Based on WM Distraction Hypothesis
Gain Loss Gain Loss
Rule-Based Information-Integration
Pro
po
rtio
n C
orr
ec
t
LowPressure
High Pressure
Pressure induces a Pressure induces a prevention focusprevention focus Will interact with Will interact with
the reward the reward structurestructure
MethodMethod 2 (Pressure-level) X 2 (Reward Structure) X 2 (Task Type) between-subjects design Performed 8 80-trial blocksRule-Based Information-Integration
Worthy, et al., 2009, Worthy, et al., 2009, Psychonomic Bulletin & ReviewPsychonomic Bulletin & Review
ResultsResults
Accuracy across all blocks
0.500.550.600.650.700.750.800.85
Gain Loss Gain Loss
Rule-Based Information-Integration
Pro
po
rtio
n C
orr
ec
t
LowPressure
High Pressure
Predictions Based on WM Distraction Hypothesis
Gain Loss Gain Loss
Rule-Based Information-Integration
Pro
port
ion
Cor
rect
LowPressure
High Pressure
Predictions Based on Regulatory Fit Hypothesis
Gain Loss Gain Loss
Rule-Based Information-Integration
Pro
port
ion
Cor
rect
LowPressure
High Pressure
Worthy, et al., 2009, Worthy, et al., 2009, Psychonomic Bulletin and Psychonomic Bulletin and ReviewReview
Decision Bound ModelingDecision Bound Modeling
Used to infer strategy use.Used to infer strategy use. Decision bound models assume stimuli are Decision bound models assume stimuli are
classified based on which side of the decision classified based on which side of the decision bound they fall onbound they fall on
Several models are fit to the dataSeveral models are fit to the data Best-fitting model gives information about Best-fitting model gives information about
which strategy each participant probably used which strategy each participant probably used to classify the stimulito classify the stimuli
Decision Bound ModelingDecision Bound ModelingBest Fit by Frequency Model
-50
0
50
100
150
200
250
300
350
200 225 250 275 300 325 350 375
Spatial Frequency
Sp
ati
al
Ori
en
tati
on
Best fit by Orientation Model
-50
0
50
100
150
200
250
300
350
200 225 250 275 300 325 350 375 400
Spatial Frequency
Sp
ati
al O
rien
tati
on
Best Fit by Optimal General Linear Classifier Model
-100
-50
0
50
100
150
200
250
300
100 150 200 250 300 350 400 450
Spatial Frequency
Spa
tial O
rient
atio
n
Best Fit By Random Response Model
-100
-50
0
50
100
150
200
250
300
100 150 200 250 300 350 400 450
Spatial Frequency
Sp
atia
l Ori
enta
tio
n
Model Fitting ProcedureModel Fitting Procedure
Fit each participant’s data on a block-by-block Fit each participant’s data on a block-by-block basisbasis
Used AIC to determine best fitting model for Used AIC to determine best fitting model for that blockthat blockPenalizes for free parametersPenalizes for free parameters
Examined the proportion of data sets best fit Examined the proportion of data sets best fit by each model over all blocks of the task.by each model over all blocks of the task.
Model-Based AnalysisModel-Based AnalysisBest Fit by Frequency Model
-50
0
50
100
150
200
250
300
350
200 225 250 275 300 325 350 375
Spatial Frequency
Sp
ati
al
Ori
en
tati
on
Best Fit by Optimal General Linear Classifier Model
-100
-50
0
50
100
150
200
250
300
100 150 200 250 300 350 400 450
Spatial Frequency
Spa
tial O
rien
tatio
n
Best strategy for Best strategy for rule-based taskrule-based task
Best strategy for Best strategy for information-information-integration taskintegration task
Proportion Fit by Best Proportion Fit by Best ModelModel
Proportion Fit by Best Model
00.10.20.30.40.50.60.70.80.9
Gain Loss Gain Loss
Rule-Based Information-Integration
Pro
po
rtio
n O
pti
ma
l
LowPressure
High Pressure
Increase in accuracy likely due to improved strategy use.
Worthy, et al., 2009, Psychonomic Bulletin & ReviewWorthy, et al., 2009, Psychonomic Bulletin & Review
SummarySummary
Pressure does appear to operate like a prevention focus during classification learning.
Not main effect where WM is decreased
Gains mismatches with pressure-induced prevention focus
Pressure hurts rule-based performancePressure helps information-integration performance.
Losses fits with pressure-induced prevention focus
Pressure helps rule-based performance.Pressure hurts information-integration performance.
Pressure and ExpertsPressure and Experts Examined effects Examined effects
of pressure after of pressure after extensive training.extensive training. RB or II taskRB or II task 5 640-trial sessions5 640-trial sessions
Difference Between Session 4 and Session 5 accuracy
-0.025-0.020-0.015-0.010-0.0050.0000.0050.0100.0150.0200.0250.030
Rule-Based Information-Integration
Category Structure
Diff
eren
ce in
Acc
urac
y
Control
Pressure
Worthy et al., 2009, Attention, Perception and Psychophysics
Supports a different account for effects of Supports a different account for effects of pressure on expertspressure on experts
Real World ChokingReal World Choking
Examined clutch free-throw performance Examined clutch free-throw performance among NBA athletesamong NBA athletes Considered point-differential between shooter’s Considered point-differential between shooter’s
team.team. Compared percentage to career percentageCompared percentage to career percentage
Expected and Observed Proportions of Free Throws Made for Each Point
Differential
0.640.66
0.680.700.72
0.740.76
0.780.80
-5 -4 -3 -2 -1 0 1 2 3 4 5
Point Differential
Pro
po
rtio
n M
ad
e
Expected
Observed
Worthy et al., 2009, International Journal of Creativity and Problem Solving
Regulatory Fit Regulatory Fit and Decision-and Decision-
MakingMaking
Worthy, Maddox, & Markman, 2007Worthy, Maddox, & Markman, 2007
Decision-making from Decision-making from experienceexperience
Basic Design‘Gambling’ task Participants choose from two or more decks of cardsMust either maximize gains or minimize losses
Gains Losses
ModelingModeling
Task is amenable to reinforcement Task is amenable to reinforcement learning modelinglearning modeling
Can estimate parameters that Can estimate parameters that describe performancedescribe performance
Expected Value (EV)Expected Value (EV)
EV – How many points one expects to gain or lose EV – How many points one expects to gain or lose from selecting a given deckfrom selecting a given deck
Used to determine which option to chooseUsed to determine which option to choose Example Example
EVEVred deckred deck= 7 points= 7 points
EVEVblue deckblue deck= 3 points= 3 points
Exploration/Exploitation Exploration/Exploitation DilemmaDilemma
Exploit Exploit the option with the highest EVthe option with the highest EVoror
Explore Explore other options with lower EVsother options with lower EVs Must balance the need to exploit with Must balance the need to exploit with
the need for new informationthe need for new information Exploration may be more frontally Exploration may be more frontally
mediated (e.g. Daw et al., 2006).mediated (e.g. Daw et al., 2006). Working hypothesis: Regulatory fit Working hypothesis: Regulatory fit
will increase exploration will increase exploration
Task DesignTask Design
Can design tasks to favor more Can design tasks to favor more exploratory or exploitative strategies.exploratory or exploitative strategies.
Experiment 1 – Exploration-optimalExperiment 1 – Exploration-optimal Experiment 2 – Exploitation-optimal Experiment 2 – Exploitation-optimal
(Gains only)(Gains only) Use behavioral and model-based Use behavioral and model-based
analyses to test the regulatory fit analyses to test the regulatory fit hypothesishypothesis
Worthy et al., 2007Worthy et al., 2007
Experiment 1Experiment 1
Designed a task where exploring the deck with Designed a task where exploring the deck with lower EV led to better-long-term performance.lower EV led to better-long-term performance.
Had to be willing to explore the Advantageous deckHad to be willing to explore the Advantageous deck Fit should increase exploration; performance Fit should increase exploration; performance
Points Based on Number of Cards Drawn from Each Deck
0
2
4
6
8
10
1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76
Cards Drawn From Deck
Poi
nts
Ear
ned
Advantageous
Disadvantageous
MethodsMethods
Used raffle-ticket manipulation to Used raffle-ticket manipulation to manipulate regulatory focusmanipulate regulatory focus
Promotion Promotion Focus Focus (Approach)(Approach)
Achieve Global Achieve Global Performance Criterion Performance Criterion Raffle ticket for $50Raffle ticket for $50
Prevention Prevention Focus Focus (Avoidance)(Avoidance)
Achieve Global Achieve Global Performance Criterion Performance Criterion Keep $50 raffle ticket Keep $50 raffle ticket given initiallygiven initially
MethodsMethods
Achieved global criterion by either Achieved global criterion by either maximize gains or minimizing lossesmaximize gains or minimizing losses
GainsGainsGained between 1 and 10 Gained between 1 and 10
points on each draw; points on each draw; maximized gainsmaximized gains
LossesLossesLost between -10 and -1 Lost between -10 and -1
points on each draw; points on each draw; minimized lossesminimized losses
Behavioral resultsBehavioral resultsAverage Distance from Criterion
-40
-35
-30
-25
-20
-15
-10
-5
0
Gains Losses
Po
ints
Bel
ow
Cri
teri
on
Promotion Prevention
Participants in a regulatory fit came Participants in a regulatory fit came significantly closer to the performance significantly closer to the performance criterion than participants in a mismatchcriterion than participants in a mismatch
Modeling Choice Modeling Choice BehaviorBehavior
EVs of each option are updated via an EVs of each option are updated via an exponential recency-weighted algorithmexponential recency-weighted algorithm
][ 11 kkkk EVrEVEV
Current EVNew EV RewardRecency Parameter
Current EV
•If reward is greater than the current EV the EV increases
•If reward is less than the current EV the EV decreases
Action SelectionAction SelectionAction selection is probabilistically determined via choice rules (e.g. Luce, 1959)
Softmax Rule
n
b
bEV
aEV
tat
t
e
eP
1
))((
))((
,
Probability of choosing option “A”
EV for option “A”
Exploitation parameter
Sum of EVs for all options
• Higher values indicate greater exploitation • Lower values indicate greater exploration • Can directly parameterize degree of exploratory vs. exploitative behavior
Model-based resultsModel-based results Fit reinforcement-learning model to estimate the degree Fit reinforcement-learning model to estimate the degree
of exploratory vs. exploitative behavior.of exploratory vs. exploitative behavior.
Exploitation parameter
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Gains Losses
Ex
plo
ita
tio
n
Promotion Prevention
Participants in a regulatory fit had significantly lower Participants in a regulatory fit had significantly lower estimated exploitation-parameter values.estimated exploitation-parameter values.
Experiment 2Experiment 2 Designed a task where exploitation of the deck with Designed a task where exploitation of the deck with
the best expected value led to the best performance.the best expected value led to the best performance.
Reward Values Given for Each Deck Based on Trial Number
0
2
4
6
8
10
Trial
Po
ints
Aw
ard
ed
Deck A
Deck B
If fit increases exploration then If fit increases exploration then participants in a fit should do participants in a fit should do worse.worse.
ResultsResults
Only ran participants with a gains reward Only ran participants with a gains reward structurestructure
Participants in a regulatory fit were Participants in a regulatory fit were further from the performance criterionfurther from the performance criterion
Average Distance from Criterion
-120
-100
-80
-60
-40
-20
0
Po
int
Belo
w C
rite
rio
n
Promotion Prevention
Model-Based ResultsModel-Based Results
Participants in a fit were less exploitative Participants in a fit were less exploitative than those in a mismatchthan those in a mismatch
Exploitation Parameter
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Exp
loit
atio
n
Promotion Prevention
SummarySummary
Regulatory fit influenced the decision-making behavior
Fit – greater explorationMismatch greater exploitation
Social pressure induces a prevention focusInfluences regulatory fitDifferential performance on category-learning tasks
Three-way interaction Regulatory focus – Promotion vs. preventionReward Structure – Maximize gains vs. minimize lossesTask Demands – Rule-based vs. information-integration; exploration-optimal vs. exploitation-optimal
Expected Reward Expected Reward ComparisonComparison
Extended decision-Extended decision-making paradigm to making paradigm to ratio vs. difference ratio vs. difference comparisonscomparisons Are EVs compared via Are EVs compared via
ratio or differences?ratio or differences? Manipulated whether Manipulated whether
difference or ratio difference or ratio preserved.preserved.
Changing the ratio Changing the ratio between EVs affected between EVs affected performanceperformance
Worthy et al., 2008, Memory and Cognition
Total Adjusted Points Earned
450
460
470
480
490
500
510
520
530
540
Control DifferencePreserving
RatioPreserving
Po
ints
Research ApproachResearch Approach Categorization and Decision-making Categorization and Decision-making
taskstasks Behavioral analysisBehavioral analysis
Mathematical modelingMathematical modeling Decision-bound modelingDecision-bound modeling Reinforcement-learning modelingReinforcement-learning modeling
Ground theories in neuroscienceGround theories in neuroscience Leads to novel predictionsLeads to novel predictions
Current & Future Current & Future DirectionsDirections
‘Why’ does regulatory fit influence behavior and cognition Working memory hypothesis
Fit increases WM memory capacityNot yet directly testedTest using regulatory focus and social pressure manipulation in WM tasks.Test by adding WM span as an additional factor on categorization and decision-making tasks.
Regulatory fit and short-term vs. long-term decision-making
Does fit reduce future discounting?People in a fit may focus more on long-term outcomes
Current & Future Current & Future DirectionsDirections
Individual DifferencesAre some less susceptible to situational factors than others?Why do some people tend to choke, while others excel?
Aging and decision-makingOlder adults appear to be more exploratory than younger adultMay value long-term over short-term outcomesPositivity biasNeural differences
Gender and decision-makingMen appear to be more exploitative than women
Current & Future Current & Future DirectionsDirections
Social vs. Monetary rewardsGive incrementally happier or angrier faces as feedback in decision-making tasks.Can use same modeling approachCompare to monetary rewardsNeural mechanisms
Current & Future Current & Future DirectionsDirections
Category learningFeedback timing
Very important for procedural learning system
Retention and generalizationDesirable difficulties
Naturalistic stimuli (x-rays – tumor detection)Interactions between multiple systems – competition vs. cooperation
Thanks!Thanks!
AcknowledgementsAcknowledgements
Todd Maddox, Art Markman, Bo Zhu, Todd Maddox, Art Markman, Bo Zhu, MaddoxLab research assistants.MaddoxLab research assistants.
Supported by NIMH grant MH077708 Supported by NIMH grant MH077708 to WTM and ABM, and a supplement to WTM and ABM, and a supplement
to DAW.to DAW.
ReferencesReferencesDaw, N.D., O’Doherty, J.P., Dayan, P., Seymour, B., & Dolan, R. (2006). Cortical Substrates for Daw, N.D., O’Doherty, J.P., Dayan, P., Seymour, B., & Dolan, R. (2006). Cortical Substrates for exploratory decisions in exploratory decisions in humans. humans. Nature,Nature, 441 (15), 876-879. 441 (15), 876-879.Grimm, L. R., Markman, A. B., Maddox, W. T., Baldwin, G. C. (2008) Differential Effects of Grimm, L. R., Markman, A. B., Maddox, W. T., Baldwin, G. C. (2008) Differential Effects of Regulatory Fit on Category Regulatory Fit on Category Learning. Learning. Journal of Experimental Social Psychology. 44Journal of Experimental Social Psychology. 44, 920-, 920-927.927.Higgins, E. T. (1997). Beyond pleasure and pain. Higgins, E. T. (1997). Beyond pleasure and pain. American Psychologist, 52American Psychologist, 52, 1280-1300., 1280-1300.
Higgins, E. T. (2000). Making a good decision: Value from fit. Higgins, E. T. (2000). Making a good decision: Value from fit. American Psychologist, American Psychologist, 5555, 1217-1230., 1217-1230.Maddox, W.T., & Ashby, F.G. (1993). Comparing decision bound and exemplar models of Maddox, W.T., & Ashby, F.G. (1993). Comparing decision bound and exemplar models of categorization. categorization. Perception and Perception and Psychophysics, 53Psychophysics, 53, 49-70., 49-70.Maddox, W. T., & Ashby, F. G. (2004). Dissociating explicit and procedural-learning based Maddox, W. T., & Ashby, F. G. (2004). Dissociating explicit and procedural-learning based systems of perceptual category systems of perceptual category learning. learning. Behavioural ProcessesBehavioural Processes, , 6666, 309-332., 309-332.Maddox, W. T., Markman, A. B., & Baldwin, G. C. (2006). Using classification to understand the Maddox, W. T., Markman, A. B., & Baldwin, G. C. (2006). Using classification to understand the motivation-learning interface. motivation-learning interface. Psychology of Learning and Motivation, 47Psychology of Learning and Motivation, 47, 213-250., 213-250.Markman, A.B., Maddox, W.T., Worthy, D.A. (2006) Choking and excelling under pressure. Markman, A.B., Maddox, W.T., Worthy, D.A. (2006) Choking and excelling under pressure. Psychological Science. 17Psychological Science. 17, 944-, 944- 948.948.Shah, J., Higgins, E. T., & Friedman, R. S. (1998). Performance incentives and means: How Shah, J., Higgins, E. T., & Friedman, R. S. (1998). Performance incentives and means: How regulatory focus influences goal regulatory focus influences goal attainment. attainment. Journal of Personality and Social Journal of Personality and Social Psychology, 74Psychology, 74, 285 - 293., 285 - 293.Spiegel, S., Grant-Pillow, H., & Higgins, E. T. (2004). How regulatory fit enhances motivational Spiegel, S., Grant-Pillow, H., & Higgins, E. T. (2004). How regulatory fit enhances motivational strength during goal pursuit. strength during goal pursuit. European Journal of Social Psychology, 34European Journal of Social Psychology, 34, 39-54., 39-54.Worthy, D.A., Maddox, W.T., & Markman, A.B. (2007). Regulatory Fit Effects in a Choice Task. Worthy, D.A., Maddox, W.T., & Markman, A.B. (2007). Regulatory Fit Effects in a Choice Task. Psychonomic Bulleting and Psychonomic Bulleting and ReviewReview,, 14 14, 1125-1132. , 1125-1132. Worthy, D.A., Maddox, W.T., & Markman, A.B. (2008). Ratio and Difference Comparisons of Worthy, D.A., Maddox, W.T., & Markman, A.B. (2008). Ratio and Difference Comparisons of Expected Reward in Decision Expected Reward in Decision Making Tasks. Making Tasks. Memory and Cognition, 36, Memory and Cognition, 36, 1460-1469.1460-1469.Worthy, D.A., Markman, A.B., & Maddox, W.T. (2009A). What is pressure? Evidence for social Worthy, D.A., Markman, A.B., & Maddox, W.T. (2009A). What is pressure? Evidence for social pressure as a type of regulatory pressure as a type of regulatory focus. focus. Psychonomic Bulletin and Review, 16, Psychonomic Bulletin and Review, 16, 344-344-349349..Worthy, D.A., Markman, A.B., & Maddox, W.T. (2009B). Choking and excelling at the free Worthy, D.A., Markman, A.B., & Maddox, W.T. (2009B). Choking and excelling at the free throw linethrow line. The International . The International Journal of Creativity & Problem Solving, 19, 53-58.Journal of Creativity & Problem Solving, 19, 53-58. Worthy, D.A., Markman, A.B., & Maddox, W.T. (2009C). Choking and Excelling Under Pressure Worthy, D.A., Markman, A.B., & Maddox, W.T. (2009C). Choking and Excelling Under Pressure in Experienced Classifiers. in Experienced Classifiers. Attention, Perception and Psychophysics, 71, Attention, Perception and Psychophysics, 71, 924-935.924-935.
Aging and Decision-Aging and Decision-MakingMaking
Estimated Exploitation Parameter Values
0
0.2
0.4
0.6
0.8
1
Gain Loss
Exp
loita
tion
Old
Young
Older adults use a more exploratory than younger Older adults use a more exploratory than younger adults.adults.
Task favored an exploitative strategyTask favored an exploitative strategyWorthy et al., in preparationWorthy et al., in preparation
Aging and Decision-Aging and Decision-MakingMaking
-Task favored an exploratory strategy-Task favored an exploratory strategy
Exploitation Parameter Values Exploratory Task
0
0.2
0.4
0.6
0.8
1
Gain Loss
Expl
oita
tion
Older
Younger
Worthy et al., in preparationWorthy et al., in preparation
Aging and Decision-Aging and Decision-MakingMaking
-Looked at “Directed Exploration” – not just more -Looked at “Directed Exploration” – not just more randomrandom
Reward Given Based on Previous Long-term Increasing Options
0
20
40
60
80
100
1 2 3 4 5 6 7 8 9 10
Number of Long-Term Increasing Options in Last 10 Trials
Rew
ard
Long-term decreasing
Long-term increasing
Worthy et al., in preparationWorthy et al., in preparation
Aging and Decision-Aging and Decision-MakingMaking
-Older adults explore the decision space-Older adults explore the decision space
-Do not focus only on short-term rewards-Do not focus only on short-term rewards
Advantageous Choices Over first 100 trials
0123456789
10
1 2 3 4 5 6 7 8 9 10
Block (10-trials)
Num
ber o
f Adv
anta
geou
s C
hoic
es Older
Younger
Worthy et al., in preparationWorthy et al., in preparation
Gender and Decision-Gender and Decision-MakingMaking
-Males tend to be more exploitative than females-Males tend to be more exploitative than females
Exploitation Parameter Values Based on Gender
0
0.2
0.4
0.6
0.8
1
1.2
Gains Losses
Exp
loita
tionnio
n
Males
Females
Feedback Delay and Feedback Delay and Category LearningCategory Learning
--Feedback timing important for II learning onlyFeedback timing important for II learning only-500ms appears to be the best time for -500ms appears to be the best time for procedural system to receive feedbackprocedural system to receive feedback
Accuracy with Different Feedback Delay Intervals
0.550.600.650.700.750.80
RB II
Pro
portio
n Cor
rect
0ms
500ms
1000ms
Worthy et al., in preparationWorthy et al., in preparation
Feedback Delay and Feedback Delay and Category LearningCategory Learning
--Separated visual and motor Separated visual and motor responseresponse feedback feedback componentscomponents-Important for system to receive visual and -Important for system to receive visual and motor information that a response has been motor information that a response has been mademade Worthy et al., in preparationWorthy et al., in preparation
Accuracy with Different Offset and Feedback Delay Intervals
0.55
0.65
0.75
250ms-250ms 250ms-500ms 0ms-500ms
Pro
portio
n Cor
rect
Desirable Difficulties in II Desirable Difficulties in II learninglearning
--Discontinuous categories are more difficult to Discontinuous categories are more difficult to learn but may lead to better transfer learn but may lead to better transfer performance.performance.
Discontinuous
Length
Ori
en
tati
on
Continuous
Length
Ori
en
tati
on
Maddox et al., in preparationMaddox et al., in preparation
Desirable Difficulties in II Desirable Difficulties in II learninglearning
--Continuous categories are learned easier, but Continuous categories are learned easier, but transfer performance is worse.transfer performance is worse.
Maddox et al., in preparationMaddox et al., in preparation
Accuracy
0.30
0.40
0.50
0.60
0.70
0.80
1 2 3 4 5 6 7 8 9 10 1 2 3 4 T
Pro
po
rtio
n C
orr
ect
DiscontinuousContinuous