human metacognition john dunlosky kent state university
TRANSCRIPT
Human Metacognition
John Dunlosky
Kent State University
Talk Overview
• Definitions and methods
• How accurate are people’s monitoring judgments?
• How do people monitor cognitive processes?
• How do people control cognitive processes?
• monitoring cognition
• controlling cognition
Metacognitive Components
• knowledge about cognition
adapted from Nelson & Narens (1990)
Monitoring and control of learning and memory processes and products
Metamemory
General Overviews
Dunlosky (2004) In Hunt & Ellis, Fundamentalsof Cognitive Psychology.
Metcalfe (2000). In Tulving & Craik, The Oxford Handbook of Memory.
adapted from Nelson & Narens (1990)
Talk Overview
• Definitions and methods
• How accurate are people’s monitoring judgments?
• How do people monitor cognitive processes?
• How do people control cognitive processes?
Judgment of Learning
How confident are you that in about 10minutes from now you will recall the secondword of the pair when shown the first word?
0 = definitely will not recall, 20 = 20 % sure,40 = 40 % sure, 60…, 80…, 100 = definitely will recall
dog - spoon dog - ?
study JOL
Items
Sample Data from 1 Participant
dog - spoon
chair - flood
daffodil - blood
closet - star
acrobat - ice
Judgment
0
80
60
0
20
Recall
1
0
0
1
1
. . .. . .
Talk Overview
• Definitions and methods
• How accurate are people’s monitoring judgments?
• How do people monitor cognitive processes?
• How do people control cognitive processes?
Accuracy is measured by the correlation between judgments and recall:
+1.0 = perfect accuracy 0 = no predictive accuracy
Relatively Accurate Relatively INaccurate
dog - spoon
chair - flood
daffodil - blood
closet - star
acrobat - ice
doctor - lobster
100
80
60
40
0
0
1
1
0
1
0
0
JOLs recall
correlation = +.78
dog - spoon
chair - flood
daffodil - blood
closet - star
acrobat - ice
doctor - lobster
1
0
1
0
1
0
JOLs recall
100
80
60
40
0
0
correlation = +.25
Relative accuracy of the judgments of learning
Older Younger
Connor et al. (1997) Study 1 (unrelated pairs) Study 2 (related pairs) Study 3 (mixed list)Dunlosky & Hertzog (2000) Trial 1 Trial 2Hertzog et al. (2003) Unrelated pairs Related pairs Across all items
.44 .29
.50 .57
.49 .55
.48 .55
.48 .43
.23 .20
.14 .08
.46 .37
Median .47 .36
study
dog - spoon dog - ?
Immediate Judgment of Learning
study
dog - spoon dog - ?
Test
about 10 min
study
dog - spoon dog - ?
Immediate Judgment of Learning
study
dog - spoon dog - ?
Test
about 10 min
study
dog - spoon dog - ?
Delayed Judgment of Learning
about 30 sec
0
0.2
0.4
0.6
0.8
1Younger adults
Older adults
Immediate Delayed
Acc
urac
y: C
orre
latio
n be
twee
n Ju
dgm
ent
and
Rec
all P
erfo
rman
ce
Immediate Delayed
Acc
urac
y:
Cor
rela
tion
betw
een
Judg
men
tan
d R
ecal
l Per
form
ance
1
.8
.6
.4
.2
0
Connor et al. (1997)
Predictive accuracy of immediate and delayed judgments of learning
Metamemory Performance (gamma for JOL accuracy)
Delayed Immediate
Nelson & Dunlosky (1991) +.90 > +.38Dunlosky & Nelson (1992) +.93 > +.45Thiede & Dunlosky (1994) recall +.86 > +.63Dunlosky & Nelson (1994) interactive imagery +.72 > +.10 rote rehearsal +.93 > +.29 distributed repetitions +.71 > +.14 massed repetitions +.83 > +.12 single presentation +.91 > +.20Connor et al. (1997) Study 1/older adults +.88 > +.44 Study 1/younger adults +.88 > +.29 Study 2/older adults +.83 > +.50 Study 2/younger adults +.82 > +.57 Study 3/older adults +.78 > +.49 Study 3/younger adults +.82 > +.55Dunlosky et al. (1998) nitrous oxide inhaled +.93 > +.47 placebo inhaled +.82 > +.42Kelemen & Weaver (1997) Experiment 1 +.80 > +.24 Experiment 2 +.77 > +.40 Experiment 3 +.72 > +.30Kennedy & Yorkston (2000) TBIb/list 1, group 1 +.93 > +.52 TBI/list 1, group 2 +.90 > +.41 Control/list 1, group 1 +.86 > +.35 Control/list 1, group 2 +.82 > +.48
adapted from Nelson & Narens (1990)
Talk Overview
• Definitions and methods
• How accurate are people’s monitoring judgments?
• How do people monitor cognitive processes?
• How do people control cognitive processes?
How do people make metacognitive judgments?
• Direct-access hypothesis
proposed by Hart (1965)
Individuals monitor the memorytrace of the sought-after response
Prediction: Accuracy will always be above chance
Benjamin, Bjork, & Schwartz (1998)
Answered trivia questions.
Made a JOL for each answer: Will you recall your answer(without the question cue) in 20 minutes from now?
Test of Free Recall
Answers that take longer to retrieve end up withstronger memory traces
Prediction from direct-access hypothesis:
JOLs > for responses that take longer to retrieve
from Benjamin et al. (1998)
from Benjamin et al. (1998)
How do people make metacognitive judgments?
• Direct-access hypothesis – disconfirmed repeatedly
• Inference-based accounts
Individuals infer whether a particular response will be (or has been) remembered based upon cues that are available when making a given judgment.
Accuracy is a function of cue diagnosticity.
Two Prominent Inference-based AccountsFor Feeling-of-Knowing Judgments
What is the capitol of California? {Don’t know.}
FOK = 80% chance of recognizing
Cue familiarity hypothesis
Accessibility hypothesis
Data from Metcalfe, Schwartz, and Joaquim (1993).
Group List 1 List 2
A-B, A-B pickle – lucky pickle – luckytable – picture table – picture butter – psyche butter – psyche
A-D, A-B pickle – carpet pickle – lucky
table – maple table – picture butter – sandal butter – psyche
Studied List 1 then List 2
Cued-recall on List 2 (critical list, identical for all groups)
FOK on unrecalled items: CUES are stimuli alone (pickle - ?)
Predictions concerning FOK magnitude
Cue familiarity hypothesis: AB,AB = AD,AB
Accessibility hypothesis: AB,AB > AD,AB
Data from Metcalfe, Schwartz, and Joaquim (1993).
Group List 1 List 2 % Recall FOK Magnitude
A-B, A-B pickle – lucky pickle – lucky 39table – picture table – picture butter – psyche butter – psyche
A-D, A-B pickle – carpet pickle – lucky 17
table – maple table – picture butter – sandal butter – psyche
Predictions concerning FOK magnitude
Cue familiarity hypothesis: AB,AB = AD,AB
Accessibility hypothesis: AB,AB > AD,AB
Data from Metcalfe, Schwartz, and Joaquim (1993).
Group List 1 List 2 % Recall FOK Magnitude
A-B, A-B pickle – lucky pickle – lucky 39 48table – picture table – picture butter – psyche butter – psyche
A-D, A-B pickle – carpet pickle – lucky 17 49
table – maple table – picture butter – sandal butter – psyche
How do people make metacognitive judgments?
Talk Overview
• Definitions and methods
• How accurate are people’s monitoring judgments?
• How do people monitor cognitive processes?
• How do people control cognitive processes?
adapted from Nelson & Narens (1990)
Controlling Study Time
How do students use monitoring to allocate study time across items?
• Standard method
• Simple answer and expected finding
• Universal finding
Standard Method
Items presented individually at a fixed rate
elimu - science
pombe - beer
buu - maggot. .
.
Participants make a judgment of learning
Standard Method
. . .
elimu - science
What is the likelihood that you will recall the translationequivalent in about 10 minutes from now? (0 - 100)
Standard Method
Item selection:Choose those items thatyou’d like to restudy.
elimu - science
pombe - beer
buu - maggot . . .
Self-paced study:Study each item as longas you’d like.
elimu - science . . .
Items
Sample Data from 1 Participant
elimu-science
pombe-beer
ndoo-bucket
zeituni-honor
kaputula-shorts
Judged Learning
0
80
60
0
20
Selection
1
0
0
1
1
Allocation (sec)
5.1
2.0
2.8
6.2
3.3
. . .. . .
Self-regulated Learning:Theory and Data
How do students use monitoring to allocate study time across items?
• Standard method
• Simple answer and expected finding
• Universal finding
Discrepancy-reduction Model
Degree ofLearning
Desired Now
PerceivedDegree ofLearning
Change inMemory
ContinueStudy
Select Item
Next Item
HasDiscrepancy
Been Reduced?Yes
No
Discrepancy-reduction Model
Degree ofLearning
Desired Now
PerceivedDegree ofLearning
Change inMemory
ContinueStudy
Select Item
Next Item
HasDiscrepancy
Been Reduced?Yes
No
0
100
Discrepancy-reduction Model
Degree ofLearning
Desired Now
PerceivedDegree ofLearning
Change inMemory
ContinueStudy
Select Item
Next Item
HasDiscrepancy
Been Reduced?Yes
No
80
100
Prediction
Inverse relation between judged learning and measures of allocation
(selection or self-paced study)
Items
Sample Data from 1 Participant
elimu-science
pombe-beer
ndoo-bucket
zeituni-honor
kaputula-shorts
Judged Learning
0
80
60
0
20
Selection
1
0
0
1
1
Allocation (sec)
5.1
2.0
2.8
6.2
3.3
. . .. . .
Self-regulated Learning:Theory and Data
How do students use monitoring to allocate study time across items?
• Standard method
• Simple answer and expected finding
• Universal finding
Selection of Items for Study
Research demonstrating inverse relation
Baldi (1996, unpublished dissertation)Cull & Zechmeister (1994)Dunlosky & Hertzog (1997)Masur, McIntyre, & Flavell (1973)Nelson, Dunlosky, Graf, & Narens (1994)
Research demonstrating positive relation
None
Pacing of Items for Study
Research demonstrating inverse relationBelmont & Butterfield (1971)Baker & Anderson (1982)Cornoldi (1990)Dufresne & Kobasigawa (1989)Dunlosky & Connor (1997)Kobasigawa & Metcalfe-Haggert (1993)Le Ny et al. (1972)Maki & Serra (1992)Mazzoni et al. (1990)Mazzoni & Cornoldi (1993)Nelson & Leonesio (1998)Owings et al. (1980)Rofoff et al. (1974)
Pacing of Items for Study
Research demonstrating inverse relationBelmont & Butterfield (1971)Baker & Anderson (1982)Cornoldi (1990)Dufresne & Kobasigawa (1989)Dunlosky & Connor (1997)Kobasigawa & Metcalfe-Haggert (1993)Le Ny et al. (1972)Maki & Serra (1992)Mazzoni et al. (1990)Mazzoni & Cornoldi (1993)Nelson & Leonesio (1998)Owings et al. (1980)Rofoff et al. (1974)
Pacing of Items for Study
Research demonstrating inverse relationBelmont & Butterfield (1971)Baker & Anderson (1982)Cornoldi (1990)Dufresne & Kobasigawa (1989)Dunlosky & Connor (1997)Kobasigawa & Metcalfe-Haggert (1993)Le Ny et al. (1972)Maki & Serra (1992)Mazzoni et al. (1990)Mazzoni & Cornoldi (1993)Nelson & Leonesio (1998)Owings et al. (1980)Rofoff et al. (1974)
Pacing of Items for Study
Research demonstrating inverse relationBelmont & Butterfield (1971)Baker & Anderson (1982)Cornoldi (1990)Dufresne & Kobasigawa (1989)Dunlosky & Connor (1997)Kobasigawa & Metcalfe-Haggert (1993)Le Ny et al. (1972)Maki & Serra (1992)Mazzoni et al. (1990)Mazzoni & Cornoldi (1993)Nelson & Leonesio (1998)Owings et al. (1980)Rofoff et al. (1974)
Research demonstrating positive relationNone
Self-regulated Learning:Theory and Data
How do students use monitoring to allocate study time across items?
• Standard method
• Simple answer and expected finding
• Universal finding: Consistent with discrepancy-reduction model
Under what conditions (if any) will discrepancy reduction fail to account for
allocation of study time?
When goal (degree of learning desired) is not to master all the items...
• Low performance goal
• Limited study time
DR prediction:
Adaptivity hypothesis (Thiede & Dunlosky, 1999):
Individuals will plan to allocate study time to obtain the goal with minimal effort.
Prediction: positive relation
Inverse relation between perceivedlearning and allocation.
When performance goal is low...
30 paired associates presented individually (1 sec)
Participants make judgments of learning
MANIPULATION:Less difficult goal : 6 of 30More difficult goal : 24 of 30
Item selection (choose items for restudy)
Self-paced study (study items that were selected)
from Thiede & Dunlosky (1999, JEP:LMC)
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
Allocation of study time
Item selection
More-difficult Goal
Less-difficult Goal
Cor
rela
tion
betw
een
Per
ceiv
ed It
em D
iffic
ulty
an
d M
easu
res
of S
tudy
Allo
catio
nC
orre
latio
n be
twee
n Ju
dged
Lea
rnin
g an
dM
easu
re o
f S
elf-
regu
late
d S
tudy
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
Allocation of study time
Item selection
More-difficult Goal
Less-difficult Goal
Cor
rela
tion
betw
een
Per
ceiv
ed It
em D
iffic
ulty
an
d M
easu
res
of S
tudy
Allo
catio
nC
orre
latio
n be
twee
n Ju
dged
Lea
rnin
g an
dM
easu
re o
f S
elf-
regu
late
d S
tudy
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
Allocation of study time
Item selection
More-difficult Goal
Less-difficult Goal
Cor
rela
tion
betw
een
Per
ceiv
ed It
em D
iffic
ulty
an
d M
easu
res
of S
tudy
Allo
catio
nC
orre
latio
n be
twee
n Ju
dged
Lea
rnin
g an
dM
easu
re o
f S
elf-
regu
late
d S
tudy
AdaptivePlanning
Non-masterygoal
Pacing ofStudy
Mechanism
Region ofproximal learning(Metcalfe, 2002)
Discrepancy Reduction (Thiede & Dunlosky, 1999)
Monitoring progress (Dunlosky & Thiede, 1997)
Perseverance (Metcalfe & Kornell, in press)
Students appear to use output frommonitoring to control study time in an efficient manner.
Controlling Study Time
How do people use monitoring to control?
Summary
• Methods
• How accurate are people’s monitoring judgments?
• How do people monitor cognitive processes?
• How do people control cognitive processes?
Thank You
30 paired associates presented individually (1 sec)
Participants make judgments of learning
MANIPULATION:Restricted study time : 15 secondsUnrestricted study time : 5 minutes
Item selection (choose items for restudy)
Self-paced study (study items that were selected)
from Thiede & Dunlosky (1999, JEP:LMC)
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
Allocation of study time
Item selection
Unrestricted Study Time
Restricted
Cor
rela
tion
betw
een
Per
ceiv
ed It
em D
iffic
ulty
an
d M
easu
res
of S
elf-
regu
late
d S
tudy
Cor
rela
tion
betw
een
Judg
ed L
earn
ing
and
Mea
sure
of
Sel
f-re
gula
ted
Stu
dy
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
Allocation of study time
Item selection
Unrestricted Study Time
Restricted
Cor
rela
tion
betw
een
Per
ceiv
ed It
em D
iffic
ulty
an
d M
easu
res
of S
elf-
regu
late
d S
tudy
Cor
rela
tion
betw
een
Judg
ed L
earn
ing
and
Mea
sure
of
Sel
f-re
gula
ted
Stu
dy