human metacognition john dunlosky kent state university

Human Metacognition

John Dunlosky

Kent State University

Talk Overview

• Definitions and methods

• How accurate are people’s monitoring judgments?

• How do people monitor cognitive processes?

• How do people control cognitive processes?

• monitoring cognition

• controlling cognition

Metacognitive Components

• knowledge about cognition

adapted from Nelson & Narens (1990)

Monitoring and control of learning and memory processes and products

Metamemory

General Overviews

Dunlosky (2004) In Hunt & Ellis, Fundamentalsof Cognitive Psychology.

Metcalfe (2000). In Tulving & Craik, The Oxford Handbook of Memory.

adapted from Nelson & Narens (1990)

Talk Overview

• Definitions and methods

• How accurate are people’s monitoring judgments?

• How do people monitor cognitive processes?

• How do people control cognitive processes?

Judgment of Learning

How confident are you that in about 10minutes from now you will recall the secondword of the pair when shown the first word?

0 = definitely will not recall, 20 = 20 % sure,40 = 40 % sure, 60…, 80…, 100 = definitely will recall

dog - spoon dog - ?

study JOL

Items

Sample Data from 1 Participant

dog - spoon

chair - flood

daffodil - blood

closet - star

acrobat - ice

Judgment

0

80

60

0

20

Recall

1

0

0

1

1

. . .. . .

Talk Overview

• Definitions and methods

• How accurate are people’s monitoring judgments?

• How do people monitor cognitive processes?

• How do people control cognitive processes?

Accuracy is measured by the correlation between judgments and recall:

+1.0 = perfect accuracy 0 = no predictive accuracy

Relatively Accurate Relatively INaccurate

dog - spoon

chair - flood

daffodil - blood

closet - star

acrobat - ice

doctor - lobster

100

80

60

40

0

0

1

1

0

1

0

0

JOLs recall

correlation = +.78

dog - spoon

chair - flood

daffodil - blood

closet - star

acrobat - ice

doctor - lobster

1

0

1

0

1

0

JOLs recall

100

80

60

40

0

0

correlation = +.25

Relative accuracy of the judgments of learning

Older Younger

Connor et al. (1997) Study 1 (unrelated pairs) Study 2 (related pairs) Study 3 (mixed list)Dunlosky & Hertzog (2000) Trial 1 Trial 2Hertzog et al. (2003) Unrelated pairs Related pairs Across all items

.44 .29

.50 .57

.49 .55

.48 .55

.48 .43

.23 .20

.14 .08

.46 .37

Median .47 .36

study

dog - spoon dog - ?

Immediate Judgment of Learning

study

dog - spoon dog - ?

Test

about 10 min

study

dog - spoon dog - ?

Immediate Judgment of Learning

study

dog - spoon dog - ?

Test

about 10 min

study

dog - spoon dog - ?

Delayed Judgment of Learning

about 30 sec

0

0.2

0.4

0.6

0.8

1Younger adults

Older adults

Immediate Delayed

Acc

urac

y: C

orre

latio

n be

twee

n Ju

dgm

ent

and

Rec

all P

erfo

rman

ce

Immediate Delayed

Acc

urac

y:

Cor

rela

tion

betw

een

Judg

men

tan

d R

ecal

l Per

form

ance

1

.8

.6

.4

.2

0

Connor et al. (1997)

Predictive accuracy of immediate and delayed judgments of learning

Metamemory Performance (gamma for JOL accuracy)

Delayed Immediate

Nelson & Dunlosky (1991) +.90 > +.38Dunlosky & Nelson (1992) +.93 > +.45Thiede & Dunlosky (1994) recall +.86 > +.63Dunlosky & Nelson (1994) interactive imagery +.72 > +.10 rote rehearsal +.93 > +.29 distributed repetitions +.71 > +.14 massed repetitions +.83 > +.12 single presentation +.91 > +.20Connor et al. (1997) Study 1/older adults +.88 > +.44 Study 1/younger adults +.88 > +.29 Study 2/older adults +.83 > +.50 Study 2/younger adults +.82 > +.57 Study 3/older adults +.78 > +.49 Study 3/younger adults +.82 > +.55Dunlosky et al. (1998) nitrous oxide inhaled +.93 > +.47 placebo inhaled +.82 > +.42Kelemen & Weaver (1997) Experiment 1 +.80 > +.24 Experiment 2 +.77 > +.40 Experiment 3 +.72 > +.30Kennedy & Yorkston (2000) TBIb/list 1, group 1 +.93 > +.52 TBI/list 1, group 2 +.90 > +.41 Control/list 1, group 1 +.86 > +.35 Control/list 1, group 2 +.82 > +.48

adapted from Nelson & Narens (1990)

Talk Overview

• Definitions and methods

• How accurate are people’s monitoring judgments?

• How do people monitor cognitive processes?

• How do people control cognitive processes?

How do people make metacognitive judgments?

• Direct-access hypothesis

proposed by Hart (1965)

Individuals monitor the memorytrace of the sought-after response

Prediction: Accuracy will always be above chance

Benjamin, Bjork, & Schwartz (1998)

Answered trivia questions.

Made a JOL for each answer: Will you recall your answer(without the question cue) in 20 minutes from now?

Test of Free Recall

Answers that take longer to retrieve end up withstronger memory traces

Prediction from direct-access hypothesis:

JOLs > for responses that take longer to retrieve

from Benjamin et al. (1998)

from Benjamin et al. (1998)

How do people make metacognitive judgments?

• Direct-access hypothesis – disconfirmed repeatedly

• Inference-based accounts

Individuals infer whether a particular response will be (or has been) remembered based upon cues that are available when making a given judgment.

Accuracy is a function of cue diagnosticity.

Two Prominent Inference-based AccountsFor Feeling-of-Knowing Judgments

What is the capitol of California? {Don’t know.}

FOK = 80% chance of recognizing

Cue familiarity hypothesis

Accessibility hypothesis

Data from Metcalfe, Schwartz, and Joaquim (1993).

Group List 1 List 2

A-B, A-B pickle – lucky pickle – luckytable – picture table – picture butter – psyche butter – psyche

 A-D, A-B pickle – carpet pickle – lucky

table – maple table – picture butter – sandal butter – psyche

Studied List 1 then List 2

Cued-recall on List 2 (critical list, identical for all groups)

FOK on unrecalled items: CUES are stimuli alone (pickle - ?)

Predictions concerning FOK magnitude

Cue familiarity hypothesis: AB,AB = AD,AB

Accessibility hypothesis: AB,AB > AD,AB

Data from Metcalfe, Schwartz, and Joaquim (1993).

Group List 1 List 2 % Recall FOK Magnitude

A-B, A-B pickle – lucky pickle – lucky 39table – picture table – picture butter – psyche butter – psyche

 A-D, A-B pickle – carpet pickle – lucky 17

table – maple table – picture butter – sandal butter – psyche

Predictions concerning FOK magnitude

Cue familiarity hypothesis: AB,AB = AD,AB

Accessibility hypothesis: AB,AB > AD,AB

Data from Metcalfe, Schwartz, and Joaquim (1993).

Group List 1 List 2 % Recall FOK Magnitude

A-B, A-B pickle – lucky pickle – lucky 39 48table – picture table – picture butter – psyche butter – psyche

 A-D, A-B pickle – carpet pickle – lucky 17 49

table – maple table – picture butter – sandal butter – psyche

How do people make metacognitive judgments?

Talk Overview

• Definitions and methods

• How accurate are people’s monitoring judgments?

• How do people monitor cognitive processes?

• How do people control cognitive processes?

adapted from Nelson & Narens (1990)

Controlling Study Time

How do students use monitoring to allocate study time across items?

• Standard method

• Simple answer and expected finding

• Universal finding

Standard Method

Items presented individually at a fixed rate

elimu - science

pombe - beer

buu - maggot. .

.

Participants make a judgment of learning

Standard Method

. . .

elimu - science

What is the likelihood that you will recall the translationequivalent in about 10 minutes from now? (0 - 100)

Standard Method

Item selection:Choose those items thatyou’d like to restudy.

elimu - science

pombe - beer

buu - maggot . . .

Self-paced study:Study each item as longas you’d like.

elimu - science . . .

Items

Sample Data from 1 Participant

elimu-science

pombe-beer

ndoo-bucket

zeituni-honor

kaputula-shorts

Judged Learning

0

80

60

0

20

Selection

1

0

0

1

1

Allocation (sec)

5.1

2.0

2.8

6.2

3.3

. . .. . .

Self-regulated Learning:Theory and Data

How do students use monitoring to allocate study time across items?

• Standard method

• Simple answer and expected finding

• Universal finding

Discrepancy-reduction Model

Degree ofLearning

Desired Now

PerceivedDegree ofLearning

Change inMemory

ContinueStudy

Select Item

Next Item

HasDiscrepancy

Been Reduced?Yes

No

Discrepancy-reduction Model

Degree ofLearning

Desired Now

PerceivedDegree ofLearning

Change inMemory

ContinueStudy

Select Item

Next Item

HasDiscrepancy

Been Reduced?Yes

No

0

100

Discrepancy-reduction Model

Degree ofLearning

Desired Now

PerceivedDegree ofLearning

Change inMemory

ContinueStudy

Select Item

Next Item

HasDiscrepancy

Been Reduced?Yes

No

80

100

Prediction

Inverse relation between judged learning and measures of allocation

(selection or self-paced study)

Items

Sample Data from 1 Participant

elimu-science

pombe-beer

ndoo-bucket

zeituni-honor

kaputula-shorts

Judged Learning

0

80

60

0

20

Selection

1

0

0

1

1

Allocation (sec)

5.1

2.0

2.8

6.2

3.3

. . .. . .

Self-regulated Learning:Theory and Data

How do students use monitoring to allocate study time across items?

• Standard method

• Simple answer and expected finding

• Universal finding

Selection of Items for Study

Research demonstrating inverse relation

Baldi (1996, unpublished dissertation)Cull & Zechmeister (1994)Dunlosky & Hertzog (1997)Masur, McIntyre, & Flavell (1973)Nelson, Dunlosky, Graf, & Narens (1994)

Research demonstrating positive relation

None

Pacing of Items for Study

Research demonstrating inverse relationBelmont & Butterfield (1971)Baker & Anderson (1982)Cornoldi (1990)Dufresne & Kobasigawa (1989)Dunlosky & Connor (1997)Kobasigawa & Metcalfe-Haggert (1993)Le Ny et al. (1972)Maki & Serra (1992)Mazzoni et al. (1990)Mazzoni & Cornoldi (1993)Nelson & Leonesio (1998)Owings et al. (1980)Rofoff et al. (1974)

Pacing of Items for Study

Research demonstrating inverse relationBelmont & Butterfield (1971)Baker & Anderson (1982)Cornoldi (1990)Dufresne & Kobasigawa (1989)Dunlosky & Connor (1997)Kobasigawa & Metcalfe-Haggert (1993)Le Ny et al. (1972)Maki & Serra (1992)Mazzoni et al. (1990)Mazzoni & Cornoldi (1993)Nelson & Leonesio (1998)Owings et al. (1980)Rofoff et al. (1974)

Pacing of Items for Study

Research demonstrating inverse relationBelmont & Butterfield (1971)Baker & Anderson (1982)Cornoldi (1990)Dufresne & Kobasigawa (1989)Dunlosky & Connor (1997)Kobasigawa & Metcalfe-Haggert (1993)Le Ny et al. (1972)Maki & Serra (1992)Mazzoni et al. (1990)Mazzoni & Cornoldi (1993)Nelson & Leonesio (1998)Owings et al. (1980)Rofoff et al. (1974)

Pacing of Items for Study

Research demonstrating inverse relationBelmont & Butterfield (1971)Baker & Anderson (1982)Cornoldi (1990)Dufresne & Kobasigawa (1989)Dunlosky & Connor (1997)Kobasigawa & Metcalfe-Haggert (1993)Le Ny et al. (1972)Maki & Serra (1992)Mazzoni et al. (1990)Mazzoni & Cornoldi (1993)Nelson & Leonesio (1998)Owings et al. (1980)Rofoff et al. (1974)

Research demonstrating positive relationNone

Self-regulated Learning:Theory and Data

How do students use monitoring to allocate study time across items?

• Standard method

• Simple answer and expected finding

• Universal finding: Consistent with discrepancy-reduction model

Under what conditions (if any) will discrepancy reduction fail to account for

allocation of study time?

When goal (degree of learning desired) is not to master all the items...

• Low performance goal

• Limited study time

DR prediction:

Adaptivity hypothesis (Thiede & Dunlosky, 1999):

Individuals will plan to allocate study time to obtain the goal with minimal effort.

Prediction: positive relation

Inverse relation between perceivedlearning and allocation.

When performance goal is low...

30 paired associates presented individually (1 sec)

Participants make judgments of learning

MANIPULATION:Less difficult goal : 6 of 30More difficult goal : 24 of 30

Item selection (choose items for restudy)

Self-paced study (study items that were selected)

from Thiede & Dunlosky (1999, JEP:LMC)

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

Allocation of study time

Item selection

More-difficult Goal

Less-difficult Goal

Cor

rela

tion

betw

een

Per

ceiv

ed It

em D

iffic

ulty

an

d M

easu

res

of S

tudy

Allo

catio

nC

orre

latio

n be

twee

n Ju

dged

Lea

rnin

g an

dM

easu

re o

f S

elf-

regu

late

d S

tudy

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

Allocation of study time

Item selection

More-difficult Goal

Less-difficult Goal

Cor

rela

tion

betw

een

Per

ceiv

ed It

em D

iffic

ulty

an

d M

easu

res

of S

tudy

Allo

catio

nC

orre

latio

n be

twee

n Ju

dged

Lea

rnin

g an

dM

easu

re o

f S

elf-

regu

late

d S

tudy

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

Allocation of study time

Item selection

More-difficult Goal

Less-difficult Goal

Cor

rela

tion

betw

een

Per

ceiv

ed It

em D

iffic

ulty

an

d M

easu

res

of S

tudy

Allo

catio

nC

orre

latio

n be

twee

n Ju

dged

Lea

rnin

g an

dM

easu

re o

f S

elf-

regu

late

d S

tudy

AdaptivePlanning

Non-masterygoal

Pacing ofStudy

Mechanism

Region ofproximal learning(Metcalfe, 2002)

Discrepancy Reduction (Thiede & Dunlosky, 1999)

Monitoring progress (Dunlosky & Thiede, 1997)

Perseverance (Metcalfe & Kornell, in press)

Students appear to use output frommonitoring to control study time in an efficient manner.

Controlling Study Time

How do people use monitoring to control?

Summary

• Methods

• How accurate are people’s monitoring judgments?

• How do people monitor cognitive processes?

• How do people control cognitive processes?

Thank You

30 paired associates presented individually (1 sec)

Participants make judgments of learning

MANIPULATION:Restricted study time : 15 secondsUnrestricted study time : 5 minutes

Item selection (choose items for restudy)

Self-paced study (study items that were selected)

from Thiede & Dunlosky (1999, JEP:LMC)

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

Allocation of study time

Item selection

Unrestricted Study Time

Restricted

Cor

rela

tion

betw

een

Per

ceiv

ed It

em D

iffic

ulty

an

d M

easu

res

of S

elf-

regu

late

d S

tudy

Cor

rela

tion

betw

een

Judg

ed L

earn

ing

and

Mea

sure

of

Sel

f-re

gula

ted

Stu

dy

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

Allocation of study time

Item selection

Unrestricted Study Time

Restricted

Cor

rela

tion

betw

een

Per

ceiv

ed It

em D

iffic

ulty

an

d M

easu

res

of S

elf-

regu

late

d S

tudy

Cor

rela

tion

betw

een

Judg

ed L

earn

ing

and

Mea

sure

of

Sel

f-re

gula

ted

Stu

dy