Emily R. Fyfe & Bethany Rittle-JohnsonVanderbilt University

Marci S. DeCaroUniversity of Louisville

The Effects of Feedback During Exploratory Math

Practice

1

SREE Fall 2011

How do children learn best?

Two schools of thought have emerged

2

• Advocates of Direct Instruction(Kirschner, Sweller, & Clark, 2006)

• Advocates of Discovery Learning(Bruner, 1961; Kuhn, 1989; Piaget, 1973)

An Integrated Perspective

No need for this strict dichotomy

3

“A mixture of guidance and exploration is needed” (Mayer, 2004)

“Proponents of constructivism and direct instruction…each have something to learn from the other” (Rieber, 1992)

“Characterizing discovery and direct instruction as diametrically opposed…has done a disservice to both approaches” (Wilson, et al, 2010)

“There’s a place for both direct instruction and student-directed inquiry” (Kuhn, 2007)

Combining Instruction and Discovery

Exploration prior to instruction facilitates learning(DeCaro & Rittle-Johnson, 2011; Schwartz & Bransford,

1998)

Hints or coaching during problem solving is better than pure problem solving alone

(Mayer, 2004)

Recent meta-analysis indicates that guided discovery is better than unassisted discovery or direct instruction

(Alfieri et al, 2010)

4

Questions remain…

Which elements of instruction are most suitable to incorporate within exploration?

For whom is guided exploration most advantageous and why?

5

Feedback During Exploration

Feedback represents one element of instruction that may be particularly effective in combination with exploration

What is Feedback?• Any information that the learner can use

to confirm, reject, or modify prior knowledge

• Accuracy information, hints, etc.

6

Types of Feedback

Outcome Feedback• Provides information

about learner’s answer

• Used extensively

• Related to strong, positive effects in past work compared to no feedback

Strategy Feedback• Provides information

about how answer was obtained

• Only examined in a few previous studies

• Better than outcome feedback in terms of strategy selection

7(Ahmad, 1988; Kluger & DeNisi, 1996; Luwel et al., 2011)

Why feedback?

May reduce disadvantages of discovery by guiding the learner’s search for information

Helps identify errors and encourages search for plausible alternatives (e.g., new strategies)

(Hattie & Timperley, 2007; Mory, 2004)

But…• Past research indicates variable efficacy

(Kluger & DeNisi, 1996)

• May only benefit a subset of learners8

What about prior knowledge?

Expertise reversal effect• Instructional technique is effective for novices,

but loses its benefits for high-knowledge learners

(Kalyuga, Ayres, Chandler, & Sweller, 2003)• Example: worked examples vs. problem solving

Low knowledge learners benefit from more external guidance; high knowledge learners benefit from less

Perhaps feedback during exploration only helps children who have low prior knowledge

9

Why the reversal?

Due to existing cognitive resources(Paas, Renkl, & Sweller, 2003)

• Novices lack schemas; need external guidance to reduce cognitive load

• High-knowledge learners have schemas; additional guidance creates more cognitive load

10

Goals of this study

Examine the effects of feedback during exploratory math practice for children with varying levels of prior knowledge

Specifically• Compare feedback vs. no feedback • Compare outcome vs. strategy feedback• Look at effects of prior knowledge

11

Hypotheses

12

Strategy Feedback > Outcome Feedback

Feedback > No Feedback

Feedback better for children with low prior knowledge

Ho 1:

Ho 3:

Ho 2:

Domain: Mathematical Equivalence

Concept that two sides of an equation represent the same amount and are interchangeable• Commonly represented by equal sign

(=)

13

3 + 7 + 8 = 3 + _3 + 7 + 8 = 3 + _ 6 + 4 = _ + 86 + 4 = _ + 8

Why Study Math Equivalence?

Fundamental concept in arithmetic and algebra

Very difficult for children in U.S.• Interpret equal sign as an operator symbol that

means “the total” as opposed to relational symbol

(McNeil, 2008; Rittle-Johnson & Alibali, 1999)

• In one study, only 24% of U.S. children in 3rd and 4th grade solved math equivalence problems correctly

(McNeil & Alibali, 2000)

14

Participants

Worked with 91 children (2nd & 3rd grade)- M age = 8 yrs, 7 mo- 53 females, 38 males- 45% white, 40% black, 15% other- 47% receive free or reduced lunch

15

Design and Procedure

Session 1: Pretest (~25 minutes)• Excluded if score >80% on pretest

measures

Session 2: Intervention & Posttest (~50 minutes)

Session 3: Two-week Retention Test (~25 minutes)

16

Tutoring Intervention

Exploratory Practice• Attempt to solve 12 math equivalence

problems• Randomly assigned to 1 of 3 conditions

• No Feedback (n = 31)• Outcome Feedback (n = 32)• Strategy Feedback (n = 28)

MidtestBrief conceptual instruction

(DeCaro & Rittle-Johnson, 2011)

17

Exploratory Practice

Find the number that goes in the blank.3 + 4 + 8 = 3 + ☐

How did you solve that problem?

No Feedback: “OK, let’s move on to the next problem.”

Outcome Feedback: “Good try, but that’s not the correct answer. The correct answer is 12.”

Strategy Feedback: “Good try, but that’s not a correct way to solve that problem.”

18

Assessment of Math Equivalence

Procedural Knowledge• Use correct strategy to solve problems

Conceptual Knowledge• Understand concept of equivalence

19

7 + 6 + 4 = 7 + _7 + 6 + 4 = 7 + _

What does the equal sign mean?

What does the equal sign mean?

4 + 8 = 8 + 4True or False?

4 + 8 = 8 + 4True or False?

(Rittle-Johnson, Matthews, Taylor, & McEldoon, 2011)

Used at Pretest, Midtest, Posttest, & Retention Test

6 - 4 + 3 = _ + 36 - 4 + 3 = _ + 3

Analysis & Results

Contrast-based ANCOVA (West, Aiken, & Krull, 1996)

• Two contrast-coded condition variables- Feedback (no feedback vs. two feedback conditions

combined)

- Feedback Type (outcome feedback vs. strategy feedback)

• Two condition x prior knowledge interactions

• Three covariates

Interaction follow-up• Categorize as low vs. high knowledge

(median split)

• Simple main effects of condition 20

Procedural Knowledge

21

Repeated Measures ANCOVA: Midtest, Posttest, and Retention Test.

Procedural Knowledge

22

Overall feedback x prior knowledge interaction, F(1, 83) = 7.05, p = .01Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 3.84, p = .05High Knowledge: Feedback vs. No Feedback, F(1, 83) = 4.61, p = .04

*

*

Procedural Knowledge Results

Expertise reversal effect

Feedback during exploratory math practice is more beneficial than no feedback, but only for children with low knowledge

For children with high prior knowledge, the reverse is true; they benefit more from no feedback

23

Intervention Activities

Subjective Cognitive Load (NASA-TLX, Hart & Staveland, 1988)

• I had to work hard to solve those problems.• I was stressed and irritated when I had to

solve those problems.• Mean rating on agreement scale from 1 to 5.

Problem-Solving Strategy Use• Variability of correct & incorrect strategies

24

Cognitive Load

25

Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 1.15, p = .29High Knowledge: Feedback vs. No Feedback, F(1, 83) = 6.05, p = .02

*

Cognitive Load

Supports expertise reversal effect explanation

Feedback may have hurt high-knowledge learners’ performance because of increased cognitive load

An effect not found for low-knowledge learners

26

Strategy Coding Scheme

27

Strategy Sample explanation (4 + 5 + 8 = __ + 8)

Correct Strategies

Equalize I added 4, 5, and 8 and got 17. And 9 plus 8 is 17.

Add-Subtract I added 4, 5, and 8 and got 17. And 17 minus 8 is 9.

Grouping I took out the 8’s and I added 4 plus 5.

Incorrect Strategies

Add-All I added the 4, 5, 8 and 8.

Add-to-Equal I just added the first three, the 4, 5, and 8.

Carry I saw a 4 here, so I wrote a 4 in the blank.

Ambiguous I used 8 plus 8, and then 5.

Perseveration

28

* P = .01

*

Perseveration = Using the same incorrect strategy on all the problems.

Incorrect Strategy Variability

29

*

Ranged from 0 to 5.Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 5.66, p = .02High Knowledge: Feedback vs. No Feedback, F(1, 83) = 4.56, p = .04

*

Correct Strategy Variability

30

*

Ranged from 0 to 3.Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 4.76, p = .03High Knowledge: Feedback vs. No Feedback, F(1, 83) = 0.16, p = .90

Strategy Variability

For children with low prior knowledge, feedback prevented perseveration and led to the generation of diverse strategies

- May explain why these children learned more when they received feedback than when they did not

31

For children with higher prior knowledge, feedback led to the generation of more incorrect, but not correct strategies

- May help explain why feedback had negative impact

Summary

Feedback led to higher procedural knowledge of math equivalence than no feedback, but only for children with low prior knowledge

For children with high prior knowledge, no feedback was better

No differences between outcome feedback and strategy feedback

32

Implications

Theoretical• Extends expertise reversal effect to

feedback• Clarifies research on discovery learning

Practical• Pay more attention to when you give

feedback during tutoring and teaching

33

Thank You

Children’s Learning Lab

Bethany Rittle-Johnson

Marci DeCaroLaura McLean

Maryphyllis CreanLucy Rice

34

Funding SourcesExpERT Training Grant, through IES

to FyfeNSF CAREER grant to

Rittle-Johnson

Instruct vs. Discover

Direct Instruction• Explore the problems on your

own with no guidance

• Find target information and new strategies independently

• But…• Overwhelms working

memory• Might never locate target

info or invent correct strategy

Discovery Learning• Told or shown how to

solve the problems

• Provides structure and reduces task ambiguity

• But…• Limits self-discovery• Might limit learner

engagement

35(Kirschner, Sweller, & Clark, 2006; Mayer, 2004; Sweller, 1988)

Exploratory Practice

36

Children’s performance on 12 problems during intervention (Percent Correct)

Instruction

Let’s take a look at this problem.3 + 4 = 3 + 4

There are two sides to this problem, one on the left side of the equal sign and one on the right side of

the equal sign…

The equal sign means that the left side of the equal side is the SAME AMOUNT AS the right side of

the equal sign. That is, things on both sides of the equal sign are equal or the same.

37

Conceptual Knowledge

38

Feedback type x prior knowledge interaction, F(1, 83) = 4.82, p = .03.Low knowledge: No effect of feedback type, F(1, 83) = 0.51, p = .48High knowledge: Main effect of feedback type, F(1, 83) = 8.21, p = .005

Intervention – Strategy Use

39

*

*

*

Note. Differences are between no feedback condition and two feedback conditions combined: * p < .05

Incorrect Correct

Strategy Variability

40

Note. Difference is between no feedback condition and two feedback conditions combined: * p < .001

* *