using r to assess mathematical sense-making in ... · using r to assess mathematical sense-making...

Using R to Assess Mathematical Sense-Making in Introductory Physics Courses

2010 UseR! Conference - July 21, 2010

by Brian Danielak, Eric Kuo, Michael M. Hull,and Andrew Elby

1

I work behind here

2

3

Students in conventional courses often treat formulas as algorithms.

Sherin, B. L. (2001). How Students Understand Physics Equations. Cognition and Instruction, 19(4), p. 479

4

v = v0 + at

if I know the initial velocity...

The uniform acceleration...

and the time...

Then I can calculate the final velocity!

5

Sense-making is about seeking coherence

v = v0 + at

8

v = v0 + at



and the time...


v = v0 + at

v = v0 + at+=t1

9

v = v0 + at



and the time...


v = v0 + at

v = v0 + at+

+

=

=

t1

t2

10

v = v0 + at



and the time...


v = v0 + at

v = v0 + at+

+

+

=

=

=

t1

t2

t3

11

The same mathematical structuremodels lots of other situations

12

+

+

+

=

=

=

t1

t2

t3

money = P + rt

13

+

+

+

=

=

=

x1

x2

x3

Y = β0 + β1X1

So here’s the bigQuestion

14

15

15

v = v0 + at



and the time...


v = v0 + at

v = v0 + at+

+

+

=

=

=

t1

t2

t3

10

v = v0 + at



and the time...


How can we test for thinking like this

When we usually test for thinking like this?

16

A Tale of 3 Instructors

1 32New Veteran Veteran

Sense-making

Traditional Approach

And one Final Exam

17

Multiple Choice Free Response

And one Final Exam

18

Multiple Choice Free Response

426 Students

3 Classes

2

1

6

Sense-makingitems

19

5

3

4

chosen a priori7

8

Class 1 Class 2 Class 3

3 4 2

e-score

20

21

Looking graphicallyat the data

22

Raw Score Distributions by Instructor

rawscore

coun

t

0

5

10

15

20

25

30

35

0

5

10

15

20

25

30

35

0

5

10

15

20

25

30

35

0 2 4 6 8

12

3

Sense-making(new)

Sense-making(vet)

Traditional(vet)

Across all 8 MC items

23

Escore Distributions by Instructor

escore

coun

t

0

10

20

30

40

0

10

20

30

40

0

10

20

30

40

−1 0 1 2 3 4 5 6

12

3

Sense-making(new)

Sense-making(vet)

Traditional(vet)

Just the 5 Sense-making Items

We ran a one-way ANOVA, then MCPs

24

Result 1

25

Across all 8 Multiple Choice items,

Sense-making > traditional approach

26

−0.15 −0.10 −0.05 0.00 0.05 0.10

3−2

3−1

2−1

95% family−wise confidence level

Differences in mean levels of instructorRawratio

27

- - -

TukeyHSD

p-adj

Cohen’s d

0.061 -0.019 -0.080

0.053 0.751 0.007

0.277 -0.0839 -0.365

Across all 8 MC items95% Family-wise Confidence Level

1 32 1 3 2

Result 2

28

Across the 5 specific sense-making items,

Sense-making >> traditional approach

29

−0.20 −0.15 −0.10 −0.05 0.00 0.05 0.10

3−2

3−1

2−1


Differences in mean levels of instructorEratio

30

- - -

TukeyHSD

p-adj

Cohen’s d

0.0450 -0.0871 -0.132

0.296 0.0130 < 0.001

0.182 -0.331 -0.510

On Sense-making MCs95% Family-wise Confidence Level

1 32 1 3 2

31

Exploring Free-Response Data

?

32

What happens to mechanical energyafter the explosion?

What Happens to the Mechanical Energy of the Two Blocks?

instructor

coun

t

0

20

40

60

80

100

120

140

1 2 3

q09dstays the sameincreasesblankdecreasesmissing valueuncodeable

33


instructor

coun

t

0

20

40

60

80

100

120

140

1 2 3


34


instructor

coun

t

0

20

40

60

80

100

120

140

1 2 3


43 4372

35


instructor

coun

t

0

20

40

60

80

100

120

140

1 2 3


43 4372

Mechanical EnergyStays the Same

36

43 4372


=

37

Sense-making(new)

Sense-making(vet)

Traditional(vet)


=

38

43 4372

Sense-making(new)

Sense-making(vet)

Traditional(vet)

Result 3

39

When considering energy, more traditional students asserted something counter-intuitive

43 4372

Sense-making(new)

Sense-making(vet)

Traditional(vet)


instructor

coun

t

0

20

40

60

80

100

120

140

1 2 3


40

5975

33

Mechanical EnergyIncreases

41

5975

33


42

5975

33


43

>

5975

33


44

Sense-making(new)

Sense-making(vet)

Traditional(vet)

>

5975

33


45

Sense-making(new)

Sense-making(vet)

Traditional(vet)

Result 4

46

When considering energy, more sense-making students were consistent with simple intuition.

5975

33

Sense-making(new)

Sense-making(vet)

Traditional(vet)

Ahead,

DRAGONS!

http://www.flickr.com/photos/aztlek/2421800480/in/set-72157605762980035/

!ar b"...

47






This stuff ’sin beta.

48





Students Who Quoted ‘‘Conservation of Energy'' as their #§*Explanation for q09d

instructor

coun

t

0

5

10

15

20

25

30

1 2 3

variable55x

49

811

32

Mechanical EnergyStays the Same...

50

811

32

Mechanical EnergyStays the Same...

Because Energyis Conserved

51

Sense-making(new)

Sense-making(vet)

Traditional(vet)

Result 5

52

When justifying their answer, more traditional students just quoted a rule, ignoring that it was tautologicaland inaccurate.

811

32

Sense-making(new)

Sense-making(vet)

Traditional(vet)

Summary

53

54

Result 3

39


43 4372

Sense-making(new)

Sense-making(vet)

Traditional(vet)

Result 4

46


5975

33

Sense-making(new)

Sense-making(vet)

Traditional(vet)

Result 5

52


811

32

Sense-making(new)

Sense-making(vet)

Traditional(vet)

29

−0.20 −0.15 −0.10 −0.05 0.00 0.05 0.10

3−2

3−1

2−1



55

http://github.com/briandk/user2010public



EricKuo

MikeHull

AyushGupta

AndyElby

Project Directors

Graduate Assistants

Thanks!

56

Many Thanks

NSF-DRL grant # 0733613NSF-EEC grant # 0835880

57

58

Result 3

39


43 4372

Sense-making(new)

Sense-making(vet)

Traditional(vet)

Result 4

46


5975

33

Sense-making(new)

Sense-making(vet)

Traditional(vet)

Result 5

52


811

32

Sense-making(new)

Sense-making(vet)

Traditional(vet)

29

−0.20 −0.15 −0.10 −0.05 0.00 0.05 0.10

3−2

3−1

2−1



using r to assess mathematical sense-making in ... · using r to assess mathematical sense-making...

Documents