Example Questions

Developing a reasoning inventory for measuring physics quantitative literacy

Trevor I. Smith,1 Suzanne White Brahmia,2 Alexis Olsho,2 Andrew Boudreaux,3 Philip Eaton4

1 2 3 4

DUE-1832880 DUE-1832836 DUE-1833050

Promoting quantitative literacy in physics

• Physics Quantitative Literacy (PQL): the ability to reason mathematically in the context of physics

• A goal of many introductory physics courses; development is often less than desired (Brahmia, 2017).

• Enhancing PQL development may: - improve students’ knowledge of mathematics

(Ellis, 2007; Thompson, 2010), - better prepare them for future demands to

think mathematically (Caballero, et al., 2015), and

- promote increased equity and inclusion in physics instruction (Brahmia and Boudreaux, 2017; Boaler, 2015).

ReferencesAdams, W. K., & Wieman, C. E. (2010).

Development and validation of instruments to measure learning of expert-like thinking. Int. J. Sci. Educ., 33(9), 1289.

Arons, A. B. (1983). Student patterns of thinking and reasoning. Phys. Teach., 21(9), 576.

Bajracharya, R. R., Wemyss, T. M., & Thompson, J. R. (2012). Student interpretation of the signs of definite integrals using graphical representations. AIP Conf. Proc., 1413(1), 111.

Boudreaux, A., Kanim, S., & Brahmia, S. (2015). Student facility with ratio and proportion: Mapping the reasoning space in introductory physics. arXiv preprint arXiv:1511.08960.

Brahmia, S., & Boudreaux, A. (2017). Signed quantities: Mathematics based majors struggle to make meaning. In Proceedings of the 20th annual conference on RUME. San Diego, CA.

Brahmia, S. S., & Boudreaux, A. (2016). Exploring student understanding of negative quantity in introductory physics contexts. In Proceedings of the 19th annual conference on RUME (p. 79). Pittsburgh, PA.

Brahmia, S. W. (2017). Negative quantities in mechanics: a fine-grained math and physics conceptual blend? In Phys. Educ. Res.Conf. 2017 (p. 64). Cincinnati, OH.

Carlson, M., Oehrtman, M., & Engelke, N. (2010). The precalculus concept assessment: A tool for assessing students’ reasoning abilities and understandings. Cognition Instruct., 28(2), 113.

Hayes, K., & Wittmann, M. C. (2010). The role of sign in students’ modeling of scalar equations. Phys. Teach., 48(4), 246.

Vlassis, J. (2004). Making sense of the minus sign or becoming flexible in “negativity”. Learn. and Instr., 14(5), 469.

Score Distribution • Scores range from 1 to 20 correct • Small increases from course to course • Broad range of item difficulty values

0 5 10 15 20

0.0

0.2

0.4

0.6

0.8

1.0

Question 17

Score

Percentage

A

A A

A

AA

A

A AAAA

A

A

A A

A

A

A

A

B

B

B

BB

B

BBB B

B B

B BBBB

B B B

C

C C C CCC C C

CCC C

CC C

C C C C

D

D

D

DD D D

DDD D D

D

D

DD

DD D D

E

E

E

EEE E E E E E E E

E EE E

E E E

F

F

F

F F

FFFFF F F F

FFF F

F F F

G

G

GG G

G G G GGG G G

G

G

GG

G G G

0 5 10 15 20

0.0

0.2

0.4

0.6

0.8

1.0

Question 9

Score

Percentage

AAA

AA A

AAA A A A

AA A

A AA A A

BB

BB B

B

B

B

BB B

B B BB B B B B B

C

C CCC

C CC

CCC C C

C C

C C

C C C

DD

D

D D DDDDD

DDD D D

DD

D D D

EEEEE E E

EE E E E

E E EE E E E E

Mean Median St. Dev. Skewness KurtosisOverall 10.9 11 3.55 -0.30 -0.29Phys121 10.3 11 3.36 -0.21 -0.33Phys122 11.0 11 3.70 -0.32 -0.34Phys123 11.7 12 3.44 -0.47 -0.03

0.000

0.025

0.050

0.075

0.100

0.125

0 5 10 15 20Score on PIQL

Frac

tion

of S

tude

nts

Multiple-Choice Multiple-Response • Most students choose at least one correct response • Four-level scoring scheme

- All Correct - Some Correct (Choosing at least one correct response, but no incorrect responses)

- Both (Choosing at least one correct and one incorrect response)

- Incorrect • Progression over time varies by question

0.0

0.2

0.4

0.6

0.8

1.0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20Question

CTT

Diff

icul

ty

Overall DifficultyFraction of students who answer correctly

0.00

0.25

0.50

0.75

1.00

Flag−Q9 Charge−Q16 Work−Q17Question

Frac

tion

of S

tude

nts

CorrectnessAll Correct

Some Correct

Both

Incorrect

0.00

0.25

0.50

0.75

1.00

Phys121 Phys122 Phys123

Frac

tion

of S

tude

nts

CorrectnessAll Correct

Some Correct

Both

Incorrect

Q9: Flag of Bhutan

0.00

0.25

0.50

0.75

1.00

Phys121 Phys122 Phys123

Frac

tion

of S

tude

nts

CorrectnessAll Correct

Some Correct

Both

Incorrect

Q16: Electric Charge

0.00

0.25

0.50

0.75

1.00

Phys121 Phys122 Phys123

Frac

tion

of S

tude

nts

CorrectnessAll Correct

Some Correct

Both

Incorrect

Q17: 1D Work

Learn about workLearn about charge

Which Responses do Students Choose?• Item Response Curves (IRCs) Plot the probability

of a student choosing a particular response, given her/his overall score on the PIQL

• Useful for seeing which of the answers students choose (particularly for MCMR questions)

• Only 1 student scored 19, only 1 student scored 20

0 5 10 15 20

0.0

0.2

0.4

0.6

0.8

1.0

Question 16

Score

Percentage

A A

A

A

A A A A A A A A AA A A A A A AB

B B

B BBBBB B

B B B

BB B

B

B

B B

C

C

C

C

CC

CCC C C C C

CC C

C

C

C C

D

DD

D

D

DD

D DD D

DDD

DD

D

D

D D

E

E

E

EE

EE E

EE E

E E E EEE E E E

Enter: the PIQL •We are developing the Physics Inventory of Quantitative Literacy (PIQL): an assessment instrument for measuring PQL across the physics curriculum.

•20 multiple-choice questions - 11 Single response - 9 Multiple response (3 have more than one correct response)

•Given as a pretest in three introductory physics courses - Phys121, Mechanics (N = 424) - Phys122, Electricity & Magnetism (N = 405) - Phys123, Thermodynamics/Waves (N = 329)

•We focus on three constructs: - proportional reasoning(Arons, 1983; Boudreaux, et al., 2015)

- reasoning with signed quantities (Bajracharya, et al., 2012; Brahmia and Boudreaux, 2016; Brahmia, 2017; Hayes and Wittmann, 2010; Vlassis, 2004)

- co-variational reasoning (Carlson, et al., 2010) • Typical test statistics (such as Cronbach’s α) may

not be relevant because we are trying to measure multiple constructs, want some challenging items that would demonstrate mastery, and don’t want students and instructors to be discouraged by extremely low scores (Adams and Wieman, 2010).

Research Questions •How does PQL develop throughout the introductory physics course sequence?

•How do students’ understanding of physics content and mathematical reasoning skills interact to impact their PQL abilities?

Important: Some of the questions on this page may have more than one correct answer. If you see theprompt “Choose all that apply”, select ALL ANSWERS you believe are correct.

9. Shown at right is a picture of a small flag of the Kingdomof Bhutan, which has a dragon along its diagonal. A secondversion of the flag, not shown, is 1.5 times taller than thesmaller flag and is also 1.5 times wider.Which of the quantities below are larger by a factor of 1.5for the larger flag compared to the smaller flag? Choose allthat apply.

a. The distance around the edge of the flag.

b. The amount of cloth needed to make the flag.

c. The length of the curve forming the dragon’s backbone.

d. The diagonal of the flag.

e. None of these quantities are larger by a factor of 1.5 for the larger flag compared to the smallerflag.

10. Jogger A is slower than jogger B (0.6 times the speed of jogger B) but runs for a longer time (1.5 timesthe amount of time that jogger B runs). How does the distance traveled by A compare to the distancetraveled by B? Select the answer with the best reasoning:

a. The distance traveled by A is greater than B because A runs for more time.

b. The distance traveled by B is greater than A because B runs faster.

c. They both run the same distance because although A runs for more time, B runs faster and itbalances out.

d. The distance traveled by A is greater because although B runs faster, A runs long enough thathe passes B and keeps going once B has stopped.

e. The distance traveled by B is greater because although A runs for more time, A doesn’t runlong enough to travel as much distance as B traveled before she stopped.

Version H, Form 1 NO CALCULATOR DO NOT WRITE ON EXAM

© Copyright Marilyn Carlson, 2007 6

14) Given that f is defined by f t( ) = 100t , which of the following is a formula for f –1?

a) ttf1001)(1 =−

b) f −1 t( ) = tln100

c) f −1 t( ) = t100t

d) f −1 t( ) = ln tln100

e) f −1 t( ) = ln t100

15) The following graph represents the height of water as a function of volume as water is

poured into a container. Which container is represented by this graph?

16) Given the function h x( ) = 3x −1 and g x( ) = x2 , evaluate g h 2( )( ) .

a) 10 b) 11 c) 20 d) 25 e) 36

11. The graph at right represents the height of water as a func-tion of volume as water is poured into a container. Whichof the containers shown below could be represented by thisgraph? Choose all that apply.

12. Assume that water is poured into a spherical bottle at a constant rate.Which of the following graphs best represents the height of the water,h, in the spherical bottle as a function of the amount of water in thebottle, V ?

a. b. c. d.

h

V

e.

h

V

Important: Some of the questions on this page may have more than one correct answer. If you see theprompt “Choose all that apply”, select ALL ANSWERS you believe are correct.

16. A student has two electrically neutral spheres, A and B. Initially, sphere A has exactly the same numberof protons and electrons as sphere B. The student touches the spheres to each other. After the spherestouch, the charge on sphere A is measured to be qA = �5 µC, and the charge on sphere B is qB = +5 µC.

Which of the following statements best describe the charges on the spheres after the spheres touch eachother? Select the statement(s) that must be true. Choose all that apply.

a. The net charge on sphere A is greater than the net charge on sphere B.

b. The net charge on sphere A is less than the net charge on sphere B.

c. The net charge on sphere A is neither greater than nor less than the net charge on sphere B.

d. The number of charged particles in sphere A is greater than that in sphere B.

e. The number of charged particles in sphere A is less than that in sphere B.

17. A hand exerts a constant, horizontal force on a block as the block moves along a frictionless, horizontalsurface. No other objects do work on the block. For a particular interval of the motion, the hand doesW = �2.7 J of work on the block. Recall that for a constant force, W = ~F • ~�s.

Consider the following statements about this situation. Select the statement(s) that must be true.Choose all that apply.

a. The work done by the hand is in the negative direction.

b. The force exerted by the hand is in the negative direction.

c. The displacement of the block is in the negative direction.

d. The force exerted by the hand is in the direction opposite to the block’s displacement.

e. The force exerted by the hand is in the direction parallel to the block’s displacement.

f. Energy was added to the block system.

g. Energy was taken away from the block system.

18. Two objects can move in either direction along the x-axis. The two objects collide. As a result of thecollision, object 1’s velocity changes, �vx = �10 m/s.

Consider the following statements about this situation. Select all statement(s) that must be true.Choose all that apply.

a. Object 1 is moving faster after the collision than it was before the collision.

b. Object 1 is moving slower after the collision than it was before the collision.

c. Whether object 1 is moving faster or slower after the collision than it was before the collisioncannot be determined using only the information given.

d. Object 1 moved in the positive direction before the collision, and in the negative direction afterthe collision.

e. Object 1 moved in the negative direction before the collision and in the positive direction afterthe collision.

f. Object 1 moved in the negative direction both before and after the collision.

**

***

**

*

*

*

• Q9: Flag of Bhutan - Choosing Perimeter (A) is more likely than choosing Diagonal (D), which is more likely than choosing the Curve (C)

- Incorrectly choosing Area (B) gradually decreases as scores get higher

• Q16: Electric Charge - Many students correctly indicate that the size of the net charge implies magnitude (C), regardless of score

- A common distractor is that negative net charge is less than positive net charge (B); this is inconsistent with the negative sign indicating type of charge (unique among physical quantities)

• Q17: 1D Work - More likely to answer correctly about the dot product (D: ) than physical implications (G: )

- All distractors common for mid-range scores (7-13)

W = ~F · ~�s<latexit sha1_base64="sLmkoQW1V/chNTbNn/L4zvpLtUM=">AAACB3icdVBNSyNBEO3xM45fUY+CNAbB09AzEuJFDK6IRwVjhEwIPZ2KNvZ80F0jhCE3L/6Ove1lD4p49eTdi/hv7CQK7rL7qILHe1V014syJQ0y9u5MTE5Nz8yW5tz5hcWl5fLK6rlJcy2gIVKV6ouIG1AygQZKVHCRaeBxpKAZXf8Y+s0b0EamyRn2M2jH/DKRPSk4WqlT3mjSPRregCiOBqHopjji4SEo5NQMOuUK8wJWs0WZx2pVf6c6JNWABQH1PTZCZf/F3ct+vrknnfJr2E1FHkOCQnFjWj7LsF1wjVIoGLhhbiDj4ppfQsvShMdg2sXojgHdskqX9lJtO0E6Ur9vFDw2ph9HdjLmeGX+9obiv7xWjr3ddiGTLEdIxPihXq4opnQYCu1KDQJV3xIutLR/peKKay7QRufaEL4upf8n54Hn73js1K/UD8gYJbJONsk28UmN1MkxOSENIsgt+UXuyYNz5/x2Hp2n8eiE87mzRv6A8/wB9mmcZg==</latexit>

W = �E<latexit sha1_base64="Y0xFmSmVQ1GK01+k7JaHHyC9aVU=">AAAB83icdVDLSgMxFM3UVx1fVZdugkVwNWSmlLopFh/gsoJ9QGcomTTThmYeJBmhDP0NNy4Udet3uHcj/o2ZVkFFD/fC4Zx7yc3xE86kQujdKCwsLi2vFFfNtfWNza3S9k5bxqkgtEViHouujyXlLKItxRSn3URQHPqcdvzxae53rqmQLI6u1CShXoiHEQsYwUpLbgfWoXtGucLwvF8qI8tBNV0QWahWtSvVnFQd5DjQttAM5eMXs548vpnNfunVHcQkDWmkCMdS9myUKC/DQjHC6dR0U0kTTMZ4SHuaRjik0stmN0/hgVYGMIiF7kjBmfp9I8OhlJPQ15MhViP528vFv7xeqoIjL2NRkioakflDQcqhimEeABwwQYniE00wEUzfCskIC0yUjsnUIXz9FP5P2o5lVyx0aZcbJ2COItgD++AQ2KAGGuACNEELEJCAG3AH7o3UuDUejKf5aMH43NkFP2A8fwDOrJPj</latexit>