Multiple Choice vs. Performance Based Tests in High School Physics Classes

Katie Wojtas

Abstract

Multiple-choice tests dominate school systems in the United States, but are such

assessments effective? Research suggests that multiple-choice tests in physics classes

may not adequately assess student understanding and application of concepts. This study

will investigate the efficacy of multiple-choice testing versus performance-based

assessments in physics class. Four high school physics classes will be administered two

forms of tests. One test will be a multiple-choice test that is designed to reflect those

predominately used in the school system. The other test will be a performance-based,

hands-on lab type test in which students must demonstrate understanding of the major

concepts covered in class as well as application of such concepts. An analytic

comparison of the results for the two test types administered will be made to determine

any difference in scores between the multiple-choice test and the performance-based lab

test.

Introduction:

The question that this experiment aims to answer is “Do multiple choice tests

really assess how well the students, in a high school physics class, know, understand, and

are able to apply the information, or is a performance-based test a better test to assess

this?”

The Merriam-Webster defines a multiple-choice test as a test that has many

answers from which one of is chosen. (http://www.merriam-

webster.com/dictionary/multiple%20choice) Each multiple-choice question is made up

of two parts. The first part is the stem. This is the actual question or statement that the

student needs to answer. The second part contains the options. This is split up into two

parts, the actual answer and distracters. Distracters are incorrect, but hopefully tempting

answers.

(http://cte.uwaterloo.ca/teaching_resources/tips/designing_multiple_choice_questions.ht

ml)

A performance-based test is a test that “requires students to demonstrate

knowledge and skills, including the process by which they solve problems”.

(http://www.projectappleseed.org/assesment.html) There are many types of

performance-based tests. They range from essays to portfolios to labs to solving word

problems. The two main types of performance-based tests used in physics are word

problem type tests and labs.

There are advantages and disadvantages to both types of test. Neither of them are

perfect, but there are some key advantages and disadvantages to discuss.

Advantages of a multiple-choice test:

In a multiple-choice test, a teacher is able to test a wide range of topics in one test.

If a multiple-choice test is well written there is only one true answer for each question.

Since there is only one answer for each question, there is little to no bias in grading these

tests. Multiple-choice tests can easily be administered to a large group of people.

(http://www.park.edu/cetl/quicktips/multiple.html)

Disadvantages of a multiple-choice test:

Multiple-choice tests are difficult and time consuming to write. These tests do not

allow students to express their thoughts on the topics that show up on the tests. Multiple-

choice tests “may overestimate learning due to the ability to utilize an elimination process

for answer selection”. (http://www.park.edu/cetl/quicktips/multiple.html)

Advantages of a performance based test:

Performance-based tests allow teachers to assess their students from multiple

perspectives. These tests allow students to express their thoughts on the topics on the test

and in some cases they are able to express their creativity. Performance based tests can

help teachers understand the student’s learning of the subject matter and also gives

teachers an insight into the student’s thought process.

(http://www.uta.edu/irp/unit_effectiveness_plans/assets/AssessMethods.pdf)

Disadvantages of performance based tests:

Where multiple-choice tests can be given to students quickly, performance-based

tests are time consuming to administer to students. These tests are also time consuming

not just to make, like multiple-choice tests, but to grade as well. If a performance-based

test is not carefully designed, the test may not accurately assess what the student actually

knows and is able to apply.

(http://www.uta.edu/irp/unit_effectiveness_plans/assets/AssessMethods.pdf)

This experiment aims to test how well multiple choice tests really assess the

knowledge of the students and how well they are able to apply this knowledge, or if a

performance based test is a better indicator of how well a student really knows and can

apply this knowledge.

Method:

To test the effectiveness of how well a multiple choice test assesses how well a

student knows, understands, and can apply the information being tested, it can be

compared to a student’s performance on a performance-based test that covers the same

material. There are many different types of performance-based tests; two that are used

the most in physics are a lab type test and a word problem test. In the lab type test, a

student is given a physics lab to do and each student has to perform this lab and then

explain why they got the results they did. In a word problem test, the students are given a

set of different physics problems and they have to go through and pick out the

information that is important and be able to know which equation they need to use to

solve the problem. These students will have to use their knowledge of what they learned

in class and apply it to be able to fully answer the questions on the performance-based

test.

The first step was to find a testing environment. After talking to many high

school teachers, one was selected. After speaking with this teacher (teacher one), it was

decided that the performance-based test was going to be a lab test. She said that it would

be easier on her and easier to grade.

After finding this testing environment it was time to figure out what was going to

be on the test, what material it was going to cover. After talking at length, she decided

that she was going to give her students both tests as their first class test and that test was

going to be on Kinematics in 1-Dimension. After this material was chosen it was time to

write the tests.

Writing the multiple-choice test was a little difficult. There are many steps to

writing a multiple-choice test. The first thing that was done was to read up on the

material. After reading the material thoroughly, key concepts from the chapters were

selected. These key concepts were: objects in free fall, acceleration due to gravity,

constant velocity, constant acceleration - not due to gravity, average velocity, and

average acceleration. After reading the material and talking to teacher one, it was time to

write the multiple-choice test. Teacher one provided a few test banks that she used for

her tests. Questions were pulled from these test banks and then revised a little bit to

make them fit the needs of the test. After the test was written, it was given to a few

professors at Christopher Newport University for review and suggestions. After these

professors reviewed the tests and gave feedback, the test was revised and sent to teacher

one for approval. After the multiple-choice test was approved, teacher one found a

performance-based lab test that she felt tested the same material as the multiple-choice

test. The material on the performance based test was also Kinematics in 1-Dimension,

but due to time constraints in the class, it unfortunately only covered constant velocity.

After this, she administered the tests to her students.

To try and eliminate the students doing better on the second test because they had

been exposed to the first tests, the class was split in half. One half took the multiple-

choice test first and the performance based test second, while the other half took the

performance-based test first and the multiple-choice test second.

Due to time constraints, teacher one was only able to test one of her four classes,

and not all four classes that she originally thought she would be able to test. Once

notified of this information, it was suggested that the test be given out to more students.

After talking to a few more high school teachers, one agreed to give the tests to

three of his physics classes. Unfortunately, his students had already been tested on this

material, so they had no incentive to do well.

After the results were in, it was time to analyze them.

Expected Results:

For those students that knew the material they were being tested on, it was

expected that they would perform well on both the multiple-choice and performance-

based tests. This was thought because those students that knew the material would be

able to analyze why they got the results they did on the performance based test. Since

they are able to analyze the results, it was thought that they would be able to do well on

the multiple-choice test.

For those students that kind of knew the material, it was expected that they would

do better on the multiple-choice test than on the performance-based test. This was

thought because the students would be able to do educated guesses on the multiple choice

test and be able to get the correct answer; and when it came time for them to tell why

something happened in the performance-based lab that they would not be able to explain.

And for those students that did not really know the material, it was expected that

they would not do well on either test. Since these students do not really know the

material, they would not really be able to make an educated guess on the multiple-choice

questions, so they would rely on guessing alone. And it was thought that since they did

not know the material, they would not be able to explain what happened in the

performance-based test.

Actual Results:

Upon looking at the results for both classes, the data showed that the students did

better on the performance based test than the multiple-choice test. The mean for the

multiple-choice test for teacher one was 68.53. The mean for teacher one’s performance-

based test was 95.29. The mean for the multiple-choice test for teacher two was 29.13.

The mean for teacher two’s performance based test was 79.52.

Teacher one’s test results.

Teacher two’s test results.

Clearly teacher one’s students did better on both tests than teacher two’s students,

but why? It could be because there was an incentive provided to teacher one’s students

and no incentive was provided to teacher two’s students. Teacher one gave the tests out

as their actual first test, so the grade they got on the tests was the grade they got as their

first test grade. Teacher two did not provide an incentive to his students; they had

already taken a test on this material. Another possible reason teacher one’s students

performed better than teacher two’s students could be the level of math that the students

have. Teacher one’s students were all in calculus, whereas teacher two’s students were

only in algebra II.

Overall the students did better on the performance-based test than the multiple-

choice test. What could be the reason for this? It could be that the performance-based

test was easier than the multiple-choice test. Due to time constraints there was a lot less

material on the performance-based test than on the multiple-choice test. So since there

was less material, the students could have known the material on the performance-based

test better than all the material on the multiple-choice test.

Motivation could have played a roll in the grades on the tests. Teacher one’s

students were given a motivation to do well on the tests, whereas teacher two’s students

were not. The tests did not have any effect on the grades of teacher two’s students.

Although there was no external motivation for the students in teacher two’s class, these

students performed much better on the performance-based test. So it can be said that the

students performed better on the performance-based test whether or not they were given

an external motivation.

This graph shows the means of the two tests and compares them.

0

10

20

30

40

50

60

70

80

90

100

MC PB

Gra

de

on

Te

st

Test Type

Multiple Choice vs. Performance Based

Teacher 1

Teacher 2

Possible Sources of Error:

The results were inconclusive do to some possible errors. These errors are: the

tests did not fully cover the same material, the multiple-choice test covered more material

than the performance-based test, there could have been some bias in grading, there could

have been a few incidents of incorrect grading of the multiple-choice tests, and there was

possibly some deliberate false data.

Due to time constraints, the performance-based test could not cover as much

material as the multiple-choice test. This is an advantage of a multiple-choice test, they

can cover a large amount of information in a short test, and a disadvantage of a

performance-based test is it is hard to cover a full range of material. The bias in grading

could be significant, and a reason the two teachers data had to be analyzed separately.

Teacher one was not able to provide teacher two with the rubric she used to grade the

performance-based test. The deliberate false data plays a roll in the grades of teacher

two’s multiple-choice test. There was a student that got a 0 on the multiple-choice test.

Statistically, if a student just guessed on every question, they would have gotten about a

25% on the test.

Further research:

If the experiment were to be repeated, more control of the variables would be

maintained. There was a lot of input from teacher one with what she thought was the best

choice for the performance-based test. The number of test subjects will be increased.

The test subjects will provide a little more background information, such as age, sex, and

current grade in their physics class. The performance-based lab will be replaced with a

word problem test. The reason for doing this is suppose something went wrong during a

lab test, the smallest error due to equipment or student error could throw off the answer,

so the student would end up getting that part wrong even if they knew and could apply

the material. There will be a third party grading of the tests. In the experiment above,

the teachers graded their own students’ tests. Teacher two most likely did not grade the

performance-based tests the same way teacher one did. Teacher one did not provide

teacher two with a rubric as to how she graded her students’ tests. Finally, each student

would be given an incentive for taking the tests.

Conclusion

Due to the sources of error in this experiment, the results were inconclusive.

There was a correlation between how well a student did on the multiple-choice test and

how well they did on the performance based test. For example, if a student did well on

the multiple-choice test, the student also tended to do well on the performance-based test.

The students did do better on the performance-based test than on the multiple choice test,

but it can not be said that giving students performance-based tests is a better way of

assessing the students, there was too much error in the experiment.

It can be said that the performance-based test yielded higher results than the

multiple-choice test, regardless of whether or not there was an incentive.

Now the question becomes, do we want the students to learn the material, or just

get good grades on tests?

References:

Multiple-choice definition- http://www.merriam-

webster.com/dictionary/multiple%20choice

“Defining Multiple Choice Questions”

http://cte.uwaterloo.ca/teaching_resources/tips/designing_multiple_choice_questions.

html

“What Should Parents Know About Performance Based Assessment?”

http://www.projectappleseed.org/assesment.html

“Effective Multiple-Choice Items” http://www.park.edu/cetl/quicktips/multiple.html

“Advantages and Disadvantages of Various Assessment Methods”

http://www.uta.edu/irp/unit_effectiveness_plans/assets/AssessMethods.pdf

Multiple Choice Test

1) Cars A and B are traveling in the same direction. If car A passes car B, car A must

be:

a) accelerating.

b) accelerating at a greater rate than car B.

c) moving faster than car B, but not necessarily accelerating.

d) moving faster than car B, and accelerating more than car B.

2) Which of the following statements is NOT true of a free-falling object? An object in

a state of free call:

a) falls with a constant acceleration of -9.8 m/s2.

b) falls with downward acceleration which has a constant magnitude.

c) falls under the sole influence of gravity.

d) falls with a constant velocity of -9.8 m/s.

3) What is the acceleration of a car that maintains a constant velocity of 55 mi/hr for

10.0 s?

a) 5.5 mi/hr/s

b) 5.5 m/s2

c) 550 mi/hr/s

d) 0 m/s2

4) The average velocity of an object which moves 10 kilometers (km) in 30 min is:

a) 10 km/hr

b) 20 km/hr

c) 30 km/hr

d) more than 30 km/hr

5) A baseball pitcher delivers a fastball. During the throw, the velocity of the ball

increases from 0 to 30.0 m/s over a time of 0.100 seconds. The average acceleration

of the baseball is:

a) 3.00 m/s2

b) 30.0 m/s2

c) 300 m/s2

d) 3000 m/s2

e) none of these

6) Which of the following is not consistent with a car that is accelerating?

a) A car is moving with a high velocity.

b) A car is moving with an increasing velocity.

c) A car is changing directions.

d) A car is moving with a decreasing velocity.

7) Consider drops of water that leak from a dripping faucet at a constant rate. As they

fall they:

a) get closer together

b) get farther apart

c) remain at a relatively fixed distance from one another

8) As an object freely falls its:

a) velocity increases

b) acceleration increases

c) both of these

d) neither of these

9) On takeoff, a rocket accelerates from rest at a rate of 50.0 m/s2 for exactly 1 minute.

The rocket’s velocity after this minute of steady acceleration will be (acceleration due

to gravity is included in this acceleration):

a) 50.0 m/s

b) 500 m/s

c) 3.00 x 103 m/s

d) 3.60 x 103 m/s

e) none

10) How far will a brick starting from rest fall freely in 3.0 seconds?

a) 15 m

b) 29 m

c) 44 m

d) 88 m

11) A ball thrown vertically upward reaches a maximum height of 30 m above the surface

of Earth. At its maximum height, the velocity of the ball is:

a) 0.0 m/s

b) 3.1 m/s

c) 9.8 m/s

d) 24 m/s

12) A basketball player jumped straight up to grab a rebound. If she was in the air for

0.8s, how high did she jump?

a) 0.50 m

b) 0.78 m

c) 1.2 m

d) 3.1 m

13) An astronaut drops a hammer from 2.0 m above the surface of the Moon. If the

acceleration due to gravity on the Moon is 1.62 m/s2, how long will it take for the

hammer to fall to the Moon’s surface?

a) 0.62 s

b) 1.2 s

c) 1.6 s

d) 2.5 s

14) A rocket initially at rest on the ground lifts off vertically with a constant acceleration

of 20 m/s2. How long will it take the rocket to reach an altitude of 9000 m?

a) 30 s

b) 45 s

c) 450 s

d) 900 s

15) An astronaut standing on a platform on Mars drops a hammer. If the hammer falls

6.0 m vertically in 2.7 s, what is its acceleration?

a) 1.6 m/s2

b) 2.2 m/s2

c) 4.4 m/s2

d) 9.8 m/s2

16) A race car starting from rest accelerates uniformly at a rate of 4.9 m/s2. What is the

car’s velocity after it has traveled 200 m?

a) 1960 m/s

b) 62.6 m/s

c) 44.3 m/s

d) 31.3 m/s

17) A ball is thrown downward with an initial velocity of 0.5 m/s from a height of 4.0 m.

What is the velocity of the ball 0.70 s after it is released? (Neglect air resistance)

a) 0.50 m/s

b) 7.4 m/s

c) 9.8 m/s

d) 15 m/s

Performance Based Test

Velocity of a Motorized Cart Mechanics: linear motion, constant velocity, graphing, slope GLX setup file: velocity cart

Qty Equipment and Materials Part Number

1 PASPORT Xplorer GLX PS-2002

1 PASPORT Motion Sensor PS-2103

1 1.2 m PASCO Track

1 Constant Speed Buggy SE-9028

1 Hooked Mass Set SE-8759

0.5 m Braided Physics String SE-8050

1 Block with eye hook

Purpose The purpose of this activity is to investigate constant velocity.

Background

Constant velocity means constant speed along a straight line. Although constant velocity

is straightforward, the graphical representation of constant velocity involves many

fundamental concepts of kinematics. The slope of a plot of position versus time is the

speed of the object. The y-intercept of a plot of position and time gives the initial position

of the object when measurement begins. The units along the vertical and horizontal axes

of a graph of position and time give the units for the object’s speed. Whether the slope of

position and time is positive or negative reveals the direction of the object relative to the

sensor.

Preview

Use a Motion Sensor to measure the motion of a motorized cart as it moves at different

speeds. Use the Xplorer GLX to record and display the motion.

Safety Precaution

Follow all directions for using the equipment.

Procedure

GLX Setup

1. Connect the Motion Sensor to one of the sensor ports on

the top end of the GLX. Put the range selection switch on

the top of the Motion Sensor to the ‘near’ (cart) setting.

2. Turn on the GLX ( ).

The Graph screen opens with a graph of Position (m)

versus Time (s).

Fig. 1: Motion Sensor setting

Equipment Setup

1. Place the PASCO track on a table and attach

the Motion Sensor to one end of the track.

2. Place the cart about 15 cm from the Motion

Sensor so that the back of the cart is facing

the sensor.

3. Aim the sensor so its signal will reflect from

the cart.

Record Data: Part 1 (Slow)

1. Press Start ( ) on the GLX to begin

measuring the sensor signal.

2. Turn on the switch on the side of the cart so the cart moves toward the other end of

the track.

3. Press to end data recording just before the cart reaches the end of the track.

Turn off the cart.

The Graph screen will display the plot of position and time for Run #1.

Record Data: Part 2 (Slower)

4. Fasten a block to the cart so it can be dragged behind the cart. Put a 500 g (0.5 kg)

mass on top of the block.

5. Place the cart and the block on the track so the mass on top of the block is about 15

cm from the Motion Sensor. Aim the sensor so its signal will reflect from the mass.

6. Start ( ) on the GLX to begin measuring the sensor signal.

7. Turn on the switch on the side of the cart so the cart moves toward the other end of

the track.

8. Press to end data recording just before the cart reaches the end of the track.

Turn off the cart.

The Graph screen will display the plot of position and time for Run #2.

Record Data: Part 3 (Slowest)

9. Replace the 500 g mass on the block with a 1000 g (1 kg) mass.

10. Repeat the data recording process.

The Graph screen will display the plot of position and time for Run #3.

Fig. 2: Equipment setup

Analysis

Find the slope of each run of data for the cart.

1. Select the first run of data. In the Graph screen, press

to activate the vertical axis label. Use the up

arrow to move to the ‘Run’ number in the upper right

corner of the screen. Press Activate ( ) to open the

‘Run’ menu. Select ‘Run #1’ and press to activate

your choice.

2. Use the arrow keys to move the cursor to the point in

the graph where the cart begins to move. Press F3 (

) to open the ‘Tools’ menu. Select ‘Linear Fit’ and

press to activate your choice.

The Slope of the linear fit is the speed of the cart.

3. Record the value of the slope in the Lab Report.

4. Repeat the data analysis procedure for the other two

runs of data.

Record your results and answer the questions in the Lab Report.

Fig. 3: Select ‘Run #1’

Fig. 4: Record the slope

Lab Report - Activity 3: Velocity of a Motorized Cart

Name ________________________________ Date ___________

Data

1. Make a sketch of one run of position versus time data including labels and units for

the y- and x-axes.

2. Record your values for the Slope for each run.

Run Slope

1 m/s

2 m/s

3 m/s

Questions

1. What is shown on the vertical axis of your graph and what are the units?

2. What is shown on the horizontal axis of your graph and what are the units?

4. How long was the motorized cart moving during each data run? (Hint: Use the

‘Smart Tool’ in the Graph screen.

Run Time of Motion (s)

1

2

3

5. What physical quantity does the slope of each plot represent?

6. What are the units for the slope of each plot?

Problem

1. If the cart moves at a constant speed of 0.33 m/s, how far will it move in 5

seconds?