International Conference on Teaching Physics Innovatively

August 17-19, 2015, Budapest, Hungary

Suttida Rakkapao

Department of Physics, Faculty of Science

Prince of Songkla University

THAILAND

An Item Response Theory Analysis

for the Test of Understanding of Vectors (TUV)

OUTLINE

• Test of Understanding of Vectors (TUV)

• Objective

• Item Response Theory (IRT)

• Results and Discussion

• Conclusions

2

3

TUV is a 20-item 5-choice research-based

conceptual test of vectors (Barniol and Zavala, 2014).

4

Ten vector concepts used in the introductory course of a university level

Vector Concept

1. Direction of a vector

2. Magnitude of a vector

3. Component of a vector

4. Unit vector in the Cartesian plane

5. Graphic representation of a vector

6. Vector addition

7. Vector subtraction

8. Scalar multiplication of a vector

9. Dot product

10. Cross product

5

Examples of TUV

6

The TUV is a reliable assessment tool based-on the classical test methods.

7

Item difficulty index (P) and Item discriminatory index (D) for item 1-10 of the TUV

Mexican students (N=423)

Thai students (N=774)

item

0.28 0.11 0.20 0.55 0.61 0.80 0.59 0.24 0.38 0.50

0.59 0.17 0.26 0.53 0.52 0.29 0.51 0.49 0.51 0.61

Desired values [0.3, 0.9]

Desired values [ ≥ 0.3]

One of the most important

shortcomings of Classical Test

Theory (CTT) is the item parameters depend on test-takers.

OBJECTIVE

8

To analyze the test of understanding

of vectors (TUV) using a one-

parameter logistic model (1PL) of

an item response theory (IRT),called Rasch model.

9

Item Response Theory (IRT)

( )

1( ) (1 )

1 a bP c c

e

where thedifficulty parameter

thediscrimination parameter

= the guessing parameter

= theability level

( ) = the probability that an examinee with that

ability willgive a correct answer to theitem

b

a

c

P

c = 0

a = 1

person parameter

Item parameters

10

(http://www.jmetrik.com/index.php)

JMetrik free software for Rasch Model

11

An example of the Item Characteristic Curve (ICC)(created by PARSCALE)

( ) 0.5P

( ) (1 ) / 2P b c

Desired values [ -2, 2]

Difficulty index by CTT

12

Difficulty index by IRT (1PL)

Desired values [ 0.3, 0.9]

Desired values [ -2, 2]

0.62 2.08 1.06 -0.88 -0.60 -2.30 -1.09 0.80 -0.01 -0.59

Item 1-10

Results and Discussion (1/3)

Difficulty index by CTT

13

Difficulty index by IRT (1PL)

Desired values [ 0.3, 0.9]

Desired values [ -2, 2]

0.10 1.64 0.09 -0.91 0.15 0.29 0.80 0.42 0.42 -1.85

Item 11-20

Results and Discussion (2/3)

14

Results and Discussion (3/3)

Item 6 of dot product concept of TUV seem to be an easy question, proved by the IRT (1PL).

CONCLUSIONS

Overall, the TUV contained items with different

levels of difficulty values for monitoring students in

different abilities.

The IRT can be used to develop multiple-choice

tests and to better understand the results, due to the

independent of test-takers.

15

[1] P. Barniol and G. Zavala, Test of understanding of vectors: A

reliable multiple-choice vector concept test, Phys. Rev. ST Phys.

Educ. Res. 10, 010121 (2014).

[2] R. K. Hambleton, H. Swaminathan and H. Rogers, Fundamentals

of Item Response Theory (Sage Publications, Inc., California, 1991).

[3] http://www.jmetrik.com/index.php

16

REFERENCES

ACKNOWLEDGEMENT

Department of Physics

Faculty of SciencePrince of Songkla University

This study was supported by the Research Support from Faculty of Science, Prince of Songkla University (budget year 2015).

17