Chapter 8

Item Analysis

What is Item Analysis?

Item analysis is a statistical technique which is

used for selecting and rejecting the items of the

test on the basis of their difficulty value and

discriminated power

In constructing a new test (or shortening

or lengthening an existing one), the final set of

items is usually identified through a process

known as item analysis.

—Linda Croker

- applicable to test formats that require students to choose the correct or best answer from the given choices

- multiple-choice test is most amenable to item analysis

PURPOSES OF ITEM ANALYSIS

- allow teachers to discover items that are ambiguous, irrelevant, too easy or difficult and non-discriminating

- enhance technical quality of examination by pointing out options that are non-functional and should be improved or eliminated

- facilitate classroom instruction

- identifies the areas of a student’s

weakness, providing information for specific

remediation

1. Check and score the answer sheets.

2. Arrange the papers from highest to lowest.

PROCEDURES IN ITEM ANALYSIS

3. Remove the 27% highest and 27% lowest of the

papers leaving the remaining 46% intact (Sax,

1989). Select the top third and bottom third for

comparison (Bergman, n.d.). Divide the papers

into two groups using median as reference. In case

of a tie between the papers in the median, assign

each paper into lower and higher group by chance

(Downie, 1984).

4. Count the number of students in the upper 27%

who responded to each option. Enter the data in

the third column. Do the same for the lower

27%.

5. Subtract the number of students in the lower group who selected the correct alternatives from the number of students who responded correctly in the upper 27%. Place the value in the fifth column.

6. Divide the difference found in the fifth column by the number of students in the upper 27% or lower 27%. The value obtained is the index of discrimination and place in the sixth column.

7. Count the number of students in the middle 46% who made the correct responses and place the value in the seventh column.

8. Add the number of individuals who responded correctly to the item (upper27%, lower 27% and middle 46%) and enter the data in the eight column.

9. Divide the value in the column 8 by N, the total number of examinees and enter the value in the last column. This is the proportion of students who responded correctly. The quotient is the index of discrimination.

Discrimination Index

distinguishes for each item between the performance of students who did well on the exam and students who did poorly.

COMPUTATION OF THE D VALUE (INDEX OF DISCRIMINATION)

1. Determine the difference between the number of students who got the correct answer from the upper 27% and the number of the students who got the correct answer from the lower 27%.

2. Divide the difference from the 27% of the total number of examinees.

COMPUTATION OF THE D VALUE (INDEX OF DISCRIMINATION)

Where N is the total number of examinees

The computed D value is 0.3, which is interpreted as discriminating item.

ofN

differenceD

%27

COMPUTATION OF THE D VALUE (INDEX OF DISCRIMINATION)

The computed D value is 0.3, which is interpreted as discriminating item.

D Value Range Interpretation

-1.00 - -0.60 Questionable Item

-0.59 - -0.20 Not Discriminating

-0.21 – 0.20 Moderately Discriminating

0.21 – 0.60 Discriminating

0.61 – 1.00 Very Discriminating

TABLE FOR INTERPRETING INDEX OF DISCRIMINATION

(D TABULAR VALUES)

ITEM DIFFICULTY

Item difficulty is the percentage of people who answer an item correctly. It is the relative frequency with which examinees choose the correct response (Thorndike, Cunningham, Thorndike, & Hagen, 1991).

COMPUTATION OF THE P VALUE (INDEX OF DIFFICULTY)

1. Determine the number of students who got the correct answer on the item from the Upper and Lower 27% and from the middle 46%.

2. Divide the sum of the total number of the students who got the correct answer form the total number of students who took the examination.

COMPUTATION OF THE P VALUE (INDEX OF DIFFICULTY)

N

RP

Where R is the total number of the students who got the correct answer.

Where N is the total number of examinees.

COMPUTATION OF THE D VALUE (INDEX OF DIFFICULTY)

The computed P value is 0.6, which is interpreted as an average item.

P Value Range Interpretation

0.00 – 0.20 Very Difficult Item

0.21 – 0.40 Difficult Item

0.41 – 0.60 Moderately Difficult Item

0.61 – 0.80 Easy Item

0.81 and above Very Easy Item

TABLE FOR INTERPRETING INDEX OF DIFFICULTY

(P TABULAR VALUES)

DIIFICULTY LEVEL

DISCRIMINATING LEVEL DECISION

Difficult Not DiscriminatingModerately DiscriminatingDiscriminating

Improbable-DiscardMay need revision

Accept

Moderately Difficult Not DiscriminatingModerately DiscriminatingDiscriminating

Needs RevisionMay need revision

Accept

Easy Not DiscriminatingModerately DiscriminatingDiscriminating

DiscardNeeds revision

DECISION TABLE

Test Scoresusually scored by marking each item separately and

finding the sum of the marks.

Crude or Raw Scores

- obtained when test are corrected

- number of points where student received credits (mere numerical description of student performance)

- common way of scoring – counting one point for each item correctly

INTERPRETING TEST SCORES(COMPUTING TRANSMUTED GRADE)

INTERPRETATING TEST SCORES(COMPUTING TRANSMUTED GRADE)

Another way …

Ranking – considered as first step in test score interpretation. It is an arrangement of the scores in order of magnitude and size. Determining the rank simply involves listing of the scores from the highest score to lowest score.

RANKING- Simple, readily obtained measure that has some

value

- useful in checking a group’s performance in a test

- use of ranking tends to overemphasize individual competition to a greater extent than the practice of assigning letter marks

- feels to indicate the extent or amount of difference in the achievement of the students being compared

Example …Scores Rank

1 78 1

2 74 2

3 73 3.5

4 73 3.5

5 68 5

6 66 6

7 65 8

8 65 8

9 65 8

10 57 10

GRAPHING/TABULATING DATA

- aids in easing the interpretation of test scores.

- give fairly clear picture of how the students performed.

- frequency distribution, histograms, frequency polygon, cumulative frequency or percent curves can make data interpretable

- tabulating may result in an appreciable sacrifice with accuracy

X % F CF % OF RANK

20 100 0 50 99+

19 95 1 50 99

18 90 1 49 97

17 85 1 48 95

16 80 2 47 92

15 75 1 45 89

14 70 2 44 86

13 65 4 42 80

12 60 4 38 72

11 55 5 34 63

X % F CF % OF RANK

10 50 7 29 51

9 45 6 22 38

8 40 5 16 27

7 35 3 11 19

6 30 2 8 14

5 25 2 6 10

4 20 1 4 7

3 15 1 3 5

2 10 1 2 3

1 5 0 1 1

0 0 0 0 -1

PERCENTILE & PERCENTILE RANKS

- another way of interpreting scores

- defined as point which a certain percent of the score fall.

- gives a person’s relative position or the percent of the students scores falling below his/her obtained score

- advantage is easy to compute and interpret

- most powerful means of interpreting scores is by statistical analysis such as measures of central tendency, measures of variability and measures of correlation.

Application.There are 80 high school students

attending a science achievement test, and 61 students pass item 1. Please calculate the difficulty for item 1 and its discrimination.

ITEM OPTIONS UPPER 27%

LOWER 27%

DIFFERENCE D VALUE MIDDLE 46%

R P VALUE

1 A 2 5 1

B 7 3 5

*C 11 9 2 0.9 20 40 .50

D 2 5 7