test construction 1

Are you the kind of Are you the kind of teacher who ask the teacher who ask the following questions?following questions?

Are you the kind of Are you the kind of teacher who ask the teacher who ask the following questions?following questions?

The poem “The Raven” ______

a. was written by Edgar Allan Poeb. was written by Elizabeth

Browningc. was written by Omar Khayyand. was written by Jose Garcia Villa

•Is it NOT true that Magellan discovered the

Philippines?

When did the People Power Revolution take in

the Philippines?A. February 23, 1986B. after the Snap ElectionC. March 1, 1956D. after Valentines’ Day in 1986

•Who was the author of the book quoted in the footnote of Chapter 1 of the present textbook

If you answered “If you answered “YESYES” ” to any of the choices to any of the choices presented then you presented then you

have a have a BIG BIG PROBLEMPROBLEM!!

If you answered “If you answered “YESYES” ” to any of the choices to any of the choices presented then you presented then you

have a have a BIG BIG PROBLEMPROBLEM!!

YOUR STUDENTS ARE NOT LEARNING!!!!

“13% of students who fail in class are caused by faulty test questions”

WORLDWATCHThe Philadelphia Trumpet

August 2005

•It is estimated that 90% of all test questions asked in the US are of “Low level” - knowledge and comprehension (Wilen, W.W., 1992)

• “Low level” doesn’t mean easy:• Write an essay explaining the decline

and fall of the Roman Empire incorporating at least five of the seven causes discussed in class from the writings of Gibbon and Toynbee

• “High level” doesn’t mean hard:• Which movie did you like more, WALL-E

or Cars? Why?

TEST CONSTRUCTION

Presented by:Arnel O. Rivera

http://www.slideshare.net/ArnelSSI

Outline:Part I Principles in Test

Construction Steps in Preparing

Test Questions Preparing Multiple

Choice Questions Preparing True or

False Questions

Part II Review of Part I Preparing Matching

Type Questions Preparing Sentence

Completion Questions Preparing Essay

Questions Other types of Test

Questions Wrap-up/Things to

Remember

“The evaluation of pupils’ progress is a major aspect of the teacher’s job.“

Evaluating Educational Outcomes

(Oriondo & Antonio)

Explain the message of the comic strip.

The Purpose of Testing

To provide a record for assigning grades.

To provide a learning experience for students.

To motivate students to learn.

To serve as a guide for further study.

The Purpose of Testing

To assess how well students are achieving the stated goals of the lesson.

To provide the instructor with an opportunity to reinforce the stated objectives and highlight what is important for students to remember.

Characteristics of Good Tests

Validity – the extent to which the test measures what it intends to measure

Reliability – the consistency with which a test measures what it is supposed to measure

Usability – the test can be administered with ease, clarity and uniformity

Scorability – easy to score

Interpretability – test results can be properly interpreted and is a major basis in making sound educational decisions

Economical – the test can be reused without compromising the validity and reliability

Other Things to Consider

“To be able to prepare a good test, one has to have a mastery of the subject matter, knowledge of the pupils to be tested, skill in verbal expression and the use of the different test format”

Evaluating Educational Outcomes(Oriondo & Antonio)

1. Multiple Choice

2. True or False

3. Matching Type

4. Fill-in the blanks (Sentence Completion)

5. Essay

5 Most Commonly used Test Format

Source: Turn-out of Test Questions in SSI (2003-2007)

General Steps in Test Construction

OUTLINE

DRAFT

ORDER

TEST ANALYZE

SUBMISSION

PRODUCE A T.O.S.

OUTLINE:– the unit learning objectives or

– the unit content or major concepts to be covered by the test

Back to Main Menu

Table of Specifications (TOS)

• A two way chart that relates the learning outcomes to the course content

• It enables the teacher to prepare a test containing a representative sample of student behavior in each of the areas tested.

• Don’t make it overly detailed.

• It's best to identify major ideas and skills rather than specific details.

• Use a cognitive taxonomy that is most appropriate to your discipline, including non-specific skills like communication skills or graphic skills or computational skills if such are important to your evaluation of the answer.

Tips in Preparing the Table of Specifications

(TOS)

• Weigh the appropriateness of the distribution of checks against the students' level, the importance of the test, the amount of time available.

• MATCH the question level appropriate to the level of thinking skills

Tips in Preparing the Table of Specifications

(TOS)

Examples of Student Activities and Verbs for Bloom’s Cognitive Levels

Table 2.1 in Jacobs & Chase (1992:19)

Bloom’s Cognitive Level

Student Activity Words to Use in Item Stem

Knowledge Remembering facts, terms, concepts, definitions, principles

Define, list, state, identify, label, name, who?, when?, where?, what?

Comprehension Explaining/interpreting the meaning of material

Explain, predict, interpret, infer, summarize, convert, translate, account for, give example, paraphrase

Application Using a concept or principle to solve a problem

Apply, solve, show, make use of, modify, demonstrate, compute

Examples of Student Activities and Verbs for Bloom’s Cognitive Levels

Table 2.1 in Jacobs & Chase (1992:19)

Bloom’s Cognitive Level

Student Activity Words to Use in Item Stem

Analysis Breaking material down into its component parts to see interrelationships/ hierarchy of ideas

Differentiate, compare/contrast, distinguish ____from ____, how does ____relate to ___, why does ____work

Synthesis Producing something new or original from component parts

Design, construct, develop, formulate, imagine, create, change, write a poem or short story

Evaluation Making a judgment based on a pre-established set of criteria

Appraise, evaluate, justify, judge, which would be better?

Tips in Preparing the Table of Specifications

(TOS)• The following array shows the most

common questions types used at various cognitive levels.

Factual Knowledge

Application Analysis and Evaluation

Multiple ChoiceTrue/FalseMatching TypeS. CompletionShort Answer/RRT

Multiple ChoiceShort AnswerProblemsEssay

Multiple Choice

Essay

Activity:

• Prepare a short TOS using the selection in your activity sheet.

Back to Main Menu

DRAFT the questions covering the content in the

outline

Back to Main Menu

ORDER the selected questions logically.

Place simpler items at the beginning to ease students into the exam.

Group item types together under common instructions.

If desirable, order the questions logically from a content standpoint (e.g. chronologically or by conceptual groups, etc.)

Back to Main Menu

Test• PUT the questions away for one or

two days before rereading them or have someone else review them for clarity.

• TEST the questions by actually taking the test.

Back to Main Menu

•ANALYZE the items to give you an idea whether the questions were well-written or poorly written as well as if there were problems in understanding instruction.

Back to Main Menu

General Rules in Writing Test Questions

Number test questions continuously.Keep your test question in each test group uniform.Make your layout presentable.Do not put too many test questions in one test group. T or F: 10 – 15 questions Multiple Choice: max. of 30 questions Matching type: 5 questions per test group Others: 5 – 10 questions

Some additional guidelines to consider when writing items are described below:

1. Avoid humorous items. Classroom testing is very important and humorous items may cause students to either not take the exam seriously or become confused or anxious.

2. Items should measure only the construct of interest, not one’s knowledge of the item context.

3. Write items to measure what students know, not what they do not know. (Cohen & Wallack)

Multiple Choice Test

When checking the stems for correctness:

Ensure that the stem asks a clear question.

Reading level is appropriate to the students

The stem is grammatically correct.

Negatively stated stems are discouraged.

What to Look for on Multiple Choice Tests

Example:

What is the effect of releasing a ball in positive gravity?

a) It will fall “down.” correct

b) It will retain its mass. true but unrelated

c) It will rise. false but related

d) Its shape will change. false and unrelated

What to Look for on Multiple Choice Tests

Multiple Choice Questions1. Use negatively stated stems sparingly and

when using negatives such as NOT, underline or bold the print.

2. Use none of the above and all of the above sparingly, and when you do use them, don't always make them the right answer.

3. Only one option should be correct or clearly best.

Multiple Choice Questions:

4. All options should be homogenous and nearly equal in length.

5. The stem (question) should contain only one main idea.

6. Keep all options either singular or plural.

7. Have four or five responses per stem (question).

Multiple Choice Questions:

7. When using incomplete statements place the blank space at the end of the stem versus the beginning.

8. When possible organize the responses.9. Reduce wordiness.10.When writing distracters, think of

incorrect responses that students might make.

Examples1. Sheldon developed a highly controversial

theory of personality based on body type and temperament of the individual. Which of the following is a criticism of Sheldon's work?a. He was influenced too much by the Freudian psychoanalysis.b. His rating of physique and temperament were not independent.c. He failed to use empirical approach.d. His research sample was improperly selected.

Examples

Better: (Eliminate excessive wording and irrelevant information)

1. Which of the following is a criticism of Sheldon's theory of personality?

Examples

1. The receptors for the vestibular sense are locateda. in the fovea.b. in the brain.c. in the middle ear.d. in the inner ear.

Examples

Better: (Include in the stem any word(s) that might otherwise be repeated in each option.)

1. The receptors for the vestibular senses are located in the _______.a. foveab. brainc. middle eard. inner ear

Examples

1. Which is not a major technique for studying brain function?a. Accident and injuryb. Cutting and removingc. Electrical stimulationd. Direct phrenology

Examples

Better: (Use negatively stated stems sparingly. When used, underline and/or capitalize the negative word.)

1. Which is NOT a major technique for studying brain function?

Examples

4. ________________ is the least form of behavior disorder.a. Psychosisb. Panic disorderc. Neurastheniad. Neurosis

Examples

Better: (When using incomplete statements avoid beginning with the blank space.)

1. The least severe form of behavior disorder is __________________.

Examples

1. The number of photoreceptors in the retina of each human is abouta. 115 millionb. 5 millionc. 65 milliond. 35 billion

Examples

Better: (When possible, present alternatives in some logical order.)

1. The number of photo receptors in the retina of each human is abouta. 5 millionb. 35 millionc. 65 milliond. 115 million

Examples6. Latane and Darley's smoke-filled

room experiment suggested that people are less likely to help in groups than alone, because peoplea. in groups talk to one another.b. who are alone are more attentive.c. in groups do not display pluralistic

ignorance.d. in groups allow others to define the situation as a non-emergency

ExamplesBetter: (All alternatives should be

approximately equal in length.)6. Latane and Darley's smoke-filled room

experiment suggested that people are less likely to help in groups than alone, because people in groupsa. talk to one anotherb. are less attentive than people who are

alonec. do not display pluralistic ignoranced. allow other to define non-emergencies

Activity:

• Prepare twotwo multiple choice questionsmultiple choice questions based on the selection in your activity sheet.

True or False

Each statement is clearly true or clearly false.Trivial details should not make a statement false.Statements are written concisely without more elaboration than necessary.Statements are NOT quoted exactly from text.

What to Look for onTrue/False Tests

Give emphasis on the use of quantitative terms than qualitative terms.Avoid using of specific determiners which usually gives a clue to the answer. False = all, always, never, every, none,

only True = generally, sometimes, usually,

maybe, oftenDiscourage the use of negative statements.Whenever a controversial statement is used, the authority should be quoted.Discourage the use of pattern for answers.

Tips in Making True/False Tests

Examples:

____ 1. Repetition always strengthens the tendency for a response to occur.

(Using "always" usually means the answer is false.)

Find the errors, and/or problems with the following true-false tests.

Examples:

_____ 2. The process of extinction is seldom immediate but extends over a number of trials.

(Words like "seldom" usually indicate a true statement.)

Examples:_____ 3. The mean, median, and mode are

measures of central tendency, whereas the standard deviation and range are measures of variability.

(Express a single idea in each statement.) e.g.“The mean and standard deviation are measures of central tendency.”

Activity:

• Prepare two true or false questions based on the selection in your activity sheet.