module 6 test construction &evaluation. lesson’s focus stages in test construction tasks in...
Post on 25-Dec-2015
309 Views
Preview:
TRANSCRIPT
Questions to ask yourself in designing a Test
What objectives will (should) I be testing?
What types of items will be included in the test?
How long will the test be in terms of time and
number of items?
How much will each objective be worth in terms
of weighting and number of items?
Writing a test specification
Stages in Test Construction Order the following stages in the test construction.
- Decide on the marks- Make a key- Decide what kind of test it is- Find appropriate text- Pilot the test- Write a list of what the test covers- Weigh the sections in the order of importance and timing- Write the marking scheme- Analyse the test and decide what can be kept/rejected- Review and revise the test and key- Write the questions- Think about length, layout and format- Write the instructions and examples
Stages in Test ConstructionSuggested order
- Decide what kind of test it is- Write a list of what the test covers- Think about length, layout and format- Find appropriate text- Weigh the sections in the order of importance and timing- Write the questions- Write the instructions and examples- Decide on the marks- Make a key- Pilot the test- Write the marking scheme- Review and revise the test and key- Analyse the test and decide what can be kept/rejected
Which stages can be repeated several times? Why?
Question Types vs Levels of Performance
Knowledge
Comprehension
Application Analysis
Synthesis
Evaluation
Multiple Choice (MC)
True/False (TF)
Matching
Completion
Short Answer
…
MC
Short Answer
Problems
Essay
…
MC
Short Answer
Essay
…
Can you add more to the columns?
Multiple Choice Items
Advantages: Can measure the higher level mental processes
(application, analysis, synthesis and evaluation) Can cover a wide range of content Efficiently and reliably scored Different response alternatives can provide diagnostic
feedback (item analysis)
Disadvantages Difficult to construct plausible alternative responses
Types of Multiple Choice Items
Four Basic Types Question Type Incomplete Statement Type Right Answer Type Best Answer Type
1. The capital city of Canada is1. The capital city of Canada is a. Vancouvera. Vancouver b. Montrealb. Montreal c. Torontoc. Toronto*d. Ottawa*d. Ottawa
Terminology: Multiple Choice
1. Stem: presents the problem
2. Keyed Response: correct or best answer
3. Distracters: appear to be reasonable answers to the
examinee who does not know the content
4. Options: include the distracters and the keyed response.
44
11
22
33
Guidelines for Writing MC
Place most of the subject matter in the Stem ensures full statement of problem
Avoid Negatively phrased Stems students may miss the qualifier
use only when learning outcome requires this type of differentiation
Ensure similarity among alternatives with regard to: grammatical structure length mode of expression
Grammatical errors provide unintentional clues to the answer When in doubt, students will select the longest alternative as
the correct answer
Example: Length of Alternatives
Neurotics are more likely than psychotics to
a. be dangerous to societyb. have delusional symptomsc. be dangerous to themselves*d. have insight into their own inappropriate
behavior but nevertheless feel rather helpless in terms of dealing with
their difficulties
Guidelines for Writing MC Randomly distribute answers across the
alternative positions inexperienced test writers emphasize “b” and “c” alternatives
do NOT use an interpretable order of keyed responses Response alternatives should not overlap,
logically or numerically ordered Avoid “All of the above” and “None of the above” Avoid Negatively phrased Stems
students may miss the qualifier
use only when learning outcome requires this type of
differentiation
What is the cost of an item that normally sells for 9.99 USD that is discounted 25%?
Order Options Numerically
WrongA. 5.00B. 7.50C. 2.50D. 6.66
RightA. 2.50 B. 5.00C. 6.66D.7.50
What are the three important concerns in achievement testing? V=validity R=reliability E=efficiency O=objectivity
Order Options Logically
WrongA. V, R, E B. O, V, RC. R, O, E D. E, O, V
RightA. V, R, O B. V, R, E C. V, E, O D.R, E, O
True-False Test Items
Best suited for testing 3 kinds of info.: Knowledge level learning Understanding of misconceptions When there are two logical responses
Advantages: Sample a large amount of learning per unit of student
testing time Disadvantages:
Tends to be very easy 50-50 chance of guessing Tends to be low in reliability
Tips for constructing True-False Items
o Provide a “T” and “F” beside each statement and ask
students to circle correct answer (students writing
illegible letters).
o Include background, qualifications, and context
"According to...., ...."
o Paraphrase, and do not directly quote
o Every part of a true sentence must be "true"
Tips for constructing True-False Items
- Avoid Unfamiliar vocabulary and concepts
Double negatives*
Specific determiners (always, never, only, etc.) =
overgeneralization*
long or complex sentences
Two ideas in each statement*
Exact quantitative (two, three, four) language is better
than qualitative (some, few, many)
A pattern of answers
True / False
negative words
Poor: It was not unheard of for Henry VIII to close
monasteries in England.T F
Better: Henry VIII closed some monasteries in England.
(T)
“never”, “none”, “always”, and “all”.
Testwise students will recognize that there are few
absolutes.
True / False
Two ideas in one statement Poor: Porpoises are able to communicate because
they are mammals. T F
Better: Porpoises are mammals. T F
Porpoises are able to communicate. T F
Test Evaluation
List of criteria for a good test:- Test purpose- Test format- Test Content- Test Validity- Test Reliability- Test Practicality- Test washback
Test Evaluation Below is the comment note of an experienced
teacher on a reading test for grade 10 students.
Read and tell what criteria have been broken:
C1: Questions 5-10 tend to test students’ grammar
rather than reading comprehension.
C2: Questions 21-23 try to test the specialized
knowledge of geography rather than language
perception and use.
C3: The checkbox for true/false questions is rather
confusing since it is not in parallel with the
statements.
Test Evaluation
C4: The test seems to be a proficiency test
than an achievement test.
C5: Some open questions seek for a very
long answer or many possible answers,
then students find it hard to write them all
and teachers find it hard to score.
C6:The test gives students a feeling that
they can get high marks if they focus on
learning grammar and vocabulary rather
than reading skills.
top related