2.3 tests of receptive skills: workshop cts-academic
DESCRIPTION
The ppt. from the workshop on day 2 (TEAM 2014.TRANSCRIPT
Tests of receptive skills developing tests for Reading and Listening
KAYSERI, 29-31 JANUARY 2014
Aims of the session
! challenges in testing for comprehension ! what sub-skills are to be tested ! selecting texts ! writing items ! task types: pros and cons
What makes testing comprehension challenging?
What makes testing comprehension challenging?
Pros ! relatively easy to create if
texts are available
! input can control learner output
! objective marking is normally possible
Cons ! no observable learner
behaviour
! familiarity and interest may affect success
! if a learner fails, difficult to know why they failed
! are we only testing what we intend to test?
receptive skills can only be tested indirectly
language performance ! inferences to comprehension abilities
set aims
select texts
write tasks
The process of writing a test
Sub-skills for reading
Sub-skills for listening
Selecting texts
! a representative sample of text types learners are likely to encounter
! appropriate extent
! texts with high information content
! clear structure
! neutral but interesting themes
! avoid relying on general knowledge
! avoid culturally loaded texts
! avoid texts learners have already read
Item writing checklist
" text and item grammatically correct
" English natural and appropriate
" item achieves test objective
" the intended sub-skills needed to answer item
" one correct key (or a limited number of responses)
" open answers within student abilities
" open answers with valid and reliable marking
" distractors all work
" item economical and practical
" key complete and correct
Common task types for testing receptive skills
Common task types for testing receptive skills
! multiple choice ! short answers ! gap fill ! information transfer ! multiple matching
! for both reading and listening
! note-taking, dictation or transcription ! for listening only
Common task types for testing receptive skills
! short answers ! multiple choice ! gap fill ! multiple matching ! information transfer
! for both reading and listening
! note-taking, dictation or transcription ! for listening only
What are the pros and cons of each task type?
Short answers
Pros ! each student can
complete an item according to their abilities
! no distractors
! links reception to controlled production
Cons ! need to decide if answer
length is specified ! unpredictable alternative
answers ! difficult to phrase stem
to determine answer ! encourages verbatim
quoting – comprehension?
Multiple choice
Pros ! very familiar format
! quick to complete
! easy to mark
! very objective
! high discrimination index
Cons ! difficult to write good
distractors
! need to produce one correct answer only
! temptation to use for every purpose
! may encourage guessing
Tips for writing good distractors
" Make sure there is only ONE correct answer.
" They should be the same form as the key.
" They should be related logically to the key.
" Key should not stand out – similar length and level.
" Avoid synonyms so they can’t be eliminated at once.
" Keep distractor extent and repetition to a minimum.
" Don’t pair up two distractors (these focus attention).
" Check for unintended clues (e.g. an … ! vowel).
" Avoid illogical or obviously impossible distractors.
" If nobody chooses a distractor, it isn’t a distractor!
Gap fill
Pros ! familiar format
! easy to create
! relatively easy to mark
Cons ! difficult to predict all
alternative answers
! some items can be answered without reading the whole item
Multiple matching
Pros ! clear task, easy to do
! easy to mark
Cons ! errors duplicate as
items aren’t independent of one another
! can be solved by elimination (where there are no distractor items)
! encourages guessing
Information transfer
Pros ! integrated ! real-life
skill
! involving and challenging task
Cons ! two or more skills
tested at once
! difficult to write tasks that limit the number of variables
! difficult to mark
Note-taking, dictation, transcription
Pros Cons
Open comprehension questions
Pros ! requires detailed output
! better differentiation between students?
! integrated: tests comprehension AND production
Cons ! answer extent is
unspecified
Thank you! [email protected]
consonantvoiced.blogspot.com