TESTING

TESTING WRITING PROBLEMS

1.Representative of the population tasks that we should expect the students to be able to perform.

2. Task should elicit valid samples of writing( truly represents the student’s ability

3. The samples of writing can and will be scored validly and reliably.

Representative Task(i)Specify all possible contentOperation

Expressing : thanks, opinions, comments, apology, information, etc.Directing : Ordering, instructing, persuading, advising, warning.

Describing : actions, objects, people. Eliciting : information, direction, etc Narration : Sequence of events

Reporting : Description, comments, decision

Types of Text : Form, letter, message, note, fax, postcard,etc

Addressees of text : ( ex, University lectures, - both native and non native)

Topics : connected with common theme

Dialect and style : ( ex, Any standard variety of English, American, British, or mixture of these)

Length of text : ( ex, 1 page)

OperationsDescribe, explain, compare, and contrast, argue for and against a position

Types of text Examination answers up to two paragraphs in length.Addressees of textNative speaker and non-native speaker university lectures.

TopicsAny capable of academic treatment. Not specialist. Relevant to the test takers.Dialect and StyleAny standard variety of English ( eg, American, British) or a mixture of these.Addressees of textNative speaker and non-native speaker university lectures.Formal styleLength of Text About 1 page

(ii) Include a representative sample of the specified content• The more tasks that we set , the more representative of a candidate’s ability( the more valid) will be the totality of the samples ( of the candidate’s ability) we obtain•If test includes a wide ranging and representative sample of specifications, the test is more likely to have a beneficial backwash effect.

Note : the example from Testing for LT page 86-88--CCSE examiners

Elicit a Valid Sample of Writing Ability Set as many separate tasks as is feasibleWe have to offer candidates as many ‘ fresh starts’ as possible, and each task can represent a fresh start, so we will achieve greater reliability and validity.

Elicit a Valid Sample of Writing Ability Test Only Writing ability, and nothing else

In LT, we are not interested In knowing whether the students are creative, imaginative, or even intelligent, have wide general knowledge, or have good reasons for the opinions they happen to hold.

Restrict CandidatesWriting tasks should be well

defined: candidates should know just what is required of them, and they should not be allowed to go too far astray.

A useful device is to provide information in the form of notes.( or pictures)

Tasks shouldn’t only fit well with the specification, but they should be made as authentic as possible.

Ensure Valid and Reliable ScoringSet tasks which can be reliably scores( a

number of suggestion made to obtain a representative performance will facilitate reliable scoring.

Set as many tasks as possible( the more scorer more reliable the total score)

Restrict candidatesGive no choice of tasks( make the

candidates perform all tasksEnsure long enough samplesCreate appropriate scales for scoring

Holistic scoring ( impressionistic scoring)involves the assignment of a single score to a piece of writing)

Advantage : Very RapidMore than one ( 4) Experienced scorers

TOEFL FOR COMPOSITION can judge and it can resulted higher scorer reliability( Harris 1968)

Appropriate to the level of candidates and the purpose of the test. ( ex, adequate for study in English in that University)

Analytic scoring( methods of scoring which require a separate score for each of a number of aspects of a task are said to be analytic. grammar, vocabulary, mechanics, fluency, form(page 101-102)

Advantages:Disposes of the problem of uneven development of subskills in individuals.scores are compelled to consider aspects of performance which they might otherwise ignorethe scoring more reliable.

http://www.edteck.com/rigor/guides/rubrics.pdf

Calibrate the scale to be usedCollecting samples of performance collected under test conditions and covering full range of the scales.

Select and train scorers.should be native or near-native speakers of the language being tested (sensitive to language, have had experience of teaching writing and making written work. Or they have had training in testing

Follow acceptable scoring procedures.assumed that the scorers have been trained.

Follow acceptable scoring procedures.assumed that the scorers have been trained.1.each task of student should be scored independently by two or more scorrers2. scoring should take place in a quiet, well-lit environment. Scorers should not be allowed to become too tired

FEEDBACK

It is done during calibration.Example of feed back on linguistic features:The following elements should be included on the feedback pro forma:Non writing –specificIncomplete performance of the task in

terms of: 1. topic : not all parts addressed

very superficial treatment 2. Operations CALLED FOR

(e.g. compare and contrast)

Writing specificMisuse of quotation marksInappropriate underliningCapitalizationStyle conventionsFailure to spit overlong sentenceshandwriting