approaches & techniques of language testing

7/22/2019 Approaches & Techniques of Language Testing

1/56

PPRO CHES NDTECHNIQUES OFL NGU GE TESTING

Group 1:

NguynThPhngThoPhmThi BoNgc

NguynThanh Bnh

NguynQunhThy

NguynL B Tng

Lecturer:

Assoc. Prof. Dr. Tung Thanh Nguyen


2/56

APPROACHES TO

LANGUAGE TESTING


3/56


4/56

1. The Essay-Translation approach

Pre-scientific stage of language testing.

No special skill or expertise in testing isrequired.


5/56

Teachersjudgement is considered important

Types of test: essay writing, translation, and

grammatical analysis

Heavy literary and cultural bias


6/56

2. The Structuralist approach

Language learning is concerned with the

systematic acquisitionof a set of habits.

Focus on structural linguistics


7/56

- Contrastive Analysis

- The need to identifyand measure the

learners mastery of separate elements of

the target language.

04 language skillsare tested separately.


8/56

Psychometricapproach: an integral part of

structuralist testing.

reliabilityand objectivity

Type of test: multiple choice items

Statisticalmeasure


9/56

3. The Integrative approach

Involve the testingof language in context

meaningand communicative effectiveness.

Assess learners ability to use twoor more

skills simultaneously.

Concern with a global view of proficiency.


10/56

Types of test:

- Cloze test

- Dictation

- Others: Oral interviews, translation,

and essay writing.


11/56

Cloze test

Base on Gesalt theory of closure

Measure the readers ability to decodeinterrupted or mutilated messages


12/56

40-50 blanks

02 methods of scoring:

- Acceptableanswer

- Exactanswer

Measure familiarity with grammar


13/56

Reliable

More closelyto the real-lifetasks

Advantageous to provide lead-in

A measure of readingdifficulty and

reading comprehension.


14/56

Requirements for successful performance:

- Linguisticknowledge

- Textualknowledge

- Knowledge of the world

Cloze test are used in general achievement test,

proficiency test, classroom placement test, and

diagnostic test.


15/56

Dictation

Measurestudents skills of listening

comprehension

Integrated skills in tests: auditory

discrimination, auditory memory span,

spelling, and so on.

Good predicators of global languageability.


16/56

Disadvantage:

No reliable way of assessing

Errorsarepenalised in the sameway


17/56

Procedure of dictation:

Readthrough the whole dictation passage

Dictatein meaningful units to challenge

students short term memory span.

Readthe whole passage once at slightlyslower than normal speed.


18/56

Translation - Oral Interviews - Writing

Complex nature of skillsand methods of

scoring.

Unreliable


19/56

4. The Communicative approach

Concern with how languageis used in

communication.


20/56

Measuredifferent language skillsin

communicativetests based on divisibility

hypothesis.

The scoresof the test will resultin several

measures of proficiency.


21/56

6 5 4 3 2 1

Listening

Reading

Listening and speaking

Writing


22/56

The requirements for the test:

Reflect the cultureof aparticular country.

Relate to real-lifesituations.

Base on precise and detailed specifications

of the needs of the learners

Focus on qualitativemodes of assessment


23/56


24/56

TECHINIQUES OF

LANGUAGE TESTING


25/56

Selection item types

Candidate - suppliedresponse item types

ITEM TYPES

NON-ITEM-BASED TASK TYPES

Mostly used in tests of reading, listening & structural competence

Mostly used in tests of reading, listening, structural competence & writing

Mostly used in tests of speaking &writing


26/56

SELECTION ITEM TYPES

Discrete point & text based multiple choice items

True/false item

Gap-filling (cloze passage) with multiple choice options

Gap-filling with selection from bank

Gap-filling at paragraph level

Matching

Multiple matching

Extra word error detection


27/56

Discrete point & text basedmultiple choice items

The singer ended the concert........................her most popular song.

A. by B. as C. in D. with

Discrete point multiple choice item

STEM

OPTIONS

DistractorsKey /

Answer

Text-based multiple choice item

Then he saw a violin in a shop. It was of such

high quality that even top professionalplayers are rarely able to afford one like it.Id never felt money was important untilthen, Colin explained. Even with themoney Id won, I wasnt sure I could affordto buy the violin, so I started to leave the

shop.

When Colin first found the violin, what did he

think?

A. He might not have enough money to buy it.

B. He should not spend all of his money on it.

C. He was not a good enough player to own it.

D. He could not leave the shop without it.


28/56

RULES FOR WRITING DISCRETE / TEXT BASEDMULTIPLE CHOICE ITEMS

1. Test only a single idea in each item

2. Base each item on a specific problem stated clearly in the stem

3. Keep the alternatives mutually exclusive

4. Use only one correct (or best) option

5. Keep option lengths similar

6. Avoid clues to the correct answer-Be grammatically correct

-Use plausible distractors

7. Avoid unnecessary length & irrelevant difficulty

-Include as much of the item as possible in the stem

-Keep the stem and options brief, concise and clear

-Put options in a logical order

8. Avoid Negative Questions

9. Avoid All of the above / None of the above

10. Present the answer in each item approximately an equal number of times


29/56

2 important points tested in a

single itemOriginal Revised

1. Peter arrived ______ the

airport ______ noon.A. at inB. in inC. in atD. at at

1. Peter arrived ___ the airport at noon.A. at

B. inC. onD. to

2. We often have lunch _____ noon.A. forB. inC. atD. on


30/56

The stem failure to present theproblem adequately

Original Revised

1. World War II was:

A. The result of the failure of the League

of Nations.

B. Horrible.

C. Fought in Europe, Asia, and Africa.

D. Fought during the period of 1939-

1945.

1. In which of these time period was

World War II fought?A. 1914-1917

B. 1929-1934

C. 1939-1945

D. 1951-1955E. 1961-1969


31/56

Overlapping options

Original Revised1. During what age period is thumb-

sucking likely to produce the greatestpsychological trauma?

A. InfancyB. Preschool periodC. Before adolescenceD. During adolescenceE. After adolescence

1. During what age period is thumb-sucking likely to produce thegreatest psychological trauma?

A. From birth to 2 years old

B. From 2 years to 5 years old

C. From 5 years to 12 years old

D. From 12 years to 20 years old

E. 20 years of age or older


32/56

Grammatical clues to the correctanswer

Original Revised

1. Albert Eisenstein was a _____.

A. anthropologistB. astronomer

C. chemist

D. mathematician

1. Albert Eisenstein was a(n) _____.

A. anthropologistB. astronomer

C. chemist

D. mathematician


33/56

Implausible distractors

Original Revised

Which of the following artists is known for painting the ceiling of the

Sistine Chapel?

a. Warhol.b. George Washington.

c. Michelangelo.

d. Santa Claus.

a. Botticelli.b. Leonardo da Vinci.

c. Michelangelo.

d. Raphael.


34/56

Unnecessary length

Original Revised

1. If the pressure of a certain

amount of gas is held constant,

what will happen if its volume is

increased?A. The temperature of the gas will

decrease.

B. The temperature of the gas will

increase.

C. The temperature of the gas will

remain the same.

1. If you increase the volume of

a certain amount of gas

while holding its pressure

constant, its temperaturewill:

A. decrease.

B. increase.

C. remain the same.


35/56

Wordy options

Original Revised

1. The term hypothesis, as used in research, asdefined as:

A. A conception or proposition formed by speculationor deduction or by abstraction and generalizationfrom facts, explaining or relating an observed setof facts, given probability by experimental

evidence or by factual or conceptual analysis butnot conclusively established or accepted.

B. A statement of an order or relation of phenomenathat so far as is known is invariable under the givenconditions, formulated on the basis of conclusiveevidence or tests and universally accepted, that

has been tested and proven to conform to facts.C. A proposition tentatively assumed in order to draw

out its logical or empirical consequences and sotest its accord with facts that are known or may bedetermined, of such a nature as to be either provedor disproved by comparison with observed facts.

1. The term hypothesis, as usedin research, is defined as:

A. An assertion explaining anobserved set of facts that hasnot been conclusively

established.B. A universally accepted

assertion explaining anobserved set of facts.

C. A tentative assertion that iseither proved or disproved bycomparison with an observedset of facts.


36/56

Illogical order

Original Revised

According to the 1991 census, approximately what percent ofthe United States population is of Spanish or Hispanic descent?

a. 25%b. 39%c. 2%d. 9%

a. 2%b. 9%c. 25%d. 39%


37/56

Logical order

Logical order Example

Numerical

a. 1939b. 1940c. 1941d. 1942

Alphabetical

a. Changing a from .01 to .05.b. Decreasing the degrees of freedom.c. Increasing the spread of the exam scores.d. Reducing the size of the treatment effect.

Sequential

a. Heating ice from -100C to 0C.b. Melting ice at 0C.c. Heating water from 0C to 100C.d. Evaporating water at 100C.e. Heating steam from 100C to 200C.


38/56

Negative questions & answers

Original Revised1. All of the following are

correct procedures forputting out a fire in a pan

on the stove except:A. Do not move the pan.

B. Pour water into the pan.

C. Slide a fitted lid onto the

pan.D. Turn off the burner

controls.

1. All of the following arecorrect procedures forputting out a fire in a pan on

the stove except:A. Leave the pan where it is.

B. Pour water into the pan.

C. Slide a fitted lid onto the

pan.D. Turn off the burner controls.


39/56


Discrete point & text based multiple choice items

True/false item

Gap-filling (cloze passage) with multiple choice options



Matching

Multiple matching



40/56

Gap-filling (cloze passage) withmultiple choice options

2 methods: - Mechanical deletion

- Selective deletion

Range of skills tested is very limited, restricted to sentence level.

Rules:

First gap should not be placed too near the beginning of the passage.

There should be between 7 and 12 words between gaps.

Words which leave an acceptable sentence when omitted should not be deleted.

Thousands of languages are spoken in the world today. Populations that.........(1)....... similar cultures and live only a short distance ..........(2)...... may stillspeak languages that are quite distinct and not ........(3)........ understood byneighbouring populations.1. A share B belong C keep D own2. A far B apart C divided D separated


41/56



42/56


Ten Days Under the Sea

1. My team and I had gone through a lot toget to where we were. There had been ayear of planning, four days of intensivetraining and, in my case, a lifetime ofambition to work underwater as a marinebiologist.

2. My team-mates, pensive and quiet,seemed to be ruminating on much the same

theme as we arrived at the barge, moored,and exchanged our dry shirts and sandals fordamp wet-suits and ungainly fins. Afteryears of use, the scuba gear I donned hadthe comfort of well-used tools, except forone crucial omission.

3. The Aquarius habitat would be our onlyrefuge and the surface a dangerous place,

where we could die in minutes. Within 24hours of submerging, our bodies wouldbecome saturated with nitrogen gas. In thisstate, a rapid return to the surface wouldinduce a severe and possibly crippling oreven fatal case of decompression sickness,better known as the bends.

A

My familiar red face mask no longer had asnorkel attached to the strap. The most

basic component of my regular equipment

was conspicuous in its

absence, reminding me that, where I was

going, the surface would no longer provide a

safe haven from trouble.B

Still, I couldnt help thinking about the

things I would miss while living underneath

the sea: sunshine, fresh air, open spaces,

even the squadrons of pelicans that soared

silently over the boat.

CFor the next ten days, in fact, we would be

aquanauts, living every marine researchers

fantasy: we would spend as many as six

hours a day working in the water and then

retire to a warm, dry, comfortable shelter for

meals, discussions, relaxation and sleep.

M t hi


43/56

MatchingSilvialikes reading true

stories which people

have written about

themselves. Shesparticularly interested in

people who have had

unusual or difficult lives.

Alienjoys reading crimestories which are

carefully written so that

they hold his interest

right to the end. He

enjoys trying to guess

who the criminal really iswhile hes reading.

A. The Last Journey

John Reynolds final trip to the

African Congo two years ago

unfortunately ended in his

death. For the first time since

then, we hear about where he

went and what happened to

him from journalist Tim

Holden, who has followed

Reynolds route.

B. The Missing Photograph

Another story about the

well-known policeman,

Inspector Manning. It is

written in the same simplebut successful way as the

other Manning stories I

found it a bit disappointing

as I guessed who the

criminal was halfway

through!

C. Free at Last!

Matthew Hunt, who spent

half his life in jail for a crime

he did not do, has written

the moving story of his

lengthy fight to be set free.

Now out of prison, he has

taken the advice of a judge

to describe his experiences in

a book.

D. London Alive

This author of many famous

novels has now turned to

writing short stories with great

success. The stories tell ofLondoners daily lives and

happen in eighteen different

places for example, one story

takes place at a table in a caf,

another in the back of a taxi

and another in a hospital.


44/56

Multiple matching


45/56



46/56


ADVANTAGES

Wide coverage of thecontent area

No spelling &

handwriting problem Marking objectivity

Marking convenience& rapidity

DISADVANTAGES

Testing: recognition >>production

Few chances to test

different abilities Guesswork

Difficulty & long timein test-designing stage

Poor classroompractice

Objectivity Reliability Validity Efficiency

CANDIDATE SUPPLIED RESPONSE


47/56

CANDIDATE-SUPPLIED RESPONSEITEM TYPES

Short answer item

Sentence completion

Open gap-filling (cloze)

Transformation

Word formation

Transformation cloze

Note expansion

Error correction / proof reading

Information transfer

Short answer item


48/56

Short answer itemSECTION 1

SECTION 3

(IELTS9,pp.33&

84)

There is only one unambiguous correct answer or a very limited number of

acceptable answers


49/56

TransformationMaria missed the ferry because her car broke down.If............................................................................................... .

OR

Mark scheme

T f i l


50/56

Transformation cloze

In Holland, people were so desperate to own tulip bulbs thattheir became quite extraordinary. It was not for people tosell all their in order to buy a single tulip bulb. The situationbecame so serious that laws were passed with the ofcontrolling this trade in tulips.

BEHAVECOMMONPOSSESSINTEND

There is a word missing in each line.Find the location of missing words and supply them in their correct form.

Note expansion

I / very pleased / meet you / teachers conference / London / last year.

Mark scheme:I was very pleased (1 mark) to meet you (1 mark)during/(while we were)at (1 mark) the (1 mark) teachers' conference (which was held) in Londonlast year (1 mark).

rather complicated

Difficult to mark


51/56

CANDIDATE-SUPPLIEDRESPONSE ITEM TYPES

ADVANTAGES

(compared with

selection types)

Difficulty & long time in marking stage examiner marking

Issue s of spelling & accuracy subjectiveassessment

DISADVANTAGES

items written easier

wider sample of contentguessing minimized creativity in language use

measure of higher & lower order skills more positive effect on classroom practice similar degree of marking objectivity


52/56

NON-ITEM-BASED TASK TYPES

Writing tasks with detailedinput

Writing tasks with titles only

Presentation

Use of picture prompts Written prompts

Information gap tasks

WRITING


53/56

WRITINGextended writing questions

WRITING TASKS ADVANTAGES DISADVANTAGES

Writing taskswith detailedinput

More uniform set ofresponsesQuicker, easier,

more reliable marking

Requiring inputcomprehensionTesting reading +

writingValid?

Writing taskswith titles only

Easy to produce

Valid in testingwriting

Varied responses

More difficult tomark fairly

Depending on thecandidate'sbackground, culture &education

W i i k i h d il d i


54/56

Writing tasks with detailed input

Writing tasks with titles only

Elementary level

Elementary levelYou must see your friend, David, before tomorrowevening.Write a note to David. Say:

Why you want to see him. Where and when to meet you. Others.

S ki t k


55/56

Speaking tasks

SPEAKING TASKS ADVANTAGES DISADVANTAGES

Presentation Topics of realconcern & interest

Memorization ofmainstream topics

Picture prompt Interest to

candidates

Varied responses

Written prompt Easy preparation Very demanding taskfor candidates

Information gap Suitability even for

elementary level

Simple & repetitive

language production


56/56

REFERENCES

ALTE (2005). ALTE Materials for the Guidance of Test Item Writers.

Retrieved March 10, 2014 fromhttp://www.alte.org/attachments/files/item_writer_guidelines.pdf

Burton, S.J.; Sudweeks, R.R.; Merrill, P. F. & Wood, B. (1991). How toPrepare Better Multiple-Choice Test Items: Guidelines for UniversityFaculty. Brigham Young University Testing Services and The Department

of Instructional Science. Retrieved March 10, 2014 fromhttp://testing.byu.edu/info/handbooks/betteritems.pdf

Cambridge IELTS 9. Cambridge: Cambridge University Press.

Zimmaro, D. M. (2004). Writing Good Multiple-Choice Exams. Austin:University of Texas. Retrieved March 10, 2014 from

http://www6.cityu.edu.hk/edge/workshop/seminarseries/2010-11/Seminar03-WritingGoodMultipleChoiceExams.pdf
http://www.alte.org/attachments/files/item_writer_guidelines.pdfhttp://testing.byu.edu/info/handbooks/betteritems.pdfhttp://www6.cityu.edu.hk/edge/workshop/seminarseries/2010-11/Seminar03-WritingGoodMultipleChoiceExams.pdfhttp://www6.cityu.edu.hk/edge/workshop/seminarseries/2010-11/Seminar03-WritingGoodMultipleChoiceExams.pdfhttp://www6.cityu.edu.hk/edge/workshop/seminarseries/2010-11/Seminar03-WritingGoodMultipleChoiceExams.pdfhttp://www6.cityu.edu.hk/edge/workshop/seminarseries/2010-11/Seminar03-WritingGoodMultipleChoiceExams.pdfhttp://www6.cityu.edu.hk/edge/workshop/seminarseries/2010-11/Seminar03-WritingGoodMultipleChoiceExams.pdfhttp://www6.cityu.edu.hk/edge/workshop/seminarseries/2010-11/Seminar03-WritingGoodMultipleChoiceExams.pdfhttp://www6.cityu.edu.hk/edge/workshop/seminarseries/2010-11/Seminar03-WritingGoodMultipleChoiceExams.pdfhttp://testing.byu.edu/info/handbooks/betteritems.pdfhttp://www.alte.org/attachments/files/item_writer_guidelines.pdf

approaches & techniques of language testing

Documents