The elusive skill: How can we test L2 listening validly?

John Field,John Field,John Field,John Field,CRELLA, University of Bedfordshire, UKCRELLA, University of Bedfordshire, UK

F l f Ed i C b id U i iF l f Ed i C b id U i iFaculty of Education, Cambridge UniversityFaculty of Education, Cambridge University

Picking up some themes…

The importance of the test takerThe importance of the test taker Local Local vsvs international testinginternational testing Testing for learningTesting for learning Testing for learningTesting for learning Testing along a developmental continuumTesting along a developmental continuum An evidenceAn evidence--based approach to testingbased approach to testing

Th t t t kThe test taker (O’Sullivan, 2011)

Individual characteristicsPh i l h l i l i i lPhysical, psychological, experiential

CognitiveCognitiveProcesses Resources

(Behaviour) (Knowledge)

Expertisepe se

An expertp Is somebody who can use a skill in way that is Is somebody who can use a skill in way that is

rapid and that does not demand a great deal ofrapid and that does not demand a great deal ofrapid and that does not demand a great deal of rapid and that does not demand a great deal of forethought.forethought.

A good driver does not have to think about theA good driver does not have to think about the A good driver does not have to think about the A good driver does not have to think about the process of changing gears.process of changing gears.

A good speaker constructs and produces aA good speaker constructs and produces a A good speaker constructs and produces a A good speaker constructs and produces a sentence without having to pause to think about sentence without having to pause to think about the words or grammar being usedthe words or grammar being usedthe words or grammar being used.the words or grammar being used.

A test of a language skill should not assess A test of a language skill should not assess knowledge of language but expertise in using thatknowledge of language but expertise in using thatknowledge of language, but expertise in using that knowledge of language, but expertise in using that language.language.

Predictive testing Many high stakes language test scores are Many high stakes language test scores are

employedemployed predictivelypredictively: e.g. to show that an: e.g. to show that anemployed employed predictivelypredictively: e.g. to show that an : e.g. to show that an individual is capable of performing in a individual is capable of performing in a particular job class or academic settingparticular job class or academic settingparticular job, class or academic setting.particular job, class or academic setting.

This places a responsibility on the test This places a responsibility on the test designer to ensure that the test elicits designer to ensure that the test elicits behaviour similar to the behaviour that behaviour similar to the behaviour that be v ou s o e be v oube v ou s o e be v ouhappens in a realhappens in a real--world context.world context.

Cognitive validity (Glaser 1991)Cognitive validity (Glaser, 1991) Clearly we cannot reproduce the Clearly we cannot reproduce the

circumstances of a real listening event in the circumstances of a real listening event in the artificial environment of a test.artificial environment of a test.

But cognitive validity requires us to find out But cognitive validity requires us to find out if theif the mental processesmental processes that a test elicitsthat a test elicitsif the if the mental processes mental processes that a test elicits that a test elicits from a candidate resemble the processes from a candidate resemble the processes that he/she would employ in nonthat he/she would employ in non--testtestthat he/she would employ in nonthat he/she would employ in non test test conditions.conditions.

At issue:At issue: How valid is the test as a predictorHow valid is the test as a predictor At issue: At issue: How valid is the test as a predictor How valid is the test as a predictor of realof real--life performance?life performance?

‘The elusive skill’e e us ve s With writing and speaking, we have evidence of With writing and speaking, we have evidence of

how test takers perform. Listening and reading take how test takers perform. Listening and reading take place internally, so we have to teach and test them place internally, so we have to teach and test them indirectly indirectly –– by asking questions.by asking questions.

This brings teaching and testing closely together. This brings teaching and testing closely together. g g g y gg g g y g But we should never forget that the results we But we should never forget that the results we

obtain tell us about theobtain tell us about the productproduct of listening not theof listening not theobtain tell us about the obtain tell us about the productproduct of listening, not the of listening, not the process (how the test taker arrived at the answer).process (how the test taker arrived at the answer).

So it is difficult to make use of test scores toSo it is difficult to make use of test scores to So it is difficult to make use of test scores to So it is difficult to make use of test scores to diagnose and deal with a class’s listening problems.diagnose and deal with a class’s listening problems.

‘The elusive skill’ 2

Testing listening usually has to involve Testing listening usually has to involve other skills other skills –– most often readingmost often reading

This raises questions of construct validity.This raises questions of construct validity. This raises questions of construct validity. This raises questions of construct validity. To what extent are we testing listening and To what extent are we testing listening and to what extent are we testing reading?to what extent are we testing reading?to what extent are we testing reading?to what extent are we testing reading?

Distribution

50

2030

4050

6070

809010 90

Construct validation

These results from piloting a test show that it These results from piloting a test show that it di i i t ll b t l It i htdi i i t ll b t l It i htdiscriminates well between learners. It might seem discriminates well between learners. It might seem to test the construct (let us say listening) well.to test the construct (let us say listening) well.

But supposing it is a test of listening with an easy But supposing it is a test of listening with an easy recording but very complicated comprehension recording but very complicated comprehension questions?questions?

It is possible that the result mainly represents It is possible that the result mainly represents learners’ reading skills and not their listening learners’ reading skills and not their listening skillsskills

Th d t d t d h t i b iThe need to understand what is being tested Weir (2005) argues strongly that we need to have Weir (2005) argues strongly that we need to have

a clearer idea of the construct we are testinga clearer idea of the construct we are testinga clearer idea of the construct we are testing a clearer idea of the construct we are testing before designing a test.before designing a test.

In the case of a test of listening we need to knowIn the case of a test of listening we need to know In the case of a test of listening, we need to knowIn the case of a test of listening, we need to know1. How does an expert listener behave (what is 1. How does an expert listener behave (what is

the target behaviour learners are workingthe target behaviour learners are workingthe target behaviour learners are working the target behaviour learners are working towards?)towards?)

2 What do test takers actually do in a listening2 What do test takers actually do in a listening2. What do test takers actually do in a listening 2. What do test takers actually do in a listening test? How closely does it resemble natural test? How closely does it resemble natural listening?listening?listening?listening?

Establishing cognitive validity

In designing tests, testers need to know more In designing tests, testers need to know more b t h t li t i i lik f th li tb t h t li t i i lik f th li tabout what listening is like for the listener:about what listening is like for the listener: About the About the speech signalspeech signal that reaches a listener’s that reaches a listener’s

ear, and the problems it might cause to an L2 ear, and the problems it might cause to an L2 listener.listener.

About the About the processesprocesses that an expert listener uses that an expert listener uses in normal circumstances and the way they in normal circumstances and the way they might vary in the case of an L2 listener.might vary in the case of an L2 listener.

We need to avoid easyWe need to avoid easy comparisons with readingcomparisons with reading The input is very different in readingThe input is very different in reading

Spelling is fixed and does not vary muchSpelling is fixed and does not vary muchThere are regular gaps between wordsThere are regular gaps between wordsThere are regular gaps between wordsThere are regular gaps between words

The process of reading is very different The process of reading is very different A reader can look back to checkA reader can look back to checkA reader can speed up or slow downA reader can speed up or slow downA reader can speed up or slow downA reader can speed up or slow down

How to find out what listeningHow to find out what listening really is?really is? Psychologists have built up quite detailed Psychologists have built up quite detailed

accounts of all four language skills.accounts of all four language skills. They draw upon hard evidence obtained inThey draw upon hard evidence obtained in They draw upon hard evidence obtained in They draw upon hard evidence obtained in

their research. So we are not just using their research. So we are not just using intuition to guess what the skill consists ofintuition to guess what the skill consists ofintuition to guess what the skill consists of intuition to guess what the skill consists of ((cfcf the subthe sub--skills approach). This is an skills approach). This is an

id b did b d hhevidence basedevidence based approachapproach

A cognitive framework for listening (Field, 2008)g g ( )

INPUT DECODINGLanguage knowledge:INPUT DECODING knowledge:Phonology

Lexis

Co-text PARSING Syntax

Text-so-far MEANING CONSTRUCTION

World knowledge

1. Decoding1. Decoding1. Decoding1. Decoding

D diDecoding

INPUT LANGUAGEINPUT LANGUAGEINPUT LANGUAGE INPUT LANGUAGE

* ! + > / ^ THE+ SUN + ISRISING

Phoneme (?)Phoneme (?) SyllableSyllableyy WordWord Phrase / clausePhrase / clause

Th i t f d diThe importance of decodingl h h i f li il h h i f li i An early approach to the testing of listening An early approach to the testing of listening

focused heavily on this level of the process, focused heavily on this level of the process, making use of dictation and even minimal making use of dictation and even minimal pair discriminationpair discrimination

In a later change of thinking, decoding was In a later change of thinking, decoding was dismissed as irrelevant.dismissed as irrelevant.d s ssed s e ev .d s ssed s e ev .

But the fact is that many failures of But the fact is that many failures of comprehension originate at the level ofcomprehension originate at the level ofcomprehension originate at the level of comprehension originate at the level of decoding.decoding.

Challenges to accurate decoding Unlike in reading, there is a high level of variation Unlike in reading, there is a high level of variation

in the form taken byin the form taken byin the form taken byin the form taken by PhonemesPhonemes

((Cl l ff d b h h b f d f )Cl l ff d b h h b f d f )((Closely affected by the phonemes before and after)Closely affected by the phonemes before and after) Words in connected speechWords in connected speech

(often compressed so as to stress the most important (often compressed so as to stress the most important word in the group)word in the group)

The voices of speakersThe voices of speakers The voices of speakersThe voices of speakers(high (high vsvs low, fast low, fast vsvs slow, accent, precision)slow, accent, precision)

Knowledge and recognitiong g Testers of listening often choose recordings on the Testers of listening often choose recordings on the

grounds of language: the transcript shows it to begrounds of language: the transcript shows it to begrounds of language: the transcript shows it to be grounds of language: the transcript shows it to be at or slightly above the linguistic level of the at or slightly above the linguistic level of the target candidatetarget candidategg

But the issue is not what the candidate knows but But the issue is not what the candidate knows but what he/she can recognise when it occurs in what he/she can recognise when it occurs in ggconnected speechconnected speech..

1. Testers need to work from the recorded 1. Testers need to work from the recorded material, not just the transcript.material, not just the transcript.

2. They need to be sensitive to the perceptual 2. They need to be sensitive to the perceptual y p py p pprominence of words and structures in a given prominence of words and structures in a given recordingrecording

Do we still test decoding?Do we still test decoding? Item writers tend to state that they are notItem writers tend to state that they are notItem writers tend to state that they are not Item writers tend to state that they are not

concerned with testing decoding skills; their concerned with testing decoding skills; their target is ‘comprehension’target is ‘comprehension’target is comprehension .target is comprehension .

T i h ?Testing what?E I’ f h f f IE I’ f h f f I-- Excuse me. I’m waiting for the ten fifteen train. Is Excuse me. I’m waiting for the ten fifteen train. Is

it on time?it on time?N ’ b lN ’ b l-- No, it’s running about seven minutes late.No, it’s running about seven minutes late.

1. She is waiting for a train that should arrive at1. She is waiting for a train that should arrive ata. 10.50 b. 7.15 c. 10.15 d. 7.50a. 10.50 b. 7.15 c. 10.15 d. 7.50

2. a. She is not in time for the train 2. a. She is not in time for the train b. She is seven minutes too late for the train.b. She is seven minutes too late for the train.c. She will need to wait a little longer for the trainc. She will need to wait a little longer for the trainc. She will need to wait a little longer for the trainc. She will need to wait a little longer for the train

3. The train will arrive at: a. 10.57 b. 10. 22. c. 10.073. The train will arrive at: a. 10.57 b. 10. 22. c. 10.07

Do we still test decoding?Do we still test decoding? In fact, many items target word and phraseIn fact, many items target word and phraseIn fact, many items target word and phrase In fact, many items target word and phrase

level recognitionlevel recognitionThi i ti l lid B t it i i t t tThi i ti l lid B t it i i t t t This is entirely valid. But it is important to This is entirely valid. But it is important to ensure a balance between this type of focus ensure a balance between this type of focus and higher level meaning construction and higher level meaning construction

Some test formats such as gap fillingSome test formats such as gap filling Some test formats such as gap filling Some test formats such as gap filling encourage a heavy dependence upon decoding encourage a heavy dependence upon decoding

d l d th t f li t id l d th t f li t iand exclude other aspects of listening.and exclude other aspects of listening.

2. Parsing2. Parsing2. Parsing2. Parsing

Listening as on-line processing

What is the speaker saying?What is the speaker saying?

ParsingParsinga. The heavy fall …a. The heavy fall …yy

clumsily.clumsily.b Th l h db Th l h db. The actor learnt the words … b. The actor learnt the words …

had been written by Shakespeare. had been written by Shakespeare. y py pc. The rescuers discovered the plane …c. The rescuers discovered the plane …

had crashed. had crashed. d. The lawyer questioned …d. The lawyer questioned …d. The lawyer questioned …d. The lawyer questioned …

by the judge admitted lying. by the judge admitted lying.

A conclusion

Listening in L1 or in L2 is a very Listening in L1 or in L2 is a very i t i hi hi t i hi happroximate process in which approximate process in which

A listener forms hypotheses of what the A listener forms hypotheses of what the speaker is saying and is about to sayspeaker is saying and is about to say

The listener often changes the hypotheses The listener often changes the hypotheses e ste e o te c a ges t e ypot esese ste e o te c a ges t e ypot esesas more evidence comes in.as more evidence comes in.

L2 listeners are much less willing to changeL2 listeners are much less willing to change L2 listeners are much less willing to change L2 listeners are much less willing to change hypotheses than L1 ones.hypotheses than L1 ones.

Sample 1

You hear an explorer talking about a journey You hear an explorer talking about a journey he’s making How will he travel once he ishe’s making How will he travel once he ishe s making. How will he travel once he is he s making. How will he travel once he is across the river?across the river?

b hi lb hi lA. by motor vehicleA. by motor vehicleB. on horsebackB. on horsebackC. on footC. on foot

((FCE HandbookFCE Handbook, 2008: 60), 2008: 60)

Recording 1 (FCE Sample Test 1:1)

The engine’s full of water at the moment, it’s very The engine’s full of water at the moment, it’s very d btf l if f th t k t thd btf l if f th t k t thdoubtful if any of the trucks can get across the doubtful if any of the trucks can get across the river in this weather. The alternative is to carry all river in this weather. The alternative is to carry all th t ff i th ld f tb id hi h ith t ff i th ld f tb id hi h ithe stuff across using the old footbridge, which is the stuff across using the old footbridge, which is perfectly possible …and then use horses rather perfectly possible …and then use horses rather th t k f th t f th t i ll thth t k f th t f th t i ll ththan trucks for the rest of the trip all the way than trucks for the rest of the trip all the way instead of just the last 10 or 15 kilometres as was instead of just the last 10 or 15 kilometres as was

i i l i t ti W l i k thi i l i t ti W l i k thour original intention. We can always pick up the our original intention. We can always pick up the vehicles again on the way back down…vehicles again on the way back down…

Recording 1 (FCE Sample Test 1:1)

The engine’s full of water at the moment, it’s very The engine’s full of water at the moment, it’s very doubtful if any of the trucks can get across the riverdoubtful if any of the trucks can get across the riverdoubtful if any of the trucks can get across the river doubtful if any of the trucks can get across the river in this weather. The alternative is to carry all the stuff in this weather. The alternative is to carry all the stuff across using the old footbridge which is perfectlyacross using the old footbridge which is perfectlyacross using the old footbridge, which is perfectly across using the old footbridge, which is perfectly possible …and possible …and then use horses rather than trucksthen use horses rather than trucks for for the rest of the trip all the way instead of just the lastthe rest of the trip all the way instead of just the lastthe rest of the trip all the way instead of just the last the rest of the trip all the way instead of just the last 10 or 15 kilometres as was the original intention. We 10 or 15 kilometres as was the original intention. We can always pick up the vehicles again on the waycan always pick up the vehicles again on the waycan always pick up the vehicles again on the way can always pick up the vehicles again on the way back down…back down…

Recording 1 (FCE Sample Test 1:1)

The engine’s full of water at the moment, it’s very The engine’s full of water at the moment, it’s very doubtful if any of the doubtful if any of the truckstrucks can get across the river can get across the river doub u a y o edoub u a y o e uc suc s ca ge ac oss e veca ge ac oss e vein this weather. The alternative is to in this weather. The alternative is to carrycarry all the stuff all the stuff across using the old across using the old footfootbridge, which is perfectly bridge, which is perfectly gg g , p yg , p ypossible …and possible …and then use horses rather thanthen use horses rather than truckstrucks for for the rest of the trip all the way instead of just the last the rest of the trip all the way instead of just the last p y jp y j10 or 15 kilometres as was our original intention. We 10 or 15 kilometres as was our original intention. We can always can always pick up the vehiclespick up the vehicles again on the way again on the way yy p pp p g yg ydown…down…

Recording 1 (FCE Sample Test 1:1) [[ The engine’s full of water at the moment], [The engine’s full of water at the moment], [ it’s it’s

very doubtful ifvery doubtful if any of the any of the truckstrucks can get across the can get across the river in this weather]. [river in this weather]. [ The alternative is to The alternative is to carrycarryall the stuff across using the old all the stuff across using the old footfootbridge], [bridge], [which is perfectly possiblewhich is perfectly possible] [] […and …and then then [[ use use horses rather thanhorses rather than truckstrucks for the rest of the trip] [for the rest of the trip] [all the wayall the way instead of just the last 10 or 15 instead of just the last 10 or 15 kilometres] [kilometres] [ as was the original intention]. [as was the original intention]. [We We can always can always pick up the vehiclespick up the vehicles again again on the way on the way downdown] [] […]…]

Double parsing

It can be quite difficult for learners to hold a series It can be quite difficult for learners to hold a series f d i th i i d f l h til thf d i th i i d f l h til thof words in their mind for long enough until they of words in their mind for long enough until they

can recognise the grammatical pattern in themcan recognise the grammatical pattern in them But a traditional test is even more difficult than But a traditional test is even more difficult than

real life. real life. The listener has to parse twiceThe listener has to parse twice –– once to once to identify the grammar structure and once to check identify the grammar structure and once to check if the group of words relates to the item or not.if the group of words relates to the item or not.

3. Meaning construction3. Meaning construction3. Meaning construction3. Meaning constructiona. Enriching meaninga. Enriching meaning

b. Handling informationb. Handling information

What is ‘comprehension’?Testers tend to characterise tests of second language Testers tend to characterise tests of second language

listening as measuring 'comprehension'; withoutlistening as measuring 'comprehension'; withoutlistening as measuring comprehension ; without listening as measuring comprehension ; without clearly defining the construct. However, they are clearly defining the construct. However, they are quite often sensitive to the need for a mixture of quite often sensitive to the need for a mixture of qq

LocalLocal questions versus questions versus global global questionsquestions ExtractingExtracting gistgist versus extractingversus extracting detaildetail Extracting Extracting gist gist versus extracting versus extracting detaildetail Extracting Extracting factfact versus interpreting speaker versus interpreting speaker intentionsintentions Handling a range of comprehensionHandling a range of comprehension task typestask types Handling a range of comprehension Handling a range of comprehension task types.task types.

h i f h h ld h b ?h i f h h ld h b ?But what proportion of each should there be?But what proportion of each should there be?In what ways are the processes different?In what ways are the processes different?

A model of meaning construction g(Field 2008)

PROPOSITIONENRICHENRICH

MEANING

MEANING REPRESENTATION

HANDLE INFO

DISCOURSE REPRESENTATION

Implications for testing

Questions may be asked at three levels:Questions may be asked at three levels: PropositionProposition: : local factual informationlocal factual information Meaning:Meaning: requiring the listener to relate what the requiring the listener to relate what the gg q gq g

speaker says to the context or to draw conclusions speaker says to the context or to draw conclusions which are not explicitly expressedwhich are not explicitly expressedp y pp y p

Discourse:Discourse: showing a global understanding of showing a global understanding of what was said (including speaker intentions etc.)what was said (including speaker intentions etc.)what was said (including speaker intentions etc.)what was said (including speaker intentions etc.)

Phase 1 Proposition -> meaning repnPhase 1 Proposition meaning repn

From the speech signal, the listener extracts From the speech signal, the listener extracts p g ,p g ,a proposition a proposition –– a literal meaninga literal meaning

He / she then has to:He / she then has to: He / she then has to:He / she then has to: Relate the meaning to its Relate the meaning to its contextcontext EnrichEnrich the meaning (drawing upon the meaning (drawing upon

world knowledge)world knowledge) Make Make inferencesinferences ResolveResolve referencereference (she it this did so)(she it this did so) Resolve Resolve referencereference (she, it, this. did so)(she, it, this. did so) InterpretInterpret the speaker’s intentionsthe speaker’s intentions

Phase 2: meaning -> discoursePhase 2: meaning > discourseHandling informationHandling information

ChooseChoose Is it important? Is it relevant?

ConnectConnect

p

How is it linked to the last utterance?ConnectConnect

CC

How is it linked to the last utterance?

Is what I think I heard consistentCompare Compare Is what I think I heard consistent with what was said so far?

ConstructConstruct What is the overall line of argument?

Structure building (Gernsbacher, 1990) Skilled listeners construct a hierarchical Skilled listeners construct a hierarchical

t ti f dit ti f direpresentation of a recordingrepresentation of a recording

Structure buildingStructure building Unskilled listeners focus their attention at local Unskilled listeners focus their attention at local

l ll llevel. level.

They build a linear structure.They build a linear structure.

A t t b ildi t kA structure building taskThree types of pollutionThree types of pollution

1..…………………..a. Example:………….b. Solution:……………2. …………………….a. Cause: ………………..b. Result: Climate change3. …………………….a. Result:……………………..b. Solution: ……………………..

Why is information handling omitted in y gpresent test design?

Choose:Choose: the tester chooses which information points the tester chooses which information points to focus on to focus on –– sometimes choosing points that are not sometimes choosing points that are not

l h dil h dicentral to the recordingcentral to the recording ConnectConnect: Much testing focuses on single points, with : Much testing focuses on single points, with

i h b f d fi h b f d fno connection to those before and afterno connection to those before and after Compare:Compare: Tests rarely ask learners to check Tests rarely ask learners to check

i f i (f l i fi f i (f l i finformation (for example, comparing two accounts of information (for example, comparing two accounts of an accident)an accident)C t tC t t T t l k f id th t lT t l k f id th t l Construct.Construct. Tests rarely seek for evidence that learners Tests rarely seek for evidence that learners can construct an outline based upon macrocan construct an outline based upon macro--and micro and micro points / headings and subheadingspoints / headings and subheadingspoints / headings and subheadingspoints / headings and subheadings

The inflexibility of high stakes tests

Large scale highLarge scale high--stakes testsstakes tests have majorhave majorLarge scale highLarge scale high stakes tests stakes tests have major have major constraints which prevent them from testing constraints which prevent them from testing listening in a way that fully represents the skilllistening in a way that fully represents the skilllistening in a way that fully represents the skill.listening in a way that fully represents the skill. RaterRater reliability and ease of markingreliability and ease of marking Highly controlled test methods, using Highly controlled test methods, using

traditional formats that the candidate knowstraditional formats that the candidate knowstraditional formats that the candidate knowstraditional formats that the candidate knows Little attention possible to individual Little attention possible to individual

variation or alternative answersvariation or alternative answers

The advantage of more local testsThe advantage of more local tests

Local smallerLocal smaller--scale testsscale tests afford the possibility of afford the possibility of t ti id f li t i itht ti id f li t i ithtesting a wider rage of listening processes with:testing a wider rage of listening processes with:

•• More open ended questionsMore open ended questions•• More scope for testing information handlingMore scope for testing information handling•• Marking on an individual basisMarking on an individual basisMarking on an individual basisMarking on an individual basis•• Possible acceptance of alternative answersPossible acceptance of alternative answers

Progress testing Properly designed, progress tests might enable the Properly designed, progress tests might enable the

tester to diagnose specific listening problems.tester to diagnose specific listening problems. In a follow up (ideally soon after), the In a follow up (ideally soon after), the

teacher/tester can ask: teacher/tester can ask: Why did you give that Why did you give that answer? What do you think you heard?answer? What do you think you heard?

In this way, a test can help to determine which In this way, a test can help to determine which aspects of listening should be focused on in later aspects of listening should be focused on in later smallsmall--scale practice exercises.scale practice exercises.

In other words, this kind of test can be formative In other words, this kind of test can be formative rather than just judgemental.rather than just judgemental.

Some referencesSome references Field, J. (2008): Field, J. (2008): Listening in the Language Classroom. Listening in the Language Classroom. , ( ), ( ) g g gg g g

Cambridge: CUPCambridge: CUP Field, J. (2009) ‘A cognitive validation of the lecture based Field, J. (2009) ‘A cognitive validation of the lecture based

ti i th IELTS li t i ’ti i th IELTS li t i ’ IELTS R h R tIELTS R h R tquestion in the IELTS listening paper’. question in the IELTS listening paper’. IELTS Research ReportsIELTS Research Reports, , VolVol 99

Field J (forthcoming) ‘Cognitive validity’ In Taylor L andField J (forthcoming) ‘Cognitive validity’ In Taylor L and Field, J (forthcoming) Cognitive validity . In Taylor, L. and Field, J (forthcoming) Cognitive validity . In Taylor, L. and GeranpayehGeranpayeh, A. , A. Examining ListeningExamining Listening. CUP . CUP

Gernsbacher,LGernsbacher,L--A. (1990) A. (1990) Language Comprehension as Language Comprehension as Structure BuildingStructure Building. Mahwah, NJ: Erlbaum. Mahwah, NJ: Erlbaum

Shaw, S & Weir, C. (2007) Shaw, S & Weir, C. (2007) Examining WritingExamining Writing. . Cambridge: Cambridge: CUPCUPCUP CUP

Weir, C. (2005) Weir, C. (2005) Language testing and validation: an evidenceLanguage testing and validation: an evidence--based approach. based approach. Basingstoke:Basingstoke: Palgrave MacmillanPalgrave Macmillanpppp gg gg

Contact details

john.field@beds.ac.ukjohn.field@beds.ac.uk

With grateful acknowledgements to theWith grateful acknowledgements to theWith grateful acknowledgements to the With grateful acknowledgements to the British Council and CRELLA for British Council and CRELLA for

i hi lki hi lksupporting this talksupporting this talk