Bridging the Gap between Theory and Practice?

J Charles Alderson

Lancaster University

What Gap?

• Between What and What?

• Whose View? From what Perspective?

• Should the Gap be Filled?

• By Whom?

• When and How?

Conference claim• In the last 25 - 30 years, research in the field of

Language Testing has grown exponentially.• A perennial concern, however, is that the day-to-

day practice of language testing has not changed as quickly as our understanding of the theory of language testing.

• Language testing practitioners complain that research reports are hard to understand and often address matters that are irrelevant or at least far removed from their daily concerns.

• Researchers and practitioners seldom co-operate. We sense, however, that this climate is changing and we think that EALTA can contribute to and perhaps accelerate the pace of change.

The dichotomies

• Long-standing – as old as the hills

• But true?

• Beware dichotomies

• Is action research a solution? Exploratory practice?

• Can theory ever be relevant?

• Is the first term always better than the second? Or the second better than the first?

• Is reality better than what we aspire to?

• Are the two in conflict?

• Why opposition? Why not synergy?

• Is practice necessarily “behind” research?

• Can theory not learn from practice?

• Is research necessarily hard to understand?

• Must everything be applicable to all realities tomorrow?

• Perhaps theory can help us understand practice, rather than be directly applicable

• Is rejection of theory and research mere Philistinism? A refusal to try to understand, a rejection of “jargon” without going beneath the surface?

• A result of practitioners’ unwillingness to come to grips with unfamiliar concepts?

• Are theorists and researchers remote from everyday practice?

Challenge

• Is there such a gap in language testing?• Between theory and practice?• Research and the concerns of

practitioners?• Test researchers and test developers?• Testers and Teachers?• Official claims and actual practice?• The Ideal and the Real?

• Does the Gap exist?

• And if so, is it a problem?

Research articles: irrelevant?

• The effect of test-taker gender, audience and topic on task performance in tape-mediated assessment of speaking

• Examining the predictive validity of a screening test for court interpreters

• Do visual chunks and planning impact performance on the graph description task in the SPEAK exam?

Research articles: irrelevant?

– To change or not to change: investigating the value of MCQ answer changing for Gulf Arab students

– Relating examinations to the Common European Framework: a Manual

– Limitations of the Common European Framework for developing comparable examinations and tests

Research articles: irrelevant?

• The development of a suite of computer-based diagnostic tests based on the Common European Framework

• The Common European Framework and the European Language Portfolio: involving learners and their judgements in the assessment process

• Assessing the language of young learners

Research articles: irrelevant?

• Progress and problems in reforming public language examinations in Europe: cameos from the Baltic States, Greece, Hungary, Poland, Slovenia, France and Germany

• Language testing in the military: problems, politics and progress

• Dark alleys and blind bends: testing the language of learning

Research articles: irrelevant?

• Stakeholders' conflicting aims undermine the washback function of a high-stakes test

• Reading to learn and reading to integrate: new tasks for reading comprehension tests?

• Linguistic sources of item bias for second generation immigrants in Dutch tests

• Generalizability of writing scores: an application of structural equation modeling

Research articles: irrelevant?

• Did they take the same test? Examinee language proficiency and the structure of language tests

• Self-assessment, preparation and response time on a computerized oral proficiency test

• Peer assessment of language proficiency

• Building and supporting a case for test use

Research articles: irrelevant?

• The impact of wearing a face mask in a high-stakes oral examination: An exploratory post-SARS study in Hong Kong

• Examining rater effects in TestDaF writing and speaking performance assessments: A Many-Facet Rasch analysis

• Individual feedback to enhance rater training: Does it work?

Research articles: irrelevant?

• Re-thinking second language admission requirements: Problems with language residency criteria and the need for language assessment and support

• Resolving score differences in the rating of writing samples: Does discussion improve the accuracy of scores?

Testers are remote from practitioners?The example of ENLTA

• 1 year experience in secondary (12 year olds)

• 17 years experience teaching adults

• 10 years experience teacher-training (primary,secondary, adult)

Testers are remote from practitioners?The example of ENLTA

• Teaching languages for 25 years:

• 9 years training school

• 5 years secondary school

• 11 years primary school

Testers are remote from practitioners?The example of ENLTA

• 3 years teaching in evening classes/study circles• 6 months teaching in secondary school• 2 months teaching in upper secondary school• Two summer courses for adult students• 15 years teaching in municipal adult education• 20 years in-service training for teachers • 15 years teaching language didactics and

assessment at university level

Testers are remote from practitioners?The example of ENLTA

• 2 years teaching language courses to adults and children

• 2 years teaching part-time continuing courses for people who did not graduate from a primary, secondary or a vocational school;

• 1 year teaching secondary school pupils; • One-to-one language teaching (mostly at home),

usually helping school children to improve their language skills

• 6 years teaching university students language classes, and language testing,

Testers are remote from practitioners?The example of ENLTA

• Teacher in international language school • In-company business English teacher• Kindergarten/primary school teacher in private school• Temporary EFL summer school teacher• Temporary pre-sessionals • Teacher Trainer British Council Teaching Operation• Director of Studies, University Language Teaching

centre, Language courses for teachers • Undergraduates Teacher Training College• Private teaching: young learners 7-14; secondary school

learners 16-19; adults

Testers are remote from practitioners?The example of ENLTA

• 3 years teacher of English at primary level,• 10 years EFL teacher, technical secondary

school,• 3 years EFL teacher, grammar secondary

school,• 9 years EFL teacher, private language

school • More than 15 years private one-to-one

English teaching

ENLTA was remote from practitioners?

Pre conference workshops

• Krakow, 2006– Assessing speaking at B1 level– Designing and reporting research

• Voss, 2005 – Using the European Language Portfolio– Classical test analysis

ENLTA was remote from practitioners?

• Activity Two: Survey of training in testing

• Activity Four: Survey of attitudes to testing

Are these books irrelevant?

Only high-stakes proficiency testing? Irrelevant to practitioners?

• DIALANG – no stakes, freely available, diagnostic

• New school-leaving examinations

• Training teachers in modern approaches to language testing

Only high-stakes proficiency testing? Irrelevant to practitioners?

• Increasing interest in diagnosing students’ strengths and weaknesses

• Researching what language develops as learners progress through the CEFR levels

• Washback of tests on teachers and learners• Teachers’ practices in assessing learners in

class

Importance of classroom testing

• Increased interest in researching classroom testing and understanding the issues and the needs.

• Important to understand better both what actually happens in classsroom assessment and what should or could happen, how it might be improved.

• EALTA’s mission is to contribute to this.

Gaps? Codes of practice as the interface with reality

• Many Codes of Practice already (ILTA, AERA/APA, JALT, ALTE, etc etc)

• Who needs another one?• Who monitors compliance, or even

activity?• What use a code of practice that is not

implemented?

Gaps? Codes of practice and Reality

• Alderson, J.C. and G. Buck (1993). ‘Standards in testing: a study of the practice of UK examination boards in EFL/ ESL testing’. Language Testing 10(1), 1-26

LINKAGE TO THE COMMON EUROPEAN FRAMEWORK

• What evidence is there of the quality of the process followed to link tests and examinations to the Common European Framework?

• Have the procedures recommended in the Manual and the Reference Supplement been applied appropriately?

• Is there a publicly available report on the linking process?

Gaps? Claims about links with the CEFR and reality

• Importance of CEFR in testing, training, publishing and curricula

• Many claims of links to CEFR• Who monitors quality?

– Council of Europe?– ALTE?– Self-monitoring?– EALTA

Gaps? Claims about links with the CEFR and reality

• Role of EALTA as Europe’s prime, independent body of language testing practitioners?

• How? Where? When? Who? How much? What?

• Why? Why Not?

Bridging the Gap between Theory and Practice?

• Necessary?

• Classroom formative and summative testing and assessment

• Codes of practice

• CEFR