language assessment and languages for spppecific purposes · and languages for spppecific purposes...

Language Assessment Language Assessment Language Assessment Language Assessment and Languages for and Languages for and Languages for and Languages for Specific PurposesSpecific Purposesp pp p

A match made in heaven?

Prof. Dr. Christian KrekelerHochschule Konstanz HTWGHochschule Konstanz HTWG

Lost in translationLost in translation

Video clip Lost in translation, CNN news report on communication difficulties between air traffic controllerscommunication difficulties between air traffic controllers and pilots.

Link to video: Fulcher, G. Language Testing Resources. languagetesting.info http://languagetesting.info/whatis/scenarios/aviation.php

Link to transcript: CNN online:Link to transcript: CNN, online: http://archives.cnn.com/TRANSCRIPTS/0707/06/ywt.01.htmlt

Argumentsa bit crowded!

language competencelanguage competence

characteristics of the language user

ValidityValidity of of languagelanguage assessmentsassessments

future performance /future performance / language use

language competencelanguage competence

ValidityValidity of of languagelanguage assessmentsassessments

future performance /future performance / language use

language competencelanguage competence

ValidityValidity of of languagelanguage assessmentsassessments

future performance /future performance / language use

test

language competencelanguage competence

ValidityValidity of of languagelanguage assessmentsassessments

future performance /future performance / language use

test score

test performanceperformance

test

language competencelanguage competence

ValidityValidity of of languagelanguage assessmentsassessments

future performance /future performance / language use

inferences

test performanceperformance

language competencelanguage competence

ValidityValidity of of languagelanguage assessmentsassessments

test fperformance

language competencecontent knowledge

ValidityValidity of of specificspecific purposepurposelanguagelanguage teststests

construct irrelevant varianceconstruct irrelevant variance

test fperformance

Sprachkompetenzlanguage competencecontent knowledge

ValidityValidity of of specificspecific purposepurposelanguagelanguage teststests

English for mechanicalengineeringSample

t tg g

writingtest

W it t t (200 t 250 d ) G dWrite a text (200 to 250 words) on „Gears and backlash“. Address the following questions:•What is backlash?•Why does backlash exist?•Why does backlash exist?•How can backlash be minimized?

future performanceon specifict k /j b

future performancelanguage use

tasks/jobs

test fperformance

Sprachkompetenzlanguage competencecontent knowledge

ValidityValidity of of specificspecific purposepurposelanguagelanguage teststests

Content knowledge is irrelevant for the Content knowledge is irrelevant for the construct „language use performance“.

Valid interpretations of test scores of LSP tests are impossible

because it is impossible to distinguish between the influences of

language knowledge and background knowledge

on the performance on the test.

ValidityValidity of of specificspecific purposepurposelanguagelanguage teststests

German for Business, Level 4 (LCCI)

Sample t t (LCCI)test

Sie arbeiten in der Produktionsabteilung von MüllerSaft, einer Firma, die Erfrischungsgetränke , , g gherstellt. Der Leiter Ihrer Abteilung, Herr Thomas, zeigt Ihnen folgendes Fax, das er gerade g g , gbekommen hat, und gibt Ihnen weitere Anweisungen. g

German for Business, Level 4 (LCCI)

Sample t t (LCCI)

Qualitätskontrolle MüllersaftPelzer Abteilungsleiter

testPelzer, AbteilungsleiterAN Herrn Thomas, ProduktionsabteilungFAX NUMMER 0232 8350233DATUM 18 Okt bDATUM 18. OktoberANZAHL DER SEITEN 1Betrifft: Probleme mit Serie AW/4030

Wie wir bei der regelmäßigen Qualitätskontrolle festgestellt haben, ist in alle Flaschen der Reihe "Orangenfizz" Reinigungsmittel gelangt, und g g g g g ,zwar in der Serie AW/4030. Diese Ladung ist leider bereits an Supermärkte und Lebensmittelgeschäfte ausgeliefert worden. Die betreffenden Flaschen müssen nun schnellstens vom VerkaufDie betreffenden Flaschen müssen nun schnellstens vom Verkauf zurückgezogen werden. Bitte benachrichtigen Sie die Supermärkte so schnell wie möglich.

future performanceon specifict k /j b

future performancelanguage use

tasks/jobs

test fperformance

Sprachkompetenzlanguage competencecontent knowledge

ValidityValidity of of specificspecific purposepurposelanguagelanguage teststests

future performanceon specifict k /j b

future performancelanguage use

tasks/jobs

ValidityValidity of of specificspecific purposepurposelanguagelanguage teststests

language performancein the specific purpose domain

ValidityValidity of of specificspecific purposepurposelanguagelanguage teststests

language performancein the specific purpose domain

“Specific purpose language ability lt f th i t tiresults from the interaction

between specific purpose background knowledge specific purpose background knowledge and language abilitylanguage ability ”and language abilitylanguage ability.

(Douglas, 2000: 40)

specific purpose language abilityp p p g g y

ValidityValidity of of specificspecific purposepurposelanguagelanguage teststests

language performancein the specific purpose domain

specific purposelanguage testlanguage test

specific purpose language ability

ValidityValidity of of specificspecific purposepurpose

p p p g g y

languagelanguage teststests

language performancein the specific purpose domain

A specific purpose language test is one in which test content and methods are derived from an analysis of a specific purpose target language use situation, so that test tasks and content are authentically representative of tasks in the target situation […]. g [ ]Such a test allows us to make inferences about a test taker's capacity to use language in the specific purpose domain (Douglas 2000: 19) domain (Douglas, 2000: 19).

specific purpose language ability

ValidityValidity of of specificspecific purposepurpose

p p p g g y

languagelanguage teststests

language performancein the specific purpose domain

A specific purpose language test is one in which test content and methods are derived from an analysis of a analysis of a specific purpose target language use situationspecific purpose target language use situation, so that test tasks and content are authentically representativerepresentative of tasks in the target situation […]. g [ ]Such a test allows us to make inferences about a test taker's capacity to use language in the specific purpose domain (Douglas 2000: 19) domain (Douglas, 2000: 19).

specific purpose language ability

ValidityValidity of of specificspecific purposepurpose

p p p g g y

languagelanguage teststests

language performancein the specific purpose domain

specific purposelanguage testlanguage test

specific purpose language ability

ValidityValidity of of specificspecific purposepurpose

p p p g g y

languagelanguage teststests

language performancein the specific purpose domain

inferences

test performanceperformance

specific purpose language ability

ValidityValidity of of specificspecific purposepurpose

p p p g g y

languagelanguage teststests

language performancein the specific purpose domain

inferences

test performanceperformance

specific purpose language ability

ValidityValidity of of specificspecific purposepurpose

p p p g g y

languagelanguage teststests

language performancein the specific purpose domain

inferences

test performanceperformance

specific purpose language ability

ValidityValidity of of specificspecific purposepurpose

p p p g g y

languagelanguage teststests

Content knowledge is an aspect of theContent knowledge is an aspect of theconstruct of LSP ability, because languageperformance varies with context andperformance varies with context andbecause academic and vocational contextsoften require specific language use often require specific language use.

LSP tests may be useful if the construct isbased on the interaction between specificpurpose background knowledge andlanguage ability.

(see also Douglas, 2000)

ValidityValidity of of specificspecific purposepurposelanguagelanguage teststests

A tA tArgumentsArguments

Investigating LSP testsVisualizing the position of LSP tests in a positioning matrixVisualizing the position of LSP tests in a positioning matrix

German for Business, Level 4 (LCCI)

Sample t t (LCCI)test

Sie arbeiten in der Produktionsabteilung von MüllerSaft, einer Firma, die Erfrischungsgetränke , , g gherstellt. Der Leiter Ihrer Abteilung, Herr Thomas, zeigt Ihnen folgendes Fax, das er gerade g g , gbekommen hat, und gibt Ihnen weitere Anweisungen. g

German for Business, Level 4 (LCCI)

Sample t t (LCCI)

Qualitätskontrolle MüllersaftPelzer Abteilungsleiter

testPelzer, AbteilungsleiterAN Herrn Thomas, ProduktionsabteilungFAX NUMMER 0232 8350233DATUM 18 Okt bDATUM 18. OktoberANZAHL DER SEITEN 1Betrifft: Probleme mit Serie AW/4030

Wie wir bei der regelmäßigen Qualitätskontrolle festgestellt haben, ist in alle Flaschen der Reihe "Orangenfizz" Reinigungsmittel gelangt, und g g g g g ,zwar in der Serie AW/4030. Diese Ladung ist leider bereits an Supermärkte und Lebensmittelgeschäfte ausgeliefert worden. Die betreffenden Flaschen müssen nun schnellstens vom VerkaufDie betreffenden Flaschen müssen nun schnellstens vom Verkauf zurückgezogen werden. Bitte benachrichtigen Sie die Supermärkte so schnell wie möglich.

German for Business, Level 4 (LCCI)

Sample t t (LCCI)

Qualitätskontrolle MüllersaftPelzer Abteilungsleiter

testPelzer, AbteilungsleiterAN Herrn Thomas, ProduktionsabteilungFAX NUMMER 0232 8350233DATUM 18 Okt bDATUM 18. OktoberANZAHL DER SEITEN 1Betrifft: Probleme mit Serie AW/4030

Wie wir bei der regelmäßigen Qualitätskontrolle festgestellt haben, ist in alle Flaschen der Reihe "Orangenfizz" Reinigungsmittel gelangt, g g g g g ,und zwar in der Serie AW/4030. Diese Ladung ist leider bereits an Supermärkte und Lebensmittelgeschäfte ausgeliefert worden. Die betreffenden Flaschen müssen nun schnellstens vom VerkaufDie betreffenden Flaschen müssen nun schnellstens vom Verkaufzurückgezogen werden. Bitte benachrichtigen Sie die Supermärkte so schnell wie möglich.

register (input and response)

influence of background background

knowledge on test performanceGB test performance

InvestigatingInvestigating LSP LSP teststests

Test of Legal English skills(TOLES)Sample

t t( )www.toles.co.uktest

Test of Legal English skills(TOLES)Sample

t t( )www.toles.co.uktest

register (input and response)

LE1LE1

influence of background background

knowledge on test performanceGB test performance

InvestigatingInvestigating LSP LSP teststests

Test of Legal English skills(TOLES)Sample

t t( )www.toles.co.uktest

Test of Legal English skills(TOLES)Sample

t t( )www.toles.co.uktest

register (input and response)

LE1LE1

influence of background background

knowledge on test performanceGB test performance

InvestigatingInvestigating LSP LSP teststests

Test of Legal English skills (TOLES)

Sample t t (TOLES)test

register (input and response)

LE1LE1

influence of background background

knowledge on test performanceGB test performance

LE2InvestigatingInvestigating LSP LSP teststests

English for mechanicalengineeringSample

t t writingtest

Write a text (200 to 250 words) on „Gears and backlash“. Address the following questions:g q

Wh t i b kl h?•What is backlash?•Why does backlash exist?y•How can backlash be obtained?•How can backlash be minimized?

register (input and response)

LE1 M1LE1 M1

influence of background background

knowledge on test performanceGB test performance

LE2InvestigatingInvestigating LSP LSP teststests

English for mechanicalengineeringSample

t t writingtest

Listen to the lecture of Professor Overdrive on „Gears and backlash“. Write a short text summarizing the following aspects:

•What is backlash?•Why does backlash exist?•How can backlash be minimized?•How can backlash be minimized?

register (input and response)

LE1 M2 M1LE1 M2 M1

influence of background background

knowledge on test performanceGB test performance

LE2InvestigatingInvestigating LSP LSP teststests

English Language Proficiency for Aeronautical Communication(EUROCONTROL)

Sample t t (EUROCONTROL)test

English Language Proficiency for Aeronautical Communication(EUROCONTROL)

Sample t t (EUROCONTROL)test

Listening: Controller: Air France 370 good morning, identified. Say againyour flight level. Pilot: We are flight level 370 Air France 370Pilot: We are flight level 370. Air France 370. C: Air France 370, roger. Be advised no SSR readout. P: Roger Air France 370. Apparently we have some problemsg pp y pwith our transponder. And we just recycled already. C: Air France 370 roger. Try your second transponder on code62636263. P: We tried that already and it doesn't seem to work. Air France370370. C: Air France 370 roger.

English Language Proficiency for Aeronautical Communication(EUROCONTROL)

Sample t t (EUROCONTROL)test

register (input and response)

LE1 M2 M1

AE1

LE1 M2 M1

AE1

influence of background AE2 background

knowledge on test performanceGB

AE2test performance

LE2InvestigatingInvestigating LSP LSP teststests

Investigating LSP testsInvestigating LSP tests

Constructing LSP assessmentsTeaching LSP

1 CLIL1. CLIL2. Purpose3. Analysis of a specific purpose

language use situationlanguage use situation◦AuthenticityPerformance testing◦Performance testing◦Complex tasks

4. Consider general language usesituationssituations

ConstructingConstructing LSP LSP assessmentsassessments

Vielen Dank!Vielen Dank!Vielen Dank!Vielen Dank!

Prof. Dr. Christian KrekelerHochschule Konstanz HTWGHochschule Konstanz HTWG

Douglas, D. (2000) Assessing language for Douglas, D. (2000). Assessing language for specific purposes. Cambridge: CUP.Fulcher, G. Language Testing Resources. , g g glanguagetesting.info

T t f L l E li h Skill TOLES Test of Legal English Skills, TOLES, www.toles.co.ukGerman for Business LCCI German for Business, LCCI, www.lccieb-germany.com

English Language Proficiency forg g g yAeronautical Communication, EUROCONTROL, www.elpac.info

ReferencesReferences