1

Language Testing as a Career Development Language Testing as a Career Development Tool: Lessons Learned from the Language Tool: Lessons Learned from the Language

Proficiency Examination for German Proficiency Examination for German Military Air Traffic Controllers in Military Air Traffic Controllers in

Accordance with ICAO Proficiency LevelsAccordance with ICAO Proficiency Levels

Dr. Dugald Sturges, Federal Office of LanguagesHürth, Germany

Dr. Christopher Hüllen, Federal Ministry of DefenceBonn, Germany

BILC Conference, Prague, May 2012

2

Remember the “Key?”Remember the “Key?”

2011: The Key to ICAO: First Experiences with a New Language Proficiency 2011: The Key to ICAO: First Experiences with a New Language Proficiency Examination for Military Air Traffic Controllers and its ImplicationsExamination for Military Air Traffic Controllers and its Implications

• Does testing military Air Traffic controllers Does testing military Air Traffic controllers according to ICAO proficiency levels according to ICAO proficiency levels make sensemake sense as as a means of maintaining and improving a means of maintaining and improving qualityquality? ?

• Is testing Is testing military military Air Traffic controllers according to Air Traffic controllers according to ICAO proficiency levels legally ICAO proficiency levels legally necessarynecessary? ?

Our questions then:Our questions then:

• Can SLP testing Can SLP testing replacereplace ICAO Certification for ICAO Certification for Military Aviators?Military Aviators?

• Despite considerable agreement between the Despite considerable agreement between the results of the two types of examination, SLP testing results of the two types of examination, SLP testing alonealone cannot entirely replace ICAO Certification. cannot entirely replace ICAO Certification.

• Testing Testing military military Air Traffic controllers according toAir Traffic controllers according to ICAO ICAO proficiency levels does proficiency levels does make a great deal of sensemake a great deal of sense as a as a means of maintaining and improving means of maintaining and improving quality.quality.

• Testing Testing military military Air Traffic controllers according to ICAO Air Traffic controllers according to ICAO ((civilcivil) proficiency levels is ) proficiency levels is notnot legally legally necessarynecessary. .

Our answers then:Our answers then:

2011: The Key to ICAO: First Experiences with a New Language Proficiency 2011: The Key to ICAO: First Experiences with a New Language Proficiency Examination for Military Air Traffic Controllers and its ImplicationsExamination for Military Air Traffic Controllers and its Implications

How to build an ICAO testHow to build an ICAO test

DRAKEN

6

Update: BSprA “ICAO Test”Update: BSprA “ICAO Test”

2010 – C2010 – Commissioned by the Air Force Office of Flight ommissioned by the Air Force Office of Flight Safety (AFSBw),Safety (AFSBw), the Federal Office of Languages at the the Federal Office of Languages at the Air Force Officer School, Fürstenfeldbrück,Air Force Officer School, Fürstenfeldbrück, developed developed testing material in Listening Comprehension and Oral testing material in Listening Comprehension and Oral Proficiency Proficiency in accordance with ICAO Level 4.in accordance with ICAO Level 4.

December 2010: Train the Trainer at the Air Force December 2010: Train the Trainer at the Air Force Officer School, Fürstenfeldbrück.Officer School, Fürstenfeldbrück.

January 2011: Testing commenced.January 2011: Testing commenced.

Late 2010: Approbation of Level 4 testing material in Late 2010: Approbation of Level 4 testing material in Hürth and Fürstenfeldbrück.Hürth and Fürstenfeldbrück.

Update: BSprA “ICAO Test”Update: BSprA “ICAO Test”Test Delivery:Test Delivery:• Teams of examiners at Fürstenfeldbrück, Teams of examiners at Fürstenfeldbrück, Hürth, andHürth, and Bückeburg.Bückeburg.• Examinations delivered regionally Examinations delivered regionally • At BSprA schools (Fürstenfeldbrück/ South, At BSprA schools (Fürstenfeldbrück/ South, Hürth/West, Hürth/West, Bückeburg/ North & East)Bückeburg/ North & East)• Where number of examinees warrant - on Where number of examinees warrant - on locationlocationTest Population:Test Population:

• Approximately 300 Military Air Traffic ControllersApproximately 300 Military Air Traffic Controllers

Testing Progress so far (May, 2012):Testing Progress so far (May, 2012):• Since January 2011 Since January 2011 210 ATCs210 ATCs have been tested have been tested

Current Procedure Excerpts from: BesAnMilFS 5-100, English Language Competence:

The goal of certifying “ English Language Competence“ in accordance with ICAO Level 4 is to enable the AT officer to effectively and efficiently communicate both in specialist (e.g. radio terminology) and in general language use.

The language testing will be conducted by the Federal Office of Languages (BSprA).

The language examination consists of a listening comprehension test in multiple choice format and a test of oral expression in the form of an interview. Both parts must be passed. (…)

• Upon passing the language examination the AT Officer receives a language certificate valid for 3 years.

• In case of failure of either or both parts of the language examination, all ratings are to be suspended and the examination is to be repeated after 3 months at the latest. During this period the AT officer is to refresh his or her language skills in self study. (…)

• “In the case of a repeat failure it is necessary to obtain a new valid 3332 SLP” before the suspension of the rating can be lifted.

11

• 180 ATCs passed Level 4 on first try180 ATCs passed Level 4 on first try

The results so far:The results so far:

• 13 passed Level 5 on first try13 passed Level 5 on first try• 3 passed Level 6 on first try3 passed Level 6 on first try• 6 failed Level 4 listening and oral on first try6 failed Level 4 listening and oral on first try

• 7 failed Level 4 just listening on first try7 failed Level 4 just listening on first try

• 1 failed Level 4 just oral on first try1 failed Level 4 just oral on first try• 6 passed Level 4 on second try6 passed Level 4 on second try

• 1 failed Level 4 repeat test (listening)1 failed Level 4 repeat test (listening)

• 7 remain to be retested7 remain to be retested

Total: 202 Certified

12

• In 2011 ICAO Language Proficiency Level exams In 2011 ICAO Language Proficiency Level exams were only conducted at Level 4.were only conducted at Level 4.

• Since January 2012, ICAO Language Proficiency ICAO Language Proficiency

Levels 5 and 6 exams have been conducted at Levels 5 and 6 exams have been conducted at Fürstenfeldbrück Fürstenfeldbrück onlyonly..

• SLP testing i.a.w. STANAG 6001 and ICAO LanguageSLP testing i.a.w. STANAG 6001 and ICAO Language Proficiency Level testing Proficiency Level testing do not replacedo not replace one another, one another, but rather are but rather are coordinatedcoordinated..

13

• As observed in 2011,examinees with a “good” As observed in 2011,examinees with a “good” STANAG Level 2 GENERAL language competency STANAG Level 2 GENERAL language competency pass according to the ICAO Level 4 descriptorspass according to the ICAO Level 4 descriptors

• HoweverHowever, knowledge of specialist aviation , knowledge of specialist aviation terminology and usage is also essential to terminology and usage is also essential to successfully deal with the oral proficiency exam i.a.w. successfully deal with the oral proficiency exam i.a.w. ICAO Proficiency Level 4.ICAO Proficiency Level 4.

ICAO Level 4 = “SLP Level 2+” ICAO Level 4 = “SLP Level 2+” PLUS specialist knowledgePLUS specialist knowledge

Consequence:Consequence: for newly trained flight for newly trained flight personnel, a personnel, a combinationcombination of a of a Level 3 SLPLevel 3 SLP PLUS PLUS specialist specialist courses in Aviation courses in Aviation English and the “US Qualification Course” English and the “US Qualification Course” are are mandatorymandatory..

14

• New conception of the final exams of the Aviation New conception of the final exams of the Aviation English (LFE) and US Qualification Course (USQC) at English (LFE) and US Qualification Course (USQC) at FFBFFB

Procedure approved as of 01/01/2012Procedure approved as of 01/01/2012

• Students who successfully complete the LFE und Students who successfully complete the LFE und USQC courses can now be certified as having ICAO USQC courses can now be certified as having ICAO Language Proficiency Levels 4, 5 and 6. Language Proficiency Levels 4, 5 and 6.

• Now contain listening comprehension and oral Now contain listening comprehension and oral proficiency items which explicitly cover the proficiency items which explicitly cover the testing testing format, skills and topic areas required i.a.w. ICAO format, skills and topic areas required i.a.w. ICAO Language Proficiency Levels.Language Proficiency Levels.

• In this way, the specialist language skills requiredIn this way, the specialist language skills required by the ICAO standard but not part of the profile by the ICAO standard but not part of the profile according to STANAG 6001 can be assessed and according to STANAG 6001 can be assessed and certified.certified.

15

But what about the “older generation?”

16

Consequence:Consequence: for personnel already in service, for personnel already in service,

RECURRENT certification of ATCs according to ICAORECURRENT certification of ATCs according to ICAO

Level 4 has been implemented as a Level 4 has been implemented as a qualityquality

managementmanagement tool (every 3 years). tool (every 3 years).

Reason: Most of the time, ATCs can get by with a Reason: Most of the time, ATCs can get by with a limited corpus of standardized terminology.limited corpus of standardized terminology.Personnel with long years of service may settle Personnel with long years of service may settle into a routine “Tower speak“, which restricts the into a routine “Tower speak“, which restricts the ability to communicate in unexpected situations.ability to communicate in unexpected situations.

Only Only oneone student leaving the USQ course at FFB did student leaving the USQ course at FFB did notnot pass at least ICAO Level 4. pass at least ICAO Level 4. All other failures wereAll other failures were OLDER (age 36+) ATCs OLDER (age 36+) ATCs WITHOUT recent SLPs or language training.WITHOUT recent SLPs or language training.

The ICAO Test as Quality Management Tool

17

The ICAO Test as Quality Management Tool

Most ATCs interviewed have had no formal Most ATCs interviewed have had no formal language training since completing ATC training – language training since completing ATC training – in some cases over 20 years ago!in some cases over 20 years ago!

They use English on a daily basis – but only routine, They use English on a daily basis – but only routine, formulaic language (“Tower-Speak”): REAL language formulaic language (“Tower-Speak”): REAL language competence deteriorates with time.competence deteriorates with time.

Perceived lack of leadership support in maintaining Perceived lack of leadership support in maintaining language skills. language skills.

Most examinees stated that the ICAO certification Most examinees stated that the ICAO certification requirement motivated them to refresh their English requirement motivated them to refresh their English language skills – if only as a preparation for the exam.language skills – if only as a preparation for the exam.

Many examinees expressed desire for Many examinees expressed desire for more SLP more SLP based trainingbased training as preparation for ICAO certification. as preparation for ICAO certification.

Summary:

The ICAO Test as Quality Management Tool

Recurrent testing is useful both as a quality management tool and as motivation to stakeholders to support further language instruction of ATCs.Our ATCs are good – but could be better

The certification requirement increases awareness of non-specialist professional language needs (SLP!)

Testing i.a.w ICAO Proficiency Levels gives us data to evaluate and improve training

In the interest of life-long learning,

In the interest of career-long certification AND

In the interest of improved air traffic safety.

19

““Which way now???”Which way now???”

The Way AheadThe Way Ahead• Enable examiners outside of FFB to test ATCs at Enable examiners outside of FFB to test ATCs at

Levels 5 & 6.Levels 5 & 6.- Testing material has been developed which- Testing material has been developed which will be implemented after Level 5 & 6 Train will be implemented after Level 5 & 6 Train the Trainer has been conducted. the Trainer has been conducted.- Benefit: Enable - Benefit: Enable lifelong certificationlifelong certification of those of those ATCs with Level 6 proficiency, thus reducing ATCs with Level 6 proficiency, thus reducing recurrentrecurrent testing load. testing load.

• Continuing two prong certification process: New Continuing two prong certification process: New personnel will be certified through completion of personnel will be certified through completion of LFE and USQC final exams; existing personnel LFE and USQC final exams; existing personnel

through recurrent testing through recurrent testing throughoutthroughout their careers. their careers.

• Both examination variants are conducted under the Both examination variants are conducted under the full authorityfull authority and and supervisionsupervision of the of the BSprABSprA, ensuring , ensuring compliance with ICAO compliance with ICAO ANDAND STANAG 6001 standards. STANAG 6001 standards.

Over to you…Over to you…

• Civil Aviation Authorities accept language tests Civil Aviation Authorities accept language tests

developed and administered by the Bundes-developed and administered by the Bundes- sprachenamt.sprachenamt.• There is no need for an “ICAO Test” as such.There is no need for an “ICAO Test” as such.• Military pilots and air traffic controllers leaving Military pilots and air traffic controllers leaving the military do not have to take ICAO tests.the military do not have to take ICAO tests.• The German Air Force wants a combination of The German Air Force wants a combination of SLP and Aviation English tests.SLP and Aviation English tests.

2011: The Key to ICAO: First Experiences with a New Language Proficiency 2011: The Key to ICAO: First Experiences with a New Language Proficiency Examination for Military Air Traffic Controllers and its ImplicationsExamination for Military Air Traffic Controllers and its Implications

That’s what we said last year.That’s what we said last year.

Well, let me add a few things…Well, let me add a few things…

• Civil Aviation Authorities accept language tests Civil Aviation Authorities accept language tests developed and administered by the Bundes-developed and administered by the Bundes- sprachenamt. – sprachenamt. – Not really…Not really…

• There is no need for an “ICAO Test” as such.There is no need for an “ICAO Test” as such. Well, yes and no…Well, yes and no…

• Military pilots and air traffic controllers leaving Military pilots and air traffic controllers leaving the military do not have to take ICAO tests.the military do not have to take ICAO tests. True, but…True, but…

• The German Air Force wants a combination of The German Air Force wants a combination of SLP and Aviation English tests. SLP and Aviation English tests. Not sure…Not sure…

The Key to ICAO: Lessons LearnedThe Key to ICAO: Lessons Learned

S L PS L PEE

PP

T S TT S T

The Key to ICAO: Lessons LearnedThe Key to ICAO: Lessons Learned

EE

PP

T S TT S T

The Key to ICAO: Lessons LearnedThe Key to ICAO: Lessons Learned

• Standardized Language ProfileStandardized Language Profile

SLPSLP

• Specialized Language ProfileSpecialized Language Profile

• ESP does not work without SLP.ESP does not work without SLP.

The Key to ICAO: Lessons LearnedThe Key to ICAO: Lessons Learned

27

www.guy-sports.com/months/jokes_aircraft.htm

And don’t forgetAnd don’t forget