RATERS:SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES

•ICAO WORKSHOP ON LANGUAGE PROFICIECY •Baku, Azerbaijan, December 7-9, 2005

•Evgenia Povaliy, Aviation English Instructor •Sergey Melnichenko, Deputy Director•Moscow ATC Training Center

CASE STUDYCASE STUDY

Accidents where language was a Accidents where language was a causal or contributing factorcausal or contributing factor

Phraseologies only are not Phraseologies only are not enoughenough

Plain language is needed in non-Plain language is needed in non-routine and urgent situationsroutine and urgent situations

English as an Instrument English as an Instrument of Communicationsof Communications

70 per cent of 28,000 safety reports 70 per cent of 28,000 safety reports were connected with communicationswere connected with communications

70 per cent of those who communicate 70 per cent of those who communicate are non-native English speakersare non-native English speakers

No other existing live or artificial No other existing live or artificial language is either better or worse than language is either better or worse than EnglishEnglish

English has become a “lingua franca”English has become a “lingua franca”

Manual on the implementation Manual on the implementation of ICAO Language Proficiency of ICAO Language Proficiency

Requirements (Doc 9835)Requirements (Doc 9835)

Licensing authorities should require Licensing authorities should require specialized evaluation of those who do specialized evaluation of those who do not demonstrate Level 6 proficiencynot demonstrate Level 6 proficiency

Phraseologies-only testing is not Phraseologies-only testing is not appropriateappropriate

Demonstration of actual speaking and Demonstration of actual speaking and listening ability is requiredlistening ability is required

Until 5 March 2008 States may continue Until 5 March 2008 States may continue to use procedures they currently have to use procedures they currently have in place to assess English proficiency of in place to assess English proficiency of flight crews and controllersflight crews and controllers

Requirements for Requirements for ratersraters

Higher education in LinguisticsHigher education in Linguistics Level 6 proficiency according to the Level 6 proficiency according to the

ICAO ScaleICAO Scale Highly developed moral standardsHighly developed moral standards Aviation background Aviation background Relevant training, preferably in an Relevant training, preferably in an

English-speaking countryEnglish-speaking country

THE SCALETHE SCALE

Who has seen it?Who has seen it? Who is happy with it?Who is happy with it? Who has used it for Who has used it for

benchmarking evaluation?benchmarking evaluation? Who has been taught to use Who has been taught to use

the Scale?the Scale?

TESTSTESTS

EntryEntry PlacementPlacement DiagnosisDiagnosis ProgressProgress ProficiencyProficiency

ENTRY TESTENTRY TEST

Enables Airline Operators and Enables Airline Operators and ANS providers to avoid new ANS providers to avoid new personnel that is likely to personnel that is likely to have problems in 2008 have problems in 2008

because of poor knowledge of because of poor knowledge of English at presentEnglish at present

PLACEMENT TESTPLACEMENT TEST

Enables Airline Operators and Enables Airline Operators and ANS providers to map out the ANS providers to map out the

level of existing personnellevel of existing personnel

DIAGNOSIS TESTDIAGNOSIS TEST

Enables linguists to identify Enables linguists to identify the needs of individuals the needs of individuals (problem areas, course (problem areas, course

duration and its content)duration and its content)

PROGRESS TESTPROGRESS TEST

Enables Airline Operators, Enables Airline Operators, ANS providers and linguists to ANS providers and linguists to

mark progress/no mark progress/no progress/regress progress/regress and to use and to use

other training path when other training path when neededneeded

PROFICIENCY TESTPROFICIENCY TEST

Shows the actual level of Shows the actual level of English language proficiency English language proficiency

according to ICAO Scaleaccording to ICAO Scale

Available TestsAvailable Tests

ChineseChinese KoreanKorean American (Oral Proficiency Interview)American (Oral Proficiency Interview) British (Mayflower College)British (Mayflower College) Russian (Moscow ATC Center)Russian (Moscow ATC Center) Eurocontrol (ELPAC – July 2007)Eurocontrol (ELPAC – July 2007)

DrawbacksDrawbacks

Computer – no interactionComputer – no interaction Oral Proficiency – time and Oral Proficiency – time and

manpower-consumingmanpower-consuming THERE IS NO IDEAL TESTING TODAYTHERE IS NO IDEAL TESTING TODAY

BUT…BUT… ““Any test at all” is not better than Any test at all” is not better than

“no test” (Doc 9835)“no test” (Doc 9835)

Steps to Make Steps to Make

Introduce ICAO qualification Introduce ICAO qualification requirements into national legislationrequirements into national legislation

Select (through independent linguistic Select (through independent linguistic institutions) and train ratersinstitutions) and train raters

Evaluate current level of pilots and Evaluate current level of pilots and controllerscontrollers

Ensure they are adequately taught Ensure they are adequately taught General and Aviation EnglishGeneral and Aviation English

Evaluate proficiency level of pilots and Evaluate proficiency level of pilots and controllers in 2008controllers in 2008

LAST BUT NOT LEASTLAST BUT NOT LEAST

Code of ethicsCode of ethics

Ethical PrinciplesEthical Principles (ILTA Code of Ethics)(ILTA Code of Ethics)

Raters shall have respect for humanity and dignity of each Raters shall have respect for humanity and dignity of each of their test takersof their test takers

Raters shall have all information in confidence and use Raters shall have all information in confidence and use professional judgment in sharing itprofessional judgment in sharing it

Raters should adhere to all relevant ethical principles Raters should adhere to all relevant ethical principles embodied in national and international guidelines when embodied in national and international guidelines when undertaking any trial, experiment or other research activityundertaking any trial, experiment or other research activity

Raters shall not allow the misuse of their professional Raters shall not allow the misuse of their professional knowledge or skillsknowledge or skills

Raters shall continue to develop their professional Raters shall continue to develop their professional knowledge, sharing it with colleaguesknowledge, sharing it with colleagues

Raters shall share the responsibility of upholding the Raters shall share the responsibility of upholding the integrity of language testing professionintegrity of language testing profession

Raters shall strive to improve the quality of language Raters shall strive to improve the quality of language testing, assessment and teaching servicestesting, assessment and teaching services

Raters shall be mindful of their obligations to the societyRaters shall be mindful of their obligations to the society Raters shall regularly consider the potential effects on all Raters shall regularly consider the potential effects on all

stakeholders, reserving the right to withhold their stakeholders, reserving the right to withhold their professional services on the grounds of conscience professional services on the grounds of conscience

THANK YOU!THANK YOU!

Evgenia Povaliy, Aviation English InstructorEvgenia Povaliy, Aviation English Instructor povaliy@atcm.rupovaliy@atcm.ru Sergey Melnichenko, Deputy Director Sergey Melnichenko, Deputy Director melnichenko@atcm.ru www.aviaenglish.ru Moscow ATC Training CenterMoscow ATC Training Center