language implications of nato’s expanding roles dr. ray t. clifford bilc conference, san antonio...

Language Implicationsof NATO’s

Expanding Roles

Dr. Ray T. CliffordBILC Conference, San Antonio

21 May 2007

Welcome to this year’sBILC conference!

What is “BILC”?

What is BILC?

• Bureau for

International

Language

Co-ordination

• A consultative and advisory body for language training matters in NATO.

Language Matters

“The single most important problem identified by almost all Partners as an impediment to developing interoperability with the Alliance has been shortcomings in communications.”

EAPC (PARP-C) D (97) 1, page 10.

BILC Within NATO

Bureau for International Language Coordination( The Consultative and Advisory Body on Language Matters )

Joint Services Sub-Group( The Action Committee for Join t T raining )

A ir Force

Finance

NATO Training Group( Responsible for Consolidation o f Multi-national T raining )

Military Committee( Senior M ilitary A uthority in NATO )

• NATO’s only language training organization– Independently chartered 1966

• The UK (1966-1981), Germany (1982-1996), and US (1997-2007) have staffed the Secretariat.

– Conducts an annual Conference each Summer– Sponsors a PfP Professional Seminar in the Fall– Responsible for STANAG 6001– Provides services to support the International

Military Staff and the International Staff:• Language Training Assessments • Language Testing Seminars

Bureau for International Language Co-ordination

Why are we here?• Language matters have never been more

important to NATO.

• NATO language needs have increased as NATO’s role has expanded.– The Cold War era.– The post Cold War transformation.– The post September 11, 2001 era.

• This year’s presentation and study group topics reveal the scope of these expanded language needs.

Expanding Needs

• There are more members and more partners in the Alliance.

• NATO Response Forces now require both:– English for interoperability.

• For staff positions.• For NCOs during unit training and operations.

– Languages Other Than English• For conflicts outside of Europe.• For counterterrorism.

The Challenge is formidable:

Language is the most complex of human behaviors.

“Kuuuurijate töööö jäääärel.”

“A moon researcher’s work-night at the edge of the ice.”

Example from a Brochure on the Estonian Language:

Language is the most complex of human behaviors.

Even professional translators can make mistakes…

Sign in Moscow Hotel

If this is your first visit to Moscow, you

are welcome to it.

Hong Kong Tailor Shop

Ladies may have

a fit upstairs.

Instructions for Thermostatin Japanese Hotel

Cooles and Heates: If you want just condition of warm in your room, please control yourself.

Cocktail Lounge in Norway

Ladies are requested not to have children

in the bar.

Even Simple Language Isn’t Simple

Expertise Needed

• When faced with complex problems…– Novices will always see more options than

are seen by the experts.– However, experts will see more solutions !

• We are here to gather the expertise needed to find solutions. – Presentations.– Study Groups.– Discussions of the Steering Committee.

Some Items on this Week’sSteering Committee Agenda

• Review of 2006 action items.– Data base of language courses taught in

support of NATO operations.– Language training support for the JSSG

TEPSO Working Group.– Upgrading the BILC website.– Recommendation that the force goals (EG

0356) are unrealistically high.– Submission of reading items for the BAT.

Some Items on this Week’sSteering Committee Agenda

(Continued)

• Progress on the BAT.

• Review of BILC Constitution and Rules of Procedure.

• JSSG tasking to create a third edition of STANAG 6001.

• Transition of BILC Secretariat responsibilities.

Some Historical Background

“Progress on the BAT”

• What is the BAT?

What is a BAT?

BAT is an acronym for…

Benchmark

Advisory

Test(s)

What is a Benchmark Advisory Test?

• Purpose. – The test is designed for NATO military and civilian

personnel who are not native speakers of English.– The purpose of the BAT is to assess international

English language proficiency, regardless of how it was acquired, by testing reading comprehension of a variety of authentic written materials representing a range of topics, and author purposes.

– The test may also be used as an external criterion for validation of nationally-developed tests.

BAT History• 2005 – BILC

– The concept of Benchmark Advisory Tests was approved, without financial support

– Reading test specifications drafted.– Working Group Volunteers began submitting items.

• 2006 – NATO, ACT– Formally asked that tests be developed.– Offered funding, then decided that the funding had to

be competitively awarded.– October: Published the bid requirements

• Deliver international language testing capability in 4 skills.• All tests must be finished within a year.• 30 days to prepare a proposal.• The lowest cost proposal would win.

BAT History• 2006 – BILC Secretariat

– October: Formed a coalition with a professional organization (The American Council on the teaching of Foreign Languages) which was already providing speaking and writing proficiency tests internationally.

– November: Submitted a proposal.– December: The BILC/ACTFL proposal won

the competition and a contract was awarded.– Financial support for travel to meetings of the

working group was at last available.

Our Professional Partner

• The American Council on the Teaching of Foreign Languages (ACTFL) currently has standardized proficiency tests and test delivery capabilities:– Speaking

• 50 languages

– Writing• 12 languages

– Reading and listening• Under construction

The Benchmark Advisory Test• Built on Voluntary National Contributions

– Pool of reading items from 12 BILC member nations

– Working Group of Testing experts from BILC member nations

• Test delivery– Online reading, listening, and writing– Telephonic speaking test with digitized

sound files• Test scoring

– Reading and listening scored online– Speaking and writing scored by trained

raters

Benchmark Advisory Test• Progress to date:

– Initial software (alpha version) developed– Test specifications for reading and listening– Review and selection of initial pool of reading

items– Conducted Modified Angoff validation of

reading test items– Beginning the design specifications for

speaking and writing tests– Developing a workshop to train speaking and

writing test raters

Benchmark Advisory Test

• Future enhancements– Online instructions and familiarization

items.

– Computer-adaptive versions of reading and listening tests

Creating BAT Testsis a Complex Process

• Language is the most complex of human behaviors.

• Testing is also a complex process.• Language Testing is complexity

squared.

3 Major Types of Tests

• Achievement

• Performance

• Proficiency

3 Major Types of Tests

• Achievement – Memorized responses using the content of

a specific textbook or curriculum.

• Performance – Rehearsed ability to communicate in

specific, familiar settings.

• Proficiency – Unrehearsed general ability to accomplish

real-world communication tasks across a wide range of topics and settings.

Different Types of Tests

• Achievement – Rehearsed, memorized responses

– Tests what is taught

– Content of a specific textbook or curriculum

Sample Achievement Test Item

Complete the following with the correct verb form in the past.

(go) I _____________ to the United States last year.

(be) My seat on the plane _______ in business class.

(have) My associates and I _________ meetings each day.

(eat) We _________ at typical American restaurants.

• Performance – Semi-rehearsed and rehearsed

responses

– Ability to communicate in specific, familiar, and predictable settings

– What one can do with what has been taught and practiced

Sample Performance Test Item

Complete the following sentences about an upcoming business trip. Add a minimum of 5 additional words to each sentence.

For an upcoming business trip I plan to ____________________________________________.

I am certain that the trip will be successful because ____________________________________________.

• Proficiency – Spontaneous, unrehearsed

communication

– General ability to accomplish communication tasks in a variety of settings

– Skill is transferable from one context to another

Sample Proficiency Test Item

You are planning a business trip abroad. You contact a travel agent. You need to purchase plane tickets, arrange hotel reservations, and get information about local transportation, but you encounter significant hurdles that you had not anticipated, including …

A Summary of the Contrasts Achievement, Performance and Proficiency

Achievement

Memorized

Performance

Rehearsed

Proficiency

Unrehearsed

Repeat, produce, choose

Show specific skills

Demonstrate a wide range of abilities

Context

Textbook, Curriculum

Focused, restricted

Broad, in-depth, variable

Accuracy

Determined by the teacher

Situation dependent

Ascending expectations

The Benchmark Advisory Tests

Are Proficiency Tests

In the next series of charts…

• Each observable increase in ability could:– Represent a week of intensive language

study. – Be tested using an achievement test.

• Multiple observable achievements could be tested with a performance test.

• Multiple performance gains across a range of topics and tasks could be tested with a general proficiency test.

Testing Second Language Skills

Incremental skill development moving from achievement to rehearsed performance andbuilding a foundation for increased proficiency by using purposefully designed curricula to:

1. Learn relevant topical content.2. Master level-appropriate communication tasks.3. Document progress using ability profiles, test results, portfolios, etc.

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

ties a

ssessed u

g infr

equent

3 Major Types of Tests• Achievement = Memorized responses

using the content of a specific textbook or curriculum.

• Performance = Rehearsed ability to communicate in specific, familiar settings.

• Proficiency = Unrehearsed general ability to accomplish real-world communication tasks across a wide range of topics and settings.

Every Level of the STANAG 6001 Proficiency Scale Describes:

• Tasks• A statement of what is to be done or

accomplished.

• Conditions• A description of the conditions under which (or

the context in which) the task is to be performed.• For language testing, this includes the topics to be

addressed.

• Accuracy• A definition of how well the task must be

performed under the conditions stated.

LEVEL TASKS CONTEXT/TOPICS ACCURACY

All expected of an educated NS

All subjects Accepted as an educated NS

Tailor language, counsel, motivate, persuade, negotiate

Wide range of professional needs

Extensive, precise, and appropriate

Support opinions, hypothesize, explain, deal with unfamiliar topics

Practical, abstract, special interests

Narrate, describe, give directions Concrete, real-world, factual

Intelligible even if not used to dealing

with non-NS

Errors never interfere with

communication & rarely disturb

Q & A, create with the language Everyday survivalIntelligible with

effort or practice

Use memorized phrases Random Unintelligible

STANAG 6001 - Speaking(Summarized) as a Standard

Therefore, Benchmark Advisory Tests

are Proficiency Tests

• STANAG 6001 is a proficiency scale.

• Tests derived from it assess unrehearsed, general language ability.

Of course, BAT development is but one of BILC’s current tasks. We’ll report on all of the Steering Committee agenda Items in our report on Thursday

The Steering Committee Report

• Benchmark Advisory Tests.

• Review of BILC Constitution and Rules of Procedure.

• JSSG tasking to create a third edition of STANAG 6001.

• Transition of BILC Secretariat responsibilities.

Thank you for attending this BILC Annual Conference!

We look forward to working with you this week to better define and address the:

“Language Implications of

NATO’s Expanding Roles”.

language implications of nato’s expanding roles dr. ray t. clifford bilc conference, san antonio...

Documents