annotated bibliography and literature survey: assessment of language proficiency in the health and...
TRANSCRIPT
Annotated Bibliography and Literature Survey:Annotated Bibliography and Literature Survey:Assessment of Language ProficiencyAssessment of Language Proficiency
in the Health and Social Services Sectorin the Health and Social Services Sector
Research Report
Presented by Juliane Bertrand
November 24, 2005
Terms of Reference
Survey the following:
1) ESL proficiency tests for adults working in a professional environment, with emphasis on health and social services workers
2) Materials generated by each such test
3) Studies on ESL proficiency assessment
Main ESL Proficiency Tests
• Administered directly by the design agency
- Test of English as a Foreign Language (TOEFL)- Michigan English Language Assessment Battery (MELAB) - Test of English for International Communication (TOEIC) - ETS Test of Professional English (ETS TOPE)
• Administered by the client institution
- Institutional Testing Program (TOEFL ITP) - Speaking Proficiency English Assessment Kit (SPEAK) - Secondary Level English Proficiency Test (SLEP) - Michigan Test of English Language Proficiency (MTELP) - Comprehensive English Language Test (CELT)
Description of ESL Proficiency Tests
Type of questions Skills assessed Environment Test Duration
Multiple choice
Short answer
Compo-sition and/or
interview
Oral compre-hension
Written compre-hension
Written produc-
tion
Oral produc-
tion
School Work-place
Area of health
and social
services
TOEFL paper
3.20 hrs X X (written)
X X X (TWE)
X *
TOEIC 2.30 hrs X X X X X
MELAB 3.30 hrs X X (both)
X X X X X X
TOEFL ITP
1.55 hrs X X X X X
SLEP 1.20 hrs X X X X
SPEAK 20 min X (oral)
X X
CELT 2.15 hrs X X X
MTELP 1.30 hrs X X X
Conditions forLanguage Proficiency Testing
Test Type of room Staff training Cost
TOEFL Regional centres N/A Test taker: US$140
TOEIC
At the institution Standard room or computer lab
TOEIC staff, or TOEIC-trained institutional staff Institution: $45 + taxes per test taker
TOEFL ITP
At the institution Standard room
Instruction manual supplied by ETS Marking by ETS representatives
Institution: $42.50 + taxes per test taker
MELAB At Ann Harbour N/A Test taker: US$120
SLEP At the institution
Standard room Starter kit:: US$235
10 question booklets: US$85 100 answer sheets: US$85
CELT At the institution Standard room N/A
Examiner's kit: $45 Cassette: $45
10 question booklets: $120.50 100 answer sheets: $55
Other costs per section: $24
MTELP At the institution
Standard room Starter kit: US$165
20 question booklets: US$75 100 answer sheets: US$10
Strengths / Weaknesses Test Strong Points Weak Points
TOEFL paper
- Standardized test - International reputation
- Does not assess oral production - Intended for the school environment - Too general (not geared to health and social services)
TOEIC - Standardized test - International reputation - Designed to assess workers
- Does not assess oral production - Too general (not geared to health and social services)
MELAB - Standardized test - Assesses the four language skills
- Not recognized in the health field - Administered in Michigan (USA)
TOEFL ITP/ SLEP/ CELT/ MTELP
- Can be administered locally - Does not assess oral or written production - Intended for the school environment - Too general (not geared to health and social services)
SPEAK - Can be administered locally - Does not assess oral production - Intended for the school environment - Too general (not geared to health and social services)
ETS TOPE - Assesses the four language skills - Referenced to typical workplace tasks
- Not tailored specifically to the health and social services sector - Administered in China
Canadian Tests
• Canadian Test of English for Scholars and Trainees (CanTEST)
Positives
- Assesses the four language skills- Uses authentic material
Negatives
- Administered only in Toronto, Ottawa, Saskatoon, Winnipeg and Halifax - Not geared to the needs of health and social services workers
Canadian Tests
• Canadian English Language Benchmark Assessment for Nurses (CELBAN)
Positives
- Assesses the four language skills- Referenced to nursing tasks
Negatives
- Administered only in Vancouver, Toronto, Edmonton and Winnipeg
Canadian Tests
• CanTEST
• Cost: about $200, at test-taker’s expense+ travel to a city where the test is administered
• Duration: 3 hrs
• CELBAN
• Cost: $300 plus taxes, at test-taker’s expense + travel to a city where the test is administered
• Duration: 3 hrs 30 min
Other Assessment Instruments
• Canadian Language Benchmarks (CLB)
- Set standards for the four language skills- Centred on the test-taker’s social and professional activities- Lay the ground for designing tests for specific job sectors (e.g. CELBAN)
• Portfolio
- Tracks the learner’s progress- Avoids psychological factors that could affect test performance- Requires ongoing assessment
Materials Generated by these Tests
• Works on theory and practice
- More than 70 research reports published by ETS (e.g. TOEFL, TOEIC, SLEP, TSE) since 1977
- Numerous CCLB documents describing the theoretical bases of Canadian language standards, the characteristics of those standards, and resulting ideas for activities
- Technical manuals on administering language proficiency tests; charts to help interpret test results
Materials Generated by these Tests
• Validity studies
- Frequently re-assessed for standardized tests
- Difficult to produce for locally administered tests given widely varying test management procedures
- CELBAN = a first study suggests a positive link between success with CELBAN and withTOEFL (CCLB, 2004)
Materials Generated by these Tests
• Testimonials from institutions using these tests
- Overall satisfaction with TOEFL and TOEIC
- TOEFL unsatisfactory for nurses
- CELBAN = a study on nurse and administrator satisfaction should be undertaken in 2006
Studies on ESL Proficiency Assessment
• Background
- Prior to the communicative approach, language proficiency was the sole focus of assessment.- Integrated test introduced- Leading works: Lado, Robert (1961). Language Testing, London, Longmans, xxiv + 390 p. Valette, R. M. (1967). Modern Language Testing: A Handbook, New York, Harcourt, Brace & World. Cooper, R. L. (1968). “An elaborated language testing model,” in Problems in Foreign Language Testing: Proceedings of a Conference. Language Learning, 18, p. 57-73. Spolsky, B. (1978). Approaches to Language Testing: Advances in Language Testing, Series 2, Arlington, VA, Center for Applied Linguistics.
Studies on ESL Proficiency Assessment
• Present trends
- With the communicative approach, assessment covers all language skills.- Introduction of alternatives to the standardized test and of tests referenced to quasi-authentic tasks.- Leading works: Canale, M. and M. Swain (1980). “Theoretical bases of communicative approaches to second language teaching and testing,” Applied Linguistics, 1, p. 1-47. Bachman, L. (1990). Fundamental Considerations in Language Testing, Oxford,
Oxford University Press. Celce-Murcia, Z. Dörnyei, S. Thurrell (1995). Communicative competence: A pedagogically motivated model with content specifications. Issues of Applied
Linguistics, 6 (2), p. 5-35. Milanovic, Michael and Nick Saville (Eds.) (1996). Performance Testing, Cognition and Assessment. Selected papers from the 15th Language Testing Research Colloquium, Cambridge and Arnhem, Cambridge, Cambridge University Press, coll. “ Studies in Language Testing 3,” 314 p.
Conclusion
• None of the tests surveyed can be used to assess proficiency in English as a second language for all job categories in the health and social services sector.
• CELBAN, designed with reference to CLB, seems to be a model to follow. It is consistent with current language acquisition theories (e.g. emphasis on language objectives rather than specific grammatical constructs and on the use of authentic tasks).