1C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

Listening. Learning. Leading.

Extend Your Recruiting Reach:The new TOEFL® Internet-based Test

Eileen Tyson - TOEFL Program, ETSTim McNamara - University of MelbourneJeffrey Smart - Swinburne University of TechnologyMelissa Banks - Banks Consulting

2C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

Listening. Learning. Leading.

Educational Testing Service

• A private, nonprofit institution, founded in 1947

• Mission: To advance quality and equity in education by providing fair and valid assessments, research, and related services for all people worldwide.

• ETS develops, administers and scores more than 24 million tests annually in more than 180 countries, at over 9,000 locations worldwide.

3C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

Listening. Learning. Leading.

TOEFL - Test of English as a Foreign Language

• Measures English proficiency of non-native speakers

• Primarily used for academic admission but also used by health professions for certification and licensing

• Began in 1964, tested 17.6 million test takers since then

• Nearly 750,000 registrants in 180 countries in 2003-04

• Test of Spoken English (TSE) and Test of Written English (TWE) created in 1980s

• TOEFL moved to computer-based testing in 1998

4C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

Listening. Learning. Leading.

TOEFL Recognition of the Test

More than 5200 institutions worldwide recognise TOEFL

• 220 institutions in Asia• 101 institutions in the U.K.• 370 institutions in other European

countries• 61 institutions in Australia and New

Zealand• 27 institutions in Mexico, Central and

South America• Increase in score reports sent to

institutions outside North America

5C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

Listening. Learning. Leading.

Increasing Your Recruitment Reach• Volumes are up in 2005,

especially in Asia, for example:– China + 14%– India + 47%– Korea + 50%– Taiwan +55%

• The TOEFL population can help you extend your recruiting reach to these and other countries

• ETS/Hobsons TOEFL Access

6C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

Listening. Learning. Leading.

Who takes the TOEFL Test?

Percentage of test taker volume• Japan 12%• China 12%• Korea 12%• India 10%• Taiwan 3%• Germany 2%• France 2%• Turkey 2%

7C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

Listening. Learning. Leading.

TOEFL iBT• Measures all four skills• Internet-delivered

– Administered at secure test centers– Fixed date testing – 30-40 per year depending on

location– Greatly expanded number of test centers

• Security– Secure ID document with photo and signature– Picture taken at test center, and included on score

report– Signature collected at test center– Official and examinee score reports on security-

enhanced paper• Registration: online, by phone, by mail• Test fee $ 185 AUD (US $140)• Phased introduction

8C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

Listening. Learning. Leading.

TOEFL iBT ScoresA new score scale• Four section scores

– Reading 0 - 30– Listening 0 - 30– Speaking 0 - 30 – Writing 0 – 30

• A total score: 0 – 120Setting appropriate TOEFL iBT score

requirements• Score comparison charts• Standard setting process

9C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

Listening. Learning. Leading.

TOEFL iBTImproved score reporting

• For test takers– Online score reporting 15 business days

after test• Paper score reports also mailed

– Performance feedback on test taker score reports

• For institutions – in 2006• Online scores available for institutions

designated by the student

1 0C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

Listening. Learning. Leading.

Academic and commercial organizations that will:

• Sell & Market ETS tests• Provide ETS learning products

and services• Advise students on ETS test use• Score/rate ETS tests

Operational iBT sites that administer ETS internet-based tests:

• Administer ETS Tests• Provide staff for ETS tests

administered at other test centers

Certified Education Partner (CEP)

Certified Test Administration Sites (CTAS)

ETS Certified Education Partner(ECEP) Program

1 1C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

Listening. Learning. Leading.

Join the ETS testing network

• Existing lab, computer lab, or other secure room with at least 15 Internet-connected computers

• PCs, not MACs, three years old or newer• Room needs to be reserved for ETS testing

purposes 10 or more times per year. • Testing on Friday evenings, Saturdays, some

Sundays– Start times vary from 8 a.m. to 7 p.m.

• Must be recognized education-related institution of good reputation.

• ETS compensates you and your staff for managing your site’s testing.

1 2C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

Listening. Learning. Leading.

• FREE TOEFL iBT Practice Test www.ets.org/toeflpractice

• Reading, Listening, Writing• Instant feedback

• Free Speaking sample questions and responses

• Free discussion boards and study tips• Practice materials

– The Official ETS TOEFL iBT Study Guide• Institutional resources

– TOEFL iBT Teacher Professional Development Workshops / Manual

– NorthStar: Building Skills for the TOEFL iBT from Pearson Longman and ETS for skill-building in the classroom

– New Institutional TOEFL in late 2006

Resources for Students and Teachers

1 3C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

MAKING TOEFL iBTRELEVANT – AND FAIR

Tim McNamaraThe University of Melbourne

tfmcna@unimelb.edu.au

1 4C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

14

Aims of research on TOEFL iBT

Relevance

Fairness

1 5C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

15

TOEFL iBT is…

A communicative test

A specific purpose, tailored test

1 6C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

16

TOEFL iBT: Distinctive features

Covers all four skills - speaking now compulsoryAll test tasks represent those facing international students (no general components; all academic)Integration of skills on some tasks in speaking and writing

1 7C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

17

The assessment argument

Claims about test takers(inferences)

viaAssessment argument

(relevance of data; its value as evidence)

Observations(Assessment

data)

1 8C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

18

Research on TOEFL iBTSkill-based research teams

Developed Framework documents for each skill

Developed research priorities

1 9C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

19

Examples of research:

(1) Relevance of test content

(2) Test tasks

(3) Scoring

2 0C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

20

(1) Relevance of test contentDoes the test provide a basis for us to draw inferences about likely performance in the target situation?

Is there a match between the content of the test and the demands of the target situation?

2 1C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

21

Relevance (continued)

Research on communicative demands facing international students

Development of computerized corpus of spoken and written academic language

2 2C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

22

(2) Test tasksWhat is special about speaking in an academic context?

Should it be scored differently from general conversational ability?

2 3C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

23

Test tasks (cont.)

Investigated judges’ comments on performances on two types of speaking task:

Independent tasks - no integration of prior materialIntegrated tasks - require integration of material presented via listening or reading

2 4C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

24

Test tasks (cont.)

Judges are oriented toContent of responseOrganization of responseHow adequately response reflects source

These new tasks require new criteriaSuch criteria are familiar in writing assessments

2 5C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

25

(3) ScoringHow much of the candidate's score is a reflection of the particular questions and texts that were used in the test? Would another judge looking at the candidate's performance in writing or speaking give the same score to the candidate?

2 6C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

26

Reliability and score agreement

41%59%0.77

10%90%0.95

64%36%0.6051%49%0.70

36%64%0.8019%81%0.90

CHANCE OF GETTING

DIFFERENT SCORE

CHANCE OF GETTING SAME

SCORE

RELIABILITY

2 7C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

27

Reliability of TOEFL iBT

0.86Speaking

0.89Writing (independent item)

0.92Writing (integrated item)0.88Reading

0.88Listening

0.95Test as a whole

ReliabilitySection of test

2 8C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

The English test that disappeared

Jeffrey Smart Swinburne Universityof Technology

Friday 14 October

2 9C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

29

Remember TOEFL?Widely accepted in the 1990s

We all knew the difference between the PBT, the CBT, a TWE and an ER

We could rattle off IELTS and TOEFL entry scores at the drop of a hat

Then came ESOS and changes to the international student visa program…

3 0C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

30

English for visasVisa requirements defined IELTS as the only acceptable

test for visa applicants in some assessment levels

TOEFL still acceptable - but for a lesser number of assessment levels

IELTS becomes more readily available to students applying for study in Australia

How did your institution respond to the changes?

3 1C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

31

Does your institution (still) accept TOEFL?Three types of institutional response to accepting TOEFL

following the introduction of Assessment Levels and new immigration rules in 2001:

1. Delete TOEFL as a valid entry tool - corporate memory forgets how to use it

2. Continue accepting TOEFL scores for academic entry purposes

3. A combination of both within the one institution

Where is your institution?

Swinburne was a Type 1 institution until 8 weeks ago.

3 2C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

32

Did TOEFL really (need to) disappear?

Hong Kong SAR

Thailand

Vietnam

India

China

Indonesia

Malaysia

Singapore

TNEP574573CountryTOEFL is still OK for visa purposes (if your institution accepts it):

3 3C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

33

The data that went missingHow can we measure the effectiveness of TOEFL as a

diagnostic tool?

No history of students admitted on TOEFL score

Correlating entry score with academic performance now impossible

Tracking studies are not available at the institutional level?

No data indicating the (likely) popularity of TOEFL as an entry test (ie how many international offices have kept records of the number of applicants wanting to submit TOEFL instead of IELTS?)

3 4C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

34

Can we start again?Swinburne recently re-introduced TOEFL as a valid entry

test

We are preparing for the introduction of the IBT

We need to educate our faculties staff, our admissions team, and work with our English Language Centre

We need to update brochures and the website

We need to educate our agents, and applicants

And some data would be helpful! TOEFL score + academic performance

3 5C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

35

My scoresReading: 95%

Listening: 97%

Writing: ?%

Speaking: ?%

3 6C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

Finding TOEFL

Melissa BanksBanks Consulting

3 7C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

37

The factsWhy do Australian institutions need to remain

informed of developments in TOEFL?

Test taker numbers increased 20% in the last 12 monthsVolumes are up in Australia’s key recruitment markets especially China, India, Korea, Japan, Taiwan, Hong Kong, Singapore, Malaysia and ThailandTest takers are increasingly seeking study options outside of the US

3 8C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

38

0%

5%

10%

15%

20%

25%

FY00 FY01 FY02 FY03 FY04

Score reports sent to institutionsoutside US over 5 years

3 9C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

39

The facts

Why do Australian institutions need to remain informed of developments in TOEFL?

The TOEFL test is the most widely recognised test in the worldAccess to TOEFL will be significantly enhanced via iBT TOEFLTOEFL is already an accepted indicator of English language proficiency by Australian institutions

4 0C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

40

The facts

Why do Australian institutions need to remain informed of developments in TOEFL?

The vast majority of TOEFL test takers will undertake tertiary studies

4 1C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

41

Company RequirementProfessional Licensure

Other

No Reason

Admission to UndergraduateStudies

Admission to graduate

studies

4%4.5%

8%

11%

28.5%

44%

Test taker purpose 03- 04

4 2C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

42

The facts

Why do Australian institutions need to remain informed of developments in TOEFL?

Discussions with DIMIA for CALS 3 and 4

You are likely to see more students applying to your institution for admission with a TOEFL score

4 3C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

43

iBT TOEFL

A new test delivered via new technologies with enhanced services

The iBT TOEFL has a number of features that makes it more convenient and attractive to prospective students

4 4C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

44

iBT TOEFL

Test centres will multiply significantly making TOEFL more accessible in more countries than ever beforeScores are reported to test takers online and on paper The test is a 4 hour test which is completed in a single sittingPerformance feedback will be provided to students on their score report

4 5C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

45

iBT TOEFL

The test is interactive making good use of the technologyA free practice test is available with instant scoresStudents can register online or via phone or mailStudents can join the TOEFL online community

4 6C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

46

Opportunities

Opportunities exist for Australian education providers to engage with TOEFL test takers before choices are made regarding future study options

Over 80% of international students predetermine their destination of choice

How can we engage with prospective students and encourage the inclusion of Australia as a study destination in their consideration set?

4 7C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

47

Opportunities

TOEFL accessA joint initiative between Hobsons and ETS

Provides Australian institutions with unique targeted access to the growing annual pool of international TOEFL test takersDelivers targeted relevant information to each individual test taker driving them to the ACCESS websiteGenerates an interactive online student community that continues to engage the test taker through their education decision making process

4 8C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

48

ACCESS Websites

ACCESS Email Communication

ACCESS Magazine and digital version

ACCESS TOEFL Tips Guide

Bespoke TOEFL Research

ACCESS wall planner/posters

Access TOEFL

4 9C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

49

Next steps

1. Attend a workshop- a series of free iBT TOEFL workshops will be held in a major city near you

Brisbane – 17 OctoberSydney – 19 OctoberMelbourne – 21 OctoberAdelaide – 24 OctoberPerth – 26 October

5 0C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

50

Next steps

2 Ensure you are registered to receive official TOEFL scores from ETS

3 Ensure your institution is prepared for the new test scores

4 Register for updates www.ets.org/toefl5 Consider the TOEFL Access

5 1C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

51

Next steps

6 Consider becoming a test centreAttract new studentsAdd additional services to your current portfolioReceive discounts on ETS products in futureLeverage off the TOEFL brand

Go to www.ets.org/ecep for information and to applyUse reference ID AIEC on your applicationFor information write ecep@ets.org

7 Join the mailing list- leave your card

5 2C o p y r i g h t © 2 0 0 5 E d u c a t i o n a l T e s t i n g S e r v i c e

52

E-mail: toeflnews@ets.org

Web: www.ets.org/toefl

TOEFL iBT Score User Help Line:+1-609-683-2008

Eileen Tyson: etyson@ets.org

For More Information