Improving Oral Proficiency in Chinese Using Simulated OPI Assessment

IntroductionOral proficiency in Mandarin is a key outcome of the Chinese program. This

skill is primarily assessed using the American Council on the Teaching of

Foreign Languages (ACTFL) Oral Proficiency Interview (OPI), which is

offered to graduating students free of charge. At this point, taking the test is

voluntary and only undergraduate Chinese major students with a high

degree of confidence have taken it. In order to enhance participation rates,

Chinese 411, a speaking capstone course offered in Fall 2013, was re-

engineered to prepare graduating Chinese majors for the OPI.

A rubric was developed for CHN 411 students to understand the

expectations of the OPI so that they can prepare for it. During this course,

students were given a series of advanced-level oral tasks, including self-

introductions, descriptions, narrations in different time frames that required

paragraph-length responses, and reports on current events as well as

various role-play scenarios. After each module, they produced recordings

of performances in simulated interviews similar to the OPI. Two assistants

outside of the class independently evaluated the recordings and provided

feedback. The result of this curricular change is that students have

expressed increased confidence in oral performance and are willing to

participate in the OPI.

Department of East Asian Languages and LiteraturesSong Jiang & Haidan Wang

Assessment Methods

Evaluation was of students’ recorded oral

performances on assigned tasks

according to rubrics aligned with the

ACTFL OPI Advanced Level descriptions

Two trained, independent assistant raters

were used to increase the inter-rating

validity and reliability

A Mock ACTFL OPI was done to assess

students’ proficiency at the end of

semester

Application of

Results• Refine the syllabus of CHN 411 in

order to further enhance student

learning

• To split some contents of CHN 411

to CHN 412

• To re-engineer lower level

speaking courses, e.g. CHN 211

and CHN 212, so that they better

prepare students to achieve the

target proficiency

• To provide feedback for Chinese

Curriculum Committee reviewing

the oral proficiency program

requirements

• To suggest increasing oral

proficiency-related SLOs in

addition to the achievement tests

in core courses

• To share the results with similar

courses in JPN and KOR sections

Chinese BA Student Learning Outcomes for Oral Proficiency

Students engage in oral communication in Chinese in various social contexts in linguistically and culturally

appropriate ways.

Chinese BA Curriculum Map for Oral Proficiency

Assessment tools

to measure

outcomes

Oral achievement tests in core language courses

Listening achievement tests in core language courses

Oral Proficiency Test [ACTFL OPI] in the final semester

Required courses CHN 301 & 302: Third-Level Mandarin (I & II)

(or CHN 305: Third-Year Chinese for Professionals)

CHN 401 & 402: Fourth-Level Mandarin (1 & II)

(or CHN 405: Fourth-Year Chinese for Professionals)

Courses for

electives

CHN 411: Advanced Mandarin Conversation (I)

CHN 412: Advanced Mandarin Conversation (II)

CHN 491: Oral Proficiency through Chinese Films

Expected oral

proficiency level

Advanced level: Low, Mid, or High on the ACTFL proficiency scale

2 or 2+ level on the Interagency Language Roundtable (ILR scale)

Mock-OPI

scores are

shared

with each

student

1. Ability to comprehend spoken Chinese

and satisfy routine social demands and

limited work requirements

SLOs Assessed in CHN 411

Action Plan

Offer CHN 411 and CHN 412 every school year and

provide explicit oral proficiency training to Chinese BA

students and Chinese Flagship participants

Ensure a speaking section for every unit or thematic

topic in all third- and fourth-level language courses

Train Chinese teachers and future teachers on the

ACTFL OPI in order to facilitate instruction

Revise assessment rubrics to align more closely to the

ACTFL OPI Advanced level descriptions

Consider increasing task difficulty for high-performing

students

Conclusions

CHN 411 helped students become more

familiar with the OPI, and thus more

confident about real-life situations

Many Chinese BA students in this class

indicated that they would like to participate

in the OPI

3. Ability to give and understand

complicated, detailed, and extensive

directions, and make non-routine changes

in such situations as travel and

accommodation arrangements

Participation Rates and Results 2011-2013

2. Ability to converse using appropriate

vocabulary and complex linguistic

structures

4. Familiarity with the format of the

ACTFL OPI, and ability to participate in

most formal and informal interactions with

cohesive discourse

A Sample of RatingsNames

Task 1 Rating resultsExcerpts of comments

Rater 1 Rater 2 Result Task 2 Task 3 Task 4

AD DP DP DP MP MP MP Much improved, especially for fluency

CJ MP MP MP ME MP ME Very fluent, need to speak louder

CA ME ME ME MP MP ME Interesting narration, need better discourse connection

GA MP ME ME MP MP MP Fluent delivery; improve tones & intonations

HM MP MP MP ME ME ME Rich & detailed, fluent, watch for intonation

LV ME ME ME ME ME ME Naturally delivered, near native fluency

LA ME MP ME ME ME ME near native fluency; interesting content; well connected

LX MP MP MP MP MP MP Need to enrich the content with details

MT MP MP MP MP MP ME Well organized in structure; please reduce the HK & TW accent

NC MP ME ME ME ME ME Very native-like, extremely fluent, naturally delivered

The Results of Simulated OPIs as Part of Course GradesNames AA AD CJ CA FA HB HM LV LA LX MB ME NJ NC

Mock-OPI IM AL AL AM AL IH AL AH AH AM IM AM AH AH