THE GLASTONBURY FOREIGN LANGUAGE ASSESSMENT

INITIATIVE: FLAP 2008-2013

Presenters:

Rita Oleksak, Director of Foreign Languages, GPSBarbara Lindsey, University of Connecticut

Dan Conrad, ACTFLNECTFL CONFERENCE

BALTIMORE, MARYLANDApril 2, 2011

Receipt of Foreign Language Assessment Grant 2008-2013

Ambitious Objective 1:

Development of national assessment tools to improve the articulation of curriculum and instruction over time and from schools to colleges and universities

Receipt of Foreign Language Assessment Grant 2008-2013

Ambitious Objective 2:

Develop a process that is replicable across the nation and across levels for the collection of student work samples in an electronic portfolio system

Historical Significance of the Glastonbury Foreign Language

Program Visionary Leadership - 1956

Expert Administrators - 1957

Collaboration of Higher Education -1957

Funding from the National Defense Education Act - 1957

Historical Significance of the Glastonbury Foreign Language

Program Creation of Elementary Foreign

Language Program - 1957

Creation of Glastonbury Foreign Language Materials - 1958

Implementation of Russian Language Program - 1958

Construction of 1st Foreign Language Laboratory - 1959

Historical Significance of the Glastonbury Foreign Language

Program Continued Funding from the Sale of

Audio-Lingual Materials (Harcourt Brace/Ivanovich) 1959-1968

Training of All Staff in ACTFL Oral Proficiency Interviews - 1983

100% Local Funding for the Continuation of Foreign Language Program - 1983-2000

Grants Received for Standards-Based Articulation of Curriculum, Assessments, and Teacher Training - 2000-2009

OUR RUSSIAN STUDENTS ARE…

successful in their first(other) foreign language(s)

love learning about another culture enthusiastic learners aware that they have been afforded a

unique opportunity sometimes native speakers with or

without formal language training standouts in the college application

process

THE COLLEGE EXPERIENCE: 6 years of Russian…now what??

Colleges ponder where to place a student who has already had an extended sequence of Russian

Dilemma solved: start over!!

American Council of Teachers of Russian (ACTR) to the rescue

Pilot AP Russian Program initiated in 2004

OUR STUDENTS SPEAK QUIA survey to upper level students and

Glastonbury graduates to probe:Metacognitive skills/recognitionSuccessful study habits/skillsEfficacy of foreign language assessments Influence on their lives and advanced studySuccess in other courses due to knowledge

of another language/cultureEffect on cultural awareness and

acceptance of cultural diversity

MLA High School to CollegeArticulation Project

Glastonbury Public SchoolsUniversity of Connecticut 1998-1999

Measuring Student Preparedness

• Student attitudinal surveyPurpose: To determine if current articulation

effortsbetween Glastonbury and the University of

Connecticut are effectively meeting the needs of these

students• Traditional placement exam

Purpose: To establish a baseline cutoff score for incoming

students wishing to place in the first two years of language study at the university and to compare performance levels of Glastonbury students (high

school levels 3-6 in predominately high achievement classes) with University of Connecticut students

Two Vehicles Used

Findings from survey

• In both groups, only 19 percent continued to study a foreign language at UCONN

• Nearly two-thirds (60 percent) in each group were studying a foreign language to fulfill a requirement

• Lack of awareness (almost 75 percent) of facilities which support language learning such as the multimedia language center, foreign language residence floor, Linkage Through Language Program

Similarities

Findings from survey

• Students who began learning a foreign language prior to sixth grade were more likely to continue studying a language in college (Numbers in the survey not large enough to draw a firm conclusion)

• Number of students who began the study of a foreign language because of a requirement but continued due to personal interest was nearly three times higher among Glastonbury graduates (eighteen versus seven percent)

• None of the general population surveyed started a new language at college, two percent of Glastonbury students did

• A slightly higher percentage of Glastonbury graduates are majoring in languages (five percent) than those from the general population (two percent)

Differences

Placement exam findings• Less than twenty percent of the variance among

test scores for UCONN students could be explained by current enrollment, total number of years studied and outside language experience.

• Additional testing and continued collection of background information should explain more of the variance between test scores

• Lack of inclusion of background information during scoring for Glastonbury students precludes a comparison between the two groups of students

DOE National FL Survey Grant• 4 years: 2000-2004• Students• Teachers• Administrators• Self-reported perceptions

on various aspects of language study in the U.S.

College Administrators: Percentage of student population enrolled in foreign language courses(2003: n = 309)

0%

20%

40%

60%

80%

100%

0% - 25% 26% - 50% 51% - 75% 76% - 100% Don't know

College Administrators: Course offerings(2003: n = 309)

0%

20%

40%

60%

80%

100%

Standard Literature Native speakers Specific purposes Methods

Literature in translation

Conversation/Composition

Content-based Linguistics Other

College FL Students: Type of foreign language course in fall of 2002 (n = 309)

0%

20%

40%

60%

80%

100%

Beginner Intermediate Advanced Don't know

College FL Students: Type of foreign language course in spring of 2003 (n = 309)

0%

20%

40%

60%

80%

100%

Beginner Intermediate Advanced Don't know

College FL Students: Took placement exam to determine which foreign language class to take in college (2003: n = 895)

College Administrators: Placement of students who do not use foreign language placement exams(2003: n = 94)

0%

20%

40%

60%

80%

100%

HS grades # of years studied Student meets withFL department

Student meets withfreshman advisor

All No procedure Other

College Instructors: Perceptions of which aspects should be measured by a college foreign language placement exam* (2003: n = 304)

0%

20%

40%

60%

80%

100%

Speaking Reading Writing Listeningcomprehension

Cultural knowledge Grammar Vocabulary

* multiple responses accepted

2003: College Administrators: Foreign language skills measured on placement test (n = 215)

0%

20%

40%

60%

80%

100%

Speaking Reading Writing Listening comprehension Cultural knowledge

College Administrators: Satisfaction with placement procedure (2003: n = 309)

0%

20%

40%

60%

80%

100%

Very satisfied Somewhat satisfied Not too satisfied Not at all satisfied Don't know

College Instructors: High school foreign language classes have adequately prepared students for college-level classes (2003: n = 304)

0%

20%

40%

60%

80%

100%

Strongly agree Somewhat agree Somewhat disagree Strongly disagree Don't know

College FL Students: College foreign language class repeated material already mastered in high school(2003: n = 309)

0%

20%

40%

60%

80%

100%

Yes No

College FL Students: High school foreign language classes prepared them well for college level classes (2003: n = 309)

0%

20%

40%

60%

80%

100%

Strongly agree Somewhat agree Somewhat disagree Strongly disagree

College FL Students: College foreign language class required material not learned in high school to be prepared for class (2003: n = 309)

0%

20%

40%

60%

80%

100%

Yes No

High School Teachers: Main cause for some students to perform less well in college foreign language classes than in high school (2002: n = 189)

0%

20%

40%

60%

80%

100%

Teachingmethods of HS

Teachingmethods of

college

Placementmethod in

college

Assessmentsused in college

Assessmentsused in HS

IndividualStudent

Other Faster pace incollege (vol.)

College Instructors: Main cause for some students to perform less well in college foreign language classes than in high school (2003: n = 92)

0%

20%

40%

60%

80%

100%

Teachingmethods of HS

Teachingmethods of

college

Placementmethod in

college

Assessmentsused in college

Assessmentsused in HS

IndividualStudent

Other Faster pace incollege (vol.)

High School Teachers: Importance of high school to college foreign language articulation initiatives(2002: n = 401)

0%

20%

40%

60%

80%

100%

Very important Somewhat important Not too important Not important at all Don't know

College Instructors: Importance of high school to college foreign language articulation initiatives(2003: n = 304)

0%

20%

40%

60%

80%

100%

Very important Somewhat important Not too important Not important at all Don't know

High School Teachers: Articulation efforts could support a student’s transition from high school to college(2002: n = 401)

0%

20%

40%

60%

80%

100%

Strongly agree Somewhat agree Neither agree/disagree Somewhat disagree Strongly disagree Don't know

College Instructors: Articulation efforts could support a student’s transition from high school to college(2003: n = 304)

0%

20%

40%

60%

80%

100%

Strongly agree Somewhat agree Neither agree/disagree Somewhat disagree Strongly disagree Don't know

High School Teachers: Familiarity with any high school to college foreign language articulation efforts(2002: n = 401)

0%

20%

40%

60%

80%

100%

Very familiar Somewhat familiar Not too familiar Not familiar at all Don't know

College Instructors: Familiarity with any high school to college foreign language articulation efforts(2003: n = 304)

0%

20%

40%

60%

80%

100%

Very familiar Somewhat familiar Not too familiar Not familiar at all Don't know

High School Teachers: Involvement in any program to address high school to college foreign language articulation (2002: n = 401)

0%

20%

40%

60%

80%

100%

Yes No

College Instructors: Involvement in any program to address high school to college foreign language articulation(2003: n = 304)

0%

20%

40%

60%

80%

100%

Yes No

High School Teachers: If not already involved, interest in participation in high school to college foreign language articulation program(2002: n = 374)

0%

20%

40%

60%

80%

100%

Very intersted Somewhat interested Not too interested Not at all interested Don't know

College Instructors: If not already involved, interest in participation in high school to college foreign language articulation program (2003: n = 221)

0%

20%

40%

60%

80%

100%

Very intersted Somewhat interested Not too interested Not at all interested Don't know

High School Teachers: Frequency of sharing information about foreign language programs between high school teachers and college instructors (2002: n = 399)

0%

20%

40%

60%

80%

100%

Never Once Twice More than twice Don't know

HS teacher shared

College instructorshared

College Instructors: Frequency of sharing information about foreign language programs between high school teachers and college instructors (2003: n = 304)

0%

20%

40%

60%

80%

100%

Never Once Twice More than twice Don't know

HS teacher shared

College instructorshared

High School Teachers: Important components of high school to college foreign language articulation initiatives* (2002: n = 401)

0%

20%

40%

60%

80%

100%

Reciprocal visits Dialogue b/w HS &college

Collaboration on learningsequences

Collaboration onassessment & placement

Other

* multiple responses accepted

College Instructors: Important components of high school to college foreign language articulation initiatives* (2003: n = 304)

0%

20%

40%

60%

80%

100%

Reciprocal visits Dialogue b/w HS &college

Collaboration on learningsequences

Collaboration onassessment & placement

Other

* multiple responses accepted

College Administrators: Areas of collaboration with school/department of education for pre-service teachers (2003: n = 309)

0%

20%

40%

60%

80%

100%

Curriculum Proficiency Pre-serviceteaching

Study abroad Methodology All areas Other Nocollaboration

Don't know

Task Force MembersMarty Abbott, Director of Education, American Council on

the Teaching of Foreign LanguagesKelly Aceto, Associate Director of the Center for International

Business Education and Research (CIBER), University of Connecticut

Gilbert Andrada, Psychometrician, Connecticut State Department of Education

Carol Any, Ph.D., Associate Professor of Language and Culture Studies (Russian) Trinity College

Dana Bourgerie, Ph.D., Associate Professor of Chinese, Brigham Young University

Christine Brown, Assistant Superintendent for Curriculum and Instruction, Glastonbury Public Schools

Lynne Campbell, Teacher of Russian, Glastonbury Public Schools

Carol Chen-Lin, Ph.D., Teacher of Chinese, Choate Rosemary Hall, Wallingford, CT

Dorie Conlon, Teacher of Spanish, Glastonbury Public Schools

Task Force MembersDaniel Conrad, Principal Assessment Specialist, American

Council on the Teaching of Foreign Languages Hanadi Dayyeh, Arabic Language and Curriculum Specialist,

American School of BeirutMuhammad Eissa, Ph.D., Consulting in Arabic/Islamic

Studies, Lecturer of Arabic, University of ChicagoCarine Feyten, Ph.D., Professor, Foreign Language

Education, Miami University OhioWafa Hassan, Ph.D., Outreach Coordinator for the Arabic

Language Instruction Program, Michigan State UniversityBonnie Hoskins, Teacher of Spanish, Glastonbury Public

SchoolsMei Ju Hwang, Lead Chinese Teacher, Springfield Public

SchoolsHong Gang Jin, Ph.D., Director of Chinese, Hamilton

CollegeCatharine Keatley, Ph.D., National Capital Language

Resource Center, George Washington UniversityBarbara Lindsey, Director, Multimedia Language Center,

University of Connecticut

Task Force Members

Kevin McKenna, Ph.D., Professor of Russian Language, Literature and Culture in the German and Russian Department University of Vermont

Priscilla Meyer, Ph.D., Professor of Russian Language & Literature Russian, Wesleyan University

Rita Oleksak, Director of Foreign Languages, ELL, Glastonbury Public Schools

Mark Pearsall, Teacher of Latin and Ancient Greek, Glastonbury Public Schools

Ken Peterson, Internet Administrator, American Council of Teachers of Russian

Tony Smith, Technology Specialist Consultant, American Council on the Teaching of Foreign Languages

Roger Travis, Ph.D., Associate Professor of Classics, Department of Modern and Classical Languages, University of Connecticut

Manuela Wagner, Ph.D., Assistant Professor of Foreign Language Education, Department of Modern and Classical Languages, University of Connecticut

Kathleen Wang, Principal, Chinese Charter School, Springfield Public Schools

Tina Wu, Chinese Professor, Central Connecticut State University

Activities Accomplished to Date

Collaboration with ACTFL on the development of the AAPPL Test item

Collaboration with the University of Connecticut

Meetings of the Portfolio Task Force, November 2009, May 2010 and October 2010

Evidence efforts in place in Glastonbury to date:

• Common monthly performance checks, midterm and final exams

• Digital speaking samples• Video and audio blogs• Digital evidence from study

abroad• Evidence from distance learning• Cross-grade level student service

learning projects• Capstone projects

How the AAPPL became part of this project:• Worked from the AAPPL

Framework• Met to discuss curricula

within GPS• Created prototype• Created proof of concept• Finalized content and met

with GPS• Conducted a very small

scale tryout in early November

• Preparing to conduct small pilot in GPS and elsewhere in December

AAPPL: A history

• Indefinite postponement of NAEP FL led to formation of task force, grant

• Grant focused on development of a Framework describing a new, standards-based national assessment

• Result was AAPPL and an AAPPL prototype

• On-line, on-demand, standards-based, curriculum linked test of four skills across three modes

AAPPL: A history

• AAPPL became a component of the Glastonbury FLAP grant

• Started with Chinese• Piloted Interpersonal

Listening/Speaking 2008• Piloted all skills/modes in 2009• Began building the delivery

infrastructure in 2009/2010• Greatly refined the concept in

2010• Continued to pilot in

Glastonbury throughout 2010• Began development on Russian

and Arabic in 2010• Expect to be operational in

2011

AAPPL: The experience• Task-based role

play in which student is him- or herself (a student of a FL in best-practice classroom)

• Sits at the computer with headphone and microphone

• Is greeted by the FL teacher who provides (in English) an overview of tasks.

AAPPL: Response modes

• Create • Compose• Converse

AAPPL: Examples

• Russian• Arabic• Chinese