dual language program evaluation...john gatta is president and chief analytics architect of ecra. in...

Community Unit School District 308

EL/Dual Language

Program Evaluation

December 2014

EL/Dual Language Program Evaluation

Ecra Group, Inc. 1

ECRA Group, Inc.

ECRA Group, Inc. (ECRA) is a premier national consulting firm and established leader in the areas of educational

assessment, research, and strategic planning. ECRA has provided support to school leaders for over 35 years. By applying

research-based methodologies, time-tested principles, and sophisticated analytic techniques, ECRA is able to meet the

unique needs of clients and provide a course of action for meaningful and lasting change.

Ann Paternostro, Project Role: Lead Consultant

Ann Paternostro, an associate with ECRA, has over 25 years of experience in the field of English Language Learning as

both a teacher (K-12 and adult) and an administrator in New Jersey, Ohio, and Illinois school districts. She has designed

and implemented ELL programs for school districts, served as a mentor to new ELL teachers and program directors, and

presented on topics related to ESL and Bilingual Education at the local, county, and state levels. She currently serves as an

adjunct faculty member at three universities in the Chicago-area, where she teaches graduate courses required for an

Illinois teaching endorsement in English as a Second Language or Bilingual Education.

John Gatta, Ph.D., Project Role: Technical Advisor

John Gatta is President and Chief Analytics Architect of ECRA. In addition to his responsibilities at ECRA, Dr. Gatta is an

Assistant Professor of Statistics and Predictive Analytics at Northwestern University, where he also serves as Director of

Research. His unique blend of academic, technical, management, and leadership experience gives him keen insights into

how organizations can function more strategically for competitive advantage and quality improvement. As President of

ECRA, he also serves as chief architect of ECRA's analytic infrastructure related to data structures, data warehousing,

statistical algorithms, and reporting. Dr. Gatta has been published and is a national speaker in both the education and

healthcare sectors.

James Fritts, Ph.D., Project Role: Financial Consultant

James Fritts, a senior associate with ECRA and Hazard, Young, Attea & Associates, has over 50 years of experience in

education and school financial management. Dr. Fritts taught graduate courses in educational leadership and finance at

Loyola University Chicago and currently teaches at Northeastern Illinois University. Dr. Fritts is the author of Essentials of

Illinois School Finance, a reference textbook published by the Illinois Association of School Boards (IASB), which is in its

sixth edition. He is also the senior editor of the fourth edition of Good School Maintenance, a manual of programs and

procedures for buildings, grounds, and equipment.


Ecra Group, Inc. 2

Table of Contents

ECRA Group, Inc. ......................................................................................................................................... 1

Executive Summary of Findings ................................................................................................................... 3

Introduction ................................................................................................................................................... 5

National Context ....................................................................................................................................... 5

State Context ............................................................................................................................................. 5

District Context ......................................................................................................................................... 6

Design and Methodology .............................................................................................................................. 7

Evaluation Questions ................................................................................................................................ 7

Data Sources ............................................................................................................................................. 7

Analysis .................................................................................................................................................... 8

Interviews and Focus Groups ................................................................................................................ 8

Financial Analysis ................................................................................................................................. 8

Achievement Analysis .......................................................................................................................... 8

Inference and Reporting ............................................................................................................................ 9

Results ......................................................................................................................................................... 10

Current State of the Program .................................................................................................................. 10

What are Students in the Dual Language Program Learning? ................................................................ 12

Student Selection ................................................................................................................................ 12

Learning Environment ........................................................................................................................ 14

Student Growth in English .................................................................................................................. 16

Student Growth in Math ...................................................................................................................... 17

Student Growth in Reading ................................................................................................................. 18

What does the Dual Language Program Cost? ....................................................................................... 20

Funding Sources .................................................................................................................................. 20

Recommendations ....................................................................................................................................... 21


Ecra Group, Inc. 3

Executive Summary of Findings

During the fall of the 2014-2015 school year, CUSD 308 (the “District”) collaborated with ECRA, a third-party research

firm, to evaluate the District’s Dual Language (DL) Program. This evaluation was designed to answer two questions:

What are students in the Dual Language Program learning?

What does the Dual Language Program cost?

The DL Program is a component of the District’s English Learners (EL) Program. The No Child Left Behind Act of 2001

(NCLB) and Illinois School and Administrative Code require school districts to provide high quality language programs for

EL students. High quality programs must assist EL students in developing and reaching English proficiency, attaining high

levels of academic achievement in core content areas, and meeting the same academic and achievement standards as all

other students are expected to meet. The District has chosen two programs to provide state-mandated bilingual education to

its Spanish speaking EL students: Dual Language and Transitional Bilingual Education (TBE).

While the primary focus of this evaluation is the District’s DL Program, other EL services and programs were studied in the

context of evaluating the DL Program.

Global Finding: The District’s EL Program, including the DL Program component, is out

of compliance, perpetuates inequities for EL students, and lacks consistent, quality

program design, implementation, and leadership.

Compliance Finding: Systemic problems with the District’s EL Program, including the DL Program, have resulted

in areas of non-compliance with state and federal regulations.

In the fall of 2011, the Illinois State Board of Education (ISBE) found the District’s EL Program to be out of compliance in

19 different areas. While the District has made an effort to remedy these findings, the program remains out of compliance

with Illinois School Code and Administrative Code in numerous areas. The EL Program also does not conform to all of the

requirements set forth under NCLB. Additional concern is raised by the District’s use of state and federal funds to educate

English speaking students in the DL Program when the funds are earmarked exclusively for EL students.


Ecra Group, Inc. 4

Equity Finding: EL students do not have equal access to educational opportunities and programs as non-EL

students.

The District EL Program, including the DL Program, does not have instructional resources and supports comparable to

mainstream program of instruction, thereby denying EL students a realistic chance of success and perpetuating the EL

achievement gap. Even within the EL Program, there is inequity between the various program models, as students in the

TBE Program do not have access to the same instructional resources as students in the DL Program. There is further

inequity in facilities, as the DL Program has been consistently housed for the past five years in one building, while

different TBE services have been located in different buildings. As a result, TBE students might change schools multiple

times. TBE families may have children attending different schools, while DL families’ children attend the same school.

Additionally, there are inequities in access to the DL Program. The program is not available to all Spanish speaking

students who qualify for TBE, and participation, by design, is limited to two percent of students in the District. This

exclusivity of participation in the DL Program has resulted in its being viewed by the community as a selective program for

monolingual English students rather than as a component of the EL Program developed to provide mandated bilingual

education to Spanish speaking EL students.

Design and Implementation Finding: The entire EL Program, including the DL Program, lacks a cohesive, quality

program design and infrastructure.

The EL Program suffers from a high teacher and administrator turnover rate. There have been seven different EL Program

administrators over the last ten years, many of whom did not have the appropriate credentials, background, or experience to

lead the EL Program. As a result, the program suffers from widespread misperceptions, a lack of vision for the program,

and inconsistent implementation. The program also lacks systemic policies, procedures, and guidelines as well as

standards-based curricula, adequate resources, and professional development for District staff.

Student Outcomes Finding: EL students in the DL Program achieve higher growth in English than EL students in

other programs; however, students in the DL Program progress slower in math than non-DL counterparts.

DL students achieve faster growth in English than non-DL students on ACCESS, a test designed to assess English

language acquisition among EL students. While Math growth is significantly slower for students in the DL Program,

reading growth for students in the DL Program is comparable to students who are not in the DL Program. The expected

outcomes for students in the DL Program are that they will attain high levels of academic achievement and become

bilingual and biliterate. However, there is no assessment in place to assess Spanish language development or Spanish

literacy, so there is no evidence to determine if students are becoming bilingual and biliterate.

Finance Finding: Cost variances between the DL Program and other District programs are negligible.

While the cost of implementing the DL Program, in its current state, is less than the general education program, the

program serves few students and does not provide some of the instructional support infrastructure of other District

programs. In order to achieve expected outcomes, considerable investments must be made in the DL Program to meet its

long-term goals.


Ecra Group, Inc. 5

Introduction

National Context EL students comprise approximately ten percent of the public school population in the United States. In the fifteen-year

period between the 1994-1995 and 2009-2010 school years, the EL student population grew nationally by more than sixty-

three percent, while the general school population grew by slightly more than four percent.1

The federal government has long recognized that students with limited English proficiency need specialized language

instruction programs. In 1968, Congress passed the Bilingual Education Act (BEA), which provided federal funds to

school districts in order to encourage the use of native language instruction along with instruction in English. The BEA

remained in effect until the passage of NCLB, when it was replaced by Title III, a formula-grant program that promotes

English acquisition and helps English-language learners meet challenging content standards.

Under Title III, states are required to develop standards for English Language Proficiency that are aligned to the state's

academic content standards. School districts are required to develop language instruction programs for EL students that are

tied to scientifically based research and demonstrated to be effective. These language instruction programs must ensure that

EL students develop English proficiency and meet the same academic content and achievement standards as non-EL

students. Authority is given to each state to determine which types of language instruction programs it will support. In the

2009-2010 school year, forty-three states offered both bilingual and English-only language programs.2 Bilingual program

models include Transitional Bilingual Education, Developmental Bilingual Education, and Dual Language; English-only

program models include Sheltered English Instruction, Content Based ESL, and ESL Pull-Out or Push–In.

State Context To meet federal requirements for educating EL students, Illinois School Code and Administrative Code require school

districts to provide either a Transitional Bilingual Education Program (TBE) or a Transitional Program of Instruction (TPI).

A TBE Program is required when there are twenty or more EL students of the same language group in an attendance

center. If an attendance center has nineteen or fewer EL students of the same language group, school districts may provide

a TPI Program. While the Illinois Administrative Code outlines the required components of TBE and TPI Programs, it

allows school districts to choose which TBE or TPI Program model(s) to implement. The most common program models

for EL students in 2011-2012, according to the Illinois State Board of Education, can be found in Table 1.

Table 1: Most Common Illinois TBE/TPI Program Models3

Program Model Percentage of Illinois

EL Students Enrolled

TBE

Transitional Bilingual Education 54%

Developmental Bilingual Education 3%

Dual Language 2%

TPI

Sheltered Instruction 13%

Content Based ESL 12%

Push-In /Pull-Out 4%


Ecra Group, Inc. 6

EL

TBE

Dual Language

Transitional Bilingual

(TBE)

TPI

Sheltered Instruction

(TPI)

ESL Resource

District Context The CUSD 308 EL Program is comprised of both a TBE and TPI Program. For TBE, the District has chosen Dual

Language and TBE as its program models. The District has chosen Sheltered Instruction, which it refers to as TPI, and ESL

Resource as its program models for TPI.

Figure 1: CUSD 308 EL Structure

The program models chosen for TBE have significantly divergent goals and intended outcomes for students. The Dual

Language Program continues instruction in the native language (Spanish) from kindergarten through eighth grade so

students can become bilingual and biliterate. In the Transitional Bilingual Education Program, the native language is

initially used for instruction while students are learning English; the goal is to transition students as quickly as possible into

mainstream English classrooms. The Dual Language Program is the focus of this evaluation.


Ecra Group, Inc. 7

Design and Methodology

Evaluation Questions The primary focus of this evaluation is the District’s Dual Language (DL) program. Other EL services and programs are

only studied in the context of evaluating the DL Program.

The District administration undertook a disciplined approach to evaluating their DL Program. During the fall of the

2014-2015 school year, the District collaborated with ECRA, a third-party research firm, to gather data through interviews

and focus groups, as well as analyze archival reports, program finances, and student achievement data.

The Dual Language Program Evaluation is designed to answer the following questions:

What are students in the Dual Language Program learning?

What does the Dual Language Program cost?

Data Sources Data were collected through a variety of means, including:

Interviews

Focus groups

Classroom observations

State reports

District documents

Nine interviews were conducted with appropriate District and building personnel, including current principals and assistant

principals of TBE and DL buildings. Focus groups were held with parents, students, teachers, and a group recommended

by the Board of Education. Stakeholders associated with the DL Program, stakeholders associated with TBE and TPI

Programs, and general education stakeholders in buildings hosting TBE and DL Programs were all heard from throughout

the focus group process. Representatives of the BPAC Advisory committee also met with the research team. In all, one

hundred seven stakeholders participated in interviews and focus groups.

Qualitative and quantitative data were also obtained through documentation provided by CUSD 308, including:

Archival reports

Assessment data

Financial budgets and reports

Meetings, phone calls, and e-mail correspondence


Ecra Group, Inc. 8

Analysis

Interviews and Focus Groups

In interviews, questions regarding the DL Program were asked of each interviewee, specific to each person’s familiarity

with the program and field of expertise. In focus groups, participants were asked to share their opinions on program

purpose, strengths, areas for improvement, and student learning. Interview and focus group data were coded for recurring

themes, and findings were used to inform the examination and synthesis of other data sources. Only findings found across

stakeholder groups or sources of information were presented.

Financial Analysis

Financial analyses were conducted to examine how much the DL Program costs and how the program is funded. Through a

review of the EL Program budget and finances, a comparative cost analysis of the DL Program was developed. Results were

reported in Table 16.

Achievement Analysis

Illinois State ACCESS Benchmark Model. Growth analyses were conducted to examine how much EL students learned, or

grew, between standardized assessments in English. Due to the small percentage of District students who took ACCESS

every year, EL growth was examined using ECRA’s Illinois State ACCESS Benchmark Model, which was built using

ACCESS data from multiple districts across Illinois between 2007 and 2013. The model was built to reflect typical student

growth in the state. To evaluate student growth, students’ actual spring 2014 ACCESS test scores were compared to the

expected values provided by the ACCESS Benchmark Model. Findings were disaggregated by program and native

language. Results were reported in Table 6 and Table 7.

Local Growth Model. Growth analyses were conducted to examine how much students learned, or grew, between

standardized assessments in reading and math. A local student growth model was built to provide a comparison within the

District. ISAT and MAP data between 2011 and 2014 were used to build a local growth model reflective of typical student

growth in the District. Through this model, each student with data from the previous school year was assigned a propensity

score based on his or her historical data. The propensity score indicated the expected achievement for that student during the

2013-2014 school year. To evaluate student growth, students’ actual spring 2014 ISAT and/or MAP test scores were

compared to the expected values provided by the prediction model. Findings were disaggregated by program, native

language, and classification. Results were reported in Table 8 through Table 15.


Ecra Group, Inc. 9

“While many statistical analyses may be carried out in a study, typically only a

subset is critical to the eventual results and interpretations. It is important to

report the results of analyses that are critical for interpretation of findings.”

Inference and Reporting ECRA follows the major principles of Standards for Reporting on Empirical Social Science Research, published by the

American Educational Research Association (AERA), when reporting research findings. AERA was founded in 1916 and is

considered a premier authority related to educational research, best practices, and standards for reporting research grounded

in the empirical traditions of the social sciences. The following are taken from Standards for Reporting on Empirical Social

Science Research.

AERA defines two overarching principles that underlie the development of reporting standards:

1. Reports of empirical research should be warranted; that is, adequate evidence should be provided to justify

the results and conclusions.

2. Reports of empirical research should be transparent; that is, reporting should make explicit the logic of

inquiry.

ECRA adheres to principle one by providing evidence in the form of data, statistics, and information from a variety of

analyses that support any claims made by the findings. All findings in this report carry with them evidence from various

sources used to infer the research finding. This is considered best practice by AERA, which states:

ECRA adheres to principle two by clearly articulating the process of data collection and analysis used during the process.

AERA states:

ECRA uses the process of triangulation, as described by AERA, to arrive at the findings in this report. Triangulation yields

accurate findings because it incorporates multiple methods and sources to verify results.

“It is the researcher’s responsibility to show the reader that the report can be

trusted… The warrant for the claims can be established through a variety of

procedures including triangulation or comparison of evidence from different

sources.”


Ecra Group, Inc. 10

EL

TBE/TPI Dual

Language

Results

Current State of the Program In the fall of 2011, ISBE conducted a monitoring review of the District’s EL Program and found the program to be out of

compliance. While the District has made an effort to remedy the majority of these findings, it remains out of compliance

with Illinois School Code and Administrative Code. Additionally, the District has failed to meet Annual Measureable

Achievement Objectives (AMAO) for six years resulting in a requirement to modify its EL Program model, curriculum,

and methods of instruction.

Based upon qualitative data collected through focus groups and interviews, it is apparent that parents, administrators, and

teachers do not have a clear understanding of the EL Program. Across focus groups, the DL Program is viewed as a

separate entity and not as a component of the TBE Program. The purpose of the DL Program is to provide state mandated

TBE services to Spanish speaking EL students. However, many stakeholders believe the program is designed primarily to

provide English speaking students the opportunity to learn Spanish and become bilingual.

The intended structure of the EL Program clearly shows that DL is a component of TBE (Refer to Figure 1). However, the

perception of the program across stakeholders is that the DL Program is separate from the EL Program, as seen in Figure 2.

Figure 2: Stakeholder Perceptions of EL & DL Program Structures

Documentation on the District website and in the English Language Learner Program Structure and Instructional

Framework helps to perpetuate this perception. Information about the DL Program is not found on the English Language

Learners Services website; the DL Program is listed under Elementary (K-5) separate from the EL Program. The English

Language Learner Program Structure and Instructional Framework manual contains a diagram showing the DL Program

as part of the EL Program but separate from the TBE and TPI Programs.

Figure 3: EL Structure in District Documentation

EL

TBE TPI

Dual Language


Ecra Group, Inc. 11

Misunderstandings about the DL Program are voiced in focus groups. According to DL teachers, the 2014-2015 school

year is the tenth year of the program, and people still do not understand it. General education teachers in the buildings that

house the DL Program report they are either not informed or misinformed about the program. General education teachers

claim the purpose of the program is not clear and that “no one knows what it is.”

Non-DL parents claim native Spanish speakers are taking away spots in the DL Program that could be going to native

English speakers, and non-DL parents believe that having native Spanish speakers in the program defeats the purpose of

DL since native Spanish speakers already know Spanish. DL and non-DL teachers state that DL is a Spanish immersion

program for native English speakers to become bilingual. Teachers believe that the DL Program is for native English

speakers and native Spanish speakers who are not EL. Teachers state that native Spanish speakers who are EL students

should be required to participate in the same lottery system as non-EL students.

There is a perceived status difference between the DL Program and the TBE Program. Parents report that DL is valued and

considered higher-level learning, while stakeholders look down upon TBE. TBE parents state that the TBE Program is “not

cared for” in the same way as the DL Program. Participants across focus groups report that DL is perceived to be exclusive

or elitist, and TBE is viewed as being “lower class” or a “last resort” for the families who cannot get into DL. Participants

at every focus group report on the disparity between DL and TBE resources.

Misunderstandings and misperceptions extend to the Bilingual Parent Advisory Council (BPAC). According to both

Illinois School Code and Administrative Code, school districts with TBE Programs are required to establish a BPAC

consisting of the parents of legal guardians of TBE students, TBE teachers, counselors, and community leaders. However,

the majority of BPAC members must be parents or legal guardians of TBE students.

While the District has a BPAC, the majority of members are not parents of TBE students. The District is cited by ISBE for

being out of compliance with school code because the majority of BPAC members are parents whose children are former

EL students or who were never EL students. While DL parents state in focus groups that the purpose of the BPAC is to

advocate for students in the EL Program, building administrators, non-DL parents, and non-DL teachers overwhelmingly

view the BPAC as a DL parent group primarily run by native English speaking parents to advocate for the DL Program.

Focus group members refer to BPAC as the “watchdog of the DL Program,” and others claim that BPAC is “hijacked” to

serve the needs of English speakers. TBE parents and teachers state that TBE parents are not involved in BPAC because

they believe it is a group for DL parents. Focus group participants voice concerns that TBE parents need a voice that BPAC

is not providing.


Ecra Group, Inc. 12

What are Students in the Dual Language Program Learning?

Student Selection

There are nine hundred thirteen EL students in the District during the 2014-2015 school year. Five hundred seventeen

students receive TBE services, one hundred fourteen of who receive services through the DL Program. Two hundred

ninety-eight students receive services through the TPI Program, and ninety-eight students refuse EL services.

DL began in the 2005-2006 school year. The program consists of two classes per grade level from Kindergarten through

eighth grade. There is no structured DL Program at the high school level. The kindergarten through fifth grade DL Program

is housed at Hunt Club Elementary, and the sixth through eighth grade DL Program is housed at Plank Junior High.

As seen in Table 2, the DL Program serves three hundred sixty-five students in kindergarten through eighth grade. One

hundred twenty of the students in the DL Program are EL students. Ten percent of students in the DL Program have IEPs,

and forty-eight percent of DL students qualify for free/reduced lunch.

Table 2: 2014-2015 DL Program Student Characteristics

Native Language Total Number

EL Students

Number

with IEPs

Number

Free/Reduced Lunch

English 116 0 7 12

Spanish 241 114 27 160

Other 8 6 2 2

For comparative purposes, demographic information is provided in Table 3 on the remaining students in the TBE and TPI

Programs. Among TPI students and TBE students not in DL, twenty-one percent of students have IEPs, and fifty percent of

students qualify for free/reduced lunch. There is an unusually high percentage of EL students with IEPs, especially in the

TPI Program.

Table 3: 2014-2015 TBE and TPI Student Characteristics

Program Total Number

with IEPs

Number

Free/Reduced Lunch

TBE (not in DL) 403 66 206

TPI 298 83 147

According to the District website, students are admitted to the DL Program through a lottery. However, siblings of students

already in the DL Program are automatically admitted into the DL Program; they do not participate in the lottery. All other

interested parents must attend a mandatory informational meeting and submit a DL Program application. Applicants are

divided into two categories – native English speakers and native Spanish speakers. Native Spanish speaking applicants are

screened for English Language Proficiency to determine whether students are Limited English Proficient. Native Spanish

speakers who are found to be English Proficient are placed in the native English speaker lottery. Therefore, some students

in the DL Program who are included in the English speaker lottery are native Spanish speakers who are proficient in

English.


Ecra Group, Inc. 13

Each year there are approximately forty-eight spaces available for kindergarten students in the DL Program. District

documentation states that fifty percent of the students in each DL classroom are native Spanish speakers and fifty percent

of the students are native English speakers. Therefore, in each year’s lottery there are twenty-four spaces allocated for each

native Spanish speakers and native English speakers. After students with siblings already in the program are admitted, the

actual number of spaces for students participating in the lottery is reduced. Table 4 details the number of spots available

during the 2014-2015 lottery. Sixteen native Spanish speakers, forty native English speakers, and six students with a native

language other than English or Spanish participated in the 2014-2015 lottery.

Table 4: 2014-2015 Dual Language Spots Available

Native

Language

Number of

DL Spots

Number of

Siblings Admitted

Number of

DL Spots After Siblings

Spanish 24 11 13

English 24 7 17

Focus group participants report the lottery is not conducted in a consistent manner, and there is no transparency in the

process. Several participants state Spanish speaking parents are not informed about the DL Program, and their children are

placed in TBE without the opportunity to participate in the lottery. Stakeholders also observe exceptions to the lottery

process.

In reviewing DL lottery information and documentation, discrepancies are found to support stakeholder perceptions

regarding consistency and transparency in the lottery process. Conflicting policies on who is granted automatic admission

are found. The Bylaws of the Oswego 308 English Language Learners Parent Council state that both siblings of students

currently in the program and children of DL teachers and administrators are granted automatic entry into the program.

There is inconsistency between the final list of students who are admitted into the program and those whose names are

selected during the lottery. In addition, the District does not adhere to parent deadlines, and students are not selected from

the wait list as per the policy stated to parents. In addition, on the list of lottery applicants, students with Hunt Club as their

home school are circled and the number noted on the top of the page. Of the eleven Hunt Club students in the lottery, ten

are selected.

Despite misunderstandings about the DL Program and perceptions surrounding the selection process, focus group

participants agree there is a strong sense of community among DL students and parents. There is also a high level of parent

involvement, teacher commitment, and the opportunity for students to become bilingual and biliterate. Focus group

participants state that everyone should have equal access to the opportunity to learn a language not just a select few. There

is strong support across focus groups for expanding the opportunity to learn a foreign language to all students.


Ecra Group, Inc. 14

Learning Environment

According to the DL Program Brochure, the goals of the DL Program are to nurture multiculturalism via cross-cultural

learning, foster language mastery in English and Spanish, meet or exceed academic success in a bilingual education

program, provide challenging and engaging instruction, and support high expectations of all students in two languages. In

addition to these outcomes, focus group participants cite high levels of academic achievement, increased cognitive ability,

better job opportunities, placement into AP Spanish, and earning a Seal of Biliteracy on their high school diploma as

additional expected outcomes.

Evidence of literacy in English can be obtained through student scores on ISAT and MAP assessments. Evidence of

English language development is obtained through scores on ACCESS for ELLs. However, there are no assessments in

place to assess Spanish language development or Spanish literacy, so there is no evidence to determine if students are

becoming bilingual and biliterate.

In focus groups, DL parents share they believe that their children are achieving intended outcomes. Parents cite anecdotal

evidence such as a student getting an “A” on a paper written in Spanish or fifth grade students’ ability to perform their

Living History Museum in both English and Spanish. Parents share that the majority of DL students go into honors classes

in middle school, and Plank and Hunt Club have the highest test scores in the District.

Teachers tell a different story. According to elementary DL teachers and administrators, both native Spanish and native

English speaking students are struggling, and middle school teachers report huge gaps in both core content areas and in

Spanish language skills. Middle school teachers report that students are “all over the place” linguistically and academically.

Middle school DL teachers believe students are low in Spanish vocabulary and grammar skills, speak in the wrong verb

tenses, and do not know how to conjugate verbs. Of the inaugural DL class now in ninth grade, only two students qualify

for AP Spanish. Both native Spanish and native English speaking students in the middle school DL Program say they read

better in English. Native English speakers say that they write better in English, and native Spanish speakers report that they

write better in Spanish. While the native Spanish speakers are comfortable speaking English, many native English speakers

say they are not comfortable speaking Spanish. Middle school DL students claim many students do not want to be in the

program, and they disrupt the educational process for those in the DL Program who want to learn.

According to Plank Junior High staff, twelve of the forty-one sixth grade DL students are enrolled in honors classes. In

seventh grade, eleven of the twenty-six DL students are in honors classes, and in eighth grade, sixteen of the thirty-three

students are in honors classes. Plank staff verifies DL students are not held to the same honors placement criteria as general

education students. DL students are placed into sixth grade Honors Math with lower percentiles than other students. In

addition, when a DL Social Studies class was not offered to eighth grade students this year, parents were given the choice

of either placing their students in the regular or honors eighth grade social studies class without having to take a placement

test. Teachers report that the DL students placed into the eighth grade Honors Social Studies class have much lower

reading skills than the other students in the class.

Parents, teachers, students, and administrators across focus groups cite the lack of resources and support for both the DL

and the TBE Programs. Neither students in the DL or TBE Programs have consistent access to Bilingual Reading

Improvement teachers, Speech/Language Pathologists, Special Education teachers, Psychologists, or Social Workers. Lack

of materials is reported for both the DL and TBE Programs, but resources seem to be more of an issue for the TBE

Program. Teachers report both programs are implemented without forethought of the resources needed.


Ecra Group, Inc. 15

Illinois Administrative Code (23IL Adm. Code Section 228.30) requires instruction in Spanish Language Arts (in a TBE or

DL Program) to be aligned to the WIDA Spanish Language Arts Standards. Instruction in English as a Second Language

(in TBE, TPI, and DL Programs) must be aligned to the WIDA English Language Development Standards. However, there

is no curriculum for any of the District’s EL Programs - DL, TBE, or TPI. Teachers create their own curricula as they go.

There is no vertical or horizontal curriculum alignment in any of the EL Programs or between EL Programs and the general

curriculum. Nor is there time for teachers to collaborate. Teachers want to see equity between the programs for EL students

and those for general education students.

Lack of direction for the program is an area of concern across focus groups. According to HR records, in the last ten years

there have been seven different EL Program Directors, the majority of whom did not have the credentials required by ISBE

for an EL Program director position. According to parents and teachers, the high rate of turnover among program

administrators has resulted in miscommunication, a lack of vision for the program, and inconsistent guidelines, policies,

and procedures.

Staffing the DL and TBE Programs and the lack of professional development for teachers are reported as problematic by

parents, teachers, and administrators. Students in the TBE and DL Programs do not have access to teachers with the same

credentials as students in the general education classrooms (Refer to Table 5). Through focus groups and a review of

District documentation, it is found that teachers of EL students, whether they are EL teachers or general education teachers,

are not receiving the annual required professional development as outlined in Illinois Administrative Code (23IL Adm.

Code Section 228.35).

Table 5: Teacher Qualifications by Program

Percentage

Highly Qualified*

Percentage with a

Master’s Degree

General Education 100% 71%

DL Program 77% 69%

TBE Program 57% 43%

*Highly Qualified for DL and TBE teachers refers to teachers who hold a

Professional Educator License with the appropriate content/grade level and

Bilingual and/or ESL Endorsements.

While the DL Program has had a consistent home base at Hunt Club Elementary for the past five years, the same cannot be

said for the TBE Program. Because the District has historically housed only one type of TBE Program at an attendance

center, students in the TBE Program have had to change schools, sometimes annually, when their level of TBE service

changes. ISBE cites the District for this in 2011 stating that this program structure “confounds parental decisions” and

“could create numerous additional challenges to the students and families.”4 ISBE finds that “it has serious implications for

families whose children might be attending different schools, their ELL children might attend different schools from year

to year, or the families might be forced to make program decisions not based on their preferences and student ELL needs

but rather on distance and transportation needs.”5 In order to remediate this compliance issue, the District and ISBE agree

that the District will develop a plan to provide appropriate EL Programs and services in each attendance center by SY 15.

However, that transition plan has not been developed or put into place.


Ecra Group, Inc. 16

Student Growth in English

EL student growth in English on ACCESS is compared to statewide growth on ACCESS for EL students with comparable

historical achievement. Each deviation from zero indicates more (or less) than expected growth based on statewide data. A

negative growth score does not indicate a student is losing knowledge, but rather that the student did not learn as much

throughout the school year as his or her peers with the same historical achievement. Results are disaggregated by program

and language.

The difference between student growth scores is considered “expected” unless the average is statistically significant and

educationally relevant. Based on research, the effect size must have an absolute magnitude of 0.20 or greater to be

educationally relevant. The color key identifies the magnitude of the difference between DL students and various

comparison groups. Blue indicates significantly higher growth observed among DL students. Green indicates no significant

difference between groups. Yellow indicates significantly lower growth observed among DL students, and red indicates

significantly lower growth observed among DL students with a difference greater than or equal to 0.40.

As seen in Table 6, DL students achieve significantly higher growth compared to non-DL students on ACCESS.

Table 6: 2014 ACCESS State Benchmark Growth Summary

Dual Language versus Not Dual Language

Program Effective Sample Size Student Growth

(z-score)

Dual Language 84 0.26

Not Dual Language 373 0.03

Difference 0.23

When comparing native Spanish speakers in the DL Program to other native Spanish speakers, as seen in Table 7, students

in DL achieve significantly higher growth in English on ACCESS.

Table 7: 2014 ACCESS State Benchmark Growth Summary

Dual Language versus Not Dual Language among Spanish Speakers


(z-score)

Spanish Dual Language 83 0.26

Spanish Not Dual Language 267 -0.09

Difference 0.35

Higher than Expected

Growth ≥ 0.20

0.20 > Expected

Growth > -0.20

-0.20 ≥ Lower than Expected

Growth > -0.40

Unsatisfactory

Growth ≤ -0.40


Ecra Group, Inc. 17

Student Growth in Math

Student growth in math for all District students is compared to historical local growth in math. Each deviation from zero

indicates more (or less) than expected growth based on District-wide data. Results are disaggregated by program, language,

and classification. As seen in Table 8, DL students achieve significantly lower growth in math than students not in DL.

However, the difference varies by student language and classification.

Table 8: 2014 ISAT and MAP Math Local Growth Summary



(z-score)

Dual Language 214 -0.15

Not Dual Language 8,781 0.13

Difference -0.28

Native Spanish speakers in the DL Program who are not classified as EL achieve growth in math that is statistically similar

to comparable students not enrolled in DL, as seen in Table 9.


Dual Language versus Not Dual Language among Spanish Speakers not classified as EL


(z-score)

Spanish (Not EL) Dual Language 92 0.01

Spanish (Not EL) Not Dual Language 447 0.03

Difference -0.02

Native Spanish speakers in the DL Program who are classified as EL achieve significantly and substantially lower growth

in math than comparable students not enrolled in DL (Refer to Table 10).


Dual Language versus Not Dual Language among Spanish Speakers Classified as EL


(z-score)

Spanish (EL) Dual Language 45 -0.56

Spanish (EL) Not Dual Language 186 0.12

Difference -0.68


Growth ≥ 0.20

0.20 > Expected

Growth > -0.20


Growth > -0.40

Unsatisfactory

Growth ≤ -0.40


Ecra Group, Inc. 18

As seen in Table 11, native English speakers enrolled in the DL Program achieve significantly lower growth in math than

students not in DL.


Dual Language versus Not Dual Language among English Speakers


(z-score)

English Dual Language 77 -0.11

English Not Dual Language 7,716 0.12

Difference -0.23

Student Growth in Reading

Student growth in reading for all District students is compared to historical local growth in reading. Each deviation from

zero indicates more (or less) than expected growth based on District-wide data. Results are disaggregated by program,

language, and classification. As seen in Table 12 through Table 15, there are no substantial differences in reading growth

by program, native language, or classification.

Table 12: 2014 ISAT and MAP Reading Local Growth Summary



(z-score)

Dual Language 214 0.13

Not Dual Language 8,780 0.07

Difference 0.06


Dual Language versus Not Dual Language among Spanish Speakers not classified as EL


(z-score)

Spanish (Not EL) Dual Language 92 0.25

Spanish (Not EL) Not Dual Language 447 0.12

Difference 0.13


Growth ≥ 0.20

0.20 > Expected

Growth > -0.20


Growth > -0.40

Unsatisfactory

Growth ≤ -0.40


Ecra Group, Inc. 19


Dual Language versus Not Dual Language among Spanish Speakers Classified as EL


(z-score)

Spanish (EL) Dual Language 45 0.17

Spanish (EL) Not Dual Language 186 0.04

Difference 0.13


Dual Language versus Not Dual Language among English Speakers


(z-score)

English Dual Language 77 -0.04

English Not Dual Language 7,715 0.07

Difference -0.11


Growth ≥ 0.20

0.20 > Expected

Growth > -0.20


Growth > -0.40

Unsatisfactory

Growth ≤ -0.40


Ecra Group, Inc. 20

What does the Dual Language Program Cost? The total of K-8 DL instructional costs (salary, benefits, services, and materials) in the 2014-2015 school year is $895,345.

The teachers’ salaries and benefits total $865,755. Purchased services and materials are estimated at $29,590. The DL

Program has 13.3 Full Time Equivalent (FTE) teachers. The numbers of classes and teachers required for the K-5 DL

Program are the same as they would be for the same number of students in a general education classroom, as class

sizes are comparable in the DL Program.

While the DL Program provides full instruction for all students in kindergarten through fifth grade, the program provides

only partial instruction to students in sixth through eighth grade. The DL Program has one teacher per section (two sections

per grade) for kindergarten through fifth grade. Only 1.3 teachers are assigned to sixth through eighth grade students, as

middle school students receive some of their instruction from non-DL teachers. Therefore, only K-5 per pupil costs are

examined in Table 16 to provide the most equivalent cost comparison between DL and general education programming.

Total K-5 DL expenditures include $612,553 for salaries, $167,493 for benefits, and $19,304 for services and materials for

a total of $799,350.

Table 16: K-5 per Pupil Cost Comparison (DL versus All Programs)

K-5 DL Program

2014-2015

All K-5 Programs

2013-2014

Salaries $2,412 $3,276

Benefits $659 $671

Services and Materials $76* $76

$3,147 $4,023

* The services and materials cost is imputed based on the services and materials cost from all

K-5 programs (1000 series accounts). Specific costs for the DL Program could not be obtained.

An average elementary school instructional cost of $4,023 per pupil is derived by dividing the 2013-2014 actual salary,

benefits, services, and materials expenditures for all K-5 programs by the K-5 enrollment of 7,872 pupils. The DL per pupil

instructional cost of $3,147 is 78 percent of the District-wide average cost to educate an elementary student. One possible

reason for the discrepancy in per pupil costs by program is that the all K-5 Programs expenditure amount includes the

cost of art, music, and PE, which DL students may also utilize.

Transportation costs are not included in the per pupil calculations. Transportation services at Hunt Club and Plank for DL

students require thirty-two routes daily. The cost of DL riders is $150,540 at Hunt Club and $100,493 at Plank for a total of

$251,033. Some students would require busing to their home schools regardless of participating in the DL Program.

Funding Sources

Most DL Program costs are locally funded. A portion of state and federal grant funds for language programs are used for

supplemental materials, professional development for all EL teachers (including DL teachers), three bilingual aides for the

DL Program, and iPads for all EL teachers (including DL teachers). Non-EL students in DL classes also benefit from these

expenditures. Outside revenue sources in the 2014-2015 budget include a Federal Title III Language Instruction Program

for Limited English Proficient Students (LIPLEPS) grant of $100,492 and a State Bilingual TBE/TPI grant of $355,962. As

noted above, a portion of these grant funds benefits the DL Program, which otherwise is locally funded.


Ecra Group, Inc. 21

Recommendations

Develop an action plan to bring the District EL Program into compliance with state and federal rules and

regulations and to rectify findings related to 2011 ISBE Monitor Visit.

As part of this plan, the District needs to create clear EL Program policies, procedures, and guidelines and share them with

all stakeholders via EL Program Handbooks and the District website. The District needs to restructure the BPAC so that it

is in compliance with Illinois Administrative Code and inclusive of all parents in the EL Program. The EL Program must

be staffed with teachers and administrators who are highly qualified and have knowledge and experience in English

Language Learning. A professional development plan for all teachers, administrators, and staff responsible for the

education of EL students must be developed that meets or exceeds what is mandated by Illinois Administrative Code. The

District must also ensure that federal and state funds earmarked for EL students is used solely for the benefit of those

students.

Restructure the EL Program to ensure equity and equal educational opportunities for all EL students.

The District has failed to meet AMAO for six years resulting in a requirement to modify its EL Program model,

curriculum, and methods of instruction. Structure the EL Program to be consistent with the District’s strategic plan. Ensure

that all EL students have access to high quality language instruction, facilities, teachers, materials, and resources to provide

EL students a realistic chance of success. Instructional infrastructure and resources must be comparable to those benefiting

students in the mainstream program of instruction. Review policies for placement into special programs such as Special

Education and Academically Talented Education to ensure that EL students are not being over-identified or under-

identified for these programs. Ensure that an effective process is in place for continual program planning, implementation,

and evaluation.

Create a structure for the EL Program models that is based on sound research and best practices that the District is

willing to fully support.

Ensure that there are program descriptions and goals that are reflective of each unique program. Educate all stakeholders

on the various program models. Develop a standards-based curriculum for each of the EL Program models, and provide all

the necessary resources and supports for teachers and students. Also, begin to administer nationally normed Spanish

language assessments to students in the TBE Program in order to assess Spanish language proficiency before students are

placed in a TBE Program and to assess that Spanish language proficiency of students in the program.


Ecra Group, Inc. 22

1U.S. Department of Education, Office of English Language Acquisition, Language Enhancement and

Academic Achievement for Limited English Proficient Students (OELA). (2011). The growing numbers

of English language learner students: 2009/10. The National Clearinghouse for English Language

Acquisition & Language Instruction Educational Programs (NCELA): Washington, DC. 2 U.S. Department of Education, Office of English Language Acquisition, Language Enhancement and

Academic Achievement for Limited English Proficient Students (OELA). (2013). The biennial report to

congress on the implementation of the title III state formula grant program: School years 2008-10.

Washington, DC.

3 Illinois State Board of Education, Data Analysis and Accountability Division, English Language

Learning Division. (2013). Bilingual education programs and English language learners in Illinois: SY

2012 (2011-2012 school year) statistical report.

4 Illinois State Board of Education (ISBE), Division of English Language Learning. (2011). Compliance

Review Report, 17.

5 Illinois State Board of Education (ISBE), Division of English Language Learning. (2013). Response to

Corrective Action Plan, 5.

dual language program evaluation...john gatta is president and chief analytics architect of ecra. in...

Documents