Reading Assessment

WYOMING LEGISLATIVE SERVICE OFFICE 09/20/2011 Denver, CO NLPES Conference 2011

Introduction

09/20/2011 Denver, CO NLPES Conference

Wyoming Legislative Service Office Katja Vermehren

Why Reading Assessment? State Senator proposed legislation (SF 131) in 2009 for

the Management Audit Committee to conduct an audit on the Reading Assessment & Intervention Program (which did not pass).

The Senator later went through the Management Audit Committee process.

Concern that the Program was not being carried out pursuant to statute.

3

Legal Requirements Wyoming has 48 independent school districts, which are

responsible for the implementation of W.S. 21-3-40.

W.S. 21-3-401 creates the Reading Assessment and Intervention Program.

4

W.S. 21-3-401 W.S. 21-3-401 (a):

“Each school district shall design and implement a reading screening program that measures student reading progress in grades one (1) and two (2). The screening program shall be designed by each school district aligned to the statewide educational program standards and shall specifically screen for student performance in reading at grades one (1) and two (2).”

5

Districts’ Responsibilities

Assessment of students (1st and 2nd grades) Placement of students on Individualized Reading Plans

(IRPs) who are not reading proficient Annual reporting towards meeting the 85% goal

(statutory requirement) of identified students reading proficiently

Longitudinal reporting for students 1st through 4th

grades re: reading proficiency Reporting reasons why the 85% goal is not met Financial reporting (presentation will focus on the last

four bullet points)

6

Background Importance of reading (pedagogic argument)

Research has shown that reading is the “foundation” for success later in life.

There was a federal push in the 90s to increase early reading skills after research showed that especially American children living in poverty demonstrated very low reading skills.

7

Background (cont.) Wyoming adopted reading

assessments in 2001 because of federal push. The Program was funded outside of

the Model from 2001 through 2005. $3.43 million for BFY 2001 (GF) $6.7 million for BFY 2003 (GF) $7.6 million for BFY 2005 (School

Foundation Program Account)

8

Background (cont.) The Program was funded within the Model from 2006 to

present via “at-risk” resources (e.g. pupil support, tutor funding, English Language Learner teacher funding, and ½ day to full day Kindergarten).

At-risk funding is available for districts to use for reading assessment programs. SY 2006 ($56.4 million) SY 2007 ($58.7 million) SY 2008 ($63.1 million) SY 2009 ($66.8 million)

9

Background (cont.) Other funding outside of the Model is also

available for districts to use. Special Education Reimbursement to districts Summer School Extended Day funding

SY 2006: $125.4 million SY 2007: $153.6 million SY 2008: $167.4 million SY 2009:$178.5 million

10

Problem Areas

WDE (Wyoming Department of Education) misinterpreted applicable statute. Longitudinal reporting towards 85% goal was

missing.

11

Problem Areas (cont.) Financial reporting was inconsistent prior to 2005

and not required after Model recalibration. Longitudinal reporting had stopped in 2001 and

was started again after recalibration. There are difficulties associated with the latest

wave of research data due to compatibility issues of various screeners used by districts.

12

Problem Areas (cont.) Influence from districts via Data Advisory

Committee impacted reporting requirements. Data Advisory Committee has purely advisory

function to the Superintendent according to W.S. 21-2-203 (d)(i),(ii),(iii).

However, Committee members representing local school districts influenced WDE not to require financial reporting for the Program.

13

Financial Reporting

At the time of our audit, WDE did not track district expenditures related to the Reading Assessment and Intervention Program.

Financial data was only available from 2001-2005. The financial data reported during this period was

inconsistent and not complete.

14

Financial Reporting (cont.)

After recalibration, WDE has not required districts to report expenditure data with respect to the Reading Assessment and Intervention Program. However, programmatic data is supposed to be reported.

Practice was in violation of current statute.

15

Financial Reporting (cont.)

Lack of reporting created a void of expenditure information for WDE and the Legislature to use when making funding decisions for the Reading and Assessment Intervention Program and future recalibrations of the Model.

16

Financial Reporting Requirement in Statute

Prior to recalibration: W.S. 21-13-333 (d)

“Each district shall, in addition to reporting information required under W.S. 21-3-401, annually report to the department expenditures of amounts made available under this section for the prior school year.”

Repealed after Model recalibration in 2006.

17

Financial Reporting Requirement in Statute

(cont.) Language was repealed after recalibration in 2006

because the Legislature thought it to be superfluous.

After recalibration: W.S. 21-2-203 (a) already contains language that makes

financial reporting mandatory for WDE. The “department shall collect” data for the state’s school

finance system.

18

Financial Reporting Requirement in Statute

(cont.) “School finance system defined as ”all statutes related to

the terms and conditions under which funds from Wyoming sources are made available under Wyoming law to the public schools for school operations.”

19

Recommendation

WDE should specify all funds spent for the Program. Expenditure reporting does not take away local discretion.

20

Recommendation The Superintendent and WDE should receive

financial data from the districts again. WDE-626 and WDE-601 reports should be used for

reporting Similar to years prior to 2005, it would be necessary to

create additional account coding unique to the Program. For example, WDE should designate a funding group code to specify all funds spent for the Program; further separation of funds by function code and object code should also occur to ensure consistent reporting of expenditures for various purposes.

21

Recommendation The Superintendent and WDE should conduct

analysis of expenditures in order to assist the Legislature with future funding and recalibration efforts.

The Superintendent and WDE should work with the Department of Audit and Data Advisory Committee to determine the types of expenditure data that districts could submit for the RAI program.

22

Inconsistent Programmatic Reporting

W.S. 21-3-401 (b) assigns the responsibility of reporting to the school districts with respect to the Reading Assessment and Intervention Program.

Inconsistent reporting by districts or inconsistencies due to the usage of various screeners that are difficult to compare. Examples of various screeners (at least more than 25

different screeners are used regularly) e.g. DIBELS, DORA, MAP, ERDA, TPRI, AIMSWEB etc.

23

Inconsistent Programmatic Reporting (cont.)

LSO visited schools and collected survey information from all districts.

Our district surveys and visits showed reporting problems or inconsistencies.

Reading screenings and interventions occurred but the reporting of effectiveness was problematic.

24

Inconsistent Programmatic Reporting (cont.)

Inconsistent reporting causes false impression that districts are doing poorly moving students from IRPs (Individualized Reading Plans) towards reading proficiency.

25

Lack of Longitudinal Reporting for Reading Proficiency

None of the visited districts and schools showed evidence of longitudinal reporting. Thus, only snapshots of single moments during school years were reported.

Schools often did not know how to report accurate information to their district offices.

Districts had problems with their student information systems.

26

Lack of Longitudinal Reporting for Reading Proficiency (cont.)

Due to lack of guidance from districts, some schools were not sure which students to report as not proficient.

WDE officials stated that technology at the state and district levels precluded accurate reporting.

27

Inconsistent Direction from WDE

Since passage of W.S. 21-3-401 in 2001, WDE has experienced difficulty administering a statutorily created program that grants local responsibility to districts.

Shift from categorical funding prior to recalibration to hybrid funding through at-risk resources

28

Inconsistent Direction from WDE (cont.)

WDE Focus shift: 2001 to 2005 WDE assisted districts with establishment

of early literacy programs Post 2005 assisted districts with reporting requirements

of W.S. 21-3-401(b)

29

Inconsistent Direction from WDE (cont.)

Lateness in reporting caused inconsistency Confusion about 85% requirement (WDE thought it

referred to all students not just students that had been placed on an IRP). Led to reporting the number of any at-risk students, as opposed to

those who are not reading proficient. Caused districts to not report to WDE or to report as neither “yes”

or “no” because data was incorrect or to report 0% as becoming proficient (Natrona County).

Created false positives with respect to the number of students placed on IRPs who were not reading proficient.

30

Examples

Willard Reported seventeen 4th graders as neither “yes” or

“no.” However, we found that ten or 59% of students were actually

reading proficient.

31

Examples (cont.) Grant

For SY 2006 through 2008, 56 students were placed on an IRP as not being reading proficient. However, we found that 5 or 9% were

actually reading proficient.

32

Examples (cont.) Different definition of the

meaning of “proficiency.” Some districts such as Platte

County #1 reported only the most at-risk students or those performing at the lowest levels.

Caused inconsistency with respect to the 85% requirement.

33

WISER IDs

District reporting process has noticeably improved with the introduction of the Wyoming Integrated Statewide Education Data System (WISER IDs which has been used starting in 2006). This system is similar to systems used nationally (longitudinal data systems). It helps to track students throughout the school career even if they move to schools in other districts.

34

WISER IDs (cont.) Usage of WISER IDs has allowed districts and

WDE to track individual student progress with respect to the 85% statutory goal.

Due to reporting inconsistencies of the districts, data that has been used to start a longitudinal analysis again might be skewed. It creates the impression that the districts performed

poorly in moving students on IRPs to proficiency.

35

Recommendation

The Superintendent and WDE should explore reestablishing requirements for school districts to report programmatic data for the Reading Assessment and Intervention Program.

36

Recommendation WDE should collect information about why the

85% statutory goal of reading proficient student has not been reached (pursuant to W.S. 21-3-401 (b)).

WDE should pay attention to deficiencies in reporting due to the use of multiple screeners on the district level.

37

Management Audit Committee Action

Management Audit Committee wrote a letter to Superintendent reemphasizing the urgency of compliance with recommendations by Legislative Service Office.

38

WDE Response

Agency acknowledged that there have been inconsistencies in the definition of reading proficiency and in the guidance given for reporting data. WDE implemented changes accordingly.

WDE further agreed that it had difficulties monitoring the implementation of statutory requirements.

39

WDE Response (cont.) Financial reporting is mandatory and is now

enforced.

The Data Advisory Committee was required to rotate members.

40

WDE Recent Actions

Recommendation that WDE conduct an analysis of expenditure and programmatic data for the Program.

Action: WDE had Education Northwest complete the 2009-2010

Wyoming Early Literacy Report. Analysis of 2009-2010 reading assessment and

intervention expenditure data was completed by the WDE Finance Division.

41

WDE Recent Actions (cont.)

Recommendation to revise the WDE-626 Early Literacy Data Collection reporting template.

Action: MAP and DIBELS are used by all

districts.

42

WDE Recent Actions (cont.)

Superintendent Memo #2011-055 outlined the purpose and process for reporting on the WDE -626. This was distributed to all Wyoming school district superintendents.

Training was offered to help ensure accurate reporting on the WDE 2010-2011 WDE-626. WDE staff provided information and a technical run-through of the reporting template and process. Information regarding resources and technical support was provided.

43

Conclusion

Questions? http://legisweb.state.wy.us/progeval/REPORTS/20

10/DOE/RAI%20Report.pdf

44