test assessment report by de barros june 2008

1

Developing a Strategic Student Assessment System in Seattle Public Schools:

Research, Stakeholder Priorities And Policy Analysis

Prepared by: Jessica de Barros

Independent Consultant

June 18, 2008

Developing a Strategic Student Assessment System in Seattle Public Schools June 18, 2008

1

Developing a Strategic Student Assessment System in Seattle Public Schools:

Research, Stakeholder Priorities and Policy Analysis

Executive Summary

Seattle Public Schools is in a unique and challenging position. Under new leadership, the mid-size urban district has approved a new strategic plan with the goal of dramatically improving student performance over the next five years.

Background

To meet the goals established in the strategic plan, the Research, Evaluation and Assessment Office is leading an effort on behalf of the Superintendent to develop a district-wide student assessment system. Seattle schools administer the Washington Assessment of Student Learning annually as required by state and federal law, yet school and district leaders need more frequent student performance data in order to meet its five-year goals. It is imperative for the district to know on a more frequent basis throughout each school year whether students are on track to make progress on state standards.

This report provides the Superintendent and other district leaders with a basis from which to select one or more assessments. First, the report describes the characteristics of high-quality assessment systems according to the research and identifies various purposes of assessment. Next, the report provides an inventory of assessments that are currently used in Seattle Public Schools and summarizes findings on outreach conducted to district stakeholders about their needs and priorities for assessment. Based on the literature and stakeholder findings, the report provides a gap analysis suggesting areas where the district could focus more deeply on assessment, and makes recommendations for the district to consider in selecting a district-wide assessment.

What This Report Provides

To provide context on how other districts have approached student assessment, the report describes and compares the assessment practices of five school districts in Washington State and five school districts outside of Washington State. Finally, the report describes six different assessment products used by other districts. The report does not recommend a specific assessment for the district, but does provide a framework for Seattle Public Schools to use in creating a strategic assessment system.

Seattle Public Schools currently administers a wide variety of assessments to meet a wide variety of purposes. The district should prioritize and narrow its purpose(s) of assessment in order to measure student performance toward clear targets. It is also necessary for the district to improve “assessment literacy,” or to train teachers and principals on how often to assess students, how to interpret the data and how to use the data to inform instruction. Data literacy currently varies across and within schools.

Highlights of Findings and Recommendations

In interviews and surveys, stakeholders identified three high-priority purposes of assessment for SPS: to drive instruction; to monitor student learning on an ongoing basis; and to diagnose special needs to place students in services. Stakeholders also identified three high-priority criteria that they believed the district should use in selecting an assessment strategy: provide data that are


2

immediately useful for instruction; be aligned to state-wide Grade Level Expectations; and, provide rich, detailed student data. Seattle Public Schools currently administers a great number of assessments that monitor student progress and inform instruction to some degree. However, the district administers fewer assessments that are immediately useful to instruction and provide rich, detailed student data. As the district moves forward to develop a strategic student assessment system, this report recommends it eliminate assessments in areas where they overlap in purpose, and enhance in the areas of district-wide reading benchmark assessments, guidelines for classroom-based formative assessments, special education assessment, and kindergarten readiness assessment. In addition, the district should increase access to data at all levels of the system in order to help all individuals understand the connection between student performance data and their daily work. In summary, this report is a starting point from which district leadership can determine the type of assessment system that will best meet its needs.

1

I. Introduction

Seattle Public Schools has approved a new strategic plan, Excellence for All (2008), with the goal of all students achieving. Within five years, the district aims to ensure that: 88 percent of third grade students meet or exceed reading standards; 80 percent of seventh grade students meet or exceed math standards; and, 75 percent of students graduate from high school in four years. These are dramatic increases from current achievement levels.

Background

The Strategic Plan maps out a plan to achieve these ambitious targets. Among the strategies the district seeks to use is developing assessment tools to consistently track student progress and using data to drive improvements. These assessments would be in addition to the statewide, annual Washington Assessment of Student Learning. It is imperative for the leaders of the district’s multiple functions (i.e., curriculum, instruction, assessment, professional development, principal leadership, policy-making) to operate as a system using data to reach the common goal of improved student achievement. Teachers, principals and administrators need common tools to measure progress toward these goals and to have access to the same types of information with which to make decisions at the classroom, building and district levels.

Seattle Public Schools has expressed interest in using formative and summative (or “benchmark”) assessments to inform instruction, monitor student and school performance, and make policy decisions at the school and district levels. The purpose of this report is to provide an objective research base that the Superintendent and her key staff can use to determine the most appropriate type of assessment(s) to implement in Seattle Public Schools classrooms.

Report Purpose and Outcome

To meet this purpose, the report will provide: • An in-depth description of the purposes of assessment, from a policy and organizational

management perspective; • An inventory of the types of assessments currently used in Seattle Public Schools; • An objective framework with which to evaluate the purposes, strengths and weaknesses of

current assessments based on stakeholder priorities; • Findings and recommendations tailored to Seattle Public Schools’ potential implementation of a

new assessment system; • Capsule descriptions of assessment strategies used by five other school districts in Washington

State and five school districts outside of Washington State; and, • A description of six assessment products that could potentially meet the priorities and criteria

identified by Seattle Public Schools stakeholders. This report will put the Superintendent in a position to determine the type of assessment(s) that are best-suited to help the district achieve its goals as laid out in the Strategic Plan, and most importantly, best-suited to help all students achieve.


2

II. Literature Review Purposes of Assessment: Assessment for What? To design an effective assessment of any kind, one must be clear on the purposes of assessment. Schools and school districts have many varying purposes of assessment. According to Ainsworth and Viegut (2006), purposes of assessment can include but are not limited to:

• Determining student mastery of particular concepts or skills; • Evaluating the effectiveness of instruction; • Motivating students to learn; • Helping students learn content through application and other reasoning skills; • Helping students develop positive attitudes toward a subject; • Communicating performance expectations to students; • Giving students feedback about what they know and can do; • Showing students what they need to focus on to improve their understanding; • Encouraging student self-evaluation; • Determining report card grades; and, • Communicating to parents what students presently know and can do.

The literature on assessment focuses on two broad categories serving distinct purposes: formative, or assessment for learning; and summative

, or assessment of learning. Two additional purposes of assessment for schools and districts, which are discussed to a lesser degree in the literature, include diagnostic (to identify specific learning needs), and evaluative (to determine how well the system is working to improve student learning).

Distinguishing Formative From Summative Assessment In recent years, much of the literature on assessments has focused on formative assessment. Formative assessments differ from summative assessments in that they are most often administered during the learning process to gauge students’ understanding of the material being taught and to adjust instruction based on their understanding. Formative assessments tend to be teacher-developed and administered, and can be as formal as paper-and-pencil tests or as informal as observations of curricular activities or class discussions. Summative assessments, on the other hand, tend to be administered after the learning process to gauge students’ mastery of the material taught. Summative assessments can take a variety of forms ranging from teacher-developed open-response questions to standardized tests. Another distinction between formative and summative assessment is the latter focuses on an “extrinsic motivation,” or outside force, for learning, as opposed to an intrinsic approach of self-motivation (McMillan, 2007).

Literature on formative assessments describes the positive impact this practice can have on student learning. In particular, research by Wiliam et al (2004) shows formative assessments, when implemented well, are one of the most cost-effective interventions compared to other interventions such as increased teacher content knowledge and class size reduction. When studying the impact of formative assessments in classrooms over the period of one year, the researchers found classrooms where teachers used frequent formative assessments showed two times the learning rate of students in other classrooms.

Formative Assessment


3

Since the positive impact of formative assessment on student learning has been recognized in the literature, formative assessment has become a popular strategy for districts to use to improve student achievement. Proprietary companies have developed and sold benchmark assessment systems, which are described as “formative,” to school districts. However, researchers caution that not all assessments advertised as formative yield the same benefits. According to Wiliam and Thompson (2007), formative assessment has a positive impact on student achievement only when the cycle between assessments is relatively short, ranging from minute-to-minute to every four weeks. Longer-cycle assessment systems designed to be administered between quarters, semesters and years (i.e., spanning more than 4 weeks between each assessment), such as many of the benchmark assessment systems being advertised as formative, do not show the same results. However, benchmark or summative assessments can meet other important purposes of assessment such as monitoring student progress and predicting student performance on state standards.

Characteristics of Effective Formative Assessment

Personalized Information on Student Learning Ideally, formative assessment is one part of a larger instructional strategy to help both students and teachers learn. According to Fullan and Crevola (2006), the most critical aspect of formative assessment is the fact that it provides personalized information about how and what students are learning, which teachers in turn use to deliver precise instruction to meet students’ individual learning needs. This process of personalized teaching and assessment described by the authors, called “breakthrough” instruction, also requires a highly focused learning community for teachers. In such a system, teachers have access to strong assessment tools that are well-aligned with learning standards, as part of a larger school- and district-wide system for monitoring learning and continuously responding to student needs. Marzano (2006) describes a process for teachers to develop a formative assessment system at the school or district level. The author points out that frequently administering formative assessments tends to motivate students to improve by providing them with information on their performance and evidence that their effort can increase performance. Assessment Leadership In order for the system to work well, there must be strong leaders at the school and district level who support and model the learning community. The importance of leadership is confirmed by Stiggins (2007), who maintains teachers need districts to put structures in place to assure sound assessment practices, and administrators should have a thorough understanding of how to administer, interpret and apply strategies that respond to assessment results. Aligned to Standards Marzano (2006) provides specific guidance on how teachers can develop formative assessments that are aligned to learning standards. This process begins with “unpacking” and prioritizing the benchmarks that are most essential for students to learn, which are included in standards documents. Teachers then group skills into three types of items: basic details and processes included in the material taught; complex ideas and processes included in the material taught; and, inferences and/or applications students might be able to make based on the material taught, but which go beyond the material taught. Within each type of item, teachers develop assessments to measure mastery of each standard.


4

School- and District-Wide Assessment Literacy Ainsworth and Viegut (2006) describe the elements that make a formative assessment system successful in a school or district. First, in order to develop a high-quality assessment system, school and district leaders and staff must be “assessment-literate.” The authors describe assessment-literacy as “the ability to understand the different purposes and types of assessment in order to select the most appropriate type of assessment to meet a specific purpose.” Assessment literacy requires not only an understanding of the different types of assessment, but also the concepts of validity, reliability and fairness and how to evaluate assessments based on these criteria. The authors describe a series of questions developed by Popham (2003) that teachers should ask when developing formative assessments:

o What am I really trying to teach? o What do my students need to know and be able to do? o How can I translate the big curricular goals…into specific teachable components? o What do my students already know about the topic I’m planning to teach?

Using Ainsworth and Viegut’s (2006) formative assessment development process, districts and schools then prioritize standards and align formative assessments to the highest-priority standards. Teacher Design & Ongoing Professional Development In order to foster and maintain assessment literacy throughout a school or district, Ainsworth and Viegut (2006) suggest teachers design, administer and score assessments collaboratively, as well as receive initial and ongoing professional development on how to determine whether formative assessments are measuring highly-prioritized standards and other skills teachers need to refine assessments on an ongoing basis. In its analysis of formative assessment in best-practice secondary schools in eight countries from 2002 through 2005, the Organisation for Economic Co-operation and Development ([OECD] 2005) identified key elements of successful formative assessment and the strategies used to achieve those elements. First, schools established a classroom environment that encouraged interaction and the use of assessment tools. Such environments were focused on student learning, as opposed to activities. Second, classrooms and schools established learning goals, monitored student progress and adjusted goals as needed. Third, teachers used varying instructional approaches to meet different student learning and assessment needs. Teachers provided feedback to students on their performance and used the data to drive instruction toward individual student needs. Finally, students were actively involved in the learning and assessment process, including peer- and self-assessment.

Summative assessments are a measure of what a student has learned over a given period of time. McMillan (2007) defines a summative assessment as one that is “conducted mainly to monitor and record student achievement, and is used for school accountability.” Summative assessments can be given at the end of an instructional unit, quarter, course, semester, or year, for example. While the results of summative assessments can be used formatively to guide further instruction, the primary purpose of summative assessments is to measure what has been learned already. Many schools and districts use summative assessments two or three times per year to monitor student progress and to predict performance on state assessments.

Summative Assessment


5

Summative assessments can be comprised of a variety of question types (e.g, multiple-choice, open response, short answer, matching, true/false). Data are particularly useful for providing performance information for groups or populations of students who have received the same instruction. It is important to note that both formative and summative assessment data can be used formatively to guide instruction. As Biggs (1998) maintains, “sensible educational models make effective use of both formative assessment and summative assessment.” The two types of assessment are not mutually exclusive in one system. The chart on the following page summarizes the purposes, characteristics and examples of diagnostic, formative, summative and evaluative assessments.


6

Types of Assessment, Their Purposes and Characteristics

Type of Assessment

Purpose Characteristics Examples Also Known As

Formative To support learning and drive instruction

• Assessment for learning • Administered frequently (every day to

every 4 weeks) • Administered and scored by teachers • Interim measure which is used to adjust

instruction • Includes multiple feedback loops to

students and teachers • Tends to be highly individualized • Students are not usually assigned grades

on performance • Internal motivator for student learning

• Classroom-Based Assessments

• Observation

Summative To measure the achievement of students after instruction

• Assessment of learning • Less frequent than formative (every 4

weeks to every year) • “Final”’ measure • Can measure student learning within

short or long periods of time (i.e., one curriculum unit or one year of instruction)

• Everyday Math benchmark assessments

• School Turnaround assessments

• WASL • DRA

• Benchmark • Interim • “Mini-

Summative” • “Early-

Warning Summative”

• “Dipstick”

Diagnostic To identify specific learning needs and place students in appropriate programs

• Assesses students’ skills through a particular lens to identify signs of a specific learning need (e.g., developmental delay, English language proficiency, advanced learning)

• Other types of assessments can be used diagnostically (e.g., DRA is used primarily for summative purposes but is also used to identify advanced learners)

• Washington Language Proficiency Test

• Brigance • Cognitive

Abilities Test

• Screener • Placement

Test

Evaluative To evaluate the quality of educational systems, curricula or programs

• Longitudinal evaluation • Evaluation of the system, not students

• NAEP • PISA

• Evaluation

Building a Comprehensive Assessment and Accountability System Regardless of the types of assessment used, researchers have identified elements of quality assessment to guide schools and districts that are developing assessment strategies. Stiggins, Arter, Chappius & Chappius (2004) identify four “keys” to quality assessment: have a clear purpose; have clear targets; serve as an accurate assessment of what was taught; and communicate

results clearly to both learners and instructors.

To place assessment systems in the context of the purposes they serve, assessments can be thought of as one element of a larger accountability system for a school or district. It is important not only for a district to assess student performance, but to make decisions based on the data.


7

To illustrate, the Mid-Continental Research Educational Laboratory (2005) has identified six characteristics of effective accountability systems: • High expectations for all students; • High-quality assessments aligned with standards; • Alignment of resources, support and assistance for improvement; • Applying sanctions and rewards; • District and school personnel data usage; and, • Informative to parents and the community. These six characteristics are inter-related. A school or district’s learning expectations for students guides the standards students are required to meet and are assessed on. Further, a district implementing an ideal accountability system would use assessment data to align resources to best support areas of need, potentially apply sanctions and rewards, assign district staff matching skills to areas of need, and communicate student data to parents and the community. Balancing the Purposes of Assessment Within an Accountability System Schools and districts often have competing purposes for assessment in the form of internal and external accountability. While an internal accountability system can be best served by formative classroom assessments to gauge individual student progress, an external accountability system can be best served by summative or benchmark assessments that provide a “snapshot” of district-wide performance. Fullan and Crevola (2006) acknowledge this tension between different purposes of accountability, noting external accountability systems tend to overlook the need to develop internal accountability systems within schools in their attempts to impact classroom practice. The authors assert that when developing a comprehensive assessment system, it is critical to ensure not only that the multiple parts of the system are aligned, but also that they are combined in a dynamic way to provide meaningful data to stakeholders at all levels of the system. The authors recommend districts seeking to develop comprehensive assessment systems build on successful formative assessment strategies already in place in classrooms, integrating existing practices into the new system. Integrating in this way can mitigate the burden to teachers of adding new assessments. Similarly, the OECD’s study of formative assessment in eight countries showed that tension between classroom-based formative assessments and school- or district-based summative and evaluative assessments was a barrier to effective implementation of a formative assessment system (2005). Abrams also notes that in fieldwork conversations with teachers, the teachers stated they do not have time to administer formative assessments due to the pressure of preparing students to meet state standards (2007). The OECD issued a set of policy recommendations for districts developing comprehensive assessment systems. At a minimum, OECD recommends formative and summative assessment purposes not compete with one another. Ideally, the two assessment types should reinforce one another. For example, data collected through summative assessments showing positive student growth should be shared with teachers to show them how well their formative classroom strategies are working, and to make any necessary adjustments to overall instructional strategies.


8

Teacher and Principal Training is Critical The effectiveness of any assessment strategy with regard to impacting teaching and learning hinges on effective training for teachers and administrators on the purposes and practices of the particular assessment. While this point is made widely across the assessment literature, the National Science Foundation offers specific characteristics of professional development that were most successful in implementing the federal Urban Science Initiative (USI) grants in 21 urban districts from 1993 through 1998. Borman’s (2005) analysis of the USI suggests effective professional development:

• Improves student learning; • Helps educators meet diverse students needs; • Allows time for inquiry, reflection and mentoring; • Is sustained and rigorous; • Recognizes teachers’ intellectual development and leadership; • Fosters greater content knowledge; • Is designed and directed by teachers and principles of adult learning; • Balances individual priorities with school and district needs, and advances the profession; • Makes the best use of new technologies; and, • Is site-based and supportive of a clear vision for student achievement.

Clearly, different types of assessment can meet different purposes for a district, school or classroom. Regardless of the type of assessment selected, it is critical that it meet the district’s clear purpose, be valid and reliable, and that the district implement it in a way that meets its intended needs.


9

III. Inventory of Assessments Currently Used in Seattle Public Schools Seattle Public Schools uses a wide variety of assessments today. Due to the long history of site-based management, assessment practices vary across schools and no comprehensive inventory of classroom-based, formative assessments exists. While teachers use many types of formative assessments, and some schools have developed common formative and summative assessments, there is not a district policy on assessment beyond a few district-wide summative tests. Results of formative assessments are usually kept at the classroom level; specific formative assessments and instructional strategies resulting from assessments are not formally shared, and were difficult to obtain for this study. District-wide, Seattle Public Schools requires all schools to administer summative assessments in reading in grades K-2 (one time per year) and in math in grades K-11 (three times per year). The reading assessment is the Developmental Reading Assessment (DRA) and the math assessments are the benchmark tests associated with the Everyday Math curriculum in elementary school, the CMP2 curriculum in middle school, and benchmark tests in Integrated Math 1, 2 and 3 in high school. In addition, district-run Head Start preschool classrooms are required to administer a curriculum-embedded assessment three times per year. These assessments are in addition to the Washington Assessment of Student Learning (WASL) required in grades 3-8 and 10 by the state and federal governments under the No Child Left Behind law. Beyond these district-required assessments, many schools administer a host of summative assessments in reading and math to meet a variety of purposes. For example: the School Turnaround reading assessments are administered three times per year in nine middle schools; the DIBELS assessment is administered one time per year in grades 1-5 for the six schools receiving federal Reading First grants; and, many schools administer school-wide writing prompts which teachers score using rubrics from the WASL writing assessment. The range of summative assessments currently used in SPS that were voluntarily provided by stakeholders interviewed for this study is summarized in the table on the following page.


10

Summative Assessments Currently Used in Seattle Public Schools

Content Area Assessment Name

Grade Level Administered

Frequency Required by District?

Reported to District?

Primary Purpose(s) of Assessment

Additional Information

Preschool Developmtl. Standards

DLM curriculum-embedded assessment

Pre-K (for District-run Head Start prgms. only)

3x/year: • Within first

45 days of enrollment

• Mid-year • End of year

Yes Yes Drive instruction Program evaluation (long-term)

District has adapted the DLM assessment many times; it is now a hybrid of off-the-shelf and home-grown

Kindergarten Readiness

Kindergarten Inventory

K 1x/year at beginning of year

No No Determine kindergarten readiness

Teacher use and reporting of results vary for this assessment.

Reading Stage A

Assessment K 1x/year mid-year Yes Yes Monitor reading

skills and growth

Developmental Reading Assessment (DRA)

1-8 2x/year (fall and spring)

Grades 1 & 2

Yes Student placement Diagnostic

Required by the state in the fall for grade 1 and required by the district in the spring for grades 1 and 2. Some schools administer the DRA to advanced grade 1 students and to students in grades 3-8 who are below grade level

DIBELS 1-5 1x/year Yes, for the 6 Reading First schools

Yes Monitor reading skills and growth Drive instruction

Columbia Teachers College (CTC) Quick Assessment

1-8 Varies Yes, for schools using CTC Readers Workshop curriculum

Yes Monitor reading skills and growth Drive instruction

Read Naturally 2-5 Varies No No Drive instruction

Reading Counts

1-2 Varies No No Student placement Drive instruction

School Turnaround

K-8 3x/year: • Beginning of

year • Mid-year • Year-end

No No Student placement Monitor reading skills and growth Drive instruction

9 middle schools and K-8s, and TT Minor, use this assessment Based on GLEs Year-end assessment used for student placement following year


11

Content Area Assessment Name



Reported to District?



Reading, Cont’d

Scholastic Read 180

9-12 Varies No No Diagnostic Monitor reading skills and growth

For Pathways students Computer-based Currently conducting a mini-evaluation of this assessment

Scholastic Reading Inventory

9-12 Varies No No Diagnostic Monitor reading skills and growth

Computer-based

Gates-McGinite

K-8 Varies No No Monitor reading skills and growth

Fountas and Pinnell Benchmark Assessments

6-8 Varies No No Monitor reading skills and growth

Associated with Guided Reading curriculum

Degrees of Reading Power

9-12 Varies No No Monitor reading skills and growth

Reading comprehension assessment

STAR K-12 Varies No No Monitor reading skills and growth

Associated with Accelerated Reader curriculum

Math Curriculum-

Based Benchmark Assessments

K-11 3x/year: • Beginning

of year • Mid-year • Year-end

Yes, but mid-year assessment not required for high school in 07-08

Yes Monitor math skills and growth Drive instruction (school purpose) Program evaluation (district purpose)

Use EduSoft to administer K-5 mid-year assessment is from the Everyday Math curriculum 6-8 mid-year assessment is from the CMP2 curriculum High school assessments are for Integrated Math 1, 2 and 3

School Turnaround

6-8 3x/year: • Beginning


No No Student placement Monitor math skills and growth Drive instruction Year-end assessment used for student placement following year

Only Mercer & Madison use this math assessment Based on GLEs

Navigator 6-8 3x/year: • Beginning


No No Student placement Diagnostic

For a limited group of students who receive after-school services


12

Seattle Public Schools also administers diagnostic assessments based on demonstrated student needs to identify and place students in specific programs such as bilingual education, advanced learning and special education. These assessments are summarized in the table below.

Need Diagnosed

Assessment Name



Reported to

District?



English Language Proficiency Washington

Language Proficiency Test 2

K-12 LEP students

Initial placement, then 1x/year

Yes Yes Identification of bilingual services needs Student placement Monitor student English language proficiency Meet state requirements

When students reach Level 4, they are advanced from the Bilingual Orientation Center to mainstream classrooms Council of Great City Schools recs will inform this area

Special Education Individual

Assessment K-12 sped students

Initial placement, then every 3 years

Yes Yes Identification of special needs Student placement

Brigance K-12 sped students

1x/year Yes Yes Drive instruction Meet federal funding requirements

Required federally by IDEA to determine “Present Level of Performance” Criterion-referenced “worksheet” Scores are included in IEPs

Advanced Learning Cognitive

Abilities Test K-7 students who are nominated for advanced learning

Initial placement No Yes Identification of advanced learning needs

Woodcock-Johnson Form 3 Achievement Test in R&M

K-1 prospective advanced learners


Individually administered 1:1.

ITBS 2-3 prospective advanced learners


DRA 1-2 District-wide scores screened for highest-achieving students in each sub-group

Yes Yes Identification of advanced learning needs, especially within sub-groups and to increase diversity


13

IV. Stakeholder Needs and Priorities For Assessment In order to collect information on the district’s current purposes of assessment and the needs and priorities of stakeholders, in-person and telephone interviews were conducted. Interviews focused on stakeholders’ perspectives on current SPS assessment practices, the key components that would comprise an ideal district-wide assessment system, and the level and types of support they would need to implement such a system. Thirty-four interviews were conducted in March 2008 with district administrators, principals and teachers (see Appendix 1 for interview list). In addition, a survey was conducted among school principals, teachers and central office staff at the elementary, middle and high school levels to identify their needs and priorities for assessment. Written surveys were distributed at a district-wide staff meeting on March 25, 2008. Seventy-four surveys were returned, of which 80 percent were completed by principals or assistant principals, 12 percent by central office staff, five percent by teachers, and three percent by staff who did not disclose their roles. Fifty percent of survey responders represented elementary schools, 16 percent represented high schools, 11 percent represented K-8s, 11 percent represented middle schools, eight percent represented all grades, and four percent of respondents did not disclose their instructional level of work. Current Purposes of Assessment Interviews with stakeholders revealed six broad purposes of assessment in the district today.

• Instructional – To drive instruction at the classroom level • Monitoring – To monitor student, school and district performance progress toward

mastery of state standards and predict performance on state assessments • Diagnostic/Placement – To identify struggling students and place them in services

that meet specific needs (i.e., special education, advanced learning, bilingual education, remediation)

• Summative – To measure student learning after instruction • Mandated – To meet district, state or federal funding requirements • Evaluative – To evaluate the impact of specific programs

Interviews showed stakeholders in SPS have many broad purposes of assessment, which are currently met on an ad-hoc basis as information is needed. The quality of assessment systems vary by school; some schools have exceptionally thorough assessment systems that are well-aligned to curricula and drive instruction, while others are focusing first on aligning curricula grade-to-grade within their school buildings and have not yet developed coherent assessment systems. The survey asked respondents to rank the six purposes of assessment based on the how well they characterized current district assessment purposes. The highest-ranked current assessment purpose was to inform and drive instruction, followed by ongoing monitoring of student learning and diagnostic/student placement. The lowest priority for assessment was to evaluate programs, followed by meeting federal, state or district requirements. The table below shows the percentage of survey respondents who ranked each purpose first, first or second, fifth or sixth, and sixth. Again, these data represent respondents’ perceptions of current priorities in SPS.


14

Percent of Survey Respondents Ranking Each Purpose of Assessment As High or Low Priorities in the Current System

Diagnostic / Student

Placement

Measurement of Learning

After Instruction

Ongoing Monitoring of

Student Learning

Inform & Drive

Instruction Meet

Requirements Program

Evaluation Percent Ranked #1 28% 19% 29% 37% 16% 5% Percent Ranked #1 or #2 48% 36% 60% 61% 21% 8% Percent Ranked #5 or #6 22% 11% 7% 12% 61% 67% Percent Ranked #6 5% 3% 3% 4% 33% 41% Stakeholder Opinions on Ideal Purposes of Assessment When asked in surveys and interviews what should be the ideal purposes of assessment, the majority of stakeholders responded they would like informing and driving instruction and monitoring student learning to remain the highest priorities, or to be even higher priorities. Respondents also indicated that meeting federal, state and district requirements and program evaluation should be low priorities. Survey respondents pointed out that there are different purposes of assessment at different times of the school year. For example, diagnostic assessment is important at the beginning of the year to obtain data on students’ initial skill levels, while monitoring student learning is important during the year and summative assessment of what students learned is most important at the end of the year. Survey respondents also made the connection between assessment and adjusting instruction, indicating an ideal system would use assessment data to change instructional strategies. Further, many thought that quick, classroom-based assessments to monitor student learning should be administered more frequently. Some respondents also wrote that assessment data should be used for teacher and principal evaluation. Many teachers and principals interviewed and surveyed said they would welcome a district-recommended (or in some cases, even district-mandated) assessment system with clear directions on how and when to use it. Additional Data Needed Overall, stakeholders expressed the need for more district-wide student assessment data. Specifically, they cited the need for detailed, strand-level assessment data in each content area that stem from curricula and are aligned with GLEs. The need for more consistent, district-wide data at the high school level was also expressed. In addition, stakeholders reported they would like data on the impact of professional development on student achievement. However, some teachers and principals cautioned against adding more assessments, as they take away from instructional time. The need to connect district-wide student assessment data systems with other student data systems such as the SOURCE was also identified. For example, administrators would like to be able to access one district-wide data system and easily view one student’s or a group of students’ benchmark assessments, WASL scores, Student Learning Plans, disciplinary actions, attendance and enrollment histories.


15

It should be noted that many survey respondents did not believe the new elementary and middle school math assessments provided them with useful data, or that assessments were aligned with the curricula. However, there were also indications of misalignment between instructional pacing and assessment. The survey comments were insufficient to draw a conclusion about the quality of these new assessments and whether they are meeting the district’s purposes, but this is an issue the district should continue to monitor carefully. Stakeholders also spoke highly of both the DRA and Direct Writing Assessment (DWA), the latter of which is no longer administered district-wide. Most stakeholders felt both assessments provided rich student data that was useful for instruction. While the focus of this report is on reading and math, it should be noted that several stakeholders requested the district reinstate district-wide administration of the DWA. Also, many stakeholders voiced the need for funding for substitute teachers during the time teachers take to administer the DRA individually to each student. This assessment takes approximately two instructional days per classroom to administer. Finally, many stakeholders stated teachers and principals need training in assessment literacy, or how to use data to inform instruction. Critical Elements of an Assessment System Stakeholders were asked to identify specific elements they would like to be part of a potential assessment system. Responses were grouped into the following categories of information:

• District-wide reading benchmark assessments for each grade level aligned to GLEs, to be administered three times per year;

• A district-mandated menu of formative Classroom-Based Assessments at each grade level, with corresponding rubrics, from which schools could select;

• District-wide diagnostic screening tools at each grade level to identify struggling students and place them in appropriate services;

• An improved special education assessment to monitor student performance progress throughout the year;

• Standardized, comprehensive performance data on students in grades K-3 to identify advanced learners earlier on, especially students of color;

• College readiness assessments administered throughout the district; • Student performance on a strand-per-strand basis; • End-of-grade or end-of-course assessment data to place students in courses in the

following year; • Assessment data that is linked to other district data systems such as the SOURCE;

and, • Assessments of how students of color are faring in SPS with regard to access to

advanced learning programs and school climate.


16

Criteria to Consider in Developing an Assessment Strategy Interviews and surveys also asked stakeholders which criteria they believed were most important for the district to consider in developing a district-wide assessment system and evaluating specific assessments. In interviews, stakeholders identified the following criteria as high-priority:

• Valid; • Reliable; • Aligned to GLEs; • Predictive of WASL performance; • Aligned to curriculum and instruction; • Ability to assess a broad range of skill levels in each grade level, or otherwise adaptable

to be inclusive of special populations such as bilingual, special education and advanced learners;

• Applicable to longitudinal data analysis and program evaluation; • Easy to administer, understand and take action on; • Reasonable time commitment (30-60 minutes per class); • Quick turnaround of results; • Sends results to teachers in a format they can immediately incorporate into instruction; • Easy to communicate results to parents; • Easy for district administrators to access data; and, • Ability to show student growth.

In general, interviewees agreed on the criteria listed above. However, individuals expressed divergent opinions on other key issues, such as whether a district-wide benchmark assessment should be organized in a format similar to the WASL (including open-response items requiring students to show their thought processes), whether assessments should be translated and administered into languages other than English, and whether teachers should be responsible for scoring assessments. These issues should be explored in more depth to determine how best to meet the needs of SPS students. During interviews, middle school principals and teachers strongly expressed the need for an assessment to provide student results to teachers in a timely fashion (i.e., one week or less) in a format that is easy to read and suggests specific skill areas to focus on for specific students. Teachers reported they were much more likely to use student data if sent to them in an easy-to-use format and expressed high satisfaction with the School Turnaround assessment in middle schools. This suggests that simply making data available to teachers does not necessarily promote the use of data to guide instruction. Rather, data organized in a way that is geared toward developing a lesson plan is more likely to be used by teachers. Based on the desired criteria obtained in interviews, survey respondents were asked to rank the importance of nine potential criteria similar to those listed above based on how important they considered each one for the district to use in selecting an assessment tool. The survey asked respondents to assume all assessments considered would be reliable and valid.


17

The majority of survey respondents felt the most important criteria were that an assessment be immediately useful for instruction, followed by being aligned to the GLEs and providing rich, detailed data on student performance. Criteria ranked lowest by survey respondents were that an assessment be adaptable to special needs students, followed by a test format that is similar to the WASL and that results be immediately useful for school management. It is important to note that the response rate for criteria ranking was lower overall than the response rate for other survey questions, and the response rate varied dramatically by criterion. The following table shows the number of survey respondents who ranked each criteria first, first or second, eighth or ninth, or ninth, along with the total number who ranked each criterion.

Number of SPS Survey Respondents Ranking Each Criteria for Assessment as High or Low Priorities

Aligned To

GLEs

Format Similar

to WASL

Quick Turn-

around for Results

Detailed Student

Data

Immediately Useful for

Instruction

Immediately Useful for

School Management

Little Time to

Administer

Adaptable to Special

Needs Students

Uses a Growth Model

Ranked #1 21 4 7 17 21 2 1 2 2 Ranked #1 or #2 33 7 15 33 39 3 10 5 3 Ranked #8 or #9 11 16 9 9 7 13 17 26 8 Ranked #9 7 6 1 4 4 6 6 17 5 Total Respondents 57 30 57 57 63 27 41 39 21 It is interesting to note there were high numbers of survey respondents who believed rich, detailed student data was one of the most important criteria, as well as high numbers of respondents who believed an assessment’s ability to provide data immediately useful for instruction was one of the most important criteria. It is difficult for one assessment to meet both of these criteria; they are usually tradeoffs. Assessments that yield more detailed data tend to take more time to administer and score, have slower turnaround time, and have the potential to provide so much data to teachers that it can be overwhelming to determine how to immediately apply data to instructional strategies. Another point to note is the difference between interview results and survey results in stakeholders’ beliefs about how important it is for a district-wide assessment be adaptable to special needs students. In interviews, this issue came up frequently as a high priority. However, only seven out of 39 survey respondents answering this question ranked “adaptable to special needs students” as their first or second priority for criteria to consider.


18

V. A Framework for SPS to Develop a District-Wide Assessment Strategy In order to assist the district in determining the type of assessment that is best-suited to help all students achieve, this report provides a framework for district leaders to evaluate and select one or more assessments. The framework takes into account the relative priority level of each criterion to help district leaders understand the strengths and weaknesses of each type of assessment based on different priorities. A district’s assessment strategy can be comprised of multiple assessments to meet multiple purposes, however, additional assessments require more time and monetary resources.

Range of Purposes and Criteria Met By Different Types of Assessment

Purpose: Inform &

Drive Instruction

Purpose: Monitor Student

Learning

Purpose: Diagnose

Special Needs

Criteria: Immediately

Useful for Instruction

Criteria: Aligned to

GLEs

Criteria: Provides

Rich, Detailed

Student Data Types of Assessment

Formative X X X Summative X X X X Diagnostic X Gap Analysis: What is Needed to Meet SPS’ Purposes of Assessment? In order to determine how best to meet the purposes of assessment identified in this report, it is necessary to identify which current assessment practices are already meeting these needs, any areas of duplication, and gaps where additional or different types of assessments are needed. The table below shows the purposes and criteria met by the major summative and diagnostic assessments used in SPS. The assessments are grouped by school level and content area. The table also shows how additional assessments not currently used in SPS but discussed later in this report meet SPS’ priorities.

Analysis of Current SPS Assessments

Purpose: Inform &

Drive Instruction


Learning

Purpose: Diagnose

Special Needs /

Placement




GLEs

Criteria: Provides

Rich, Detailed

Student Data Preschool Assessments DLM X X X X X Elementary / K-8 Reading Assessments Kindergarten Inventory

X X X

Stage A X X X DRA X X X X X DIBELS X X X X CTC Quick Assessment

X X X X

Read Naturally X X Reading Counts X


19

Purpose: Inform &

Drive Instruction


Learning

Purpose: Diagnose

Special Needs /

Placement




GLEs

Criteria: Provides

Rich, Detailed

Student Data Gates-McGinite X STAR X X X Elementary / K-8 Math Assessments EM Benchmarks X X X X Middle School Reading Assessments School Turnaround

X X X X X

Gates-McGinite X Fountas and Pinnell Benchmarks

X X X

STAR X X X Middle School Math Assessments CMP2 Benchmarks

X X X X

School Turnaround

X X X X

Navigator X X X X

High School Reading Assessments Scholastic Read 180

X X X X

Scholastic Reading Inventory

X X X

STAR X X X Degrees of Reading Power

X X X

High School Math Assessments Integrated 1, 2, 3 Benchmarks

X X X X

Diagnostic Assessments WLPTII (ELL) X X X Individual Special Ed Assessment

X X

Brigance (Sped) X X Cognitive Abilities Test (Adv. Learning)

X X

Woodcock-Johnson Form 3 (Adv. Learning)

X X

ITBS X X DRA X X X


20

Purpose: Inform &

Drive Instruction


Learning

Purpose: Diagnose

Special Needs /

Placement




GLEs

Criteria: Provides

Rich, Detailed

Student Data Additional Assessments Discussed in Report MAP X X X X ESD 189 CBAs X X X X School Turnaround

X X X X

FoS X X X Scantron X X X X Tungsten X X X X The table above shows that Seattle Public Schools administers many assessments that inform and drive instruction, monitor student progress, and are aligned with GLEs. However, there are far fewer tests that are immediately useful for instruction or provide rich, detailed student data.


21

VI. Findings and Recommendations Findings To summarize, this report has identified findings from the literature review and stakeholder outreach that have implications for Seattle Public Schools’ approach to developing a strategic student assessment system. • SPS does not have a common purpose for student assessment. Teachers, schools and the

central office assess students for various purposes. • Most decisions about data, including how often to assess student performance, actions to take

based on the data, and who to communicate the data to are made at the school or classroom level, with the exception of district-mandated assessments such as the DRA reading test in grades 1 and 2 and benchmark math tests in grades K-11.

• Student performance data (except for WASL data) are examined on an ad-hoc basis by teachers and principals, with different schools using different strategies to analyze and take action on data.

• Formative assessments vary widely by classroom. There are no formal district guidelines on formative assessments.

• The district administers many assessments that inform instruction, monitor student progress and are aligned with GLEs, yet fewer assessments that are immediately useful for instruction or provide rich, detailed student data.

• Since the district has a clear policy on the elementary and middle school math curriculum, the purposes of assessment are most clear in math. In other areas, such as reading, there are fewer district-wide policies for and purposes of assessment.

• Detailed student assessment data are not readily accessible to central office administrators who rely on making data-driven decisions to guide their work.

• Teachers and principals are not satisfied with the Edusoft assessment platform as it is currently used. Teachers feel the data are not easily accessible and do not inform instruction.

• Simply making data available to teachers does not necessarily promote the use of data to guide instruction. Rather, data organized in a way that helps teachers develop lesson plans is more likely to be used by teachers.

• Stakeholders have competing priorities for assessment, requesting both rich, detailed student data, as well as data that immediately drives instruction. It is difficult for one assessment to meet both of these needs; they are tradeoffs.

Recommendations As Seattle Public Schools embarks on developing a district-wide strategic student assessment system, it is suggested the following recommendations be considered based on the findings in this report.

• The assessment system should have a clear purpose; have clear targets aligned with the Strategic Plan; serve as an accurate assessment of what was taught; and communicate results clearly to students, teachers, administrators and families. The district must prioritize its most important purposes of assessment in order to achieve its goals.

Clear, Narrow Purpose


22

• Formative, summative, diagnostic and evaluative assessment should have distinct purposes and provide different types of student performance data that, when put together, provide a comprehensive picture of student achievement in Seattle Public Schools.

• Based on stakeholder input, the highest priorities of assessment should be to inform and drive instruction for teachers and principals, and monitor student progress by providing ongoing data multiple times throughout the school year on whether students are mastering standards.

• The district should identify current assessments that could be eliminated because they meet the same purposes of assessment or are not high priorities, and replace those with assessments that better meet its purposes. The district should take caution not to add many new assessments, recognizing the time and resources assessments consume.

o Possible areas to eliminate are reading assessments that monitor student progress but are not administered district-wide as benchmarks.

o Possible areas to add are frequent math and reading assessments that provide data immediately useful for instruction.

• Implement district-wide reading benchmark assessments for each grade level aligned to GLEs at least three times per year, similar to math.

Areas to Enhance

• Develop guidelines for Classroom-Based Assessments in each grade level, aligned to standards at the strand level, with corresponding rubrics.

• Select an improved special education assessment to monitor student performance throughout the year. This should replace Brigance.

• Implement a district-wide kindergarten readiness assessment that provides diagnostic information on special needs, bilingual services and advanced learning eligibility.

• The district should continue to carefully monitor math benchmarks to ensure successful implementation of the new math curriculum.

• Teachers and administrators should receive initial and ongoing training in administering assessments and analyzing data. This should include site-based technical assistance.

Assessment Literacy & Training

• An entire culture change is necessary in order to transform the district to be data-driven. Assessing students is only the first step; district leaders must take deliberate steps to ensure stakeholders at all levels of the system see the connection between student assessments and their daily work.

• Summative, diagnostic and evaluative assessment data should be available on the SOURCE for every student.

Access to Data

• Assessment data should be more accessible to district administrators who rely on making data-driven decisions to guide their work.

• Summative assessment data should be provided to teachers in an easy-to-read report within one week after students complete tests. Reports should suggest specific skill areas to focus on for individual and groups of students.

• The district should be deliberate about the degree of detail to provide teachers and principals in student data reports. Reports that are provided more than one time per year and are designed


23

to immediately impact instruction should be short and easy-to-read, with tangible strategies for teachers to quickly absorb and apply. Reports that are provided less frequently and are designed for reflective analysis should include more rich, detailed student data.


24

VII. How Other School Districts Have Approached Student Assessment As Seattle Public Schools develops a district-wide student assessment strategy, it may be helpful to learn how other districts have approached this issue. Below are capsule descriptions of ten school districts – five in Washington State and five in other states – that have implemented assessment strategies. These districts were selected based on recommendations from experts and stakeholders, the literature review, and availability of information. Washington State School Districts Bellevue1

Enrollment: 16,500

Schools: 29 The Bellevue School District developed a unique assessment system based on a common district-wide math curriculum. First, the district moved from a system where curricular decisions were made at the school level to a common math curriculum with common textbooks. At the elementary level, the selected curriculum was Math Expressions (which is similar to Everyday Math); at the middle school level, the curriculum was CMP2; and at the high school level, the curriculum was Core Plus. In 2006, Bellevue received a grant from The Boeing Foundation to create a “Curriculum Web” mapping out lesson plans for each unit of each curriculum. The format is on a shared platform in a “wiki” format, meaning that teachers can share feedback about their experiences using each lesson plan, and add suggested strategies, for all other teachers to see. After creating common lesson plans, teachers collaboratively designed common assessments. Assessments are scored using a common rubric, written by classroom teachers, district-wide. The assessments are also linked to the shared Curriculum Web platform, allowing teachers to go on-line to see examples of student work reflecting the rubric. Besides the consistent, district-wide curricula, lesson plans and assessments, another core element of Bellevue School District’s assessment strategy is collaborative scoring of assessments. All data are shared; the assessment results of every classroom are posted district-wide. Teachers are encouraged to share and view one another’s classroom data and earn clock hours for the time they spend on collaborative scoring. The Bellevue School District also places a high priority on communication with families about curriculum, instruction and assessment. Parents and other members of the public can view detailed course descriptions, “movies” of lessons on each unit, and assessments. The Curriculum Web is designed for both teacher and family use, with information translated into multiple languages. Another unique feature of Bellevue’s curriculum and assessment system is its high level of customization. District teachers analyzed the selected curricula and determined the pacing calendar that they thought made the most sense for their district; they do not follow sequence of textbooks, per se. The district also used customization to develop its own assessments based on some elements of the assessments that came with the curricula. Although there are not enough data yet to draw 1 Information on Bellevue School District’s assessment strategy was obtained from Eric McDowell, Math Curriculum Developer, Bellevue School District, on March 19, 2008.


25

conclusions about how well the district’s assessments correlate with the WASL, the Bellevue School District appears to have a well-aligned, coherent district-wide assessment strategy. Clover Park2

Enrollment: 11,704

Schools: 28 The Clover Park School District implemented Measures of Academic Progress (MAP) to monitor student progress in reading and math in 2006-07. In addition, the district administers the DIBELS reading assessment in grades K-3. The district selected MAP after interviewing and surveying several schools and districts on their assessment strategies. Clover Park started by implementing the test in middle school grades, then added 9th

grade the following year. The greatest demand for MAP came from elementary schools, however, many elementary schools do not have computer labs in which to administer the tests. The district does not have a formal or mandatory process for analyzing data, but staff use MAP data to guide discussions at building-level meetings between instructional coaches, teachers and principals. According to the assessment office, the impact MAP has on teaching varies by school, depending on how instructional coaches and teachers use the data. There has been a strong relationship between WASL and MAP scores.

One issue the district would like improved with MAP is the fact that the tests only show growth from fall to spring, not including winter assessments in the growth model. Therefore, the cohort reports for fall and winter do not match and growth cannot be determined. This issue makes the winter reports less useful to teachers since they cannot assess the impact of their teaching and adjust instruction. Clover Park is working with the MAP vendor to address this issue. The district strongly emphasizes the need for intensive professional development on how to link assessment analysis with instruction. The assessment office also recommends starting MAP implementation with one or two grade levels and expanding year-to-year to establish teacher buy-in and properly train teachers and principals on how to use the data. Highline3

Enrollment: 17,000

Schools: 36 The Highline School District uses MAP to monitor student progress toward state standards. The district has been using MAP for six years. In selecting MAP as its primary assessment tool, the district looked closely at the correlation between MAP and WASL scores, and found strong alignment, particularly in math. Schools administer MAP to all students in grades 3-10 three times per year in reading and math. In addition, elementary schools with advanced learning programs administer the test to 2nd grade students, and some high schools test students in 11th and 12th grade if they have not met 10th

2 Information on Clover Park School District’s assessment strategy was obtained from Feng-Yi Hung, Director of Assessment & Program Evaluation, Clover Park School District, via telephone interview on March 10, 2008.

grade WASL standards. Testing windows are typically two to three weeks long, or 15 school days. Highline is considering making the second (winter) MAP assessment

3 Information on the Highline School District’s assessment strategy was obtained from David Dreher, Project Coordinator, Highline School District, via telephone interview on March 13, 2008.


26

optional for high schools, mainly because of scheduling issues. The district uses the DIBELS reading assessment to monitor student progress in grades K-2 because it is less expensive than MAP. Highline designates a building MAP Coordinator in every school. At the end of every day of testing, MAP Coordinators upload tests results, and results can be accessed 24 hours later. According to the central assessment office, the MAP reports are generalized, as opposed to being differentiated for individual students. MAP offers an optional reporting tool called the Dynamic Reporting Suite, at an added cost, and affords the level of versatility that teachers and principals need to analyze the data in different ways. Highline does not subscribe to the Dynamic Reporting Suite but devotes a full-time staff person to analyze results and provide similar reports. Highline administers MAP to bilingual students who are eligible to take the WASL. For the math assessment, they offer a Spanish audio version of MAP that provides the student with a prompt in Spanish. MAP results for bilingual students are similar to results on the WASL. According to the central assessment office, use of MAP data by Highline teachers is fairly limited. Initially, teachers were highly dependent on MAP building coordinators in each school for data use, and principals had more control over the data. That is, principals would decide whether and how to give the data to teachers, and practices varied building-to-building. The district has tried to raise awareness among teachers about how to access MAP data to decentralize use of data. Since the assessment was implemented top-down, the district struggles to break this into a distributed model. Overall, however, the district is satisfied with MAP as an assessment tool. Highline is embarking on a Shared Accountability Initiative to encourage all participants in the education system to analyze data and continuously improve student achievement.4

Six schools are piloting this initiative in 2007-08, and a collaborative team of teachers, principals, central office staff, union representatives and a school board member is planning expansion of the Initiative next year. In addition, the district will launch a new Data Warehouse in the 2008-09 school year that merges student assessment data with other student information and administration, providing access to all school and district-level staff. The Data Warehouse will also include a principal dashboard to monitor school accountability.

Kennewick5

Enrollment: 14,915

Schools: 22 Kennewick has a multi-pronged assessment strategy. The district has developed a consistent grade-to-grade curriculum with suggested and required assessments and instructional strategies in the content areas of reading, writing, communications, math, science, health and fitness, social studies and the arts.6

4 Highline School District Web Site:

In reading, all schools in the district are required to administer the Kindergarten

http://www.hsd401.org/directory/accountability/ 5 Information on the Kennewick School District’s assessment strategy was obtained from Bev Henderson, Staff Development & Assessment Coordinator, Kennewick School District, via telephone interview on March 7, 2008. 6 Kennewick School District Curriculum Framework. http://www.ksd.org/Portal/Content/Resources/CurriculumFramework.pdf

http://www.hsd401.org/directory/accountability/�

http://www.ksd.org/Portal/Content/Resources/CurriculumFramework.pdf�


27

Reading Assessment, Reading Continuum (K-2), CORE Assessment (K-3), Basal tests (K-2), the DRA (1-2), Functional Level Reading Tests (2-8, high school for students not meeting standard), Iowa Test of Basic Skills (ITBS) (3, 6), Iowa Test of Educational Development (ITED) (9). In math, the district-wide assessments are the Functional Level Math Tests (3-9), the ITBS (3, 6), and the ITED (9). For the Functional Level Reading and Math tests in grades 2-9, Kennewick uses MAP to assess student progress throughout the year and to guide instruction. Since the district has been using MAP for approximately 15 years, it is very integrated into everyday work and there is a high level of support for it, particularly at the elementary and middle school levels. MAP is less relevant to high school assessment for Kennewick, since the assessments only cover standards through 10th

grade. Therefore, high school teachers also administer end-of-course assessments.

Kennewick schools administer MAP two times per year for grades 2-10, in the fall and spring. In addition, students who are struggling or need to retest take the test in the winter. Schools are only required to administer the test in the spring (fall testing is optional), yet all schools opt to test in the fall as well. The district uses results to monitor student progress, inform instruction, diagnose students for remediation, placement in double-dose classes of reading and math, and placement in advanced learning. Results are widely shared among teachers, parents and students. The assessment office reports that teachers strongly support MAP, mainly because the results are immediate and teachers understand what they mean. They also like the fact that the assessment is adaptive, so not every student is taking the same test. One hurdle the district has faced is not having enough computer labs to administer the test to all students at the same time. The district specifies a three-week window in which to complete all testing. In addition, MAP offers a Dynamic Reporting Suite that provides additional detail on specific skills individual students need to work on, but this feature is at an additional cost. Spokane7

Enrollment: 30,132

Schools: 49 Spokane Public Schools built a unique assessment system from the ground up, beginning with a district-wide curriculum. The district’s assessment strategy is comprised of several summative, end-of-unit assessments throughout the school year, and the data are used both summatively and formatively. The district’s curriculum department writes the curriculum for each subject, breaking each down into units of instruction. The department carefully spells out the GLEs and state standards that are covered in each curriculum within a period of instructional time. Included in the curricular materials are summative end-of-unit assessments, which are locally developed. Teachers administer these assessments district-wide at the end of each curriculum unit, within specified testing windows, several times throughout the year. Each curriculum coordinator assembles teams of assessment writers to write “mini” WASL-like assessments, guided by WASL item and test specifications. Spokane Public Schools considers its home-grown end-of-unit assessments to be

7 Information on the Spokane Public Schools’ assessment strategy was obtained from Jack Monpas-Huber, Director, Assessment & Evaluation Program, Spokane Public Schools, via telephone interview on March 13, 2008.


28

“blueprints” for the WASL. Items are written to measure the GLEs and skill strands that are covered in the curriculum. While schools administer end-of-unit summative assessments several times throughout the year, they only report scores to the central office three times per year in elementary and middle schools and four times per year in high schools (at the end of each quarter). In the cases where schools do not report their data centrally, the district expects teachers to use data immediately to adjust instruction and that is the sole purpose of the assessment. For the assessment data that is collected centrally, the data is sent to the district assessment office, which subsequently provides reports, item analyses, and validity tests. The district assessment office also provides data to curriculum content coordinators, who share the feedback with schools to inform instruction. Instructional coaches are highly involved in using data to adjust instruction and inform assessment development. Teachers score the end-of-unit assessments, in some cases transcribing students’ work to scan sheets (mainly in the secondary grades). In elementary schools, Spokane does not yet have a district-wide data collection mechanism. Therefore, teachers gather together a random sample of student work (generated by the assessment office) and send them to the central office where they are keyed into a spreadsheet. In addition to end-of-unit assessments, Spokane has developed the Spokane Assessment of Student Learning (SASL), which is intended to be a simulation of the WASL, for elementary schools. The SASL is slightly longer than the end-of-unit assessments. Students in grades 1 through 6 take the SASL in the fall and winter, and students in grades 1 and 2 take the SASL again in the spring. According to the Spokane assessment office, teachers have varying opinions of Spokane’s assessment strategy. The largest change associated with Spokane’s approach was not the assessment, but rather, the fact that it developed a district-wide curriculum. While there was some resistance to this, there was a surprising level of demand for it, as well.


29

School Districts Outside of Washington State Aldine, TX8

Enrollment: 56,292

Schools: 66 The mid-size, urban Aldine Independent School District, outside of Houston, TX, has earned national recognition for its data-driven management and assessment system. In order to improve performance on state standards, the district contracted with a local software vendor to develop the TRIAND data-management system. TRIAND allows teachers to share student achievement data from state, benchmark, diagnostic and formative assessments with other teachers, principals, administrators, and parents. All teachers are required to submit student data into TRIAND. All schools administer benchmark assessments specified by the district, which were developed by the district from state standards. Benchmark assessments are given every nine weeks. Schools also administer common formative assessments every two to three weeks, developed at the school level. Each school has several “skills specialists” in each subject area, who are trained teachers who support teachers in their instruction and in regularly reviewing student data. Principals meet weekly with the skills specialists to review data.

Enrollment: 59,429 Atlanta, GA

Schools: 99 In 2000, facing chronically low student achievement, Atlanta Public Schools developed an accountability system to support instruction. The district began by setting annual student performance targets, then organized School Reform Teams (SRTs) in the central office to support clusters of schools in achieving targets. Overall district-wide targets were converted into targets for individual schools in every grade in core subject areas. SRTs monitor student data and progress toward targets in each subject. In addition to state and federal assessment requirements, Atlanta established mid-year exams linked to each school’s targets. Schools are expected to incorporate mid-year exams, as well as classroom-based assessments, into a School Achievement Plan. All student data is available on an instructional management system called INsight.

Enrollment: 77,000 Boston, MA

Schools: 144 Boston Public Schools has had a common district-wide curriculum since 1996, when the district approved the Citywide Learning Standards and Curriculum Frameworks. The learning standards specify the skills each student must master to advance to the next grade level and graduate from high school. To measure mastery of learning standards, the district administers benchmark assessments in reading and writing three times per year in all grades, end-of-unit assessments in math in grades 8 Peterson, J.L. (2007). The brave new world of data-informed instruction. Education Next, 7(1). Palo Alto, CA: Hoover Institution.


30

K-5, math performance tasks eight times per year in grades 6-12, and mid- and end-of-course exams in the core subjects in grades 6-12. In addition, Boston developed the Formative Assessment of Student Thinking in Reading (FAST-R) in 2003 in partnership with the Boston Plan for Excellence (BPE) to provide teachers with student learning data they could immediately apply to instruction.9

This assessment is truly formative; scores are not reported to the central office and the sole purpose is to inform instruction. Teachers are given broad guidelines for administration of FAST-R, but they ultimately decide when to give the assessments. Assessments are electronically scored by BPE, followed by a meeting with the teacher one week later to review the data. The assessment covers grades 3, 4, 7 and 10. Schools volunteer to participate in FAST-R; the assessment is optional. In 2004-05, 40 schools expressed interest and 30 schools participated (the maximum number of schools that could participate within the program budget). According to the BPE, participating schools have been pleased with the FAST-R, finding data very useful for instruction and aligned with state standards. In particular, teachers found the reports much more user-friendly than other formative and summative assessments that had been administered district-wide.

Houston, TX10

Enrollment: 202,000

Schools: 293 The Houston Independent School District recently created Curriculum Benchmark Assessments to inform instruction and make intervention decisions. The benchmark assessments, administered every nine weeks in elementary grades and every six weeks in secondary grades, are aligned to the state standards and district curricula in the core subject areas. Each benchmark assessment includes 10 to 15 items modeled after the state assessment. Lower-performing schools are required to administer benchmark assessments, while regional administration offices decide on a case-by-case basis whether other schools are required to use them. Assessments are scored centrally, and results are posted to the district’s on-line data management system. Teachers decide whether students with disabilities participate on a case-by-case basis, and limited English proficient students are provided with accommodations including unlimited time, use of a dictionary, clarification of word context, use of highlighters, and clarification/ restatement/ translation of instructions. Beginning LEP students are not typically included in benchmark assessments.

Enrollment: 353,283 Miami-Dade, FL

Schools: 378 The Miami-Dade County Public Schools use Interim Assessments to measure student progress and adjust instruction throughout the school year. Interim Assessments are criterion-referenced benchmark tests administered three times per year in grades 3-10 in reading, math and science.

9 Chrismer, S.S. (2005). Formative Assessment of Student Thinking in Reading (FAST-R) year II evaluation. Boston, MA: Boston Plan for Excellence. 10 Houston Independent School District curriculum web site: http://dept.houstonisd.org/CURRICULUM/CBA/HISD_CIA_CBA.html.

http://dept.houstonisd.org/CURRICULUM/CBA/HISD_CIA_CBA.html�


31

Interim Assessments are aligned to state standards, the district’s instructional pacing calendar, and state assessment item specifications.11

The assessments must be administered during district-specified testing windows. Tests are scanned and scored using Edusoft, the district’s assessment management platform. In addition to the Interim Assessments, teachers assess students formatively on a biweekly basis.

Miami-Dade also uses assessment as a key strategy in its School Improvement Zone (SIZ) reform.12

The SIZ, comprised of a network of the district’s most struggling schools, provides intensive support and intervention strategies to accelerate achievement. Schools in the SIZ use site-specific weekly or bi-weekly assessments to provide detailed student performance information to teachers. Teachers adjust instruction based on frequent assessment data. This process also promotes rigorous instruction, since students are frequently challenged based on their progress.

11 Miami-Dade County Public Schools Assessment, Research & Data Analysis web site: http://oada.dadeschools.net/IAP/IAP.asp. 12 Miami-Dade County Public Schools School Improvement Zone web site: http://thezone.dadeschools.net/.

http://oada.dadeschools.net/IAP/IAP.asp�

http://thezone.dadeschools.net/�


32

VIII. Assessment Products To provide Seattle Public Schools with an idea of the types of assessment products that are available, this report provides summaries of six assessment systems. These assessment products were selected because they were used by other school districts similar to Seattle, information was readily available on them, and they appeared to meet some of the needs expressed by Seattle stakeholders. This selection is not comprehensive and the summaries are intended to provide only a sense of what each product provides. ESD 189 Classroom-Based Assessments13

Educational Service District 189 developed Classroom-Based Assessments (CBAs) in reading and math that are aligned with state standards. CBAs are administered every nine weeks in reading in grades 3-10 and in math in grades K-12. Teachers administer and score the assessments, enter data into Excel spreadsheets, and submit data to the ESD. The ESD then works with teachers to develop targeted instructional strategies based on the data. Seattle Public Schools purchased and piloted the reading CBAs in 2006-07 and 2007-08 and has received mixed feedback from teachers. However, this could be attributed to the distribution of scores through the district’s Edusoft platform.

ETS Focus on Standards14

Focus on Standards (FoS) is a product of the Educational Testing Service (ETS) comprised of a comprehensive school improvement plan aligned to state standards. FoS helps elementary schools develop interim assessments to measure progress toward state standards and inform instruction. Schools analyze state and district assessment data through ETS’ Instructional Data Management System (IDMS) platform. The IDMS can also be used to analyze performance data of individual students or groups of students. Teachers can access web-based IDMS reports from their classroom computers. After interim assessments are scored, ETS staff lead teachers in structured planning time to review student performance and instructional strategies to address problem areas. Teachers then analyze intervention effectiveness and discuss any necessary instructional adjustments.

Measures of Academic Progress15

Developed by the Northwest Evaluation Association (NWEA), MAP provides school districts with summative benchmark assessment systems in grades K-10 in reading and math and 2-10 in science. MAP is used by a variety of school districts across the U.S. and internationally, with over 3,000 clients ranging from small schools to large metropolitan districts. Assessments are taken on a computer and are adaptive, meaning the test adjusts the difficulty of each question depending on how well a student answers the previous question. MAP is given in English, although a Spanish audio accommodation is available. MAP has aligned its assessment to Washington State standards with a correlation rate between MAP and WASL passage rates of between 0.7 and 0.8.

13 Information on the ESD 189 CBAs was obtained from Kathy Shoop, Assistant Superintendent for Curriculum, ESD 189, on June 10, 2008, and Dan Coles, Literacy Manager, Seattle Public Schools. 14 Information on the Focus on Standards system was obtained from the ETS web site: http://www.ets.org. 15 Information on the MAP assessment was obtained from Jeff Tilton, Northwest Evaluation Association, via telephone interview on February 29, 2008.

http://www.ets.org/�


33

The primary purposes the MAP assessment serves are to identify gaps in knowledge for individual students and groups of students, to individualize instruction based on this data, and to diagnose students for special education or advanced learning. This is a useful system for identifying how well students are mastering skills at a specific point in time. MAP can be thought of as a bridge between an annual summative assessment such as the WASL and an informal formative assessment developed by a teacher. Not only does MAP provide information on student skills, but it also provides external feedback by comparing student performance to the performance of peers in similar districts. In addition, MAP measures student growth, or how much each student improves within a school year and year-to-year, as opposed to measuring only whether students meet standards. NWEA provides intensive training and ongoing consultation. Scantron16

Scantron offers a comprehensive product with two types of assessments to meet different purposes. The Achievement Series consist of formative assessments for a range of grade levels at the district and classroom level to monitor student progress, guide instruction, and predict mastery of state standards. The Performance Series consists of computer-adaptive diagnostic testing to identify students who are behind and measures student performance growth over time.

The Scantron platform for the Achievement Series allows districts to use test items from the Scantron item bank, its own test items, or assessments from other vendors. However, this places the onus on the district of aligning assessments with state standards since districts select the questions or assessments used. Districts can also administer tests online or on paper, allowing the same assessment to be used district-wide using a mixture of computer- and paper-based formats to accommodate schools that do not have computers. Results are available immediately after students complete the assessments. Districts can score the assessments themselves or purchase this service from Scantron. School Turnaround17

The School Turnaround assessment is part of a larger strategy to dramatically improve performance in elementary and middle schools. The intervention consists of a school partnering with the School Turnaround organization to establish and commit to goals, followed by training, implementation of focused strategies, and assessment and professional development. School Turnaround uses specific short-answer and extended response assessments it has developed to measure progress. Teachers administer the assessments, send the scores to School Turnaround, and receive score reports in an easy-to-read format within one week. School Turnaround then works with teachers to provide them tools to differentiate instruction based on the use of the assessment data. In Seattle Public Schools, School Turnaround is working with nine middle/K-8 schools and one elementary school in reading. Teachers report them to be user-friendly and not impactful on teacher time.

16 Information on the Scantron assessment solution was obtained from the Scantron web site: http://www.scantron.com. 17 Information on the School Turnaround assessment was obtained from the School Turnaround web site: http://www.schoolturnaround.org/index.php and personal interviews with Gillian Williams, President, School Turnaround.

http://www.scantron.com/�

http://www.schoolturnaround.org/index.php�


34

Tungsten Learning, a division of Edison Schools, offers the Benchmark Assessment System, which is a formative, electronic assessment tool for grades 2-8 in reading and math. Exam-level benchmarks are available for students in grades 9-12. Benchmark assessments are administered monthly in both reading and math, and are aligned with state standards. Students take the assessments on-line and can see their scores immediately after completion. The primary purpose of the Benchmark Assessment System is to inform instruction. Student data can be tracked and reported regularly to make instructional adjustments.

Tungsten Learning Benchmark Assessment System


35

IX. Conclusion Seattle Public Schools leaders have an opportunity to develop a district-wide assessment system with the potential to provide an accountability framework to meet the goals they have set out in their Strategic Plan. In developing an assessment system, the district should clearly define its purpose of assessment, select the top-priority criteria the system must meet, provide initial and ongoing training to participants at all levels, and frequently review data to ensure progress toward results. This report provides a framework for the district to develop such a system. In developing an assessment system, the district should include district-wide reading benchmark assessments, develop guidelines for Classroom-Based Assessments, improve assessment to diagnose special education and kindergarten readiness, and continue to carefully implement the math benchmark assessments. Regardless of the assessments used, the data should guide major decisions in the district so that all stakeholders at every level of the system understand the connection between student data and their daily work.


36

Works Consulted

Abrams, L.M. (2007). Implications of high-stakes testing for the use of formative classroom assessments. In McMillan, J.H. (Ed.). (2007). Formative classroom assessment: Theory into practice (Ch. 6). New York: Teachers College Press. Ainsworth, L. & Viegut, D. (2006). Common Formative Assessments: How to connect standards-based instruction and assessment. Thousand Oaks, CA: Corwin Press. American Institutes for Research. (2007). A field guide to student success in mathematics and science: A sourcebook for Washington state educators. Washington, DC: American Institutes for Research. Bambrick-Santoyo, P. (2007-2008). Data in the driver’s seat. Educational Leadership, 65, 43-46. Alexandria, VA: Association for Supervision and Curriculum Management. Barton, P. (2007-2008). The Right Way to Measure Growth. Educational Leadership, 65, 70-73. Alexandria, VA: Association for Supervision and Curriculum Management. Bass, Kristin M. & Glaser, R. (2004). Developing Assessments to Inform Teaching and Learning. CRESST/Learning Research and Development Center, University of Pittsburgh. Biggs, J. (1998). Assessment and classroom learning. Assessments in education: Principles, policy and practices, 5 (1), 103-110. Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2003). Assessment for learning: Putting it into practice. Maidenhead: Open University Press. Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2004). Working inside the black box: Assessment for learning in the classroom. Phi Delta Kappan, 86. Borman, K.M. (2005). Meaningful urban education reform: Confronting the learning crisis in mathematics and science. Albany: State University of New York Press. Brookhart, S.M. (2007). Expanding views about formative classroom assessment: A review of the literature. In McMillan, J.H. (Ed.) (2007). Formative classroom assessment: Theory into practice (Ch. 4). New York: Teachers College Press. Chappius, S. & Chappius, J. (2007-2008). The best value in formative assessment. Educational Leadership, 65, 14-18. Alexandria, VA: Association for Supervision and Curriculum Management. Chappius, R., Stiggins, R., Arter, J., & Chappius, J. (2004). Assessment for learning: An action guide for school leaders. Portland, OR: Assessment Training Institute. Chrismer, S.S. (2005). Formative Assessment of Student Thinking in Reading (FAST-R) year II evaluation. Boston, MA: Boston Plan for Excellence. Dahlin, M.P. (2008). A study of the alignment of the NWEA RIT score with the Washington assessment system. Lake Oswego, OR: Northwest Evaluation Association.


37

Fullan, M., Hill, P. & Crevola, C. (2006). Breakthrough. Thousand Oaks, CA: Corwin Press. Elmore, R.J. & Rothman, R. (1999). Testing, teaching and learning: A guide for states and school districts. Washington, DC: National Academy Press. Englert, K., Fries, D., Martin-Glenn, M., & Michael, H. (2005). How are educators using data? A comparative analysis of superintendent, principal and teachers’ perceptions of accountability systems. Denver: Mid-continental Research for Education and Learning. Epstein, A.S., Schweinhart, L.J., DeBruin-Parecki, A., & Robin, K.B. (2004). Preschool policy matters, 7. New Brunswick, NJ: National Institute for Early Education Research. Guskey, T.R. (2007-2008). The rest of the story. Educational Leadership, 65, 28-35. Alexandria, VA: Association for Supervision and Curriculum Management. Keller, Bess. (2008). Data Yield Clues to Effectiveness. Education Week, 27(18), 20, 22-24. Guskey, T.R. (2007). Formative classroom assessment and Benjamin S. Bloom: Theory, research and practice. In McMillan, J.H. (Ed.). (2007). Formative classroom assessment: Theory into practice (Ch. 5). New York: Teachers College Press. Marzano, R.J. (2006). Classroom assessment and grading that works. Alexandria, VA: Association for Supervision and Curriculum Development. McMillan, J.H. (Ed.). (2007). Formative classroom assessment: Theory into practice. New York: Teachers College Press. McMillan, J.H. (2007). Formative classroom assessment: The key to improving student achievement. In McMillan, J.H. (Ed.). (2007). Formative classroom assessment: Theory into practice (Ch. 1). New York: Teachers College Press. McNair, S., Bhargava, A., Adams, L., Edgerton, S. and Kypros, B. (2003). Teachers speak out on assessment practices. Early Childhood Education Journal, 31(1), 23-31. Olson, L. (2007). California center gauges novice teachers with tools, mentors. Education Week, 26 (7), 8-9. Olson, L. (2007). Homegrown Tests Measure Core Critical Reading Skills. Education Week, 26 (35), 32-33. Organisation for Economic Co-operation and Development. (2005). Formative assessment: Improving learning in secondary classrooms. Perie, M, Marion, S., Gong, B., & Wurtzel, J. The Role of Interim Assessments in a Comprehensive Assessment System. The Aspen Institute Education and Society Program, Achieve, Inc. and the National Center for the Improvement of Educational Assessment (2007).


38

Peterson, J.L. (2007). The brave new world of data-informed instruction. Education Next, 7(1). Palo Alto, CA: Hoover Institution. Popham, J.W. (2003). Test better, teach better: The instructional role of assessment. Alexandria, VA: Association for Supervision and Curriculum Development. Reeves, D. (Ed.). Ahead of the curve: The power of assessment to transform teaching and learning. Bloomington, IN: Solution Tree. Schmoker, M. (1999). Results: The Key to Continuous School Improvement. Alexandria, VA: Association for Supervision and Curriculum Development. Stiggins, R.J. (2007). Conquering the formative assessment frontier. In McMillan, J.H. (Ed.) (2007). Formative classroom assessment: Theory into practice (Ch. 2). New York: Teachers College Press. Stiggins, R., Arter, J.A., Chappius, J., & Chappius, S. (2007). Classroom assessment for student learning: Doing it right – Using it well. Prentice Hall. Wiliam, D. (2007-2008). Changing classroom practice. Educational Leadership, 65, 36-42. Alexandria, VA: Association for Supervision and Curriculum Management. Wiliam, D. & Leahy, S. (2007). A theoretical foundation for formative ssessment. In McMillan, J.H. (Ed.). (2007). Formative classroom assessment: Theory into practice (Ch. 3). New York: Teachers College Press. Zemelman, D. & A. Hyde & Daniels, H. (1998). Best practice: New standards for teaching and learning in America’s schools. Portsmouth, NH: Heinemann.

test assessment report by de barros june 2008

Documents

districtwide assessment

strategic assessment

assessment literacy

assessment office

assessment practices

specific assessment

assessment strategy

type of assessment system