the many faces of benchmark assessments: changing the culture of teaching and learning

Proprietary & Confidential

The Many Faces of Benchmark Assessments: Changing the Culture of Teaching and Learning

Hardin Daniel, Discovery Education AssessmentNancy Bonesho, Milwaukee Public Schools

Paul Changas, Metro Nashville Public SchoolsKathy Strunk, Discovery Education Assessment

Milwaukee Public SchoolsMetro Nashville Public Schools

Universal Screener Hardin Daniel & Kathy Strunk

Classroom Teachers – All LearnersInterventionists - Struggling Learners

Interim Benchmarks in the Instructional Process Nancy Bonesho

Classroom TeachersStudents

Longitudinal and Real-Time Data Paul Changas

Professional Development for Principals

What are we trying to accomplish?

Balanced, Comprehensive Assessment Solution

• Progress Zone – 60,000+ Item Bank – Remediate with digital content– K-HS Reading, Math, Science

• Interim Benchmarks– K-HS Predict Student Proficiency– Universal Screener

• RTI Progress Monitoring– Grades K-HS

• VAL-ED Principal Effectiveness

DEA Balanced Assessment System

Data serves any level of intensity or frequency, knowing student needs may vary throughout learning careers

DEA Balanced Assessment System

Unique types of data meet the needs of all educators, including both general and special education.

NCLB and IDEA Both Require Standards Based Accountability

• NCLB 2001 standards based accountability provisions require each state to develop content and achievement standards in several subjects, administer tests to measure students’ progress toward these standards, develop targets for performance on these tests, and impose a series of interventions on schools and districts that do not meet the targets.

• IDEA § 300.309, (a), states a child may be determined to have a specific learning disability, if—(1) The child does not achieve adequately for the child’s age or to meet State-approved grade-level standards in one or more of the following areas, when provided with learning experiences and instruction appropriate for the child’s age or State-approved grade-level standards.

External Review

National Center for Response to Intervention

www.rti4success.org

Challenges to the Use of Interim Assessment as a Universal Screener

• “Brief”• Multiple Grades and Subjects• National Norm vs State Specific• Foundational Skills vs State Standards• Minimum of 3-months before external validation• Diverse Population Data

Research on Standards-Based Screening

• Herman and Baker, (2005), “Unless benchmark tests reflect state standards and assessments, their results tell us little about whether students are making adequate progress toward achieving the standards and performing well on the assessment.”

• Joseph Jenkins, (2008),“I'm convinced screening instruments should be "locally validated" according to the future measure used to judge adequate/inadequate reading outcomes. Schools (and states) can validate the screening cut points on CBM and on other types of screens that predict adequate/inadequate reading levels on the local criterion outcome measure (e.g., state standards test). Because these outcome measures (and the score used to distinguish between adequate and inadequate performance) differ by state and school district, schools should evaluate screening measures (and screening cut points) in relation to the specific (local) outcome measure.”

National Center on Response to Intervention-cut back

• Discovery Education Assessment Predictive Benchmarks is posted on the Center’s website at www.rti4success.org as a universal screening tool for struggling learners.

• At this time, the DEA universal screener is the only state-specific screener for Math and for grades higher than 3rd in Reading and Math.

• DEA received highly favorable scores, with Classification and Reliability ranked highest.

• All screenings are rated for – 1) Classification Accuracy– 2) Generalizability– 3) Reliability– 4) Validity– 5) Disaggregated Data for Diverse Populations

National Center for Response to Intervention

www.rti4success.org

Universal Screener Hardin Daniel & Kathy Strunk

Classroom Teachers – All LearnersInterventionists - Struggling Learners

Interim Benchmarks in the Instructional Process Nancy Bonesho

Classroom TeachersStudents

Longitudinal and Real-Time Data Paul Changas

Professional Development for Principals

What are we trying to accomplish?

• 85,000 Students in 185 schools• About 5,500 teachers• Have been using Discovery Ed. Assessment for 4 years• Started DEA with 22 schools in September 2006• Increased to 109 by December 2006• District wide implementation by September 2007• Over 600 Professional Developments sessions have

been conducted since Sept. 2006 (differentiated)• MPS employs conservative cut scores to avoid under-

identifying students in need of assistance• Predictive Validity: Reading: 86.4% Math 90.1%

1. Teacher understanding of the connection between the benchmarks and instruction

2. Consistent analysis and use of the data by teachers and school communities

3. Formative use of assessments and results with students/parents in a timely manner

* Goal Setting

Reading Results - School A Consistent use of all critical components

School A - Consistent Efforts Yield Sustained Results

• Teachers meet frequently to discuss student DEA data

• Decisions on how to instruct students are based on DEA data – flex groups and differentiated work

• Reports are immediately shared with students and parents; students review difficult items and set class and individual goals for next benchmark

• Periodic ‘check-ins’ occur to keep everyone on task

Math Results - School B Inconsistent use of critical components

School B - Random Acts of Improvement

• Inconsistent use of data

• Inconsistent delivery of targeted instruction

• Inconsistent utilization of differentiated activities

• Complacency in the classroom

• Results were not sustained

Universal Screener Hardin Daniel & Kathy Strunk

Classroom Teachers – All LearnersInterventionists - Struggling Learners

Interim Benchmarks in the Instructional Process Nancy Bonesho

Classroom TeachersStudents

Longitudinal and Real-Time Data Paul Changas

Professional Development for Principals

What are we trying to accomplish?

PK-12 Enrollment 76,033

African American 47.4%

Asian / Pacific Islander 3.8%

Hispanic 15.9%

Native American 0.1%

White 32.6%

Limited English Proficient 10.4%

Students with Disabilities 12.2%

Economically Disadvantaged 68.1%

Mobility Rate 35.4%

MNPS Student Demographics

Where We Were (Spring 2008)

On verge of NCLB Corrective Action

Not consistently data driven

District assessments not well utilized

DEA in some schools, but not a district initiative

Planning for Interim Benchmarks

Information gathering Research literature review School visits Visits to other districts Discussion with test company representatives Focus groups (principals, teachers, exemplary educators)

Options identified Build our own assessments DEA (recommended)

Project Management

Project Charter – Desired outcomes Administration of interim benchmark assessments

(Reading & Math in grades 3-8; Alg. I, Bio I, Eng. II)

Teachers routinely use results to make classroom decisions

Teachers administer classroom formative assessments or probes to identify specific deficits

Change in culture to data driven decision making

Communications Plan - Targets District executive staff District coordinators and facilitators Instructional coaches School administrators Teachers Exemplary Educators School Board members Parents Students Community

Project Management

Project Management

Professional Development Plan – Expectations Principals will take leadership role in each school C&I executive staff will hold principals accountable Lead teachers selected for professional development will

assist/train other school staff Curriculum coordinators and instructional coaches will

support teachers as needed DEA training/discussion will occur routinely at principal

meetings DEA and other formative assessments will be routinely

addressed in school improvement plans Professional development will be ongoing within the school

Project Management

Professional Development Plan – TargetsPrincipals

Key central office staff

School teams (Principal and 3-5 key staff)

Exemplary Educators

Instructional coaches

Additional school staff on as needed basis

Where We Are Now

Common language among our educators

Ongoing and sustained professional development

Sharing of best practices and effective strategies

Monthly meetings of principals, APs and coaches

Principal institutes

Teacher workshops

MNPS Tube (online PD)

Rubric Developed from Kim Marshall, “Interim Assessments: A User's Guide,” Phi Delta Kappan, 9/2008

Use of DEA Interim Benchmarks

District level District and School AYP reports Prioritization of schools for interventions Identification of district-wide PD needs

School level Monitoring of classroom teachers Identification of school-wide PD needs Identification of students for school-wide interventions

Classroom level Identification of student academic strengths and needs Instructional feedback Prediction of student success on high stakes assessments

2009 Relationship Between DEA and TCAP

Correctly predicted TCAP proficient or higher: 87.2%

Correctly predicted exact performance level: 68.6%

Below Proficient Advanced Total

Below 1,898 1,796 50 3,744

Proficient 2,056 10,218 1,183 13,457

Advanced 93 4,639 9,299 14,031

Total 4,047 16,653 10,532 31,232

TCAP Achievement Test3rd DEA

AYP Report Example (Data Warehouse)

Metropolitan Nashville Public Schools

Standards Proficiency Predictor – By Subject _____________________________________________________________________________________________________

School: AAA Middle School School Year: 2009-2010 Subject: MATH

Test: DISCOVERY EDUCATION ASSESSMENT Test Date(s): 02/08/2010 Students: AYP (1-2) (Active Students Only) _____________________________________________________________________________________________________

1 2 3 4

37 43 13 4

38.1% 44.3% 13.4% 4.1%

27 36 30 3

28.1% 37.5% 31.3% 3.1%

45 70 37 5

28.7% 44.6% 23.6% 3.2%

0 5 4 2

0.0% 45.5% 36.4% 18.2%

83 128 77 14

27.5% 42.4% 25.5% 4.6%

26 26 7 0

44.1% 44.1% 11.9% 0.0%

4 25 18 6

7.5% 47.2% 34.0% 11.3%

105 129 66 8

34.1% 41.9% 21.4% 2.6%

86 148 82 12

26.2% 45.1% 25.0% 3.7%

23 6 2 2

69.7% 18.2% 6.1% 6.1%

ED: N

Disability: N

Disability: Y

ELL: N

ELL: Y

ED: Y

Summary - School Level

MATH - 02/08/2010

B

W

H

A

DEA in Combination with Value Added

DEA Growth Report

School Report for TN METRO NASHVILLE DAVIDSON CO SD, TN Change in Proficiency Level and Growth: Test 2 to Test 3

AAA Middle School Grade 5 Mathematics

Comparison of Test 2 to Test 3

Test 2 to Test 3 Actual Change vs. Prediction

DEA Growth Report

Student System School Sex Race Enrolled Grade

Gif Mig ELL ED SpED FD CTS Achievement Probability

7. BASS, R Davidson County

AAA Middle School

M W 5 N N N Y Y N U 13.8

8. BETTERS, A Davidson County

AAA Middle School

M B 5 N N N Y N N U 1.7

13. DEADRICK, I Davidson County

AAA Middle School

M B 5 N N N N Y N U 1.1

20. GARRETT, K Davidson County

AAA Middle School

F H 5 N N N Y N N U 15.2

21. GIBBS, C Davidson County

AAA Middle School

F W 5 N N N Y N N U 23.8

22. GILES, T Davidson County

AAA Middle School

F B 5 N N N Y Y N U 0.1

25. GRAHAM, R Davidson County

AAA Middle School

F B 5 N N N Y N N U 7.2

38. MARIS, C Davidson County

AAA Middle School

M H 5 N N Y Y Y N U 10.1

41. MCELROY, T Davidson County

AAA Middle School

F B 5 N N N Y Y N U 3.3

2009 Accelerate IFor Students Enrolled at AAA Middle School

5th Math ACHIEVE

Merging TVAAS and DEA Projections

Student System School Sex Race Enrolled

Grade Gif Mig ELL ED SpED FD CTS

Achievement Probability

7. BASS, R Davidson County

AAA Middle School

M W 5 N N N Y Y N U 13.8

8. BETTERS, A Davidson County

AAA Middle School

M B 5 N N N Y N N U 1.7

13. DEADRICK, I Davidson County

AAA Middle School

M B 5 N N N N Y N U 1.1

20. GARRETT, K Davidson County

AAA Middle School

F H 5 N N N Y N N U 15.2

21. GIBBS, C Davidson County

AAA Middle School

F W 5 N N N Y N N U 23.8

22. GILES, T Davidson County

AAA Middle School

F B 5 N N N Y Y N U 0.1

25. GRAHAM, R Davidson County

AAA Middle School

F B 5 N N N Y N N U 7.2

38. MARIS, C Davidson County

AAA Middle School

M H 5 N N Y Y Y N U 10.1

41. MCELROY, T Davidson County

AAA Middle School

F B 5 N N N Y Y N U 3.3

2009 Accelerate IFor Students Enrolled at AAA Middle School

5th Math ACHIEVE

Grade/Class/Date:

Level 1 Level 2 Level 3 Level 4 ?Maris, C Garrett, K Bass, RMiller, J Gibbs, C Wells, DSampson, I Giles, TVail, C McElroy, T

Smith, CStreet, IBetters, ADeadrick, IGraham, RSharp, BVince, Z

Parker, IVogel, JWoods, K

Diller, R Chandler, A Crew, A Strong, PEller, A Dorris, W Gross, C Voice, J

Nails, SOrton, JArnold, NCarter, ASawyer, DTiller, MYearly, I

Hill, L Silver, H Moore, JWilson, A Little, IWillis, R Mullins, K

Accelerate II

Advanced

?

Grd. 5 / J ones (10/2009)

TVAAS Projections

ThinkLink Test

Worksheet for Merging TVAAS and DEA (ThinkLink) Data

Accelerate I

Universal Screener Hardin Daniel & Kathy Strunk

Classroom Teachers – All LearnersInterventionists - Struggling Learners

Interim Benchmarks in the Instructional Process Nancy Bonesho

Classroom TeachersStudents

Longitudinal and Real-Time Data Paul Changas

Professional Development for Principals

What are we trying to accomplish?

Contact Information

• Hardin_Daniel@discovery.com• Kathy_Strunk@discovery.com• Paul.Changas@mnps.org• boneshnb@milwaukee.k12.wi.us• Discovery Education Assessment

– Toll Free Number: 1-866-814-6685