scientific research-based interventions (srbi) assessment report

Scientific Research-Based Interventions (SRBI) AssessmentJanuary 2014

Summary Report of Qualitative and Quantitative Findings

Glen Martin Associates 41 State Street, Suite 604-02Albany, NY 12207Phone: (518) 486-8885 Fax: (518) 486-8886www.glenmartinassociates.com

New LondonPublic Schools

NEW LONDON PUBLIC SCHOOLSSRBI ASSESSMENT

Introduction

The Scientific Research-Based Interventions (SRBI) Assessment Toolkit, developed by the Connecticut State Department of Education (CSDE) and the State Education Resource Center (SERC), is comprised of (1) a self-assessment instrument for educators, (2) a site visit interview protocol for external consultants, and (3) a team consensus form that is completed collaboratively by educators and consultants. All three components of the toolkit were used as part of SERC’s program quality review in New London.

SERC is currently coordinating a State Personnel Development Grant (SPDG) for the CSDE. With SPDG funding, the SRBI Assessment Toolkit is being refined for statewide use. For example, during the 2013-2014 academic year, SERC is field-testing online versions of the Self-Assessment and Team Consensus tools, along with a standard template for school- and district-level reporting. The online tools are being piloted at SPDG and non-SPDG sites. Herewith, SERC is pleased to pilot a reporting format that includes both quantitative data (ratings from team members after a consensus-building session) and qualitative observation summaries completed after SERC consultants visited school sites. The latter are designed to address some of the limitations inherent in the self-report nature of the Self-Assessment instrument.

Methodology

1. SRBI Self-Assessment Instrument. The self-assessment is comprised of 45 indicators that describe a school’s level of SRBI implementation across four general areas: System of Instruction, System of Assessment, System of Decision Making, and Leadership. Each indicator is rated according to a 4-point scale where “0” = no implementation; “1” = initial implementation; “2” = partial implementation; and “3” = full implementation. The Self-Assessment is completed prior to the site visit made by SERC consultants. A team of educators, typically representing the School Leadership Team, (e.g., principal, literacy specialist, behavior specialist, ESL teacher, classroom teacher, and/or special education teacher) individually completes the self-assessment online. Individuals are asked to rate the school’s level of implementation of each indicator and cite specific evidence to support their answer. SERC consultants review all individual responses prior to making the site visit.

2. Site Visit. As per the site visit protocol, SERC consultants interview administrators and teachers, toured the building, conducted informal classroom walkthroughs, and observed an instructional/data intervention team. Consultants take detailed notes during these activities and use their observations to inform the consensus-building process in the next step.

3. SRBI Team Consensus. SERC consultants facilitate a collaborative review of individual self-assessment data and assist team members to reconcile or discern the level of “best fit” based on content gathered during Step 1 (i.e., individual self-reported rankings) and Step 2 (i.e., site visit observations). The goal of this process is to reach consensus regarding implementation levels (not yet, initial, partial, or full) and priority rating (low, medium, or high) for each of the 45 indicators. SERC staff record consensus data via an online tool, referred to as the SRBI Team Consensus Form.

Summary Results

In fall 2013, SRBI Self-Assessments and Team Consensus Forms were submitted for five New London schools. The following figures present data from the Team Consensus Forms, i.e., the mean scores and average percentage of points assigned, per section and overall by the schools. As is shown, the overall mean score across all five schools was 2.1, which is just slightly above “partial” implementation.

New London Public Schools Page | 1

Figures 1: Aggregate SRBI Self-Assessment Team Consensus Results (n=5 schools)

Figure 1: Mean Score per Section and Overall

Instruction Assessment Decision Making

Leadership Overall0

1

2

32.2

1.9 1.92.3

2.1

Variation per Checklist Section and Overall

On the next page, Figure 3 demonstrates the variation across individual schools in the percentage of points per section and overall on the SRBI Team Consensus Form. As is shown, the greatest variation occurred on the section related to Decision Making, with schools’ implementation ratings ranging from a low of 38% (Winthrop Elementary) to a high of 95% (New London High). The smallest variation (although still quite large) occurred on the Leadership section, with implementation ranging from a low of 59% (Winthrop Elementary) to a high of 96% (New London High). Overall, New London High School had the highest consensus ratings across all 4 sections. Figure 3 also includes a summary of the areas of strength and improvement noted by SERC consultants during the site visit. These summaries are informed by the practices that consultants observed in each building.


Figure 2: How to Interpret Figure 3

0%

20%

40%

60%

80%

100%

Median district aggregate score

Highest school score

Lowest school score

Contains the middle 50% of aggregate scores for ALL participating schools. This box allows for comparison between an overall district aggregate score and an individual school’s data. (See explanation of data points in the three items below). The line in the middle of the box is the median.

Mean (average) for an individual school

Highest score for an individual school

Lowest score for an individual school


Figure 3: % of Points per Section & Overall – Variation Across New London Schools

JENNINGS ELEMENTARYSite Visit Observations Team Consensus

Areas of Strength: Expressed commitment

among staff to value and respect what children bring to the learning environment, including native language, and to utilize this understanding to scaffold new learning.

Active involvement of leadership in grade level meetings. Leadership models asking questions to promote instructional decision-making among grade level teachers.

Evidence that grade-level teams are held accountable for instructional decisions.

Areas for Improvement: A need to articulate an

approach to teaching in the school that can be explained by teachers and reflected in practice.

Information used to make instructional decisions should extend beyond quantifiable data regarding discrete skills and include, for example, qualitative data regarding academic focus and learner responses to instruction.


Leadership Overall0%

20%

40%

60%

80%

100%

NATHAN HALE MAGNET ELEMENTARYSite Visit Observations Team Consensus

Areas of Strength: Assessments, intervention

logs, and student work is reviewed on a weekly basis and feedback is provided to teachers by the administration.

School staff engage in a very thorough and formal process for identifying specific weekly learning goals/focus areas for intervention; students are involved in tracking progress.

Evidence of a strong professional and

Areas for Improvement: Consistent

implementation and monitoring of effective teaching strategies.

Redesign school structures to provide time for delivery of Tier 2 interventions that are supplemental to core instruction.



20%

40%

60%

80%

100%


collaborative culture for continuous learning and collective responsibility for every child’s learning.

WINTHROP ELEMENTARY MAGNETSite Visit Observations Team Consensus

Areas of Strength: Additional time is provided

for interventions and enrichment.

Continuous professional development is provided by administration, math and literacy coaches, and partner organizations.

Staff meets regularly to review data, discuss students, and plan to address behavior and academic needs.

Teachers recognize and feel a sense of responsibility to respond to the fact that some students are not engaged and some families are not satisfied with the level of rigor in the learning environment.

Areas for Improvement: Develop collective

understanding and consistency in implementing quality instruction.

Maximize the increased use of planning time via clear protocols and procedures for reviewing data and planning instruction/intervention.

Prioritize professional learning to focus greater attention, understanding, and application of an inquiry-based approach to teaching and learning to fulfill theme of magnet school.



20%

40%

60%

80%

100%

BENNIE DOVER JACKSON MIDDLESite Visit Observations Team Consensus

Areas of Strength: Double-block schedule

allows for small group intervention.

Weekly time for common planning and data team meetings.

Clear focus on school improvement by using weekly staff meeting to reflect on current school

Areas for Improvement: Use of research-based

instructional practices for academics and behavior that are appropriate and responsive to students’ strengths and needs.

Clearly articulated plan and delivery of interventions to small groups and individual



20%

40%

60%

80%

100%


practices in relation to school improvement goals.

Professional Learning Practice, one block every day is used by instructional coaches to provide professional development, observe practice, and model instruction.

students with similar needs.

NEW LONDON HIGHSite Visit Observations Team Consensus

Areas of Strength: Strong leadership team with

an understanding and clear focus on improving student achievement.

School schedule provides opportunity to provide Tier 2 and Tier 3 instruction.

Content area data teams meet more than once a week to review data, look at student work, and plan instruction.

Staff encourages family engagement by implementing school-wide initiative to provide assignments that require communication between students and parents/guardians.

Professional development is fluid and informed by real-time data collected through walkthroughs. Administrative team uses videos and peer visits to support teacher growth.

Areas for Improvement: Greater consideration for

student voice, social emotional needs, strengths, and interests when discussing students in need of intervention.

Improvement of instruction and monitoring of implementation of instruction as detailed by school improvement plan.

Interventions should take into consideration current instructional practices and their effectiveness with like peers.

Further clarify, explain, and improve the “gradual release model” of instruction used in the school, so that teachers can discuss and represent in practice.



20%

40%

60%

80%

100%


Results per Individual Indicators

SERC consultants observed many high quality practices throughout all schools. As shown above, many of these practices are common among a majority of the schools. Other high quality practices were present in one or two schools, and are worthy of replication in all. Below are recommendations that will strengthen the district’s multi-tiered system of academic and behavior supports to improve student achievement.

1. Decision-Making: Grade-, school-, and district-level data teams will allow teachers and administrators the opportunity to evaluate the effectiveness of programs and instruction as they make the shift to the Common Core State Standards. Data teams can smoothly integrate processes to help educators develop insight and capacity for “what works” and assist in discerning what instructional approaches should look and sound like.

2. Improvement of Core Instruction: The district would benefit from adopting procedures for intervention planning and progress monitoring for all schools. Many great things are happening. It would be most beneficial to conduct analysis of instructional effectiveness across schools to replicate what’s working, with specific attention to strategies validated by research and practice.

3. Data Management: A district data management system would enhance the disaggregation and use of multiple data sources by data teams, support teachers’ progress monitoring efforts across grades, and strengthen understanding of the needs of whole child in order to more effectively target instruction and allocate staffing.

Tables 1-4 on the following pages present results for each indicator on the SRBI Team Consensus Form. For each indicator, you will see the consensus level of implementation (not yet, initial, partial, or full) assigned by each school. The last column provides the number of schools to assign the indicator a priority rating of “high.”

Table 1: Section I - System of Instruction

SRBI IndicatorJennin

gsNathan

HaleWinthro

pBennie Dover

New London

HS

# to Assign High

Prioritya. System in place to evaluate curriculums, instruction,

and learning environments on a regular basis to determine effectiveness based on outcomes of all students.

1

b. Evidence-based and culturally relevant practices are known and utilized by all staff.

1

c. An alignment of curriculum, instruction, assessment and professional development is evident.

0

d. Core general education practices are accessible by all students (regardless of language spoken at home, culture, ability).

0

e. Continuum of support for all students is clearly defined and articulated (addressing both academics and behavior).

2

f. Core Reading curriculum is reflective of components of comprehensive literacy (phonemic awareness, phonics, fluency, vocabulary, oral and written comprehension).

0

g. Core Math curriculum addresses State Framework (Algebraic Reasoning: Patterns and Functions, Numerical and Proportional Reasoning, Geometry and Measurement and Working with Data: Probability and Statistics).

0


h. A social/emotional learning (SEL) curriculum is in effect and represents core competencies (self-management, social awareness, relationship skills, and responsible decision making).

1

i. School-wide behavior expectations are clearly defined, taught, and reinforced.

1

j. There is a culture of strengthening and building relationships (adult to adult, adult to student, student to student).

0

k. There is a focus on implementation and replication of evidence-based practices (e.g., practices that have been effective with all subgroups as evidenced by . . .).

0

l. Reflective practice fosters continuous improvement of teaching and learning.

0

m. Time for supplemental intervention is in addition to core instruction/programming.

1

n. Developed or is planning to develop scientific research-based intervention materials.

0

o. Interventions have clearly described protocols/procedures.

2

p. Intervention intensity is determined through review of data for considerations about frequency, duration, size of grouping, and effectiveness of instruction/intervention.

0

q. Use and allocation of staff to provide various interventions is reviewed on a regular basis.

2

r. Core general education curriculum is reflective of multiple racial/cultural perspectives.

1

Level of Implementation: Not Yet Initial Partial Full

Table 2: Section II - System of Assessment

SRBI IndicatorJenning

sNathan

HaleWinthro

pBennie Dover

New London

HS

# to Assign High

Priority

a. Common district assessments are used to evidence effectiveness of core instruction, predict which students might need supplemental supports, and clarify/target specific instructional needs.

0

b. Assessment methods are in place to provide the necessary data to inform instructional decisions and demonstrate improvement in student performance over time (e.g., common assessments, performance assessments, portfolio assessment, curriculum-based measurements).

0

c. Teachers collaborate to examine student work and create probes/short and quick assessments for targeted skill areas.

1

d. Assessments are sensitive to students’ instructional level and language proficiency and are culturally relevant.

1

e. An established data-management system, tied to grade level content standards and benchmarks, allows ready access to students’ progress monitoring data for staff and families.

2

f. Data are collected, analyzed, summarized and displayed to inform instruction in the core curriculums based on students’ true peer group (e.g., SES, race, language, ability).

3


g. Teachers use progress monitoring data to evaluate instructional effectiveness and to be informed about the changes necessary to better meet students’ needs (e.g., Data Teams).

2

h. Decision-making rules and cut points for interventions are clearly defined.

1

i. Teachers use progress monitoring data frequently to monitor students’ response to instruction/intervention and to identify students in need of additional support early on.

0

j. Progress monitoring during supplemental instruction (tier 2 and tier 3) focuses on targeted skills and occurs systematically to determine progress.

0

k. There is sufficient time allotted to interpret and use data and other information related to student performance for the improvement of instruction.

1



Table 3: Section III - System of Decision Making


sNathan

HaleWinthro

pBennie Dover

New London

HS

# to Assign High

Prioritya. There is a school-wide data team that disaggregates

and analyzes multiple sources of student data (e.g., achievement and behavioral expectations) to find patterns/trends in order to make appropriate programmatic changes.

2

b. A variety of teams (e.g., grade level, content) meet regularly (once a week) to review students’ progress and to determine effective strategies and interventions.

0

c. There is evidence of collaboration through data teams and coaching/collaborative opportunities across disciplines (e.g., general ed. and special ed.) for improved student outcomes.

0

d. Decisions regarding changes in instruction and intervention are based on a systematic, data driven decision-making model.

3

e. Function of student behavior is assessed early on. 1

f. School staff collaborates with families to determine student needs and intervention plans.

2

g. There is a focus on improved student outcomes vs. eligibility for special education.

0


Table 4: Section IV - Leadership


sNathan

HaleWinthro

pBennie Dover

New London

HS

# to Assign High

Priority

a. School Leadership Team communicates effective schools’ research and evidence-based practices.

0

b. School leadership articulates rationale for SRBI practices and integrates its principles with school values and mission.

0

c. There is a plan for the implementation of SRBI and staff are committed to long-term commitment for transformational change.

1

d. Leadership organizes allocation of staff time and resources and identifies resources for SRBI implementation and improved student performance.

1

e. Leadership builds capacity of staff to ensure sustainability of effective practice (e.g., connects professional growth plans with school improvement goals).

0

f. Leadership provides consistent and systematic observations of teachers to ensure fidelity of instruction/intervention (e.g., walkthroughs).

0

g. School Leadership Team provides ongoing professional development and support for school staff to maximize the utilization and effectiveness of the interventions in the general classroom.

1


h. Leadership facilitates PD around school climate and cultural competence.

0

i. Leadership models and supports a culture of collaborative inquiry and continuous improvement.

0



Appendix A SRBI Self-Assessment Team Consensus Snapshots

Page 7Page 8Page 9

Page 10Page 11

Jennings Elementary SchoolNathan Hale Magnet Elementary SchoolWinthrop Elementary Magnet SchoolBennie Dover Jackson Middle SchoolNew London High School


APPENDIX ASCIENTIFIC RESEARCH-BASED INTERVENTIONS (SRBI) SELF-ASSESSMENT

2013-14

JENNINGS ELEMENTARY SCHOOLTEAM CONSENSUS

Mean Implementation Level per Section and Overall


Leadership Overall0

1

2

3

1.41.3 1.3

1.9

1.5

Le

ve

l o

f Im

ple

me

nta

tio

n

System of Instruction System of Assessment

a b c d e f g h i j k l m n o p q r0

1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n

a b c d e f g h i j k0

1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n

System of Decision Making Leadership

a b c d e f g0

1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n

a b c d e f g h i0

1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n

Level of Implementation: 0=Not Yet 1=Initial 2=Partial 3=Full

Jennings Elementary School Page | 13


2013-14

NATHAN HALE MAGNET ELEMENTARY SCHOOLTEAM CONSENSUS



Leadership Overall0

1

2

32.7

2.4 2.4

2.82.6

Le

ve

l o

f Im

ple

me

nta

tio

n



1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n


1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n


a b c d e f g0

1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n

a b c d e f g h i0

1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n


Nathan Hale Magnet Elementary School Page | 14


2013-14

WINTHROP ELEMENTARY MAGNET SCHOOLTEAM CONSENSUS



Leadership Overall0

1

2

3

2.0

1.5

1.1

1.8 1.7Le

ve

l o

f Im

ple

me

nta

tio

n



1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n


1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n


a b c d e f g0

1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n

a b c d e f g h i0

1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n


Winthrop Elementary Magnet School Page | 15


2013-14

BENNIE DOVER JACKSON MIDDLE SCHOOLTEAM CONSENSUS



Leadership Overall0

1

2

3

2.1

1.4

2.0

2.3

1.9

Le

ve

l o

f Im

ple

me

nta

tio

n



1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n


1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n


a b c d e f g0

1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n

a b c d e f g h i0

1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n


Bennie Dover Jackson Middle School Page | 16


2013-14

NEW LONDON HIGH SCHOOLTEAM CONSENSUS



Leadership Overall0

1

2

32.7 2.7

2.9 2.92.8

Le

ve

l o

f Im

ple

me

nta

tio

n



1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n


1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n


a b c d e f g0

1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n

a b c d e f g h i0

1

2

3

SRBI Indicator

Le

ve

l o

f Im

ple

me

nta

tio

n


New London High School Page | 17

scientific research-based interventions (srbi) assessment report

Documents