RE-THINKING HOW SCHOOLS IMPROVE:

A Team Dialogue Model for Data-Based Instructional Decision Making

Dr. Michael E. Hickeymehickey@towson.edu

Dr. Ronald S. Thomasrathomas@towson.edu

Center for Leadership in Education at Towson University

CCSSO Education Leaders ConferenceSeptember 12, 2007

2

The Big PictureIn today’s session, we are going to:

1. Re-think our understanding of how schools improve—moving from the dysfunction of the old model to the requirements for what a “new model” might look like.

2. Focus on a “new model” for improving performance that enables content, vertical, or departmental teams to use data more effectively for classroom instructional improvement and increased student learning

Part 1: What are we trying to do and why?

4

“Every organization is perfectly designed to get the results it achieves.”

--W. Edwards Deming

5

Think about how long you have been engaged in the school improvement

process.

Has the school gotten better each year?

Has the performance of each student improved as a result of each year he/she spends in the school?

If your answer to one or both questions is no, what will it take to change it to yes?

6

What are data?

Data are observations, facts, or numbers which, when collected, organized

and analyzed, become information and, when used productively in

context, become knowledge.

7

The DRIP Syndrome

8

Being Data Rich

Your school may suffer from

You may need ways to organize the data.

9

Sources of Student Achievement Data

• External assessment data

• Benchmark or course-wide assessment data

• Individual teacher assessment data--Supovitz and Klein (2003)

10

Data-driven schools and school districts use data for two major purposes:

• Accountability (to prove)• Instructional decision making (to improve)

11

The Hierarchy of Data for Accountability Purposes

External (State & National) Assessments

System Benchmark Assessments

Common School or Course Assessments

Classroom Assessments of Student Work

12

The Hierarchy of Data for Instructional Decision Making

Classroom Assessments of Student Work

Common School or Course Assessments

System Benchmark Assessments

External (State & National Assessments)

13

Think about it . . .

Do you have a school improvement plan?

Or a school accountability plan?

A SIP ? Or a SAP?

Have a three minute conversation with someone sitting near you about what you think most schools

currently have.

14

The Old Model

The School Improvement Team, a Data Committee, or one person analyzes data, using primarily state test data. These data are mined for every possible nuance.

15

The Old Model

The data are presented at a faculty, SIT, or department meeting, and faculty members brainstorm ideas for what to do to increase student

performance.

16

The Old Model

Faculty or team members “average opinions” and put forth the solutions that are acceptable to the largest majority of people.

17

The Old Model

This results in school-wide or department-wide initiatives that may or may not be implemented.Data expert Mike Schmoker has estimated that about 10% of what is planned in SIPs actually is implemented at a high level of quality.

18

Results of the Old Model

19

Why is the old model not working anymore?

20

Why? Wrong Data

We have been using the wrong data. State test data are: Way too general Instructionally insensitive – not designed for instructional improvement

21

Why? Wrong Time

The data come at the wrong time. State test data are: Out of date when they arrive For students we no longer have

The results of the changes that are implemented will not be known for a year.

22

Why? Wrong Team

The SIT, a full department, or a Data Committee is the wrong team to do the analysis.Membership is too diverse (often including parents)Meets too infrequentlyNot connected to immediate classroom needs

23

Why? Wrong Plan

The initiatives that are put in place are: Too global to address

the diversity of students Aimed at performance increases

of groups on average Looking for the “silver bullet” that

will have a schoolwide impact

24

We need a new model.

Real time Specific to each grade and subject Addresses individual students’ needs Results in instructional improvements that will actually

occur at a high level of quality Can be re-directed frequently Has meaning for teachers (seen by teachers as a

worthwhile use of their time)

THREE MINUTE CONVERSATION: How do the data conversations in schools that you know of

rate against these criteria?

25

What should that new model look like?

“School improvement is most surely and thoroughly achieved when teachers engage in frequent, continuous, and increasingly concrete and precise talk about teaching practice . . . adequate to the complexities of teaching, capable of distinguishing one practice and its virtue from another.”

--Judith Warren Little

26

In other words . . .

A Classroom-Focused Improvement Process (CFIP)

27

Education After Standards

28

The Classroom-Focused Improvement Process is the work

that professional learning communities do.

A professional learning community is not an

organizational structure.

It is a WAY OF DOING BUSINESS.

29

CFIP: A WAY TO MOVE SCHOOLS From To

• Focus on teaching• Emphasis on what

was taught• Coverage of content

• Curriculum planned in isolation

• Infrequent summative assessments

• Focus on average scores

• Focus on learning• Fixation on what

students learned• Demonstration of

proficiency• Shared knowledge of

essential curriculum• Frequent common

formative assessments

• Monitoring individual proficiency on every essential skill

30

CFIP: A WAY TO MOVE SCHOOLSFrom To

• Remediation• One opportunity to

demonstrate learning• Isolation• Each teacher

assigning priority to different learning standards

• Privatization of practice

• Focus on inputs

• Intervention• Multiple

opportunities • Collaboration• Teams determining

priority of learning standards

• Sharing of practice

• Focus on results

31

Fundamental Concepts of Collaborative Learning Communities

• Teachers establish a common, concise set of essential curricular standards and teach to them on a roughly common schedule.

• Teachers meet regularly as a team for purposes of talking in “. . . concrete and precise terms” about instruction with a concentration on “thoughtful, explicit examination of practices and their consequences.”

• Teachers make frequent use of common assessments.

Continued on next slide

32

“These elements, so rarely emphasized in school . . . improvement plans, deserve our attention more than anything else we do in the name of school improvement.”

--Mike Schmoker (2006)

33

Our Goal in the Data Dialogues:

Frequent, continuous, and increasingly concrete and precise dialogue by school teams, informed by data

34

IS IT WORTH THE EFFORT?

Take a look at the following results. Then you tell us.

35

Maryland School Assessment: 5th Grade Reading Disaggregated Data

49.9

67.6

33.3

16.7

46.2

57.1

85.6

71.4

75

61.5

0 10 20 30 40 50 60 70 80 90

Target

BBES

FARMS

SPED

AF. AMER

Su

bg

rou

ps

% Proficient

2005

2004

2005 57.1 85.6 71.4 75 61.5

2004 49.9 67.6 33.3 16.7 46.2

Target BBES FARMS SPED AF. AMER

36

Maryland School Assessment: 5th Grade Math Disaggregated Data

38.3

60.8

25

22.2

30.8

47.2

83.8

75

60

61.5

0 10 20 30 40 50 60 70 80 90

Target

BBES

FARMS

SPED

AF. AMER

Su

bg

rou

ps

% Proficient

2005

2004

2005 47.2 83.8 75 60 61.5

2004 38.3 60.8 25 22.2 30.8

Target BBES FARMS SPED AF. AMER

37

Reading: 2004 3rd Graders/ 2005 4th Graders

68.7

52.7

21.1

41.7

81.3

66.7

73.1

73.3

0 10 20 30 40 50 60 70 80 90

BBES

FARMS

SPED

AF. AMER

Su

bg

rou

ps

% Proficient

2005

2004

2005 81.3 66.7 73.1 73.3

2004 68.7 52.7 21.1 41.7

BBES FARMS SPED AF. AMER

38

Math: 2004 3rd Graders/ 2005 4th Graders

65.2

36.8

26.3

41.6

75.6

52.4

53.8

60

0 10 20 30 40 50 60 70 80

BBES

FARMS

SPED

AF. AMER

Su

bg

rou

ps

% Proficient

2005

2004

2005 75.6 52.4 53.8 60

2004 65.2 36.8 26.3 41.6

BBES FARMS SPED AF. AMER

39

Grasonville Elementary School Maryland School Assessment -

Reading

 

MSA Percent at Proficient and Advanced - Reading

50

60

70

80

90

100

2003 2004 2005 2006

Grade 3

Grade 4

Grade 5

40

Grasonville Elementary School Maryland School Assessment -

Mathematics

 

MSA Percent at Proficient and Advanced - Mathematics

50

60

70

80

90

100

2003 2004 2005 2006

Grade 3

Grade 4

Grade 5

41

THE CLASSROOM-FOCUSED IMPROVEMENT PROCESS (CFIP):

A Team Data Dialogue Protocol

Part 2:

Components of

THE NEW MODEL

42

What are the right teams to conduct data dialogues?

Grade-levelVertical

Content

43

When is the right time to conduct data dialogues?

•At a minimum, devote at least one hour to data dialogues every two weeks.

•According to several studies, schools that realized the greatest results from a shift to a data culture scheduled data dialogues at least once a week.

44

Frequency of Data Dialogues

Source: Stanford University, Stanford Research Institute, Education Week, January 24, 2004

20

45

35

4548

7

0

10

20

30

40

50

60

A Few Timesa Year

A Few TimesA month

A Few TimesA Week

Gap Closers

Non-gap Closers

45

What are the right data to use in the data dialogues?

Triangulate three types of data:• External Assessment Data• Course-wide Benchmark Assessment Data• Classroom Assessment Data

--Supovitz & Klein (2003)

46

THE GPS ANALOGY

47

Conclusions are specific to students in the class.

Conclusions are used to plan upcoming daily instruction.

The plans are implemented.

What is the right plan where the results of the data dialogues should be used?

48

What is the right way to use the results of the data dialogues?

Conclusions are used to identify enrichments and interventions for the students in the class. Conclusions are used to

plan upcoming daily

instruction.

49

The new process needs to be built on:

1. Dialogue

2. Protocols

3. Triangulation of

Data

50

Why True Dialogue?

“In dialogue, a group accesses a larger ‘pool of common meaning,’ which cannot be accessed individually.

People are no longer primarily in opposition, rather they are participating in generating this pool of common meaning….

We are not trying to win in a dialogue. We all win if we are doing it right.”

- Senge, The Fifth Discipline (2006)

51

Team Learning

Team learning is the process of aligning and developing the capacities of a team to create the results its members truly desire.

The discipline of team learning starts with “dialogue,” the capacity of members of a team to suspend assumptions and enter into a genuine “thinking together.” It also involves learning how to recognize the patterns of interaction in teams that undermine learning.

--Peter Senge (2006)

52

What Is a Data Protocol?

A protocol consists of guidelines for dialogue – which everyone understands and has agreed to – that permit a certain kind of conversation to occur, often a kind of conversation which people are not in the habit of having.

Protocols build the skills and culture necessary for collaborative work. Protocols often allow groups to build trust by doing substantive work together.

53

Using a Data Protocol

Protocols can help us to navigate difficult and uncomfortable conversations by:

Making it safe to ask challenging questions

Making the most of scarce time Providing an opportunity for all to be

involved Resulting in an analysis that will lead to

positive action

54

Using a Data Protocol

The point is not to do the protocol well, but to have team dialogue that is:

In-depth Insightful Concrete Precise

55

The Big Six of Data Analysis

1. Begin with a question.2. Understand the data source.3. Look for the big picture.4. Look for patterns in the data.5. Identify and act on the implications of the patterns for your students.6. Identify and act on the implications of the patterns for your instruction.

56

CFIP DATA DIALOGUE PROTOCOL FORMATS

• One-page overview of the model, page 15• CFIP model with reflection questions, pages

17-18• CFIP model worksheets, pages 19-22• Reflection Guide to Instructional Changes,

pages 23-24• Examples of CFIP model as completed by

school teams, pages 25-38

Take a few minutes to preview these pages.

57

SIX-STEP PROCESS - TEAM DATA DIALOGUE PROTOCOL: MOVING FROM DATA TO INCREASED STUDENT LEARNING

DATA SOURCE(S): __________________________________________________________________________

Step 1: Identify the questions to answer in the data dialogue.Step 2: Build assessment literacy. Define terms (if needed).Step 3: Identify the “big picture” conclusions from the data.Step 4: Identify the patterns of class strengths and weaknesses (using more than one data source, if possible).

STUDENT STRENGTHS STUDENT WEAKNESSES

Step 5: Drill down in the data to individual students. Identify and implement needed enrichments and interventions.

STUDENTS WHO EXCELLED

ENRICHMENTS TO BE PUT IN PLACE

STUDENTS NEEDING FURTHER WORK

INTERVENTIONS TO BE PUT IN PLACE

Step 6: Reflect on the reasons for student performance. Identify and implement needed instructional changes for the next unit.

58

CFIP Step 1: When analyzing data, begin with a question.

All data analyses should be designed to answer a question.

Unless there is an important question to answer, there is no need for a data analysis.

59

CFIP Step 2: Understand the data source

Build ASSESSMENT LITERACY with questions like these:What assessment is being described in this data report? What were the characteristics of the assessment?

Who participated in the assessment? Who did not? Why?

Why was the assessment given? When?

What do the terms in the data report mean?

Be sure you have clear and complete answers to these questions before you proceed any further.

60

CFIP Step 3: Look for the “big picture” views in the data.

Identify:

What do we “see” in the data?

What “pops out” at us from

the data?

What questions do the data raise?

61

CFIP Step 4A: Look for data patterns in a single data source.

What do you see over and over again in the data?

What are the students’ strengths? What knowledge and skills do the students have?

What are their weaknesses? What knowledge and skills do the students lack?

62

CFIP Step 4B: Identify Patterns of Class

Strengths and Weaknesses from Multiple Data Sources.

TRIANGULATION

•In what ways are the results similar among data sources? For example, how do benchmark test results compare with ongoing classroom assessment data?•In what ways do the results among data sources differ?•Why might these differences occur?

63

Power When Multiple Types

of Data Are Used Reduces the anxiety and the mistakes

of relying on a single measure as the only definition of student successProvides more frequent evidence on which to actDevelops and sustains a culture of inquiry in the school based on data

64

CFIP Step 5: Drill Down to Individual

Students. Identify and Implement Needed Enrichments and Interventions.

What are the implications for enrichments and interventions from

what you learn from the data? Which students need enrichments and interventions?What should enrichments and interventions focus on?

65

CFIP Step 6: Reflect on the reasons for student performance -- What in our teaching might be

preventing all students from being successful?

To what extent did we implement research-based instructional practices as we:

Planned instruction? Introduced instruction? Taught the unit? Brought closure to instruction? Assessed formatively?

66

CFIP Step 6: Reflect on the reasons for

student performance. Identify and implement instructional changes

in the next unit.

How will we change instruction in our next unit?Content focusPacingTeaching methodsAssignments

67

CFIP Step 6: Reflect on the reasons for

student performance. Identify and implement instructional changes

in the next unit. When will we review the data again to determine the success of the enrichments, interventions, and instructional changes?What do the data not tell us? What questions about student achievement do we still need to answer? How will we attempt to answer these questions?

68

The Next Steps

1. Unless teams emerge from the data analysis process with a clear plan of action for their classroom, they have wasted their time.

2. Implement the plan of interventions, enrichments, and changes in instruction.

3. Collect the next set of data.

69

Where does a school go from here in becoming more data-driven?

The Drivers The Barriers

DISCUSSION: What drivers and barriers would you see schools facing in implementing the CFIP model?

70

Typical School Improvement Plan (SIP)

Classroom Focused Improvement Process (CFIP)

Process established at district level

Process designed at team level

Linear and prescriptive Non-linear/non-prescriptive

Annual strategic plan Short-cycle operational plan

Impact: total school Impact: students in class

SIT develops Classroom-level team develops

Purpose: meet AYP Purpose: adjust practice

Results determined end of year

Results determined when unit is taught

71

So what about the School Improvement Team?

The School Improvement Team (SIT) as typically constituted is designed to do exactly what its name implies: IMPROVE THE SCHOOL. It is not designed to improve teaching and learning at the classroom level. That is the focus of the content or grade-level team or the department.

72

Core Functions of the SIT

• Keep the vision alive.• Develop and monitor school-wide plan for

meeting state accountability standards.• Build a data-driven culture.• Establish priority focus on instruction.• Provide a safe and supportive environment

for all students.• Connect school with parents and

stakeholders.• Provide needed resources.

73

Caveats about CFIP

• It is a paradigm shift from traditional lesson planning format.

• It is not easy, especially at first.• Follow the steps faithfully until they

become second nature.• The CFIP is a guide until you make the

process your own.• Expect mistakes and imprecision in the

data.• The results are worth the effort.

74

Coming together is a beginning,

staying togetheris progress,

and working togetheris success.

- Henry Ford