Download - Data informed decision-making
Learning-Centered Leadership Development Program for Practicing and Aspiring PrincipalsWestern Michigan UniversityKalamazoo, MI 49008
A Project funded by the United States Department of Education (USDOE), Washing, DC: 2010
Module 1: Data-Informed Decision-Making
2
INTRODUCTION
• Do you Believe in me? - Dalton Sherman• Reflections• Learning Objectives
3
Dalton Sherman
http://www.youtube.com/watch?v=HAMLOnSNwzA
4
Reflection Upon Dalton Sherman’s Speech• Do staff in your school believe that all students can learn?• What does this belief look like in your school?• How do you know that all students are learning?• What changes do you need to make to align practices with
beliefs?
5
Learning ObjectivesAs a consequence of participating in this module, participants will:• Understand and experience the importance of data in a continuous
improvement cycle;• Utilize a data mining tool D4SS (Data for Student Success, MDE) that will
equip you with an understanding about how to disaggregate student data and identify learning gaps in students performance by gender, ethnicity, SES, and learning impairments;
• Learn from other practicing and aspiring school leaders about the effect and challenges of their evidence-based instructional initiatives; and
• Develop and implement a renewal activity in a high priority content area that is designed to improve student achievement.
6
Conceptual Framework for Date-Informed Decision-Making
• What is Data?• Putting your Fear on the Table Regarding the Use of Data• Conceptual Model for Data Use• Collaborative Inquiry Process
7
What is Data?
8
Examples of Data
Numbers Opinions Observations Essays Science projects Demonstrations and …….
Data Can Answer These Questions
1. How are we doing?
2. Are we serving all students well?
3. In what areas must we improve?
Other Questions?
9
10
Principals’ Perception Regarding the Use of Data
Of Teachers
Teachers uncomfortable with data
Teachers cannot read data Data has meaning to classroom Do not know what to do with
data Data not part of teacher training Lack of knowledge data-
instruction No data link to teaching
practices
Of Themselves
Do not understand data use What data do you use Teacher collection of data Need systematic disaggregation Find better assessment tools PD for teachers and principals
11
Principals’ Perception of Time Constraints to Analyze Data
Time to complete tasks Data vs. classroom duties Limited instructional time Time to analyze data Time for collaboration
Time to monitor teacher use Time in getting test results Time in getting data back A year behind-results Holistic approach in working
with teachers
12
Principals’ Perception of Teacher and Student Issues
Teacher cooperation in assessment
Teacher- team cynicism Teachers see data as important Teacher-staff cooperation in data
assessment Inconsistent teacher collection of
student data Student do not take testing
seriously A few teachers see testing as a fad Utility of data
No relevance to individual
students
Results do not reflect current
students
Needs to make sense
Students mirror teacher attitude
Too much student testing
Teacher buying into data
Unsure if data use beneficial
Quality of instruction
No consistence in teacher use of
tools
Put Your Fears on the Table
No,I don’t see
any problem with the data!
What concerns you most about using data to make school decisions?
Internal?
External?
Do the following concerns sound familiar? 13
“Putting data on the table will damage union
negotiations.”
Fear of Data
“My questions about data will sound silly.”
“Will we get sued if we look at student data? What
about privacy issues?”
“Can we trust the data? What if the
numbers are ‘cooked’?”
“If people know the truth about how our
district is doing, we’ll get pummeled.”
14
“I don’t understand the
data.”
“People will take the data out of context to
further their own agendas.” ?
Take Away Your Fear
You don’t have to be a statistician Data are actionable Data must be viewed in relationship to something else Data should be used to establish a focus of inquiry
15
16
School Processes
Description ofSchool Programsand Processes
PerceptionsPerceptions of
Learning Environment
Values and BeliefsAttitudes
Observations
Enrollment, Attendance,Drop-Out Rate
Ethnicity, Gender,Grade Level
Demographics
Standardized Tests
Norm/Criterion-Referenced Tests
Teacher Observations of Abilities
Authentic AssessmentsStudentLearning
Multiple Measures of Data
Demographics Perceptions Student Learning
Demographics- Gender- Grade- Teacher- Age- Time in Building- Behavior- Attendance- Poverty Level- Racial/ethnic- Socioeconomic- Single Parent- Siblings in household- Free/Reduced Lunch
Parental Involvement- Preparedness- Transience-Out of school experiences
Community Support- Programs e.g., Head Start- Services e.g, FIA
Opportunity to Learn- Current Offerings- Extra Curricular Activities
Teacher quality- Qualifications & Credentials- Instructional Practices- Professional Development- Collective Efficacy - Learning Communities- Professional Affiliations
Leadership- Vision, Mission, Goals- Staff Engagement & Perceptions- Parent Engagement & Perceptions- Supervision Practices- Professional Affiliations
Resource Allocation- Budget Allocation- Staffing Patterns - Professional Development- Facility Usage/Maintenance- Technology Distribution
Results Data (Static Data)- MEAP/MME- ACT- AP Testing- District Benchmark Assessments- Standardized Assessments- Graduation Rate- Postgraduate Follow-up
Process Data (Real-Time Data)- Instructional Strategies- Classroom Assessments- Instructional Time on Task- Behavioral Referrals- Books- Writing Samples- Homework Assigned/Completed- Positive Parent Contacts
School Processes
Perception Data- Student Engagement- Student motivation- Student perceptions of success- Values- Beliefs- Culture- Attitudes- Observations
Data Streams Examples
17
1. My school has a written vision that focuses on student achievement.
❏ Yes No I Don’t Know❏ ❏
6. My school is willing to explore ways to use data to measure progress.
❏ Yes No I Don’t Know❏ ❏
2. My school has a general awareness about why data are significant.
❏ Yes No I Don’t Know❏ ❏
7. Everything my school does aligns with our vision.
❏ Yes No I Don’t Know❏ ❏
3. My school has a mission statement that reflects core values and beliefs.
❏ Yes No I Don’t Know❏ ❏
8. My school knows that staffs role is using data to improve student achievement.
❏ Yes No I Don’t Know❏ ❏
4. My school agrees data shows evidence of progress in achieving student goals.
❏ Yes No I Don’t Know❏ ❏
9. My school uses data to set goals? ❏ Yes No I Don’t Know❏ ❏
5. My school has stated, measurable goals that are tied to our vision.
❏ Yes No I Don’t Know❏ ❏
10. My school make decisions based on data based research?
❏ Yes No I Don’t Know❏ ❏
Are We Ready To Use Data More Effectively?
18
19
Types of data
• Input• Process• Outcome• Satisfaction *Set and assess progress toward goals
*Address individual or group needs*Evaluate effectiveness of practices*Assess whether client needs are being met*Reallocate resources in reaction to outcomes*Enhance processes to improve outcomes
Information Actionable knowledge
District School Classroom
SOURCE: Marsh, J. A., Pane, J. F., and Hamilton, L. S. (2006). Making sense of data-driven decision making. Rand Corporation. p. 3.
Conceptual Framework for Data Use
20
Duh!!!
School
Improvement
21
Establishing the Bridge for Student Improvement
Collaborative Inquiry
22
Enabling Collaborative Work
• Schools have an abundance of data. There is the propensity of school officials to relegate the technical work in organizing data to a small group of individual – i e., principal, principal and select teachers, or data specialist.
• This responsibility needs to be shared among all teachers, and ideally, among all members of the school community.
• It is quite apparent that when people are involved in analyzing and interpreting data collaboratively, they become more invested in the school improvement efforts that are generated out of those discussions.
• The more people involved in data analysis and interpretation, the more effective the resulting school improvement efforts will be.
23
John Dewey
What the best and wisest parent wants for his own
child, that must the community want for all of
our children.
24
It’s Easy to Get Lost in the Numbers
63542
63542
37620
609155629098625
8762987620
980098
89365and forget that the numbers represent the hope and future of real children with strengths as well as challenges,
each deserving the kind of education we want for our very own children
Bridging the Data Gap
Imagine two shores with an river in between. On one shore are data—the masses of data now
overwhelming schools:
On the other shore are the aspiration, intention, moral assurance, and directive to improve student learning and close repetitive achievement gaps.
course-taking patterns attendance data survey data and on and on graduation rates
state test data sliced and diced local assessments demographic data dropout rates
25
How do we get there?
26
I saw this new reading program at
the State conference, let’s
try it, it can’t hurt!
If we put more resources into
“Bubble Kids” our scores will
improve
It is evident that those kids cannot learn as efficiently
as others
ResultsData
? ? ?
27
ResultsData
Build and identify the parts of a bridge that is needed to get Data to the Results?
Collaborative Inquiry Is The Bridge
• Schools know that they have to improve• But they often do not know how to improve• Collaborative inquiry is the how• As collaborative inquiry grows, schools shift away from
traditional data practices and toward those that build a high-performing culture of data use
• When engaged in collaborative inquiry, Data Teams investigate the current status of student learning and instructional practice and search for successes to celebrate and amplify.
28
29
Setting the Stage for Collaborative Inquiry
Participants’ Activity:• In this particular activity, participants will discuss the type of
external and internal data they use in their schools. After this participants will identify trends associated with these administrations.
• On post- it notes, participants will be asked to make the following observations and report out in groups the following questions:
30
Group Activity
Question:
1. Is there one particular data type (external or internal) used more often than the other? If so, why?
2. What decisions are made by the use of external and internal data tools? Who make these decisions (by data type)?
3. To what extent is there a close relationship between the gaps in student learning, as identified by the data types, and the initiatives that were developed?
4. What challenges are you facing implementing the initiative in your school?
5. To what extent is the initiative producing the intended results you originally sought? How do you know? What data are you using? If you are not getting the desired results, what are you doing about it?
Demographics Perceptions Student Learning
Demographics- Gender- Grade- Teacher- Age- Time in Building- Behavior- Attendance- Poverty Level- Racial/ethnic- Socioeconomic- Single Parent- Siblings in household- Free/Reduced Lunch
Parental Involvement- Preparedness- Transience-Out of school experiences
Community Support- Programs e.g., Head Start- Services e.g, FIA
Opportunity to Learn- Current Offerings- Extra Curricular Activities
Teacher quality- Qualifications & Credentials- Instructional Practices- Professional Development- Collective Efficacy - Learning Communities- Professional Affiliations
Leadership- Vision, Mission, Goals- Staff Engagement & Perceptions- Parent Engagement & Perceptions- Supervision Practices- Professional Affiliations
Resource Allocation- Budget Allocation- Staffing Patterns - Professional Development- Facility Usage/Maintenance- Technology Distribution
Results Data (Static Data)- MEAP/MME- ACT- AP Testing- District Benchmark Assessments- Standardized Assessments- Graduation Rate- Postgraduate Follow-up
Process Data (Real-Time Data)- Instructional Strategies- Classroom Assessments- Instructional Time on Task- Behavioral Referrals- Books- Writing Samples- Homework Assigned/Completed- Positive Parent Contacts
School Processes
Perception Data- Student Engagement- Student motivation- Student perceptions of success- Values- Beliefs- Culture- Attitudes- Observations
Data Streams Examples
31
32
BREAK
15 Minutes
33
Creating A Data Team
A data team is a team that meets regularly to analyze data and make educational decisions to improve student achievement.
THE DATA TEAM PROCESS1. Collect and Chart
Data
2. Analyze Data and Prioritize
Needs
3. Establish SMART
Goals
4.Select Instructional
Strategies
5. Determine Results
Indicators
Source: Allison, E. et al.. (2010). Data teams. Lead + Learn Press.: Englewood, CO.
6. Monitor and
Evaluate Results
Data Teams
The data team members must:• Be seen as leaders.• Be willing to learn about data in depth.• Must have skills in collaboration,
communication, and leadership.
The functions of the data team are:• To develop expertise on data.• To share data information with staff members
of the schools.• To assist is setting up support systems at the
schools.• To create and complete action plans based on
the data.
The data team members need:• Information about systems to support data
based decision making.• Training in the problem solving process.
Data team members should be expected to:• Meet regularly as a team to develop a plan to
establish using data to improve student performance.
• Have conversations about student achievement.
• Show examples of successful schools.• Set up a system that supports the collection
and use of student data.• Help staff members understand how to use
student data to guide decision making.• Work to secure commitment from staff
35
36
D4SS
Data Analysis Activity
http://www.data4ss.org
User Name: demo_test1Pass Word: fall_01
Data Narrative Statements Criteria
Data Narrative Statements are objective statements of FACT about the school data
They:
1. Represent student achievement, demographics, school programs, school processes, and stakeholder perceptions
2. Communicate a SINGLE idea
3. Are clear and concise – written in sentences or phrases
4. Describe the data; they do not evaluate the data!
5. MUST stand alone; they do not require the data source to accompany them in order to be understandable.
37
Data Narrative StatementsDo they meet criteria from Previous Slide?
Narrative Statement 1 2 3 4 5
1- Spring 2010 Math Assessment shows that our girls do slightly better than the boys.
2- The Spring 2010 Math Assessment shows that 20.5% of our 11th grades students were proficient and 79.5% were not.
3- The Spring 2010 Math Assessment shows that we really need a new math series.
4- In 2009-2010 21.4 % of all our students taking the Math Assessment are proficient; while 20.5% of our 11th graders are proficient and 33.3% of our 12th graders are proficient.
5- Parents do not like the math program.
38
Year: _________ Building: _________________________
Which grade level(s) is not meeting the criteria for grade level proficiency?
What do we need to know more about?
%Proficient
%Proficient
%Proficient
AYP Target (see slide 47
for AYP Target)
%Proficient
AYP Target(see slide 47
for AYP Target)
Content Area Reading Writing Total ELA ELA Math Math
Overall Building – All
Students
Grade 3
Grade 4
Grade 5
Grade 6
Grade 7
Grade 8
39
MEAPBuilding: Content Analysis
MEAPOverall Building Sub-Group Level Achievement Analysis
GradeNumber
of Students
% Proficient
Number of
Students%
ProficientNumber
of Students
% Proficient
AYP Target(see
slide 47 for AYP Target)
Number of
Students%
Proficient
AYP Target(see slide 47 for AYP
Target)
Content Area Reading Reading Writing Writing Total ELA Total ELA ELA Math Math MathAmerican Indian or Alaska Native
Black or African American
Hispanic or Latino
White Asian American, Native Hawaiian or other Pacific
Islander
Multiracial
Economically Disadvantaged
Students with Disabilities
Limited English Proficient
Non AYP-Migrant
40
MEAPSub-Group Analysis
Sub-Group Level Achievement: Choose sub-group for analysis
Year ___________ Group: ____________________
Grade Number of Students
% Proficient
Number of Students
% Proficient
Number of Students
% Proficient
AYP Target (see
slide 47 for AYP Target)
Number of Students
% Proficient
AYP Target(see
slide 47 for AYP Target)
Content Area Reading Reading Writing Writing Total ELA Total ELA ELA Math Math Math
Building
3
4
5
6
7
8
41
Date: _______________________________
Building: ____________________________
Data Team Members: __________________________________________________________________________________
________________________________________________________________________________________________________________
42
Narrative Statement:
MEAPData Narrative Statements for Sub-Group Analysis
Year: _________ Building: _________________________
MMEBuilding: Content Area Proficiency
In which subject area(s) is your building not meeting the criteria for proficiency?
What do you need to know more about?
%Proficient
%Proficient
%Proficient
AYP Target(see slide 47
for AYP Target)
%Proficient
AYP Target (see slide 47
for AYP Target)
Content Area Reading Writing Total ELA ELA Math Math
Overall Building – All Students
Grade 11
Grade 12
43
MMEOverall Building Sub-Group Level Achievement Analysis
GradeNumber
of Students
% Proficient
Number of
Students%
ProficientNumber
of Students
% Proficient
AYP Target(see
slide 47 for AYP Target)
Number of
Students%
Proficient
AYP Target(see slide 47 for AYP
Target)Content Area Reading Reading Writing Writing Total ELA Total ELA ELA Math Math Math
American Indian or Alaska Native
Black or African American
Hispanic or Latino
White Asian American, Native Hawaiian or other Pacific
Islander
Multiracial Economically
Disadvantaged
Students with Disabilities
Limited English Proficient
Non AYP-Migrant
44
MME Sub-Group Analysis
Sub-Group Level Achievement: Choose sub-group for analysis
Year ___________ Group: ____________________
Grade Number of Students
% Proficient
Number of Students
% Proficient
Number of Students
% Proficient
AYP Target(see
slide 47 for AYP Target)
Number of Students
% Proficient
AYP Target (see
slide 47 for AYP Target)
Content Area
Reading Reading Writing Writing Total ELA Total ELA ELA Math Math Math
Building
45
Date: _______________________________
Building: ____________________________
Data Team Members: ___________________________________________________________________________________
________________________________________________________________________________________________________________
46
Narrative Statement:
MMEData Narrative Statements for Sub-Group Analysis
Michigan Annual AYP Objectives
47
Reflections On Today’s Session
1. What do you remember from today's session (scenes, events, and conversations)?
2. What words are still ringing in your ears?
3. What image captures for you the emotional tone of today's session?
4. What is a key insight from today's session?
5. What name would you call today's session? (Try a poetic title that captures your responses.)
48
49
End
Session 1
50
ResultsData
ContinuousImprovementData UseCollaborationLeadership
Capacity
TrustCultureEquity
51
PANEL DISCUSSION WITH DATA EXPERTS
Kathryn Parker Boudett
http://bcove.me/v4ccqy6q
Joel Klein
http://bcove.me/byt742or
Rudy Crew
http://bcove.me/emn3jz77
Martha Greenway
http://bcove.me/4cf0bkzq
Aimee Guidera
http://bcove.me/trsldt2h
Dan Katzir
http://bcove.me/gy3wbiac
http://www.edweek.org/ew/section/video-galleries/april10-event-data.html
Sm
art
Go
als
Data Teams
Summative Assessments
Perceptual
Demographic
Achievement
Formative Assessments
Ali
gn
me
nt
Qu
estion
s and
Inq
uiry
Process
Revised Instructional Strategies
Revised Instructional Strategies
DataTeam
s
Data Feedback Model
Dis
tric
t W
ritt
en a
nd
Ta
ug
ht
Cu
rric
ulu
m
Dat
a In
ters
ecti
on
A
nal
ysis
53
54
55
56
57
58
59
60
61
62