consider the evidence tpu ti cs
DESCRIPTION
TRANSCRIPT
Consider the Evidence
Data analysis and evidence-driven decision making
for TIC’s Teen Parent Units
27 August 2009
Naomi KinnairdCentre for Educational Development
Insert Pic
Evidence-driven decision making
Today we aim to :• consider how we use data and other evidence to
improve teaching, learning and student achievement
• use evidence to support an inquiry based approach
• improve our understanding, confidence and capability in using data to improve practice
• set our own evidence-based goal for learning in our TPU
Consider
1. What data do you have/collect?
2. Why do you collect this data?
3. How do you use this data?
Evidence-driven decision making
Just having data offers very little. For a skilled
leader, however, data can be a compelling force for improving schools. The value associated with data come from being able to discern the quality of the data and to organise it, think about what it might mean and use it to make decisions. This is a human activity that requires capturing and organising ideas and turning the information into meaningful actions (Senge, 1990).
Evidence-driven eatingYou need to buy lunch. Before you decide
what to buy you consider a number of factors: • how much money do you have?• what do you feel like eating?• what will you be having for dinner?• how far do you need to go to buy food?• how much time do you have?• where are you going to eat it?
Evidence-driven teachingI had a hunch that Ana wasn’t doing as well as she could in her
research assignments, a major part of the history course. What made me think this?
Ana’s general work (especially her writing) was fine. She made perceptive comments in class, contributed well in groups and had good results overall last year, especially in English.
How did I decide what to do about it? I looked more closely at her other work. I watched her working
in the library one day to see if it was her reading, her use of resources, her note taking, her planning, or what. At morning tea I asked one of Ana’s other teachers about Ana’s approach to similar tasks. I asked Ana if she knew why her research results weren’t as good as her other results, and what her plans were for the next assignment.
I thought about all of this and planned a course of action. I gave her help with using indexes, searching, note taking and planning and linking the various stages of her research.
Teacher inquiry
Evidence• Any facts, circumstances or perceptions that can
be used as an input for an analysis or decision
• how classes are compiled, how classes are allocated to teachers, test results, teachers’ observations, attendance data, portfolios of work, student opinions …
Data are one form of evidence
Data• Known facts or measurements, probably expressed
in some systematic or symbolic way (e.g. as numbers)
• assessment results, gender, attendance, ethnicity …
Data are one form of evidence
Evidence-driven decision making
We have more evidence about what students know and can do than ever before - their achievements, behaviours, environmental factors that influence learning
We should draw on all our knowledge about the learning
environment to improve student achievementexplore what lies behind patterns of achievement decide what changes will make a difference
What evidence does a school have?
• Demographics
• Student achievement
• Perceptions
• School processes
• Other practice
DemographicsWhat data do we have now to provide a profile
of our school? What other data could we create?
School Students Staff Parents/caregivers and community
Student achievementWhat evidence do we have now about student
achievement?What other evidence could we collect?
National assessment results Standardised assessment results administered
internallyOther in-school assessmentsStudent work
PerceptionsWhat evidence do we have now about what
students, staff and others think about the school?Are there other potential sources?
Self appraisal Formal and informal observations made by teachers Structured interactions Externally generated reportsStudent voice Other informal sources
School processesWhat evidence do we have about how our school is
organised and operates?
TimetableClasses Resources Finance Staffing
Other practiceHow we can find out about what has worked in other
schools?
Documented researchExperiences of other schools
The evidence-driven decision making cycle
Trigger Clues found in data, hunchesExplore Is there really an issue?Question What do you want to know?Assemble Get all useful evidence togetherAnalyse Process data and other evidenceInterpret What information do you have?Intervene Design and carry out actionEvaluate What was the impact?Reflect What will we change?
The evidence-driven decision making cycle
cycle EXPLORECheck data and evidence to explore the issue
QUESTIONClarify the issue and ask a question
ASSEMBLE Decide what data and evidence might be useful
ANALYSE data and evidence
TRIGGERData indicate a possible issue that could impact on student achievement
EVALUATE the impact on the intervention
INTERVENEPlan action to improve student achievement
INTERPRET Insights that answer your question
SPECULATEA teacher has a hunch about a problem or a possible action
ACTCarry out the intervention
REFLECT on what has been learned, how practice will change
The evidence-driven decision making cycle
Explore data Survey of students shows that this is only partially true
QuestionWhat are the characteristics of students who are poor at reading?
Assemble more data & other evidence: Probe reading, homework, extracurricular,attendance, etc.
Analyse asTTLe reading (standardised) results.
TriggerSignificant numbers not achieving well in reading.
Analyse non standardised data and evidence
InterveneCreate multiple opportunities for reading; include topics that can use sport as context; connect reading with curriculum areas. PD for staff.
Interpret informationPoor readers likely to play sport, read less, do little homework.
A teacher has a hunch - poor readers might spend little time on homework.
Evaluate Has reading improved?
ReflectHow will we teach reading in the future?
The length of the cycle will vary for different situations. We might wait a year to evaluate the effects of our actions - but sometimes we’ll be able to (and ought to) work to shorter (or maybe longer) cycles.It’s important that we reflect, evaluate and make professional judgements at each stage of this cycle.
Now we will invent scenarios that might apply in our school (or department). Do this in groups. Draw up your scenario as a cycle as in this slide.
The next slide provides a blank template for this exercise. You might like to photocopy this template for groups to fill in. A sample scenario is given in the following slide.
The length of the cycle will vary for different situations. We might wait a year to evaluate the effects of our actions - but sometimes we’ll be able to (and ought to) work to shorter (or maybe longer) cycles.It’s important that we reflect, evaluate and make professional judgements at each stage of this cycle.
Now we will invent scenarios that might apply in our school (or department). Do this in groups. Draw up your scenario as a cycle as in this slide.
The next slide provides a blank template for this exercise. You might like to photocopy this template for groups to fill in. A sample scenario is given in the following slide.
The evidence-driven decision making cycle
EXPLORE
QUESTION
ASSEMBLE
ANALYSE
TRIGGER
EVALUATE
INTERVENE INTERPRET
SPECULATE
ACT
REFLECT
Evaluate and reflectSummative evaluation – assess how
successful the intervention was; decide how our practice will change; report to board
Formative evaluation - at every stage in the cycle we reflect and evaluate
Are we are on the right track? Do we need to fine-tune?Do we actually need to complete this?
Types of analysisWe can compare achievement data by
subject or across subjects for • an individual student• groups of students• whole cohorts
The type of analysis we use depends on the question we want to answer
Trigger questionsHow good/poor is …?What aspects of … are good/poor?Is … actually changing?How is … changing?Is … better than last year?How can … be improved?Why is … good/poor?What targets are reasonable for …?What factors influence the situation for …?What would happen if we …?
Formative or summative?
Questions from hunchesI suspect this poor performance is being caused by
… Is this true?
We reckon results will improve if we put more effort into ... Is this likely?
I think we’d get better results from this module if we added … Is there any evidence to support this idea?
Questions with purposeWhat do we know about attendance for TPU
students?
MAY BE BETTER ASWho has been absent? When? Why? Where have they been? How long?What are students telling us?What does pastoral care data tell us? Were some interventions more effective with some students / groups of students than others?
Professional decision makingWe have evidence-based information that we see as
reliable and valid
What do we do about it?
If the information indicates a need for action, we use our collective experience to make a professional decision
Professionals making decisionsYou asked what factors are related to poor
student performance in formal writing. The analysis suggested that poor homework habits have a significant impact on student writing.
You make some professional judgements and decideStudents who do little homework don’t write enoughYou could take action to improve homework habits -
but you’ve tried that before and the success rate is low
You have more control over other factors - like how much time you give students to write in class
So you conclude - the real need is to get students to write more often
Deciding on an actionInformation will often suggest a number of options
for action. How do we decide which action to choose?
We need to consider what control we have over the actionthe likely impact of the actionthe resources needed
Planning for evaluationWhat evidence do we need to collect before we
start?Do we need to collect evidence along the way, or just
at the end? How can we be sure that any assessment at the end
of the process will be comparable with assessment at the outset?
How will we monitor any unintended effects? Don’t forget evidence such as timetables, student
opinions, teacher observations …
Evaluate the impact of our action
Did the intervention improve the situation that triggered the process?
Was any change in student achievement significant? What else happened that we didn’t expect? How do our results compare with other similar studies we can find?
Does the result give us the confidence to make the change permanent?
Future practice
What aspects of the intervention will we build into future practice?
What aspects of the intervention will have the greatest impact?
What aspects of the intervention can we maintain over time?
What changes can we build into the way we do things in our school?
Would there be any side-effects?
What now?
How can we apply this model in our TPU?
Develop a specific task to review / implement (in regard to evidence / data) for learning in your TPU.
What evidence already exists?
How is this collected / recorded / analysed and used in the interests of improving student achievement?