using observation to analyze effectiveness marybeth flachbart, ed.d [email protected]
TRANSCRIPT
WWW.NEUHAUS.ORG
OBJECTIVES
3
1. Review research
2. Clarify types of walk-throughs
3. Identify areas of need through observational data
RESOURCES USED:LEADING FOR INSTRUCTIONAL IMPROVEMENT, FINK, MARKHOLT, 2011VISIBLE LEARNING, HATTIE, 2009 VISIBLE LEARNING FOR TEACHERS, HATTIE, 2012
4
LEADING FOR INSTRUCTIONAL IMPROVEMENTSTEPHEN FINK, ANNEKE MARKHOLT
5
6
VISBILE LEARNING: A SYNTHESIS OF OVER 800 META-ANALYSES RELATING TO ACHIEVEMENT
John Hattie, Professor of Education and Directorof the Visible Learning Labs, University of Auckland
EFFECT SIZE MOVES US BEYOND DID IT WORK
7
• Effect size is a way of quantifying the size of the difference between two groups.
• How well does it work? • Generally used in meta-analysis for
combining and comparing estimates from different studies.
BASIS OF HATTIE’S CONCLUSION
800+ META-ANALYSIS 50,000 STUDIES, AND 200+ MILLION STUDENTS
HATTIE’S RANGE OF EFFECT SIZE
9
.07
Effect Size
Benchmark .40
Enhanced – closer to 1.0
10
VISBILE LEARNING: A SYNTHESIS OF OVER 800 META-ANALYSES RELATING TO ACHIEVEMENT
John Hattie, Professor of Education and Directorof the Visible Learning Labs, University of Auckland
11
Raise your hand if you’ve read any or all of these books
12
If you have read any of the books:
Which one(s)?
What did you think of it?
Please provide answers in the comment box
DISPELLING THE WIDGET MYTH
13
THE WIDGET EFFECT, NEW TEACHER PROJECT, 2009
• Surveyed 14 large American School Districts• 98% of teachers were evaluated as “satisfactory”
“Based on such findings, many have characterized classroom observation as a hopelessly flawed approach to assessing teacher effectiveness. “
14
http://widgeteffect.org/
CAN CLASSROOM OBSERVATIONS IDENTIFY PRACTICES THAT RAISE ACHIEVEMENT?
15
http://educationnext.org/evaluating-teacher-effectiveness/
YES!
17
Defining Classroom Management• Room setup• Access to materials• Transition time• Procedures • Grouping• Climate
EFFECT SIZE:CLASSROOM BEHAVIORAL .80
CLASSROOM MANAGEMENT .52
18
19
Defining Instructional Practice• Link to standards• Use of questioning• Teacher led vs. student led• Cognitive demand of selected activities• Differentiation • Assessment practices• BICS vs. CALP
EFFECT SIZE:
FORMATIVE ASSESSMENT .90MICRO TEACHING .88TEACHER CLARITY .75RECIPROCAL TEACHING .74META-COGNITIVE STRATEGIES .69COOPERATIVE LEARNING .59QUESTIONING .64
20
21
Which is easier to observe, classroom management or instructional practices?
Respond in comment box
GATHERING DATA (FINK, MARKHOLT)
22
Types of walk-throughs• Learning walk• Goal setting• Implementation• Supervisory• Other
YOUR PRACTICE
23
Which types of walk-through do you do most frequently? Why?
1. Learning walk
2. Goal setting
3. Implementation
4. Supervisory
5. Other
OBSERVATIONAL DATA IS NOT EVALUATIVE!
“Stick to the facts m’am, just the facts.”
A Profession in Search of a Practice City and Elmore 2009
26
Developing the discipline of seeing• Seeing is a discipline• It is like a muscle – it gets stronger with
repetition• Foundation of Instructional Rounds
Creating Literacy Leaders Observation Protocol Classroom Observation Date: _____________ Grade Level: _____________________ Subject: ________________________________
Teacher Name: ___________________________________ Begin: _________________ End: _____________ Indicator Estimate % Teacher Directed Observer: _______________________________________________________ Student Directed Independent Work Other Total Y = Yes – Observed or Present N = No – Not observed or no occasion to be observed Classroom Management/Instruction Est.
Time Teacher Directed Whole Class/ Teacher Directed Small Group Est. Time
When waiting for assistance from the teacher, students are occupied with literacy related activities provided by the teacher.
Y N
Teacher clearly states the lesson’s topic, theme, and objectives. (Objective is linked to standards)
Y N
Teacher uses modeling, demonstration, and graphics. Y N
Teacher uses a variety of instructional modes. Y N Teacher explains directly and thoroughly. Y N Transitions between whole group and small group are brief and orderly.
Y N
Teacher uses prompting/cueing. Y N
Teacher has a Reading block schedule posted in the room. Y N Teacher reviews with questioning. Y N Students are engaged and on task. Y N Teacher Student Interaction Teacher re-teaches following questioning. Y N Student- Directed Groups/ Independent Work Teacher uses open-ended questions and encourages
elaboration. Y N
Teacher interacts instructionally with students (explaining, checking, giving feedback). Y N
Teacher encourages peer interaction. Y N
Teacher interacts managerially with students (reinforcing rules, procedures).
Y N Teacher encourages students to paraphrase, summarize, and
relate. Y N
Teacher encourages students to help each other with their work.
Y N
Teacher encourages students to check their own comprehension.
Y N
Observation Statistics (mark all that apply) Teacher encourages students to speak in complete sentences. Y N Observer present when lesson began? Y N Observer present when lesson ended? Y N Other Activity: Observer present during middle of lesson? Y N # of Students _____________ Aide present? Y or N How many?_____________ (If present, please make note of aide activity)
Using the Observation Protocol collect data that is:
• Descriptive not evaluative- just the facts!
• Specific
• About instructional core
• Related to problem of practice (or data needed to be collected)
OBSERVING CLASSROOM PRACTICE
“Teacher introduced the concept of fractions and students applied it in a hands-on activity.”
vs.
“Prompt for student essays: “What role did symbolism play in foreshadowing the main character’s dilemma?”
SPECIFICITY OF EVIDENCE
“Student 1 asked student 2: ‘what are we supposed to write down?’ Student 2 said, ‘I don’t know.’”
“The teacher read from a book that was not at the appropriate level for the class.”
“Teacher introduced the concept of fractions and students applied it in a hands-on activity.”
DESCRIPTION VS. JUDGMENTWITH
WITHOUT
“Students followed directions in the text to make circuit boards.”
“There was too much time on discussion, not enough time on individual work.”
“The students conducted a sophisticated lab experiment.”
DESCRIPTION VS. JUDGMENT
32
• What is the teacher doing and saying?
• What are the students doing and saying?
• What is the task?
WHEN OBSERVING, WHAT DO YOU SEE?
33
Possible data points:• Percentage of teacher vs. student talk• Formative assessment practices• Standard clear to students• Meta-cognitive strategies• Number of questions (types of questions)• Who is doing the questioning?
LEARNING WALK
LEARNING WALK CONTINUED
34
• Amount of time spent reading/writing/researching
• Cognitive demand of student work• Academic Vocabulary use• Level of engagement
What else have you learned by walking around?
TEACHER CHANNEL VIDEOS
HTTPS://WWW.TEACHINGCHANNEL.ORG.
35
36
PRACTICE:
Observing:
What do you see?• What is the teacher doing?• What are the students doing?
WHAT DO YOU WONDER?WHAT QUESTIONS WOULD YOU HAVE
FOR THE TEACHER?WHAT QUESTIONS WOULD YOU HAVE
FOR THE STUDENTS?
37
GOAL SETTING WALK THROUGH
38
GATHER DATA BEFORE YOU SET THE GOAL!
GOAL: IMPROVE WRITING ACHIEVEMENT BY 10%
40
Define the problem of practice:
Is it a lack of instruction in writing?
Is the lack of instruction in mechanics, process, or both?
Is it a lack of practice opportunities to write across the content areas?
Is it a lack of vocabulary?
Are the vocabulary deficits in
oral or written language?
SUPPOSE YOUR STUDENTS WRITING SKILLS WERE WEAK, AND YOU THEN OBSERVED INSTRUCTION…
41
“Based on my observation of 15 classrooms (visited once a week for three weeks), we need to improve the quality of interaction between faculty and students. We need to model and require academic language in both our oral and written communication.”
42
“If we model and then scaffold our students ability to use academic language, we will see an improvement in our students reading, writing, and speaking proficiency.”
THEORY OF ACTION
43
IMPLEMENTATION WALK-THROUGH
IMPLEMENTATION WALK
45
• When do you want to see this change in practice?
• How frequently will you gather implementation data?
• Who will gather the data?• How will that refine your theory of action?• Grade level, whole school, individual
teacher?
WHAT IF YOU SEE VERY LITTLE IMPLEMENTATION?
THOU SHALT…
47
• Only schedule professional development for faculty that you can measure in terms of a change in instructional practice
• Prepare a standard protocol for faculty and share it before observation
• Give sufficient time to implement and resources for job embedded coaching
BE SPECIFIC!
CLARIFY:
49
Message from observer:
Last Friday we had an in-service day where faculty shared their strategies for teaching writing across content areas. I am anxious to see how you’re incorporating these practices in your classroom. I will be doing walk-throughs beginning next week to see how the students are responding to their new learning. Attached is a form I’ll be use to gather data so that we can discuss student work at our PLCs in November. I look forward to seeing you in action.
INSPECT WHAT YOU EXPECT
50
Ask professional development providers (including staff) for an observation protocol
“If you can’t see it in the classroom, it doesn’t exist.”
Elmore
Inquiry Cycle: Habits of Thinking for Instructional Leadership
• Noticing – Describe specific facts related to the Observation Protocol that you recorded.
• Wondering – Based on what you saw (or didn’t see), what are you curious or would like to know more about?
• Analysis – Using what (collective) knowledge is known about what learning should look like in the ideal in relation to this context observed, think about this information as it compare with my noticing and wonderings.
• Theories – Based on multiple observations across all classrooms and preponderance of data, this is one theory (or problem of practice) about student learning is …..
• Leadership Implications – Using Theory as a guide this is one implication for professional development and my decisions as a leader. ( Gathering additional information and data to measure effectiveness)
TICKET OUT
As a result of this presentation what if anything will you add to your classroom observations?
Email response to:[email protected]