1 you’re the judge now…getting the scores right calibration training, continued
Post on 24-Dec-2015
213 Views
Preview:
TRANSCRIPT
1
You’re the Judge Now…Getting the Scores Right
Calibration Training, continued
Determining an Indicator’s Rating
Each Indicator often has several definitional narratives for each level of proficiency. However, Principals must enter only one (1) score as to each Indicator (1, 2, 3, 4, 5, NO, NA).
2
3
Determining an Indicator’s Rating
“Best fit” approach
3
The Goal of Rater Accuracy
To move from subjectivity to higher levels
of objectivity and insight.
4
How Do You Become More Accurate?
Practice, discipline, and
intentional study of the rubric.
5
Let’s Review!!
• Indicator 1• Indicator 6• Indicator 7
• Indicator 11• Indicator 12
• “I think that’s a Revulsion!”• “I like Chuck E. Cheese too!”• “If you can express your answer in
the numeric way, you can also express it in the geometric way and in words.”
• “I luv puzzles!!”• “In science, you have to be
flexible. Everyone be flexible”6
Calibration Activity One
7
Your First Job…
• Using your rubric, working in PAIRS, sort observation evidence into their appropriate indicator category or categories:– Indicator 1– Indicator 6– Indicator 7– Indicator 11– Indicator 12
8
Answer Key to Sorting the Evidence, Calibration Activity #1
9
EvidenceIndicator 1 B, O, R, TIndicator 6 C, F, H, L, Q (also I)Indicator 7 D, E, I, M, S (also H)Indicator 11 N, P (also A, G, J and S)Indicator 12 A, G, J, K
Your Second Job…
• Using your rubric, working in PAIRS, determine the “best fit score” for each piece of evidence. – Note that the evidence
typically relates to just one component of the indicator. Right now, you should ignore the other components of the indicator/rubric that are not addressed on the evidence sheet.
10
Suggested* Scores
11
Evidence Indicator ScoreA 11, 12 3B 1 1-2C 6 1D 7 2E 7 5F 6 2G 11, 12 4H 6, 7 5 (Ind. 6), 3-4 (Ind. 7)I 6, 7 1-2J 11, 12 1 (Ind. 12); 2 (Ind. 11)
Suggested* Scores
12
Evidence Indicator ScoreK 12 5L 6 4M 7 4N 11 2O 1 1-2P 11 2-3Q 6 3-4R 1 3-4S 7, 11 5T 1 3
Wait…there’s something we haven’t talked about!
13
Make Your Own “Professional Preference” Lists
• Pet Peeves
and
• Preferences
14
Common Pet Peeves/Preferences
Pet Peeves/Negative Bias--messy rooms--sloppy attire--food/drink in the classroom--accents, loud speech--typohs --particular strategies
Preferences/Positive Bias--neat room--attractive appearance--humor--casual demeanor--students in uniforms--particular strategies
15
Give me a 5 if you’re ready for a break!
16
We’re Back…ready or not!!!
17
Three Additional Indicators for Calibration
• Indicator 2: Clearly defined behavior
• Indicator 9: Active learning, questioning, incorporation of interests, and involving all students
• Indicator 13: Monitoring for progress
18
19
Indicator 2
Indicator 2—Teacher clearly defines expected behavior.
• Ensures that students are engaged and clear as to the expectations of the classroom with few reminders given.
• Monitors the behavior of students during whole-
class, small group and seat work activities and during transitions between instructional activities.
• Stops inappropriate behavior promptly and consistently with an appropriate voice level / word choice.
20
Clearly Defining Expected Behavior: What Do you Expect…
• teachers to be doing/saying if they are performing at an “effective” level?
• students to be doing/saying if teachers are performing at an “effective” level?
21
22
Indicator 9
Indicator 9—Engagement of learners: active learning/ involvement/ questioning/ incorporation of interests.
• Engages most students in active learning experiences 80 percent of the class time.
• Uses questioning techniques throughout the lesson, scaffolding to at least the mid-level of Bloom's taxonomy. Provides wait time for some student response and does random checking to ensure the involvement of all learners.
• Engages students by incorporating their general skills and interests into the lesson.
23
Engagement of LearnersWhat Do you Expect…
• teachers to be doing/saying if they are performing at an “effective” level?
• students to be doing/saying if teachers are performing at an “effective” level?
24
25
Indicator 13
Indicator 13—Teacher checks to determine if students are progressing toward stated objectives.
• When appropriate, moves to all areas of the room while students are working on guided practice to promote and reinforce positive student behaviors.
• Uses different types of student response techniques, both individual / group.
• Uses student response techniques to increase active engagement.
• Student’s understanding is evaluated by feedback. • Uses wait time of 3-5 seconds (more for more complex
questions) after voicing the question. Provides opportunity for students to formulate more thoughtful responses and allows time for the student to consider supporting evidence.
26
Checks/Monitors for ProgressWhat Do you Expect…
• teachers to be doing/saying if they are performing at an “effective” level…
• students to be doing/saying if teachers are performing at an “effective” level…
27
The Three-Step Process of Scoring
• Step One: Gather evidence
• Step Two: Sort evidence into indicators, as appropriate
• Step Three: Determine rating
28
29
Calibration Activity Two
30
• During the video– Step One, Part A: Take
notes, documenting the evidence you observed relevant to indicators 2, 9 & 13.
– We will pause briefly at the 5 min. marker to give you add’l time to write your evidence down on notepaper
• After the video– Step One, Part B:
Complete your notes and review them for the most significant evidence relevant to indicators 2, 9 & 13.
– Transfer the key evidence onto sticky notes—one per piece of evidence.
31
Practice Video 1…just indicators 2, 9 & 13Step One
Independent Work
Additional Notes About the Video
• Your “universe of evidence” is the video alone.– “consistently implements” = consistently within
the video– Disregard rubric requirements that require
knowledge outside of video (stricken in the rubric)
• Video format is similar to what will be used on the calibration test.– One screen on the board; one panoramic screen
on the class.
32
33
Everyone Ready?
Calibration Activity #2…Brief Practice Video …just 2, 9 & 13
CCOSA--Tulsa Model Video for Calibration Activity No. 2 (7th grade ELA) -- view first 12 minutes only
34
• During the video– Step One, Part A: Take
notes, documenting the evidence you observed relevant to indicators 2, 9 & 13.
– We will pause briefly at the 5 min. marker to give you add’l time to write your evidence down on notepaper
• After the video– Step One, Part B:
Complete your notes and review them for the most significant evidence relevant to indicators 2, 9 & 13.
– Transfer the key evidence onto sticky notes—one per piece of evidence.
35
Practice Video 1…just indicators 2, 9 & 13Step One
Independent Work
Step Two, Part A:
Using the worksheets as placemats, sort the evidence (on sticky notes) into the appropriate indicator category•If evidence applies to more than one indicator, you may copy the evidence onto multiple sticky notes
Step Two, Part B:
Share worksheets with Table Team
Consolidate the table’s findings evidence onto one placemat per indicator.
36
Practice Video 1…just indicators 2, 9 & 13Step Two
Independent Work Table Work
Pause and Share Evidence with the Room
• What evidence did you have for Indicator 2?
• What evidence did you have for Indicator 9?
37
Pause and Share Evidence with the Room
• What evidence did you have for Indicator 13?
38
Practice Video 1…just indicators 2, 9 & 13Step Three
Step Three
Using the placemats and the rubric, determine the appropriate rating for each indicator and write the “best fit” rating on the placemat in the designated spot.
39
Table Work
Pause and Share Ratings with the Room
• What rating did you have for Indicator 2?
Target: 2
• What rating did you have for Indicator 13?
Target: 2
• What rating did you have for Indicator 9?
Target: 2
40
41
Calibration Activity #3
Practice Test video!
42
Practice Video…eight indicators
• Rating only Indicators 1, 2, 6, 7, 9, 11, 12 & 13• Viewing just the first 20 minutes of an
Observation Engine video (secondary Math).– We will NOT stop at the video while we view it.
• We will:– Independently categorize evidence, and – Independently rate the indicators; then– Share our findings with table teams and report.
43
44
Everyone Ready to Cue Music?
Link to Practice Test Video (Secondary Math)
CCOSA--Tulsa Model Video Practice Test (Secondary Math) -- view first 20 minutes only
45
Practice Test -- Secondary
46
• Ind. 1: 3• Ind. 2: 3• Ind. 6: 3• Ind. 7: 2
• Ind. 9: 2• Ind. 11: 3• Ind. 12: 4• Ind. 13: 2
Target Scores…
47
Are We Ready?
48
Calibration Test Basics
• Required by the state, but it’s TPS-developed• Online platform called Observation Engine.• The testing time will last 90 minutes in all.• Scoring only the 20 minutes of 3 videos– Indicators 1, 2, 6, 7, 9, 11, 12, & 13– Customized by elementary/secondary (your choice)• Either test provides K-12 calibration certification
– After each video, you will have 10 minutes to score before we proceed with the next video.
49
Calibration Test Basics, continued
• Target scores determined by expert panel• Your “grade” is determined by comparing your
scores against the target scores.• Goal is 80% on target• The tests will be graded electronically using
the scantrons on which you enter your scores.• Pass/Fail data will be available in about 2
weeks, if not earlier.
50
Context of Calibration Test
• Your “universe of evidence” is the video alone.–“consistently implements” = consistently within the video–Disregard rubric requirements that require knowledge
outside of video (stricken in the rubric)
• Tips–Use the truncated rubric form for each video, focusing on
each required indicator separately. –Don’t stress! You will do fine!
– Re-takes are available (with new videos)
–Don’t let your professional preferences bias you.
Are we Ready? …. Questions?
52
For Tulsa Model or Calibration Questions…
Jana Burk – burkja@tulsaschools.orgGene S. Kleindienst - kleinge@tulsaschools.org
Katy Ackley – ackleka@tulsaschools.orgJennifer Gripado – gripaje@tulsaschools.org
top related