observing classroom instruction
DESCRIPTION
Observing Classroom Instruction. Sally Atkins-Burnett, Ph.D. Senior Researcher. Process for Developing Protocols. Review literature on fidelity Review classroom observation tools Review literature on quality instruction Identify key features of curriculum. Observing Classroom Practice. - PowerPoint PPT PresentationTRANSCRIPT
Observing Classroom Instruction
Observing Classroom Instruction
Sally Atkins-Burnett, Ph.D.Senior Researcher
Sally Atkins-Burnett, Ph.D.Senior Researcher
Process for Developing ProtocolsProcess for Developing Protocols
Review literature on fidelity Review classroom observation tools Review literature on quality instruction Identify key features of curriculum
Review literature on fidelity Review classroom observation tools Review literature on quality instruction Identify key features of curriculum
Observing Classroom PracticeObserving Classroom Practice
Adherence to instructional practices Exposure (dose) Quality of delivery Student engagement Program differentiation
Adherence to instructional practices Exposure (dose) Quality of delivery Student engagement Program differentiation
Methods of Collecting Classroom Data
Methods of Collecting Classroom Data
Rating scales Checklists Time samples Interactive coding systems Narrative descriptions Teacher logs or diaries Teacher assignments Student work Interviews and surveys
Rating scales Checklists Time samples Interactive coding systems Narrative descriptions Teacher logs or diaries Teacher assignments Student work Interviews and surveys
Recording Observations with Interactive Coding Systems
Recording Observations with Interactive Coding Systems
Most widely used Presence or absence of behaviors in a time
sample Use of a rubric or rating Low inference Key to reliable measurement is operational
definition
Most widely used Presence or absence of behaviors in a time
sample Use of a rubric or rating Low inference Key to reliable measurement is operational
definition
Making It ReliableMaking It Reliable
Describe in behavioral terms –what would you see and hear
Offer both examples and non-examples (is and is not) or differentiate as needed
Use classroom video when training – use range of practice
Discuss disagreements and come to common understanding
Check for rater drift
Describe in behavioral terms –what would you see and hear
Offer both examples and non-examples (is and is not) or differentiate as needed
Use classroom video when training – use range of practice
Discuss disagreements and come to common understanding
Check for rater drift
Example of Item: “Asks close-ended question”
Example of Item: “Asks close-ended question”
Description: Close ended questions are ones in which the teacher accepts only one correct answer.
Example: Q: What is 2+5? (only one answer)
Description: Close ended questions are ones in which the teacher accepts only one correct answer.
Example: Q: What is 2+5? (only one answer)
Differentiating CategoriesDifferentiating Categories
Sometimes the question may have more than one answer but the teacher only accepts one answer:
Q: How do you add 2+5? (more than one possible answer) But teacher only accepts one answer as correct and then moves on to other questions
Answer: You start from 5 and add on 2.Others students could have solved it differently
but teacher only accepts this one.
Sometimes the question may have more than one answer but the teacher only accepts one answer:
Q: How do you add 2+5? (more than one possible answer) But teacher only accepts one answer as correct and then moves on to other questions
Answer: You start from 5 and add on 2.Others students could have solved it differently
but teacher only accepts this one.
Areas of Math Instruction to ConsiderAreas of Math Instruction to Consider
Teacher-initiated instruction Teacher responses or feedback Student to student interaction Student work Activity setting Materials Types of Representations Problem-Solving Approaches
Teacher-initiated instruction Teacher responses or feedback Student to student interaction Student work Activity setting Materials Types of Representations Problem-Solving Approaches
Quality InstructionQuality Instruction
Contingent feedback Positive climate Student engagement Positive behavior management High productivity Higher order thinking
Contingent feedback Positive climate Student engagement Positive behavior management High productivity Higher order thinking
Identify Key FeaturesIdentify Key Features
Review Publisher materials Examine teacher manuals and student
materials Outline key features to assess – may get
developer /publisher input Consider best way to collect information on
key features Triangulate data using multiple-methods
Review Publisher materials Examine teacher manuals and student
materials Outline key features to assess – may get
developer /publisher input Consider best way to collect information on
key features Triangulate data using multiple-methods
ReferencesReferences
American Association for the Advancement of Science (AAAS) Project 2061 (Evaluating curricular materials). Retrieved June 2, 2007 from http://www.project2061.org/publications/textbook/default.htm
Baker, Jean A. “Teacher-Student Interaction in Urban At-Risk Classrooms: Differential Behavior, Relationship Quality, and Student Satisfaction with School.” The Elementary School Journal, vol. 100, no 1, 1999, pp. 57-70.
Clare, Lindsay. “Using Teachers’ Assignments as an Indicator of Classroom Practice.” CSE Technical Report 532. Los Angeles, CA: CRESST University of California. November 2000.
Hilberg, R. Soleste, Hersh C. Waxman, and Roland G. Tharp. “Introduction: Purposes and Perspectives on Classroom Observation Research. In Hersh C. Waxman, Roland G. Tharp, and R. Soleste Hilberg (Eds.). Observational Research in U. S. Classrooms: New Approaches for Understanding Cultural and Linguistic Diversity (pp. 1-20). New York: Cambridge University Press. 2004.
American Association for the Advancement of Science (AAAS) Project 2061 (Evaluating curricular materials). Retrieved June 2, 2007 from http://www.project2061.org/publications/textbook/default.htm
Baker, Jean A. “Teacher-Student Interaction in Urban At-Risk Classrooms: Differential Behavior, Relationship Quality, and Student Satisfaction with School.” The Elementary School Journal, vol. 100, no 1, 1999, pp. 57-70.
Clare, Lindsay. “Using Teachers’ Assignments as an Indicator of Classroom Practice.” CSE Technical Report 532. Los Angeles, CA: CRESST University of California. November 2000.
Hilberg, R. Soleste, Hersh C. Waxman, and Roland G. Tharp. “Introduction: Purposes and Perspectives on Classroom Observation Research. In Hersh C. Waxman, Roland G. Tharp, and R. Soleste Hilberg (Eds.). Observational Research in U. S. Classrooms: New Approaches for Understanding Cultural and Linguistic Diversity (pp. 1-20). New York: Cambridge University Press. 2004.
ReferencesReferences
Huntley, Mary Ann. “Operationalizing the Concept of ‘Fidelity of Implementation’ for NSF-Funded Mathematics Curricula.” Presentation at the National Science Foundation K-12 Mathematics, Science & Technology Curriculum Developers Conference. 2005. Retrieved November 1, 2006 from www.agiweb.org/education/nsf2005/speakers.html
Melde, Chris, Finn-aage Esbensen, and Karin Tusinski. “Addressing Program Fidelity Using Onsite Observations and Program Provider Descriptions of Program Delivery.” Evaluation Review, vol. 30, no. 6, 2006, pp. 714-740.
Mowbray, Carol T., Mark C. Holter, Gregory B. Teague, and Deborah Bybee. “Fidelity Criteria: Development, Measurement, and Validation.” American Journal of Evaluation, vol. 24, no. 3, 2003, pp. 315-340.
Huntley, Mary Ann. “Operationalizing the Concept of ‘Fidelity of Implementation’ for NSF-Funded Mathematics Curricula.” Presentation at the National Science Foundation K-12 Mathematics, Science & Technology Curriculum Developers Conference. 2005. Retrieved November 1, 2006 from www.agiweb.org/education/nsf2005/speakers.html
Melde, Chris, Finn-aage Esbensen, and Karin Tusinski. “Addressing Program Fidelity Using Onsite Observations and Program Provider Descriptions of Program Delivery.” Evaluation Review, vol. 30, no. 6, 2006, pp. 714-740.
Mowbray, Carol T., Mark C. Holter, Gregory B. Teague, and Deborah Bybee. “Fidelity Criteria: Development, Measurement, and Validation.” American Journal of Evaluation, vol. 24, no. 3, 2003, pp. 315-340.
ReferencesReferences
National Research Council. “On Evaluating Curricular Effectiveness: Judging the Quality of K-12 Mathematics Evaluations.” Committee for a Review of the Evaluation Data on the Effectiveness of NSF-Supported and Commercially Generated Mathematics Curriculum Materials. Jere Confrey and Vicki Stohl (Eds.). Mathematical Sciences Education Board, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. 2004.
O'Donnell, C. L. (2006). Fidelity of implementation in scaling up
highly rated science curriculum units. In A. Benbow (Ed.) NSF K-12 Mathematics, Science, and Technology Curriculum Developers' Conference 2005: Dealing with Challenges to Effective and Widespread Implementation of IMD Curricula. Alexandria, VA: American Geological Institute.
National Research Council. “On Evaluating Curricular Effectiveness: Judging the Quality of K-12 Mathematics Evaluations.” Committee for a Review of the Evaluation Data on the Effectiveness of NSF-Supported and Commercially Generated Mathematics Curriculum Materials. Jere Confrey and Vicki Stohl (Eds.). Mathematical Sciences Education Board, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. 2004.
O'Donnell, C. L. (2006). Fidelity of implementation in scaling up
highly rated science curriculum units. In A. Benbow (Ed.) NSF K-12 Mathematics, Science, and Technology Curriculum Developers' Conference 2005: Dealing with Challenges to Effective and Widespread Implementation of IMD Curricula. Alexandria, VA: American Geological Institute.
ReferencesReferences
O’Donnell, Carol. “Fidelity of implementation in scaling up highly rated science curriculum units.” Presentation at NSF K-12 Mathematics, Science, and Technology Curriculum Developers Conference. 2005. Retrieved November 1, 2006 from http://www.gwu.edu/%7Escale-up/document/ODonnell_Fidelity_NSFIMD.pdf
O'Donnell, C.L. & Lynch, S. (2005, April). Examining the fidelity of implementation of highly rated middle school science curriculum materials. Paper presented at the Annual Meeting of the American Educational Research Association, Montreal, Canada.
Padron, Y. N., and H. C. Waxman. “Classroom observations of the Five Standards of Effective Teaching in urban classrooms with ELLs.” Teaching and Change, vol. 7, no. 1, 1999, pp. 79-100.
Pianta, Robert C., Karen M. La Paro, and Bridget K. Hamre. “CLASS Classroom Assessment Scoring System Manual K-3 Version.” Charlottesville, VA: Center for Advanced Study of Teaching and Learning.
O’Donnell, Carol. “Fidelity of implementation in scaling up highly rated science curriculum units.” Presentation at NSF K-12 Mathematics, Science, and Technology Curriculum Developers Conference. 2005. Retrieved November 1, 2006 from http://www.gwu.edu/%7Escale-up/document/ODonnell_Fidelity_NSFIMD.pdf
O'Donnell, C.L. & Lynch, S. (2005, April). Examining the fidelity of implementation of highly rated middle school science curriculum materials. Paper presented at the Annual Meeting of the American Educational Research Association, Montreal, Canada.
Padron, Y. N., and H. C. Waxman. “Classroom observations of the Five Standards of Effective Teaching in urban classrooms with ELLs.” Teaching and Change, vol. 7, no. 1, 1999, pp. 79-100.
Pianta, Robert C., Karen M. La Paro, and Bridget K. Hamre. “CLASS Classroom Assessment Scoring System Manual K-3 Version.” Charlottesville, VA: Center for Advanced Study of Teaching and Learning.
ReferencesReferences
Rivera, Hector, Roland G. Tharp, D. Youpa, S. Dalton, G. Guardino, and S. Lasky. “ASOS: Activity Setting Observation System coding and rulebook.” Santa Cruz: Center for Research on Education, Diversity and Excellence, University of California. 1999.
Rowan, Brian, Eric Camburn, and Richard Correnti. “Using Teacher Logs to Measure the Enacted Curriculum: A Study of Literacy Teaching in Third Grade Classrooms.” The Elementary School Journal, vol. 105, no. 1, 2004, pp. 75-101.
Stodolsky, S. S. (1990). Classroom observation. In J. Millman & L. Darling-Hammond (Eds.), The new handbook of teacher evaluation: Assessing elementary and secondary school teachers (pp. 175-190). Newbury Park, CA: Sage.
Unrau, Y. A., & Wehrmann, K.C. “Evaluation of a home-based family literacy program.” In Y.A. Unrau, J.L. Krysik, & R.M. Grinnell, Jr. (Eds.), Student study guide for the sixth edition of social work research and evaluation: Quantitative and qualitative approaches (pp. 183-190). Belmont, CA: Wadsworth. 2001.
Woodward, J. and J. Baxter (1997). “The effects of an innovative approach to mathematics on academically low-achieving students in inclusive settings.” Exceptional Children, 63(3), 373-388.
Rivera, Hector, Roland G. Tharp, D. Youpa, S. Dalton, G. Guardino, and S. Lasky. “ASOS: Activity Setting Observation System coding and rulebook.” Santa Cruz: Center for Research on Education, Diversity and Excellence, University of California. 1999.
Rowan, Brian, Eric Camburn, and Richard Correnti. “Using Teacher Logs to Measure the Enacted Curriculum: A Study of Literacy Teaching in Third Grade Classrooms.” The Elementary School Journal, vol. 105, no. 1, 2004, pp. 75-101.
Stodolsky, S. S. (1990). Classroom observation. In J. Millman & L. Darling-Hammond (Eds.), The new handbook of teacher evaluation: Assessing elementary and secondary school teachers (pp. 175-190). Newbury Park, CA: Sage.
Unrau, Y. A., & Wehrmann, K.C. “Evaluation of a home-based family literacy program.” In Y.A. Unrau, J.L. Krysik, & R.M. Grinnell, Jr. (Eds.), Student study guide for the sixth edition of social work research and evaluation: Quantitative and qualitative approaches (pp. 183-190). Belmont, CA: Wadsworth. 2001.
Woodward, J. and J. Baxter (1997). “The effects of an innovative approach to mathematics on academically low-achieving students in inclusive settings.” Exceptional Children, 63(3), 373-388.