cera 87 th annual conference- effective teaching & learning: evaluating instructional practices...
TRANSCRIPT
CERA 87th Annual Conference- Effective Teaching & Learning: Evaluating Instructional Practices
Rancho Mirage, CA – December 4, 2008
Noelle C. Griffin, PhD.
Implementation of a Formative Assessment-based Middle School Math Intervention: Findings, Considerations and Lessons Learned
CRESST/UCLA
Victoria Schumacher, PhD. Norwalk/ La Mirada Unified School District
• NLMUSD Background and Considerations
• POWERSOURCE© Overview and Implementation
• POWERSOURCE© Findings and Trends
• Broader Math Achievement Context
Presentation Overview
NLMUSD: Background and Considerations
• Priority 1! 9 out of 10 students reading at grade level.
• Math: initial focus on middle school
• Overarching district framework: “The Work”
Instructional Context
• Authentic student engagement
• Effective questioning
• Making meaning/text analysis
• Assessment driven instruction
• Standards-based instruction across content areas
Components of the Work
Comparison of Math CST 2008 Results across Schools with Free or Reduced Lunch and English Learners
Using CRESST Collaboration to Support “The Work”
• Initial conversations/planning re: implementation and experimental design (POWERSOURCE and “alternative groups”)
• Coordination with other district math partners (Dr. Marty Bonsangue, CSUFullerton)
• Key consideration: How can POWERSOURCE© project compliment existing district initiatives?
District Stakeholder Involvement
• Key role of stakeholders in decision to participate
• Joint CRESST/NLMUSD presentations/question answer sessions to middle school principals/teachers
• Involvement of Teachers’ Association (TANLA)
• Inclusion of CRESST Senior Researcher (Terry Vendlinski) in other district math professional development activities and sessions
• Teachers invited to participate, vote for “site coordinators” for each school
POWERSOURCE© : Overview and
Implementation
POWERSOURCE© Background
• Five-year research project
• Objective: Develop formative assessments and associated instructional support materials to improve student performance in Algebra/pre-Algebra
• Funder: U.S. Department of Education, Institute of Education Sciences (National Assessment Center Grant)
• Emphasis on empirical rigor and experimental testing
• Involves participation of multiple districts over multiple years
POWERSOURCE© Foundations: Formative Assessment
• Formative assessments can be used to guide teaching and instruction
• When used correctly, it can produce increased learning and achievement. (e.g., Black & William, CRESST)
• Many teachers don’t have the time, resources, or knowledge to use formative assessments effectively. (e.g., Herman)
POWERSOURCE© Foundations: Cognition
• Expert knowledge is schema-based, organized around complex, interconnected concepts
• Expert knowledge organization supports problem solving, inferences, transfer
• Experts provide the basis for material design, scoring, and formative assessment process
• Application of what we know about experts to support content area expertise
POWERSOURCE© Foundations: “Big Ideas”
• Fundamental principles that underlie Algebra operations, when learned, will enhance the speed and quality of learning
• Context of expert knowledge/ cognition
• Identified/vetted through participation of math experts (theory and instruction)
• POWERSOURCE Focus: Rational Number Equivalence, Arithmetic Properties, Solving Equations, Functions
Big IdeasMap of Algebra Knowledge
POWERSOURCE© Components
• Four “units” during the school year (one for each “Big Idea”)
• A series of brief formative assessment tasks
Focused on core concepts in algebra and their interconnection, cognitive demands
Using a variety of formats to demonstrate understanding (computation, problem-solving, explanations, word problems, partially worked examples)
• Focused instructional resources for teachers
Teacher Handbook, exemplar lessons, web-based support
• Professional development activities
Explanation Task-Example• An open-ended question to see if students can
transfer the ideas about the multiplicative identity to a novel situation.
Sample Student Responses
Sample Student Responses (Cont’d)
Sample Teacher Resources (Introduction to Handbook)
Information to say to the class
Information to write on the board
Overheads
Reconciling Design/ Rigor with District Needs
• How to integrate POWERSOURCE© content with on-going initiatives?
• How to be respectful of teacher time but maintain implementation fidelity?
• How to make project useful for all participants while maintaining experimental rigor? (I.e., random assignment)
• How to collect data needed while minimizing district burden?
Alternative Professional Development in NLMUSD
• Developed specifically for implementation in NLMUSD
• Focus on use of district benchmark data (DataDirector)
• Teachers randomly assigned to POWERSOURCE© or Alternative
• Provide a meaningful experience for all teachers (including “controls”) while still maintaining empirical rigor
• Collaboration with districts to find workable approaches to control group
POWERSOURCE© : Findings and Trends
POWERSOURCE© 6th Grade Field Testing
• Random assignment of teachers, 4 districts (6th grade)
• Preliminary results:
POWERSOURCE© students perform significantly better than control students on math learning outcomes.
Based on HLM analyses, there were significant differences between PS and control students for all districts and content areas. The average effect size was .88 (with a district by content area range of .44 to 1.75), and the effect size averages by district ranged from .75 to 1.01.
No initial broad impact on CST (follow-up being tracked).
POWERSOURCE© 2007-08 6th Grade Extended Pilot Test
• Multiple Districts (7, including NLMUSD)
• Mixed Design (random assignment by school or teacher)
• Start with 6th grade in 2007-08, then phase in 7th and 8th grades
• Analysis in progress (multiple student and teacher outcomes)
• Preliminary trends NLMUSD
Preliminary Findings: Change in Proficient and Above 2006-07 to 2007-08 (Grade 6 Math CSTs)
Broader Math Achievement Context
NLMUSD Math Achievement
• POWERSOURCE© part of larger initiative, starting at middle school and eventually moving to other grades
• Overall trajectory of math gains across grades in the district from 2006-07 to 2007-08
Math 6 Benchmark Comparison 2006 - 07 to 2007 - 08
Percent Proficient and Above = 1%
Percent Proficient and Above = 5%
Percent Proficient and Above = 6%%
Percent Proficient and Above = 11%
Percent Proficient and Above =17%
Percent Proficient and Above = 23%
Change +10%
Change +12%
Change +17%
Math 7 Benchmark Comparison 2006 - 07 to 2007 - 08
Percent Proficient and Above = 6%
Percent Proficient and Above = 2%
Percent Proficient and Above = 3%
Percent Proficient and Above = 10%
Percent Proficient and Above =6%
Percent Proficient and Above = 12%
Change +4%
Change +4%
Change +9%
Math 8 - Algebra I Benchmark Comparison 2006 - 07 to 2007 - 08
Percent Proficient and Above = 2%
Percent Proficient and Above <1%
Percent Proficient and Above <1%
Percent Proficient and Above = 15%
Percent Proficient and Above =12%
Percent Proficient and Above = 5%
Change +13%
Change +12%
Change +5%
Grade 6CST Comparsion2006 - 07 to 2007 - 08
10%
15%
20%
25%
30%
35%
40%
Total Number of Students
% Proficient &
Above1766 28%
28%
32%
Total Number of Students
% Proficient &
Above1702 32%
14% increase
Grade 7CST Comparison2006 - 07 to 2007 - 08
10%
15%
20%
25%
30%
35%
40%
26%
37%
Total Number of Students
% Proficient &
Above1752 26%
Total Number of Students
% Proficient &
Above1735 37%
42% increase
10%
15%
20%
25%
30%
35%
40%
13%
24%
Total Number of Students
% Proficient &
Above1778 13%
Total Number of Students
% Proficient &
Above1741 24%
Grade 8CST Comparison2006 - 07 to 2007 - 08
85% increase