math matters formative evaluation final report
DESCRIPTION
During the 2014-15 academic year, K-12 teachers in nine districts in Fairfield and Franklin counties in Ohio, and selected staff of the Fairfield County Educational Service Center participated in year-one implementation of the Math Matters Project. The Project was funded by the Ohio Department of Education Straight A Fund, providing support for transitioning to a blended learning model over a five-year period utilizing the ST Math® Program for math instruction in K-12 classrooms. The PAST Foundation Knowledge Capture Program (KC) conducted formative evaluation of the implementation process across the nine districts during the period beginning August 2014 through June 2015.TRANSCRIPT
The PAST Foundation | 1003 Kinnear Road, Columbus, OH 43212 614.340.1208 | www.pastinnovationlab.org
Math Matters Formative Evaluation Final Report October 30th, 2015
Monica Hunter, Ph.D.
Change comes from knowledge.
Our research supports implementation of transformative programs. Through qualitative and quantitative analysis we provide an in-depth,
culturally relevant understanding of your community.
Math Matters Final Formative Evaluation Report October 30th, 2015
Submitted to:
Fairfield County Educational Service Center
Submitted by THE PAST FOUNDATION MIND Research Institute
Knowledge Capture Formative Evaluation Team Monica S. Hunter Ph.D. Annalies Corbin Ph.D. Maria Green Cohen
Meghen Matta Nichole Carelock Ph.D.
Alyssa Reder Lisa Beiswenger Kayla Galloway Ashley Jackson Rachel Orsborn
Bret Roberts
MIND Research Institute Project Implementation Team Andrew Coulson
Eric Pryor Doug Bruno
Twana Young
Copyright © 2015 PAST Foundation All Rights Reserved
Math Matters: Implementation of Blended Learning in Math Instruction for K-12 Education in Fairfield and Franklin Counties, Ohio
EXECUTIVE SUMMARY During the 2014-15 academic year, K-12 teachers in nine districts in Fairfield and Franklin counties in Ohio, and selected staff of the Fairfield County Educational Service Center participated in year-one implementation of the Math Matters Project. The Project was funded by the Ohio Department of Education Straight A Fund, providing support for transitioning to a blended learning model over a five-year period utilizing the ST Math® Program for math instruction in K-12 classrooms. The PAST Foundation Knowledge Capture Program (KC) conducted formative evaluation of the implementation process across the nine districts during the period beginning August 2014 through June 2015.
Three Key Benchmarks of Success to Sustain Blended Learning During year-one implementation, the KC Evaluation Team identified three significant areas of implementation associated with successful execution of blended learning for students in math instruction. Implementation strategies designed by the MIND Research Institute for the Math Matters Straight A Project offered a structured process to support transition to a blended learning environment that allowed individual school districts to identify and advance a unique path toward attaining strategic goals for innovation in math instruction that best suit their particular needs.
Three important aspects of the year-one experience have been identified based on teacher and administrator perspectives. These sustaining factors are key to assuring continued growth and success in attaining goals for blended learning in math instruction.
Three Key Benchmarks of Success to Sustain Blended Learning
Benchmark 1 Building and sustaining stakeholder buy-in Benchmark 2 Technological tools for consistent use of digital resources Benchmark 3 Formal and informal collaboration to support blended learning
These benchmarks reflect understanding of significant, emerging dimensions of the impact of ST Math that also point to potential barriers to success over years two through five. In this approach, sustaining blended learning for math instruction is a more likely outcome when essential scaffolding is addressed, increasing stakeholder effectiveness in achieving blended learning in the long term. Benchmark 1: Building and Sustaining Stakeholder Buy-in The first significant benchmark centers on garnering interest and gaining committed buy-in among key stakeholders. It is clear from teacher and administrator perspectives that regardless of the mode for initiating use of ST Math, teachers reported they were primarily influenced by a set of related factors that led to increases in teachers’ interest, perseverance and commitment to sustaining exploration and use of ST Math. These reflect four basic aspects of buy-in:
1) Engaged leadership that initiated use of ST Math framed by a vision of blended
learning; 2) Professional development to facilitate a collaborative approach to increased use
of ST Math; 3) Digital resources to enable teachers to explore new instructional strategies; and, 4) Growth in student mastery of ST Math materials in ways that encouraged
teachers to persist in use of digital learning resources. Benchmark 2: Technological Tools for Consistent Use of Digital Resources Two aspects of technology are key to establishing a blended learning environment: first, building teacher skills in use of technology, and second, identifying institutional factors that may influence teachers’ willingness to increase technology skills. The question of how teachers are helped or hindered by use of technology in the classroom is an important dimension of understanding what may limit their ability as well as discourage willingness to persist in transitioning to a blended learning environment. Benchmark 3: Formal and Informal Collaboration to Support Blended Learning Creating an effective blended learning environment requires that teachers are provided not only professional development to gain essential skills in facilitation of math learning, but also ongoing development opportunities to collaborate in efforts to increase competencies that support success through sharing of best practices among teachers. As initial, intensive teacher training supported by the ST Math Team is phased out, teachers in following years will benefit from continued access to formal and informal communication, as well as collaborative coordination for effective application of program components. Providing teachers with the opportunity to collaborate in discovering the benefits and strengths of a blended learning approach to math instruction can nurture an organic development of “what blended learning looks like in my school,” and incentivize ongoing investment of time and resources to achieve an effective blended learning approach.
Sustained Use of ST Math for Blended Learning in Math Instruction
The long-term, shared view of sustainability of the Math Matters Project involves a related set of factors that reflect essential shifts from implementation support to embedded capability with use of ST Math by administrators, instructional leaders, classroom teachers and specialists. In building a vision and path toward attaining blended learning in math instruction, the experience of the Fairfield County Educational Service Center Math Matters Project offers insight on complex dimensions of the transition gained from implementation of ST Math in nine districts. The rich texture of the unique dimensions of implementation across grade levels also presents the reader with the opportunity to learn from first-hand experience across districts in meeting diverse needs and constraints, as well as in building upon effective strategies to grow and sustain blended learning through use of ST Math.
Math Matters Formative Evaluation Final Report October 30th, 2015
Table of Contents
Section 1: Math Matters Formative Evaluation Final Report……………………………………………...10 Section 2: Math Matters Knowledge Capture Appendix ………………………………………………...32
MATH MATTERS FORMATIVE EVALUATION October 30, 2015
FINAL REPORT During the 2014-15 academic year K-12 teachers in nine districts in Fairfield and
Franklin counties in Ohio, and selected staff of the Fairfield County Educational Service
Center (ESC), participated in year-one implementation of a five-year process to initiate
the Math Matters Project. The grant was awarded by the Ohio Department of
Education Straight A Fund, providing resources for transitioning to a blended learning
model utilizing the ST Math® Program for math instruction in K-12 classrooms. The
Math Matters Program was launched in August 2014 by the MIND Research Institute,
providing access to ST Math software for use in K-12 classrooms in 100 buildings
across Fairfield and Franklin counties.
The PAST Foundation Knowledge Capture Program (KC) conducted formative
evaluation of the implementation process during the period beginning August 2014
and continuing through June 2015. Formative evaluation was reported on a quarterly
basis (October 30, 2014 and January 30, April 30, and July 30, 2015). This report
constitutes the final review of year-one implementation for Math Matters. This report
also provides a summary of all evaluation activities conducted over the 2014-15
academic year, as well as a final analysis of data gathered during year one of the
project.
Quarterly reports issued in October 2014, January, April, and July 2015 detailed
implementation progress, including evaluation based on observation of project
planning activities, both on-site and in professional development settings.
Additionally, key informant interviews, focus groups, and pre- and post-surveys were
conducted with Math Matters administrators and teachers across the districts. See the
Appendix to this report, Table A: Chronology of Math Matters Knowledge Capture
Activities 2014 - 2015.
10
FORMATIVE EVALUATION ACTIVITIES
Table 1: Knowledge Capture Math Matters Formative Evaluation Methods presents an
overview of research methods including a brief description of the process involved,
type of analysis, and report product. These five types of evaluation activities are
further described in the following section, characterizing methods and purpose of each
type of activity.
TABLE 1: Knowledge Capture Math Matters Formative Evaluation Methods
Research Activity
Process Conducted by Evaluation Team Evaluation Product
Observation of ST Math Implementation Activities
Observe Part 1 and Part 2 training sessions, school site visits, classroom modeling, data and planning meetings, and implementation review meetings.
Bullet point reports providing summary of emerging themes; submitted in quarterly grant reports.
ST Math District Administrator Interviews
Conduct Key Informant interviews with district program leads to develop the context for formative evaluation based on individual district program design and goals for year-one implementation; follow-up interviews were conducted with each district regarding perspectives on achievements and challenges, as well as plans for year two.
Qualitative analysis to identify achievements, challenges and barriers to attaining project goals; narrative analysis and bullet point reports submitted with quarterly grant reports.
ST Math Teacher Focus Groups
Conduct small group discussion with teachers in Math Matters schools regarding ST Math implementation strategies in the classroom; identify challenges, benefits, and gains experienced during year one of project implementation; explore year two implementation strategies with “Train the Trainer” participants.
Qualitative analysis to identify challenges and barriers to attaining project goals; narrative analysis and bullet point reports submitted in quarterly grant reports.
ST Math Teacher Surveys
Design and post online surveys for teachers in all Math Matters districts and the Fairfield ESC, including June Academy participants. Survey data included aspects of ST Math implementation from classroom perspective including teacher perceptions of program impacts related to math instruction, challenges, and ideas about blended learning and student engagement.
Qualitative and quantitative analysis submitted with quarterly grant reports.
Formative Evaluation Monthly Meetings
Monthly team meetings to review implementation schedules; review logistics of evaluation team involvement in project implementation activities; review interim stages of analysis of qualitative and quantitative data to inform implementation strategies; planning and review of quarterly report preparation (October 2014 and January, April, and July 2015).
1 – 1.5 hr. monthly meetings held at the PAST Foundation Office coordinated and conducted by Knowledge Capture.
11
During the 2014-15 school year, the KC team conducted (71) observations of on-site
training, planning sessions and program reviews involving 842 individuals across the
nine districts (see Appendix Table B: Knowledge Capture Research Participants by
District and Grade Level). Additionally, KC conducted (20) administrator interviews,
(15) focus groups involving (91) participants, and (91) teachers participated in an online
post-implementation survey. Table 2: Knowledge Capture Research Activities by
Elementary School, Middle School, High School, and K-12 presents a summary
overview of stakeholder involvement across grade levels.
Table 2: Knowledge Capture Research Activities by Elementary School, Middle School, High School, and K-12
School Designation
Observations Interviews Teacher Focus Groups (FG)
Teacher Surveys
(n=71) (n=20) FG Sessions (n=15)
FG Participants
(n=91)
Respondents (n=91)
Elementary School 36 8 10 48 71
Middle School 14 3 4 22 15
Junior High School 2
3
High School
2
All Grade Levels 19 9 2 21
TOTALS: 71 20 15* 91 91 *One focus group contained participants from both elementary and middle schools.
The approach to formative evaluation employed in this project follows a mixed
methods research design outlined in the National Science Foundation Directorate for
Education and Human Resources, Division of Research and Learning in Formal and
Informal Settings (2010) for conducting qualitative analysis of program implementation.
Formative evaluation developed through a mixed methods approach produces real
time data in a systematic process designed to document substantive implementation
issues and challenges identified by the implementation team as well as by
stakeholders. Utilizing real time, evaluation data allows for an iterative and responsive
process conducted over the course of the project that defines emerging issues
identified by participants and addressed as implementation is underway.
12
Observation of Project Activities: Gathering data in the early stages of formative
evaluation was initiated with observation of orientation sessions with district teams
conducted by ST Math field staff. Data from observation of training sessions, planning
sessions and on-site school visits throughout the course of the school year also
provided the opportunity for the KC Evaluation Team to document the on-site
implementation process, track modifications to the implementation design within
districts, and develop a coordinated plan for engaging with successive phases of
implementation activities related to critical project goals.
Key Informant Interviews: Early data collection also involved conducting one-on-one
interviews with program leads, including district administrative staff, curriculum
coordinators, principals and building program coordinators. Interviews conducted
during October through December of 2014 were designed to develop the context for
formative evaluation of Math Matters implementation based on individual district
program design and goals for year-one implementation. One-on-one interviews were
also conducted during spring 2015 within each district to follow-up on program
implementation and perspectives on achievements and challenges, as well as plans for
year two. Interview data was targeted to gain insight on the experience of elementary
schools, middle schools, and district level and/or a K-12 perspective. Administrator
interview questions were developed from analysis of early observations of training
sessions in schools, as well as district planning sessions with key implementation staff.
Focus Groups: The KC team conducted teacher focus groups during the 3rd and 4th
quarter of the school year. Focus groups conducted at various on-site locations across
the districts involved classroom teachers, content specialists, curriculum coordinators,
and special education/intervention or enrichment specialists. Additionally, the KC team
held focus groups during the “Train the Trainer” sessions held in June 2015 with
participants from nine districts and the ESC. All question sets were developed through
analysis of data including onsite training observations, and administrator interviews.
Post-Implementation Surveys: Surveys provided the opportunity for individual classroom
teachers to give feedback on aspects of the implementation process including
perceptions of program impacts related to math instruction, challenges, and ideas
13
about blended learning and student engagement. These surveys were conducted via a
secure online web-based platform, SurveyMethods®. Information to project
participants about the survey and web link was distributed to classroom teachers by the
ST Math team via email notification. Surveys were conducted during the 4th quarter of
the school year. Survey questions were developed from evaluation data gathered
throughout the year (observations, interviews, and focus groups). Final survey
questions were sent out for review and comment by the ST Math Implementation Team
and the Fairfield ESC grant manager.
Formative Evaluation Monthly Meetings: Members of the Knowledge Capture Team
met monthly with the ST Math Implementation Team and the Math Matters Fairfield
ESC grant manager to review and coordinate modifications to implementation.
Discussions included reviews of interim stages of analysis of qualitative and quantitative
data intended to inform ongoing implementation strategies. Time was also allocated
to plan and review coordination of quarterly report preparation. Additionally, quarterly
report debrief sessions were conducted with ST Math staff immediately following
submittal of quarterly reports to gain insight on interim analysis and data presented in
each quarterly report to inform all Implementation Team members.
A detailed description about the role of Knowledge Capture and approach to
formative evaluation for Math Matters Straight A Grant implementation was developed
to provide information about the purpose and scope of formative evaluation (see
Appendix, Figure A-1: Knowledge Capture Formative Evaluation for the Straight A
Grant Program). The information presented in Figure A-1 was circulated as a one-page
document to ST Math district leads during September 2014. Additionally, a virtual
Q&A session was offered to all participants in September to provide an opportunity to
discuss formative evaluation and provide any additional information to participants
about the PAST Foundation and its role in Math Matters implementation.
Overview of Math Matters Formative Evaluation
Table 3A: Math Matters Formative Evaluation Research Activities 2014 and Table 3B:
Math Matters Formative Evaluation Research Activities 2015 present key data collection
14
points during year one of Math Matters implementation. Formative evaluation
activities are listed by month and year, and include the number of districts involved
with different activities.
As shown in Tables 3A and 3B, the KC Program employed a range of methods that
include both qualitative and quantitative data collection designed to identify unique
dimensions of the implementation process and stakeholder experience across the nine
school districts. The evaluation also involved identifying key opportunities to gather
data that could demonstrate and define important milestones and benchmarks of
project implementation in ways that could better support project participants during
year one to attain strategic goals and objectives.
Table 3A: Math Matters Knowledge Capture Research Activities 2014
Month Type of Formative Evaluation Event Number of Events
Number of
Districts
Aug. 2014 Telephone Orientation with MRI and district 1 1
Conversation with MIND staff 1 10
Sept. 2014 Observations of Part 1 Training 3 2
Meeting with Fairfield ESC Grant manager 1 10
Conference in Irvine 1 10
Oct. 2014 MIND Conference Call 1 10
Observations of Part 1/2 Training 10 3
Key Informant Interview 1 1
Nov. 2014
Key Informant Interview 1 1
Site Visit Observations 11 6
Observations of Part 1 Training 1 1
Monthly Formative Evaluation Meeting 1 10 Conference in Irvine 1 10
Dec. 2014
Key Informant Interviews 5 5 Site Visit Observations 7 3 Program Implementation Meetings 2 1
Monthly Formative Evaluation Meeting 1 10
Observations of Part 1/2 Training [4 abbrev.] 6 6
15
Table 3B: Math Matters Knowledge Capture Research Activities 2015
Month Type of Formative Evaluation Event Number of Events
Number of
Districts
Jan. 2015 Site Visit Observations 5 4
Monthly Formative Evaluation Meeting 1 10
Observations of Part 2 Training; Follow-up Training 3 2
Feb. 2015 Observations of Part 1 Training 1 1
Conference in Irvine 1 10
Monthly Formative Evaluation Meeting 1 10
Mar. 2015
Formal Interviews 2 2
Observations of Part 1 Training 1 1
Monthly Formative Evaluation Meeting 1 10
Site Visit Observations 3 2
Apr. 2015
Formal Interviews 8 6
Monthly Formative Evaluation Meeting 1 10
Teacher Focus Groups 2 2
Site Visit Observations 4 2
May 2015
Conference in Irvine 1 1
Site Visit Observations 1 1
Formal Interviews 2 1
Program Implementation Meetings 4 4
Monthly Formative Evaluation Meeting 1 10
ST Math Teacher Survey 1 10
Teacher Focus Groups 10 5
June 2015
Teacher Focus Group 1 1
June Academy Observations 4 7
June Academy Teacher Surveys 4 7
TTT Focus Groups 2 10
TTT Observations 3 10
Monthly Formative Evaluation Meeting 1 10
Summer School Training 2 1
July 2015 Monthly Formative Evaluation Meeting 1 10
Aug. 2015 Conference in Irvine 1 10
Monthly Formative Evaluation Meeting 1 10
The design for data collection was developed by the KC Team and reviewed during
the Math Matters monthly formative evaluation project team meetings. This involved
review of implementation events and opportunities for structured observation of
planned activities at school sites, as well as mapping out a schedule for conducting
other types of research and data collection, including one-on-one administrator
interviews, teacher focus groups, and teacher surveys.
16
The Knowledge Capture Team conducted interviews with ST Math leaders in all nine of
the Math Matters districts during the 3rd and 4th quarters of year one. Administrator
interview questions were developed from analysis of early observations of training
sessions in schools, as well as district planning sessions with key implementation staff. Analysis of interview data provided insights on expectations and experience of the
Math Matters Project from the perspective of building leaders and content area
coordinators and other program specialists involved with implementation at the district
level.
The final components of the research and evaluation design consisted of teacher focus
groups (April to June) and the teacher survey (May to June). Focus group questions
and survey questions were also developed through analysis of all data including onsite
training observations and administrator interviews. The ST Math implementation team
and the Fairfield County ESC grant manager reviewed and commented on the
questions.
Analysis and reporting conducted during the course of the project is shown in Table 4:
Overview of Math Matters Formative Evaluation Reporting 2014-2015. All supporting
data, including research instruments with question sets for administrator interviews,
focus groups, and surveys were submitted with the quarterly reports as shown in
Table 4.
Preliminary and mid-project data analyzed and presented in the four quarterly reports
introduced and detailed the unique approach to implementation selected by each
district. However, this final analysis centers on three key benchmarks identified across
districts that offer significant benefit for continued success with implementation by
educators, whether launching ST Math in a single building, a K-12 feeder system, or an
entire district. In the following section of this report, these three key benchmarks are
explored more fully, demonstrating the value of year-one implementation knowledge
gained for the Math Matters Project.
17
TABLE 4: Overview of Math Matters Formative Evaluation Reporting 2014-2015
Mode of Evaluation Dates Conducted Report Submitted
Intro to ST Math Part 1 Observations (8) 9/20*, 9/30, 10/9,10/15*, 10/16* October 30, 2014 Intro to ST Math Part 2 Observations (2) 10/15*
Intro to ST Math Part 1 Observations (2) 11/13, 12/16
January 30, 2015
Intro to ST Math Part 2 Observations (4) 10/22*, 10/23, 12/10
Key Informant Interviews (7) 10/23, 11/6, 12/15, 12/16, 12/18*, 12/19
District-led Teacher Debrief Observations (2) 11/4*
Site Visit Observations (10) 11/12*, 11/13, 11/24, 12/5*, 12/9, 12/11*
Data Meeting and Site Visit Observations (5) 11/18, 11/20, 12/3, 12/16, 1/7
Data Meeting Observations (3) 11/21, 11/25, 1/9
Intro to ST Math Part 1 (abbrev.) Observations (2) 12/1, 12/11
Intro to ST Math Part 2 (abbrev.) Observations (2) 12/3, 12/15
Planning Meeting Observations (2) 12/8, 12/9
Data Meeting Observation 1/13
April 30, 2015
Data Meeting and Site Visit Observations (2) 1/13, 1/14
Follow-up Training Observations (2) 1/13, 1/16
Intro to ST Math Part 2 (abbrev.) Observation 1/16
Intro to ST Math Part 1 Observations (2) 2/25, 3/25
Classroom Modeling Observation 3/26
Site Visit Observations (2) 3/31*
Program Administrator Interviews (13**) 3/5, 3/10, 4/8, 4/10, 4/13*, 4/14, 4/15, 5/11
July 30, 2015
Focus Groups (15) 4/20, 4/27, 5/8, 5/12, 5/14*, 5/15, 5/18, 5/20, 5/27, 6/3, 6/24*
Site Visit Observations (4) 4/29*, 5/2, 6/12
Classroom Modeling Observation 4/30
Math Matters Post-Implementation Teacher Survey (Year One)
5/15-6/17
District Meeting for Year 2 Observations (4) 5/15, 5/19, 5/20, 5/21
June Academy Tracks A & B Observations (10) 6/9*, 6/17*
Intro to ST Math Part 1 (abbrev.) Observations (2) 6/12*
Train the Trainer Observations (3) 6/23-6/25
*Indicates dates with more than one evaluation activity. **Some interview sessions included more than one administrator.
18
Summary Overview of 2014-15: Three Key Benchmarks of Success in Sustaining Blended Learning for Math Instruction
In a comprehensive review of all year-one data sources (observation, interviews, focus
groups, and surveys), the KC Evaluation Team identified three significant areas of
implementation associated with successful execution of ST Math to achieve a blended
learning approach in math instruction. For purposes of this discussion, blended
learning is defined as an approach to math instruction that combines traditional
classroom instruction with digital modes of learning. Generally, this approach is one
that offers diverse implementation strategies designed to maximize existing learning
resources (classroom design, access to computers and other devices, digital
infrastructure, etc.) integrated with teacher facilitation of student-centered or
differentiated learning.
Year-one implementation strategies designed by the MIND Research Institute for the
Math Matters Straight A Project offered a structured process to support transition to a
blended learning environment. This allowed individual school districts to identify and
advance a unique path toward attaining strategic goals for innovation in math
instruction that best suit their particular needs and vision of blended learning. (See
Appendix, Table C: Math Matters Implementation Overview 2014-15).
Table 5: Three Key Benchmarks of Success to Sustain Blended Learning shows
significant aspects identified in the year-one experience based on teacher and
administrator perspectives. These three sustaining factors are key to assuring continued
growth and success in attaining goals for blended learning in math instruction.
Table 5: Three Key Benchmarks of Success
to Sustain Blended Learning Benchmark 1 Building and sustaining stakeholder buy-in
Benchmark 2 Technological tools for consistent use of digital resources
Benchmark 3 Formal and informal collaboration to support blended learning
19
ST Math year-one implementation in Fairfield and Franklin counties evidenced both a
high level of interest as well as commitment reported by teachers and administrators
supported by Math Matters. These benchmarks reflect understanding of significant,
emerging dimensions of the impact of ST Math that also point to potential barriers to
implementation over years two through five. In this approach, sustaining blended
learning for math instruction is a more likely outcome when essential scaffolding is
addressed, increasing stakeholder effectiveness in achieving blended learning in the
long term.
Benchmark 1: Building and Sustaining Stakeholder Buy-in
The first significant benchmark centers on garnering interest and gaining committed
buy-in among key stakeholders that include administrators, teachers, and students.
Building stakeholder interest entails planning for all phases of implementation in ways
that systematically recognizes the role of individuals who are fundamental to successful
adoption of a blended learning approach to math instruction. Math Matters project
leads noted that essential staff should not only include district and building leaders,
classroom teachers, intervention and other special education staff, but also content
specialists, educators in technology, and district and building IT support staff.
Figure 1: Stakeholder Buy-In shows relationships across stakeholders that reflect four
basic aspects of buy-in for use of ST Math that occurred during 2014-15 among
administrative staff, teachers, and students. These include:
1) Engaged leadership that initiated use of ST Math framed by a vision of blended learning;
2) Professional development to facilitate a collaborative approach to increased use of ST Math;
3) Digital resources to enable teachers to explore new instructional
strategies; and,
4) Growth in student mastery of ST Math materials in ways that encourage teachers to persist in use of digital learning resources.
20
Figure 1: Stakeholder Buy-In
Figure 1: Attention to four major aspects of ST Math implementation has increased and or maintained stakeholder buy-in through: 1) Engaged leadership that initiated use of ST Math framed by a vision of blended learning; 2) Professional development to facilitate a collaborative approach to increased use of ST Math; 3) Digital resources to enable teachers to explore new instructional strategies; and, 4) Growth in student mastery of ST Math materials in ways that
encourage teachers to persist in use of digital learning resources.
21
Districts varied in their approach to initiating professional development (PD) and
support for K-12 teachers during the early stages of implementation. Differences in
ways of introducing ST Math to teachers ranged from on-site training to other methods
that supported initial use of the math software, including access to online ST Math
instructional resources and training modules. These resources were also available for
supplemental support following on-site training to all teachers to reinforce basic skills
and knowledge of the ST Math Program. Additionally, two districts pursued a “Train-
the-Trainer” model, creating district trainers at the start of the fall 2014 school term to
launch ST Math use by classroom teachers.
In considering these different approaches to teacher training, it is clear from teacher
and administrator perspectives that regardless of the mode for initiating use of ST
Math, teachers reported they were primarily influenced by a set of related factors that
led to increases in teachers’ interest, perseverance and commitment to sustaining
exploration and use of ST Math. The four aspects of stakeholder buy-in are worth
exploring further in building the context for the three key benchmarks.
Engaged leadership that initiated use of ST Math framed by a vision of blended learning. Launching the program with clarity about overall goals involves building a shared vision
of successful math instruction by instructional leaders. Leadership to guide teachers in
growing blended learning skills is also essential to building interest in effective use of
digital resources in the classroom. The diversity of training options demonstrates that
teachers can successfully employ multiple approaches to gain skills in use of ST Math.
However, this in itself does not automatically ensure that a teacher will pursue broader
goals associated with blended learning. A major factor derives from the level of
engagement of the instructional leader, whether this individual is a principal, curriculum
coordinator, content specialist, or grade-level teacher leader. In this approach, the
more engaged the instructional leader is, the more likely it is that the other essential
aspects of stakeholder buy-in will take on importance in sustaining the vision for
blended learning (Benchmarks 2 and 3).
Professional development to facilitate a collaborative approach to increased use of ST Math. Professional development designed to support a blended learning classroom
strategy involves a major transition for the teacher to expand from traditional direct
22
instruction to include a range of instructional strategies. A primary goal of blended
learning is for the teacher to facilitate math learning. ST Math is designed to support
this transition for teachers. However, this transition is more likely to occur if the shift
conforms to an established practice of learning through sharing best practices. This
allows teachers to build their skills through support that ranges from PD workshops to
modeling in the classroom. In the case of ST Math, teachers were given optional
support for classroom modeling, augmenting initial skills gained in PD in using a digital
resource, to integrating use of the instructional resource into the classroom setting.
Teachers reported that the support provided by ST Math Educational Consultants was
essential during year one, and helped to build experience in exploring ways to work
collaboratively through formal and informal networking as use of ST Math grew over
the course of the 2014-15 school year.
Digital resources to enable teachers to explore new instructional strategies. Student
learning with ST Math requires that teachers are willing to allocate class time to use of
ST Math with confidence that students will achieve learning outcomes daily, weekly
and over the course of the school year, allowing teachers to also gain facilitation skills
in the process. During the early stages of use, some teachers reported they found it
challenging to make the shift from direct instruction to facilitation. However, teachers
were generally encouraged by a positive student response, leading teachers to
increase interest in using the program for math instruction. Increased use strategies
shared among teachers (e.g., overcoming schedule constraints for meeting desired
time/week to syllabus completion, approaches to integrating ST Math with math grade
level curriculum) helped to give teachers a way to more fully explore program features
that included learning to apply the ST Math student data reports to support
differentiated learning. While this phase of implementation was reached by the end of
year one for most administrators, many teachers are now approaching this goal during
year two of the Math Matters Project. Both administrators and teachers felt that they
planned to focus on gaining fluency with ST Math student data in year two.
Growth in student mastery of ST Math materials in ways that encouraged teachers to persist in use of digital learning resources. The range of ways in which students were
exposed to ST Math during the 2014-15 academic year included full classroom
integration during math instruction in a blended learning context, to use of the
23
program as a supplement to grade-level instruction as an independent activity. ST
Math was also used during intervention for special populations, including low
performing students, gifted students, and English Language Learners (ELL). Whether
all students were exposed to ST Math, or in cases where only select grade levels or
specific student populations within a building began use of the program, enthusiasm
among students for ST Math significantly influenced buy-in by administrators and
teachers across all nine districts. This was reported by administrators and teachers
alike as a major factor contributing to acceptance and willingness to further commit to
maximizing the perceived benefits of the ST Math Program for creating a blended
learning math environment.
These aspects of buy-in form an interrelated set of factors associated with a positive
experience in learning to use ST Math for both teachers and students. In the following
section, factors to consider in sustaining use of ST Math are examined in terms of
understanding barriers and constraints to use regardless of the level of enthusiasm for
the program by teachers and their students.
Benchmark 2: Technological Tools for Consistent Use of Digital Resources
Figure 2: Technological Tools for Consistent Use of Digital Resources presents two
dimensions of implementation that involve building teacher skills in use of technology,
and institutional factors that may influence teachers’ willingness to increase technology
skills. In Figure 2, the question of how teachers are helped or hindered by use of
technology in the classroom is an important dimension to understanding their ability
and willingness to persist in transitioning to a blended learning environment.
Across the nine districts, teachers encountered similar kinds of technology challenges,
some of which were short term and temporary in nature (e.g., acquisition and
distribution of devices funded by the Straight A Fund). Beyond the resolvable
challenges in launching ST Math, other factors emerged that can be examined as
potential barriers to sustained use as the program advances from years two through
five of implementation.
24
Figure 2: Technological Tools for Consistent Use of Digital Resources
Figure 2: Ensuring consistent use of ST Math presents a view of two dimensions of implementation that involve building teacher skills in use of technology, and institutional/infrastructure factors that may influence a teacher’s willingness to advance technology skills. S h a r in g b e s t p r a ct i ce s for t e ch n o lo g y s u p p or t is a strategic development for building teacher confidence in facilitating a blended learning environment.
25
Institutional constraints as shown in Figure 2 involve different dimensions of access to
devices, as well as connectivity issues. The latter was surprisingly a feature of both
older and newer school buildings. If we begin from the premise that with experience,
teachers value digital resources, then awareness of how specific technology constraints
can work against adoption of digital resources and related instructional strategies
becomes an outcome that must be better understood. If left unaddressed, these
issues can discourage a teacher from investing time and effort to gain new or more
advanced skills in technology if success with the instructional resource diminishes due
to technology constraints.
As teachers encounter significant hardware and connectivity challenges, it is likely they
will require some level of technical support at different stages of implementation, or in
managing actual and perhaps permanent building/hardware constraints. This prospect
is an important dimension in considering limited resources to provide technical
assistance within a building or across a district. Under these conditions, monitoring
locally developed solutions (specific to a building) to promote a menu of “work
arounds” involves strategic, coordinated planning and action. Teachers
communicating in their grade-level or content area networks, newly designated district
ST Math Trainers, technology teachers, and IT specialists among others, are all in
position to share emerging strategies that can potentially overcome common
technology constraints. Strategies may have already begun to develop and will likely
continue to take shape as technical issues emerge and/or district resources to support
technology shrink. Raising awareness of the value of sharing best practices for
technology support should be considered a critical development in year two.
The ultimate benefit of this strategy is again aimed at supporting teachers’ willingness
and persistence in using the ST Math program. Most important, as the five-year
program advances from year two through year five across the nine districts, if teachers
are assisted in maintaining persistence in use of the program, students will have the
opportunity to explore success in math through a blended learning strategy for math
instruction supported by ST Math.
26
Benchmark 3: Formal and Informal Collaboration to Support Blended Learning
In Figure 3: Formal and Informal Collaboration the spotlight is directed at the
intersection of the three benchmarks formed through collaborative work to engender
and sustain blended learning. In Benchmarks 1 and 2, collaboration to support buy-in,
engagement, and persistence is viewed as fundamental to effectively sustaining
blended learning environments. In focusing more closely on collaboration, it is
apparent that the formal implementation process supported by the Straight A Fund is
planned for the short term as designed by conditions of the grant, and similar to all
grant funding in the current era, intended to serve as a catalyst for much needed
critical change to advance 21st century learning. However, ultimate capacity for
sustaining this one-time investment resides within the systemic nature of the
implementation design to engender a new paradigm that embraces blended learning,
and inherent tendencies of stakeholders to invest in success.
In year one, teachers and administrators reported that ST Math opened up new
dialogues in reaction to early signs of improved student interest in math, and increased
levels of engagement in math activities across grade levels. (See Appendix, MIND
Research Institute ST Math Final Report, October 30, 2015 for final statistics on student
syllabus completion for 2014-15.) This is an important early outcome that has helped
to develop a platform for sustaining ST Math use at an institutional level as teachers
attend district-wide PD, exchange ideas during staff meetings, or simply discuss current
math instruction issues in common planning time or informally in the course of their
day.
As intensive teacher training supported by the ST Math Team is phased out, teachers
in years two through five will benefit from continued support for formal and informal
communication, as well as collaborative coordination for effective application of
program components. Shared exploration in use of the program at different grade
levels is a major element of sustaining successful use of ST Math over time. This
includes communication about common experiences in observing student outcomes,
and related instructional strategies in different contexts (e.g., special education).
27
Figure 3: Formal and Informal Collaboration
Figure 3: Formal collaboration starts with implementation and organically grows with stakeholder buy-in to define “what blended learning looks like in my school.” Teacher leaders collaborating through informal networking includes modeling of successful classroom integration of ST Math and sharing best practices
28
Figure 3 shows that initial buy-in by administrators together with on-site PD, initiated a
collaborative experience in using ST Math. Once training was underway, teachers
reported that collaboration was fostered in ongoing PD specifically linked with ST Math
implementation. Increased buy-in through both formal and informal channels created
a sense of value for digital resources by teachers who were gaining skills and
experience with technology, and who are now exploring and sharing strategies for
increased, effective use of digital learning resources.
Early adopters are those teachers who may already have successfully used digital
resources prior to the introduction of ST Math, and who now play a role in influencing
others through demonstration of best practices. These individuals may also be tapped
by district instructional leaders or principals to support others in building technology
skills and becoming proficient in facilitating a blended learning environment for math
instruction. Early adopters are also among those selected for certified training in ST
Math, who will play a key role as a district-level resource in sustaining use of ST Math,
including training newly hired teachers coming into any of the nine districts in year two.
Sustained Use of ST Math for Blended Learning in Math Instruction
Organic developments in collaborative nurturing of blended learning are inherent to
success in exploring and defining “what blended learning looks like in my school.”
Collaborative networks can also incentivize continued investment of time and resources
at every level, across stakeholders, in ways that will continue to increase individual
teacher comfort level with building competency in math instruction in a blended
learning environment.
Figure 4: Sustaining Blended Learning for Math Instruction offers a more robust view of
stakeholder roles and relations in building and sustaining a blended learning
environment. As shown in Figure 4, the long-term, shared view of sustainability of the
Math Matters Project involves a related set of factors essential to progress to long term
outcomes. This includes a shift from implementation support to embedded capacity in
use of ST Math by administrators, instructional leaders, classroom teachers and
specialists, revolving around the experience of teachers who report success in use of
29
Figure 4: Sustaining Blended Learning for Math Instruction
Figure 4: Sustaining a blended learning environment for math instruction involves a related set of factors that reflect essential shifts from implementation support to embedded capacity in use of ST Math by administrators, instructional leaders, classroom teachers and specialists. In this view, the experience of teachers who report success in use of digital resources and gains in facilitating math learning through use of ST Math can influence continued buy-in and commitment to using digital resources for math instruction.
30
digital resources and gains in facilitating math learning (i.e., teachers value digital
resources).
Figure 4 also shows the relationship between technology constraints and sustaining
blended learning. This is perceived by stakeholders as an ongoing factor that will
continually challenge successful implementation. These constraints are also likely to
evolve with changes in infrastructure, as well as access to digital resources and
technical support. Factors affecting teachers’ ability to overcome challenges to
consistent use of ST Math will involve tailored support provided by the ST Math Team
in the short term, and in the long term, teacher willingness to persist in use of digital
resources in the classroom.
Teachers and administrators who see value in building skills and experience with digital
resources are among those who will likely achieve goals for student success in
differentiated learning approaches to math instruction. Conducting an annual survey in
years two through five of the Math Matters Project is proposed as a key tool for
evaluating ongoing stakeholder investment in growing and sustaining a blended
learning environment for math instruction (see Appendix, Table D: Schedule of Teacher
Survey Questions, Years Two through Five). This survey will be administered in each
district as an online survey supported by the MIND Research Institute.
In building a vision and path toward attaining blended learning in math instruction, the
experience of the Fairfield County Educational Service Center offers insight on
complex dimensions of the transition gained through implementation of ST Math in
nine districts. The rich texture of the unique dimensions of implementation across
grade levels also presents the opportunity to learn from first-hand experience across
districts in meeting diverse needs and constraints, as well as in building upon effective
strategies to grow and sustain blended learning through use of ST Math.
31
Knowledge Capture
APPENDIX Math Matters
Knowledge Capture Tables
Table A: Chronology of Math Matters Knowledge Capture Activities 2014-2015
Table B: Knowledge Capture Research Participants by District and Grade Level
Table C: Math Matters Implementation Overview
2014-2015
Table D: Schedule of Teacher Survey Questions, Years Two through Five
Math Matters, MIND Research Institute Quarterly Report on ST Math October 30, 2015
Submitted directly to the Fairfield ESC This report is included here for reference only
32
*Obsv=Observation; **BP=Bullet Point Report
KC Staff Date Event Product Participants
MSH, MGC 8/8/14 District Orientation Obsv*Program Administrator, District Leaders, MIND Research Institute Staff
MM, MGC 9/3/14District Wide Kickoff Event
ObsvProgram Administrator, District Leaders, ESC Partners (Samsung), 7 District Teams
MSH, MGC, MM
9/17/14 Straight A Fund MeetingProject Review
Grant Manager
MGC 9/20/14 Intro to ST Math Part 1 Obsv/BP**Elementary School Teachers, MIND Educational Consultant
MGC 9/20/14 Intro to ST Math Part 1 Obsv/BPElementary School Teachers, MIND Educational Consultant
MGC, LB 9/30/14 Intro to ST Math Part 1 Obsv/BPElementary and Middle School Administrators, Math Coaches, and Teachers, MIND Educational Consultant
MH 10/9/14 Conference with MINDProject Review
MIND Research Institute Staff
MGC, LB 10/9/14 Intro to ST Math Part 1 Obsv/BPProgram Administrator, K-12 Teachers/Aides, MIND Educational Consultant
LB 10/15/14 Intro to ST Math Part 2 Obsv/BPElementary School Teachers, MIND Educational Consultant
LB 10/15/14 Intro to ST Math Part 2 Obsv/BPElementary School Teachers, MIND Educational Consultant
MGC 10/15/14 Intro to ST Math Part 1 Obsv/BPProgram Administrator, K-12 Teachers, MIND Educational Consultant
MGC 10/15/14 Intro to ST Math Part 1 Obsv/BPProgram Administrator, K-12 Teachers, MIND Educational Consultant
MGC 10/16/14 Intro to ST Math Part 1 Obsv/BPProgram Administrator, K-12 Teachers, MIND Educational Consultant
MGC 10/16/14 Intro to ST Math Part 1 Obsv/BPProgram Administrator, K-12 Teachers, MIND Educational Consultant
MSH, MGC, AR
10/20/14KC Formative Evaluation Meeting
Project Review
MIND Research Institute Staff, Grant Manager
August 2014-August 2015Table A: Math Matters Chronology of Knowledge Capture Activities
33
*Obsv=Observation; **BP=Bullet Point Report
KC Staff Date Event Product Participants
MGC 10/22/14 Intro to ST Math Part 2 Obsv/BP
District Instructional Coach, Elementary School special education, ELL, gifted, and intervention Teachers, MIND Educational Consultant
MGC 10/22/14 Intro to ST Math Part 2 Obsv/BP
Director of Elementary Curriculum, Elementary School special education, ELL, gifted, and intervention Teachers, MIND Educational Consultant
MSH 10/23/14 Intro to ST Math Part 2 Obsv/BP
District Instructional Coach, Elementary School special education, ELL, gifted, and intervention Teachers, MIND Educational Consultant
MSH 10/23/14 Interview Notes Program Administrator
MGC 11/4/14District-led Teacher Debrief
Obsv/BPTeacher facilitated ST Math debrief with K-12 teachers
MGC 11/4/14District-led Teacher Debrief
Obsv/BPTeacher facilitated ST Math debrief with K-12 teachers
MSH, MGC 11/6/14 Interview Transcript Program Administrator
MGC 11/12/14 Site Visit Obsv/BPPrincipal, Middle School Teachers, MIND Educational Consultant
MGC 11/12/14 Site Visit Obsv/BPPrincipal, Instructional Coach, Elementary School Teachers, MIND Educational Consultant
MGC 11/12/14 Site Visit Obsv/BPInstructional Coach, Elementary School Teachers, MIND Educational Consultant
MGC 11/13/14 Intro to ST Math Part 1 Obsv/BPProgram Administrator, Elementary School ESL Teachers/Aides, MIND Educational Consultant
MGC 11/13/14 Site Visit Obsv/BPProgram Administrator, Middle School ESL Teachers, MIND Educational Consultant
MSH, MGC, AR
11/17/14KC Formative Evaluation Meeting
Project Review
MIND Research Institute Staff, Grant Manager
MGC 11/18/14 Data Meeting; Site Visit Obsv/BPPrincipal, Assistant Principal; Middle School Teachers, MIND Educational Consultant
34
*Obsv=Observation; **BP=Bullet Point Report
KC Staff Date Event Product Participants
MSH 11/19/14Quarterly Meeting with MIND
Project Review
MIND Research Institute Staff, Grant Manager, PAST CEO
MGC 11/20/14 Data Meeting; Site Visit Obsv/BPPrincipal, Assistant Principal; Middle School Teachers, MIND Educational Consultant
MGC 11/21/14 Data Meeting Obsv/BPPrincipal and Intervention Coach, MIND Educational Consultant
MGC 11/24/14 Site Visit Obsv/BPST Math Lead Teacher and Curriculum Coordinator, MIND Educational Consultant
LB 11/25/14 Data Meeting Obsv/BP Principal, MIND Educational Consultant
MGC 12/1/14Intro to ST Math Part 1 (abbreviated)
Obsv/BPElementary School Teachers, MIND Educational Consultant
MGC 12/3/14 Data Meeting; Site Visit Obsv/BPAssistant Principal; Middle School Teachers, MIND Educational Consultant
MGC 12/3/14Intro to ST Math Part 2 (abbreviated)
Obsv/BPElementary School Teachers, MIND Educational Consultant
MGC 12/5/14 Site Visit Obsv/BPMiddle School Teachers, MIND Educational Consultant
MGC 12/5/14 Site Visit Obsv/BPElementary School Teachers, MIND Educational Consultant
MGC, KG 12/8/14 Planning Meeting Obsv/BPProgram Administrator, MIND Educational Consultant
MGC 12/9/14 Site Visit Obsv/BPMiddle School Special Education Teachers, MIND Educational Consultant
MGC 12/9/14 Planning Meeting Obsv/BPProgram Administrators, MIND Educational Consultant
MGC, KG 12/10/14 Intro to ST Math Part 2 Obsv/BPMiddle School Teachers, MIND Educational Consultant
MGC, KG 12/11/14 Site Visit Obsv/BPProgram Administrator, Elementary School Teachers, MIND Educational Consultant
MGC, KG 12/11/14 Site Visit Obsv/BPProgram Administrator, Elementary School Teachers, MIND Educational Consultant
MGC, KG 12/11/14Intro to ST Math Part 1 (abbreviated)
Obsv/BPProgram Administrator, Elementary School Teachers, MIND Educational Consultant
MSH, MGC 12/15/14 Interview Transcript Program Administrator
35
*Obsv=Observation; **BP=Bullet Point Report
KC Staff Date Event Product Participants
LB 12/15/14Intro to ST Math Part 2 (abbreviated)
Obsv/BPElementary School Teachers, MIND Educational Consultant
LB 12/16/14 Data Meeting; Site Visit Obsv/BPAssistant Principal; Middle School Teachers, MIND Educational Consultant
MSH, MGC 12/16/14 Interview Transcript Program Administrator
KG 12/16/14 Intro to ST Math Part 1 Obsv/BPElementary and Middle School Teachers, MIND Educational Consultant
MSH, MGC, AR
12/17/14KC Formative Evaluation Meeting
Project Review
MIND Research Institute Staff, Grant Manager
MSH, MGC 12/18/14 Interview Transcript Program Administrators (n=2)
MSH, MGC 12/18/14 Interview Transcript Program Administrator
MSH, MGC 12/19/14 Interview Transcript Program Administrator
KG 1/7/15 Data Meeting; Site Visit Obsv/BPPrincipal, Elementary School Teachers, MIND Educational Consultant
LB 1/9/15 Data Meeting Obsv/BPPrincipal, Assistant Principal, and Instructional Coach, MIND Educational Consultant
LB 1/13/15 Data Meeting; Site Visit Obsv/BPRTI Tutor; Junior High School Teachers, MIND Educational Consultant
LB 1/13/15 Data Meeting Obsv/BPPrincipal and Title 1/ESL Teacher, MIND Educational Consultant
LB 1/13/15 Follow-up Training Obsv/BPK-12 Special Education Teachers, MIND Educational Consultant
KG 1/14/15 Data Meeting; Site Visit Obsv/BPPrincipal, Elementary School Teachers, MIND Educational Consultant
LB 1/16/15 Follow-up Training Obsv/BPElementary School Teachers, MIND Educational Consultant
KG 1/16/15Intro to ST Math Part 2 (abbreviated)
Obsv/BPElementary School Teachers, MIND Educational Consultant
MH, AC 2/4/14 Conference with MINDProject Review
MIND Research Institute Staff
MSH, MGC, AR
2/17/15KC Formative Evaluation Meeting
Project Review
MIND Research Institute Staff, Grant Manager
36
*Obsv=Observation; **BP=Bullet Point Report
KC Staff Date Event Product Participants
LB 2/25/15 Intro to ST Math Part 1 Obsv/BPK-12 ESL Teachers/Aides, MIND Educational Consultant
MSH, MGC 3/5/15 Interview Transcript Program Administrator
MSH, MGC 3/10/15 Interview Transcript Program Administrator
MSH, MGC, AR
3/16/15KC Formative Evaluation Meeting
Project Review
MIND Research Institute Staff, Grant Manager
LB 3/25/15 Intro to ST Math Part 1 Obsv/BPK-12 ESL Teachers/Aides, MIND Educational Consultant
MGC 3/26/15Site Visit: Classroom Modeling
Obsv/BPElementary School Teachers, MIND Educational Consultant
AJ 3/31/15 Site Visit Obsv/BPElementary School Teachers, MIND Educational Consultant
AJ 3/31/15 Site Visit Obsv/BPPrincipal and ESL Teacher, MIND Educational Consultant
MSH, MGC, AR
4/6/15KC Formative Evaluation Meeting
Project Review
MIND Research Institute Staff, Grant Manager
MSH, MGC 4/8/15 Interview Transcript Program Administrator
MSH, MGC 4/10/15 Interview Transcript Program Administrator
MSH, MGC 4/13/15 Interview Transcript Program Administrator
MSH, MGC 4/13/15 Interview Transcript Program Administrator
MSH, MGC 4/14/15 Interview Transcript Program Administrator
MSH, MGC 4/15/15 Interview Transcript Program Administrators (n=3)
MGC, KG 4/20/15 Focus Group Obsv/BP Elementary School Teachers (n=5)
MGC, KG 4/27/15 Focus Group Obsv/BP Middle School Teachers (n=8)
MGC 4/28/15 Site Visit Obsv/BPJiJi visit to Middle School, MIND Educational Consultant
37
*Obsv=Observation; **BP=Bullet Point Report
KC Staff Date Event Product Participants
LB 4/29/15 Site Visit Obsv/BPJiJi visit to Elementary School, MIND Educational Consultant
LB 4/29/15 Site Visit Obsv/BPJiJi visit to Elementary School, MIND Educational Consultant
MGC 4/30/15Site Visit: Classroom Modeling
Obsv/BPElementary School Teachers, MIND Educational Consultant
RO 5/2/15 Site Visit Obsv/BPST Math demonstration during district-wide event
MSH 5/6/15Quarterly Meeting with MIND
Project Review
MIND Research Institute Staff, Grant Manager, PAST CEO
MGC, KG 5/8/15 Focus Group BP Elementary School Teachers (n=5)
MSH, MGC 5/11/15 Interview Transcript Program Administrators (n=2)
MGC, KG 5/12/15 Focus Group BP Elementary School Teachers (n=7)
MSH, MGC 5/14/15 Focus Group BP Middle School Teachers (n=4)
MGC, LB 5/14/15 Focus Group BP Elementary School Teachers (n=5)
MSH, KG 5/14/15 Focus Group BP Elementary School Teachers (n=2)
MSH, KG 5/14/15 Focus Group BP Elementary School Teachers (n=4)
MGC, LB 5/15/15 Focus Group BP Middle School Teachers (n=8)
MSH 5/15/15District Meeting: Review Plan for Year 2
BPProgram Administrator, MIND Educational Consultant
MGC 5/15/15 ST Math Teacher SurveySurvey Report
Teachers in 9 Districts and Fairfield ESC (n=91), SurveyMethods® (web based)
MSH, RO 5/18/15 Focus Group BP Elementary School Teachers (n=9)
MSH, MGC, AR
5/18/15KC Formative Evaluation Meeting
Project Review
MIND Research Institute Staff, Grant Manager
MSH 5/19/15District Meeting: Review Plan for Year 2
Obsv/BPProgram Administrators, MIND Educational Consultant
38
*Obsv=Observation; **BP=Bullet Point Report
KC Staff Date Event Product Participants
MGC 5/20/15District Meeting: Review Plan for Year 2
Obsv/BPProgram Data Team, MIND Educational Consultant
MGC, RO 5/20/15 Focus Group BP Elementary School Teachers (n=5)
MGC 5/21/15District Meeting: Review Plan for Year 2
Obsv/BPProgram Administrator, MIND Educational Consultant
MGC, KG 5/27/15 Focus Group BP Elementary School Teachers (n=4)
MGC, BR 6/3/15 Focus Group BP Elementary and Middle School Teachers (n=4)
LB 6/9/15 June Academy Track A Obsv/BPEducators from Math Matters districts, MIND Educational Consultant
MSH, MGC 6/9/15 June Academy Track B Obsv/BPEducators from Math Matters districts, MIND Educational Consultant
LB 6/12/15Intro to ST Math Part 1 (abbreviated)
Obsv/BPElementary Summer School Teachers, MIND Educational Consultant
LB 6/12/15Intro to ST Math Part 1 (abbreviated)
Obsv/BPElementary Summer School Teachers, MIND Educational Consultant
KG 6/17/15 June Academy Track A Obsv/BPEducators from Math Matters districts, MIND Educational Consultant
LB 6/17/15 June Academy Track B Obsv/BPEducators from Math Matters districts, MIND Educational Consultant
MSH, MGC, AR
6/15/15KC Formative Evaluation Meeting
Project Review
MIND Research Institute Staff, Grant Manager
LB6/23-25/15
Train the Trainer Obsv/BPEducators from Math Matters districts, MIND Educational Consultant
MSH, KG 6/24/15 Focus Group BP Educators from Math Matters districts (n=10)
MM, LB 6/24/15 Focus Group BP Educators from Math Matters districts (n=11)
MSH, MGC, AR
7/20/15KC Formative Evaluation Meeting
Project Review
MIND Research Institute Staff
MSH 8/13/15Quarterly Meeting with MIND
Project Review
MIND Research Institute Staff, PAST CEO
MSH, MGC, AR
8/27/15KC Formative Evaluation Meeting
Project Review
MIND Research Institute Staff
39
Knowledge Capture Activity
Grade Level C
CS
ESC
Gah
anna
Ham
ilton
Hill
iard
Lanc
aste
r
Lib
erty
Uni
on
Pick
erin
gto
n
Wal
nut
Tow
nshi
p
Wor
thin
gto
n
June
Aca
dem
ies
Trai
n th
e Tr
aine
r
Total Participants by Grade Level and Activity
ES 2 5 11 4 5 21 48MS 2 8 12 22
K-12 21 21ES 3 1 9 8 1 16 1 17 15 71MS 3 2 6 2 2 15
Jr HS 1 1 1 3K-12 1 1 2ES 25 7 111 16 22 70 19 39 45 354MS 4 17 3 2 98 7 131
Jr HS 11 4 15K-12 31 10 96 2 21 160
70 11 27 15 131 43 28 202 20 176 77 42 842
Source: MIND Research Institute, October 30, 2014; email correspondence September 16, 2015, and September 21, 2015; KC Observation Bullet Point Reports.
*Several schools were observed more than one time.
2014-2015 (n=842)
Focus Groups
Surveys
Teachers Observed*
Total Participants by District
Table B: Knowledge Capture Research Participants by District and Grade Level
40
Table C: Math Matters Implementation Overview 2014 - 2015
ST Math Activit ies Month
Fai
rfie
ld E
SC
Lan
cast
er
Lib
ert
y U
nio
n
Pic
keri
ng
ton
Wal
nu
t
CC
S
Gah
ann
a
Ham
ilto
n
Hil
liar
d
Wo
rth
ing
ton
Training [Pt. 1 & 2; Abbrev.]
August n September n n n n n
October n n n n n
November n n December n n
January n n February n n
March n n April n May n June n
Classroom Support [Site Visits]
September October n n n n n n n
November n n n n n n n
December n n n n n n n n
January n n n n
February n n n n n
March n n n n n n
April n n n n n n n n
May n June n July n
Classroom Modeling April n n May June n
Webinars April n
Data Meetings
November n n n December n n n
January n n n n
February n n n n
March n n n n
April n n n
May n n n n n
July n JiJI School Visits April n n n n n n n
Implementation Planning Meetings
September n October n
December n March n April n n
May n n n n June Academy [6/9 &
6/17] June n n n n n n n n
Train the Trainer [6/23-6/25} June n n n n n n n n n n
Source: MIND Research Institute, July 30, 2015, and email correspondence July 22, 2015, September 10, 2015.
41
Table D: ST Math Teacher Survey Questions for Implementation Years 2-5 This schedule of survey questions is designed for use during years 2-5 of the Math Matters Project in Fairfield and Franklin counties. These question sets are designed for use by Math Matters school districts during the 4th quarter of each year, administered via MIND Research Institute to monitor ongoing strategies for continued successful use of ST Math in K-12 math instruction. Districts can select from the question categories to target particular priorities for monitoring ongoing use of ST Math and are not required to administer all categories in their annual end-of-year survey. Note questions showing an asterisk were included in the Year-One Implementation Survey, Spring 2015. Group A: Respondent Profile questions provide the foundation for contextual analysis of survey responses. Groups B-D include the following categories: B. Stakeholder Buy-in C. Technological tools and support for consistent use of ST Math D. Formal and Informal collaborative development to support blended learning E. Sustaining use of ST Math Group A: Respondent Profile
1. *What grade level is your school building? (Please check all that apply.) a. Elementary School b. Middle School c. Junior High School d. High School e. K-12 or pre-K-12
2. *What is your current position at your school?
a. Classroom teacher b. Special Education Teacher c. Gifted Teacher d. ESL/ELL Teacher e. ESL/ELL Aide f. Intervention Specialist g. Instructional Coach h. Technology Teacher i. N/A
42
3. What is your position in your district?
a. Curriculum Coordinator b. Certified ST Math Trainer c. Program Coordinator/Intervention d. Program Coordinator/ELL e. Program Coordinator/Gifted f. N/A
4. *What do you teach?
a. All subjects b. Math c. English as Second Language d. Math Intervention e. Reading Intervention f. Technology g. If other, please describe
43
Group B: Stakeholder Buy-In
1. How were you introduced to ST Math? a. Learned about it in a district meeting b. Learned about it during PD c. Learned about it in a staff meeting d. Learned about it from a colleague in my building e. Learned about it from a math curriculum coordinator f. Learned about it from a student g. Learned about it from a family member
2. In which school year were you introduced to ST Math?
a. 2013-14 b. 2014-15 c. 2015-16 d. 2016-17 e. 2017-18 f. 2018-19
3. *How were you trained in using ST Math? (Please check all that apply.)
a. On-site training sessions led by ST Math staff b. Classroom visits from ST Math staff c. Self-guided online courses d. Webinar participation e. Training by others in your building/district f. No formal training g. If other, please describe
4. Were you tasked by your administrator to assist with training other teachers in your building?
(Y/N)
5. If yes, have you/will you work across grade levels, or only within your grade level? (Please check all that apply.)
a. All grades (elementary) b. All grades (middle school) c. Junior high d. All grades (High School) e. All grades (K-12) f. Only my grade level
44
6. How would you describe your initial level of buy-in to using ST Math with students?
a. Highly enthusiastic b. Somewhat enthusiastic c. Neutral d. No interest
7. What is your current level of buy-in to using ST Math with students?
a. Highly enthusiastic b. Somewhat enthusiastic c. Neutral d. No interest
8. Has your current level of buy-in been influenced by your colleagues? (Y/N)
9. What is the current level of buy-in by your principal?
a. Highly enthusiastic b. Somewhat enthusiastic c. Neutral d. No interest
10. What is the current level of buy-in by students in your school?
a. Highly enthusiastic b. Somewhat enthusiastic c. Neutral d. No interest
11. What is the level of buy-in by parents?
a. Highly enthusiastic b. Somewhat enthusiastic c. Neutral d. No interest e. I don’t know
45
Group C: Technological tools and support for consistent use of ST Math
1. Did you have experience using computers to support classroom instruction prior to using ST Math? (Y/N)
2. How comfortable are you with using technology with your students in the classroom? a. Very comfortable b. Somewhat comfortable c. Not comfortable
3. Do you have tech support in your building? (Y/N)
4. How are students accessing ST Math? (Please check all that apply.)
a. Tablets/Laptops from shared cart b. Tablets/Laptops in the classroom c. Desktop computers in the classroom d. Computer lab e. Computer/devices at home
5. How many opportunities have you had to work with MIND Research Institute Educational
Consultants this school year? a. Ongoing as needed b. 1-3 times c. More than 3 times
6. How many opportunities have you had to work with your ST Math District Trainers this school
year? a. Ongoing as needed b. 1-3 times c. More than 3 times
7. Did you experience technology related challenges this past year? (Y/N)
8. If yes, were you able to solve the problem? (Y/N)
9. Did you seek assistance from anyone else? (Please check all that apply.)
a. Other teachers in your building b. Someone in your building who is not a teacher c. District ST Math Trainer d. Other district staff e. ST Math Technical Assistance Staff (on-site) f. ST Math “hotline” assistance
46
Group D: Formal and informal collaborative development to support blended learning
1. How is ST Math used in your classroom? a. Instructional classroom time b. Special needs student (intervention) c. Tutoring d. Small group e. Rotation f. If other, please describe
2. Do you team-teach math? (Y/N)
3. Do you plan to engage in a collaborative sharing of instructional strategies with other teachers
in your building? (Y/N)
4. *Do you have opportunities to discuss ST Math with others in your building? (Y/N)
5. If yes, please describe when you discuss ST Math. (Scale selection where 1=daily, 2=weekly, 3=bi-monthly, 4=monthly, 5=not at all)
a. Professional Development b. Staff Meetings c. Planning Time d. Grade Level Meetings e. Content Planning/Coordination Meetings f. Informally g. If other, please describe
6. Do you have opportunities to discuss ST Math with your instructional leader? (Y/N)
7. Do you have common planning time to work with other teachers in your building on
developing ST Math classroom strategies? (Y/N)
8. Have you had opportunities to share/observe best practices for using ST Math? (Y/N)
9. *What kinds of opportunities have you had to share best practices with others in your building? (Please check all that apply.)
a. During PD b. During staff meetings c. During planning time d. Informally e. If other, please describe
47
Group E: Sustaining Use Of St Math
1. *When do your students play ST Math games? (Please check all that apply.) a. During math class time b. During other class time c. After school (in the building) d. Recess e. Free time during the school day f. At home g. If other, please describe
2. How often are you using ST Math for instruction in your classroom?
a. 1 time per week b. 2-3 times per week c. 4-5 times per week d. I do not use ST Math for instruction
3. Do you have support in the classroom?
a. ESL b. Teaching aides c. Intervention teacher d. If other, please describe
4. *What type of instructional strategy do you use with ST Math? (Please check all that apply.)
a. Direct Instruction b. Coaching students through facilitating questions (small group) c. Coaching students through facilitating questions (whole class) d. N/A e. If other, please describe
5. Do you use student facilitators? (Y/N)
a. If so, how did you train them?
6. *Are you currently using other instructional math software with your students? (Y/N)
7. *If yes, which ones are you using?
48
8. *Have you accessed any of the following from the ST Math Teacher Resource website? (Please check all that apply.)
a. ST Math Teacher online manual b. Training videos c. Videos to share with students d. Game mats e. Fluency worksheets f. Parent letter template g. “JiJi culture” materials, such as postcards, etc. h. I haven’t explored the Teacher Resource Site i. I haven’t had any information about the Teacher Resource website j. I explored the website and didn’t find what I was searching for
9. *Do you or your students use any of the following during ST Math time? (Please check all that
apply.) a. Whiteboard b. Worksheets c. Paper and pencil d. Manipulatives e. Game mats f. If other, please describe
10. *Do your students share what they have learned during ST Math time in ways that help
advance their ability in “thinking about thinking”? (Please check all that apply.) a. I haven’t observed my students share what they have learned b. With other students in small group settings or one-on-one c. With the class as a whole d. Working with the teacher one-on-one e. If other, please describe
11. *How would you characterize student outcomes at this point in the school year?
a. I do not see any change in student math performance b. Students have a positive attitude about math and math learning c. Students show more perseverance when facing challenging problems d. Students more frequently talk about math concepts with each other e. Students exhibit a greater depth of knowledge when talking about math concepts f. Students score better on class quizzes and tests g. Students who are hardest to reach in math learning are more willing to engage in ST
49
1
Math Matters: Transforming Math Education
for 21st Century Success
Year 1 Report on Large Scale Blended Learning
ST Math Adoption
October 30th, 2015
50
2
SECTION 1 – GENERAL IMPLEMENTATION OVERVIEW........................................................................ 3
SECTION 2 – MIND PROFESSIONAL DEVELOPMENT ACTIVITY ............................................................. 4
MIND TRAINING & SERVICE EVENTS ....................................................................................... 4
MIND PROFESSIONAL DEVELOPMENT PLAN OPTIONS ............................................................ 5
ADDITIONAL MIND PROFESSIONAL DEVELOPMENT DELIVERABLES ......................................... 6
MIND SELF-GUIDED COURSE OVERVIEW ................................................................................. 7
MIND SELF-GUIDED COURSE SEQUENCE ................................................................................. 8
SECTION 3 – ST MATH IMPLEMENTATION ........................................................................................ 9
ST MATH STUDENT ENROLLMENT BY MONTH ........................................................................ 9
YEAR 1 START-UP VS SYLLABUS PROGRESS ........................................................................... 10
PROJECTIONS FOR ST MATH PROGRESS IN 2015-2016 .......................................................... 11
PROJECT CHALLENGES .......................................................................................................... 12
PROJECT SUCCESS ................................................................................................................ 13
51
3
Section 1 – General Implementation Overview The Math Matters project was proposed with the goal of transforming mathematics education across nine Ohio school districts within Fairfield (Lancaster City Schools, Liberty-Union Thurston Local Schools, Pickerington Local Schools, Walnut Township Schools) and Franklin (Columbus City Schools, Gahanna-Jefferson Local Schools, Hamilton Local Schools, Hilliard City Schools, Worthington City Schools) Counties over the course of five years. Districts participating in the Math Matters project (as well as the Fairfield County Education Services Center) each recognized the need to address an existing achievement gap in mathematics that was unique to their individual student populations. Depending on the district, that achievement gap may have existed between students identified with special needs, those with limited English proficiency, or between students of varied cultural or socioeconomic background. Through the implementation of the ST Math program from MIND Research Institute, Math Matters schools have begun to develop a blended learning approach to mathematics instruction which allows students and teachers access to a language free, visual approach to teaching and learning math concepts. Over the first year implementing the ST Math program, participation grew to include over 1,000 teachers working with just under 27,000 students in a total of 82 of the 100 schools funded through the Math Matters project. As initial logistic and technical challenges were identified and resolved, student participation totals in the program steadily grew each and every month over the course of the entire 2014-2015 school year. This continued expansion of student and teacher participation can also be attributed to the increased interest in the ST Math program that was generated among additional grade levels or student populations outside of those originally targeted for involvement in the project. The Professional Development services delivered by MIND Research Institute have enabled each district to successfully begin their implementation of ST Math, plan for its expanded use, begin making connections to classroom instruction, and increase their internal capacity to support teachers using the program. An efficient shared services model was implemented within each of the individual districts whenever possible, as well as at the overall project level through joint training initiatives such as the June Academy sessions and MIND’s Train the Trainer workshops. In addition to the instructor led services provided by MIND, a majority of the Math Matters teachers took advantage of digital self-paced Professional Development content that is available through MIND’s Teacher Resource Site.
The introduction of ST Math into classrooms across Franklin and Fairfield Counties has created opportunities for students to gain a deeper conceptual understanding of mathematics by solving visual puzzles that require them to think critically and make meaning through hands-on learning. Participating teachers are now equipped with a tool that supports a blended learning environment in their classrooms and allows students the ability to learn at a personalized pace based on their own understanding. The Math Matters project is positioned to well exceed the targeted numbers of students and teachers utilizing the ST Math program in the second year of this five year initiative. The successful implementation of ST Math in year 1 of the Math Matters project will serve as a strong foundation on which to further strengthen the connection between ST Math and classroom instructional practices in Mathematics.
52
4
Section 2 – MIND Professional Development Activity
Chart 1
MIND Research Institute delivered more than 230 training and service related events over the 2014-2015 school year in support of the 9 districts implementing ST Math as a part of the Math Matters project (see Graph 1). The Implementation Planning process that was initiated with each district in July 2014 included guidance on the technical activation of devices to be used with ST Math, the creation of all necessary teacher accounts (also student accounts if district chose to roster), and the development of a Professional Development plan best suited to each district’s individual needs (see pg. 5). All nine districts participating in the project completed initial ST Math trainings prior to the Benchmark date of October 31, 2014. In most cases multiple initial training sessions were delivered in order to meet demand based on the original number of teachers identified for participation in the project. Subsequent initial trainings were necessary throughout the first year of the grant as districts continued to identify additional segments of students that they felt would benefit from the ST Math program.
In addition to the standard training components in the plans listed on pg. 5, additional events such as site visitations and data meetings were delivered in all districts after initial start-up of the program. Student performance data from ST Math was reviewed at this time and used to identify struggling students. An initial focus of these additional events was centered around how to facilitate student thinking through questioning as they work on ST Math puzzles. As schools mastered this important element of implementing ST Math, MIND’s Education Consultant was then able to begin providing guidance on a deeper level integration of ST Math. This customized support included workshops and modeling to address the areas of blended learning, mathematical practice standards, and effective lesson design.
0
5
10
15
20
25
30
35
40
45
50
MIND Research Institute- Event Counts by District
Number of MathMatters Schools
Number ofMIND EventsDelivered
53
5
MIND Professional Development Plan Options
54
6
Additional MIND Professional Development Deliverables Train the Trainer - With the ultimate goal of building sustainability for the continued implementation of ST Math, the MIND Research Institute completed a three-day Train-the-Trainer certification workshop (June 23-25th) for 21 attendees representing the nine districts involved in the Math Matters project. The candidates chosen to participate in the workshop primarily held a role as classroom teachers or math coaches within the schools or districts they represent. Nineteen of the twenty-one attendees successfully earned certification as a trainer of ST Math and are now capable and equipped to deliver MIND’s three-hour introductory workshop on ST Math. MIND is committed to providing continued support and development opportunities for these individuals who have obtained training certification. These participants will have access to quarterly online Q&A sessions with MIND Technical Support representatives and a yearly webinar outlining any changes made to the ST Math program itself and/or any of the training support resources that have been provided to them. The increased capacity of internal support (such as a Train the Trainer program) for any new initiative can have a positive and lasting impact on a new program’s adoption and long term sustainability when utilized. At the time of training most attendees were unsure of the specific expectations that would be placed upon them after receiving this training certification. MIND will continue to promote and develop the capacity for this, and other, internal support structures at the district and school levels. June Professional Development – The MIND Research Institute team in Ohio was able to make additional Professional Development sessions available to teachers from school’s participating in the Math Matters project through Fairfield County Educational Service Center’s June Academy professional development event. A variety of sessions were offered in order to meet the needs of a broad base of ST Math teachers that had grown tremendously over the course of the 2014-2015 school year. Due in part to the expanded interest and growing scope of the ST Math implementation at many schools, Track A sessions (Sessions 1 & 2) were offered to provide any teachers new to ST Math with an additional opportunity to receive the initial professional development necessary to implement the program. Track B sessions (Sessions 1, 2, & 3) were advanced courses designed for experienced ST Math teachers who were prepared for a deeper level use of ST Math within their classrooms. The Common Core Mathematical Practice Standards, development of a Blended Learning environment, and the integration of ST Math games into the instructional routine were topics covered in these sessions. These June Professional Development sessions offered teachers an opportunity to collaborate with other ST Math users from within and outside of their own district. This sharing of best practices and lessons learned from the first year of the Math Matters implementation will serve to improve the efficient and effective implementation of the program in future years. Instructional Consulting Workshops – Plans are in place for the upcoming 2015-2016 school year to provide a selected group of Math Matters teachers (representation from multiple districts) the opportunity to participate in a 5 part professional development series from MIND Research Institute. These teachers will continue to learn strategies on how to implement ST Math classroom lessons that engage all students in developing the Common Core Standards of Mathematical Practice 3 & 4. Activities include lesson planning, analyzing student work, discussing pedagogical shifts, sharing best practices, and implementing facilitation strategies that promote student-centered learning. The series is comprised of a full day Workshop, ½ day Lesson Studies, 2 follow-up Webinars, and an onsite Coaching opportunities with MIND Research Staff.
55
7
MIND Self-Guided Course Overview
Chart 2
In addition to the instructor led training and service events delivered by MIND Research Institute over the first year of the Math Matters project, participating teachers took advantage of the Self-Guided Courses available through MIND’s Teacher Resource Site. The ST Math Self-Guided Courses are available to any teacher or administrator with ST Math account credentials (username and password) and are able to be accessed and completed in a timeframe convenient to their schedule. The eight courses available (see pg. 8) cover content from an introduction to ST Math, how to get students started, the use of available performance data and student/class reports, and ultimately how a teacher may incorporate ST Math into the classroom as an instructional tool.
58% of the 1017 teachers participating in the Math Matters project utilized the Self-Guided Courses as either a supplement to the instructor led professional development they received or in some cases as their main source of learning how to implement ST Math. On average these teachers had just over 5 recorded experiences interacting with this course content. In two districts the number of individuals utilizing the Self-Guided Courses actually surpassed the total number of teachers actively using ST Math. This can be accounted for through the use of these courses by administrators or coaches that do not have students assigned to them within the ST Math system. The 3,161 individual uses of these Self-Guided Courses have provided participating teachers with a significant opportunity for additional learning about the effective implementation of ST Math program.
0
50
100
150
200
250
300
MIND Self-Guided Courses - District Usage
Number ST MathTeachers
NumberIndividuals UsingSG Courses
56
8
MIND Self-Guided Course Sequence
57
9
Section 3 – ST Math Implementation
Chart 3
The initial start-up of ST Math in the first year of the Math Matters project was delayed by a number of factors that were eventually resolved. First, although initial outreach from MIND began as early as July 2104, in most cases it took at least several weeks in order to finalize all of the implementation planning details necessary for each of the districts to proceed. Once district/school teams were organized information relative to device activation and account creation was identified and provided to MIND prior to the beginning of formal training. A delayed rollout of the grant funded hardware also impacted the number of students able to start using ST Math early in the school year. In many cases, schools/districts needed additional time in order to setup and configure new Chromebooks and iPads to work on their networks. MIND provided onsite and virtual technical troubleshooting support in order to identify issues restricting the use of ST Math on these devices. Once these issues were resolved within the Oct-Nov timeframe the number of participating students using the program rose rapidly and continued to rise all the way through July (see Graph 3 blue bars). As shown on Graph 3 in red, the comparative amount of time it has taken schools to get students actively enrolled into ST Math this school year (2015-2016) has dropped significantly. By the end of September 2015 there were more than 4 times the number of students (23,000) enrolled in ST Math compared to that same time the previous school year (5,000). This early start in using ST Math will have a dramatic impact on the amount of ST Math content that students across the project will be able to progress through over the course of the second year of the Math Matters project.
0
5000
10000
15000
20000
25000
30000Total Students Enrolled in ST Math by Month
2014-2015School Year
2015-2016School Year
58
10
Year 1 Start-up vs Syllabus Progress The general start-up delays experienced by districts as well as the positive expansion of new ST Math users as a result of mid-year exposure to the program created a wide range of individual start dates for the initial use of the program. The average start date for the 26,531 students who used ST Math last school year was actually December 5th, which ends up being about 40% through the school year calendar. In comparison, the average start date for the 23,000 students that were already enrolled in ST Math by the end September 2015 was September 7th. This average will continue to fluctuate as more students are enrolled in the program over the coming weeks/months, but early indications show that schools/districts have been able to get students started on ST Math much earlier than last year. The final ST Math Syllabus Progress Averages for individual districts and the project-wide average overall for the first year were less than ideal when viewed as a whole, but understandable due to the wide variance in the number of weeks/months in which students may have been using ST Math across the project. It is important to note that calculation of school, district, or project-wide Syllabus Progress Averages may include students who have had access to ST Math for an entire 180 day school year or those who’ve had access for as little as 1 day. As shown in Chart 4 below, students who started earlier in the year had on average many more opportunities to use ST Math (Logins) and therefore completed more of the ST Math content (Syllabus Progress) at their assigned grade level. The nearly 5,000 students who were able to login to ST Math 72 or more times (average of 2 times per week) completed 65% of their assigned content.
Chart 4
0
10
20
30
40
50
60
Aug-Oct Nov-Jan Feb-Apr May-Jun
Time Period Student Started Using ST Math
ST Math Logins/Avg Progress vs Start Date
Average ST MathLogins
Average FinalSyllabus Progress
59
11
Projections for ST Math Progress in 2015-2016 As noted previously, the start-up of ST Math in year 2 of the Math Matters project is off to a much faster start than year 1. There are multiple reasons for the more efficient launch of the program this year. The need for initial ST Math training has been reduced to those staff members who are new to the program and no longer exists as a barrier to start-up. Returning teachers have a year’s worth of valuable experience in the process of setting up their classes and the day to day use of the program. The technology issues and considerations relative to device activation and student scheduling have in most cases been resolved. All of these add up to a huge reduction in time necessary to get students up and running on the program. The benefit of this quick start will be the increased number of opportunities that students will have to work through the assigned ST Math content in year 2. MIND’s best practice recommendation is to use ST Math at least twice a week for a total of 60 – 90 minutes (depending on grade level). Due to the previously discussed start-up delays in year 1 only 18% of the students were able to meet or exceed the suggested number of logins for the year (72). The students who were able to do so completed an average of 65% of their assigned grade level content which was more than twice the overall project average (30%). As shown in Charts 5 & 6, the positive impact of an efficient and timely start-up of ST Math is already quite evident in the year over year comparisons of Average ST Math Syllabus Progress by month. The student’s that are currently assigned Grade K-6 ST Math (Chart 5) content have double the amount of average syllabus progress as compared to last year in both September and October of 2015. Those students currently assigned ST Math’s Middle School Supplement (MSS) or High School Intervention Content (HSI) have four times the average syllabus progress over the same time periods.
Chart 5
0
5
10
15
20
25
30Average K-6 ST Math Syllabus Progress
2014-2015School Year
2015-2016School Year
60
12
Chart 6
Project Challenges The most common challenge that was faced by schools implementing ST Math as a part of the Math Matters project was centered on the availability of time. It was an issue that was raised by half of the districts that participated in the project. The availability of time had an impact in a number of ways. The request to fit 60 to 90 minutes of ST Math time per week into an already crowded schedule is a common initial concern that is confronted when adopting a new program. Decisions about where that time comes from are not always immediately clear and need to be worked out through a process of trial and error. In a number of cases the integration of ST Math was further impacted by the presence of other programs also competing for any available time outside of core instruction. In addition to already existing supplemental programs, a few districts were also transitioning to new core instructional programs which took up a considerable amount of their focus. Another aspect of the time challenge that was appropriately realized by several districts was not only the time in which students get to work on ST Math but the time in which teachers have available to facilitate struggling student’s use of the program. This is an important element of the successful use of ST Math. Although students are challenged to be independent learners through the use of ST Math relying on the informative feedback provided by the program, all teachers will have students for which additional assistance is eventually required. Finding and ensuring that not only students but teachers alike have time to work with ST Math is very important.
0
1
2
3
4
5
6
7
8
9
10Average MSS/HSI ST Math Syllabus Progress
2014-2015School Year
2015-2016School Year
61
13
Time was also a challenge impacted by the availability of technology devices to be used with the program. There were a number of large schools that saw value in utilizing the ST Math with many grade levels across their buildings. Along with finding the time in class schedules to incorporate 60 to 90 minutes for ST Math was the challenge of matching that time up with available devices. The eventual setup and distribution of grant funded devices from the Math Matters project lessoned the burden on the existing technology demand but did not resolve it in all cases. Project Success The first year of the Math Matters project was a strong first step toward reaching the goals around which the project was designed. Over 1,000 teachers and nearly 27,000 students have already been introduced to ST Math and a new approach to teaching and learning math. The excitement that has been built around the program will serve to counteract some of the initial challenges that were faced. Greater awareness of and first-hand experience in seeing how ST Math effectively supports students learning will lesson concerns about the time needed to implement the program. In the first year of this project ST Math has grown to be seen as a compliment to teacher’s core math instruction and a valuable support toward the common goal of deepening student’s true understanding of math leading to an increase in math achievement. The continued month to month expansion of ST Math users through the first year was a very positive sign of increased interest and acceptance of the program. Many districts experienced growth beyond the initially targeted student populations which was possible due to their access to site licenses of the program. Expansion was also seen in the different ways in which Math Matters schools see the program being used. Several schools incorporated the use of ST Math into structured Summer Programs that incorporated a blended learning approach in which students engaged in the use of ST Math both independently and as a whole group. As schools use of ST Math continues MIND will encourage creative thinking in how students are provided access to this powerful learning tool. The experience gained over the first year of the Math Matters project has many schools and teachers ready to take the use of ST Math to the next level. The sessions offered in the June Academy and the joint Instructional Consulting Workshops planned in the coming year were/are focused on creating a successful blended learning environment. This goal of bridging the use of technology and the core math instruction will continue to be a focus of MIND’s work in schools and of the professional development that we provide.
62