Slide 1

Building a Culture of Collaborative Inquiry to Improve Adolescent Literacy Day 2, Logic Modeling June 5, 2014 Nashville, TN 1

Slide 2

Todays presenters Zoe Barley, Ph.D. Event facilitator, REL Appalachia, and Zbarley Consulting Stephanie Wilkerson, Ph.D. Alliance lead, REL Appalachia, and Magnolia Consulting 2

Slide 3

Yesterdays workshop... Identified: Key root causes of barriers to effective data use. Interventions currently under way in the district. Gaps between root causes and interventions. Developed long-term outcomes for the district, school, and classroom in resolving root causes. Identified needs for professional learning and resources in order to improve data use. 3

Slide 4

The day ahead Learn about logic models purposes and creation. Create an initial MNPS alliance logic model: Outcomes refined and time frame projected. Strategies/activities to accomplish. Aligned to root causes. 4

Slide 5

What is a logic model? Many people say a logic model is a roadmap. 5

Slide 6

A logic model is A depiction of planned work showing: What the work will do. What the work will accomplish. 6

Slide 7

Complex logic model Source: State University of New York, 2011 7

Slide 8

A linear model Source: University of Wisconsin-Extension, 2003 8

Slide 9

The basis of the logic A series of if-then relationships that, if implemented as intended, lead to the desired outcomes. 9

Slide 10

Theory of change A theory of change is a description of how and why a set of activities be they part of a highly focused program or a comprehensive initiative are expected to lead to early, intermediate, and long-term outcomes over a specified period. 10

Slide 11

An example of a theory of change Source: Center for Advancement of Informal Science Education, n.d. 11

Slide 12

Programs fail for 3 reasons 1.Theory of change failure: Program conceptualization and design cannot address the problem or cannot generate desired outcomes, no matter how well implemented. 2.Translation failure: Theory is sound, but translation into practice or action is not. 3.Implementation failure: Failure to properly implement programs, strategies, or activities. Adapted from Peter Rossi. Source: Shadish, Cook, & Leviton, 1991 12

Slide 13

How is a logic model useful? Helps differentiate what we do from results. Clarifies the results; keeps focus on priorities. Represents the big picture; blueprint. Provides common language. Identifies gaps or inconsistencies. Fosters ownership and accountability. Provides a foundation for evaluation. 13

Slide 14

Simple logic model 14

Slide 15

Target population The audience you will affect through your activities: Students. Teachers. Parents. Leaders. Community. 15

Slide 16

Inputs The resources you invest in a program: Funding. In-kind contributions. Staff. Time. Materials and equipment. Partnerships. Planned activities/strategies. The state and local interventions already in place. 16

Slide 17

Activities What do you do to reach and affect the target audience: Changes to team agendas to incorporate collaborative inquiry. Collection of new data. In-house professional development on data visualization. How you address a clearly defined need (root cause) in the target population or community. 17

Slide 18

Outputs Short-term/intermediate outcomes First steps toward the desired impact: Time for teachers to collaborate. Faster data turnaround. New ability to visually display data. 18

Slide 19

Outputs vs. outcomes Example: Teachers meeting for collaborative inquiry is an output. Change in instructional practice based on the learning from the inquiry is an outcome. Outcomes Not how many worms the bird feeds its young, but how well the fledgling flies. - United Way of America 19

Slide 20

Outcomes Short-term Intermediate Long-term 20

Slide 21

Hierarchy of outcomes Source: Adapted from Rockwell & Bennett, 1995 21

Slide 22

Recap: The logic of the logic model The long-term outcomes are the impact we desire. The intermediate outcomes will lead to the long-term outcomes: If we accomplish them, then The short-term outcomes will lead to the intermediate outcomes: If we accomplish them, then 22

Slide 23

Recap: The logic of the logic model The activities we have planned and the target audiences we have engaged will lead to the short-term outcomes. The activities address the root causes that create barriers to effective data use, hindering the improved adolescent literacy we seek. 23

Slide 24

Next steps Where are you going? How will you get there? What will show that youve arrived? Source: University of Wisconsin-Extension, 2003 24

Slide 25

Todays focus If we implement a collaborative inquiry approach to data use, what classroom-, school-, and district-level outcomes do we intend to achieve? Articulating outcomes for: How we work together. The way in which we use data. The knowledge and skills for each. 25

Slide 26

Overview of logic modeling steps Step 1: Review root causes and long-term preferred outcomes. Step 2: Develop short-term and intermediate outcomes. Step 3: Identify strategies and activities. Step 4: Review as a whole group. 26

Slide 27

Three groups: Classroom, school, district Classroom outcomes (teachers) What will teachers know and be able to do in their literacy instruction as a result of participating in collaborative inquiry? School outcomes (coaches, principals/assistant) What will coaches and administrators know and be able to do in their schools as a result of participating in and supporting a collaborative inquiry process for data use in literacy instruction? District outcomes (district staff, directors, executives) What will district staff know and be able to do in MNPS as a result of fostering a districtwide culture of collaborative inquiry for data use in literacy instruction? 27

Slide 28

Logic Modeling Step 1 1.Review root causes and long-term preferred outcomes (from yesterday). Facilitators have made suggestions. What additional suggestions does the group have? 28

Slide 29

Logic Modeling Step 2 2.Develop short-term and intermediate outcomes. Given a long-term outcome For example: Literacy instruction is based on findings from collaborative inquiry. what would be an intermediate step toward obtaining it? For example: Teams of literacy teachers practice with collaborative inquiry processes. 29

Slide 30

Logic Modeling Step 2 What would be a short-term step toward the intermediate outcome? For example: Literacy teachers will know how to engage in collaborative inquiries. 30

Slide 31

Logic Modeling Step 3 3.Identify strategies and activities. What current work (activity of strategy) do we need to modify, or what new work should we consider, to attain the short-term outcomes? For example: Professional learning on collaborative inquiry is provided to middle school teachers. 31

Slide 32

Is the model logical? Is our progression from activity to shorter-term outcomes logical? 32

Slide 33

Do the If Thens work? Source: Center for Advancement of Informal Science Education, n.d. 33

Slide 34

Logic Modeling Step 4 4.Review as a whole group. How do the outcomes support addressing the root causes identified yesterday? Is there continuity across levels? Are there gaps? How can we address these? How could we use these logic models to guide our collaborative inquiry work and promote collaborative inquiry in middle schools and throughout the district? 34

Slide 35

Next steps Ongoing learning based on Dr. Liptons presentation and in preparation for Day 2 of her workshop on July 15, 2014. Distribution of additional resources, as identified over the course of the technical assistance. Digital transfer and clean-up of logic model development from today. Preparation for follow-on session on innovation configurations on July 16, 2014. 35

Slide 36

REL Appalachia Stakeholder Feedback Survey Did we meet our goals and objectives for the fishbone process and logic modeling activities? Do you have other support or information needs? Anonymous. 5-10 minutes. 36

Slide 37

References Anderson, A. (2000, November). Using theory of change in program planning and evaluation. Paper presented at the annual meeting of the American Evaluation Association, Honolulu, HI. Center for Advancement of Informal Science Education. (n.d.). Principal investigators guide: Managing evaluation in informal STEM education projects. Retrieved March 13, 2014 from http://informalscience.org/evaluation/evaluation-resources/pi-guide#introduction. http://informalscience.org/evaluation/evaluation-resources/pi-guide#introduction Office of Financial Management, State of Washington. (2008). Enterprise risk management. Retrieved March 21, 2014 from http://performance.wa.gov/GE/GE111908/RiskManagementWorkerSafety/ERMMaturity/Documents/E RM%20Logic%20Model.jpg. http://performance.wa.gov/GE/GE111908/RiskManagementWorkerSafety/ERMMaturity/Documents/E RM%20Logic%20Model.jpg Rockwell, K., & Bennett, C. (2004). Targeting outcomes of programs: A hierarchy for targeting outcomes and evaluating their achievement. Faculty Publications: Agricultural Leadership, Education & Community Department. Retrieved October 20, 2013 from http://digitalcommons.unl.edu/aglecfacpub/48.http://digitalcommons.unl.edu/aglecfacpub/48 Shadish, W., Cook, T., & Leviton, L. (1991). Foundations of program evaluation: Theories of practice. Thousand Oaks, CA: Sage Publications. State University of New York. (2011). Catchup and complete enhanced blended learning initiative. Retrieved March 21, 2014 from http://wiki.sln.suny.edu/display/SLNNGLC/revised-updated+logic+model. University of Wisconsin-Extension. (2003, February). Enhancing program performance with logic models. Retrieved October 20, 2013 from http://www.uwex.edu/ces/pdande/evaluation/pdf/lmcourseall.pdf.http://www.uwex.edu/ces/pdande/evaluation/pdf/lmcourseall.pdf. 37