learning what works - amazon s3slid… · learning what works data & documentation in an...
TRANSCRIPT
![Page 1: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/1.jpg)
Learning What WorksData & Documentation in an
Evidence-based WorldSummer 2017Kristin Wolff, SPR
![Page 2: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/2.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
What & Who
Project Resources Online at:http://cte.ed.gov
The work reported herein was supported under the Community College Career and Technical Education (CTE) Stackable Certificates Initiative; award number ED-VAE-15-D-0007, as administered by the U.S. Dept. of Education, Office of Career, Technical, and Adult Education. The contents do not necessarily represent the positions or policies of OCTAE or the U.S. Dept. of Education and you should not assume endorsement for the Federal Government.
![Page 3: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/3.jpg)
Evaluation. It doesn’t have to be this way.
![Page 4: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/4.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
Why Evaluation?
• To plan and launch effectively(formative, process)
• To improve (empowerment, participatory, collaborative, adaptive, action)
• To see if what you did worked(summative, outcome, impact)
• To find better ways to meet your needs (developmental, complexity, social innovation, network-centered)
![Page 5: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/5.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
Common Data Types
• Process documentation• Activity-level data• Outcome data• Demographic data• Experiential data (participants, employers,
partners)
![Page 6: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/6.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
Evaluation Considerations
• Use what you have• Gather what you need• Document your practice
[Note: You may opt to contract out components of this work, but if you want to learn from it, engage in it.]
![Page 7: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/7.jpg)
1. Use Ongoing Evaluation and Reporting
![Page 8: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/8.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
What do you have? How can you use it?
• Got a TAACCCT? Part of an ATE? Enlisted philanthropic friends? – Figure out what’s being collected, by whom, and whether it’s
accessible– Join evaluation calls and webinars where relevant– Use this TA, other TA, and peers to identify approaches to
getting existing information that will help you learn, scale– Review evaluation reports and materials from partner or
parallel projects– Influence existing data collection practices where relevant
![Page 9: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/9.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
2. Gather what you need
![Page 10: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/10.jpg)
Do you know what you need?
![Page 11: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/11.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
2. Are you clear about what you are doing and why?
• Logic model • Theory of change• Blank Canvas• Strategic plan
![Page 12: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/12.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
Expectations
![Page 13: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/13.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
The Basics
![Page 14: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/14.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
Examples
![Page 15: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/15.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
Examples
![Page 16: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/16.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
Examples
![Page 17: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/17.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
Examples
![Page 18: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/18.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
How will you measure what matters?
• Capture the essentials (highest level outcomes)• Test the value of other indicators (process and
outcome)– Build on existing systems– Experiment (until confident) – Count manually – Use debriefs to address measurement issues (test against
use cases)
![Page 19: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/19.jpg)
3. Document your practice(Yea! The fun bit!)
![Page 20: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/20.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
What to Document
• Your Process (it matters more than you think—especially if you want to grow or scale)– Context – Meetings– Events – Materials and products– Successful ventures (and those less successful)– Anything that’s hard to explain
• Participant/Stakeholder Experience & Impact
![Page 21: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/21.jpg)
Get Creative
![Page 22: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/22.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
Methods• Write, record, photograph, video, publish/share• Capture perspectives of individuals and groups• Share and reflect (graphic recording?)• Use multimedia (yes, social, too)
![Page 23: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/23.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
Crowd Favorites
![Page 24: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/24.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
To Recap…
• Find out what you have/can access• Gather what you need (even informally)• Document your practice (you’ll be glad you
did, we promise)
![Page 25: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/25.jpg)
And, of course, analyze & course correct.
![Page 26: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/26.jpg)
Mapping Upward: Stackable Credentials That Lead to Careers
Reflection/Debrief
• Assign responsibility for reflection/debrief process
• Conduct the process as a group/team• Take actions as needed (and stop collecting
data that does not matter)
![Page 27: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/27.jpg)
A word about external
![Page 28: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/28.jpg)
Evaluation. It doesn’t have to be this way.
![Page 29: Learning What Works - Amazon S3slid… · Learning What Works Data & Documentation in an Evidence-based World Kristin Wolff, SPR Summer 2017](https://reader035.vdocuments.us/reader035/viewer/2022063005/5fb3b62ba63fe90b0839dc05/html5/thumbnails/29.jpg)
Thank you