Measuring our Impact: The Open.Michigan Initiative
Emily Puckett RodgersOpen Education Coordinator, Open.Michigan
OpenCourseWare Consortium Global Meeting 2011
Except where otherwise noted, this work is available under a Creative Commons Attribution 3.0 License.Copyright 2011 The Regents of the University of Michigan
CC
: BY-
NC
-SA
, C
hoco
nan
cy1
Flic
kr
Evaluation team
• Emily Puckett Rodgers
• Ted McCarthy• Pieter Kleymeer
Introduction
Evaluation: started September 2010
http://open.umich.edu
“Through a combination of existing and new qualitative (including surveys, interviews and user feedback) and quantitative (including web analytics,
published resources) data, we aim to measure Open.Michigan’s impact on the University of Michigan campus and on
the broader open education landscape, as well as its progress toward overall objectives and mission.”
Evaluation Design
Implementation
Mission and Objectives
Environmental Scans
Document Study and Review
Organizational analyses
Research development and deployment
Analysis
Review
Environmental Scan
General evaluations, activities in Open Education
Specific evaluation projects and documentation: • Tufts OpenCourseWare (2010)• Free to Learn Guide (2010)• MIT OpenCourseWare (2006)• Oxford’s Technology-Assisted Lifelong Learning (2010)
Document Study
• dScribe data • Wiki• Shared internal documents
and spreadsheets
Environments
Open.Michigan’s Strategic Vision
Open.Michigan enables University of Michigan faculty, students, staff and others to share their educational resources and research
with the world.
Open.Michigan’s efforts contribute to two primary goals:
1. to sustain a thriving culture of sharing knowledge at U-M2. to provide comprehensive public access to all of U-M’s scholarly output
Strategic Vision
http://tinyurl.com/openmichiganvision
Measurable Activities
• Building communities of OER producers and users
• Consulting and Outreach Services to facilitate OER production
• Development of Processes and Software to support OER production and publishing
Next Steps: Goals and Objectives document, Communications Plan
Strategic Plan
Guiding Objectives • How and Why is our OER being
used? • Who is using our OER? • What value does OER bring to U-
M? • In what contexts are people
using our OER?
From the Vision document
From the Evaluation plan
Impact: Growth
Open.Michigan platform migrations
eduCommons OER
OERbit /education OER
static Open.Michigan pages
4/1/2008 9/16/20108/27/2009 4/2011
Google Analytics
Impact: Growth
Impact: Contributions
Resources available• ~70 courses published • 2 campuses • 12 schools• ~9 OER modules published • ~48 resources deposited in Deep
Blue
http://open.umich.edu/education
Resource facilitation• Anishinaabe language and
literature• African Health OER Network• “Textbook of the Future”
discussions• Center for Global Engagement
Handbook
• OERca, released 2008 (copyright and content clearance tool)
• OERbit, released 2011 (publishing platform)
• dScribe, process for using volunteers and community members to clear content and publish OER
• Code of Best Practices in Fair Use for OpenCourseWare (2009)
• African Health OER Network
Contributions to Open Education
Impact: Contributions
Institutional Support (U-M)
Dean support: • Dean Woolliscroft (UMMS)• Dean MacKie-Mason (SI)• Dean Potempa (Nursing,
retired)• Paul Courant (MLibrary)
Financial support: • UMMS• Hewlett Foundation• FAIMER• Open Society Institute• Local grants
Institutional support and partnerships: • Medical School• School of Information• MLibrary• MERLOT
African Health OER Network: • Global Reach• African Studies Center• Ghana-Michigan Charter• Ghana Emergency Medicine
Collaborative
Impact: Investment
“Having the big block M on a lot of high quality produced teaching modules that are made available to the world… is mission consistent, so what’s the business that we’re in here? We create and distribute knowledge.” -Paul Courant, “Why Open is Important” interview
Impact: Surveys
40.7% instructors never heard of OCW75.6% students never heard of OCW
CTools survey
http://tinyurl.com/ctools2010survey
Impact: Surveys
Population Size Response Rate Completion Rate
39,958 students 27.5%71%
6,149 faculty 30.4%68%
2,268 staff 16.5%55%
Open.Michigan and Open Educational Resource survey to U-M
Themes: • Sharing habits • Awareness of
Open.Michigan and OER• Use of OER• Impact of OER • Support of OER efforts and
Open.Michigan
Sharing
Impact: Surveys
"I share learning materials with my colleagues to..." ? (faculty responses)
Use
Impact: Surveys
“What type of OER have you used?” (student responses)
Next Steps: participating faculty surveys
(Some) ResultsThemes from survey comments
• Establish Open.Michigan in main channels of U-M communication
• Clarify copyright in OER and provide resources
• Consistently define Open.Michigan and OER
• Describe why/how to use/create OER• Make short-term improvements to OER• Encourage cultural shifts in learning
CC: BY Emily Puckett Rodgers
Impact: Surveys
Next Steps
In the next three years, Open.Michigan will:
① Produce more and richer content as OER with the various campus units, improve modularity, instructional design, and accessibility of U-M OER
② Increase the visibility and discoverability of U-M resources through a combination of marketing and metadata
③ Draw participants from more parts of campus to expand its disciplinary coverage
④ Ensure OER production is an embedded part of the academic life on campus
Evaluation • Finish survey
analysis• Focus Groups
Strategy• Goals and
Objectives• Communications
plan • Advisory
committee
Resources
Contact
Emily Puckett Rodgers
[email protected]@umich.edu
Use
• Evaluation wiki page • Evaluation plan• Survey questions• Strategic planning• Aggregate data
• Our Google Analytics data • Aggregate survey results
http://open.umich.edu/wiki
http://tinyurl.com/omevaluation