what is a library ‘outcome’ and how do you measure one? andy stewart library director c. l....
TRANSCRIPT
What is a Library ‘Outcome’ and How Do You Measure One?
Andy Stewart
Library DirectorC. L. Wilson Library
Missouri S&T
A “Reality” Check…
"Outcomes-based WHAT?"
"Moderately aware of the
concept"
"Well-read and chomping at the
bit to try one"
"Conducted 3 or more OBE projects"
0%
40%
80%
Session Audience – 10/7/10Outcomes-Based Evaluation Ex-
pertise
"Evaluation based on WHAT?"
"Moderately aware of the
concept"
"Well-read and chomping at the
bit to try one"
"Conducted 3 or more OBE projects"
0%
20%
40%
60%
80%
100%
Outline:
Introduction Definitions Logic Models Examples Related Resources Looking Beyond Questions
Introduction
Definitions:“What is outcome evaluation?
IMLS defines outcomes as benefits to people: specifically, achievements or changes in skill, knowledge, attitude, behavior, condition, or life status for program participants.”
A little more from IMLS:“Note that for IMLS, while academic knowledge and skills are
foundations, learning is broader. It includes the cognitive, social, and physical; the practical, entertaining, and personal; the formal and informal; and many, many more domains and settings. Learning represents knowledge, attitudes, skills, and behaviors that support individual success in our complex world—in short, outcomes. Outcomes allow us to know something about the extent to which we have (or haven’t) reached our audiences. Information about outcomes allows us to strengthen our services. Equally important, it communicates the value of museums and libraries to the broadest spectrum of those to whom we account. Without data, it’s been said, “you’re just another guy with an opinion.”
From IMLS FAQ: Do I have to do this?
“IMLS does not currently require its grantees to conduct outcome-based evaluation, but it supports and encourages it as a valuable management tool. At the same time, IMLS is required to report to Congress in outcome-based terms; we cannot do that without input from you. We consider the consistent use of outcome-based evaluation to be an effective and efficient way for all programs to capture critically important information and to tell their story persuasively. IMLS is gradually strengthening information about outcome-based evaluation in guidelines for its discretionary grant programs and its program for State Library Agencies, and is considering the benefit of making outcome-based evaluation for funded programs a requirement at some future time.”
Logic Models
Andy’s definition:
A systematic way to plan programs and activities, applicable to a wide array of interests and disciplines, which :
* aids in identifying and surfacing assumptions * includes the interests of disparate stake-holders* clarifies the link between beginning,
intermediate, and final program/project steps* helps insure there is a logical connection
between the activity plans and the anticipated goal, and* facilitates the evaluation process.
Outputs
Activities
Resources / Inputs Outcome
sImpact
Your planned work
Your intended results
Page source: WKKF LMDG, p. 1, (modified Figure 1)
“Basically, a logic model is a systematic and visual way to present and share your understanding of the relationships among the resources you have to operate your program, the activities you plan, and the changes or results you hope to achieve.”
Outputs
Activities
Resources/ Inputs Outcome
sImpact
Your planned work Your intended results
Page source: WKKF LMDG, p. 3, (modified Figure 2)
Certain resources
are needed to operate
your program (Library).
IF you have access to
them, THEN you can use them to
accomplish your
planned activities.
IF you accomplish
your planned
activities, THEN you
will hopefully
(sic) deliver the amount of product
and/or service that
you intended.
IF you accomplish
your planned
activities to the extent
you intended, THEN your
participants will benefit in certain
ways.
IF these benefits to
participants are achieved, THEN certain changes in
organizations,
communities, or systems might be
expected to occur.
Real-life example (or two)
OSEDA - p. 39UIUC – 2003
Bibliography
-- Session handout
1. In-person or Online NNLM classes; right here in Missouri. Find out more: http://nnlm.gov/training/schedule/index.html?_prefs=&go=Y&areg=nnlm&keyword=®ion[]=OER#results
2. Sample NNLM Outreach Class (One day):
Measuring Your Impact: Using Evaluation for Library Advocacy
http://nnlm.gov/evaluation/workshops/measuring_your_impact/ [accessed 9-7-2010]
Looking BeyondAction Step 1
Read / Download:Perspectives on Outcome Based Evaluation for Libraries and Museumshttp://www.imls.gov/pdf/pubobe.pdf
Looking BeyondAction Step 2
Read / Download:
Association of College and Research Libraries ,The Value of Academic Libraries, Researched by Oakleaf, Megan for (ACRL, 2010).www.acrl.ala.org/value
Looking BeyondAction Step 3
Self-Guided Course, Shaping OutcomesAvailable here:http://www.shapingoutcomes.org/course/index.htm
[accessed 8-31-10]
Questions?
What is a Library ‘Outcome’ andHow Do You Measure One?
Andy Stewart
Library DirectorC. L. Wilson Library
Missouri S&[email protected]