surveying the state of library assessment
TRANSCRIPT
Surveying the State
of Library Assessment
MICHAEL HOLT Valdosta State UniversityLOUISE LOWE Mercer University
EMILY THORNTON University of North GeorgiaCRISTINA HERNÁNDEZ
TROTTEROconee Regional Library System
Presentation graphics by Abbie Gettys
SURVEY GOALSWhat were our goals with the
survey?What is being assessed in libraries today?
What tools and methods are being used?
What should be assessed, but is not? Why?
SURVEY BACKGROUNDDATES:
Survey open May 3rd – June 3rd, 2016
INTENDED PARTICIPANTS:
Anyone who self-identifies as being responsible for any type of assessment or evaluation in any kind of library.
DISTRIBUTION OF INVITATIONS:
Acrlassesdg - ACRL Assessment listservArl-assess, GLA-L , MLA, PUBLIB, ALA Spectrum Scholars
List
SURVEY CONTENTS
How they are responsible for
library assessment
Job Title
Years of experience in
libraries overall and in assessment
Perception of effectiveness of assessment plan/practices
Examples of what is being assessed at their library
Examples of what is not
being assessed at their library
• What tools/methods?• How are results used?
• What are the barriers?
281 TOTAL RESPONDENTS1.4%
RESPONDENTS’ EXPERIENCE IN LIBRARIES
0-5 years 6-10 years 11-15 years 16-20 years 20+ years0102030405060708090
100
4350 54
39
95
(15%)(18%)
(19%)
(14%)
(34%)
RESPONDENTS’ EXPERIENCE WITH ASSESSMENT
0-1 years 2-3 years 4-5 years 6-10 years 11-15 years 16-20 years 20+ years0
10
20
30
40
50
60
70
25
61
48
60
38
2227
(9%)
(21%)
(17%)
(21%)
(14%)
(8%)(10%)
RESULTS: IS IT AN OFFICIAL RESPONSIBILITY?
WHAT ARE THEY DOING BESIDES ASSESSMENT?
HOW WOULD YOU RATE YOUR KNOWLEDGE OF ASSESSMENT IN
LIBRARIES?
Not kn
owled
geab
le at
all
Slight
ly kn
owled
geab
le
Moder
ately
know
ledge
able
Very k
nowled
geab
le
Extre
mely kn
owled
geab
le0
40
80
120
330
12995
23
HOW KNOWLEDGEABLE ARE RESPONDENTS ABOUT
ASSESSMENT?
RESULTS: CORRELATION BETWEEN KNOWLEDGE AND
COMFORT?
HOW COMFORTABLE ARE YOU WORKING
WITH ASSESSMENT?
04080
120160
416 40
13783
HOW COMFORTABLE ARE THEY WORKING WITH ASSESSMENT?
EXTREME COMFORT WITH ASSESSMENT AND RELATION TO
JOB TITLE
DOES YOUR LIBRARY CURRENTLY HAVE AN ASSESSMENT PROGRAM
OR PLAN?
ASSESSMENT PLANS/PROGRAMSBY LIBRARY TYPE
HOW EFFECTIVE IS YOUR LIBRARY’S ASSESSMENT PROGRAM
OR PLAN?
EFFECTIVENESS BY LIBRARY TYPE
DOES EXPERIENCE CHANGE PERCEPTIONS
OF EFFECTIVENESS?
• Most effectiveness ratings fell into very effective or moderately effective no matter what the experience level with assessment was.
• There is a slight upward shift in effectiveness ratings as experience with assessment increases.
• Moderately effective was the most popular rating across all experience levels.
DOES EXPERIENCE CHANGE PERCEPTIONS OF EFFECTIVENESS?
Having a plan seems to make for a higher effectiveness rating of assessment practices.
DOES HAVING A FORMALASSESSMENT PLAN AFFECT
PERCEPTIONS OF EFFECTIVENESS?
of those with a formal plan rated their practices as very or extremely effective.
without a plan rated their practices that highly.
of those unsure whether or not they had a plan rated their practices that highly.
41% 23.4% 25%
EFFECTIVENESS BY JOB TITLEThose with assessment explicitly listed in their job description rated their
effectiveness slightly higher than their counterparts.
of those with assessment in their job title/description rated the effectiveness of their practices as extremely or very effective.
44%
of those without assessment as an explicit component of their job rated their practices as extremely or very effective.
20% The largest percentage of both (46% assessment in title/description,57% without) rated their practices as moderately effective.
DO THEY FEEL THE IR ASSESSMENT EFFORTS ARE EFFECT IVE?
EFFECTIVENESS BY TYPE OF LIBRARY
PRELIMINARY F INDINGS FROM ANALYSIS OF OPEN-ENDED
QUESTIONS
RESPONSES CODED
1,604CODES
CREATED AND APPLIED
38
DATA AREAS BEING ASSESSED
460 unique responses were coded.
Two categories of codes:• Data Point/Focus - What is the
thing being measured this is actually increasing or decreasing?
• Area of Assessment - What aspect of the library is being assessed?
DATA POINT/FOCUS
ASSESSMENT AREAS BY LIBRARY TYPE
AREAS ASSESSED
SERVICES ASSESSED
SERVICES ASSESSED BY LIBRARY TYPE
DATA AREAS BEING ASSESSED:TOOLS AND METHODS
2.7%
HOW ARE THE RESULTS BEING USED?
To track and improve user services and spaces
Most common
response:
• Reporting - Internal and External• Collection Development• Strategic Planning• Staff Performance
Other popular
responses:
• Adherence to Policy• Promotion and Marketing
Less popular
responses:
HOW ARE THE RESULTS BEING USED?4.8%
WHAT ARE WE UNABLE TO ASSESS?
In these three main assessment areas:
• Instruction & Information Literacy
• Collections• General Services
MOST EXPRESSED A DESIRE,
BUT DIFFICULTY IN ASSESSING:
• Impact• Correlation• Adequacy• Value-added services
In these three main assessment areas:
• Instruction & Information Literacy
• Collections• General Services
IN THESE THREE MAIN ASSESSMENT
AREAS:• Instruction &
Information Literacy• Collections• General Services
EXAMPLES OF COMMON CHALLENGESHow can we show direct correlations between library
instruction and student retention/academic
performance?
How effective are ‘one shot’ library instruction
sessions?
How can we build adequate collections
through faculty support and usage data?
How can we evaluate electronic collections using various vendor-
provided data?
EXAMPLES OF COMMON CHALLENGES
How can libraries prove they are
meeting needs?
Are library services and programs
reaching specific patrons groups?
Are library services and programs
relevant for specific patron groups?
How can libraries prove activities,
collections, services are worth the investment?
How can libraries prove their overall
value to the community/institution
?
WHAT ARE THE LIMITING FACTORS
IN ASSESSMENT?
“OTHER” LIMITING FACTORS
FROM ACADEMIC LIBRARIANS:• Difficulty defining measurable goals• Lack of an organized mission statement to
align goals to• No cooperation from teaching faculty
FROM PUBLIC LIBRARIANS:• Trouble with vendor standards• Trouble defining what is being assessed
DOES TYPE OF LIBRARY AFFECTLIMITING FACTORS?
DISCUSSIONHow do you address the
common assessment problems in your library?
Do you have a problem not mentioned here?
Are there other obstacles to assessment?
QUESTIONS?
This presentation is available at:slideshare.net/chtrotter
Presentation graphics by Abbie Gettys - [email protected]
MICHAEL HOLT [email protected]
EMILY [email protected]
LOUISE [email protected]
CRISTINA HERNÁNDEZ TROTTER
COMMENTS?